S3 Performance

Question 1

What is an S3 prefix?

Answer 1

• An S3 prefix is the part of the file path between the bucket name and the object (ex. ‘mybucketname/folder1/subfolder1/myfile.jpg’ has prefix ‘/folder1/subfolder1’)

Question 2

What does KMS have to do with performance?

Answer 2

• Uploading and downloading files count towards the KMS quota

_{The exact amount for the quota is region-specific, but is either 5,500; 10,000; or 30,000 requests per second.}

Question 3

How can you request a quota increase for KMS?

Answer 3

Currently, there is NO way to request a quota increase for KMS

Question 4

What is Multi-Part Upload?

Answer 4

• The Idea behind Multi-part uploads is to parallelize an upload (and thus increase its efficiency)
• This is done by splitting a big file into parts and then uploading the individual parts in parallel

Question 5

What are S3 Byte Range Fetches?

Answer 5

• S3 Byte Range Fetches allow you to speed up downloads by parallelizing them.
• You parallelize by specifying exact byte ranges
• If there is a failure in the download, it is only for a specific byte range.

Question 6

What would you use if you just want to download partial amounts of a file?

Answer 6

S3 Byte Range Fetches with just the relevant bytes

Question 7

At what file size is Multi-Part Upload recommended?

Answer 7

100MB

Question 8

At what file size is Multi-Part Upload required?

Answer 8

5GB

Question 9

What do S3 prefixes have to do with performance?

Answer 9

• They are important for performance because requests are measured per second per prefix
• So, y​ou can get better performance by spreading your reads across different prefixes

Question 10

How many PUT requests can AWS handle?

Answer 10

3,500 per prefix per second

Question 11

How many COPY requests can AWS handle?

Answer 11

3,500 per prefix per second

Question 12

How many POST requests can AWS handle?

Answer 12

3,500 per prefix per second

Question 13

How many DELETE requests can AWS handle?

Answer 13

3,500 per prefix per second

Question 14

How many GET requests can AWS handle?

Answer 14

5,500 per prefix per second

Question 15

How many HEAD requests can AWS handle?

Answer 15

5,500 per second per prefix

Question 16

What is a good common way to ensure lots of different prefixes in S3?

Answer 16

Go by timestamp of upload

_{(there are many right answers here)}