Amazon S3 request count optimization to minimize cost

jodmarch

Question: when transferring many small files to / from Amazon S3, does Multcloud optimize request count by consolidating many small files into a single request? Or does it process small files one at a time? 

Background: I am interested in using Amazon S3 as a long-term backup service (backing up Google Drive) for intermittent access. Storage costs should be very manageable but I am concerned about Amazon's per-request charges. If the data are consolidated into large blocks or chunks then there are not too many requests, but if every small file requires several requests, then these would add up very fast. (About 10 years ago, I had this problem when I naively used some file management program to transfer files to or from S3 (I don't remember which). While my monthly storage bill was about $2, I received a bill for almost $1000 because of the per-request charges.

I complained and they credited me the amount back, but ever since then I have been avoided using S3 to store many small files. I realize that I can zip lots of files together and upload those instead, but I'd like to avoid that friction point if possible.

Any information or suggestions appreciated. I have a Multcloud lifetime unlimited account.

Thanks! 

Admin

@Jodmarch, MultCloud will send the requests for every file unless you put single files to a folder and zip the folder as a file. This is the mechanism of API of Amazon s2.

jodmarch

Thanks for the info!