AWS Open Source Blog
Category: Amazon Simple Storage Service (S3)
fMRI data preprocessing on AWS using fMRIprep
A typical fMRI study often produces imaging data of terabytes or more. Storing and preprocessing this data can be challenging on a single computer because it often has neither enough disk space to store the data nor enough computing power to preprocess it. Traditionally, researchers use a combination of cloud-based storage and on-premises high-performance clusters […]
Centralized Container Logging with Fluent Bit
September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. Visit the website to learn more. by Wesley Pettit and Michael Hausenblas AWS is built for builders. Builders are always looking for ways to optimize, and this applies to application logging. Not all logs are of equal importance. Some require real-time analytics, […]
Parallelizing S3 Workloads with s5cmd
This open source project comes from our customer community. It was developed by Peak Games to assist with their own S3 workflow, and includes features such as tab completion and built-in wild card support for files in S3 commands. Enjoy! – Deirdré Background Up until now, working on multiple objects on Amazon S3 from the […]