AWS Storage Blog
Category: AWS Step Functions
University of California Irvine backs up petabytes of research data to AWS
The University of California, Irvine (UCI) is a public land-grant research university with troves of research data stored on servers in lab environments on about 1500 faculty-research lab environments on campus. UCI needed a solution to address the practical and economic challenge of providing a centralized backups for these independently-administered servers. The goal for the […]
Using Amazon S3 Express One Zone as a caching layer for S3 Standard
Data caching is a critical strategy for optimizing application performance in today’s data-intensive environments. By storing frequently accessed information in high-speed storage locations, organizations can dramatically reduce access times, optimize the use of compute resources, and improve overall system responsiveness. Effective caching strategies become particularly essential for workloads that require consistent low latency, such as […]
How Pendulum achieves 6x faster processing and 40% cost reduction with Amazon S3 Tables
Pendulum is an AI-powered analytics platform that aggregates and analyzes real-time data from social media, news, and podcasts. Designed to help organizations stay ahead, it enables reputation monitoring, early crisis detection, and influencer activity tracking. Using machine learning (ML) enables Pendulum to surface key insights from multiple channels, providing a comprehensive view of the digital […]
How FICO modernizes file transfers with ETL automation using AWS Transfer Family
FICO powers decisions that help people and businesses around the world prosper. Using FICO solutions, businesses in more than 80 countries do everything from protecting four billion payment cards from fraud, to improving financial inclusion, and increasing supply chain resiliency. As a global leader in credit scoring and analytics, FICO processes massive volumes of sensitive […]
How London Stock Exchange Group migrated 30 PB of market data using AWS DataSync
Update (5/29/2025): On May 29, 2025, AWS DataSync launched Enhanced mode support for cross-cloud transfers. Enhanced mode simplifies data transfers between AWS and other clouds by removing the need for a DataSync agent. It also provides higher performance and scalability when compared to Basic mode. For more details, see the What’s New announcement or review the documentation for guidance […]
Optimizing storage costs and query performance by compacting small objects
Applications produce log files that should be reliably stored for ad-hoc reporting, compliance, or auditing purposes. Over time, these collections of relatively small log files grow in volume and cost-effective storage and data management becomes crucial. Accessing the data in these files and querying them can also be useful for getting insight from the data. […]
How Visual Layer builds high quality datasets on Amazon S3
Companies from different industries use data to help their Artificial Intelligence (AI) and Machine Learning (ML) systems make intelligent decisions. For ML systems to work well, it is crucial to make sure that the massive datasets used for training ML models are of the highest quality, minimizing noise that can contribute to less-than-optimal performance. Processing […]
How to restore archived Amazon EC2 backup recovery points from the Amazon S3 Glacier storage classes
This is the second post in a two-part series. In part one, we described a process to automatically archive Amazon EC2 backup recovery points from AWS Backup to an Amazon S3 bucket in one of the Amazon S3 Glacier storage classes. In this post, we describe the process to restore an archived EC2 backup recovery point from […]
How to archive Amazon EC2 backup recovery points to Amazon S3 Glacier storage classes
Centralizing and automating data protection helps you support your business continuity and regulatory compliance goals. Centralized data protection and enhanced visibility across backup operations can reduce the risks of disasters, improve business continuity, and simplify the auditing process. Many organizations have requirements to retain backups of their compute instances for a certain time based on […]
Automating AWS Backup pre- and post-script execution with AWS Step Functions
Customers execute custom scripts before or after a backup job to automate and orchestrate required and repetitive tasks. For example, customers running applications hosted in Amazon Elastic Compute Cloud (EC2) instances use scripts to complete application transactions, flush the buffers and caches, stop file I/O operations, or ensure that the application is idle, bringing the […]