Optimizing database backup strategy: How Service NSW achieved 70% cost reduction with AWS Backup

Service NSW, an executive public sector agency within the Department of Customer Service in New South Wales (NSW), Australia, provides unified access to government services for 8.2 million NSW residents. Operating as a digital-first organization born in the cloud, Service NSW manages services on behalf of over 70 NSW government agencies, including Transport for NSW, Revenue NSW, and Births Deaths and Marriages, across multiple Amazon Web Services (AWS) accounts.

Service NSW maintains a large fleet of Amazon Relational Database Service (Amazon RDS) and Amazon Aurora databases. In 2022, Service NSW implemented AWS Backup as a straightforward and quick solution to manage its database backups. The initial approach was designed to meet the NSW government’s 7-year data retention requirement through daily snapshots of all critical databases.

However, this one-size-fits-all approach, while facilitating compliance, led to an unexpected challenge. The continuous accumulation of snapshots over 2 years resulted in significant month-over-month cost increases. Furthermore, the initial backup policy, though comprehensive, didn’t align optimally with Service NSW’s specific Recovery Point Objective (RPO) and Recovery Time Objective (RTO) requirements.

Collaborative solution development

Recognizing the need for optimization, the Service NSW team engaged with their AWS account team and AWS storage specialists to develop a more sophisticated backup strategy. Through collaborative sessions, the team identified opportunities to balance cost optimization with compliance requirements while promoting robust disaster recovery capabilities.

The resulting solution introduced a tiered backup strategy that intelligently managed data retention across different time horizons. The new approach implemented point-in-time recovery (PITR) providing granular recovery options for recent data. PITR allows you to restore databases to a specific point in time, going back a maximum of 35 days. This was complemented by weekly, monthly, and yearly snapshots meeting various recovery objectives including the NSW government’s 7-year data retention requirement. This structured approach significantly reduced storage costs while maintaining comprehensive data protection.

Technical implementation and innovation

Service NSW’s cloud platform team went beyond basic backup optimization by implementing an innovative automated testing solution. By combining AWS Backup restore testing with AWS Lambda functions, they created a sophisticated system capable of automatically restoring and validating hundreds of databases in a few hours at a remarkably efficient cost. Refer to AWS Backup pricing for details.

The implementation of AWS Backup Audit Manager transformed Service NSW’s compliance reporting process. Instead of manual documentation and validation, the system now automatically generates comprehensive compliance reports, streamlining interactions with auditors and providing consistent regulatory adherence. To learn more about automating compliance, refer to Streamline and automate compliance monitoring and reporting with AWS Backup Audit Manager in the AWS Storage Blog.

Significant results and benefits

The transformation of Service NSW’s backup strategy yielded impressive results. The organization achieved a 70 percent reduction in overall backup costs, effectively eliminating the concerning trend of month-over-month cost increases. This significant cost optimization was achieved while enhancing, rather than compromising, the organization’s backup capabilities.

The automated validation system now provides regular assurance of backup integrity, and the streamlined audit processes reduce the administrative overhead. The new strategy successfully balances cost optimization with operational requirements, which means that databases can be restored within target RPO/RTO while meeting long-term data retention requirements. As Service NSW’s cloud platform team puts it, backups are critical, but recovery is what truly matters.

Looking ahead

Service NSW continues to push the boundaries of backup strategy innovation. The organization’s roadmap includes the development of customized data validation tests for mission-critical databases and the implementation of end-to-end restore testing capabilities that will include application restoration alongside database recovery.

The organization’s commitment to quarterly backup integrity validations demonstrates its dedication to maintaining robust disaster recovery capabilities while optimizing operational efficiency. This forward-thinking approach means Service NSW stays ahead of evolving backup and recovery requirements while maintaining cost effectiveness.

Service NSW’s journey demonstrates how public sector organizations can use AWS services to optimize their backup strategies while maintaining strict compliance requirements. By moving from a straightforward turnkey solution to a sophisticated, tiered approach, Service NSW achieved significant cost savings while enhancing its disaster recovery capabilities. The organization’s innovative use of automation for backup testing sets a new standard for efficient backup validation in large-scale database environments.

How can AWS help?

The successful implementation showcases the value of collaboration between customers and AWS teams in developing solutions that balance cost optimization, operational efficiency, and compliance requirements in the public sector.

AWS offers in-person training, online training, and certification programs. AWS has many AWS Partners and the AWS Professional Services team who can help you build and improve strategies specific to your requirements.

To learn more about how you can use AWS to support your agency’s unique use case, contact the AWS Public Sector team.