AWS Backup and CI/CD Tools to Automate Centralized Backup Across AWS Services 

By Ibukun Oyewumi, senior security consultant, AWS,

Khurram Nizami, operations management consultant, AWS, and

Sabith Venkitachalapathy, enterprise solutions architect, AWS

Automating and scaling your data protection and backup strategy helps you reduce manual overhead from time-consuming configuration, minimizes the risk for errors, provides visibility on drift detection, and enhances backup policy compliance across distributed AWS workloads or accounts. Incorporating backup in your DR and business continuity plan (BCP), along with the automation of backup operations, are important security best practices to secure your backup data and operations in AWS.

Organizations are leveraging modern software development practices such as continuous integration (CI) to implement frequent, reliable, and incremental code changes and continuous delivery (CD) to deliver code. CI/CD workflows are orchestrated by a pipeline to automate steps in the software delivery process based on the release process models you define, such as initiating automatic builds when there is a code change and then deploying to Amazon Elastic Container Service (Amazon ECS).

In previous blog posts, we covered how you can automate centralized backup at scale using AWS Backup and AWS CloudFormation infrastructure as code (IaC). In this blog post, we demonstrate how you can align with modern software development practices to scale your backup operations using AWS CI/CD tools and AWS Backup as the fully managed backup service to centralize and scale the backup of data across AWS services. Using this solution, you can build and implement a cloud data protection strategy to centrally automate and scale the process of implementing AWS Backup policies, backup vaults, and cross-region, cross-account replication across your multi-account AWS environment using software development practices that mitigate the risk of data loss.

Solution overview
The architecture uses AWS Organizations to centrally manage and programmatically scale data backup across AWS services. It consists of at least 4 accounts: a CloudFormation delegated administrator account, an AWS Organizations Management account, a centralized backup account, and multiple member accounts belonging to the same organization. To implement the solution, we use AWS CodeCommit for managed source control service, AWS CodeBuild for continuous integration, AWS CodeDeploy for automated code deployment, AWS CodePipeline for continuous delivery, CloudFormation as the IaC tool, and AWS Backup as the fully managed backup service to implement the backup of data across AWS services.

Click to enlarge

The solution provides a CI/CD-based deployment method for implementing backup and recovery with AWS Backup across your AWS organization spanning multiple accounts and regions. It uses a service-managed StackSet to configure AWS Backup in accounts within the AWS organization units (OUs) you specify. New accounts added to the target OUs will automatically be configured with the solution described in this post.

Solution includes following features:

• CloudFormation templates for setting up the baseline solution architecture.

• CodePipeline for orchestrating and automating the management, testing, and deployment of solution components.

• AWS Lambda-backed CloudFormation custom resource for managing AWS Backup policies to define how you back up your resources using backup plans.

• Lambda function to automatically reapply tags to restored resources from AWS Backup.

• Amazon S3 bucket for centralized and automated deployment of lambda functions to member accounts and Regions.

• CodeBuild project for static security analysis of CloudFormation templates using cfn_nag to look for patterns that may indicate insecure infrastructure.

• CodeBuild projects for packaging and deploying the solution’s Lambda functions.

• AWS Backup Vault, AWS Key Management System (AWS KMS) key, and AWS Identity and Access Management (IAM) service role for deployment of AWS Backup resources to each member account and region

• AWS Backup Vault, KMS key, and IAM service role for centralized AWS Backup resources to provide secondary backup storage across all accounts and regions.

• Sample AWS Organizations service control policy to limit sharing and copying of AWS Backup vaults to only AWS accounts within the AWS Organization.

We recommend you modify and customize this solution to meet your needs. Optionally, you can deploy this solution in Customizations for AWS Control Tower.

The source code and latest solution instructions can be downloaded from this GitHub repository.

Prerequisites
Following prerequisites are required to deploy the solution.

1. Your AWS accounts must belong to the same organization (see the documentation on AWS Organizations).

2. Basic knowledge of CloudFormation StackSets, Lambda functions, Python, and setting up a CI/CD pipeline on AWS.

3. Install the latest version of the AWS CLI or use the AWS CloudShell. To use the AWS CLI, we recommend you assume a role in the destination accounts.

4. Opt in to use AWS Backup. If this is your first time using the AWS Backup service, you must opt in to use AWS Backup and cross-account management features using the AWS Management Console or CLI. See the documentation on how to Opt in to use AWS Backup for additional information.

5. Verify that the solution prerequisites in the GitHub solution repository are satisfied.

Walkthrough
We recommend that you fork the GitHub repository for the solution and then clone your forked GitHub repository.  This will allow you to receive and apply updates to your deployed solution if it changes over time.

Deploy the solution by following these steps.

A link is provided in each step that details the commands you must execute:

1. Setup the central backup account: This will deploy an AWS Backup vault and related solution resources to store backup copies for all your AWS Organization accounts participating in the solution. This provides a secondary, isolated fail-safe for your backups improving the security and resiliency of your AWS backup architecture.

2. Setup the solution home account: You will need to select a solution home account and region. This account and region will contain the CodePipeline and CodeCommit repository and will enable the CI/CD of the BackupOrgPolicyManager CloudFormation custom resource used to manage and deploy your AWS Organizations backup policies using an Infrastructure as Code approach.  The solution uses SSM to lookup configuration information such as the target AWS organizational units containing the AWS accounts that will participate in the solution.  We recommend that you specify a test AWS Organizational Unit as a target first to allow you to test your solution before wider deployment to multiple organizational units.

3. Setup the BackupOrgPolicyManagerOrgAdmin IAM role: The BackupOrgPolicyManager AWS Lambda Function assumes the BackupOrgPolicyManagerOrgAdmin role in your AWS Organizations Management account in order to manage backup policies for the AWS organization.  You must deploy the role, BackupOrgPolicyManagerOrgAdmin into your AWS Organizations Management account so that the solution home account can assume this role to manage your AWS organizations backup policies.

Usage
When you complete step 2 in this blog, the solutions CodePipeline will automatically execute and will fail until step 3 is completed. You should review and update the aws-backup-org-policy CloudFormation template integrated and deployed with the solution CodePipeline to implement your desired AWS Backup policy.

This example implements the following policy properties:

• For all supported resources in the target regions and target OUs that are tagged with the tag key backup and the tag value daily, perform a nightly backup at 05:00 UTC. The backup will have a retention period of 35 days. The backups will also be copied to the selected central backup vault. If the backup doesn’t complete in 1,200s (20 hours), then it will be canceled.

• For all supported resources in the target regions and target OUs that are tagged with the tag key backup and the tag value monthly, perform a monthly backup at 05:00 UTC on the first day of each month. The backup will have a retention period of 366 days. The backups will also be copied to the selected central backup vault. If the backup doesn’t complete in 1,200s (20 hours), then it will be canceled.

The solution’s BackupOrgPolicyManager CloudFormation custom resource makes it easy for you to centrally manage your backup policies using a CI/CD, IaC approach.  As demonstrated in this example, you can provision multiple instances of this custom resource so you can deploy multiple backup policies with varying properties.

Once you have made changes to the aws-backup-org-policy CloudFormation template you can commit your changes to the CodeCommit repository in order to automatically initiate the solution CodePipeline to test & deploy your backup policy.

You should be able to confirm a completed pipeline initiation (in green) similar to the image in the following figure in your solution home account.

If you deployed the example template, you should also be able to confirm 2 backup policies similar to the following figure implemented in your management account.

Click to enlarge

Customizing backup policies with BackupOrgPolicyManager AWS CloudFormation custom resource
The BackupOrgPolicyManager CloudFormation custom resource allows you to define backup policies for different organizational units and regions in your AWS Organization. It is recommended that you create a separate custom resource for each backup policy. This will allow you to make smaller, incremental changes to your backup policies. The aws-backup-org-policy CloudFormation template example integrated in the open-source solution follows this approach. You may also choose to create separate custom resources aligned to your organizational units. For example, you may have an organizational unit with different backup schedule and retention requirements.  You can define separate custom resources for each of these so you can manage the backup policies for them via CloudFormation separately. When defining policies, you should observe the inheritance impact of the backup policies you define to make sure they have the desired effect.

Cleaning up
To avoid incurring future charges, delete the resources. First, delete the stack instances, then delete the StackSets from the AWS CloudFormation console or CLI. To reduce costs, clean up the backups that are no longer required for recovery or retention purposes. You can use AWS Backup and Amazon Data Lifecycle Manager to automate your retention policy for a portion of your backups.

Conclusion
In this blog post, we showed you how to implement and automate your data protection and backup strategy using modern CI/CD software development practices to manage organization-wide backup policies using AWS Backup. We demonstrated how to apply your backup policies from a delegated administrator account to scale and manage backup across AWS services in your member accounts.

Automating backup using CI/CD pipelines is important to implement frequent, reliable, and incremental code changes with quick and seamless delivery of code. Automation reduces manual overhead, provides central code source control, continuous integration and deployment, and enhances backup policy compliance at scale.

A backup strategy is an essential part of an organization’s data protection plan to withstand, recover from, and reduce any impact that might be sustained due to a security event. You should define an extensive backup strategy with detailed backup plans and resource assignments to reflect your enterprise data protection policies. Automating and deploying backup policies or organization-wide backup plans using CI/CD deployment methods allows you to standardize and scale your backup strategy to mitigate the risk of data loss.

To get started with AWS Backup in the console, visit the AWS Backup console.