City of Avondale Establishes BC and Protection for Critical Services

The City of Avondale, a community in the Phoenix, AZ metropolitan area, is committed to making sure that critical health, safety, utility, and financial services are always available for its residents.

Phoenix International Raceway

Avondale is adjacent to Phoenix, AZ and is part of the largest metropolitan areas in the US. It is a recreation and sports mecca, providing acres of parks for hiking and outdoor activities. The city is also home to the Phoenix International Raceway where, twice a year, hundreds of thousands of visitors watch NASCAR races, and the nearby University of Phoenix Stadium is the home playing field for the NFL’s AZ Cardinals.

A few years ago, Avondale moved to a hybrid cloud IT environment, using vSphere to build an internal cloud computing infrastructure on a FlexPod environment that integrates server, storage, and networking components from Cisco Systems, Inc. and NetApp, Inc. This strategy allows the IT organization to expand to meet the city’s needs while running fewer servers, reducing capital and operating costs. After deploying the FlexPod environment, the IT department recognized it was positioned to improve the city’s DR coverage in ways previously unattainable. The Avondale team wanted a mature solution that’s able to meet its objectives for cost with minimal overhead and tightly integrated with its existing environment.

Enabling BC Avondale chose to utilize VMware’s vCloud Air DR solution, as the city had been successful with VMware and saw the ability to absorb, use, and maintain the solution as long-term advantages.

“We reached our goal of 70% virtualization within a year of using vSphere. VMware continuously advances its technology and provides options that allow us to build onto its solutions. As a CIO, that’s critical when looking at strategic investments,” says Avondale CIO Rob Lloyd.

VMware documentation, training, and support made it easy for the IT staff to learn new skills.

“The tools are easy and the functionality is readily accessible,” says Aaron Favata, IT systems engineer at the city. “VMware provides an excellent solution for DR and offers a natural path to expand use of the platform. The user community and available resources have been superior, and we’ve always been impressed with VMware support.“

Avondale used vCloud Air DR to build a cloud-based failover environment from which it runs 15 of its most critical business applications, including financial, human resources, water and wastewater utilities, traffic, SharePoint, databases, and web servers.

“Now that we’re using vCloud Air DR, we can assure city managers that we remember how everything looked before a disaster happens and we can get everything backup and running as quickly as possible.“

Leveraging built-in failover testing failover and failback operations are crucial to the success of disaster planning. Avondale wanted to be able to conduct affordable, periodic testing of its cloud-based environment with minimal administrative overhead.

“Our goal is to test at least twice a year,” says Lloyd. “vCloud Air DR provides unlimited failover testing, which makes it easy. The test routine is built-in and I can see it working. Now that we’re on vCloud Air DR, we’ll be able to assure the city that we remember what we looked like before a disaster happens and that we have the ability to get back into running mode as quickly as possible. At the end of the project, the team had to show a clean cut-over of a key internal system running on vCloud with a series of business tasks performed successfully. It hit all targets.“

Meeting RPO
For DR, Avondale set a three-hour RPO for important services.

“For standard services, we’ve set a 24-hour RPO,” Lloyd explains. “For critical services, it’s immediate to three hours, depending on the system. Of course, we want as short a recovery time as possible. Traditional approaches called for expensive cold, warm, and/or hot sites, or a longer time to rebuild and environment. vCloud Air DR closes both physical and time gaps.“

Furthermore, the city has built-in retention objectives to meet government policies and laws for records management. Depending on the type of information, the range for keeping records can span from one to 30 years, or forever for capital crimes. The IT team decided that a nightly replication cycle to local and semi-local recovery points would enable them to recover copies quickly.

“We create nightly snapshots locally from our data center and weekly and monthly copies for local offsite backup,” explains Favata. “For regional separation, whatever is replicated to our local data center is also replicated nightly to a different state location using vCloud technology. We can increment that to near-real time as needed. That means we can recover critical municipal services, even if there is a catastrophe that causes major impairments in the region.“

Business benefits
“We wanted to do a better job with fast recovery and business resumption,” says Lloyd. “When we started looking at solutions, some providers were more rudimentary than we expected and others were too expensive. VMware solutions are priced realistically. With VMware for DR, we finally have a comfortable level of assurance at a reasonable price. And we can take advantage of VMware technology to bring solutions in and out of the cloud easily.“

The city designed, tested, and launched a DR strategy in less than three months. As a result, Avondale has established an active backup environment in another state with replication. The city’s IT staff can take advantage of built-in test features for routine background failover testing to validate business resumption capacity for business lines and auditors.

“Now that vCloud Air DR is set up, the solution mostly drives itself,” says Favata. “I check daily to make sure all the servers are replicated, but it really hasn’t added any time to my work.“

Case study