Disaster Recovery Testing: Best Practices by Company Size and Cloud Provider

A disaster recovery (DR) plan is only as effective as the test you put it through. Most companies have something written down. Far fewer have actually run through it under realistic conditions. This article covers:

• Types of DR testing that can be done

• General best practices

• Best practices for: AWS, GCP, and Azure

• Best practices by company size

What Is a DR Test?

A disaster recovery test is a planned exercise where you simulate a system failure and verify that your team can restore operations within acceptable timeframes. The two numbers you're trying to hit are:

• RTO (Recovery Time Objective): How long it takes to get back up and running after an incident

• RPO (Recovery Point Objective): How much data you can afford to lose (measured in time - e.g. "up to 4 hours of data")

Testing tells you whether your real-world RTO and RPO actually match what's written in your DR plan. They usually don't on the first attempt. That's the point.

Types of DR Tests

There are four main types, ranging from low-effort to full simulation:

Tabletop exercise - The team walks through a scenario verbally. No systems are touched. Good for checking whether people know their roles and whether the plan makes logical sense. Fast, zero risk, easy to do quarterly.

Walkthrough / document review - Using an existing document, you verify that backup configurations exist, that they're running on schedule, and that restore procedures are documented accurately.

Recovery test - You restore a non-production copy of your data (staging/dev environment) or spin up a standby environment and verify it works. You're touching real systems but not affecting production.

Full test - This is a documented recovery from a real incident. If your system went down and your team recovered it, that counts - as long as it was recorded properly. What makes it count as evidence is the write-up: what failed, what steps were taken, how long it took, and whether you hit your RTO and RPO. A planned full failover exercise (where you intentionally cut over to a backup environment) also qualifies here.

For compliance purposes, you need evidence that testing of one of the above happened. Having written records of who did it, when, what was tested, what the outcome was, and whether you hit your RTO/RPO targets.

General Best Practices (All Companies)

Test on a schedule, not just after incidents. Once a year is the absolute minimum for compliance. Quarterly tabletop exercises should be the norm, with a partial restore test at least twice a year.

Document everything before you test. Your DR procedure needs to be written down before you run through it - not after. Auditors want to see that you followed a procedure, not that you reverse-engineered one from what actually happened.

Define success criteria in advance. Before starting the test, write down exactly what you're trying to prove. Did the restore complete in under 30 minutes? Did data loss stay within 4 hours? Was the restored environment fully functional? You can't grade a test you haven't defined.

Include the right people. At minimum: whoever owns the infrastructure (usually engineering), and whoever would be responsible for declaring an actual disaster (usually a founder, CTO, or VP Eng at startups). If you have customers, someone from customer-facing teams should understand the communications plan too.

Test restores, not just backups. A backup that's never been restored is not a verified backup. The test should include actually pulling data from the backup and confirming the restored system works. This sounds obvious but it's the most commonly skipped step.

Track findings and close the loop. Every test should produce a short report: what worked, what didn't, and what's being fixed. If you find gaps, log them as action items and track them to completion. Auditors want to see that testing leads to improvements, not just checkboxes.

Don't test in production during peak hours. Schedule any tests that touch live systems during low-traffic windows. Always have a rollback plan.

AWS

AWS gives you a lot of built-in tools for DR. The challenge is using them correctly and verifying they work.

Key services: RDS automated snapshots, S3 versioning and cross-region replication, EC2 AMIs, AWS Backup (centralized), Route 53 for DNS failover, and AWS Elastic Disaster Recovery (DRS) for full server replication.

What to test:

• Restore an RDS snapshot to a new instance and confirm the database is accessible and data-complete

• Confirm S3 versioning is enabled on all production buckets and test rolling back a specific object to a previous version

• If you're using multi-AZ deployments, simulate an AZ failure and confirm automatic failover triggers correctly

• If you use Route 53 health checks with failover routing, confirm DNS actually routes to the backup target during a simulated failure

Common gaps:

• Backup retention periods set too short (1-day default on RDS - confirm your retention window matches your RPO)

• S3 lifecycle policies that delete versions sooner than expected

• IAM permissions that prevent the designated restore person from actually triggering a restore (test permissions separately)

• Restores tested only on one region when the failure scenario involves that region being unavailable

Evidence to capture: AWS Backup job logs, RDS event logs showing restore completion, screenshots of the restored environment, and the time elapsed from start to finish.

Google Cloud Platform (GCP)

GCP's DR tooling is solid but less centralised than AWS Backup. You're often managing backup policies per service, which means more to verify.

Key services: Cloud SQL automated backups, Cloud Storage versioning, Persistent Disk snapshots, Filestore Backups (for managed file storage), and Google Cloud Backup and DR (the dedicated service). For full instance recovery, you'll use custom machine images or Terraform/Deployment Manager to recreate environments.

What to test:

• Restore a Cloud SQL backup to a new instance and verify connectivity and data integrity

• Test Point-in-Time Recovery (PITR) for Cloud SQL - confirm you can restore to a specific timestamp within your backup window

• Verify Cloud Storage object versioning is enabled and test restoring a deleted or overwritten object

• If using multi-region storage or Cloud Spanner, test behaviour during a simulated regional disruption

Common gaps:

• Cloud SQL PITR works differently depending on the database engine. For MySQL instances, it requires binary logging to be explicitly enabled - this is not on by default. For PostgreSQL instances, PITR uses WAL archiving and is enabled through the automated backup settings. Either way, check that it's actually turned on before assuming PITR is available.

• GCP backup configurations are set per resource (per Cloud SQL instance, per Persistent Disk, etc.) - there's no project-level policy that automatically covers everything. Each resource needs to be individually verified. A common gap is assuming a backup schedule is running on a new resource just because it was configured on an older one.

• Persistent Disk snapshot schedules don't automatically snapshot all disks unless explicitly configured per disk

• No centralised backup dashboard by default - you need to check per-service or use Cloud Backup and DR to consolidate

Evidence to capture: Cloud SQL backup restoration logs, Cloud Storage access logs showing version retrieval, timestamps from Cloud Console or gcloud CLI showing restore duration.

Microsoft Azure

Azure has strong native DR tooling, particularly Azure Site Recovery (ASR) for full VM replication and Azure Backup for database and file-level backups.

Key services: Azure Backup (VMs, SQL, files), Azure Site Recovery (full replication + orchestrated failover), Azure SQL automated backups with geo-restore, Azure Blob Storage soft delete and versioning, and Recovery Services Vault as the central backup management layer.

What to test:

• Restore an Azure SQL database from an automated backup to a new server and confirm it's accessible

• If using Azure Site Recovery, run a test failover - ASR has a built-in "Test Failover" option that spins up a replica environment in an isolated network without affecting production

• Confirm Azure Blob soft delete is enabled and test recovering a deleted blob within the retention window

• Verify that Recovery Services Vault backup jobs have been completing successfully (check the job history, not just assume they're running)

Common gaps:

• Azure Site Recovery replication needs to be configured per VM - it doesn't replicate everything by default

• Backup policies in Recovery Services Vault have a default retention that may not match your RPO - review and customise

• Test Failover in ASR is isolated from production, but teams often confuse it with a real failover - make sure the team understands what they're running

• Geo-redundant storage (GRS) doesn't automatically make your data recoverable in the secondary region - you need geo-restore to actually access it during a real failure

Evidence to capture: Recovery Services Vault job logs, ASR test failover job history from the portal (timestamps, status, and completion details are logged automatically and serve as audit evidence), screenshots of the restored database/environment, and elapsed restore time.

By Company Size

The right DR testing approach depends heavily on how much infrastructure you're running and how many people you have to run it.

0-25 Employees

At this stage, you're probably running on a single cloud provider with a small infrastructure footprint. One or two engineers own everything, and DR testing tends to fall off the calendar because there's always something more urgent.

What's realistic:

A tabletop exercise twice a year is achievable and doesn't require much engineering time. Get the founder or CTO in the room, walk through the scenario: "Our database goes down at 2am on a Tuesday. What do we do?" Write down the steps. Identify who does what. Time it on paper.

Once a year, actually run a restore. Pick a low-risk time (weekend, low usage), spin up a test database from your latest backup, confirm the data looks right, and document the time it took. This is the evidence your auditor wants.

Things to confirm at this size:

• Backups are running and someone is notified if they fail

• At least one other person (beyond the primary engineer) knows how to trigger a restore

• RTO and RPO are written down somewhere - even a Google Doc is fine

• If the primary technical person leaves or is unavailable, someone else can access the cloud console

Common mistake: Relying entirely on one engineer who "knows where everything is." If that person is sick or unreachable during an actual incident, the DR plan falls apart. Document the steps.

25-50 Employees

At this size, you're likely serving real customers, possibly in regulated industries, and the stakes of a real outage are higher. You probably have a small engineering team (3-8 people) and may be going through your first audit.

What's realistic:

A quarterly tabletop exercise with the engineering lead and a member of the customer team. At least two partial restore tests per year - one for your primary database, one for a secondary system (file storage, auth, etc.). If you have a staging environment that closely mirrors production, use it for DR testing.

Start building an on-call rotation if you don't have one. Your DR plan needs to name who gets paged and in what order, not just say "contact the engineering team."

Things to confirm at this size:

• Backup retention matches your stated RPO - if you say "4-hour RPO," daily snapshots aren't enough

• Two people (minimum) have the credentials and permissions to trigger a restore

• There's a communications template for notifying customers if there's an outage - tested and ready to go

• Your RTO has actually been measured, not just estimated

• DR testing results are documented in writing, even if informally

Common mistake: Setting RPO/RTO targets in the DR plan that don't reflect actual backup frequency. If your backups run once daily, your real RPO is up to 24 hours - not the 4 hours that's written down. Auditors check for this.

50+ Employees

At 50+ employees, you're likely serving a larger customer base, possibly with enterprise contracts that include SLAs. Your DR program needs to be formalised, tested regularly, and have clear ownership.

What's realistic:

Quarterly tabletop exercises for relevant teams (engineering, ops, customer success). Semi-annual partial restore tests. At least one full failover test per year if your architecture supports it. A dedicated DR runbook - not just a paragraph in your security policy.

You should have formal roles defined: incident lead, communications lead, executive sponsor. Each person should know their role before an incident happens.

Things to confirm at this size:

• You have a tested runbook with step-by-step restore procedures (not just high-level notes)

• DR testing is on a formal calendar with an assigned owner

• Test results feed into a risk register or action item log that gets reviewed

• You've tested cross-region or multi-cloud failover if your architecture uses it

• Customer-facing SLAs are backed by RTO/RPO targets that are actually achievable and tested

• Backup monitoring includes alerting - someone is notified if a backup job fails, not just if an incident occurs

Common mistake: DR testing exists on paper (policy says "annually") but hasn't happened in 18 months because it keeps getting deprioritised. Assign it to a specific person, put it on the engineering calendar, and treat missed tests as a compliance gap - because that's exactly what they are.

What Auditors Look For

Regardless of your cloud provider or company size, auditors reviewing DR as part of SOC 2, ISO 27001, or HIPAA will typically want to see:

• A written DR plan with defined RTO and RPO

• Evidence that backups are running and completing successfully (logs, not just "we think they're running")

• Evidence that at least one restore test has occurred within the past 12 months

• A written record of the test results - what was tested, the outcome, and any gaps identified

• Evidence that gaps found during testing were addressed (action items closed out)

If you can produce those five things, you're in good shape for most DR-related audit questions.

TLDR