Use 3-2-1-1-0 without counting a fourth copy by accident. Test protected versions, recovery keys, actual RPO/RTO and a clean business restart.
Auteur
Anichur Rahaman
The backup job is green. The owner asks a different question: “Can we run the shop from it on another server?” In a planned drill, the database opens, but invoices point to missing files and nobody can retrieve the decryption key. This is a fictional scene, not a customer incident.
The job copied something. The business has not recovered. Ransomware makes that gap urgent because an attacker may also compromise reachable backups, identities or the controls needed to restore them. Backups also do not undo stolen data or remove an intruder.
A useful backup plan ends with a working, validated service and a named person who can make the recovery decision. The numbers in 3-2-1-1-0 help organise the plan; they are not a guarantee.
The 3-2-1-1-0 guideline means three copies, two media, one off-site, one offline/air-gapped/immutable and zero errors in recovery verification. The production copy normally counts among the three.
An illustrative layout is the live system, a separate local backup and an off-site protected backup. The third can satisfy both off-site and protected-copy requirements; the rule does not automatically demand a fourth retained copy. A temporary restore target is for testing, not a substitute for retained history.
Different folder names on one disk are not independent copies. Look beyond the storage diagram: shared admin login, identity provider, physical location, encryption key, billing account and retention controls can create common failure. Replication and RAID can improve availability while faithfully reproducing deletion or corruption; they are not the whole backup strategy.
An offline device must actually be disconnected and protected between controlled runs. A connected pull-based backup server has useful separation, but is not automatically air-gapped or immune to attack. Test the access boundaries rather than relying on the label.
AWS Object Lock documentation protects specified object versions; newer versions and delete markers can still exist. Compliance and governance have different override rules. Do not treat the visible latest object as the only recovery copy.
AWS also warns about encryption-key loss: locked objects can become unreadable if required keys are lost or deleted. Immutability protects a retention property, not every dependency needed for recovery.
Ask your provider about actual version protection, retention shortening, privileged override, account closure and restoration of a non-current version. Test on disposable objects before committing valuable data to an irreversible retention setting. S3 compatibility does not prove identical lock semantics.
Separate routine backup writers, protected-storage administrators and recovery readers. Grant the permissions each workflow really needs, including verification or multipart operations where applicable; “write-only” is a design goal for the production writer, not a universal magic permission set. A writer may still upload garbage or consume capacity. Protect administration with separate controlled identities and strong authentication, and monitor missing jobs, failures, retention changes, key changes and unexpected growth.
Recovery objectives distinguish tolerable lost-data age(RPO) from tolerable service interruption(RTO). Choose them with business owners and dependencies, not by copying a tier table.
In an illustrative drill, interruption starts at 09:12. The selected recoverable data contains committed transactions through 08:45. Staff validate the agreed business service at 11:05. The lost-data gap is 27 minutes; end-to-end recovery is 113 minutes. Against RPO 15 minutes and RTO 120 minutes, the data objective misses by 12 minutes while the time objective fits with 7 minutes to spare.
A file uploaded at 09:10 does not change that data point if it only contains 08:45 data. Likewise, a database booting at 10:00 does not stop the recovery clock if staff still cannot open invoices or reconcile payments. Record the agreed start, data point, validation scope and completion.
Frequency must support the promise: one monthly copy cannot by itself deliver a one-week RPO for changing data. For PostgreSQL point-in-time recovery, the documentation requires a suitable physical base backup and continuous required WAL. A logical pg_dump is not a base for WAL replay. Verify the complete chain and the version-specific recovery procedure.
A ransomware-safe restore point may be older than the newest available copy. Investigate compromise and assess which point is safe; do not assume that “before encryption” means “before intrusion”. Reflect the resulting data gap and reconciliation work honestly.
| Drill measure | Illustrative target | Observed result | Interpretation |
|---|---|---|---|
| Data gap(RPO measure) | 15 minutes | 09:12−08:45=27 minutes | Misses by 12 minutes |
| Service recovery(RTO measure) | 120 minutes | 09:12→11:05=113 minutes | Within this target by 7 minutes |
| Business validation | Defined critical workflows | Files, keys, access and external reconciliation included | A booted database alone is insufficient |
Inventory databases, uploaded invoices/media, object versions, configuration, software/build versions, jobs, integrations and the documented rebuild procedure. Select a consistent recovery set; restoring database and files from unrelated points can leave missing or wrong attachments. A copying command over live database files is not automatically a consistent database backup.
Protect encryption in transit and at rest according to the threat model. Keep the required decryption material under separate controlled custody, with tested emergency access. Preserve necessary historical keys for retained backups and encrypted application records. Do not blindly replace an application encryption key: rotation may need a controlled migration and access to earlier keys.
Test recovery when the normal administrator, password vault, identity provider or primary region is unavailable. An encrypted archive is not useful if the only way to unlock it disappeared with production. Keep a protected offline runbook and current contacts; do not print usable secrets into a general-purpose handbook.
Choose retention for business, legal and incident-discovery needs, restore-point diversity and storage cost. A fixed 30/90/365-day schedule is not a universal rule. Alert on age of the last recoverable point, not just presence of a newly named file. Checksums help detect alteration; they do not prove a backup is clean, complete or useful.
Use an isolated, access-controlled restore environment, with outbound payments, email, courier calls, scheduled jobs and webhook side effects blocked or safely substituted before services start. Restored customer data still needs privacy controls and secure disposal after the drill. Never point a test restore at live integrations just to see what happens.
Verify sample records and balances at the chosen recovery point, open representative attachments, check decryption and permissions, and exercise critical workflows with safe test identities. Include provisioning, download, replay, configuration, authentication and business validation in the elapsed time. Merely comparing a current production row count to an older backup can produce a false alarm.
Reconcile external truth before resuming: a payment provider may have captured money after the database recovery point, and a courier may already hold a parcel. Do not rerun old queues, charge again or ship again blindly. Assign exceptions and use stable references and duplicate protection.
Plan frequent lightweight verification plus representative full restores and business exercises on a risk-based calendar. Test a non-current protected version, missing key, broken log chain, absent administrator and restored-job replay. Keep failures visible with owners and deadlines. “Zero errors” describes the verification result and scope; one clean drill cannot promise all future recoveries.
Name an incident lead and recovery approver, with clean out-of-band contact paths. Isolation, evidence preservation, backup protection and specialist response should begin promptly; calling for help is not reserved for when all backups fail. The Canadian Cyber Centre playbook separates response and recovery and warns against credential changes that would block restoration.
Contain affected access and systems with the responders. Avoid a universal “never power off” instruction: preserving volatile evidence and stopping ongoing harm must be balanced by the incident team. Choose clean rebuild targets, address persistence and entry risks, and approve reconnection in stages after validation.
Rotate compromised access credentials from a clean context while retaining controlled recovery/decryption capability. Keep backup copies protected and preserve evidence. Do not make a broad “change every key now” action that destroys the ability to restore.
Assess data theft, contractual duties, insurer requirements and applicable notification obligations with the responsible advisers. Restoring operations does not automatically close the security incident or reverse disclosure. Record decisions and actions; review the plan after the incident or drill.
Pick one important service and have a person other than its usual maintainer follow the runbook. Show the protected version, usable key path, clean target, validated business workflow and measured RPO/RTO. Log what did not work and fund that gap before promising a recovery time to customers.
The existing interface walkthrough below is a product demonstration, not proof that your own setup has immutable copies or meets a target. Keep the acceptance test specific to your environment.
A green backup job is a useful signal. The stronger evidence is a repeatable recovery whose limits you can explain.
Reviewed 5 November 2026. Anichur Rahaman is the creator of StoreConsole. Scenarios, targets and timings are illustrative; provider capabilities and recovery duties must be checked for your environment.
À propos de l'auteur
Anichur Rahaman
Continuer la lecture
