What to take away
- Redundant components can still share a dependency or a control decision that prevents recovery.
- A successful backup job does not demonstrate that the application can be restored within the business’s tolerance.
- Test recovery through a customer journey, including data, identity, capacity and communications.
What happened in London
On 20 October 2026 UK time, a Google Cloud facility serving the europe-west2-b availability zone lost power after an external electrical fault. Google’s updated report describes a control-system design and configuration problem that prevented the generators from starting automatically in the facility’s maintenance state. Battery reserves were exhausted, and workloads in the zone became unavailable. Failover also increased pressure on other infrastructure in the region. Google reported resolution of the main impact within six hours and eight minutes. [1]
This was an operational outage, not a disclosed cyberattack. The incident illustrates how physical power, control logic and application recovery can interact. It does not establish that every Google Cloud customer was affected or that every workload in London had the same recovery experience. The date also appears as 19 October in the report’s US/Pacific timeline; this article uses the UK date. [1]
For a smaller business, the valuable question is not whether it could have repaired a cloud provider’s power system. It could not. The question is what the business expected its own service to do when one part of the provider became unavailable—and whether that expectation had ever been tested from a customer’s point of view.
An infrastructure diagram may show two servers, a replicated database and a backup bucket. Those are useful components, but they do not automatically add up to recovery. The path back to service also includes permissions, routing, available capacity, configuration, people and decisions. A recovery plan becomes credible when these pieces have been exercised together under a defined failure scenario.
Count independent paths, not just duplicate components
Two copies of something are only as useful as the independence of the paths that make them work. Two application servers may share one database. Two databases may depend on the same identity service or configuration store. Two recovery operators may both need access through a platform that is part of the outage. These are general design questions, not additional findings about the London incident.
Draw the customer journey and identify the dependencies behind each step. A person visits the website, signs in, retrieves a record and submits a transaction. Which services make those steps possible? Where is the authoritative data? How are secrets obtained? What routes traffic to the healthy application? A component-level availability claim should be translated into this end-to-end behavior before the business relies on it.
Ask what changes during maintenance or degraded operation. A system may have several independent paths in its normal state but fewer while one component is being repaired. That temporary condition deserves explicit attention because it changes the assumptions behind failover. A recovery exercise that always begins with every backup component healthy may miss the conditions that make a real incident difficult.
Document the scenario you are designing for. Surviving a single process failure is different from surviving a zone outage, a region outage or loss of administrative access. A small business does not need the most expensive design for every possible event. It does need to understand which failures its current design is intended to tolerate and what the plan is when the incident exceeds that boundary.
Translate downtime into a business decision
Start with the operation the business must preserve. A marketing website, a customer support portal and a payment-processing workflow may have very different tolerances. Ask how long each can be unavailable, what work can continue manually and what commitments have been made to customers. Without those answers, an engineer cannot sensibly choose between a simple restore process and a continuously available secondary environment.
Two planning terms help make the discussion concrete. The recovery time objective describes the target time to restore service after a disruption. The recovery point objective describes the acceptable period of data loss. Google’s planning guide treats these as business-driven design inputs and notes that tighter objectives generally require more cost and complexity. [2] They are targets to validate, not guarantees created by writing numbers in a document.
Consider a hypothetical appointment platform used by clinics in the Philippines. It might tolerate a short period of read-only access while staff continue checking existing appointments, but it cannot silently lose confirmed bookings. The business may therefore choose different recovery behavior for viewing schedules and accepting new appointments. This example is invented; the point is that one uptime percentage does not express every operational requirement.
Write down who accepted the tradeoff. If the company chooses a lower-cost restore approach that may take several hours, customer promises and support procedures should reflect that choice. If sales promises near-continuous availability, the architecture and operating process need evidence to support it. Recovery planning is a shared business decision, not an engineering team’s private guess about how much downtime people will tolerate.
Follow the complete route back to service
- Recover data
Restore the required records and check integrity.
- Restore access
Make authorized identity and secrets available.
- Prove capacity
Support the essential customer workload.
- Verify the journey
Confirm the business action works end to end.
A backup is an input to recovery
A completed backup job establishes that a backup process reported success. It does not establish that the application can use the result, that the data is complete or that the team can restore it quickly enough. A useful recovery test follows the backup through restoration, configuration and a representative customer workflow. That is where hidden dependencies often become visible.
Check that recovery material remains available during the chosen failure scenario. The database backup may exist, but the deployment instructions could be stored inside the unavailable application. The encryption key might depend on an inaccessible account. The infrastructure definition may be on a developer’s laptop. These are hypothetical examples, but they explain why the plan must include the means of recovery as well as the data being recovered.
Use an isolated environment for routine restore exercises. Restore an approved backup, verify representative records and confirm that sensitive information remains protected. Avoid casually cloning production data into an environment with weaker access controls. If realistic data is needed, agree the handling requirements and retention before the exercise. Resilience testing should not create a new, poorly governed copy of customer information.
Measure the whole process. Include the time needed to obtain approval, authenticate, provision resources, restore data, apply configuration and verify the service. A database restore taking twelve minutes is useful information, but it is not the same as the customer-facing application recovering in twelve minutes. Record the start and end conditions so repeated exercises can be compared honestly.
Measure recovery with meaningful checkpoints
A recovery record should distinguish the moment of failure, the most recent accepted business operation, the newest operation present after restoration and the time the customer journey works again. These checkpoints answer different questions. The service may be reachable before recent records are available, or the data may be restored while users still cannot sign in. Collapsing those milestones into one “recovered” timestamp hides important gaps.
The local Python example below calculates two values from fictional exercise timestamps. The first is the elapsed recovery time. The second is the age of the newest recovered write relative to the last accepted write. It makes no cloud calls and does not verify the contents of a database. A real exercise must establish those timestamps from trustworthy evidence and separately check that the recovered records are correct.
Be careful interpreting the data gap. Two timestamps do not reveal how many transactions are missing or whether partially completed operations need reconciliation. A five-minute gap during an inactive period has a different consequence from five minutes of busy checkout activity. Record both timing and business impact. If the system accepts work asynchronously, define exactly what “accepted” means and what acknowledgment the customer received.
Keep the result useful to the business. Report which journey was tested, whether the target was met, what failed and the next improvement. A short, repeatable exercise record is more valuable than an impressive recovery estimate nobody has demonstrated. Over time, the evidence can guide spending toward the actual bottleneck rather than whichever infrastructure component is most visible.
from datetime import datetime
def utc(value):
return datetime.fromisoformat(value.replace("Z", "+00:00"))
failed_at = utc("2026-11-19T02:00:00Z")
customer_journey_restored = utc("2026-11-19T02:42:00Z")
last_accepted_write = utc("2026-11-19T01:59:50Z")
newest_recovered_write = utc("2026-11-19T01:56:00Z")
recovery_minutes = (customer_journey_restored - failed_at).total_seconds() / 60
data_gap_seconds = (last_accepted_write - newest_recovered_write).total_seconds()
assert recovery_minutes >= 0 and data_gap_seconds >= 0
print(f"Observed recovery time: {recovery_minutes:.1f} minutes")
print(f"Observed write-time gap: {data_gap_seconds:.0f} seconds")The recovery environment needs enough room to work
Failover changes load. If a secondary environment normally serves little traffic, its ability to run a health check does not establish that it can handle the full customer workload. Consider application workers, database connections, queues, external APIs and account quotas. The limiting resource may be somewhere other than the server instance that appears on the architecture diagram.
Define a manageable degraded mode. A business may be able to preserve account access and essential transactions while temporarily suspending large reports or nonessential background processing. That decision should be designed and communicated before an incident. A system that tries to do everything during recovery can make its most important function harder to restore, particularly if delayed work arrives in a large burst.
Review retry behavior with the development team. Repeated requests can be useful when a service briefly fails, but unlimited retries can amplify pressure and confuse transaction outcomes. The appropriate design depends on the operation: reading a status page is different from creating a payment or booking. Use bounded behavior and an explicit way to determine whether a business action already succeeded before repeating it.
Test the return to normal operation as well as the move away from failure. Reconnecting the original environment, draining queues and reconciling records can introduce a second period of risk. Decide which system is authoritative and how duplicate or conflicting operations will be handled. Recovery is not finished merely because both environments are running again; the business needs a consistent account of what happened.
Recovery also depends on people and access
A technical runbook should identify who can declare an incident, who can authorize failover and who communicates with customers. In a small company, one person may hold several roles, but the responsibilities still need to be explicit. If the only administrator is asleep or travelling, the plan should say how another authorized person can obtain the necessary access without improvising unsafe shortcuts.
Protect emergency access while keeping it usable. A recovery account that nobody has tested may fail when needed; an unrestricted shared password in a chat history creates another problem. Agree a controlled method with appropriate authentication, accountability and review after use. Check whether the emergency route depends on the same identity provider or device that the scenario assumes is unavailable.
Keep essential instructions available through an appropriate independent route. This may include contacts, service ownership, recovery order and the location of approved backups. Do not duplicate live credentials into ordinary documents to achieve convenience. The aim is to preserve the ability to coordinate and locate authorized access, not create an unprotected alternative to the company’s normal security controls.
Exercise the human handoffs. Ask a colleague who did not write the runbook to follow it in a safe test environment. Notice where they need undocumented knowledge or an approval the document never mentioned. Those pauses are valuable findings. They reveal whether the organization has a repeatable recovery process or merely a knowledgeable person who can usually repair things.
Keep customers informed without guessing at the cause
An outage message should begin with the customer impact: what is unavailable, which functions remain usable and when the next update will arrive. The precise root cause may take longer to establish. It is better to say that a provider incident is under investigation than to label every service disruption a cyberattack or make unsupported claims that no data risk exists.
Use a status channel that remains reachable in the scenario you are planning for. If the main application is down, a status page hosted through exactly the same dependency may be unavailable too. The right arrangement depends on the business, but test it. Also decide how support staff will receive consistent information when their ordinary collaboration tools are affected.
Explain temporary behavior clearly. If the service is read-only, tell customers which actions are unavailable. If a submitted transaction has an uncertain outcome, provide a safe way to check its status instead of encouraging repeated attempts. These details reduce confusion and can limit extra work arriving while the team is restoring service. Communication is part of the operational response, not decoration added after engineering has finished.
After recovery, publish a proportionate account of what happened and what changed when evidence supports it. Distinguish the provider’s findings from your application’s behavior. A cloud outage may explain the initial disruption, while your own review reveals that the support contact list or restore process needs improvement. Owning those practical lessons builds more confidence than presenting a supplier’s failure as the end of the investigation.
Know which review answers which question
Resilience work and penetration testing overlap in their attention to assumptions, but they answer different questions. A recovery exercise examines whether the business can restore an agreed service under a failure scenario. A web or API penetration test examines defined security boundaries and application behavior. Neither should be sold as a universal guarantee that a cloud provider will remain available.
A free external review of one approved domain can document visible security exposure and identify sensible next questions. It cannot validate generator behavior, internal cloud failover or the completeness of your backups. If the immediate concern is availability, start by defining the critical customer journey and the recovery evidence you need. If the concern is unauthorized access, agree the relevant application and permissions to test.
The London incident is useful because it turns an abstract resilience discussion into a concrete question: when a dependency fails, what has to work next? Your answer should include the data, the application, the people and the customer. Test that sequence, record the result and improve the weakest step. That is how a collection of backup components becomes a recovery capability the business can understand.
Sources & editorial notes
This analysis uses public reporting and the sources below. Confirmed facts are distinguished from our recommendations and hypothetical examples. We did not investigate these organisations’ systems. Code examples are educational and should be adapted and tested in an authorised environment.
- Google Cloud: europe-west2 incident report
Primary post-incident report updated 4 November 2026; incident dates are also shown in US/Pacific time.
- Google Cloud: disaster recovery planning guide
General guidance on recovery objectives and planning; separate from the incident report.
Incident accounts can change as investigations develop. This article reflects sources checked on 19 November 2026. Illustrations are conceptual; no affiliation or endorsement is implied.



