01 THE HOUR
­
Morning
Wednesday, 20 June 2012 . Edinburgh / London
Customers of RBS, NatWest and Ulster Bank logged into online banking on Wednesday morning and found account balances that had not moved. Wages due overnight had not arrived. Direct debits due to leave had not left. ATM screens showed figures from the previous day. The batch processing run that updates every account, every night, had not completed.
The money was not missing. It was in transit, somewhere inside a queue that had stopped moving.
02 THE SEQUENCE
­
17 Jun On Sunday, the RBS Group's centralised Technology Services function applied an upgrade to the batch scheduler software that sequences overnight account updates. The batch scheduler coordinated the order in which deposits, withdrawals, interbank clearing and payroll data posted to customer accounts. On an average day, the system processed roughly 20 million transactions across RBS, NatWest and Ulster Bank.
17-19 Jun Technology Services noticed problems with the upgrade and decided to uninstall it. The back-out was performed without testing whether the new version was compatible with the old. It was not. The Financial Conduct Authority later found that the upgraded software could not be cleanly rolled back to the previous version. On Tuesday night, batch processing failed.
20 Jun Wednesday. Customers found that online banking did not show correct balances. ATMs were generally available but presented out-of-date balances. Wages, benefit payments and standing orders that should have posted overnight were absent from accounts. The RBS Group confirmed technical issues and kept 1,000 branches open until 7pm on Thursday and Friday.
23 Jun Saturday. RBS chief executive Stephen Hester apologized publicly. The bank announced that 1,200 branches would open on Sunday, described as the first time that had been done. Call centre staffing was doubled. The bank stated that no customer would be left permanently out of pocket.
25 Jun Monday, day six. Technology Services had stabilised batch processing for RBS and NatWest. The backlog of unprocessed transactions was being cleared, but not all accounts had caught up. The UK Financial Services Authority urged other banks to be lenient with their own customers who had missed payments because transfers from RBS Group accounts had not arrived.
26 Jun-
10 Jul 		RBS and NatWest systems were largely restored by the end of June. Ulster Bank, whose transactions ran in both sterling and euro through the same Edinburgh batch, took longer. Its ATMs were not fully functional for 19 days. Its online banking was disrupted for 20 days. The FCA later recorded that disruptions to the majority of Ulster Bank's systems continued until 10 July. A House of Commons committee found that some Ulster Bank issues stretched into August.

Elon Musk: This Could be 180 Times Bigger than SpaceX IPO

Elon Musk is on record predicting this new opportunity is set to be 180 times bigger than the SpaceX IPO.

That IPO minted 4,400 millionaires overnight.

If this is 180 times bigger…

Just imagine how many more Americans will become millionaires.

Will you be one of them? Click here now.

03 THE HIDDEN DEPENDENCY
­
The batch scheduler. It was a single piece of software that determined the order in which overnight jobs ran: deposits, withdrawals, interbank clearing, payroll, standing orders. Every customer-facing channel, from ATMs to online banking to point-of-sale terminals, displayed the output of the last completed batch. None of those channels calculated a balance on its own.
When the scheduler failed, the team could not access the record of which transactions had been processed up to the point of failure. Establishing where the batch had stopped had to happen before the next batch could run. Each day of delay added another day of unprocessed transactions to the queue. Within days, the backlog reached 100 million items.
None of those channels calculated a balance on its own.
04 WHAT STILL WORKED
­
RBS BRANCH TELLER SERVICES
The FCA's final notice recorded that teller services at RBS branches were unaffected throughout the incident. Branches could handle transactions that the digital channels could not. The bank opened 1,200 branches on Sunday, with extended weekday hours, to substitute for the frozen online and ATM systems. Customers who could show they should have received payments were offered emergency cash.
­
LENIENT TREATMENT AT OTHER BANKS
The FSA urged banks outside the RBS Group to treat their own customers leniently where payments from RBS accounts had failed to arrive. The failure had reached beyond the institution that caused it: non-customers could not receive monies owed to them and could not honour their own commitments as a result.
05 THE DOCUMENT
THE DOCUMENT
August 2010
The FCA's final notice, issued in November 2014, recorded that the period of breach began on 1 August 2010: the date of a Group Internal Audit on mainframe batch processes. That audit identified the risk of a batch scheduler failure. The batch scheduler failed twenty-two months later. The FCA's enforcement boundary starts at the audit rather than at the incident, marking the point at which the institution possessed the knowledge and did not act on it.
06 THE PATTERN LIBRARY
1 When one retail withdrawal channel closes or fails, demand concentrates on whichever channel is left open, and the capacity of that channel sets what a depositor can obtain.
Observed: Northern Rock 2007, Cyprus 2013, India 2016, Greece 2015, United States 1933, Argentina 2001, Lebanon 2019, Worldpay 2026, TSB 2018, Afghanistan 2021, Nigeria 2023, Square 2023, United States 2025, Fiserv 2025, United States 2021, RBS Group 2012.
4 Domestic payment channels can continue while cross-border wire transfer routes fail. Where credit card settlement runs through a separate institution on an independent network, it may persist after the wire transfer channel closes. In some cases the central bank intermediates selected transfers through its own remaining correspondent accounts.
Observed: Iceland 2008, Germany 1931, Belize 2015.
8 A cash withdrawal ceiling is paid only in the denominations the machine is permitted to hold, so a ceiling and a denomination rule set together decide how many notes a withdrawal is, and a ceiling has to be raised alongside the denominations that pay it when the existing notes no longer carry it.
Observed: Zimbabwe 2008, Nigeria 2023, Venezuela 2018.
9 A payment outage can hit card authorisation and merchant funding as separate faults, so acceptance at the terminal is restored while the money owed to the merchant arrives a day late under its own incident notice.
Observed: Worldpay 2026, United States 2021, Marqeta 2026, Square 2023.
34 A failure in the batch processing layer that posts transactions to accounts overnight left balances readable but stale across ATMs, online banking and point-of-sale systems, because those channels displayed the last completed batch output rather than generating their own figures.
Observed: RBS Group 2012.
CHANGED TODAY: Entry 1 gained RBS Group 2012. New entry 34 added, on the batch processing layer leaving balances readable but stale when the overnight posting sequence failed.
Hour Zero is a production of Conditions. Reconstructions from the public record.