01THE HOUR
13:54
Thursday 7 September 2023. Reconstructed to the hour from the company's post-incident report.
At 1:54 PM Eastern, Square products and services became unavailable. Sellers mid-transaction found readers that would not complete a sale. The devices were powered and connected; what they could not reach was the authorization layer behind them.
What had changed was not in the payment code. A host-based firewall rule deployed that morning at 11:04 had increased on-node firewall rule sizes, and the internal DNS caches that had been absorbing the resulting load for nearly three hours expired at once. Every service that depended on DNS to find its counterpart went down together.
02THE SEQUENCE
7 Sep 2023, 11:04 AM ETA host-based firewall rule change is deployed to enable communication between regions. The change increases the size of firewall rules stored on each node. Internal DNS caches continue resolving service requests and no payment surface registers a problem.
7 Sep 2023, 1:54 PM ETSquare products and services become unavailable. The DNS caches have expired under the enlarged firewall rule set, and services can no longer locate one another. Two minutes later, at 1:56 PM, a DNS zone change is applied.
7 Sep 2023, 2:02 PM ETEngineers are notified of infrastructure issues. Incident response begins, focused on DNS investigation.
7 Sep 2023, 2:52 PM ETWork begins to recover internal access and tooling. The internal tools and services used for troubleshooting and recovery resolve through the same DNS layer and are also down. Engineers must rebuild their own access before they can begin restoring payment services.
7 Sep 2023, 3:56 PM ETNetworking traffic is shed to DNS servers. Manual work begins to bring up new DNS servers, because the Square deployment pipeline is unavailable.
7 Sep 2023, 6:29 PM ETInternal access services recover, more than four hours after the outage began. Engineers can now work in parallel on authorization and the control plane.
7 Sep 2023, 8:25 PM ETThe status page confirms Balance and Transfers as separately affected, a distinct item from the payment processing outage acknowledged hours earlier. Sellers whose terminals were down and whose funds were frozen encountered both faults at once.
7 Sep 2023, 8:36 PM ETThe deployment pipeline recovers. Automated code and configuration changes are possible again, after nearly five hours of manual work.
8 Sep 2023, 12:55 AM ETAfter hours of manual rebuilding and a reverted configuration push, all DNS servers report healthy.
8 Sep 2023, 2:40 AM ETPayment traffic fully recovered. The incident is marked resolved on the status page at 5:19 AM.
8 Sep 2023, 9:52 AM to 1:57 PM ETBackground processing of offline payments begins at 9:52 AM. Completed at 1:57 PM for sellers on supported configurations. Transactions queued on devices since the previous afternoon clear through the system over the course of four hours.

The move Washington made in 1934

They did it once. Trump can do it again.

In 1934, the government executed a legal maneuver that transferred billions in wealth overnight.

Most Americans had no idea it was coming.

A small group who saw it early walked away wealthy.

Everyone else paid for it.

Trump has the same legal authority today. Advisors close to the administration believe he's considering using it. If he does, the transfer happens fast — and the window to be on the right side of it is already closing.

We put together a free report on exactly what this move is, why the timing points to now, and the one step ordinary Americans can take to position themselves before it happens.

It costs nothing. Takes 30 seconds to request.

The people who moved early in 1934 didn't have a warning.

You do.

Send me the free report
03THE HIDDEN DEPENDENCY
The firewall rule deployed that morning was an unrelated change to its host-based firewalls; it changed the size of host-based firewall rules stored on each node. Square services used DNS and mesh-based routing infrastructure to find service dependencies and serve requests. When those DNS caches expired under the larger rule set, the services lost the ability to find one another and the stack went down as a unit.
The dependency ran deeper than the payment surface. The failure that stopped payments also stopped the tools for restoring them. Engineers spent hours recovering internal access and tooling before the authorization and control plane services could be addressed.
The failure that stopped payments also stopped the tools for restoring them.
04WHAT STILL WORKED
OFFLINE MODE
Sellers using Square Terminal or Square Register with offline mode enabled continued accepting payments. The company confirmed that many sellers used offline mode during the disruption. The queued payments were processed the following day, completing at 1:57 PM on 8 September.
CASH AND MANUAL RECORDS
Some merchants switched to cash during the outage. Others wrote down card numbers by hand to charge later.
05CONTESTED
CONTESTED
7 September 2023
Square and Cash App are both operated by Block and share a domain name system. Both went down on 7 September 2023, in the same window. Square's post-incident report covers Square services and does not mention Cash App.
Cash App posted its own, separate status updates. Payments Dive reported seven days later that it was not clear how, or if, the two outages were linked.
06THE PATTERN LIBRARY
5 of 31 entries shown.
1When one retail withdrawal channel closes or fails, demand concentrates on whichever channel is left open, and the capacity of that channel sets what a depositor can obtain.
Observed: Northern Rock 2007, Cyprus 2013, India 2016, Greece 2015, United States 1933, Argentina 2001, Lebanon 2019, Worldpay 2026, TSB 2018, Afghanistan 2021, Nigeria 2023, Square 2023, United States 2025, Fiserv 2025, United States 2021.
9A payment outage can hit card authorisation and merchant funding as separate faults, so acceptance at the terminal is restored while the money owed to the merchant arrives a day late under its own incident notice.
Observed: Worldpay 2026, United States 2021, Marqeta 2026, Square 2023.
27A payment network that connects most of its enrolled institutions through third-party resellers can remain operational while a reseller's outage severs the connection for that reseller's clients, so the failure boundary follows the reseller relationship rather than the network itself.
Observed: Fiserv 2025.
28A payment gateway can fail independently of the card network it connects to, so a merchant with a connected terminal loses the ability to take cards when the gateway between the terminal and the network goes down, while merchants routed through other gateways retain the ability to process.
Observed: BridgePay 2026.
31A shared internal routing layer beneath a payment processor can fail and take dependent services down as a unit, including the internal tools used to diagnose and recover, extending the outage while engineers rebuild access to their own systems.
Observed: Square 2023.
CHANGED TODAY: Entry 1 and Entry 9 gained Square 2023. Entry 31 added, on a shared internal routing layer whose failure disabled both the payment services and the recovery tooling.