Solana Outage Post Mortem (Patreon)

In the early hours of Feb 6, 2024, Solana experienced a major outage that lasted approximately three hours. The root cause of the outage was a bug in the durable nonce transactions feature, which allowed a failed transaction to be processed twice under specific circumstances. This led to a stalled consensus among validators, causing the network to stop producing blocks.

During the outage, the Solana team, along with other core contributors and over a thousand dedicated validator and RPC operators, worked together to identify the issue and restart the network. The durable nonce transactions feature was temporarily disabled to prevent further issues.

The Solana community responded quickly to the incident, with various teams collaborating to restore the network and ensure that client services were back up and running as soon as possible. The incident highlighted the importance of the Solana community's dedication to maintaining the network's stability and performance.

PER OX MERT on X 

lmao  as a test, I followed ALL news coverage on over 8 blockchain outages this past year (many L2s, and even multiple ETH finality failures)  (I even was the first to report multiple outages, ok)  none of them had even an ounce of media coverage multiple hours in  and even the post-mortems were "temporary block halt" or "block explorer buggy"  Solana is already frontpage EVERYWHERE  this industry has a weird hard-on against things pushing the frontier  enjoy the clicks — we're not going anywhere  every battle scar makes the system that much more resilient in the long run  if crypto is truly "early" in its adoption, much better to get this out of the way now before we take off  if Solana ever stops pushing the frontier and boundaries of speed, it will no longer be the place for me