Solana Mainnet Outage Report: Due to a surge of bots for minting, introducing a fee priority mechanism announced

share
Solana Mainnet Outage Report: Due to a surge of bots for minting, introducing a fee priority mechanism announced

The Solana mainnet was temporarily paused for seven hours on 5/1 at 04:30 Taiwan time and resumed normal operation at 11:30. The main reason was the influx of up to 6 million transactions per second, causing verification nodes to run out of memory and crash, leading to a halt in the mainnet consensus mechanism and block production. Solana released an incident report on 5/3 and announced the introduction of transaction fee priority settings, expected to be implemented in version v1.11.

Cause: Influx of Bots

The Solana NFT minting platform Metaplex acknowledged on Twitter that there were too many speculators deploying crawlers against its NFT issuance mechanism, the Candy Machine.

In response, Metaplex plans to implement a penalty mechanism, where wallets attempting to complete invalid transactions will be charged a fine of 0.01 SOL.

According to the Solana incident report, there is no evidence of a denial-of-service attack, but it was confirmed that a large number of NFT minting bots flooded the network, processing transactions at a rate of up to 6 million per second, with single-node traffic exceeding 100 Gbps at the time of the incident.

Solana emphasized that network performance has been optimized, and although the mainnet experienced a nearly 17-hour outage in September last year, the characteristics of this delay were similar, but the volume of transaction requests during this outage reached 10,000% of the September level, reflecting subsequent optimizations in the verification mechanism.

Introduction of Transaction Priority Fee Mechanism

Currently, version 1.10 is running stably on the test network, including memory optimizations to allow nodes to extend the consensus mechanism stalled by a high volume of transactions, and version 1.11 is expected to introduce transaction fee priority settings.

Recognizing the need to use network bandwidth more efficiently, Solana points out the necessity of a priority fee mechanism to end the long-standing practice of Solana accepting transactions indiscriminately without considering the source of the transaction, and further enhancing the quality of transaction services is a natural decision.

Official Solana data shows that this is the seventh performance degradation or network delay event experienced by the mainnet.

Commentary from Li Tingting, Founder of The Z Institute

Li Tingting, founder of The Z Institute, was also consulted for her views on the Solana incident report and the transaction fee priority mechanism.

She stated that if the gas price for each transaction on Solana remains the same in the short term, it will still be susceptible to DDoS attacks because the cost of failed transaction submission is too low.

The optimization methods as per the official response are:

  1. Changing the originally designed protocol to one developed by Google.
  2. Nodes with more stake can prioritize transaction processing.
  3. Increasing fees to expedite transaction processing.

Regarding the blame on Solana being a centralized blockchain, Li Tingting believes it is not directly related to this issue. The key lies in the concept of "user pays," where under the previous unified transaction fee setting of Solana, bots often engaged in random transactions.

Li Tingting explained that since both chains have Tips transaction fees, why does Solana perform better? It is because Solana's underlying design only affects specific state statuses, unlike Ethereum, which affects the entire blockchain state single thread. This underlying protocol design of Ethereum is insurmountable unless more Layer2 L2 is used, but the approach of Layer2 differs from Solana's.