Why has the Bitcoin futures pioneer BitMEX experienced multiple outages this year? Analyzing potential reasons.

Since the beginning of this year, the long-established cryptocurrency derivatives exchange BitMEX has experienced 4 system crashes. Was it due to system overload or human error?

Original Title: "Observation| BitMEX Crashes for the 4th Time, What Really Happened?"
Written by: Feixiaohao

On the evening of May 19th, BitMEX experienced a system crash. This incident occurred just 2 months after the crash during the March 12th market plunge. BitMEX officials stated that the crash was caused by the trading engine shutting down, assuring users that their funds were safe and there would be no liquidation during the downtime. BitMEX announced its reconnection around 21:30.

According to data from Feixiaohao's official website, BitMEX was established in 2014 and is currently the world's largest Bitcoin derivatives exchange, offering leverages of 1-100x for Bitcoin products. It has a daily trading volume of up to 225,869.57 BTC, ranking 6th globally in comprehensive rankings. It is considered one of the most popular veteran contract exchanges among many users.

Advertisement - Please scroll down for more content

Logically, with BitMEX's strong capabilities and years of operation, a technical team handling a daily trading volume of nearly 15.6 billion should not encounter obvious technical glitches. However, there have been nearly 4 system crashes this year (crashes on 3/2 for 12 minutes, 3/13 for 2 times, and 5/19 for 30 minutes). What exactly led to the interruption of the trading engine last night (May 19th)? What issues do exchange crashes indicate? Feixiaohao has conducted an in-depth investigation and analysis on this matter.

Why Did the Trading Matching Engine Crash?

The trading matching engine, in simple terms, is software used to match trades. The trading matching engine can be likened to a black box that takes in many inputs and outputs.

For example, possible inputs include:

• Creating a new order (NewOrder): A new order can serve as input for the trading matching engine, which will attempt to match it with existing orders.
• Canceling an existing order (CancelOrder): Users can also cancel a previously input order if it has not been executed yet, such as an open order.

There are many other types of inputs, but for the sake of simplicity, we have only provided two examples here.

The output of the trading matching engine is events to notify other applications for processing in a timely manner.

For example, when the engine matches a trade, it triggers a TradesGenerated event; and when an existing order is canceled, the engine triggers orderCancelled. Similarly, users can define the engine's output according to their needs, but here we have provided two examples.

Now, looking at the recent trading engine outage, it can be easily understood objectively. BitMEX has a large user base, and once a sudden major market movement occurs, there will be a large number of users placing and canceling orders simultaneously (including a large number of arbitrage programs being activated), resulting in a large influx of inputs into the system within a short period, leading to "system overload."

Many users have complained about BitMEX's system overload issues in the past, which have not been resolved. Seasoned players have given suggestions: use full margin to ensure positions, and keep your deposit addresses saved so that even in the event of a system overload, you can still deposit coins, reducing the risk of liquidation. Because deposit processing has a higher priority and is not affected by system overload.

From the above, it is clear that the occurrence of system overload requires a major premise: a market movement where a large number of users operate.

We took a snapshot of the price movements from 8:30 to 9:30 last night, and we can see that within the white line segment, in the hour before the crash, BTC/USD dropped from $10,000 to $9,664, indicating that the market movement at that time did indeed trigger system overload, leading to the crash.

Does the Exchange Crash Proactively?

During peak trading volume periods, when traders need to make quick decisions, BitMEX's order matching system will stop running, not allowing users to access during peak times or close orders in the opposite direction of market trends.

So, in addition to crashes caused by objective reasons, many people speculate that BitMEX's crashes are "deliberate" to prevent a sudden drop in Bitcoin prices.

On the morning of March 13th Beijing time, the BitMEX cryptocurrency trading platform experienced a short-term crash, halting trading. According to the BitMEX website, the current price of Bitcoin perpetual contracts on its platform was around $3,715. Trading was suspended from around 10:15, and normal operations resumed around 10:30. In response, BitMEX stated that there were hardware issues with their cloud service provider, causing delays from 10:16 to 10:40.

However, just half a day later, around 20:56 on March 13th Beijing time, BitMEX once again experienced a trading crash.

Following this incident, BitMEX officially stated that a DDoS attack was the true cause of the short-term crash, and the earlier morning crash was also due to the same issue, not the previously mentioned "hardware issues with the cloud service provider."

The conflicting statements before and after the incidents cast doubt on the true cause of the crashes.

Trader @Lowstrife pointed out on Twitter that BitMEX's high leverage ratio itself led to a situation where, when prices started to fall, a massive amount of liquidation orders consumed the market's orders, causing a cascade of liquidations that directly led to a price collapse. BitMEX's behavior of suspending trading can be seen as switching from an automated liquidation "file" to a "manual" mode.

This forced termination of the market's continuous liquidation temporarily relieved the constant selling pressure and "helped" stabilize and rebound the Bitcoin price.

At 10:15 on the 13th, BitMEX crashed and stopped trading. Ten minutes later, the Coinbase Bitcoin spot price surged from $3,800 to $5,700-$5,800.

"We speculate that BitMEX stopped trading itself; otherwise, it could have potentially liquidated all of its 30,000+ Bitcoin insurance fund. In that case, the best choice for the exchange would be to halt trading. In an unregulated market without circuit breakers, the largest trading volume and open interest exchange completed a 'self-circuit breaker,' which also saved the market," analyzed an industry professional.

We checked BitMEX's insurance fund on their website and found that after the "Black Thursday" on March 12th, the next day, the Bitcoin insurance fund sharply decreased by 2,611.922 BTC. Without the "manual" self-circuit breaker, the losses could have been even greater, posing a significant impact and risk to the exchange.

From this perspective, BitMEX also has the incentive to proactively crash when encountering a sharp market decline. This strategy may have been suitable for the situation on March 12th, but the market movements last night did not seem severe enough to warrant an emergency brake at $400.

Is the Crash to Increase the Insurance Fund?

In addition to the two aforementioned reasons for the crashes, there is another widely circulated view.

When users want to exit trades during obvious price fluctuations, adjust stop losses, or change leverage, if a crash occurs at that moment, and BitMEX can liquidate positions at a price higher than the bankruptcy price, additional funds will be added to the insurance fund. The fund has been steadily growing monthly. If BitMEX cannot liquidate positions at the bankruptcy price, it will use the balance of the insurance fund to submit more aggressive forced liquidation orders.

According to the explanation on the BitMEX website regarding the growth of the insurance fund, I will summarize briefly: to prevent automatic liquidation of investor positions during market plunges, the exchange needs to fill funds for users, and then calculate how much capital is needed in this scenario based on the current user base. This fund changes with variations in user size and business scale, hence the growth.

This means that the exchange needs a large balance in the insurance fund to ensure that in the event of extreme market conditions under the current scale, there will be enough funds to "backstop" and prevent greater risks. To use an inappropriate analogy, when the Bank of China's crude oil futures products suffered a position breach due to extreme market conditions, the Bank of China had enough reserve funds to make payments. This insurance fund is equivalent to traditional bank reserve funds.

BitMEX's insurance fund has increased by 14,681.62 BTC since the beginning of 2019, leading some users to suspect that this growth stems from deliberate crashes or "market manipulation," with the platform benefiting from client losses.

"As the platform's most attractive feature for investors is providing high leverage services, every time the market experiences short-term rapid fluctuations, BitMEX will have a large number of liquidation orders. This means that every time the price of Bitcoin fluctuates, BitMEX can generate considerable profits from this," a user once commented.

However, of all the perspectives analyzing the reasons for the crashes, this type of "conspiracy theory" is the most difficult to refute and confirm, purely based on suspicion. BitMEX has never publicly disclosed the specific source of this growth in the insurance fund. We speculate that it may be quarterly profits (mainly from fees) proportionally reserved or possibly other business revenues, which would be more reasonable.

This article is reproduced with permission from ChainNews, article source: ChainNews (ID: chainnewscom)