For close to 30 minutes, users of CloudFlare experienced a massive outage that led to their customers staring at 502 errors (“Bad Gateway”) on their screens. The massive outage was followed by speculations from left right and center with many saying the disruption was because of an attack on CloudFlare.
However, according CloudFlare, the massive outage was because of a misconfigured rule. The latter was being deployed on the CloudFlare network on its web application firewall, but things didn’t go as planned. The configuration contains new rules, which are updated continuously on the net. With the deployment underway, the misconfiguration caused an unfortunate spike on the CPUs; hence the worldwide outage accompanied with 502 errors. At its worse, the CPU was spiked to 100% leading to many sites getting affected.
Effects of the Outage
Due to the outage, many crypto related sites were down while others reflected different Bitcoin price causing massive panic. For the latter, Coindesk was showing Bitcoin was exchanging hands at 26 US dollars. On the other hand, CoinMarketCap had its servers down, and Poloniex had to suspend trading on its platform. Similarly, reports from Itbit came out indicting only its front-end web servers were affected but no back end functions crashed.
With the exchanges coming out to provide information to calm their customers down, CloudFlare also did the same. Matthew Prince CloudFlare CEO also came out on Twitter to provide information about the outage. He also assured customers that they are working to find the root cause of the issue. Technicians sprang into action to save the situation and rolled back the misconfiguration. To make the network function as usual, technicians initiated a global overkill on the WAF managed rule sets at 14:02 UTC. The latter action rolled back the effects of the CPU spike, putting an end to the outage.
Although this time around CloudFlare came out and clarified it was not an attack, many thought it was an attack. It was only recently that the CloudFlare network had another outage. However, according to the information from CloudFlare, the two are not related in any way. For the June 24 outage, Prince came out saying Verizon’s network issues caused the outage. Although the network is ok, many still have questions.
