The CrowdStrike fail and next global IT meltdown already in the making

When computer screens went blue worldwide on Friday, flights were grounded, hotel check-ins became impossible, and freight deliveries were brought to a stand-still. Businesses resorted to paper and pen. And initial suspicions landed on some sort of cyberterrorist attack. The reality, however, was much more mundane: a botched software update from the cybersecurity company CrowdStrike.

"In this case, it was a content update," said Nick Hyatt, director of threat intelligence at security firm Blackpoint Cyber.

And because CrowdStrike has such a broad base of customers, it was the content update felt around the world.

"One mistake has had catastrophic results. This is a great example of how closely tied to IT our modern society is — from coffee shops to hospitals to airports, a mistake like this has massive ramifications,"  Hyatt said.

In this case, the content update was tied to the CrowdStrike Falcon monitoring software. Falcon, Hyatt says, has deep connections to monitor for malware and other malicious behavior on endpoints, in this case, laptops, desktops, and servers. Falcon updates itself automatically to account for new threats.

"Buggy code was rolled out via the auto-update feature, and, well, here we are,"  Hyatt said. Auto-update capability is standard in many software applications, and isn't unique to CrowdStrike. "It's just that due to what CrowdStrike does, the fallout here is catastrophic," Hyatt added.

Even though CrowdStrike quickly identified the problem, and many systems were back up and running within hours, the global cascade of damage isn't easily reversed for organizations with complex systems.

"We think three to five days before things are resolved," said Eric O'Neill, a former FBI counterterrorism and