The consequences of code

BitDepth#1470

ON JULY 19, cybersecurity firm Crowdstrike sent an automatic update to Microsoft Windows computers that was intended to upgrade the Falcon sensor security solution it sells to enterprise.

The worst possible thing happened. A bug in the code sent the computers that received into a death spiral of blue screens. The update was just 40 kilobytes in size and was intended to adjust the sensor's ability to detect malware.

Instead, it caused more than US$6 billion in real world damage.

Delta Airlines alone, which deployed the software widely in its computer network, reported losses of more than US$500 million over the week it struggled to normalise operations after the Crowdstrike bug crippled the company's ability to function.

Microsoft estimates that more than eight million Windows computers were affected by the bug.

Crowdstrike quickly deployed a patch that corrected the issue, but for many customers, it fixed nothing.

Falcon is an endpoint sensor widely used in computers that run systems like automated kiosks and customer interface panels that were also secured by Microsoft's BitLocker encryption software.

On those computers, it was necessary to decrypt the hardware, apply the patch then restart. Roughly 20 minutes work, multiplied by hundreds of devices.

Delta's long path to restoring operations was apparently compounded by outsourced IT, which meant fewer people available to "touch" stricken computers.

TT was largely unscathed by the incident (https://cstu.io/36e5d9), and most organisations affected by the bug reported resumption of transactions within 24 hours.

"Do I think that TT dodged a bullet because Crowdstrike is expensive? Yes," said cybersecurity specialist Shiva Parasram.

"The fact that Crowdstrike is very popular but very expensive might be one of the factors limiting its impact in Trinidad.

"But it's not necessarily a good thing. The reason why there was minimal impact is because we don't really spend much on cybersecurity."

The cruel reality of Crowdstrike is that it wasn't a cybersecurity attack. It was a quality of service lapse and the incident puts IT professionals in an odd space, sandwiched between determined and sustained attacks by hackers and ransomware organisations and hastily deployed software that ends up fragging their systems from the inside.

Do IT pros do all recommended updates as they are issued and risk buggy updates like Crowdstrike?

Do they wait a few days and risk compromise because of outdated security measures or unplugged security holes?

Do they create a sandboxed update system to confirm that updates are safe? If so, how practical would that be for typically underpaid, overworked local IT teams?

Parasram believes that sandboxed test systems to confirm updates are something that companies will have to build into their IT management.

"It's not going to get any easier for TT," he said.

"But we have a lot more graduates coming out, new professionals who are looking for a start. Companies will have to get serious about disaster recovery and that includes cloud service providers and software as a service.

"Companies have to do third-party risk assessments on these businesses, ensure that they are certified, that they have qualified teams, that they are on the ground. What is their response time (when disaster strikes)?

"People don't take on service-level agreements, but you have to look at how much downtime and uptime are guaranteed and if it's not provided, you are due compensation. Service-level agreements and contracts have to be studied quite carefully to ensure that these critical services are supplied."

As the immediacy of Crowdstrike disruptions gave way to analysis of the incident, talk of legal liability began to surface.

What should TT take away from the Crowdstrike bug?

Top of the list is that businesses and government agencies are responsible for the sanctity of their computer systems and every business decision should be predicated on maximising cybersecurity.

Contingency planning must be thorough, exhaustive and well-exercised.

When systems fail, customers and the public don't actually care and often don't understand distributed responsibilities, so blaming other companies and services is always going to fall flat.

While TT customers have a high tolerance for service abuse, they should not be expected to offer eternal grace for digital failures.

TSTT weathered the humiliation of having private customer information exposed on the dark web and later the open internet by offering its CEO and CFO as public sacrifice.

iGovTT managed to dodge public opprobrium after its proud achievement, TTConnect, simply disappeared for months.

With no legal requirement to notify anyone of cybersecurity breaches, other exposures of personally identifiable information remain largely unknown.

What we don't know can, in fact, hurt us.

Mark Lyndersay is the editor of technewstt.com. An expanded version of this column can be found there