Technology

59685 readers

3329 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

[email protected]

819

CrowdStrike downtime apparently caused by update that replaced a file with 42kb of zeroes (twiiit.com)

submitted 4 months ago* (last edited 4 months ago) by [email protected] to c/[email protected]

189 comments fedilink hide all child comments

…according to a Twitter post by the Chief Informational Security Officer of Grand Canyon Education.

So, does anyone else find it odd that the file that caused everything CrowdStrike to freak out, C-00000291-
00000000-00000032.sys was 42KB of blank/null values, while the replacement file C-00000291-00000000-
00000.033.sys was 35KB and looked like a normal, if not obfuscated sys/.conf file?

Also, apparently CrowdStrike had at least 5 hours to work on the problem between the time it was discovered and the time it was fixed.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 1 points 4 months ago (1 children)

You know there's a whole other scenario where the system can simply boot the last known good config.

[–] [email protected] 1 points 4 months ago (1 children)

And what guarantees that that "last known good config" is available, not compromised and there's no malicious actor trying to force the system to use a config that has a vulnerability?

[–] [email protected] 1 points 4 months ago* (last edited 4 months ago) (1 children)

The following:

An internal backup of previous configs
Encrypted copies
Massive warnings in the system that current loaded config has failed integrity check

There's a load of other checks that could be employed. This is literally no different than securing the OS itself.

This is essentially a solved problem, but even then it's impossible to make any system 100% secure. As the person you replied to said: "this is poor code"

Edit: just to add, failure for the system to boot should NEVER be the desired outcome. Especially when the party implementing that is a 3rd party service. The people who setup these servers are expecting them to operate for things to work. Nothing is gained from a non-booting critical system and literally EVERYTHING to lose. If it's critical then it must be operational.

[–] [email protected] 1 points 4 months ago

The 3rd party service is AV. You do not want to boot a potentially compromised or insecure system that is unable to start its AV properly, and have it potentially access other critical systems. That's a recipe for a perhaps more local but also more painful disaster. It makes sense that a critical enterprise system does not boot if something is off. No AV means the system is a security risk and should not boot and connect to other critical/sensitive systems, period.

These sorts of errors should be alleviated through backup systems and prevented by not auto-updating these sorts of systems.

Sure, for a personal PC I would not necessarily want a BSOD, I'd prefer if it just booted and alerted the user. But for enterprise servers? Best not.