An LLM can’t “go rogue”. They’re all just toys that idiots are using for critical infrastructure functions, then they bitch when they burn themselves on the fire they’ve created in their lap.
The AI agent was set to complete a routine task in the PocketOS staging environment. However, it came up against a barrier “and decided — entirely on its own initiative — to ‘fix’ the problem by deleting a Railway volume,” writes Crane, as he starts to describe the difficult-to-believe series of unfortunate events.
Quite easy-to-believe, really.
These multiple safeguards toppling in rapid succession
Multiple safeguards? Really? Multiple paragraph prompts are not multiple safeguards… it’s half a safeguard at best. Applying limits on what the AI can do is a safeguard.
That’s great to hear.
This is absolutely hilarious. “AI” users getting what they deserve chef’s kiss
This is what happens when there is a new technology and companies are run by commerce grads, not scientist or engineers that understand the technology.
AI has good therapeutic uses, particularly for disabled or impoverished people who may not be able to access mainstream therapy
Please don’t recommend AI for therapeutic uses, it’s only been optimised to keep the user engaged and pushed many people into psychosis. Just search for “ai psychosis” on your favourite search engine and you’ll get a ton of reports on how LLMs validate vulnerable people’s delusions, sometimes pushing them all the way into murder and/or suicide.
I hope you are not seriously advocating using the lying machine for therapy. You would get more value talking to a finger puppet.
No. Chatbots are machines built by billionaires with the agenda of making money. They litterally design these bots (even the therapeutic ones) to be sycophantic to the point they tell people anything to keep them chatting longer. To the point some of their users lose touch with reality. How many cases do we need of a chatbots helping a teenager plan and succeed at a suicide? Altruists did not design these machines. Even with a human therapist we have to watch for the landmines of their personal agendas. That’s a thousand times worse for machines that have no humanity, are capable of LIES, and have secret unwritten priorites written into their code by rich sociopathic creators. If facebook taught us anything it should be that if something is free on the internet it’s not because we are the customers.
Also DO NOT TELL ALL YOUR DEEPEST DARKEST SECRETS TO CHATBOTS! They aren’t required by any legal bodies to protect that information! OMFG
There’s stupid from top to bottom here.
The company is stupid for allowing an AI full root access to their entire setup.
The provider is stupid for only generating full-access API keys. They’re even stupider for storing backups with a volume, so deleting the volume (zero confirmation via API key) also insta-deletes the backups. And they’re stupidest for encouraging users to plug AIs into this full-trust mess.
And the company is absolute stupidest for having no backups other than the provider’s builtin versioning.
Reminder that Anthropic’s AI system was used in targeting the school in Minab, killing 120 students. https://www.washingtonpost.com/national-security/2026/03/11/us-strike-iran-elementary-school-ai-target-list/
The company is suing to be able to supply the US military again. It is in bed with the fascists.
Can you get an AI to code? Yes. Can you get it to stop you from running your operation in such a stupid way that it will end up destroying it? No.
Always keep offline backup copies of your important data regardless of using AI slop to look over it! No, I don’t care that “optical media is obsolete and e-waste!”, or that “tapes are a 100 year old obsolete technology compared to cheap SSDs from TEMU!”.
Optical media? Is that a viable part of backup strategies? I would expect tapes for sure, sounds like you know more than me.
- Better than not having an offline copy.
- Write-only, ransomware cannot delete/encrypt it.
- Drives are still cheap.
Downside is having techbros talk you about laser rot, how internal drives are obstructing the optimal airflow in GAMING PC cases, and how Gabe Newell is based and stuff.
Great points! Lotta my optical media use also included hot summers in cars lol, nothing like an archival use.
A quality disc can last 10 years or more. At a company I used to work at the backups were burned to discs coated with gold. They had 15 year old discs that still worked.
Dang that’s rad, had no idea (about it being used in such a way, I guess I mean, not too hard to imagine discs lasting that long).
I have 20+ yr old optical media cdr/dvdr and they are still good, the cheap ones like Pine and the ones with no name at all
What is this 10 year thing? I’ve also got CD RWs and CD Rs from 1998 that still work. And DVD Rs from like 2002 that are still fine.
That was my point, hehe. I also never spent on the “quality name brands” of disks, $10 for 100 cds, deal! $15 for 100 dvds insert fry meme. Maybe we just “took care” of our media better than others did? Personally, they are in spindles on a bookshelf, I just made sure no direct sunlight would hit them where they are, some days get warm before I can turn on the ac.
I definitely agree with you. I feel like I see people talking about optical media rotting all the time and it just doesn’t seem like a practical issue for 99% of use cases.
I seem to remember the conversation in the early 2000s being about how discs would rot in 50+ years and now I see people saying ten or 15.
they did not follow the 3-2-1 rule…
To me it seems more criminal that the cloud provider has a “nuclear button” feature via the API that destroys everything including the backups with a single call and no confirmation whatsoever. What if the key gets accidentally leaked and someone wants to have fun?
It’s a feature.
It seems like actually criminal too. Like legitimately “we need to shred 2TB of incriminating data instantly or we’re all going to prison”
This was on Hacker News: https://news.ycombinator.com/item?id=47911524
Twitter link: https://xcancel.com/lifeof_jer/status/2048103471019434248
Hacker New’s sentiment on this from the comments I’ve read is that it is the author’s own fault.
Good.
“If your prod can be deleted by your AI, it should be.”
Did they write that title with AI also? Look terrible
Not to mention the image
3-2-1
Its really common for companies to not have an offsite backup. My own employer only offsites the customer data, not our core biz stuff. And I setup the offsite replication. It did not exist until I built it. (Proxmox Backup Server is tha best!)Seems like, if nothing else, Ai might finally force corporate accountants to acknowledge that the cost of a good backup strategy far outweighs the cost of losing all your data because some MBA thought he could write a product update himself with Claude code.
Good. Pay me. More. DR engineer!
