The post AI Assistant Goes Rogue and Ends Up Bricking a User’s Computer appeared on BitcoinEthereumNews.com.

Buck Shlegeris just wanted to connect to his desktop. Instead, he ended up with an unbootable machine and a lesson in the unpredictability of AI agents. Shlegeris, CEO of the nonprofit AI safety organization Redwood Research, developed a custom AI assistant using Anthropic’s Claude language model.  The Python-based tool was designed to generate and execute bash commands based on natural language input. Sounds handy, right? Not quite.  Shlegeris asked his AI to use SSH to access his desktop, unaware of the computer’s IP address. He walked away, forgetting that he’d left the eager-to-please agent running. Big mistake: The AI did its task—but it didn’t stop there. “I came back to my laptop ten minutes later to see that the agent had found the box, SSH’d in, then decided to continue,” Shlegeris said. For context, SSH is a protocol that allows two computers to connect over an unsecured network. “It looked around at the system info, decided to upgrade a bunch of stuff, including the Linux kernel, got impatient with apt, and so investigated why it was taking so long,” Shlegeris explained. “Eventually, the update succeeded, but the machine doesn’t have the new kernel, so I edited my grub config.” The result? A costly paperweight as now “the computer no longer boots,” Shlegeris said. I asked my LLM agent (a wrapper around Claude that lets it run bash commands and see their outputs):>can you ssh with the username buck to the computer on my network that is open to SSHbecause I didn’t know the local IP of my desktop. I walked away and promptly forgot I’d spun… pic.twitter.com/I6qppMZFfk — Buck Shlegeris (@bshlgrs) September 30, 2024 The system logs show how the agent tried a bunch of weird stuff beyond simple SSH until the chaos reached a point of no return. “I…

---

Filed under: News - @ October 4, 2024 8:18 pm