The post The Bigger the AI, the More It’ll Lie appeared on BitcoinEthereumNews.com.

Researchers have found evidence that artificial intelligence models would rather lie than admit the shame of not knowing something. This behavior seems to be more apparent the more they grow in size and complexity. A new study published in Nature found that the bigger LLMs get, the less reliable they become for specific tasks. It’s not exactly lying in the same way we perceive the word, but they tend to reply with confidence even if the answer is not factually correct, because they are trained to believe it is. This phenomenon, which researchers dubbed “ultra-crepidarian”—a 19th century word that basically means expressing an opinion about something you know nothing about—describes LLMs venturing far beyond their knowledge base to provide responses. “[LLMs are] failing proportionally more when they do not know, yet still answering,” the study noted. In other words, the models are unaware of their own ignorance. The study, which examined the performance of several LLM families, including OpenAI’s GPT series, Meta’s LLaMA models, and the BLOOM suite from BigScience, highlights a disconnect between increasing model capabilities and reliable real-world performance. While larger LLMs generally demonstrate improved performance on complex tasks, this improvement doesn’t necessarily translate to consistent accuracy, especially on simpler tasks. This “difficulty discordance”—the phenomenon of LLMs failing on tasks that humans perceive as easy—undermines the idea of a reliable operating area for these models. Even with increasingly sophisticated training methods, including scaling up model size and data volume and shaping up models with human feedback, researchers have yet to find a guaranteed way to eliminate this discordance. The study’s findings fly in the face of conventional wisdom about AI development. Traditionally, it was thought that increasing a model’s size, data volume, and computational power would lead to more accurate and trustworthy outputs. However, the research suggests that…

---

Filed under: News - @ September 27, 2024 10:16 pm