AI chatbots like OpenAI's ChatGPT promise to revolutionize the workplace, but a new study is sounding a warning: Purdue University researchers found that ChatGPT returned incorrect answers when tackling computer programming questions a whopping 52% of the time.
This worrying statistic comes from a study presented at the Computer-Human Interaction conference in Hawaii, which entered 517 programming questions taken from Stack Overflow, a popular platform for programmers, into ChatGPT.
The results were a double-edged sword: while a significant portion of the answers (52%) contained errors, the study also found that ChatGPT stood out for its comprehensiveness and clear language. This led to a surprising result: 35% of study participants preferred ChatGPT's answers despite the inaccuracies.
Even more worrying is the fact that the programmers in the study missed critical errors in nearly 40% of ChatGPT's answers, highlighting a significant risk: seemingly well-written answers can mislead users into believing incorrect information.
This study highlights the need to improve the accuracy of AI assistants like ChatGPT, especially for critical tasks like programming. Additionally, the study highlights the importance of users being aware that AI-generated responses may contain misinformation.
ChatGPT isn't the only one suffering from hallucinations: Google's AI is also suffering from them
Google recently introduced a new AI Overview feature that, Bloomberg reports, has been churning out errors that have Alphabet CEO Sundar Pichai fed up: When asked why cheese was sliding off a pizza, the feature recommended the user apply glue, while the AI ​​chatbot reportedly suggested to another user that pythons are mammals.
Google says it's improving its AI search service as new reports of hallucinations come in. A Google spokesperson told The Verge that the errors “typically occur for very rare queries and are not representative of most people's experiences.”
