In an astonishing revelation, a new study has claimed that the capabilities of ChatGPT, the artificial intelligence-driven chatbot by OpenAI, seem to be deteriorating with time. Researchers from two prestigious institutions, Stanford and UC Berkeley, found that the chatbot’s newest models are providing significantly less accurate solutions to identical questions over the span of a few months. Ironically, it seems as though the technology on the forefront of advancement is experiencing a regression.
ChatGPT, known for answering math problems, generating code lines, conducting spatial reasoning, and responsibly handling sensitive questions, was tested by the researchers. They used the ChatGPT-3.5 and ChatGPT-4 models to conduct these examinations. The continuous downhill slide in performance level certainly raises a few perplexing questions. For instance, back in March, the ChatGPT-4 could pinpoint prime numbers with a staggering 97.6% accuracy rate. However, when the same test was conducted in June, its accuracy had incredibly fallen to a mere 2.4%.
Simultaneously, its abilities to generate new code lines have also deteriorated. Comparatively, the older GPT-3.5 model surprisingly had improved in spotting prime numbers in the same time frame. The researchers noted that the responses to sensitive questions have also seen a significant change. Initially, ChatGPT provided detailed reasons for refusing sensitive queries, but later, it simply apologized and refused to answer.
The entire scenario is quite disconcerting considering the reliance of numerous individuals and organizations on such AI-powered chatbots. Due to the possible significant changes in these large language model services within a brief period, continuous monitoring and quality check has indeed become imperative. As such, the study recommends users to implement some monitoring analysis to ensure the optimal functioning of the chatbot.
OpenAI, in response to these revelations, announced plans to form a team for managing the potential risks arising out of superintelligent AI systems. This step highlights the substantial influence and expectation the world has from AI technology in the forthcoming decade. Although it is too soon to speculate or hastily label these initial deterioration instances as a possible threat to the future of AI, we cannot neglect the difficulty presented in maintaining the stability and growth of these complex systems. Thus, emphasizing again the necessity for regular tracking and monitoring, to upkeep the tech gem that AI truly is.
Source: Cointelegraph
