The new technology which will drive ChatGPT in the future managed to score better than nine in ten humans when taking the bar exam to become a lawyer, the company behind it has said.
OpenAI’s new technology GPT-4 takes over from the earlier model GPT-3.5, which had a much worse score in the bar exam.
The previous model, which has taken the world by storm since ChatGPT was released late last year, scored in the bottom 10% of test-takers, the business said.
The new model, GPT-4, is able to handle “much more nuanced instructions” than its predecessor, its creators said.
A demonstration by the company behind the technology showed them drawing a simple website design on a sheet of paper and asking the system to write code to make that website reality.
One online user claimed to have recreated the 1972 computer game Pong in less than 60 seconds by asking the chatbot to write the code.
Another said that it had taken 20 minutes of interaction with GPT-4 to create the game Snake.
OpenAI also said that subscribers will be able to build on GPT-4 to give it a unique personality.
It demonstrated by asking the software to behave like a tutor, teaching the human chat participant how to solve an equation, rather than just telling them the answer straight away.
But the software also has limitations, one of the most serious being what AI developers call “hallucinating”. The system will still confidently make up facts and make reasoning errors, it said.
“Great care should be taken when using language model outputs, particularly in high-stakes contexts, with the exact protocol (such as human review, grounding with additional context, or avoiding high-stakes uses altogether) matching the needs of a specific use-case,” OpenAI said.
The software tends to lack knowledge of what happened after September 2021, as it is largely trained on information which took place before this date. It also does not learn from experience.
Open AI added: “Depending on the traffic patterns we see, we may introduce a new subscription level for higher-volume GPT-4 usage; we also hope at some point to offer some amount of free GPT-4 queries so those without a subscription can try it too.”
