\n
![](\"https:\/\/gizmodo.com\/app\/uploads\/2024\/05\/72304a9885903a2bfcba79ca40608c30-1024x683.jpg\")
<\/p>\n
\n<\/p>\n
Earlier this year, Sam Altman was confronted directly with a video from what has become a viral trend: people showing off the significant shortcomings of OpenAI\u2019s voice model. It seems he didn\u2019t particularly enjoy that, because OpenAI is taking steps to save Altman from future embarrassment. On Thursday, the company announced three new voice models meant to open up the technology to developers who might be able to do groundbreaking things like program a functional timer.<\/p>\n
Per the company, it is releasing GPT-Realtime-2, its first voice model with \u201cGPT-5-class reasoning\u201d that can allegedly handle difficult prompts and better maintain conversations than its predecessors. It also introduced GPT-Realtime-Translate, which it claims can translate speech from more than 70 input languages into 13 output languages while \u201ckeeping pace with the speaker.\u201d The final model, GPT-Realtime-Whisper, is meant for live speech-to-text transcription.<\/p>\n
\u201cVoice is becoming one of the most natural ways for people to use software,\u201d the company said in a statement. \u201cBut building useful voice products takes more than fast turn-taking or a natural-sounding voice. A voice agent needs to understand what someone means, keep track of context, recover when a request changes, use tools while the conversation continues, and respond in a way that feels appropriate to the moment.\u201d<\/p>\n
The challenges that building AI models have presented have become the subject of many a meme over the past year or so. TikTok user @huskistaken, aka Husk, is perhaps the master of the genre, regularly poking holes in the capabilities of OpenAI\u2019s previous voice models\u2014though instead of doing so as a red teamer preventing issues from making it into the final product, he primarily encourages OpenAI to make changes via embarrassment.<\/p>\n
It was one of Husk\u2019s videos that made its way to Altman earlier this year. The CEO was made to watch ChatGPT\u2019s voice model very obviously lie about starting a timer. Husk would ask the model to time how long it took him to run a mile, then immediately say he was done, only for the model to claim he finished his mile in 10 minutes. Altman, visibly annoyed about the whole thing, said it\u2019d be \u201cMaybe another year before something like that works well.\u201d<\/p>\n
The new models are meant to speed up solutions to this confounding problem. Per OpenAI\u2019s press release, the new releases are adept at \u201cvoice-to-action, where people can describe what they need and the system can reason through the request, use tools, and complete the task.\u201d They provide an example like asking Zillow to \u201cfind me homes within my BuyAbility, avoid busy streets, and schedule a tour for Saturday.\u201d That certainly feels a bit more advanced than \u201cstart a timer,\u201d but it stands to reason that\u2019d fall under the same functionality.<\/p>\n
The real test of OpenAI\u2019s new models will be the jailbreakers like Husk. Earlier this year, former OpenAI founder Andrej Karpathy argued that people simply haven\u2019t updated their priors on AI models, which he argued are advancing all the time in ways that don\u2019t garner the same attention as voices messing with the voice model. But those videos aren\u2019t old\u2014Husk uploads new ones regularly. If he stops posting with the release of this new model, chalk up a win for the true believers like Karpathy.<\/p>\n<\/p><\/div>\n