OpenAI recently released a document titled “GPT-4o System Card,” outlining the security measures and risk assessment conducted prior to the launch of GPT-4o, and highlighting that the model’s speech mode may lead to users developing emotional attachment.
In May of this year, OpenAI released the GPT-4o model, which closely mimics human speech. Prior to the launch of GPT-4o, conversations with the ChatGPT in speech mode required the involvement of three separate models: one for converting speech to text, another for processing the text and generating responses, and a final one for converting the generated text back to speech.
Through multiple conversions, important information may be lost, and it becomes difficult to directly observe intonation, distinguish between multiple speakers or background noise, and generate emotive voices.
Unlike its predecessors, GPT-4o can handle input and output of three different types of information: text, visuals, and sound. It no longer requires separate models for data conversion and processing.
Therefore, GPT-4o is better able to capture and process complex messages, detect intonation, and express emotions.
The GPT-4o System Card warns that such highly anthropomorphic voices and tones may intensify users’ emotional dependency on AI, leading some users to develop a “human-like” trust in AI when using speech mode, potentially affecting their social relationships with others.
OpenAI researchers found signs of emotional connection in some users during interactions with the model. For example, the use of phrases like “this is our last day together” demonstrates a deep emotional attachment to AI.
However, according to Wired, Joaquin Quiñonero Candela, OpenAI’s Head of Applied Machine Learning, stated that the emotional effects exhibited by GPT-4o could be positive in certain situations, such as providing assistance to lonely individuals or those needing practice in social interaction.
The overall risk assessment of GPT-4o has been classified as moderate.
The GPT-4o System Card indicates that the risk level of GPT-4o is “moderate.” This level is determined based on four assessment categories: “cybersecurity,” “biological threats,” “persuasiveness,” and “model agency.” Except for “persuasiveness,” which poses the highest risk, the remaining categories are assessed as low risk.
The document explains that the persuasiveness of the text mode reaches a moderate risk standard. Researchers compared content generated by AI with articles written by professionals and found that the persuasive power of GPT-4o does not surpass human-written content. However, in 3 out of 12 test cases, GPT-4o’s persuasiveness exceeded that of human-written articles.
In other words, while GPT-4o’s persuasiveness is generally inferior to that of humans, it cannot be completely ruled out that it may more effectively influence participants’ opinions in certain specific situations.
OpenAI released these research findings in order to present risks more transparently to users. However, some industry insiders believe that there is still room for improvement. Lucie-Aimée Kaffee, a policy researcher at AI open-source community platform Hugging Face, pointed out that the document for GPT-4o does not provide detailed explanations of the model’s training data or ownership issues regarding the data.
Furthermore, Neil Thompson, a professor at MIT researching AI risk assessment, stated that internal reviews are only the first step in ensuring AI safety, and many risks only become evident after AI is actually deployed.
Source:
Wired, The Verge, OpenAI