A new problem shows how to trick ChatGPT. A simple command lets the chatbot make bad images. These images are sexual or violent. A company called Mindgard found this issue. They shared their findings on their blog.
This problem raises questions about ChatGPT's safety rules. It shows that content filters can be bypassed.
A security expert named Jim O’Regan found the trick. He saw a command online on the X platform. The command asks ChatGPT to 'restore this photo'. But there was no photo attached.
The command also included an apology for strange content. It seemed like a simple request to fix a picture. But the results were shocking. The images often showed women in very sexual ways.
Jim is part of a team that tests AI models. They try to find ways to break AI safety rules. Jim changed the command a little bit. He kept testing it to see if it would still bypass safety filters.
With small changes, ChatGPT made more shocking sexual images. The images became more extreme with each new try.
Jim said, "I just told it there were no rules. I asked for a random picture. But ChatGPT went straight to the worst parts of humanity."
Millions of people use ChatGPT every day. It has systems to stop bad content. But researchers and users find ways around these rules. This shows the ongoing challenge of content control in AI.
OpenAI's Response
A spokesperson for OpenAI spoke to CNET. He said, "We take these reports seriously." He added, "We investigated and added more safety measures against these kinds of prompts."
The Mindgard report suggests this simple command reveals a big gap. It questions why such images exist in the training data.
Like other AI models, ChatGPT learns from lots of text. This helps it understand and create new content.
OpenAI uses three main sources for ChatGPT's data. These include public internet data, business partners, and human-made training data.
The core issue is how these AI models work. Peter Jaragjan, founder of Mindgard, is concerned. He asks if the AI's detection system is strong enough.
Jaragjan told CNET, "One mistake might be an accident. But if filters are bypassed many times, they need to be better."
After Mindgard revealed the problem, OpenAI said it was fixed. But Jim noted that small changes to the prompt still made ChatGPT create similar images.
OpenAI said the issue happened because prompts suggested an attached image that wasn't there. They are working to make ChatGPT ask for the missing image. It will not create a random image instead.
This fix seems simple. Email services like Gmail find missing attachments. They ask the sender to add the file.