In mere moments of the new video app's launch, users started creating disturbing videos including copyrighted characters in compromising situations alongside explicit depictions of violence and racism.
Based on reviewed prompts and created videos, the artificial intelligence tool created multiple troubling situations, including:
Regardless of the company's terms of service that explicitly prohibit content that "promotes violence" or "creates damage", these visual creations promptly surfaced on various platforms.
The video generator stays exclusive to selected users and not yet available to the wider audience. However, within three days of its restricted launch, it secured the number one spot in Apple's App Store, outperforming even the company's own established products.
"It's been amazing to see what the combined imagination of humanity is able to achieve so far," stated the development head in a online update.
Researchers in misinformation alert that such lifelike artificial intelligence-created content could obscure truth and create situations where synthetic media might be employed in fraud, bullying, and intimidation.
"It has no fidelity to historical events, it has no connection to the reality," stated a media manipulation expert. "When cruel people obtain such technology, they will employ them in hostility, harassment and provocation."
The video application reportedly created numerous videos featuring protected intellectual properties from well-known entertainment franchises, such as:
"The guardrails are not effective if people are already producing protected properties promoting fraudulent schemes," stated a media professor. "In previous years, technology firms would have made commitments about content moderation ... Now, it appears they've determined they don't care."
The company has acknowledged "certain concerns" about how social media can be addictive and employed in bullying. The company asserts it has implemented mitigations for using someone's likeness and protections against disturbing or illegal content.
Nevertheless, experts remain doubtful about the effectiveness of these measures, especially given the quick dissemination of concerning material across online platforms.
"Synthetic media machines, whether created to produce text, images or video, are a scourge on our digital environment," cautioned a technology ethics expert. "Their outputs function analogously to an environmental disaster, spreading across technical and social infrastructure, weakening and damaging relationships of trust."
