ElevenLabs, a voice AI startup co-founded by former Google machine learning engineer Piotr Dabkowski and ex-Palantir deployment strategist Mati Staniszewski, launches an AI model for generating detailed sound effects using text prompts.
This innovative leap forward fills the silent void left by OpenAI’s Sora, a state-of-the-art text-to-video model praised for its high-resolution video clips devoid of sound.
We were blown away by the Sora announcement but felt it needed something…
What if you could describe a sound and generate it with AI? pic.twitter.com/HcUxQ7Wndg
— ElevenLabs (@elevenlabsio) February 18, 2024
Following the buzz around Sora’s unveiling, which showcased the model’s capacity to create visually captivating videos from textual descriptions, ElevenLabs announced its pioneering project on February 18, 2024.
The project aims to transcend the limitations of current AI video technology by introducing an auditory dimension to the silent films produced by Sora.
Eleven Labs wrote in a recent blog post, “We used text prompts like “waves crashing,” “metal clanging,” “birds chirping,” and “racing car engine” to generate audio that we overlaid onto some of our favorite clips from the OpenAI Sora announcement. “We’re thrilled by the excitement and support from the community and can’t wait to get it into your hands,” it added.
By utilizing text prompts such as “waves crashing,” “metal clanging,” “birds chirping,” and “racing car engine,” ElevenLabs successfully launches realistic AI sound effects onto video clips, thus enriching the sensory experience of AI-generated content.
ElevenLabs, valued at over $1 billion following a successful $80 million Series B funding round, has been trying to eliminate linguistic barriers in content through its AI voice technologies.
When the news hit the internet, people globally rushed to social media to express their excitement and anticipation for using the technology.
Very cool! Do I have to describe it with text prompts or can it infer it directly from input images or videos?
— Bilawal Sidhu (@bilawalsidhu) February 18, 2024
The company’s dedication to creating accessible and immersive digital experiences has propelled it to the forefront of AI voice cloning and synthesis.
With this latest venture into sound generation, ElevenLabs is not only broadening its technological repertoire but also challenging the traditional boundaries of video production and content creation.
Movies are 50% visuals and 50% sound, so great idea to use AI for soundscapes too!
— Everett World (@WorldEverett) February 18, 2024
Industries ranging from filmmaking and gaming to virtual reality and digital marketing stand to benefit from this technological advancement.
The ability to generate custom sound effects through simple text descriptions could revolutionize how creators approach audiovisual projects, enabling more seamless integration of visual and auditory elements and potentially reducing the costs and complexities associated with traditional sound design processes.
Now, THAT’S how you ride a hype train. Well done, eleven labs.
— Jessa • Femularity.ai (@IversusAI) February 20, 2024
As with any disruptive technology, the advent of AI-generated sound effects raises questions about the future of creative jobs and the authenticity of artistic expression.
The quote “AI will not take your jobs but the people that can use AI will” becomes more real by the day
As a content creator, this is a bit unsettling tbh
— francis O-F🎨🎥 (@FrancisOF4) February 19, 2024
The dialogue surrounding ElevenLabs’ innovation is multifaceted, embracing both the excitement of technological possibilities and the concerns over the implications for human creativity and employment within sound design and broader creative industries.
For more AI news and trends, visit the news section of our website.
