Recently, Lalamu Studio has launched the beta version testing: Lalamu Studio. This tool integrates TTS and lip sync functions, allowing characters in any video to speak and accurately simulate mouth movements.
For example, select a video clip, add synthesized speech by AI, and you can achieve real-time playback and lip synchronization. Click on the link above to visit Lalamu Studio. On the left, you can synthesize speech and complete lip synchronization.
Select an animation, add a video script to the frame on the left, choose a voice actor below, click on generate audio, after completing speech synthesis, click on Generation below to start synthesizing the video. To save computing power, Lalamu Studio does not provide high-definition video output. However, from the generated animation, it can be seen that the character's mouth movements match the speech perfectly.
Next, select a live-action animation to test the effect of mouth movement simulation. Lalamu Studio is still in the testing phase, and after the official launch, it will support high-definition output. Lalamu Studio will also introduce multi-character voice synthesis and editing functions to make video editing more fun.
After a brief understanding of Lalamu Studio, I will demonstrate how to use Lalamu Studio to synthesize speech for animations.
First, use any tool to generate an image of TINKERBELL. Log in to Pikalabs: Pika, enter "a pirate of the Caribbean" prompt, generate a short video of a few seconds, and save it locally. Return to Lalamu Studio, upload the created short video. Next, log in to the TTS speech synthesis website Elevenlabs: Text to Speech & AI Voice Generator – ElevenLabs, click on the voice library option, and choose a suitable voice actor from the community.
Click on add to voicelab below, add the voice actor to your own voice library. Then, enter the video script and complete the speech synthesis:
"Hi, I'm Jack Sparrow, a pirate of the Caribbean."
Upload the voice file to Lalamu, click on the lipsync option, and achieve mouth movement simulation.
Finally, test the video materials provided by Lalamu, through multiple tests, Lalamu can accurately simulate the mouth movements of most cartoon characters, everyone can try it out. Then, test the simulation effect of real portraits.
Lalamu can be used to create fun short videos and animations, supporting scripts of up to 300 characters. Currently, Lalamu only supports German and English, not Chinese, so everyone can try it out on the website above.