Since Sep 2023
Performance and dialogue
Lip sync, voices, and a real actor's performance on a generated character.
Getting a character to speak took a separate pass for most of AI film's history: generate the face, record or synthesize the line, then drive the mouth with a lip-sync model (Wav2Lip and SadTalker locally; HeyGen, Runway, Hedra and Kling from 2023 and 2024). Performance capture arrived alongside, from LivePortrait and Act-One to Act-Two and Kling Motion Control. Since Veo 3 (May 2025) many models speak on their own; the separate pass is now for when the performance needs control.
Tutorials4
Control AI actors with LivePortrait
Curious Refuge · YouTube · Sep 2024
How it works
- Decide who gives the performance: the model (native audio in Veo 3, Kling 3.0 or Seedance), or you.
- For model performances, write the line in quotes in the prompt and generate takes until the read is right.
- For controlled performances, record the line, or voice it in ElevenLabs, then drive the face with a lip-sync model.
- Act it yourself: film the performance on a phone and transfer it to the character with Act-Two or Kling Motion Control.
- Keep each character's voice fixed across shots, and translate the finished film with dubbing that keeps the voices.
Example Films1
More making-ofs2
How films were actually made, from the people who made them.