# Performance and dialogue

Period: 2023 – now · Era: 2023 · The Cambrian Explosion

Lip sync, voices, and a real actor's performance on a generated character.

Getting a character to speak took a separate pass for most of AI film's history: generate the face, record or synthesize the line, then drive the mouth with a lip-sync model (Wav2Lip and SadTalker locally; HeyGen, Runway, Hedra and Kling from 2023 and 2024). Performance capture arrived alongside, from LivePortrait and Act-One to Act-Two and Kling Motion Control. Since Veo 3 (May 2025) many models speak on their own; the separate pass is now for when the performance needs control.

## How it worked

1. **Decide who gives the performance:** the model (native audio in Veo 3, Kling 3.0 or Seedance), or you.
2. **For model performances,** write the line in quotes in the prompt and generate takes until the read is right.
3. **For controlled performances,** record the line, or voice it in ElevenLabs, then drive the face with a lip-sync model.
4. **Act it yourself:** film the performance on a phone and transfer it to the character with Act-Two or Kling Motion Control.
5. **Keep each character's voice fixed** across shots, and translate the finished film with dubbing that keeps the voices.

## Why it mattered

A face that cannot speak is mostly limited to trailers and montage. Lip sync and performance transfer are what turned AI clips into scenes, and they keep the actor in the loop even as models learn to perform.

## Tools of the era

- HeyGen Video Translate (2023): https://www.heygen.com
- Runway Lip Sync (2024): https://runwayml.com
- Hedra Character-1 (2024): https://www.hedra.com
- LivePortrait (2024): https://github.com/KwaiVGI/LivePortrait
- Kling Lip Sync (2024): https://klingai.com
- Runway Act-One (2024): https://runway.com/research/introducing-act-one
- Hedra Character-3 (2025): https://www.hedra.com
- Veo 3 (2025): https://deepmind.google/models/veo/
- Eleven v3 (2025): https://elevenlabs.io
- Runway Act-Two (2025): https://runway.com
- Kling Motion Control (2026): https://klingai.com
- ElevenLabs Dubbing: https://elevenlabs.io/dubbing

## Example videos and posts

- Control AI actors with LivePortrait · Curious Refuge: https://www.youtube.com/watch?v=xfzK_6cTs58
- Act-Two first look: a short film from a phone performance · Theoretically Media: https://www.youtube.com/watch?v=Ni_6i3LSBcQ
- Behind the scenes of an AI film: The Bridge (Veo 2, Hedra, ElevenLabs) · Theoretically Media: https://www.youtube.com/watch?v=6MRSaaxhz7A
- Three ways to keep a voice consistent across generations · Curious Refuge: https://www.youtube.com/watch?v=3qDL7A4sS2k
- You are the workflow now: Motion Control plus Seedance in Magnific · techhalla: https://x.com/techhalla/status/2071952099298955574
- How Neural Viz makes an entire universe · Neural Viz: https://x.com/NeuralViz/status/1921732202934788551

## On the timeline

- [The Bridge](https://aifilmhistory.org/entry/the-bridge-veo-2) (2025-03-24)

---
Source: https://aifilmhistory.org/workflows/craft-performance