All workflows

Act V · Direct the production · 1 of 2

Film it, then generate

Since Jul 2025

Film it, then generate

Shoot the performance or block the shot in 3D, then let the models build the world around it.

The loud half of AI filmmaking generates the whole film; the quiet half still shoots one. A crew films what only a person can give, the performance, and models build the rest: backgrounds, weather, crowds, light and coverage nobody shot. Unveil did it for Salomon's Olympic campaign, CJ ENM for The House, the Rolling Stones with their own 1970s faces. The roots are AI as a VFX assistant (Runway's Green Screen, Wonder Studio in 2023) and studios like Asteria, which put generative steps inside a 2D and 3D animation pipeline.

Tutorials9

How to enhance live action footage with AI

Runway · YouTube · Feb 2026

How it works

  1. Previs the whole film, then shoot against it. Unveil built a thirty-shot storyboard in Runway before a two-day shoot near Paris, so the sixty people on set knew what the generated world would be. Martin Haerlin did the same on Griesson: pace, angles, wardrobe and set worked out in AI before the camera arrived.
  2. Shoot only what a person has to do. Faces, hands, the take. Haerlin shot Merle Schwietert on green screen and put the argument in one line: nuanced performance is irreplaceable. CJ ENM shot The House indoors in four days and generated every background around it.
  3. Replace the world with an edit model instead of a plate. Runway Aleph takes the footage you shot and swaps objects, changes the weather, relights and reframes it from a prompt. Luma Modify Video does it while pose, expression and lip sync survive, and Aleph 2.0 carries one frame's change across the sequence.
  4. Put the face or the character on last. Francois Rousselet shot the Rolling Stones video conventionally with credited body doubles, and Deep Voodoo laid the 1970s band over them. The version anyone can afford is Runway Act-Two or Kling Motion Control: film the performance on a phone and drive a generated character with it.
  5. Relight instead of compositing. Beeble SwitchLight turns a live plate into normals, albedo and roughness, so an actor can be lit after the shoot like a CG asset. Corridor rebuilt season three of Son of a Dungeon around it: green screen, CorridorKey for the matte, a phone for camera tracking, straight into Unreal.
  6. Generate the coverage you did not shoot. Feed an existing take back in with the character sheet and ask for the wide; the audio and the performance come with it. Christopher Gwinn found it in MiniMax H3 in August 2026 and Curious Refuge published the recipe two weeks later.
  7. Finish where you already cut. Since September 2026 Runway's plugins put Gen-4.5, Seedance 2.5, Kling 3.0, Veo 3.1 and Aleph 2.0 inside Premiere Pro and After Effects, and edits land at the playhead. Topaz and Runway Ruby still do the last pass: upscale and real HDR.
  8. Or block it in 3D instead of shooting it. Gray-box the scene in Blender, move only the camera and rough stand-ins, and give the video model the playblast as a motion reference with a start frame. Uncanny Harry's '90s test followed his Blender animatic shot for shot in Dreamina, and JSFILMZ had Claude build the blockout over MCP for Seedance 2.5.
  9. Or stage it in a 3D director. Autodesk's Flow Studio 3D Editor (Aug 2026) and Pippit's 3D Director Studio stage characters, props and camera paths in 3D inside the AI tool; Pippit hands the layout to Seedance 2.5, the pipeline behind The Dor Brothers' The Last Show.
  10. Leave room for the performance. The model follows the blocking closely, not frame for frame; as on a set, give a character a beat of slack and direct the rest in the prompt.

Example Films9

  • The film on the timeline
  • The film on the timeline
  • The film on the timeline
  • The film on the timeline
  • The film on the timeline
  • The film on the timeline

More making-ofs13

How films were actually made, from the people who made them.

Related on the timeline8

Sources & further reading14
NextThe agent as producerIn use now