Apr 2021 – mid 2022
Frame by frame
Warping one image forward, then re-dreaming it toward a prompt.
Before video models, AI artists made animations one picture at a time. A program would nudge, zoom or rotate the previous frame, then redraw it to match a text prompt, hundreds of times over. Stitched together, those frames became the dreamy, ever-shifting films of 2021 and early 2022.
How it worked
- Write a prompt, or a dictionary of
{frame: prompt}keyframes, in a Colab cell and pick a 2D or 3D animation mode. - Each frame starts from the previous one, warped by a camera schedule (
zoom,translation_x/y/z,rotation_3d_*as0:(1.02)strings; 3D mode borrows depth from MiDaS). - The warped frame is re-optimized or re-diffused toward the prompt for a set number of steps (init strength and skip steps decide how much survives).
- Wait hours on a Colab GPU while frames land in Google Drive, then assemble with ffmpeg and smooth 1–4 fps output with RIFE or FILM interpolation.
- For live footage, feed each source frame as the init (Disco v4.1 "video input") or stylize a handful of keyframes and let EbSynth carry them across the shot.
- Latent-walk videos went the other way: interpolate between seeds or prompt embeddings instead of warping, for the morphing "dreaming" look.
Example Films1
More making-ofs1
How films were actually made, from the people who made them.
Related on the timeline2