Aug 2022 – 2023
Deforum - Infinite Morphing Zooms
The zoom that ate 2022: programmable cameras on top of Stable Diffusion.
Deforum put a programmable camera on top of Stable Diffusion. You typed schedules for zoom, rotation and how much each frame could change, hit go, and got a flowing, morphing animation, often timed to music. It was the first animation tool thousands of people used, and its look defined a year.
Tutorials4
AI animation with Deforum in Stable Diffusion
Vladimir Chopine (GeekatPlay) · YouTube · May 2023
How it worked
- Generate frame zero, or supply an init image.
- Warp it by the schedule:
zoom,angle,translation_x/y/z,rotation_3d_*, with MiDaS depth in 3D mode. - Run img2img at a scheduled
strength(0.4–0.65 was the sweet spot: higher fell apart, lower froze into mush) and repeat for every frame. - Layer
cfg_scale_schedule,noise_schedule,seed_scheduleand color coherence to a reference to keep the drift musical instead of random. - Sync to music: Parseq mapped beats and amplitude to zoom pulses and prompt switches across forty-plus parameters.
- Video-input mode used each source frame as the init; Hybrid Video (2023) pulled optical flow from a driving clip.
Related on the timeline2