Phenaki

Google researchers, including Dumitru Erhan, introduce Phenaki, a text-to-video model that does something no other system could yet do: it takes a sequence of prompts that change over time and turns them into one continuous video, minutes long. Their showcase is a 2 minute 28 second story told through a chain of prompts, a first-person motorcycle ride that becomes a run through the woods, a backyard where an elephant and then a robot appear, and the robot dances a tango. Another chains a teddy bear swimming in the ocean at San Francisco, diving under the water and ending on a panda.

Under the hood, a new video tokenizer (C-ViViT) compresses footage into a small set of discrete tokens with causal attention in time, which is what lets clips run to any length, and a bidirectional masked transformer generates those tokens from text. Training jointly on a large set of image-text pairs and a smaller set of video-text pairs let it render things its video data never showed.

While other models of the day maxed out at a few seconds, Phenaki showed that long-form AI video was possible in principle, and that a story could be written as a timeline of prompts rather than a single one. Like Google's Imagen Video, announced the same week, it never left the lab, but the idea of a prompt sequence as a script came back in the storyboard and multi-shot tools that followed. Erhan went on to co-lead Gemini Omni at Google DeepMind.