Google Research Introduces an AI Video Co-Director: 4 Agentic Frameworks for Coherent, Minutes-Long Video Generation
https://www.marktechpost.com/2026/09/27/google-research-introduces-an-ai-video-co-director-4-agentic-frameworks-for-coherent-minutes-long-video-generation/📌 Google 用 4 套框架,治好 AI 長片的「失憶症」
TL;DR:Google Research 發表 4 個 agentic 框架,鎖定長影片生成中的身份漂移與連鎖錯誤問題。
Diffusion model 幾秒鐘就能生出一段畫質細膩的短片,但要把好幾段片段串成一則連貫的故事,卻是完全不同層次的難題——小偷的帽子莫名消失,寶石的顏色說變就變,這正是目前多數 agentic 影片生成 pipeline 的常見窘境。
🤔 語意漂移與連鎖失敗,是一個「歸因」問題
多數 agentic pipeline 是把多個模組串接起來,各自用手動撰寫的 prompt 驅動,這會造成兩種失敗:一是語意漂移(semantic drift),服裝或場景在不同鏡頭之間悄悄改變;二是連鎖失敗(cascading failures),上游一個壞掉的素材會拖垮後面所有鏡頭。Google 團隊把這個問題框定為「credit assignment problem」——當最終影片出錯,很難回溯是哪一句 prompt導致的。這套系統建構在 Gemini 與 Veo 之上,但屬於 model-agnostic 設計,理論上可以換用其他生成模型,輸出也繼承了底層模型的 SynthID 浮水印。
🧩 四套框架,各自解決一塊拼圖
- Co-Director(COLM 2026 論文):採用 multi-armed bandit(MAB)。Orchestrator Agent 負責在 Creative Strategy、Narrative Mode、Aesthetic Archetype 三個維度中挑選配置,Pre-Production Agent 建立分鏡腳本,再由 Keyframe、Video、Audio 三個 sub-agent 產出對應媒材,最後由 MLLM Judge 為成品評分,並把拆解後的獎勵訊號回饋給 bandit,持續調整選擇策略。
- CANVAS(EMNLP 2026 論文):持續追蹤角色、地點與物件狀態隨劇情演進的變化,當某個場景重新出現時,會取回先前儲存的視覺錨點。在 Google 的博物館竊案測試中,對照組 AutoStudio 弄丟了小偷的帽子、Gemini-3.1-Pro 改變了寶石的顏色,而 CANVAS 讓兩者都維持一致。
- A²RD(Agentic Autoregressive Diffusion):一套免訓練(training-free)架構,每個片段都對一個多模態影片記憶庫執行 Retrieve、Synthesize、Refine、Update 的迴圈。遇到全新的劇情轉折時採用 extrapolation,遇到角色重新登場則切換為 interpolation。Google 用這套方法展示了一支 10 分鐘長片。
- VQQA(Video Quality Question Answering):針對每一句 prompt 生成對應的視覺問題,再用 VLM 的評論作為「語意梯度」反過來改寫文字 prompt,整個過程不需要存取模型內部參數。搭配的 Global Selection 步驟,會在所有迭代版本中挑出品質最好的一支影片,而不是單純採用最後一次迭代的結果。
📊 三個新 benchmark,專門壓測長片一致性
為了驗證這些方法,Google 建立了三個新 benchmark:GenAD-Bench 涵蓋 50 個品牌、200 個虛構產品下的 400 個廣告場景;HardContinuityBench 專門壓測場景重現與道具狀態變化;LVBench-C 則設計了 120 個場景,讓關鍵資產在消失至少 10 個 segment 後才重新出現,考驗系統的長程記憶能力。
🎯 實務啟示
對正在做多鏡頭、多 agent 影片生成 pipeline 的工程師來說,這四套框架示範了幾個值得參考的設計模式:用 bandit 做全域策略選擇、用視覺錨點做跨場景記憶、用 retrieve-synthesize-refine-update 迴圈處理長時序一致性,以及用 VLM 反饋當作沒有梯度可用時的「語意梯度」。這些拆解出的子問題,比起單純堆更大的 diffusion model,可能更貼近長片生成真正卡關的地方。
🔗 來源
- 標題:Google Research Introduces an AI Video Co-Director: 4 Agentic Frameworks for Coherent, Minutes-Long Video Generation
- 作者／機構:Asif Razzaq,MarkTechPost
- 連結:https://www.marktechpost.com/2026/09/27/google-research-introduces-an-ai-video-co-director-4-agentic-frameworks-for-coherent-minutes-long-video-generation/
#GoogleResearch #VideoGeneration #AgenticAI #Gemini #Veo #COLM2026 #EMNLP2026 #MultiAgent #DiffusionModels #AIVideo
原始資料 MarkTechPost · 收集於 2026-09-28
摘要原文
Google Research has introduced an AI video co-director for long-form video generation. The suite of 4 agentic frameworks turns short clips into coherent, minutes-long stories. It targets identity drift and cascading errors, the 2 failures that break most multi-shot AI video pipelines today. Diffusion models render high-fidelity clips in seconds. Stitching those clips into a story is harder. Most agentic pipelines chain modules with independent, handcrafted prompts. That causes semantic drift , where attire or scenery shifts between shots. It also causes cascading failures , where one bad upstream asset corrupts every later shot. Google team frames this as a credit assignment problem. A broken final video is hard to trace back to the prompt that caused it. The system sits on top of Gemini and Veo. It is model-agnostic, so the same layer can drive other generators. Outputs inherit SynthID watermarking from the base models. Co-Director , accepted at COLM 2026, uses a multi-armed bandit (MAB). An Orchestrator Agent picks a configuration across Creative Strategy, Narrative Mode, and Aesthetic Archetype. A Pre-Production Agent builds the storyboard. Keyframe, Video, and Audio sub-agents produce the media. An MLLM Judge then scores the cut and sends a factored reward back to the bandit. CANVAS , accepted at EMNLP 2026, tracks characters, locations, and object states as the story evolves. It retrieves stored visual anchors when a scene returns. In Google’s museum heist test, AutoStudio lost the thief’s cap and Gemini-3.1-Pro changed the gemstone. CANVAS kept both consistent. A²RD (Agentic Autoregressive Diffusion) is a training-free architecture. Each segment runs a Retrieve, Synthesize, Refine, Update loop against a multimodal video memory. The agent switches between extrapolation for new story beats and interpolation for returning entities. Google shared a 10-minute film generated this way. VQQA (Video Quality Question Answering) generates visual questions for each prompt. VLM critiques act as “semantic gradients” that rewrite the text prompt. It needs no access to model internals. A Global Selection step picks the best video across all iterations, not simply the last one. Google built 3 new benchmarks. GenAD-Bench has 400 ad scenarios across 200 fictional products from 50 brands. HardContinuityBench stresses scene reappearances and prop state changes. LVBench-C has 120 scenarios where key assets vanish for at least 10 segments before returning. Sources: linked papers, project pages, and GitHub repositories. Verified September 27, 2026. Check out the Technical Details . All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter . Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Google Research Introduces an AI Video Co-Director: 4 Agentic Frameworks for Coherent, Minutes-Long Video Generation appeared first on MarkTechPost .
由 claude-code/sonnet 自動生成