Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics

來源原標題

Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics

作者

Latent Space

原始標籤

blog

原始連結

https://www.latent.space/p/dyna

摘要原文

Watch along AIE NYC and join us at AIE CODE in SF next month! This episode includes a live demo of the Taku robot that is best enjoyed on YouTube . Last month, Dyna made waves by announcing Dyna-2 , a world-action model pre-trained on one million hours of human video, claiming several new scaling laws on human data across four orders of magnitude . They have now followed up the model with a physical robot agent, Dyna-2.1 , that has to be seen to be believed: In his talk at AIE this year, Jason talked about how the real challenge for robotics in the real world is reliable, long-horizon autonomy , and how Dyna got its models to a 99.4% success rate folding restaurant napkins for 24 hours straight, including recovering after it pulled over the whole stack. From training a world-action model on over one million hours of human video to deploying robots that fold thousands of napkins daily at Din Tai Fung , Dyna Robotics is pushing physical AI beyond impressive demos and into real-world production. In this episode, Dyna Robotics co-founder and Chief Scientist Jason Ma joins swyx on-site to unpack how robots can learn new tasks from as little as 45 minutes of demonstrations, why predicting the future of the physical world may be the key to general-purpose robotics, and what it actually takes to build robots that work reliably for real customers. We go deep on Dyna-2, Dyna’s million-hour world-action model that combines future video prediction with robot actions, the scaling laws behind human-to-robot transfer, diffusion transformers and flow matching, and why synthetic environments and self-generated training data could unlock the next wave of robotics progress. Jason also explains why Dyna builds its own hardware instead of relying on off-the-shelf humanoids, how agentic harnesses enable robots to execute hour-long workflows, why dexterous hands and missing sensory data remain major bottlenecks, and what separates real commercial deployments from the graveyard of robotics startups. We also cover world-action models versus vision-language-action models, GPT-6 and coding agents in robotics, and Jason’s journey from AlphaGo and Penn to NVIDIA, Google DeepMind, and founding Dyna. Why Dyna-2 was trained on over one million hours of human video World-action models: predicting future video to guide physical robot actions Diffusion transformers , flow matching, and real-time robotic control How robots learn new tasks with as little as 45 minutes of demonstrations Why robotics has no equivalent of the internet and data remains its biggest scaling challenge Human egocentric video , robot demonstrations, and cross-embodiment transfer Synthetic environments , simulation, and models that generate their own training data Why robots need on-device inference , model distillation, and custom hardware Why Dyna chose wheeled semi-humanoids instead of traditional bipedal robots The hardware bottleneck: why dexterous robot hands still overheat Best Western, Din Tai Fung , and the difference between robotics pilots and scaled deployments The Bitter Lesson of Robotics: scaling data instead of engineering specialized solutions Agentic harnesses and robots executing multi-step, hour-long physical workflows GPT-6, coding agents , and AI-generated simulation environments for robotics research World-action models vs. VLAs , and why vision alone may not be enough Jason’s journey through AlphaGo, Penn, NVIDIA, Google DeepMind , and the Eureka project Dyna’s research culture , deployment-first philosophy, and vision for millions of robots LinkedIn: https://www.linkedin.com/in/jason-ma-742224a2/ 00:00:00 Introduction and Dyna-2’s World-Action Model 00:03:08 Diffusion Transformers, Flow Matching, and Generalist Robots 00:08:19 The Robotics Data Bottleneck 00:13:50 Synthetic Data and Self-Improving World Models 00:16:51 Real-Time Inference and Model Distillation 00:19:45 Full-Stack Robotics: Hardware and Supply Chains 00:25:02 Why Dyna Built a Wheeled Semi-Humanoid 00:29:35 Teaching Robots New Tasks and Evaluating Performance 00:31:59 Autonomous Workflows and Hardware Limitations 00:36:06 Real-World Deployments at Best Western and Din Tai Fung 00:41:07 The Bitter Lesson of Robotics 00:46:13 Agentic Harnesses for Physical AI 00:49:07 GPT-6, Coding Agents, and AI-Native Robotics 00:52:02 World-Action Models vs. VLAs 00:59:17 From AlphaGo to Founding Dyna Robotics 01:04:20 Dyna’s Research Culture and the Future of Robotics Swyx [00:00:00]: All right, we’re here on site with Jason Ma, the co-founder, chief scientist of Dyna Robotics. Thanks for having us in your office. Jason Ma [00:00:06]: Of course. Thank you for coming. Vibhu [00:00:07]: It’s been a blast. We got to see—you’ll see plenty of B-roll—drinks being pulled, a lot of evals, robots with knives for the first time. So under the hood of all this you have a world action model. What is that? Jason Ma [00:00:22]: So last month, we released our world action model called Dyna-2. So it’s the first world action model trained on more than one million hours of data. So going back to your question first, world action model is a model that’s able to predict the future video, the future state of the world, so hence the name world model. But in addition, you can also use that future generation to also produce actions robots can execute in the physical world to control the robot. So why do we do that? It’s a very powerful architecture, because if you think about how humans act in the physical world, in our brain, we are implicitly thinking about how the world will evolve based on our actions. So now we have a model architecture that’s able to do that to guide robot actions. So what we find is that by doing so, the model is much more dexterous, much more robust, and much more capable of generalizing to environments, to scenarios it’s never seen before. Vibhu [00:01:17]: I wanna prompt this question a bit. I think a lot of people hear world model, and they think you’re outputting video or world or 3D state. You’re also doing action. I think most people don’t understand what models are under the hood of robotics. So what are the inputs and outputs? What’s the training data? What is the architecture? Give us all the details. Jason Ma [00:01:39]: So, for inputs, the most important input is video, right? You can think about videos that the robot is capturing with its cameras, right? So video or images of the past many seconds, that’s the input. But for robots, what’s really interesting is that robot also has a lot of actuators and joint encoders on itself. So the robot also knows, the angles of its joints. So those can also be inputs to this world model. That’s something that’s very different from just a pure video model that’s only looking at videos, right? So that’s the input. And additionally, for robots, you want to be able to talk to it, to instruct it to do different tasks, right? So you can also feed it language instruction. So those are what I consider the basic set of inputs that go into any robotics models, including world action models. And once we have that the output, at the basic level, it needs to output actions the robots can execute on the hardware. But in the case of world action model, what’s really interesting is that in addition to the actions, it can also output future video, right? So basically, you can almost output the future video or some representation of it first, and then use that to guide robot actions. So the way I think about it is this is kind of analogous to chain of thought, you know. And basically, the models are reasoning in the visual, the physics space, and then outputting actions. So that’s the key difference between world action models outputs versus other kind of models, robot models outputs. Vibhu [00:03:08]: You put out a good blog post around your model. I think a lot of people understand language models or transformers, next-token predictors. In this case, you shared quite a bit.