Part 1: Where Physical AI Stands Now — An Extension of LLMs, or Something Else?

Conclusion: Physical AI is not a simple extension of LLMs

“Physical AI—isn’t that just an LLM getting a robot body?” That’s the common read, but the reality is a little different.

Let me give the conclusion first: the mainstream of Physical AI today is hybrid. High-level meaning and planning use LLMs/VLMs (large language and vision-language models), while the actual action generation combines VLA, diffusion models, flow matching, world models, and classical control. Rather than doing everything with a single giant model, stacking models with different roles has become the norm.

In this article, I’ll map where things stand from the angle of “which models are working, in which lineage.” Let’s start with the big picture.

Where we stand: the central term is “VLA (Vision-Language-Action)”

Physical AI today is a research field that unifies “seeing, understanding through language, and physically acting” in a robot. The keyword at its center is VLA (Vision-Language-Action).

What set the direction was Google’s RT-2. It connected a web-scale vision-language model to robot actions, showing that you can output robot control directly from images and language (RT-2 project page).

That said, LLMs/VLMs alone are too slow and too coarse for the continuous control of arms, fingers, and legs. So recent implementations often use a two-tier structure: VLM/LLM on top, a fast action policy below. Figure AI’s Helix is explained as exactly this—a VLA that combines a “System 2” for meaning and a “System 1” for fast continuous control. It’s close to how humans switch between “thinking it through” and “moving reflexively.”

Below, I’ll walk through the field in three lineages.

Lineage 1: The outgrowth of LLMs/VLMs — VLA

The first lineage adds “action” to ChatGPT- or Gemini-style multimodal models. Representative examples are RT-2, Gemini Robotics, OpenVLA, GR00T, and Helix.

Google DeepMind’s Gemini Robotics is a robot-oriented VLA built on Gemini 2.0, aiming for robots that understand, act, and react in the physical world. The derivative Gemini Robotics-ER strengthens the embodied-reasoning side—spatial understanding, 3D detection, and grasp reasoning.

OpenVLA is an open, 7B-parameter VLA (OpenVLA project page). It’s trained with a Llama 2-family language model, DINOv2/SigLIP-family visual features, and roughly 970,000 real-world robot demonstrations. What matters here is that it shifted robot control away from “building a dedicated model from scratch for each task” toward “fine-tuning a pretrained foundation model.”

NVIDIA’s GR00T N1 is also a VLA, but aimed specifically at humanoids. On paper it’s a dual structure: a vision-language module interprets the environment and language instructions, while a downstream diffusion transformer generates real-time motion. In other words, GR00T is a fine example of a hybrid that is both “LLM-lineage” and “diffusion-lineage.”

Lineage 2: Action generation via diffusion models and flow matching

The second lineage specializes in generating the action itself.

Robot motion is hard to handle with discrete token sequences like language. Grasping, pushing, folding, walking, balancing—these are continuous-valued, and there are multiple valid trajectories. There’s no single correct way to grip a cup; there are many patterns that work. Diffusion models are well suited to this multimodality.

Diffusion Policy expresses a robot’s visuomotor policy as a conditional denoising diffusion process and reportedly outperformed prior methods across multiple tasks and benchmarks (Diffusion Policy project page). RDT-1B is a diffusion foundation model for bimanual manipulation—pretrained with 1.2B parameters, 46 datasets, and over a million episodes, predicting the next 64 actions from language instructions and multi-view RGB images. This is less an LLM than a Diffusion Transformer for robot action.

Physical Intelligence’s π0 is an even clearer hybrid: it stacks a flow matching architecture on top of a pretrained VLM, aiming for a generalist robot policy that handles complex, dexterous tasks.

Lineage 3: World models, simulation, and synthetic data

The third lineage confronts the data problem head-on.

Physical AI’s biggest bottleneck is that no giant dataset like web text exists. Real robot data is expensive, slow to collect, and risky—move the robot and it can break. So world models, physics simulation, and synthetic data generation have become another mainstream.

NVIDIA Cosmos is a World Foundation Model platform for Physical AI, providing foundation models that simulate the physical world plus a framework for data processing, training, and evaluation. The paper positions Physical AI as needing both “a digital twin of the robot itself” and “a digital twin of the world.” Covariant’s RFM-1 likewise puts forward a “physics world model” that predicts the outcome of a robot action as AI-generated video. The idea is to internally predict “how the object will respond if I grip it this way” before moving.

A summary of the key models

Here are the models so far, laid out by lineage.

Model	Owner	Lineage	Key point
RT-2	Google DeepMind	VLA	Connects a VLM to robot action
Gemini Robotics	Google DeepMind	VLA / embodied reasoning	From Gemini 2.0, for the physical world
OpenVLA	Stanford / UC Berkeley, etc.	open VLA	7B, 970k robot demos
π0	Physical Intelligence	VLM + flow matching	Generalist robot policy
GR00T N1	NVIDIA	VLA + diffusion transformer	For humanoids
RDT-1B	Tsinghua-affiliated	diffusion transformer	Bimanual, 1.2B
Octo	Berkeley, etc.	transformer diffusion policy	800k trajectories, Open X-Embodiment
Helix	Figure AI	two-tier VLA	Extending to full-body humanoid control
Cosmos	NVIDIA	world foundation model	Synthetic data, world simulation

So, is it “an extension of LLMs”?

Half yes. VLA especially is a direct outgrowth of LLMs/VLMs. Understanding language instructions, visual understanding, common sense, task decomposition, tool understanding, spatial reasoning—these are transplanted from LLM/VLM capabilities.

But the heart of robotics isn’t there. Low-level control, continuous trajectories, multimodality, contact, friction, recovery from failure, real-time control—these are beyond an LLM alone. So the current mainstream is a configuration that uses LLMs/VLMs for meaning, planning, and interface, and uses diffusion/flow/Transformer policies for the body’s motion generation.

On the industry side, too, the 2025–2026 movement is shifting from a research phase to “the eve of large-scale deployment.” Figure is expanding Helix into logistics and household tasks, and Skild AI is putting forward a general-purpose “robot brain” usable across diverse robots. In Japan, a plan by Noetra and AIST for a Physical AI foundation model has been reported, aiming to release a foundation model as early as fiscal 2026 and improve it yearly with data from manufacturing and other sectors.

Summary

• The core of Physical AI isn’t “an LLM getting a body”—it’s “a control system that has a body taking in the meaning-understanding of LLMs/VLMs”
• Where we stand is a combination of three lineages: VLA (outgrowth of LLMs/VLMs) / diffusion & flow (action generation) / world models (the data problem)
• The likely winning path is neither pure LLM nor pure diffusion, but their composite architecture

In short, while Physical AI is adjacent to LLM research, in practice it is a fusion field of robotics, control, generative models, simulation, and the data flywheel.

Next time (Part 2), I’ll dig into how to design this “intelligence with a body,” drawing an analogy to the human nervous system. If the LLM is the cerebrum, then what plays the role of the cerebellum, the spinal cord, and the autonomic nerves?