Part 3: Don't Do Everything with One Giant Model — An Architecture of Hierarchy and Separated Responsibilities

Doing everything with one giant model is a waste of resources

To close the series, I’ll back up the “blueprint for embodied intelligence” from Part 2 from a different angle: resource efficiency.

Honestly, making a single giant model do everything feels like a waste of resources. And this intuition has an old backing in computer science: hierarchy and separation of responsibilities.

Think about a human:

• Spotting a person 100m away
• Tying shoelaces
• Breathing
• Keeping balance
• Holding a conversation

We don’t compute all of these in the cerebrum. If the cerebrum inferred “move the right foot 5mm forward, bend the knee 2 degrees…” dozens of times a second with every step, there’d be no room for conversation. The nervous system distributes processing across different circuits by type.

The same thing is starting to happen in AI.

Don’t run things on different time scales through the same model

Break robot control down and you get a hierarchy like this:

LLM
    ↓  "Bring me the cup"
VLA
    ↓  "Go to the shelf" "Use the right hand"
Motion model
    ↓  Joint trajectory generation
MPC
    ↓  Posture control at 1ms granularity
Motor driver
    ↓  PWM output

Within this, the LLM runs only at the very start. The layers below keep turning, each at its own speed.

What matters even more is that the time scales are utterly different. Even in humans it’s layered: the breathing reflex is tens of milliseconds, posture control 10–20ms, walking about 100ms, conversation hundreds of milliseconds to seconds, planning minutes to hours. It’s the same for AI; the natural periods are roughly these.

A giant model has no need to infer at 1kHz. Waste arises precisely because you cram the layer that should run at 1kHz and the layer that only needs to run once every few seconds into the same model.

This “separate slow thinking from fast action” idea has become explicit on the research side too. Hume, a 2025 VLA study, combines System 2 slow thinking (low-frequency value evaluation and resampling) with System 1 fast action denoising—splitting processing exactly by time scale (Hume paper). Hi Robot, too, combines a high-level LLM planner, a low-level VLA policy, and synthetic data generation, adopting a division close to “cerebrum = planning / cerebellum and spinal cord = execution” (Hi Robot).

Energy efficiency: solve locally what can be solved locally

The resource story is also, directly, an energy-efficiency story.

Today’s LLM fires billions of parameters just to make the judgment “I’m hungry.” Living things don’t work that way. If you’re hungry, a fair amount of processing finishes in a local circuit alone—stomach → vagus nerve → hypothalamus. The cerebrum only makes the higher-level judgment, “let’s go to the convenience store today.”

The principle here is simple: solve locally what can be solved locally.

Keeping balance, avoiding collisions, holding a motor at constant torque—calling the LLM for these every time is excessive. Leave them to dedicated lightweight circuits, and call the LLM only when it’s truly needed. AI is starting to adopt this division of labor that living things reached over hundreds of millions of years.

AI is approaching an “OS-like structure”

Thinking about it this way, I suspect AI will gradually approach a structure like an OS.

LLM
├─ Planning
├─ Language
├─ Reasoning

Local AI
├─ Vision
├─ Speech
├─ Motion
├─ Touch
├─ Navigation

Reflex
├─ Collision
├─ Balance
├─ Emergency stop

Autonomic
├─ Battery
├─ Cooling
├─ Resource management
├─ Network

The LLM is like “user space,” with dedicated services and daemons resident in the lower layers. It’s the same as how a user-space app (the LLM) doesn’t take over the kernel or drivers (the reflex and autonomic layers) every time. The “autonomic nerves” and “homeostasis” from Part 2 also correspond, in this diagram, to mechanisms that stay resident as lower-layer daemons.

What’s required is “organizing” the control software

This division of labor isn’t only about how you choose models or how you implement the robot hardware. The control software that ties them together also needs the same kind of “organization.”

Which module, at which period, holds which responsibility. Under what state it delegates the decision upward, and in an anomaly which layer takes the lead and how it stops safely. These rules of role-splitting and coordination have to be designed as software. Simply placing a high-level reasoning model and low-level control side by side won’t make the whole thing work.

The limits of a giant model on its own are being clearly discussed, too. A recent Embodied AI review organizes it as: MLLMs (multimodal large language models) are strong at contextual reasoning but tend to overlook physical constraints, while world models are strong at physical simulation but weak at high-level meaning-understanding—and it proposes combining the two (Embodied AI: From LLMs to World Models). NVIDIA likewise argues that the winning path ahead is likely neither a pure VLA nor a pure World Action Model, but a hybrid of the two (NVIDIA: The Rise of World Action Models). In short, “splitting models by time scale, responsibility, and distance from the body” beats “doing everything with one giant model.” And designing the control software that binds those split models together is where the real skill will show from here.

Summary: looking back over the three parts

• Doing everything with one giant model wastes exactly as much as it mixes processing on different time scales
• Like the nervous system, AI should be layered by type, period, and responsibility of processing (separate the 1kHz layer from the once-every-few-seconds layer)
• The principle is “solve locally what can be solved locally”; call the LLM only when truly needed
• Where it lands is an “OS-like structure” with the LLM in user space and the reflex/autonomic layers as daemons
• What’s required is “organizing” not just the models and hardware, but the control software that binds them

The core of the Physical AI we’ve traced over three parts was, in the end, not “an LLM getting a body.” A control system that has a body takes in the meaning-understanding of LLMs/VLMs only as much as needed, and leaves the rest to the right specialized circuits—the design of that division of labor is, I believe, the real battleground from here.