World Models: A Founder's Guide

A world model is a model that predicts how an environment changes when something acts on it. Give it the current state of a scene and an action, and it hands back the next state. That is the whole idea.

It sounds narrow. It is not, and the reason is that almost nothing useful in the physical world can be done without one. A robot deciding whether to push a box, a car deciding whether to merge, a planner deciding where to put a shelf: all of them need an internal answer to the question what happens if. Language models never needed that, because text is a record of a world rather than the world. This is the piece that was missing.

The material here comes from an a16z conversation with Fei-Fei Li and Yunzhu Li, hosted by a16z general partner Martin Casado, recorded after World Labs acquired the robotics company SceniX in July 2026. Fei-Fei Li is the Stanford computer scientist behind ImageNet and a co-founder of World Labs, which raised $1 billion in February 2026. Yunzhu Li co-founded SceniX and is an assistant professor at Columbia University, with a PhD from MIT and a postdoc in Fei-Fei Li's lab at Stanford. Because two of the three share a surname, this piece uses their first names after the introduction.

What makes the conversation worth a founder's time is not the acquisition. It is that two people who sell this technology spend most of it explaining what world models cannot do yet, and in the process describe a market that is smaller, nearer, and more addressable than the humanoid headlines suggest.

What a world model actually is

Which side of the model the action sits on decides what you have actually built, and Yunzhu Li lays out both cases. If actions go in, "that is essentially a forward simulator that is going to predict how the environment is going to change when you apply a specific action." If actions come out, "this is essentially a policy model that is trying to predict given a specific goal like what should be the action you take in the real environment to get you closer to that goal."

Same architecture, opposite jobs. The forward simulator is the world. The policy is the thing living in it. A robotics foundation model will likely do both. Yunzhu describes it as necessarily multimodal, taking in frames, text, images, and depth, with action as one of the most important modalities in that set.

This is the distinction to hold onto, because it tells you which business you are in. Selling the policy means competing with well-funded robot foundation model labs. Selling the forward simulator means selling infrastructure to all of them. Those are not the same company, and the second one is far easier to start.

Fei-Fei Li's framing sits one level up: "we are building the next frontier of AI which is what we call spatial intelligence," she says, meaning AI that can generate, understand, reason with, and interact with spaces, physical or virtual. Large world models are the means to that end rather than the end itself. That distinction matters commercially: it is why World Labs sells into visual effects, gaming, and design today, and treats robotics as one application of the same capability rather than the only one.

Why is a video model not a world model?

The most common approach in robotics right now is to use a video model, and Casado asks directly how the 3D and simulation approach differs. Yunzhu's answer is consistency. A world for a robot to learn in has to hold together "over space over time over different viewpoints and over different type of interactions." Then the failure case, which is worth memorizing because it explains the whole category:

"Imagine if a robot push an object forwards. The object just magically disappear," he says, describing a problem with many existing video prediction models. A clip like that looks fine to a human. It is poison to a robot, because it teaches the wrong physics.

This is the difference between predicting plausible pixels and predicting a consistent world. A video model optimizes for the frame looking right. A world model has to keep the box existing. Photorealism does not save you: a gorgeous render where objects blink out of existence is worse than a crude one where they do not, because the crude one at least teaches conservation of matter.

Yunzhu is careful not to declare a winner. Video models are getting stronger quickly, and his expectation is convergence: a stack where the model is "somewhere in the middle" between physics-only and learning-only, one that can "capture the essential structure of the problem" while still improving as data accumulates. If you are building on top of this, that middle is where to place your bets, not on either pure extreme.

Why data is the binding constraint, and why it is not the same constraint LLMs had

The data problem in the physical world is different in kind from the one language models faced, not just in degree.

Fei-Fei Li states it plainly: robotics faces a lack of data in training and a lack of data in evaluation, and "This is very very different from language models where data is abundant on the internet." Language models worked because a decade of human text was already sitting there. Nothing equivalent exists for physical action. And yet, as she puts it, "we have to somehow unlock the power of scaling law." You cannot scale what does not exist.

Collecting it by hand runs into arithmetic. Teleoperation, where a human drives the robot with a rig, is the standard method, and Yunzhu Li points out the flaw people miss: "you're actually collecting the data at a speed that is actually slower than human actually doing the task." You are paying a person to do the job worse than they would do it unaided, in order to get data. That is a hard ceiling, and it gets worse when customers want speed: "human speed to them is not good enough. They want faster than human speeds," Yunzhu says, and you cannot simply run the robot faster to collect faster data, because, as he puts it, the gravity does not change.

We covered the data side of this in more depth in physical AI, including the shift toward egocentric video, and the broader question of what data an AI product actually needs sits underneath both. The point here is narrower: the data shortage is what creates the market for world models. Simulation is not a nice-to-have optimization. It is the only way to manufacture the data that does not exist.

What simulation gives you that real data cannot

Fei-Fei Li's argument for simulation is not about cost.

"There's a very important role simulation plays that real world data doesn't play," she says: counterfactual reasoning. You play out events that have not happened, cannot happen, or that you do not have enough data to make happen, and you learn how to act in them anyway. Humans do this constantly. We simulate in our heads.

Real-world data can only ever show you what did happen. It structurally cannot show you the thing that almost happened, which is precisely the case a safety-critical system needs the most practice on.

The most mature commercial proof is autonomous driving, and Fei-Fei reaches for it directly, noting that "cars are the simplest kind of robots." Waymo says its Driver has traveled nearly 200 million fully autonomous miles on public roads and billions of miles in virtual worlds. In February 2026 it introduced the Waymo World Model, built on Google DeepMind's Genie 3 and adapted for driving, and the justification is exactly counterfactual: it lets Waymo simulate rare events, from a tornado to an encounter with an elephant, that are close to impossible to capture at scale in reality. A company with 200 million real autonomous miles still leans on simulation, which should settle the argument about whether simulation is a crutch for teams that lack data.

Yunzhu Li breaks the benefit into two, and this is the framing to steal for your own product reasoning: "simulation can provide two levels of benefits. The first one is reliability and the second one is efficiency."

Reliability comes from coverage. For a robot to work reliably in the real world, he says, you need data that provides "systematic coverage of all the state space and the variations that robots might encounter." In simulation you get that deliberately: "you can do systematic randomizations and control and the variations of lighting, frictions, geometries, object types" until the state space is actually covered. In the real world you wait and hope the edge case shows up.

Efficiency comes from breaking the physical constraints. In simulation you can systematically speed up the robot's behavior while still accounting for the dynamics, which is the only path to the faster-than-human performance customers ask for.

He also names two use cases for the platform, training and evaluation, and the evaluation half is the one founders should weigh heaviest. It gets its own section.

Why is evaluation the underpriced half of this market?

Evaluation is the part teams skip, and Yunzhu says it is routinely overlooked in robotics. It is also not optional: "if you are tuning like robotic models you have to know how well it works and that is the only source of information for you to iterate."

The concrete problem is wall-clock time. The question a robotics team asks every day is whether this checkpoint is better than the last one, and telling a 90 percent checkpoint from a 92 percent one requires enough real trials to see through the noise. Doing that with physical robots is brutal. Yunzhu's summary is that the iteration speed in the real world is "multiple orders of magnitude slower than iterations of those language models." He returns to the other costs a moment later: "not only it's slow, it's dangerous, it's costly."

If your simulator has demonstrated alignment with reality, that changes. "If a checkpoint is working better in the simulation is also highly likely to also work better in the real environments," Yunzhu says, and that inference is the entire product. You are not selling synthetic data. You are selling a faster iteration loop, which is the thing every ML team already knows how to value.

Notice how much easier this is to sell than training data. Evaluation is a smaller, better-scoped promise: you do not have to claim your simulated world is good enough to learn from, only that it ranks checkpoints in the same order reality does. Some SceniX customers buy only the real-to-sim half for exactly this reason, digitizing the specific task they care about so they can evaluate against it. That is a wedge a small team can actually deliver.

Note the boundary, because it is easy to blur. That evaluation is expensive in robotics is a known cost, and physical AI treats it as one. What is new here is evaluation sold as a product: a simulated environment whose only promise is that it ranks your checkpoints in the order reality would. The pattern generalizes past robotics, which is why we wrote LLM evals for founders. Most teams building on models are flying without instruments, and the fix is the same one here: build the measurement before you scale the thing being measured.

How photorealistic does a world model need to be?

Less than you would guess, and realism is the wrong bar to measure it by. Casado poses the sharper version: robotics needs precision, but creative uses do not, so how do those reconcile?

The answer is that the model does not have to be perfect in robotics either. It has to capture the essential structure of the problem. Yunzhu's example is locomotion. Quadruped and bipedal robots walk on snow and through bushes, and nobody simulated the snow or the bushes precisely. The simulator captured the structure, then randomized hard across everything else.

That reframes the engineering target and it should reframe your buying criteria. When a vendor demos a beautiful scene, the question is not whether it looks real. It is whether behavior transfers: does what happens in their world happen in yours? SceniX's stated goal is that "whatever happens in the digital world is also going to happen in the real environments," which is a testable claim in a way that visual realism is not.

Ask for the transfer evidence. Ask which parameters they randomize. A vendor who leads with render quality is selling to your eyes rather than to your robot.

Where do software founders fit in the world model stack?

Casado makes the point most people miss when they hear a company is going into robotics: "that's not building a robot it's building an environment which another company can place their robot brain" into.

That is a software business with software margins sitting underneath a hardware industry, and choosing your layer deliberately is the argument the AI for startups pillar makes across the rest of the stack. The layer here is defined by two properties Yunzhu names: it is model agnostic and embodiment agnostic. "our platform right now is just naturally embodiment agnostic," he says, so customers running single arms, fixed arms, mobile manipulators, grippers, and more elaborate end effectors all use the same infrastructure. On the model side, generated worlds can train models from scratch or post-train existing vision-language-action models.

Two things follow for founders.

First, you do not need a robotics lab to sell into robotics companies. You need to be genuinely indifferent to whose robot it is, which is a narrower and more buildable position than the one most robotics startups take.

Second, look at how this company found demand. Before any acquisition talk, SceniX signed up as a Marble customer. World Labs was separately fielding inbound: "our inbound customers for Marble were already seeing this kind of demands," Fei-Fei Li says, from early-stage robotics companies, and the honest admission is the useful bit, "we just cannot serve these customers." Unserved inbound from a segment you did not target is about as clean a demand signal as a founder gets. She also singles out the habit that made the team credible: two of its three co-founders came from academia, and even so "their first instinct is work with design partners and customers in real industry," in industry labs, warehouses, and electronics assembly. That is a founder behavior you can copy this week regardless of what you are building.

Which environments do world models pay off in first?

Casado notes that Yunzhu Li has told him the predictions around humanoids were somewhat aggressive, and that more constrained rollouts are the likelier path. The reasoning is a ladder every robotics deployment climbs, and it doubles as a market-selection framework.

Fully structured: you have knowledge and control over every configuration in the environment. Car manufacturing lines. Automated for decades.

Semi-structured: you control some of it. Amazon warehouses, restaurants, hotels. You can arrange the space to make the task easier, but clothes and stray objects still show up.

Unstructured: your house. Yunzhu calls this the grand challenge, and nobody in the conversation pretends it is close.

Robustness, as Yunzhu puts it here, comes from "a sufficient coverage of the scenarios that robots might encounter," so the size of that set is the difficulty. Semi-structured is where the coverage problem is tractable today, which is why Yunzhu describes moving there first as the sustainable and realistic approach.

Fei-Fei Li adds the argument against humanoids as the default form factor, and it is evolutionary rather than engineering: "evolution has optimized human body for unstructured environment," she says, which makes it general but not best at anything in particular. Commercially, "this unstructured environment and a generalized body is actually the hardest problem to solve. It's not necessarily even the right way to solve the problem." Specialize the body, narrow the problem.

For a founder this is a rule about where to point: pick a semi-structured environment and a specialized form factor, and you have made both halves of the problem easier at once.

The reliability bar is higher than anything you shipped with an LLM

This is the part that should change how you plan, and it is where the security discipline earns its keep.

Yunzhu Li draws the contrast exactly. Language models have incredible capabilities, "but still you don't just blind trust it to book your flight tickets or make your hotel reservations you still hopefully there's still a person who reading the output from those language models." There is a human in the loop absorbing the errors, and every LLM product you have shipped has quietly depended on that.

Robots do not get that. "for robotic models out of the box the robots has to work reliably in the real environment." No reviewer sits between the model and the consequence. The error budget you have been operating with does not transfer.

Three things follow if you are building or buying in this stack.

Sim-to-real alignment is a safety control, not a feature. The moment you rely on a vendor's claim that behavior in their world predicts behavior in yours, that claim is load-bearing for physical safety. Treat it the way you would treat any control: ask for evidence, ask how it is measured, ask what happens when the model is updated. A simulator that silently drifts out of alignment after a version bump is a control that failed without alerting anyone.

Your simulator is now in your supply chain. It shapes the behavior of a machine that moves in the world, which makes it a more consequential dependency than most of your infrastructure. Version it, pin it, and know what changed when behavior changes.

Digitized environments are sensitive assets. A real-to-sim pipeline produces a detailed geometric model of a customer's actual warehouse or lab. That is facility reconnaissance in a file. It deserves the handling you would give production customer data, including retention limits and clear terms on whether it can be used to train shared models. Most early vendor contracts in this space will not address it unless you raise it.

Yunzhu is also clear that no single model solves this. Every working robot in the real world, he says, is a system: hardware, software, brain, down to the friction coefficient of the fingers. Systems get reliable through iteration, not through a checkpoint.

Which is why Yunzhu's closing note is the right one to plan against. The state of the art, he says, keeps moving faster than he expected, and still "we also have to be like calibrated about our predictions." Expect real progress and long timelines to human-level efficiency at the same time. Both are true.

If you want the broader map of where this fits alongside agents, pricing, and go-to-market, start with the AI for startups pillar, and see data for AI for the data infrastructure question underneath all of it.

What to do this week

1. Decide which side of the model you are on. Write one sentence: are you building the forward simulator, the policy, or an application on top of someone else's? If you cannot answer, you are probably drifting toward the most expensive option by default.
2. Look for the evaluation wedge in your own product. Whatever you build, ask what your customer's iteration loop currently costs in wall-clock time. Selling a faster loop is easier than selling a better model, and the buyer already knows the number.
3. If you are buying simulation, ask for transfer evidence, not renders. Which parameters do they randomize? What is the measured correlation between simulated and real performance? How is alignment re-verified after a model update? Put the answers in the contract.
4. Pick a semi-structured environment. If your roadmap depends on unstructured spaces, move the first deployment to somewhere you control part of the layout. Coverage is the constraint, so shrink the space you have to cover.
5. Audit one counterfactual you cannot currently test. Name the failure case your product would handle badly and that you have no real data for. That gap is what a world model buys you, and naming it tells you whether you need one yet.
6. Classify your digitized environments. If you scan a customer facility, decide today who can access it, how long you keep it, and whether it can train shared models. Write it down before a customer asks.

The pattern underneath all of this is the one this course is built around: figure out which layer of the stack you are actually operating at, then build the measurement loop before you scale. That is what AI Operating System for Startups teaches, applied to whatever you are building.

Sources

• Fei-Fei Li is Solving the Hardest Problem in Robotics, World Labs with a16z is the conversation this article distills, with Fei-Fei Li, Yunzhu Li, and Martin Casado.
• World Labs Acquires SceniX, the company's own announcement of the July 2026 acquisition.
• Marble: A Multimodal World Model, World Labs on the model referenced throughout, publicly released in November 2025.
• The Waymo World Model is the source for Waymo's simulation mileage, its rare-event argument, and its use of Google DeepMind's Genie 3.
• World Labs Announces New Funding, the company's own February 2026 post on the $1 billion round, also reported by Reuters.
• Profiles: Fei-Fei Li at Stanford, Yunzhu Li at Columbia, and Martin Casado at a16z.
• Cross-source context from Nvidia's Jim Fan at Sequoia's AI Ascent on simulation as the pre-training substrate for robotics, and Waymo co-CEO Dmitri Dolgov on YC's channel, who argues that closed-loop simulation is required for evaluating physical AI systems.