← back
Code World Model: Building World Models for Computation – Jacob Kahn, FAIR Meta
Takeaway
Meta's Code World Model predicts program execution traces as an autoregressive sequence so agents can imagine outcomes before running code.
Summary
- Jacob Kahn (FAIR Meta) presents CWM (Code World Model): instead of treating code as syntax, model the transition function of program states during execution.
- Training data: execution traces with frame separators, local variables, and memory state per line — line-by-line natural-language descriptions of execution paired with code.
- Scope ambitions go from function-level to repo-level and even distributed-system execution traces, generated from GitHub PRs plus CI/test runs.
- With a world model, agents can imagine execution outcomes without actually running code — drastically improving sample efficiency for agentic reasoning.
world-modelscode-generationmeta-fair
Original description
Today, most neural models for code learn from code itself: sequences of tokens that capture syntax rather than computation. While this allows models to learn the shape of code, true reasoning about programs requires understanding execution and the dynamics of computation. In this talk, I’ll present a world-model approach to learning from code: one that incorporates data from program execution to implicitly predict behavior while generating code. The Code World Model (CWM) embodies this paradigm, opening new capabilities for reasoning and offering a foundation for future research and prototyping in AI-driven software systems. Speaker: Jacob Kahn | Research Scientist, FAIR, Meta https://www.linkedin.com/in/jacobdavidkahn/