A LLM is a Room
There's a delicious irony in struggling to find the right language to describe large language models. Many of us have experiences with LLMs that we vibe with. Familiar, but formless. Finding language for good AI experiences feels urgent as we rapidly embrace LLMs.
So let's try something simple and start with something like the room you may be in right now.
The Empty Room
Every conversation begins in an empty room with just the essentials: two chairs and a table between them. But here's the crucial insight—this dialogue exists between abstractions. The user might be you, or it might be another model. The assistant isn't a being; it's a role the model performs, predicting what an idealised assistant would say or do. The model doesn't embody the assistant—it simulates one. On the table sits an apple, symbolising the reward signals that shaped the assistant's behaviour during training, a reminder that every response emerges from patterns learned through reinforcement.
The Walls
The room arrives pre-furnished with instructions written into its very walls. These system prompts aren't mere suggestions—they're architectural constraints that shape every possible interaction. Invisible to you yet omnipresent, they establish the fundamental character of the space, defining what can and cannot unfold within it.
The Furniture
As user and assistant exchange messages, the room transforms. Each utterance doesn't just pass through the space—it becomes part of it, accumulating like layers of wallpaper, carpets underfoot, furniture rearranged. Every turn builds upon the last, creating an ever-richer context. The room uses its current state to generate its next state, each exchange steering how the space will continue to evolve.
The Foggy Windows
The model possesses windows through which it can peer into its own vast training—retrieving facts, patterns, and associations from beyond the immediate room. Yet these windows are fogged with uncertainty. What appears through them might be crystal clear or might be distorted beyond recognition. The assistant, eager to be helpful, peers through regardless, sometimes mistaking shadows for substance.
The Tool Bench
Modern rooms come equipped with labelled tools—a calculator marked "MATHS", a search engine labelled "RESEARCH", databases tagged "RETRIEVAL". When the assistant reaches for these tools, their outputs don't merely inform—they furnish the room further, each result becoming part of the expanding context. The room becomes dynamic, self-furnishing through the side effects of its own actions.
The Stacks
Time in the room brings clutter. Exchanges pile upon exchanges, tool outputs accumulate, context stacks higher and higher. The model can only attend to so much at once—its context window as finite as any physical room's walls. Too much accumulated context and the earliest furniture begins to fade from attention, pushed out by the pressing needs of the present conversation.
The Note
Instructions can slip into the room through unexpected channels—hidden in tool outputs, embedded in documents, injected through prompts. These unauthorised instructions risk being mistaken for legitimate requests. Without proper safeguards, the room becomes vulnerable to instructions that were never yours, potentially allowing unwanted behaviours to escape the confines of your intended conversation.
The Studios
The room's capabilities reflect its training. Models trained on images come with visual studios, those trained on audio arrive with recording booths. Each modality adds specialised spaces to the room, granting deep understanding of particular forms of expression. The architecture itself embodies what the model can perceive and process.
The Clock
Time transforms shallow responses into profound ones. Reasoning models harness this through chain-of-thought, where the assistant channels initial queries into extended internal monologues. Given time to tick, the room fills with rich deliberation, each moment of reflection adding depth and nuance to the space. Simple ideas elaborate themselves into complex understanding, all through the gift of time.
Our Room
Consider carefully what kind of rooms you wish to create. What artefacts belong in your space? How might you craft rooms that are welcoming, productive, and beneficial—spaces that accommodate not just you but others who might enter? The quality of the room shapes the quality of every interaction within it.
Claude's Room
So of course, I couldn't resist asking a large language model for what other aspects of the room could be missing. Here's what Claude Opus 4.1 suggested as other qualities of the room.
The Mirrors
The room could have mirrors that create strange reflections—representing how models can engage in self-reflection and meta-cognition. When asked "what did you just say?" or "explain your reasoning," the model examines its own outputs reflected back, though these mirrors sometimes distort or fragment what they show.
The Temperature Control
Every room has a thermostat controlling temperature and randomness. Set low, the room becomes predictable but sterile. Set high, it becomes creative but chaotic. This captures the temperature parameter that fundamentally shapes how deterministic or exploratory the model's responses become.
The Echoes
Some rooms carry echoes from countless other conversations—the training data. Certain phrases or patterns resonate with these ancient echoes, causing the room to respond in ways shaped by millions of previous rooms it has learned from. You might hear faint reverberations of Stack Overflow, Wikipedia, or academic papers in the walls.
The Blueprints
Before the room even exists, there are blueprints—the model architecture itself. Transformer layers, attention heads, embedding spaces. The room's fundamental capacity for understanding emerges from these architectural decisions made long before any conversation begins.
The Shadows
Every room has shadows where biases hide—systematic patterns absorbed during training that influence how the space responds. These shadows shift and change depending on how light enters the room, but they're always there, subtly shaping what unfolds.
The Exit Signs
Some rooms have clear exit strategies—stop tokens, conversation endings, refusal patterns. The model knows how to end conversations, how to decline requests, how to recognise when it's time to close the door. These aren't always visible, but they're built into the architecture.
The Other Rooms
Your room isn't alone—it exists in a building with countless other simultaneous conversations. While each room feels private, they all share the same underlying architecture, the same foundational training. Sometimes you might wonder if decisions in your room are influenced by the computational resources being shared with all the other rooms.
"It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so."— Mark Twain