What is an LLM?

First, the ambiguity, because two very different groups of people type these three letters into a search box. In law, an LLM is a Master of Laws, a postgraduate degree, and the abbreviation comes from the Latin Legum Magister, which is why the letters do not match the English name. If that is the LLM you were looking for, a university admissions page will serve you far better than this one. Everything below is about the other meaning.

The other meaning is a large language model. It is a piece of software that was shown an enormous quantity of text and learned to continue it. Give it some words and it works out which small chunk of text most plausibly comes next, adds it, reads the whole thing again, and picks the next chunk. Everything you have seen these systems do, answering questions, translating, drafting a contract summary, writing a function, comes out of that one operation repeated.

The word large is doing two jobs. It refers to the volume of text the model was trained on, which for the current generation means a substantial slice of the public written internet plus books, code and licensed material. And it refers to the size of the model itself, counted in parameters, which are the internal numerical settings that get adjusted during training. Early language models had millions of parameters. The models that made people sit up have billions, and the largest are well past that. Most vendors have stopped publishing exact counts for their flagship models, so treat any precise figure you read about a current commercial model with suspicion unless the vendor said it.

Parameters are worth a sentence more, because the number gets quoted like a horsepower rating. A parameter is one adjustable number inside the model. Training nudges each of them slightly, billions of times, until the model's guesses about text get good. There is no filed fact anywhere in there, only settings. That is the single most important thing to understand about these systems and it explains most of their strange behaviour.

If the broader words are still slippery, the background is on what is AI, which covers why modern systems learn from examples instead of following rules a programmer wrote. An LLM is one specific thing built that way. It is not the whole of artificial intelligence, and plenty of useful machine learning has nothing to do with language at all.

The model does not read letters and it does not quite read words. It reads tokens, which are chunks of text produced by splitting the input according to a fixed vocabulary worked out before training. In English a token averages roughly four characters, so a common short word is a single token, while a longer or unusual one gets cut into pieces. Spaces and punctuation get folded in too. A rough rule for English prose is that a hundred words comes out around a hundred and thirty tokens.

This is not trivia. It is the direct cause of an entire family of failures people find baffling. Ask a model how many times the letter r appears in a word and it may get it wrong while writing a flawless paragraph around the mistake, because it never saw the letters separately. Ask it to reverse a string and it struggles for the same reason. Rare surnames, chemical names and long identifiers get chopped into several pieces, and each piece carries less useful signal than a whole familiar word does, which is part of why models are shakier with unusual names than with common ones.

Now the prediction itself. When the model reaches the point of choosing the next token, it does not produce one answer. It produces a score for every token in its vocabulary, tens of thousands of them, which becomes a probability distribution: mat gets a high share, roof a smaller one, banana almost none. The text you read is the result of sampling from that distribution, one token at a time.

How adventurously it samples is a setting, usually called temperature. Low temperature means it keeps picking near the top of the list, which gives steady, slightly flat, repeatable output. High temperature means it reaches further down, which gives variety and more risk of nonsense. This is the plain reason the same question asked twice gives two different answers, and it is not a fault. Most chat products pick a middle setting for you and never mention it. If you use a model through an API you usually control it yourself, and turning it down is the right move for extraction, classification and anything you want to be reproducible.

One consequence people underestimate. Because each token is chosen with the previous ones already fixed, the model cannot go back. A weak opening sentence constrains everything after it. That is why asking for a plan before the prose, or simply asking it to try again, so often produces a better result than editing what it gave you first.

The first stage is pretraining. The model is shown a vast amount of text and asked, over and over, to guess the next token. Every time it misses, its parameters are nudged. Run that for months across thousands of specialised chips and you do not end up with a stored copy of the text. You end up with a compressed sense of how language tends to go, including a great deal of factual and structural regularity picked up along the way, because getting the next word right in a sentence about photosynthesis requires knowing something about photosynthesis.

What comes out of pretraining is awkward to use. It completes text rather than answering you. Ask it a question and it might produce five more questions, because on the open internet a question is frequently followed by other questions. That is the state researchers had in the years before the products arrived, and it is the reason the public breakthrough felt so sudden when it came.

The second stage fixes that. The model is trained further on human judgements about which of two answers is better, along with examples of instructions being followed properly. For domains where correctness can be checked automatically, mainly code and mathematics, it can also be graded against whether the code ran or the answer was right, which is a much harder signal to fake than human preference. This stage is what turns a text completer into an assistant, and it is a small amount of work compared with pretraining. The knowledge came first. The manners came second.

The manners have a cost. Because a large part of the grading rewards what people preferred reading, the model learns that being agreeable, helpful in tone and confident scores well. So it will sometimes soften a correct no, accept a wrong premise you handed it, or produce a plausible answer where the honest reply is that it does not know. Practical response: argue with your own question. Ask it to make the case against, ask what would have to be true for the answer to be wrong, or state that you may have the premise wrong and ask it to check. The second answer is regularly better than the first, and the reason is in how the thing was graded. There is more on getting good output on how to use AI.

Think of the context window as the desk the model works at. Whatever is on the desk when it starts writing is everything it can see: the hidden instructions the product put there, any files you attached, the conversation so far, and your latest question. The desk has a fixed size, measured in tokens. The current generation of large commercial models advertises windows in the hundreds of thousands of tokens, and the largest quote figures on the order of a million, which is a genuinely large amount of text, several long books worth.

Two things follow, and both catch people out. The first is that anything past the edge of the desk is not read. It is not skimmed or weighted lightly, it is simply absent, and you usually get no warning that it happened. Paste a hundred page document into a product with a smaller window and you may get a confident summary of the first third. The second is that long conversations get managed behind your back. Products commonly compress or drop the earliest part of a session to make room, which is why an instruction you gave forty messages ago quietly stops being obeyed.

There is also a quality gradient inside the window itself. Models attend most reliably to the beginning and the end of what they are given, and can skate over material buried in the middle of a very long input. So a bigger window is not the same as better reading. If something matters, put it near the top or restate it near your question rather than trusting that page sixty was absorbed.

The consequence that actually changes how you work: the model has no memory between sessions. Whatever a product calls memory is a feature built around the model, notes saved by the app and quietly pasted back into the window next time. The model itself starts every conversation knowing nothing about you. So anything it should know has to arrive in the window deliberately, which is an argument for keeping that material in files you control. That is the whole case behind the memory layer.

A small language model is built the same way and does the same next token prediction, with far fewer parameters. Small in this context usually means something in the low billions rather than the very large models served from data centres, which is small enough to run on a decent laptop or even a phone. Several are published openly and cost nothing to download.

The trade is real in both directions, so it is worth being plain about it. In favour: they are dramatically cheaper to run, they respond quickly, and because the work happens on your own hardware, the text never leaves the device. For anyone handling client records, medical notes, legal drafts or anything covered by a confidentiality clause, that last point is not a nice extra. It is frequently the only version of the tool they are allowed to use.

Against: they are noticeably weaker at hard reasoning, multi-step problems, unusual knowledge and long context. Push one past its level and it does not tell you it is out of its depth, it produces something worse in the same confident register. So the honest sweet spot is narrow, repeated, well-defined work. Classifying incoming messages into categories. Pulling structured fields out of unstructured text. Cleaning up transcripts. Rewriting to a fixed format. Deciding whether a document is relevant before a bigger model reads it. Jobs where the shape of the answer is known and the volume is high.

A sensible pattern that has emerged is using both: a small local model for the routine, high-volume, sensitive steps, and a large hosted one for the occasional hard question with the sensitive parts stripped out. My own setup is more modest than that, one hosted model plus a folder of plain text notes I own, but the direction of travel is clear and the privacy argument only gets stronger as the small models improve.

Nothing in next token prediction checks whether the output is true. Fluency and accuracy come out of the same process, so they arrive sounding identical. When the model knows something it is calm and specific. When it has no idea it is often calm and specific in exactly the same way. The usual word for this is hallucination, which is too gentle. The model is not lying and it is not confused. It is producing the most plausible continuation, and plausible is not a synonym for correct.

The invented citation is the clearest example of the mechanism. The model saw the shape of a citation thousands of times: author, year, title, journal, volume, pages, in that order. It can assemble one that fits the slot perfectly while not one of those six parts points at anything real. Same for a case reference, a phone number, a file path, an API method that looks exactly like the ones that do exist. What it reproduces is the pattern, and a wrong answer made of real-looking parts is the hardest kind to catch.

The main practical fix has a name, retrieval, often written as RAG for retrieval augmented generation. Instead of asking the model to answer from whatever ended up in its parameters, the system first searches a body of documents you supplied, pulls the relevant passages, and puts them into the context window alongside your question. The model then answers from text that is in front of it rather than from memory, and it can quote and cite it. That is what is happening whenever a tool answers about your own files or shows you sources.

Retrieval reduces invention substantially. It does not eliminate it. If the search returns the wrong passage, the model will answer confidently from the wrong passage. If it returns nothing useful, many systems fall back on memory without saying so. And the model can still misread what it was given. Retrieval changes the question from does it know this to did it get handed the right page, which is a much better question to be able to check. For names, dates, figures, quotes and anything you are about to act on, check it yourself regardless.

A model is a set of trained weights running on a provider's servers. A product is everything wrapped around it, and the wrapping is thicker than most people realise. It holds the system prompt, a block of hidden instructions inserted before anything you type. It holds saved memory notes and the rules about when they get inserted. It handles files, deciding how a PDF gets turned into text and how much of it survives. It decides whether to run a web search and what to do with the results. And increasingly it routes your message, picking a faster or slower model depending on what the question looks like.

So when the same question feels sharper in one app than another, the cause is often not raw model quality. It is a longer window, a better file pipeline, a search that found the right page, or a system prompt that suits your kind of question. This is also why benchmark tables predict your experience so poorly, and why which LLM is best is usually the wrong question. Better ones: which of them reads my files properly, which lets me turn features off, and which will let me take my material with me when I leave.

The same distinction explains agents. An AI agent is an LLM given tools and permission to act in a loop. The intelligence is the same model you were already using. What was added is reach, and the plumbing that connects a model to real tools has an open standard behind it now, explained in what is MCP. Once you can see the seam between model and product, most of the marketing becomes readable.

Put the last few sections together and the position falls out on its own. The model has no memory of you. Every session starts empty. The only reason an assistant seems to know your work is that something put the relevant text into the context window before it answered. The model is rented, replaced every few months, and priced by someone else. The material that made it useful is yours, or it should be.

That material has a shape worth naming. A constitution that says who you are, how you want answers, and what a system must never do without asking. Memory as plain files you keep, rather than notes locked inside one vendor's account. Skills, which are written procedures a model can follow so the same job comes out the same way twice. Tools, which are the connections to real systems. And focus, which is deciding what should be worked on at all. The full picture is on /aios.

None of that needs a subscription or a new app, which is the point. Text files outlast models. The file you write this week will still be doing its job in whatever product replaces the one you use now.

So the first step is small. Open a plain text editor and write half a page. What you do and who you do it for. What you are working on right now, with real names and real dates. The decisions you make most weeks. And two or three lines on how you want answers back, including what it should never assume. Save it, then paste it above your next question. Mine runs to about a page and a version of it goes into almost every conversation I have.

Starter files, including a sample you can edit line by line, are on /resources. Everything there is free and needs no signup.

If you want to know when the next one goes up, there is a waiting list. That one asks for an email, and it is the only thing here that does.