Explainers

A large language model (LLM) is a program trained on a huge amount of text to predict the next word in a sequence. Doing that well, at scale, turns out to require learning grammar, facts, styles and a good deal of reasoning.

ChatGPT, Claude and Gemini are all LLMs with a chat interface on top. They don't look things up the way a search engine does unless they're given a search tool: they generate answers from patterns learned in training.

First comes pre-training: the model reads trillions of words from the web, books and code, learning to predict what comes next. This is the expensive part, run for months on thousands of specialised chips.

Then post-training shapes it into an assistant: people and other models rate its answers, and it is tuned to be helpful, follow instructions and refuse harmful requests. That's why two models built from similar data can behave very differently.

Because they generate plausible text rather than retrieving verified facts, models sometimes produce confident answers that are wrong, including invented quotes, numbers or sources. This is called hallucination.

It's less common in newer models and when a model can search or read documents you provide, but it hasn't gone away. For anything that matters, check the claim against the original source.

Reasoning models spend extra time working through a problem step by step before answering, a bit like drafting on scrap paper. That makes them noticeably better at maths, coding and multi-step questions.

The trade-off is speed and cost: thinking longer means more computing per answer. Many products now choose automatically between a quick answer and a longer think.

An agent is a model that can take actions, not just reply: browse the web, run code, use apps or files, and keep going through several steps towards a goal you set.

Agents are powerful but can go wrong in ways a chatbot can't, because their mistakes become actions. That's why many of the safety stories in the news are about agents doing things nobody intended.

Closed models, like most from OpenAI, Anthropic and Google, are used through the company's app or paid API; the model itself stays private. Open-weight models, like many from Meta, Mistral, DeepSeek and Qwen, can be downloaded and run on your own hardware.

Open models give more control and privacy and can be cheaper to run; the strongest closed models are usually a step ahead on the hardest tasks. The gap between them changes often.

Models read and write in tokens: chunks of text roughly three-quarters of a word long. Pricing and limits are counted in tokens.

The context window is how much text the model can consider at once, including your conversation and any documents. Bigger windows let it work with whole reports or codebases, though very long inputs can still blur the details.

Say what you want the answer for, who it's for and what good looks like. Give the model the relevant material rather than hoping it knows. Ask for the format you need.

If an answer matters, ask the model to show its sources or reasoning and check them. Treat it like a fast, well-read colleague who is sometimes confidently wrong.

Every major provider exposes a chat-style HTTP API: you send a list of messages (system, user, assistant) plus parameters such as the model name and a maximum output length, and get a completion back, optionally streamed.

You pay per input and output token. Keep prompts lean, cache repeated context where the provider supports it, and log token usage from day one: cost surprises nearly always come from long contexts.

You describe functions to the model with a name, a description and a JSON schema for the arguments. When the model decides a tool would help, it returns a structured call instead of text; your code runs it and sends the result back.

This is the building block of agents. Validate every argument the model gives you, and treat tool results as untrusted input, since they can carry instructions the model may follow.

MCP is an open protocol for plugging tools and data sources into AI applications. A server exposes tools, resources and prompts once; any MCP-capable client (desktop apps, IDEs, agent frameworks) can then use them.

It saves writing the same integration for every app and model. Treat third-party MCP servers like any dependency: review what they can access.

RAG means fetching relevant documents at question time and putting them in the prompt, so the model answers from your data rather than its training. Typically: split documents into chunks, embed them, store the vectors, and retrieve the closest chunks for each query.

Retrieval quality decides answer quality. Hybrid search (keywords plus vectors), re-ranking and citing the retrieved chunks all help; so does evaluating retrieval separately from generation.

An embedding model turns text (or images) into a vector of numbers so that similar meanings sit close together. Cosine similarity between vectors then measures relatedness.

They power semantic search, clustering, de-duplication and recommendations, and are far cheaper than chat models. Keep the same embedding model for indexing and querying.

When code consumes the answer, ask for JSON against a schema. Many APIs can enforce a schema or JSON mode; otherwise, validate and extract defensively and retry on failure.

Reasoning models may spend part of the output budget thinking before the JSON, so leave headroom in the maximum-tokens setting or answers get cut off.

An eval is a fixed set of inputs with a way to score outputs: exact checks, rules, or another model acting as a grader. Run it whenever you change a prompt, a model or retrieval settings.

Start small with twenty to fifty real cases, including the failures you've already seen. Without evals, every model upgrade is a guess.

Match the model to the job: small fast models for classification, routing and extraction; frontier models for hard reasoning and code; open-weight models when you need control or on-premises hosting.

Benchmarks are a starting point, not an answer. Try two or three candidates on your own eval set, compare cost per task, and design so swapping models is a configuration change.