LLMs, models and harnesses: the AI vocabulary

The AI world is full of jargon, and much of the confusion comes from terms used wrongly as synonyms: "model," "LLM," "AI," "neural network." To compare prices smartly and choose what to run, you first need to understand exactly what you're talking about. Here are the essential terms, from the most fundamental to the most technical.

What is a model?

A model is a trained neural network: concretely, a large file of weights (often several GB) that, given an input, predicts the most likely output. For a language model, the input is text and the output is the most probable next token. It's also called "weights," "checkpoint," or sometimes "parameters."

The model is the result of training: we showed the network billions of examples, and it adjusted its weights to minimize its prediction errors. Once trained, the model is frozen: it doesn't "relearn" during inference (except in special cases like adaptive models). This is important: a model has no memory between two conversations — each call is independent, except for what you send back in the prompt.

The LLM (Large Language Model)

An LLM is a large language model, trained on billions (sometimes trillions) of tokens of text. GPT-4, Claude, Gemini, Llama, Mistral, Qwen, DeepSeek are LLMs. The "L" for Large refers to the number of parameters (often billions) and the volume of training data.

Size is measured in parameters: a "7B" has 7 billion parameters, a "70B" has 70 billion, a "405B" has 405 billion. The more there are, the more capable the model is in theory, but the more memory it needs to run. That's why we talk so much about VRAM in the hardware guide.

Foundation model vs fine-tune

A foundation model (base model) is trained from scratch on a general corpus (web, books, code). It "knows" how to predict text, but it's not aligned to follow instructions: if you say "Write me a poem," it might reply "Write me a song" (it just continues the text). To make it useful, you retrain it on targeted data: that's fine-tuning.

• Foundation model (base): Llama-3-8B, Mistral-7B-v0.3. Raw, predictive, not aligned.
• Instruct / chat: retrained to follow instructions and hold a conversation. E.g. Llama-3-8B-Instruct, Mistral-Instruct. This is what you want 95% of the time.
• Specialized fine-tune: retrained on a domain (code, medical, legal, math). E.g. DeepSeek-Coder, CodeLlama.
• RLHF / DPO: the instruct model is then "aligned" with reinforcement learning from human feedback (RLHF) or direct preference optimization (DPO) so it's helpful, honest and harmless.

In practice, on an aggregator or locally, you almost always pick the "Instruct" or "Chat" version — unless you want to fine-tune yourself from the base.

Parameters and quantization

The number of parameters gives an idea of capacity, but the memory size also depends on precision (the number of bits per weight). An FP16 (16-bit) model weighs 2 bytes per parameter; in 4-bit, 0.5 byte. Quantization reduces precision to shrink the size and speed up inference, at the cost of a slight quality loss.

The GGUF format popularized by llama.cpp lets you store a quantized model in a single file, with different precisions (Q4_K_M, Q5_K_M, Q8_0…). For local use, Q4_K_M is the best compromise: quality nearly identical to FP16 for half the Q8 size.

Token, context and window

A token is the base unit an LLM manipulates — not a word, but a chunk of a word (about 0.75 words in English, ~1 token per 2-3 characters in French). API prices are "per million tokens," hence the importance of this site's comparator.

The context window is the amount of tokens the model can "see" at once: prompt + reply included. An 8k-token model processes ~6,000 words; a 128k model, ~96,000 words (a book). The bigger the window, the more the model can reason over long documents — but the more expensive (in tokens) and the more VRAM it needs (the "KV cache" grows with context).

What is a "harness"?

An evaluation harness is a software framework used to test a model reproducibly. It loads the model, runs it through a battery of standardized benchmarks (MMLU, HumanEval, GSM8K, HellaSwag, TruthfulQA…), and computes a score. The best known is EleutherAI's lm-evaluation-harness, used by Hugging Face and nearly every lab to compare models.

The harness doesn't create the model: it "harnesses" it to submit it to standardized tests, like a test bench in mechanics. Without a harness, you can't compare two models objectively — each would publish its own optimized metrics.

Common benchmarks measure distinct capabilities:

• MMLU: general knowledge (57 subjects).
• HumanEval: Python code generation.
• GSM8K: grade-school math problems.
• GPQA: PhD-level questions (guessing resistance).
• IFEval: ability to follow precise instructions.

Beware of pitfalls: contamination (benchmarks leak into training), overfitting on benchmarks, and the gap between benchmark score and real utility. A model can excel on MMLU and be frustrating in conversation. Benchmarks are useful for comparing, not for predicting user experience.

Modality: text, multimodal, vision

Some models process only text (Llama, Mistral, DeepSeek), others are multimodal: text + image (GPT-4o, Gemini, Qwen-VL, Llama-3.2-Vision), even text + image + audio + video. The modality indicates which input types the model accepts. Multimodal models are heavier (they also carry an image encoder, like ViT/CLIP) and often pricier on API.

• Text only: Llama-3, Mistral, DeepSeek, Qwen (text).
• Text + image: GPT-4o, Claude 3.5, Gemini, Qwen-VL, Llama-3.2-Vision.
• Text + audio: GPT-4o-audio, some Gemini models.
• Image generation: another model type (diffusion) — Stable Diffusion, FLUX, Midjourney. Not LLMs, but often compared.

Inference engines: llama.cpp, vLLM, Ollama, MLX

The model is a file; you need an inference engine to load it and generate text. The main ones:

• llama.cpp: the Swiss army knife, in C++, optimized for CPU and GPU, GGUF format. The base of almost the entire local ecosystem.
• Ollama: a friendly wrapper around llama.cpp, with a simple CLI and API. The easiest way to start locally.
• LM Studio: a GUI (desktop) on llama.cpp, for those who'd rather click than type commands.
• vLLM: high-performance server engine, optimized for production (PagedAttention, continuous batching). The choice for serving a model to multiple users.
• MLX: Apple's framework for the unified memory of M chips. Very efficient on Mac.
• TensorRT-LLM: NVIDIA's ultra-optimized engine, but complex to deploy.

In summary

• Model = the weights, the frozen file.
• LLM = a large language model (billions of parameters).
• Base vs instruct = raw vs aligned for instructions.
• Quantization = compress (FP16 → 4-bit) for less memory.
• Token / context = the unit of measure and the working memory.
• Harness = the tool that evaluates the model reproducibly.
• Engine = the software that runs the model (llama.cpp, vLLM…).

With this vocabulary, the price comparator on the home page becomes crystal clear: each row is a model (instruct, at a certain quantization), with a price per input and output token. To know which machine to run it on, check the guide on hardware.