Welcome to the NZRT Wiki Podcast. Today we’re looking at LLM Overview.

So, what exactly is a Large Language Model? At its core, it’s a neural network that’s been trained on enormous amounts of text. We’re talking billions of web pages, books, articles, code repositories — you name it. The model learns statistical patterns across all of that data, and what makes it interesting is that after training, you can point it at all kinds of language tasks without having to retrain it for each one specifically.

Let’s talk about how one of these models actually processes your input, because understanding the flow helps a lot. You start with whatever text you type in — your prompt. That text gets broken down into tokens, which are small chunks of text, sometimes whole words, sometimes just parts of words. Those tokens then pass through a series of transformer layers, which use a mechanism called self-attention. Self-attention is what lets the model relate any token to any other token in the input — so it can understand context across the whole passage, not just word by word in sequence. After that, the model predicts the next token, and then the next, and the next — building up its response one piece at a time until it’s done. That overall design — the transformer architecture — was introduced in a 2017 paper and is the foundation of every major LLM you’ll encounter today.

Now let’s look at some key properties you’ll see referenced when people talk about these models. There are six worth knowing. First, parameters — these are the learned weights inside the model, and they’re what the model actually “knows.” Sizes range from about one billion parameters on the smaller end up to over a trillion for the largest models. Second, the context window — this is the maximum amount of text the model can hold in mind at once, measured in tokens. Older models handled around eight thousand tokens; newer ones go up to two million or more. Third, temperature — this controls how random or creative the outputs are. Set it to zero and you get very consistent, deterministic responses. Push it higher and the outputs get more varied and creative. Fourth and fifth are top-p and top-k, which are sampling filters — they give you finer control over how diverse the model’s word choices are. And sixth, RLHF — Reinforcement Learning from Human Feedback — which is a technique used to tune the model so it follows instructions well and behaves in line with human preferences. There’s also quantization, which is a compression technique that reduces the size of model weights so the model can run more efficiently.

Training one of these models happens in phases. The first phase is pre-training, where the model learns next-token prediction across that massive corpus of text — web content, books, code, all of it. The second phase is supervised fine-tuning, where the model is trained on curated question-and-answer pairs to get better at following instructions. The third phase involves RLHF or a similar technique called RLAIF, where a reward model and optimisation process align the model’s outputs more closely with what humans actually want. And finally, the model gets evaluated against benchmarks — standardised tests like MMLU for general knowledge, HumanEval for code, HellaSwag for commonsense reasoning, and MATH for mathematical problem-solving.

When it comes to what these models can actually do, the range is broad. They can generate text across formats — prose, poetry, code, structured data. They can summarise long documents down to the key points. They can classify and categorise content, including sentiment analysis. They can extract structured fields out of unstructured text, answer questions based on context you give them, and work through multi-step reasoning problems. They can write, explain, refactor, and debug code. They handle translation between languages. And increasingly, they can use tools — meaning they can call external APIs and functions as part of completing a task.

But there are real limitations you should keep in mind. Hallucination is probably the most important one — models can generate text that sounds completely plausible but is factually wrong, and they won’t always flag it. Second, there’s a knowledge cutoff — the model only knows what was in its training data, so recent events won’t be reflected. Third, the context limit means you can’t just feed it an arbitrarily large document and expect it to handle the whole thing. Fourth, models have no persistent memory by default — every time you start a new conversation, the model starts fresh, unless you’re using retrieval tools or external memory systems. And finally, models can be quite sensitive to how you phrase things — a small change in your prompt can meaningfully shift the output.

Those limitations aren’t reasons to avoid LLMs, but they’re important to design around. Knowing where the edges are is half the battle.

That’s it for this episode of the NZRT Wiki Podcast. Thanks for listening.