Module 01How LLMs Work
§ workbook
Learning Outcomes
# Module 1: How LLMs Work — A Practical Model for Social Scientists
Estimated time: 2–3 hours
Type: Conceptual workbook
Prerequisites: None
What you'll produce: A one-page "AI capability map" for your research area
1. What Is a Large Language Model?
A large language model is a system trained on an enormous quantity of text — typically trillions of words drawn from the open internet, books, academic papers, code repositories, and other sources. Its training objective is simple: predict the next word in a sequence, given all the words that came before.
That is genuinely the whole mechanism. The model receives text as input and produces text as output, one piece at a time, each time asking: "Given everything I have seen so far, what is the most statistically plausible thing to come next?"
This sounds too simple to produce the fluent, knowledgeable-seeming output we observe. But three things make it work at scale:
- Scale. The model has seen more text than any human could read in a thousand lifetimes. The patterns it absorbs — grammatical structures, argumentative forms, academic conventions, factual associations — are drawn from that vast training corpus.
- Attention. The transformer architecture (Vaswani et al., 2017) gives the model a mechanism for weighting the importance of every previous piece of text when predicting the next one. This is why modern LLMs can maintain coherence over long passages: they can "attend" to a key term defined three pages ago when it becomes relevant again.
- Training, not programming. No one wrote rules for how to form a paragraph or what counts as a plausible academic citation. The model learned these patterns from the statistical structure of its training data.
What the model IS NOT
The model does not:
- Know things in the way a person knows them. It has no stable internal representation of facts that it can update when it learns new information.
- Check its own output against reality. The only test of a token is whether it is statistically plausible given the preceding text, not whether it is true.
- Understand the text it processes in any semantic sense. It models the distribution of language, not its meaning.
- Remember anything between conversations (unless the system around it provides memory). Each conversation starts fresh.
2. Tokens, Context, and Prediction
Tokens
LLMs do not process words as words. They break text into tokens — the smallest units the model can read or produce. A token might be:
- A whole common word: "the", "research"
- Part of a longer word: "method" → "meth", "od"
- A punctuation mark: ".", ",", "?"
- A special symbol marking the start or end of text
Most modern models use roughly 1.3 tokens per English word on average, though this varies by language and domain. A 500-word abstract is about 650 tokens.
The Context Window
The context window is the model's working memory — the maximum number of tokens it can "see" at once. Everything counts against this limit:
- Your instructions (the prompt)
- Any examples you provide
- The conversation history
- Any documents or data you include
- The model's own responses
- Outputs from tools the model calls
When the context window fills, earlier material drops out. This has practical consequences:
- A model cannot "remember" a detail from the beginning of a very long conversation.
- If you give it a 50-page document and ask about page 3, it may not have page 3 in its active context if the document was processed in chunks.
- Long conversations drift because the early constraints and instructions fall out of the window.
Context windows have grown dramatically — from 2,048 tokens (GPT-3, 2020) to 1 million tokens or more in current models (2025–26). But a larger window does not mean the model uses all of it equally well; attention tends to concentrate on the beginning and end of the context.
Next-Token Prediction in Practice
Here is what happens, step by step, when you type a prompt and receive a response:
- Your prompt is tokenized — broken into tokens.
- The model processes all tokens in its context window through its transformer layers.
- It produces a probability distribution over every possible next token.
- It selects one token according to a sampling strategy.
- That token is appended to the sequence.
- Steps 2–5 repeat until the model produces a special "stop" token or hits a length limit.
At no point in this process does the model pause to ask "is this true?" or "do I have evidence for this claim?" It is always and only predicting the next token.
3. Sampling: Why the Same Prompt Gives Different Answers
If the model always picked the single most probable next token, it would produce the same output every time — and that output would be repetitive and dull. Instead, models use sampling strategies that introduce controlled randomness:
| Strategy | What it does | Effect |
|---|---|---|
| Temperature | Scales the probability distribution. Low temperature (0.1) → peaked, deterministic. High temperature (1.5) → flattened, creative. | Controls the creativity–predictability trade-off. |
| Top-p (nucleus sampling) | Only considers tokens whose cumulative probability reaches p (e.g., 0.9). | Cuts off the long tail of very unlikely tokens. |
| Top-k | Only considers the k most probable tokens. | A simpler version of the same idea. |
Why this matters for research: The same prompt, run twice on the same model with the same settings, can produce different outputs. This is a feature (it allows variation and creativity), but it is also a reproducibility problem. If your classification label depends on which run you happened to get, your measurement is not reliable. We return to this in Module 4 (annotation) and Module 7 (validation).
4. What LLMs Can and Cannot Do
What they tend to do well
- Produce fluent, well-structured text in the languages and registers represented in their training data
- Summarise text by identifying and condensing key information
- Translate between languages, especially high-resource pairs
- Classify text according to explicit criteria, when the categories are clearly defined
- Generate code for common programming tasks, especially in widely-used languages
- Answer factual questions when the answer appears frequently in the training data
- Restructure or reformat information (e.g., "convert these notes into a paragraph")
What they tend to do poorly
- Produce factual claims about low-frequency or specialised knowledge without hallucination
- Reason about quantities, dates, or logical relationships that require computation rather than pattern matching
- Maintain consistency across very long outputs or multi-step reasoning chains
- Express appropriate uncertainty — they tend to sound confident regardless of whether they are correct
- Identify their own errors — they cannot check their output against reality
- Handle tasks requiring world knowledge that postdates their training cutoff
The hallucination problem
Hallucination — the production of plausible but false information — is not a separate failure mode. It is what the model always does: produce statistically plausible text. Sometimes that text corresponds to facts in the world; sometimes it does not. The model has no internal mechanism for distinguishing the two cases.
This is why every module in this course requires evidence from outside the model. You cannot trust the output by reading the output. You must check it against sources, against human judgment, against repeated runs, against alternative prompts — against something that is not the model itself.
5. The Vocabulary of AI Systems
People use "AI," "LLM," "ChatGPT," and "language model" interchangeably, but these are different things. Here is the vocabulary we will use throughout the course:
Model
The trained neural network — the thing that takes tokens in and produces tokens out. GPT-4, Claude 4, Llama 3, and Qwen are models. A model has no interface, no memory, and no tools. It is a pure text-in/text-out function.
Interface
The application wrapped around a model. ChatGPT, Claude.ai, and Gemini are interfaces. The interface may add features the model does not have:
- Conversation history (memory across turns)
- File upload and processing
- Web search capability
- Image generation or analysis
- Custom instructions or system prompts
When you use an interface, you are using the model plus whatever the interface provides. The model itself does not "know" it can search the web — the interface intercepts certain outputs, runs a search, and feeds the results back into the context window.
System Prompt / Custom Instructions
Text provided to the model before the conversation that sets its behaviour, constraints, and available tools. In most interfaces, this is invisible to the user. Some interfaces let you edit part of it (often called "custom instructions" or "project instructions").
Tool Use / Function Calling
The ability of a system to call external tools: a calculator, a web search, a code interpreter, or a file system. The model does not execute the tool — it produces a structured request that the surrounding system executes. The result is then fed back into the context window as if the model had produced it.
Retrieval-Augmented Generation (RAG)
A pattern where the system searches for relevant documents before generating a response. The retrieved documents are placed in the context window, and the model generates its response with those documents "in view." This grounds the output in specific sources — but it does not guarantee the model will use those sources faithfully.
Agent
A system combining an LLM with tools, memory, and a loop: the model decides what to do, the system executes it, the result enters the context, and the model decides the next action. Agents can operate at different levels of autonomy:
- Assisted: The agent proposes an action but asks before executing it.
- Agent: The agent acts without asking but reports what it did.
- Unattended: The agent runs without a human in the loop at all.
We cover agents in depth in Module 6.
Why the distinctions matter
When someone says "I used ChatGPT to analyse my data," it is unclear what happened. Did they paste data into a chat window? Use the code interpreter? Run an agent? The model, the interface, and the tools matter for reproducibility and validity. Being precise about what was used is the first step toward a credible disclosure.
6. The Course Lens: Four Questions
Every AI use you evaluate in this course — whether a demonstration, an exercise, or your own experiment — will be examined through four questions. Learn them now; you will use them in every module.
1. What is being delegated?
Name the task and the decision separately. "Summarise these papers" is a task delegation. "Decide which papers are relevant" is a decision delegation. They have different risk profiles and require different evidence.
2. What would make the result believable?
Name evidence outside the model output. A verified source. A human comparison. A repeated run. A test on a known case. The model's confidence is not evidence.
3. What could be lost or flattened?
Look for variation, minority cases, ambiguity, provenance, and tacit knowledge. What does the non-AI version preserve that the AI version might erase?
4. What must remain visible or human?
Name the decision, check, or record a reviewer must be able to inspect. If the answer is "nothing — the model can handle everything," you have probably delegated a decision you should not have.
modules/shared/four_questions.md — keep it open as a reference throughout the course.7. Demonstration: Same Task, Different Runs
Let's make this concrete. Here is a simple research task:
I will run this prompt on the same abstract three times, on the same model, with the same settings. The only thing that varies is the sampling — the random seed that determines which tokens are selected at each step.
Abstract (from a real published paper):
Run 1: quantitative
Run 2: quantitative
Run 3: quantitative
In this case, the model was consistent — but it could have been otherwise. Try this yourself with a more ambiguous abstract, or a more nuanced classification task, and you may see variation across runs. That variation is not random noise to be ignored — it is evidence about the reliability of the measurement.
Exercise: Before reading the next section, take a research task you know well. Write the prompt you would use to ask an LLM to help with it. Then answer the four questions for that task — before you run anything.
8. Your AI Capability Map
The final exercise for this module is to produce a one-page AI capability map for your research area. This is a living document you will update as you work through the course. For now, complete the first version:
Capability Map Template
```
AI Capability Map: [Your Research Area]
- Tasks AI could plausibly assist with:
- [List 3-5 specific tasks: writing, searching, classifying, coding, etc.]
- Tasks AI should probably not touch:
- [List tasks where delegation would be inappropriate: final interpretive judgments,
ethics decisions, confidential data handling, etc.]
- What I would need to verify before trusting AI output on #1:
- [For each task: what external evidence would make the result believable?]
- What could be lost if AI handled #1:
- [Variation, minority perspectives, tacit knowledge, provenance, etc.]
- My current baseline (how I do these tasks now):
- [Describe your current workflow briefly]
- One task I want to experiment with during this course:
- [Pick one bounded task from #1 to use across modules]
```
Keep this map. You will revisit it in Module 7 when you design your validation plan, and in Module 8 when you build your portfolio prototype.
9. Summary
- LLMs predict the next token. That is the whole mechanism. Fluency is not truth.
- Tokens are the unit of processing; the context window is the model's working memory.
- Sampling introduces controlled randomness. The same prompt can produce different outputs.
- Models, interfaces, tools, and agents are different things. Be precise about what you used.
- The four questions are your lens for every AI use in this course: delegate, verify, watch for loss, keep visible.
- Your capability map is your starting point. Update it as you learn.
What's Next
Module 2 applies the concepts from this module to writing tasks: drafting, editing, style transfer, and criteria-based evaluation. You will use the four questions on real writing examples.
If you want to go deeper on the technical side before moving on:
- Read the optional deep-dive in
slides.md(a content-first slide script covering the transformer architecture, training, and the RLHF alignment process) - Browse the annotated reading list in
readings.md
Module 1 of 8 · AI as a Research Tool: Workflow Design for the Social Sciences · University of Zurich
§ exercises
# Module 1: Exercises
Each exercise includes an answer key. Work through the exercise before reading the key. The point is to form a judgment before seeing a model's output or a prepared answer — this is the "human first" principle you will use throughout the course.
Exercise 1.1: Predict the Token
Goal: Experience next-token prediction from the model's perspective.
Below is the beginning of a sentence. Without overthinking it, write the single most likely next word:
Your answer: _______________
Now read these alternative continuations. Which do you think a model trained on social-science text would assign the highest probability?
(a) "...political polarisation."
(b) "...happiness."
(c) "...the amount of time spent outdoors."
(d) "...a statistically significant coefficient."
The model assigns probabilities based on patterns in its training data. In a corpus heavy with social-science abstracts, "political polarisation" and "a statistically significant coefficient" would both receive high probability — they are common completions in that genre. "Happiness" would be plausible but less specific. "The amount of time spent outdoors" would be unlikely because it breaks the expected register and topic.
The point is not which answer is "correct" — there is no correct answer. The model is not retrieving a fact. It is making a statistical bet about what typically follows the words "positively associated with" in academic text. When the bet is good, the output looks knowledgeable. When it is bad, the output looks like hallucination. But the mechanism is the same either way.
Exercise 1.2: Model, Interface, or Agent?
Goal: Practice distinguishing components of AI systems.
Classify each scenario as primarily involving a model, an interface, a tool, or an agent:
- You type a prompt into a chat window and receive a response.
- The system searches the web, retrieves three articles, and incorporates them into its answer.
- The system reads a CSV file, writes Python code to analyse it, executes the code, and reports the results.
- You download a file containing the trained neural network weights and run it on your own computer.
- The system, without asking you, reads your Zotero library, identifies papers on a topic, drafts a literature review, and saves it to your drive.
Your answers:
- _______________
- _______________
- _______________
- _______________
- _______________
- Interface — You are using a chat interface (like ChatGPT or Claude.ai) built around an underlying model. The interface handles the conversation history and display; the model handles the text generation.
- Interface + Tool — The interface provides web search as a tool. The model does not "know" how to search the web; the interface intercepts, executes the search, and feeds results back into the context window. This is a Retrieval-Augmented Generation (RAG) pattern.
- Interface + Tool (code interpreter) — The interface provides a code execution environment. The model writes code, the interface executes it, and the results re-enter the context. This is tool use, not full agency — the system is following a single instruction, not running an open-ended loop.
- Model — You have the raw model (the trained weights). You still need software to run it (like LM Studio, Ollama, or vLLM), but what you downloaded is the model itself, not an interface or agent.
- Agent — The system is making multiple decisions without asking: what to read, what to include, what to write, where to save. It has tools (Zotero access, file writing) and is operating autonomously. This is the "unattended" end of the agent spectrum.
Exercise 1.3: The Four Questions on a Real Task
Goal: Apply the course lens to a concrete task before running anything.
Choose a research task you actually do — or might do. It can be anything: drafting a paragraph, searching for literature, classifying text, analysing data. Write it down:
Your task: _______________
Now answer the four questions for this task:
1. What is being delegated?
- Task: _______________
- Decision: _______________
2. What would make the result believable?
- Evidence outside the model: _______________
3. What could be lost or flattened?
- _______________
4. What must remain visible or human?
- _______________
There is no single correct answer — the four questions are a lens, not a checklist with right answers. But here is an example of what a good response looks like:
Task: Classify 500 news articles as covering "economic policy" or not, for a dataset I am building.
1. What is being delegated?
- Task: Reading each article and assigning a binary label.
- Decision: The classification itself — the model decides which articles count as economic policy coverage. I am also delegating the interpretation of my category definition, since the prompt is my operationalisation.
2. What would make the result believable?
- Double-code a random sample of 100 articles myself and check agreement with the model.
- Test the prompt on 10 articles I already know the answer to.
- Check whether errors are concentrated in a particular type of article (e.g., articles that mention economics only in passing).
3. What could be lost or flattened?
- Ambiguous articles that partially discuss economic policy might be forced into one category.
- The model might miss economic policy discussions that use non-standard terminology.
- My own tacit knowledge about what "counts" as economic policy in my theoretical framework might not be captured in the prompt.
4. What must remain visible or human?
- The codebook (category definitions and decision rules) must be written by me, not the model.
- The final dataset must record which articles were model-labelled and which were human-checked.
- Any article where the model's label disagrees with mine must be resolved by me, not by accepting the model's label.
Exercise 1.4: Hallucination Hunt
Goal: Experience hallucination directly and understand why it happens.
Ask any LLM (whatever chat interface you have access to) the following question:
Write down what the model returns:
Model's response: _______________
Now try to find the paper. Search for the journal, the DOI, the author. Does it exist?
Does it exist? _______________
If the paper does not exist: the model produced a plausible-looking reference — right journal name format, plausible author name, plausible title, plausible volume and page numbers — that is entirely fabricated. Why?
The model produced the reference the same way it produces everything else: by predicting tokens that are statistically plausible given the prompt. The prompt asked for a reference in a specific format, with a specific journal name and a plausible-sounding title. The model's training data contains millions of real references in exactly this format. So it produces something that looks like a reference, with components that go together plausibly — and it has no mechanism to check whether the specific paper exists.
This is why the course rule on citations is absolute: every reference you submit must have been personally verified. A model cannot tell you whether a paper exists. It can only tell you what a paper that exists would probably look like.
If the paper did exist: you got lucky — or the model's training data happened to include that specific reference. But the mechanism is the same. The model was not "retrieving" the reference from a database. It was predicting tokens, and those tokens happened to correspond to a real paper. You cannot tell the difference from the output alone.
Exercise 1.5: Build Your Capability Map
Goal: Create the starting document for your course-long portfolio.
Using the template from Section 8 of the workbook, create your AI Capability Map. Write it in a text file, a note-taking app, or on paper. You will return to it in Modules 7 and 8.
Keep it somewhere you can find it. This is not submitted — it is your working document.
Module 1 Exercises · AI as a Research Tool
§ readings
Nothing on this list is required before starting the module. Read in any order — before, during, or after. Each entry explains why it is useful and what to take from it.
---
Core Readings
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). "Attention Is All You Need." Advances in Neural Information Processing Systems, 30.
Why read it: This is the paper that introduced the transformer architecture — the foundation of every modern LLM. You do not need to understand the mathematics. Read the abstract, the introduction, and Section 3 (Model Architecture) for the diagrams. The key insight is that "attention" lets the model weight the importance of every previous token when predicting the next one, which is what enables coherent long-form generation. What to take: The transformer is an architecture for processing sequences. Its core innovation — attention — is why modern LLMs can maintain coherence over long passages. Everything else (scale, training data, RLHF) is built on this foundation. ---
Shanahan, M. (2024). "Talking About Large Language Models." Communications of the ACM, 67(2), 68–79.
Why read it: The best short introduction to what LLMs are and — critically — what they are not. Shanahan argues that anthropomorphic language ("the model knows," "the model thinks") leads researchers and users astray. He offers precise alternative language: models are "sequence predictors," not knowledge bases or reasoning engines. What to take: The vocabulary you use shapes what you expect. Saying a model "hallucinated" implies it normally tells the truth and occasionally slips — but the model is always producing statistically plausible text. Some of it corresponds to facts; some does not. The model does not know which is which. ---
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency.
Why read it: The paper that popularised the term "stochastic parrot" — the argument that LLMs do not understand language but rather probabilistically reassemble patterns from their training data. Whether you agree with the full argument or not, the paper forces you to articulate what you believe the model is doing when it produces plausible output. What to take: The central question — "is the model understanding, or is it pattern-matching at scale?" — is not merely philosophical. It has practical consequences for when and how you trust the output. If the model is a stochastic parrot, every factual claim requires external verification. If you believe it does something more, you still need to specify what that "something more" is and how you would test for it. ---
Grimmer, J., Roberts, M. E., & Stewart, B. M. (2022). Text as Data: A New Framework for Machine Learning and the Social Sciences. Princeton University Press. Chapters 1–2.
Why read it: The standard social-science introduction to computational text analysis. Chapters 1–2 establish the conceptual framework: text as data, representation, and the relationship between human judgment and automated methods. This framework predates LLMs but applies directly to them. The distinction between discovery, measurement, and inference is one you will use throughout the course. What to take: The methodological questions Grimmer, Roberts, and Stewart raise about automated text analysis — what is being measured? how is it validated? what are the consequential errors? — are the same questions the course lens asks about LLMs. The technology changed; the methodological demands did not. ---
Becker, J., Rush, N., Barnes, E., & Rein, D. (2025). "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity." arXiv:2507.09089.
Why read it: This paper is about programmers, not social scientists. Read it for the self-assessment gap: developers who used AI reported feeling more productive, but objective measures showed smaller and less consistent gains. The gap between perceived and actual improvement is a pattern that appears across AI application domains — and it is one of the reasons this course requires evidence from outside the model. What to take: The abstract and the discussion of the self-assessment gap (not the full methods). The key finding: your feeling that AI helped is not evidence that it did. You need an external check. ---
Optional Deep Dives
Brown, T. B., Mann, B., Ryder, N., et al. (2020). "Language Models are Few-Shot Learners." Advances in Neural Information Processing Systems, 33.
Why read it: The GPT-3 paper. Read the introduction and the sections on limitations (Section 8). The paper that established the scaling paradigm — bigger models, more data, emergent capabilities — and also documented the failure modes that remain: hallucination, inconsistency, bias, and the inability to distinguish truth from plausibility. ---
Hicks, C. (2025). "Learning Principles" (PRINCIPLES.md). github.com/DrCatHicks/learning-opportunities (CC-BY-4.0).
Why read it: A short, practical guide to learning effectively when using AI tools. Written about programmers; read it about yourself. The principles — predict before seeing, explain to someone else, vary the task, test on new cases — are the pedagogical foundation of this course's "human first" approach. ---
Kapoor, S., & Narayanan, A. (2023). "Leakage and the Reproducibility Crisis in Machine-Learning-Based Science." Patterns, 4(9).
Why read it: A clear explanation of how data leakage — when information from the test set bleeds into training — can inflate performance claims. The relevance to LLMs: if a model was trained on a corpus that includes the papers you are asking it to classify or the facts you are asking it to retrieve, its performance may look better than it would on truly unseen material. Knowing what was in the training data is often impossible, which makes independent verification essential. ---
Read Later (Across Modules)
These readings are listed here because they span multiple modules. You will see them referenced again:
- *Bail, C. A. (2024). "Can Generative AI Improve Social Science?" PNAS, 121(21). — A broad argument for AI's potential in social science. Read with Grossmann et al. (2023) for the counterpoint. Revisit in Modules 7 and 8. - Grossmann, I., et al. (2023). "AI and the Transformation of Social Science Research." Science, 380(6650). — A more cautious assessment. Read with Bail (2024) for the debate. Revisit in Modules 7 and 8. - Doshi, A. R., & Hauser, O. P. (2024). "Generative AI Enhances Individual Creativity but Reduces the Collective Diversity of Novel Content." Science Advances, 10(28). — AI helps individuals, but when everyone uses it, outputs converge. A pattern to watch for in your own experiments. --- Module 1 Reading List · AI as a Research Tool*