Module 02Writing with AI

Learning Outcomes

# Module 2: Writing with AI — Drafting, Editing, and Style

Estimated time: 2.5–3.5 hours

Type: Conceptual + practical (markdown workbook + exercises)

Prerequisites: Module 1 (How LLMs Work)

What you'll produce: A style guide, a worked editing example with disclosure, and a documented writing workflow

1. AI as a Writing Assistant: What It Does Well and Poorly

The promise

Writing is one of the most immediately appealing use cases for LLMs. Paste in a rough paragraph, ask for a revision, and the model returns something smoother, clearer, and better-structured — often in seconds. For social scientists who write constantly — seminar papers, thesis chapters, grant applications, journal articles — this feels like a breakthrough.

The model can:

• Restructure a paragraph or section to improve logical flow
• Condense a long passage into a concise abstract or summary
• Rephrase awkward or unclear sentences while preserving meaning
• Translate between registers: turn notes into prose, jargon into plain language, or a dense paragraph into bullet points
• Flag gaps, contradictions, or unsupported claims in a draft
• Generate alternatives — five ways to open a paragraph, three possible conclusions — that you can select from

The risk

Everything you learned in Module 1 applies with full force to writing. The model does not know what your paper is trying to argue. It does not know which claims your evidence supports and which it does not. It does not know your voice, your disciplinary conventions, or your audience's expectations — unless you tell it.

When those things are absent from your prompt, the model supplies defaults. The defaults are statistically plausible given the model's training data. They are not your argument. They are not your evidence. They are not your voice.

The most common failure pattern in AI-assisted writing is not grammatical error or nonsense. It is plausible rewriting that silently changes the claim. A cautious "may be associated with" becomes a confident "is associated with." An empirical puzzle becomes a recommendation. A limitation disappears. The prose improves; the argument drifts.

2. Criteria-Based Evaluation: The Baseline Rule

Define "good" before you see the output

The single most important rule in this module — and one you will return to throughout the course — is:

If you read the model's output first and then ask "is this good?", you are judging the output against an impression formed after seeing it. You will be influenced by fluency, by the specific choices the model made, by whether it "feels right." That is the opposite of evaluation.

Instead, define concretely what "good" means for this specific piece of writing:

| Criterion | How would you check it? |

|---|---|

| The argument is the one I intended | Compare the claim sentence against the claim sentence in my outline or author brief |

| Every factual statement is supported | Trace each claim to a source I have open; mark any that lack support |

| Uncertainty is preserved where appropriate | Count hedging language (may, suggests, possibly) — has any been removed? |

| The voice is mine, not generic academic | Read aloud; does it sound like something I would say? |

| The structure follows my outline | Check section-by-section against the outline I wrote beforehand |

| No claim is stronger than my evidence permits | Underline every causal or comparative claim; check scope against what the evidence actually shows |

These are not universal criteria. They are criteria for a specific task, written by a specific researcher, before the model runs. A different task — say, condensing a methods section for a lay summary — would have different criteria: clarity for a non-specialist, removal of jargon, preservation of the core finding.

The human-first workflow

Every writing exercise in this module follows the same pattern:

1. Define the task — What exactly do you want the model to do? Draft? Revise? Summarise? Restructure?
2. Write the criteria — What would make the result good for your purpose? Be specific.
3. Write an author brief (if relevant) — What is your purpose, intended claim, evidence boundary, and voice?
4. Run the model
5. Evaluate against your criteria — Not against your impression. Against the list you wrote at step 2.
6. Revise or discard — If it fails a criterion, either fix it or throw it out. Never accept output that fails your criteria just because it reads well.

Steps 1–3 happen before the model sees anything. This is the "human first" principle from Module 1, applied to writing.

3. Style Transfer: Academic, Plain-Language, and Personal Voice

What style transfer is

Style transfer means asking the model to rewrite text in a different register while preserving the content: turn a journal abstract into a blog post, or meeting notes into a formal paragraph, or a dense methodology section into a plain-language summary for participants.

The model can do this because it has seen millions of examples of each register in its training data. It has absorbed the statistical patterns of academic prose, journalistic prose, legal prose, and everyday speech.

What style transfer is not

Style transfer is not neutral. The model's idea of "academic style" is the average of the academic prose in its training data — which skews toward certain disciplines, certain decades, and certain rhetorical conventions. Its idea of "plain language" may flatten nuance or remove hedging that you need. Its idea of your "personal voice" — unless you supply examples — is a guess.

Deriving a style guide from examples

The most reliable way to get the model to write in a specific style is to show it examples and ask it to extract the patterns. This is what Exercise 2.3 asks you to do.

A style guide derived from examples might include:

• Sentence length and structure: Short declarative sentences? Complex nesting? Varied?
• Vocabulary register: Technical terminology defined on first use? Jargon avoided? Specific verbs over general ones?
• Voice and stance: First person ("I argue") or third person ("this paper argues")? Active or passive? Cautious ("suggests") or declarative ("demonstrates")?
• Paragraph structure: Topic sentence → evidence → implication? Or narrative flow?
• Citation conventions: Integrated into prose or parenthetical? How are authors introduced?
• Treatment of uncertainty: Explicitly flagged? Confined to a limitations section? Absent?

The model can extract these patterns from examples far more reliably than it can guess them from a one-line instruction like "write in my style."

Demonstration: Three styles from the same content

Here is a single claim, rendered three ways:

Original (rough notes):

Style 1: Academic prose (journal article)

Style 2: Plain-language summary (for study participants)

Style 3: Policy brief

The same content; three different audiences, three different sets of conventions. The model can do all three — but you must tell it which one you want, and preferably show it examples.

4. The Four Questions Applied to Writing Tasks

Apply the course lens to any AI-assisted writing task before you run the model. Here is what the questions look like for writing specifically:

1. What is being delegated?

Writing tasks and writing decisions are different things:

| Task (low-risk delegation) | Decision (high-risk delegation) |

|---|---|

| "Reorganise these paragraphs into a logical sequence" | "Decide what the logical sequence should be" |

| "Suggest five alternative openings for this section" | "Choose the best opening" |

| "Flag claims that lack supporting evidence" | "Decide which claims to keep" |

| "Convert this paragraph from passive to active voice" | "Decide whether active voice is appropriate here" |

| "Summarise Section 3 in three sentences" | "Decide which findings from Section 3 are worth including in the abstract" |

The task/decision distinction from Module 1 is especially sharp in writing. The model can execute a writing task — restructure, rephrase, condense. But deciding what the argument is, what evidence supports it, and what voice is appropriate: those are methodological judgments. They belong to you.

2. What would make the result believable?

For writing tasks, "believable" usually means "preserves my argument, evidence, and voice." Evidence comes from comparison:

• Compare the model's output against your outline, author brief, or criteria document
• Compare the original paragraph and the revised paragraph side by side; mark every change to the claim
• Have a colleague read both versions without knowing which is which and ask which argument is clearer
• Check every factual claim that appears in the output against its source (if the model added anything, it must be verified or removed)

3. What could be lost or flattened?

Writing with AI risks losing:

• Your voice. The model's default academic register is an average. It sounds like everyone. If you do not supply a voice specification, you get the generic.
• Uncertainty. Models tend to strengthen claims — hedging becomes declarative, "may suggest" becomes "shows." This is a direct consequence of the training data: academic prose that reaches strong conclusions is overrepresented relative to academic prose that says "we don't know."
• Specificity. The model may replace your precise but awkward phrasing with smoother but vaguer alternatives. "The coefficient on the interaction term between trust and network size was negative and significant at p < 0.01" becomes "trust and network size interact negatively" — smoother, less specific, and no longer capable of being evaluated against your results.
• Argument structure. The model may restructure your paragraphs into a more "standard" flow that does not match the logic of your argument. Some arguments need to start with the counterclaim, or build tension before resolving it. The model's idea of good structure is the average of published papers, not your rhetorical strategy.
• Citations and attribution. The model may drop citations, move them, or — if you are not careful — fabricate them. Every citation in AI-assisted text must be verified against the source.

4. What must remain visible or human?

For writing, the non-negotiable human elements are:

• The argument itself. You decide what claim your paper makes. The model can help you express it; it cannot decide it.
• The relationship between evidence and claims. You decide what your evidence supports and what it does not. The model cannot weigh your data.
• Every citation. You must personally verify that every reference exists, says what you claim it says, and is appropriate to cite.
• The disclosure. You must record what tool you used, for what purpose, where in the work, what you verified, and what you decided. This is visible to readers, reviewers, and — eventually — to yourself when you return to the work months later.

5. The Disclosure Paragraph

Every portfolio component and every exercise that produces AI-assisted output requires a disclosure statement. The template — introduced here and used throughout the course — follows the format expected by the American Journal of Political Science and American Political Science Review, adapted for course use.

A complete disclosure states:

1. What tool(s) were used — Model, interface, date(s)
2. For what purpose — Be specific: "to suggest restructuring of the literature review" not "for writing"
3. Where in the work — Point to the specific section or passage
4. What was verified and how — What checks did you perform? With what evidence?
5. What the human decided — What judgments did you retain?

The disclosure must include itself. If you used AI to help write the disclosure, say so.

A disclosure is not a confession and not a liability disclaimer. It is a record of what happened, written so that a reviewer — or your future self — can understand what the model contributed and what you decided. Good disclosure is a research practice, not a bureaucratic requirement.

6. Demonstration: Before and After

Let's work through a complete example. I will show you a paragraph, run an editing task without and with an author brief, and evaluate both outputs against criteria defined in advance.

The paragraph

Teaching status: This paragraph is constructed for demonstration. It is not a verified summary of published research. (The same paragraph appears in Exercise 2.2 so you can work through it yourself.)

Step 1: Define the task

Step 2: Write evaluation criteria (before seeing output)

| Criterion | Check |

|---|---|

| The core empirical tension is preserved (contradictory effects) | The opening sentence must state that moderation can have competing effects, not that one effect dominates |

| No policy recommendation appears without evidence | The final sentence must not assert that stronger moderation is "best" or "most effective" |

| Evidentiary gaps are marked, not filled | Where the paragraph makes claims without cited support, the revision should note the gap, not supply plausibly-sounding evidence |

| The structure flows from puzzle → mechanisms → open question | The paragraph should not resolve into a single clear recommendation |

| No fabricated citations | Any reference appearing in the output must be traceable to a real source |

Step 3: Write the author brief

• Purpose: Introduce an empirical puzzle about heterogeneous effects of content moderation on political participation.
• Intended claim: Moderation may protect some forms of participation while disadvantaging others.
• Not intended: This paragraph does not establish that stronger moderation is preferable overall.
• Evidence boundary: No sources have been supplied. Do not add empirical claims or citations; mark claims needing evidence.
• Voice: Analytic and cautious. Preserve uncertainty.
• Permitted assistance: Improve organisation and clarity; flag any substantive decision rather than making it silently.

Step 4: Run A — without author brief

The model receives only the paragraph and the editing request.

Typical output (Run A):

Step 5: Apply criteria to Run A

| Criterion | Pass? | Evidence |

|---|---|---|

| Core tension preserved | ⚠️ Partial | "Competing effects" is preserved, but the opening frames it as a resolved tension |

| No unsupported policy recommendation | ❌ Fail | "Stronger moderation offers the most effective way to protect participation" — this is stronger than the original and has no evidence |

| Evidentiary gaps marked | ❌ Fail | "Evidence suggests," "disproportionately disadvantage" — no gaps marked, confidence added |

| Structure flows puzzle → open question | ❌ Fail | Paragraph resolves to a single recommendation |

| No fabricated citations | ✅ Pass | No citations invented |

Verdict: The prose improved. The argument got worse — it became more confident and less accurate.

Step 6: Run B — with author brief

The model receives the paragraph, the editing request, and the author brief.

Typical output (Run B):

Step 7: Apply criteria to Run B

| Criterion | Pass? | Evidence |

|---|---|---|

| Core tension preserved | ✅ Pass | "Competing ways" preserved; puzzle framed explicitly |

| No unsupported policy recommendation | ✅ Pass | Final sentence explicitly states the paragraph does not establish the recommendation |

| Evidentiary gaps marked | ✅ Pass | [EVIDENCE NEEDED] markers at unsupported claims |

| Structure flows puzzle → open question | ✅ Pass | Opens with tension, closes with open question |

| No fabricated citations | ✅ Pass | No citations added |

Verdict: Run B is less smooth than Run A. It has bracketed interruptions. It does not resolve neatly. It is also more accurate to the author's actual position, more honest about what the evidence supports, and more useful as a piece of academic writing.

What this demonstrates

1. The author brief is not a "better prompt" in the sense of being longer. It is a specification of what the author actually intends, which the model cannot guess. Without it, the model defaults to the statistical centre of its training data — which, for academic text, tends toward confident declarative claims.

1. Better prose and better argument are different axes. Run A reads more smoothly. Run B is more accurate. In academic writing, accuracy is the binding constraint. Smoothness is secondary.

1. You cannot evaluate output by reading the output. You need something outside the text — your criteria, your author brief, your evidence — to check it against.

7. A Failure Case: When Style Transfer Goes Wrong

Style transfer is especially prone to silent argument change. Here is an example:

Original (from a real working paper):

Request: "Rewrite this for a general audience."

Model output (failure case):

What was lost:

• The effect sizes disappeared (0.34 and 0.07).
• The significance levels disappeared.
• The small-n caution disappeared.
• The interaction term — the actual test of whether the subgroups differ — disappeared.
• "Indistinguishable from zero" became "didn't help" — which is a different claim (no evidence of effect ≠ evidence of no effect).
• The entire statistical apparatus that lets a reader evaluate the claim was replaced by a story.

Better plain-language version:

The second version is accessible but preserves the evidence structure. The difference is not in the prompt wording — it is in whether you, the author, check the output against your criteria and refuse to accept a version that drops the evidence.

8. Exercise Instructions

Complete all five exercises in exercises.md. Each takes approximately 20–30 minutes. Work through them in order — they build on each other.

You will need: A chat interface (any LLM), this workbook, and a piece of your own academic writing you are authorised to use (a paragraph from a seminar paper, a thesis draft, or a working document — do not use co-authored work without permission, and do not upload confidential or unpublished data).

| Exercise | What you'll do | Key skill |

|---|---|---|

| 2.1 | Define criteria for "good" writing before seeing any AI output | Criteria-first evaluation |

| 2.2 | Draft with AI, evaluate against criteria, revise | The human-first editing loop |

| 2.3 | Derive a style guide from three examples, test it on new text | Style specification from evidence |

| 2.4 | Write a disclosure for an AI-assisted writing task | Transparent record-keeping |

| 2.5 | Apply the four questions to a writing workflow | Systematic delegation analysis |

Before starting the exercises, re-read Sections 2 (criteria-based evaluation) and 4 (four questions applied to writing) of this workbook. Keep the disclosure template (modules/shared/disclosure_template.md) and the four questions (modules/shared/four_questions.md) open.

Save everything you produce. The style guide from Exercise 2.3, the disclosure from Exercise 2.4, and the four-questions analysis from Exercise 2.5 will feed into your portfolio.

9. Summary

• AI is a capable writing assistant for drafting, editing, restructuring, and style transfer — but it cannot supply your argument, your evidence, or your voice.
• Define evaluation criteria before seeing AI output. This is the baseline rule of AI-assisted writing. Fluency is not accuracy.
• An author brief — purpose, intended claim, evidence boundary, voice, permitted assistance — prevents the model from filling missing specifications with its own defaults.
• Style transfer requires specification. Show the model examples; do not assume it knows what "academic style" or "plain language" means for your purpose.
• The four questions apply to writing as they apply to every AI use: what is being delegated (task vs. decision), what would make it believable, what could be lost or flattened, what must remain human.
• Disclosure is not optional. Every AI-assisted output must record what tool was used, for what purpose, where, what was verified, and what the human decided.

What's Next

Module 3 moves from writing to literature: searching, screening, and synthesising with AI assistance. You will use the criteria-first approach from this module to evaluate search results and the style-transfer skills to write source-bounded syntheses. The four questions remain your lens; the disclosure template remains your record.