← course index

SharedResources

glossary four questions validation menu disclosure template

§ glossary

# Glossary: AI as a Research Tool

This glossary defines key terms as they are used across all eight modules. Each entry is written for a social-science audience with no assumed programming background. Terms marked (M1) through (M8) indicate the module where the concept is first introduced in depth.

This glossary is a living reference. Terms are introduced gradually across modules; the module label (M1–M8) indicates where each concept receives its first full treatment.

§ the four questions

# The Four Questions: A Course Lens

These four questions are the core analytical framework used across all eight modules. Apply them to every AI use you evaluate, whether it is your own experiment, a demonstration, or a published study.

Using the Four Questions

Before using AI on a task: Answer all four questions on paper. This is your prediction and your safety boundary.

After using AI on a task: Answer them again. What was different? What did you learn?

When reading research that uses AI: Apply the questions to the published description. Can you answer them from what the paper reports? If not, the disclosure is incomplete.

In your portfolio: Each of the four questions should have a concrete answer, supported by evidence, for the AI use you evaluated.

The four questions are introduced in Module 1 and used throughout every module. They replace the nine-column tool audit and special workflow notation from earlier versions of this course.

§ validation menu

# The Validation Menu: Seven Checks

This menu is introduced in Module 4 and used throughout Modules 5–8. For any AI-assisted result, choose the checks that are appropriate to the claim. No single check is sufficient for every claim, and running every check is usually wasteful. Select the ones that could actually change your interpretation.

How to Choose Checks

  1. Start with the claim. What are you asserting? "This search found all relevant papers." "These labels are accurate." "This summary is faithful to the sources."
  1. Ask what would falsify the claim. What kind of evidence would show you were wrong?
  1. Pick the checks that would detect that falsification. If fabricated references would falsify your claim, run check 1. If label inconsistency would falsify it, run checks 2 and 3. If the conclusion depends on a specific prompt wording, run check 4.
  1. One check is better than none; two well-chosen checks are better than five run superficially.

Example: Literature Search Claim

Claim: "This search identified all relevant papers on AI-assisted classification in political science published since 2020."

Falsification: A known relevant paper is missing; an irrelevant paper is included; the date or field filter excluded something it should not have.

Checks to run:

  • Check 1 (source verification): Verify that each returned paper exists and is about the claimed topic.
  • Check 2 (human comparison): Have a colleague suggest three papers they would expect to find. Are they in the results?
  • Check 5 (boundary cases): Test whether the search returns papers from adjacent fields (sociology, computational linguistics) that use the same methods.

Example: Annotation Claim

Claim: "The LLM-labelled dataset can be used in a regression as if it were human-coded."

Falsification: The regression coefficient changes sign or significance when human labels replace LLM labels; errors are concentrated in one category.

Checks to run:

  • Check 2 (human comparison): Double-code a random sample and compute agreement and disagreement patterns.
  • Check 7 (downstream consequence): Run the regression with human labels and with LLM labels on the same cases. Does the substantive conclusion differ?
  • Check 5 (boundary cases): Examine cases where human and LLM disagree. Is there a systematic pattern?

The validation menu is introduced in Module 4 (Text Annotation and Measurement) and revisited in Module 7 (Validation, Reproducibility, and Research Integrity).

§ disclosure template

# Disclosure Template

Every portfolio component and every module exercise that produces AI-assisted output should include a disclosure statement. This template follows the format expected by the American Journal of Political Science (AJPS) and American Political Science Review (APSR), adapted for course use.

Template

```

AI Use Disclosure

Tool(s): [Model name, version, interface, date(s) of use]

Purpose: [Specific task the AI performed]

Location: [Where in this work the AI contribution appears]

Verification: [What checks were performed, with evidence where available]

Human decisions: [What judgments were retained by the author]

The author takes full responsibility for all content in this document.

No AI tool was used as a source of fact, and no AI-generated reference

was included without personal verification.

```

Examples

Example 1: Writing Assistance

```

AI Use Disclosure

Tool(s): Claude 4 (Anthropic), via claude.ai, 12 September 2026

Purpose: To suggest alternative organisations for the literature review section

Location: The structure of Section 2 (Literature Review)

Verification: All sources cited in Section 2 were located, read, and verified

by the author before inclusion. The AI's structural suggestions were evaluated

against the author's own outline and modified substantially.

Human decisions: The final organisation, the selection of which studies to

include and emphasise, and all interpretive claims are the author's.

```

Example 2: Annotation

```

AI Use Disclosure

Tool(s): GPT-5 (OpenAI), via API, 15 October 2026

Purpose: To generate initial binary labels (relevant/not relevant) for

2,000 abstracts using the codebook in Appendix A

Location: The screening data reported in Section 3 and Table 1

Verification: A random sample of 200 abstracts was independently coded by

a second human coder. Agreement: 89% (Cohen's kappa = 0.76).

Disagreements concentrated in the "ambiguous" category; all disagreements

were resolved by the author.

Human decisions: The final inclusion/exclusion decision for every abstract

flagged as "ambiguous" was made by the author. The codebook was developed

by the author before any AI-assisted coding.

```

Example 3: No AI Used

```

AI Use Disclosure

No AI tools were used in the preparation of this work.

```

Important Rules

  1. The disclosure must include itself. If you used AI to help write the disclosure, say so.
  1. "Verified" means you personally checked. You cannot delegate verification to the same tool that produced the output.
  1. Fabricated references are never acceptable, disclosed or not. If you include a reference you have not personally confirmed exists and says what you claim, the disclosure does not protect you.
  1. A tool is not a co-author and not a source of fact. These are the positions of ICMJE, COPE, the SNSF, the ERC, and every major journal.

This template is introduced in Module 2 and used throughout the course. See Module 7 for the full treatment of validation, reproducibility, and research integrity.