Marin T. Kael
DE / EN

Research · Methodology Lab

How does the machine read an author?

Since 11 May 2026 three answer engines with web search – OpenAI, Gemini and Claude – have answered the same 16 questions about a new author. Knowledge graphs, search indices and reading platforms are sampled daily alongside. The programme measures whether and how reliably the machines find the author, and how stable the measurement itself is. Phase 1 ended on 7 October 2026; since the book launch on 8 October 2026 Phase 2 measures the effect of the published book.

Methods are pre-registered, source code and raw data are open, failure logs count equally with findings. Addressed to authors with visibility questions — and to practitioners of search and answer engine optimisation (SEO, AEO, GEO) who seek a validated measurement basis rather than anecdote.

Programme lead
Marin T. Kael
Active since
2026-05-11
Cadence
Measurement every 3 days · reports quarterly
Licence
CC BY 4.0 · MIT · CC0
Lines of inquiry · Active 3-phase design

Active programme across three temporally overlapping phases.

Marin T. Kael’s research programme operates in three temporally overlapping phases. Phase 1 (May to October 2026) combines seven pre-registered interventions Q0–Q6 with testing of the measurement instruments. Phase 2 (from the book launch on 8 October 2026 to Q3 / 2027) measures the effect of the published book. Phase 3 (from Q3 / 2027) runs long-term controlled experiments on the tested apparatus. Effect claims in Phase 2 remain subject to the instrument validation, which continues in parallel until Q1 / 2027.

Line 01

Citation Inventory

What does each measurement instrument show today? Eight measurement surfaces are sampled daily and the visibility of the author identity per identity cluster — person, work, genre, world-mechanic — is documented as a coverage matrix. In Phase 1 without effect interpretation: first count what the instruments actually depict.

The core is the citation measurement: 16 fixed questions in four groups (directly about author and work, about terms from the work, about genre recommendations, about the research programme) put to three answer engines with web search – OpenAI, Gemini and Claude (claude.ai, the newest Opus, Sonnet and Haiku model, incognito; Claude counts as one provider) – every three days. Sampled daily alongside: Wikidata, Wikipedia, Google Knowledge Graph, Bing, Google Search Console, Common Crawl, Goodreads, Hardcover, Reddit and Bluesky. Every answer is scored on a fixed scale from −3 (hallucination) to +3 (complete, sourced citation); the headline figure is the share of citing answers per provider (descriptive).

Line 02

Measurement-Instrument Validation

An instrument is reliable when repeated measurement of the same reality yields the same values. Phase 1 tests every measurement surface for test-retest reliability, intra-set consistency, and model drift — and determines which surfaces are even suitable for later effect measurements.

Test-retest correlation r over 24-h replication probes (threshold r ≥ 0.9 for API sources, ≥ 0.7 for language-model probes). Cronbach’s α intra-query-set ≥ 0.7 as a precondition. CUSUM charts on hit-rate over a 90-day window with alarm threshold h = 5. Model-version logs separately recorded; drift events are reported with drift profiles per measurement surface.

Line 03

Codebook Iteration

What even counts as a "correct citation"? Which answer is a hallucination, which is correct-but-incomplete? Phase 1 publicly versions the annotation schema, documents edge cases, and gathers public feedback — as a precondition for later effect claims to rest on an unambiguous measurement schema.

Codebook v0.x → v1.0 as a published milestone; until v1.0 is released (planned Q1 / 2027) results count as descriptive. Each new schema version comes with example annotations, edge-case discussion, and a diff to the predecessor version. Inter-rater agreement (Cohen’s κ) is collected from Q4 / 2026 with external annotators; threshold κ ≥ 0.7 as a condition for codebook version release.

Line 04

Open Materials

Everything the programme produces is open: methodology notes, pre-registrations, measurement source code, raw data, quarterly reports, failure logs, codebook versions. Methodology reviewers with a Python environment and internet access can trace any evaluation — that is the only audit mechanism a single-case study has.

Methodology notes and quarterly reports under CC BY 4.0, source code on GitHub under MIT, raw data under CC0 (where platform terms permit). Reports and methodology notes receive a Zenodo DOI. One replication archive per quarterly report with frozen version pins, environment.yml, user-agent strings, and endpoint snapshots.

Example finding · Phase 1

What a validation finding looks like.

Phase 1 measures not effects, but measurement properties. Test-retest reliability r tells us whether a surface yields the same values on repeated probing of the same reality. The threshold r ≥ 0.9 applies to API sources, r ≥ 0.7 to language-model-based browser snapshots.

The analysis is in the Validation Report Q3 / 2026. On an ongoing basis the results page shows the running measurements and, per question, how often repeated measurements yield the same score.

Publications

Published and in preparation.

Reports, working papers, methodology notes and pre-registrations, newest first. Every publication has a stable URL and a citation scheme; reports and methodology notes also a Zenodo DOI.

Validation Report · Q3 / 2026 DOI 10.5281/zenodo.23145378

How reliably does the programme measure? A test of the six instrument hypotheses of pre-registration Q0-INST

First validation report. None of the six pre-registered instrument hypotheses is confirmed: three cannot be tested with the data, three are not confirmed. Repeated measurements yield the same score in 87 to 94 percent of cases per provider, but the question set does not form a single scale (Cronbach’s α 0.41 to 0.61). With coverage balance, drift alarms and the status of all pre-registrations.
Report · 03 DOI 10.5281/zenodo.22015495

Found, not recommended: How language models learned who a new author is in 99 days

Analysis of 99 measurement days and 11,130 scored answers from three answer engines with web search. Questions about author and work reach a citation rate of about 71 percent after seven weeks and stay there; questions about terms from the work grow more slowly; in genre recommendations the author is named in 0.6 percent of answers. With confidence intervals, control channel and open data.
Working Paper · WP-04

Five Hidden Failure Modes in AEO/GEO Measurement Pipelines

A seven-day pre-registered experiment exposed five systematic failure modes that make citation rates look artificially stable or artificially low. For each: reproduction recipe, detection heuristic and fix. The corrected analysis lies on average 10.6 percentage points above the faulty one.
Working Paper · WP-02

Pre-Launch LLM Citation: Top-3 LLMs Hit 19.8–24.0% for a Pre-Launch Author Without Published Work

An author identity without a published book, reviews or press reaches measurable citation rates within seven days: OpenAI search 24.0 percent, Claude Haiku and Sonnet 19.8 percent each. The only input was identity work (Wikidata, ORCID, Zenodo, llms.txt).
Field Note · T+23 DOI 10.5281/zenodo.20549020

Zero to Cited in Six Days — A 23-Day Measurement of AI Visibility

Field note on the first 23 days: how fast do AI search systems and language models start reading, understanding, and citing a brand-new author? With tempo, provider, category, and grounding analysis. Open data, code, and the full bilingual report linked.
Project Journal · Living

Project challenges and solutions

Engineering journal of the first ten days: pipeline obstacles, methodological drift and reach-related brakes with symptom, cause, solution and methodological consequence. Later outages and method corrections are listed with date and cause in the outage register of the results page.
Methodology Note · 01 DOI 10.5281/zenodo.20170615

Baseline Measurement: Author Identity in the Citation Behaviour of Language Models (Active Pre-Launch Design)

v2.0 revises the phase model: the programme does not operate in 'first validation, then action', but in three temporally overlapping phases with continuously pre-registered interventions. Seven active pre-registrations (Q0–Q6) on eleven measurement surfaces — including the Cross-LLM Trust Graph, Common-Crawl Snapshot Probe, and machine-readable identity surfaces.
Pre-Registration · Q0 DOI 10.5281/zenodo.20125967

Pre-Launch Instrument Validation · Active Pre-Launch Window 2026-05 → 2026-09

Locks in, before data collection, the full Phase-1 measurement plan: six instrument hypotheses H-Q0-INST-01 through 06 (test-retest reliability, multi-snapshot aggregation, CUSUM drift sensitivity, Cronbach’s α, Wikidata anchor stability, inter-surface agreement), sampling plan, stop criteria, contingency plan, and reproducibility specification. CC BY 4.0.
Codebook · v0.1 DOI 10.5281/zenodo.20125976

Annotation Schema for AI Citation Behaviour

Operationalises, before every measurement probe, what is to count as a 'correct citation': four binary dimensions (Hit, Correct, Hallucination, Completeness), anti-pattern catalogue, four documented edge-case classes, version plan v0.1 → v1.0 with inter-rater threshold Cohen’s κ ≥ 0.7. CC BY 4.0.
Software Release · v0.3 DOI 10.5281/zenodo.20262669

marin-research-tools — Phase-1 Tooling

Source-code release of the Phase-1 instrument-validation tooling: style_lint.py (style-sheet linter for outbound), source_attribution_parser.py (Cross-LLM Trust Graph), Pre-Registrations Q0–Q6, operator-policy documentation. MIT licence. GitHub tag v0.3. Predecessors v0.1 (10.5281/zenodo.20126017) and v0.2 (10.5281/zenodo.20189714) remain permanently citable; v0.3 supersedes v0.2 by design refactor (see CHANGELOG).
Dataset · T+0 DOI 10.5281/zenodo.20126038

Wikidata Identity Snapshot · Zero Point

Wikidata items Q139720807 (author) and Q139720798 (book) on the reference date 11 May 2026 as a full EntityData export and SPARQL property listing. Serves as ground-truth anchor for H-Q0-INST-05 (coverage stability > 0.85 over the active pre-launch window). CC0 1.0.
Validation Report · Q4 / 2026 Forthcoming

Inter-Rater Agreement & Codebook v0.2

Second validation report: inter-rater agreement (Cohen’s κ) with external annotators, codebook version difference from v0.1 to v0.2, ongoing drift statistics per measurement surface, first statements on inter-surface agreement (which surfaces are redundant, which orthogonal?).

Validation Report · Q1 / 2027 Forthcoming

Consolidation of the Measurement Apparatus

Third validation report: combined reliability, drift, and inter-rater findings across three quarters. Decision on codebook v1.0, on which robust effect claims for Phase 2 rest. Surfaces not meeting the validation thresholds will be excluded from Phase 2 or replaced by alternatives.

Transition Note Forthcoming

Close of the Instrument Validation · Phase-3 Pre-Registrations

Methodology note with the conclusion of the instrument validation and the first pre-registered hypotheses for Phase-3 long-term controlled experiments from Q3 / 2027 on the validated apparatus.

Open Materials

Replication and auditability.

Style-sheet (canonical truth)
The reference document compared against every survey is maintained under versioning. The current state is available on request.
Measurement pipeline (source code)
Survey and linter scripts are public on GitHub: github.com/marintkael/marin-research-tools (Python, MIT licence). With each quarterly report there is also a replication archive with frozen version pins.
Pre-registrations
Every survey is published, before data collection, with hypothesis, measurement operationalisation, and stop criterion. Later changes are made transparent.
Failure log
Every material blocked by the linter is logged with reason and date. Aggregated excerpts are part of the quarterly reports. The policy boundary of the pipeline remains externally auditable.

About the programme

Marin T. Kael is an author — and runs in parallel an openly documented field laboratory on the question of how language-model- based search systems and AI answer engines take in, understand, and cite a literary author identity. Phase 1 (May to October 2026): pre-registered interventions Q0–Q6 before the launch and testing of the measurement instruments. Phase 2 (from the book launch on 8 October 2026): effect of the published book. Phase 3 (from Q3 / 2027): long-term controlled experiments on the tested apparatus.

Addressed to authors who want to understand their visibility in AI search, and to practitioners of search and answer engine optimisation (SEO / AEO / GEO) who seek a reproducible measurement basis rather than anecdotal claims. The work is self-funded and not affiliated with an academic institution.

A detailed description — mission, methodology, ethics, publication plan — is at /en/research/programme.

Contact

Enquiries on methodology, replication, or auditing: research@marin-t-kael.de. Source code and replication archives on GitHub.