The Synthetic Respondent

An old fraud, newly cheap to commit

Long before large language models, methodologists had a name for the interview that never happened. Curbstoning — an enumerator sitting on the curb and inventing plausible responses rather than knocking on the door — has been documented across professional survey operations for decades, and the empirical literature indicates it is neither vanishingly rare nor confined to unprofessional teams.¹ In one large South African survey, forensic re-analysis concluded that a meaningful share of interviews had been fabricated by fieldworkers despite standard controls — a finding from that study and context, not a universal rate, but a caution against assuming one's own data is immune.⁴ Separately, an audit of many widely used public-opinion datasets reported that a substantial minority contained exact or near-duplicate records above a five-percent threshold.² Fabrication, on this evidence, is a recurring risk in field research under pressure rather than an exotic failure.

Generative AI does not introduce this problem; it changes its economics. What previously required a fabricator to imagine internally consistent answers — cognitively taxing, and historically detectable partly because human fabricators tend to under-produce variance³ — can now be produced quickly, in volume, with fluent narrative texture in the open-ended fields that once resisted fabrication most. The historical tell of the fabricated qualitative response — flat, repetitive, thin — is precisely the weakness a capable language model can mask.

Four questions that are not the same question

Most confusion in this area comes from collapsing four distinct questions into one. Keeping them separate is the analytical spine of everything that follows, because a record can trigger one without triggering the others.

These are orthogonal. A record may have AI involvement (a translator used a model, and disclosed it) while remaining authentic, faithful, and non-fabricated. Conversely, a record may have zero AI involvement and still be pure fabrication — classic curbstoning. It follows that detecting AI involvement is neither necessary nor sufficient for establishing fabrication. The protocol proposed here therefore targets B, C, and D — authenticity, fidelity, and integrity — and treats A as, at most, a weak and easily-mistaken signal.

Why this matters acutely in Somalia and similar settings — and why now

The risk concentrates wherever three conditions coincide: remote management, high enumerator turnover, and thin independent verification. In the author's evaluation practice across Banadir, Puntland, Jubbaland, Galmudug, South-West State and Somaliland, all three are structural rather than incidental: international staff frequently manage programmes remotely, so a large share of collection runs through local enumerators and partners; access is uneven and security-contingent, so a proportion of sampled locations cannot be independently re-visited; and turnover means the person who collected a figure is often gone before anyone asks how. Somali-to-English translation adds a further layer, since much narrative data is rendered into English by field staff for whom it is a second or third language — the very population, as Section 02 shows, that AI-text detectors are most prone to misjudge.

These conditions are tightening, though the evidence varies in strength and should be read at the right level. At the sector level — across 22 monitored humanitarian operations, not Somalia specifically — the OCHA Centre for Humanitarian Data estimates that the share of crisis data assessed as available and up-to-date in its HDX Data Grids fell from 74 percent to 68 percent between its 2025 and 2026 assessments, a decline it attributes largely to the contraction in humanitarian financing.⁹ Beyond that primary figure, secondary sector analysis, aggregating agency reports, describes reductions in the in-person verification capacity that underpins data quality, including disbanded enumerator networks; some Somalia-specific figures — such as a sharp contraction in the number of locations where routine collection remains feasible — appear in that analysis only as reported projections, which I have not been able to trace to a primary agency publication and therefore treat as indicative rather than established.¹¹ The overall direction, however, is not seriously in doubt: the verification layer is thinning at the same moment the tooling for fluent fabrication has become increasingly accessible and capable. That coincidence — rising opportunity, falling scrutiny — is why this is a present problem, not a future one.

Why the obvious defence is not a defence

Confronted with suspected AI text, the reflexive move is to run it through an AI-content detector and treat the score as a verdict. This paper's central methodological claim is narrow but firm: whatever the future of detection technology, today's stylometric detectors do not have the evidentiary reliability to support an integrity finding in this context — and using them to accuse, discipline, or exclude field staff is both unsound and unjust.

The consequence is both epistemic and ethical. Epistemically, a tool with these error characteristics cannot by itself support a defensible finding: a positive result is ambiguous, and a negative result is easily manufactured, since light manual editing or paraphrasing of machine text has been shown to push a large share of it past these tools undetected.⁷ Ethically, building quality assurance around such a tool means constructing a process whose false accusations fall hardest on national staff — reproducing, in miniature, the asymmetry of trust that the localisation agenda exists to dismantle.

Four vectors by which AI text enters a dataset

Not all contamination is equal, and conflating the cases produces both false alarms and missed fabrication. Two of these vectors are integrity violations; two can be honest practice that nonetheless corrupts the evidentiary record when undocumented. Each maps onto the four questions of Section 01.

The taxonomy yields the governing principle of this paper: the issue is not whether AI touched the data, but whether the transformation is disclosed, traceable, faithful, and consistent with the research protocol. Vectors 01 and 02 are integrity violations to be caught and addressed through formal procedures; Vectors 03 and 04 are process risks to be governed by disclosure and by preserving the source. A system that cannot tell these apart will either wave through fabrication or wrongly treat honest, efficient practice as misconduct.

A provenance-first detection protocol

If the dependable signal is not in the text, it is in the record's provenance and in the respondent's corroboration. The five layers below move from cheapest and most automatable to most resource-intensive, and are designed as routine data-quality assurance rather than as tools reached for only after suspicion. None is individually conclusive; their value is cumulative and triangulated. The pre-AI fabrication literature approached detection through multiple, convergent statistical indicators paired with targeted re-interviews rather than any single test;¹ the stronger position that no single signal should ever be treated as decisive on its own is this paper's methodological conclusion, and it governs everything that follows.

From indicators to claims: an inference ladder

The protocol's honest output is a position on a ladder of claims, not a verdict. Conflating these rungs is the same error the AI detectors commit — presenting a probabilistic signal as a categorical fact. Each step up requires more, and more independent, evidence.

Note that the protocol never needs to name the technology an enumerator did or did not use. It assesses record integrity and provenance — questions B, C, and D — and is deliberately agnostic about question A, because the invisible presence of a model is neither knowable with confidence nor, in itself, the thing that matters.

What a MEAL manager should actually do

A framework that stops before the decision is of little use in the field. The following tiers translate the inference ladder into proportionate action. They deliberately avoid arbitrary numeric cut-offs; the organising principle is that multiple independent indicators raise confidence, while no single anomaly establishes fabrication. "Independent" is doing real work here — three anomalies that all stem from the same cause (a misconfigured device clock, say) are one indicator, not three.

An anomaly is a reason to investigate, not a reason to accuse

A provenance protocol concentrates scrutiny on the people with the least power in the system: national enumerators, translators, and field officers, often employed on short contracts and managed remotely by international staff. Deployed carelessly, it becomes an instrument of exactly the injustice this paper set out to avoid. The following safeguards are not an appendix to the method; they are part of it.

Guard against the non-native-English trap at every layer, not just the detector. The same bias that makes AI-text detectors unreliable can seep into human judgement — treating fluent-but-regular English, or "too clean" narrative, as inherently suspect. Fluency is not evidence. Anomaly signals should be tied to provenance and occurrence, never to how the prose reads.

Presumption of good faith, and due process. A flag is a question, not a charge. Anyone whose work is investigated should know the concern, see the basis for it, and have a genuine opportunity to explain before any adverse decision — with the recognition that power asymmetry makes it easy for a national staff member to be presumed guilty and hard for them to contest it.

Proportionality. The intensity of investigation should match the strength and independence of the indicators, not the convenience of blaming the nearest contractor. Quarantining records is reversible; a misconduct finding is not.

Privacy and consent in the forensic layers themselves. GPS traces, device identifiers, and audio recordings are personal data. Collecting and scrutinising them for integrity purposes must rest on informed consent, clear retention limits, secure handling, and applicable data-protection standards — the QA process must not itself become a surveillance or protection risk for respondents or staff. Audio audits in particular require that respondents consented to recording for this use.

The stylometric reflex versus the provenance discipline

The argument reduces to a single reorientation. The arrival of generative AI in field research is widely received as a novel detection challenge demanding a novel detection tool. It is better understood as a stress test of a discipline the sector already possessed and had let atrophy: maintaining an unbroken chain of provenance from a reported figure back to a real person in a real place. The fabrication literature that predates the language model already showed how this is done — through metadata, duplication tests, appropriate distributional checks, and re-contact.¹ The conclusion this paper draws from it is that such work must be done by triangulation, holding any single signal as a question rather than an answer.

What generative AI changes is the cost of the two failures. It makes fabrication cheaper to commit; and, through the biased detector, it makes false accusation cheaper to level against the very national staff the system depends on. A serious response refuses both. It declines to treat fluent text as evidence of a real interview, and equally declines to treat fluent text as evidence of a fake one. It rebuilds, instead, the unglamorous infrastructure of provenance — and pairs it with the safeguards that keep an integrity process from becoming an injustice.