CounterProof Research
Contents

Agentic Adjudication of Clinical Reports

An evaluation design for second reads

Luuk Joseph Soons
CounterProof Research, a practice of Clavestra Capital Limited, Malta
ORCID: 0009-0000-3088-6373
Conceptual methods proposal · companion to Correction Across Clinical Handovers · preprint v0.6 · 6 November 2026 · DOI 10.5281/zenodo.23978361
Deposited on Zenodo on 6 November 2026: 10.5281/zenodo.23978361. This web page is a reading copy of the deposited text.

Not peer reviewed. No study proposed here has been conducted. The literature gap this proposal rests on is provisional: one near-neighbour paper has been read only as an abstract (section 2.4). AI assistance and author responsibilities are disclosed below.

Abstract

A signed radiology report can carry an error that a second reader would catch: a side reversed, a measurement copied wrongly, a finding asserted and then denied, a dictation phrase that changes the meaning, an impression item with no supporting finding. Large language models now detect such text-internal errors at rates that match or exceed radiologists reading the same text in some published comparisons, and have been run retrospectively over report streams of ten thousand reports and more. The question this proposal asks is the one Study B of the companion paper Correction Across Clinical Handovers poses in general form: does an additional, separately obtained read add anything worthwhile to the strongest single-model workflow, and how would one know?

A screen of thirteen near-neighbour papers, with three more read since, sixteen in all, finds the question open for clinical reports, provisionally in one named respect: one near-neighbour paper was reached only as an abstract (section 2.4). No published study adds one separately obtained read from a different model family to the strongest single-model workflow on reports and counts, at the level of individual findings, what it adds, what both miss, and what it costs to adjudicate. What is published argues against a free gain: a majority of eight open-weight models scored below its best single member against a human reference; an identical evidence packet given to thirty-four models raised their agreement, around correct answers on most questions and around wrong ones on eleven, and raised the number of questions on which the majority was wrong from 16 to 20; the most accurate models are the most error-correlated on fixed-option tasks; and, where both precisions are reported, synthetic error sets inflate the precision of Weng's GPT-4o methods and of ClinicalBERT.

The proposal specifies a study on de-identified, already-signed radiology report text with three error strata kept apart: a sealed, script-planted stratum with a hard sub-stratum of reasoning-dependent errors; a model-planted public stratum reported separately; and an untouched real stratum for prevalence and precision, which needs two radiologists and an arbiter and is not reachable in 2026. The arms are a rules checker, the strongest single-model workflow, the same model run repeatedly at a stated temperature, a cross-vendor panel split by data licence, and a base-model witness. The measures are the companion paper's seven: additional correct findings, shared misses, false objections, findings lost during filtering, adjudication effort, the packet each reader received before any shared discussion, and agreement not counted as confirmation. The decorrelation measure is defined for free-text finding sets. The kill rule is a paired comparison; the test and the item count are computed from development-set miss rates before pre-registration and are not computed in this draft. The 2026 form of the study is declared a ceiling check. The contribution is an evaluation design. If the strongest single model plus five repeats of itself does as well as the panel, the panel should be abandoned.

Keywords: radiology reports; report quality assurance; second read; error detection; large language models; model panels; error correlation; evaluation design; agentic epistemics; agentic institutions

1. The question

The companion paper, Correction Across Clinical Handovers [1], proposes an evaluation design for a correction workflow across simulated clinical handovers. Its Study B asks a further question, to be run only after its Study A is interpretable: "Start with the strongest single-model workflow and compare it with the same workflow plus a read from the same model, a different model family or a qualified clinician." It fixes how that comparison must be run: "Record the packet each reader receives before shared discussion. Agreement after receiving a common packet does not by itself establish independent confirmation." And it fixes what to measure: "Measure additional correct findings, shared misses, false objections, findings lost during filtering and adjudication effort. More disagreement or more citations do not establish independence or usefulness. Remove the multi-model procedure if it makes no worthwhile additional contribution."

This proposal belongs to the same programme as the companion paper: the institutional frame of Agentic Institutions [2], the account of evidence and warrant in AI-assisted reports in Agentic Representation [3], and the panel procedure of independent readings, verbatim quotation and provenance in Agentic Stemmatics [4] and in the programme's research note on collation and interpretability [26]. That procedure was built in the author's practice, CounterProof, for the review of financial-infrastructure software, where a finding names a line and a defect or vulnerability that a test can settle, and the programme's aim is sound agentic institutions in finance and in healthcare alike. The companion paper was the programme's first bounded step into clinical work; this proposal is the second, carrying the procedure to clinical text, where the settling read is a radiologist's, and asking whether it earns its cost there. It takes the companion paper's question to a different object. Its unit is one signed clinical report, in the first instance a radiology report, and within it one candidate finding: a quoted line or line pair with a stated error class. It does not depend on Study A and does not wait for it. It shares Study B's measures, its packet rule and its removal rule, and changes the material and the oracle.

The author's starting phrase was "a second opinion on a diagnosis report". This proposal reads that as the detection of text-internal errors in a signed report, not as a judgement on the diagnosis the report states. The narrowing is a decision, not a finding. It was made because every working oracle for report errors in the published work is text-based (section 2.3), while diagnostic truth has no reference standard that arrives in time for a study of this size. A diagnostic-level second read, on case vignettes with gold diagnoses, is a different study and is named as such in section 5.

2. What is known

2.1 The near-neighbour screen

Thirteen papers were read in full text, except one read as an abstract, between 4 and 5 November 2026, and graded against six sub-questions: the marginal value of a different-family read (Q1), error correlation across model lineages (Q2), the effect of a shared evidence packet on independence (Q3), clinician error diversity (Q4), the oracle and its noise (Q5), and precision at real prevalence (Q6). Three further papers were read on 5 November at abstract or full-text level. Every number below was located in the paper's text, by a script independent of the reader or by hand where the script could not match a table row or a sentence broken by extraction; the supplement records which, and the step-zero note carries the sentences that reached the reviewers only as a reader's gloss.

2.2 Marginal value and error correlation

No paper adds one separately obtained read to the strongest single-model workflow on reports and counts what it adds. Wang and colleagues [5] screened five model families on 10,000 Chinese radiology reports, carried one forward (Qwen3), ran it on 95,582 reports at a report-level sensitivity, specificity, positive predictive value and F1 of 0.99, 0.99, 0.90 and 0.94, and raced it against three radiologists on 300 error-positive reports, where it classified 320 of 335 error instances against 223, 178 and 152 for the three readers. The four other families' outputs on the same 10,000 reports exist and are reported only as an appendix figure; no overlap of misses is published. Flanders and colleagues [12] ran eight open-weight models and GPT-4o over 29,766 head-CT impressions; against a human reference on 1,490 cases the four-of-eight majority scored a Matthews correlation of 0.29 where the best single model scored 0.57. That reference set was built from the 1,668 cases on which the consensus and the triage tool disagreed plus 58 concordant cases, with 236 of the 1,668 discordant plus 58 concordant reports excluded as ambiguous, and the authors' own conclusion rests on a different measurement, the tool's apparent performance with each ensemble as reference: "An ensemble of open-source LLMs provides a more consistent and reliable method to derive a ground truth retrospective evaluation of a clinical AI triage tool over a single LLM alone." The whole-body CT filter of Wachinger and colleagues [13] added four other open-weight models from three lineages to its strongest single model on real reports and found identical recall, 0.92 against 0.92, with higher precision; that task was abnormality extraction, not error audit. On medical question answering, MedAgentBoard [18] found that multi-agent collaboration "does not consistently surpass the best single LLM configurations", with a mixed-family consensus scoring 78.00 on MedQA against 89.90 for single-model self-consistency. Zöller and colleagues [15] found that an ensemble of five model families beat the best single model on top-5 accuracy and "performed better than four of the five individual LLMs and approximately at the level of the best-performing LLM" on top-1.

Error correlation across lineages is measured on fixed-option tasks and nowhere on report auditing. Kim, Garg, Peng and Garg [16] found that two wrong models choose the same wrong option about 60% of the time on HELM, where "choosing between incorrect answers uniformly at random would lead to an agreement rate of 1/3", and that "more accurate models (and especially if both models are accurate) are more correlated." Bean and colleagues [17] found question-level correctness correlations of 0.39 to 0.58 among the seven current models of the sixteen tested on medical licensing questions, with a cross-vendor pair as correlated as two models of one family. Zöller and colleagues found that "LLMs agree more among themselves than with physicians ( Fig. 4 C ) and this difference is particularly pronounced in situations where both human and LLM diagnoses are incorrect". Akinci D'Antonoli and colleagues [8] found that GPT-4.1 and Llama 3.3 70B "exhibited the same performance hierarchy despite differing architectures" across four planted error classes, a shared-miss pattern recorded without a pairwise statistic. The MedJUDGE scoping review [19] states that no included study quantified inter-judge error correlation and that only two of forty-nine avoided a same-family generator and judge.

2.3 Shared packets, clinician reads, oracles and base rates

Farajiamiri and colleagues [14] gave thirty-four models the same retrieved evidence report on 169 radiology questions: "The median entropy decreased from 0.48 (IQR 0.55) under zero-shot to 0.13 (IQR 0.51) under agentic reasoning"; the median majority fraction rose from 0.85 to 0.97; agreement "increased with an incorrect majority in 11 (7%)" of questions; and the count of questions on which the majority was wrong rose from 16 to 20 (Table 4, pooled). The authors write that the identical-context design "may amplify correlated errors when retrieval is misleading or incomplete". The whole-body CT filter shows the same shape on real reports: pairwise Jaccard agreement among five models rose from 0.541 to 0.803 after a shared packet while the majority vote's overlap with the manual labels after the packet "was slightly lower than Stage 1". No paper has an arm in which readers receive different or independently obtained packets.

On clinician reads, Zöller and colleagues report that physician and model errors are complementary on vignettes: when a model missed the diagnosis entirely, an individual physician held it in 30 to 38% of cases and a five-physician collective in 55 to 65%; when physician collectives missed, models held it in 17 to 32%. On reports, Wang's radiologists were given text only and complementary catches were not reported. Schmidt and colleagues' discussion [9] offers anchoring as a hypothesis for an attending radiologist checking a resident's report, not for a clinician shown model output; whether a clinician read taken after model output is still an independent read is measured in none of the papers read, and four papers on clinician anchoring after model output are listed as unread in section 2.4.

Every working oracle for report errors is text-based. Wang's two senior radiologists read text only and agreed on 79.35% and 85.60% of positives before consensus; Schmidt's single resident labelled 3,233 reports, with two consultants on 100 at kappa 0.79 to 0.88; Weng's single radiologist [10] classed ambiguous cases as error-free, with a second rater at kappa 0.998 on an oversampled 500; the multi-pass study's reviewers saw the model's rationale and never reviewed unflagged reports [11]. Planted errors give an exact oracle and a known inflation: Gertz and colleagues [7] inserted 150 errors into 100 of 200 reports and found GPT-4 at 82.7% detection against radiologist groups at 80.0 to 89.3%, with one senior radiologist above it at 94.7%; Weng's synthetic balanced set put the precision of the GPT-4o methods and of ClinicalBERT at 0.979 or more where the real set put GPT-4o alone at 0.219.

Base rates and precision span an order of magnitude and track prevalence and oracle, not model quality alone: Wang, 4.6% of reports with a confirmed error after routine double reading and a report-level PPV of 0.90; the multi-pass study, 14 errors in 1,000 MIMIC-III reports and a PPV of 0.063 rising to 0.159 over three passes, with review burden falling from 192 to 88 reports per thousand; Weng, 1.2% laterality prevalence and PPV 0.219 for the model alone; Kim and colleagues [6], on 10,000 head-CT reports deemed error-free by physicians, "GPT-4 detected 96 errors, with a low positive predictive value of 0.05"; Schmidt, 44.2% of reports with any dictation error and 3.2% with a significant one. No paper tests whether a separately obtained second read moves PPV; the multi-pass study's rise from 0.079 to 0.159 comes from a same-model verifier pass primed with the detector's own candidates, and the step from 0.063 to 0.079 is its extraction pass; neither is a separately obtained second read.

2.4 What this gap claim rests on, and what it does not

The claim that the question is open survived two outside reviewers reading the screen and a confirmation pass, with all six sub-question verdicts unchanged. It is provisional in one named respect: the NeurIPS 2026 workshop paper on post-deployment monitoring of LLM-generated radiology indications [20], which reports that pooling several model judges raised sensitivity, was reached only as an abstract, its full text being behind a human-verification page this work did not pass. It must be read before the gap is claimed in print without this qualification. Also unread: Goh, Qazi, Dratsch and Chanda on clinician anchoring; Goel on judge similarity; Taylor and Potts on human double reading; Rao on the ReXErr dataset; Wang's appendix with the four non-selected models; Zöller's complementarity matrix; Farajiamiri's released per-question data.

2.5 What a non-incremental study must do

  1. Add exactly one separately obtained read to the strongest single-model workflow, the workflow allowed its own multi-pass structure, and count additional correct findings and shared misses at the level of findings, by error class.
  2. Run the same model repeatedly at a stated sampling temperature as the null for "another read"; every paper read ran at a stated temperature of 0 to 0.1 or once, except Schmidt, which averaged three runs per model.
  3. Vary the packet: identical shared evidence against independently obtained evidence.
  4. Capture any human read before model output is shown, and record the order.
  5. Use an oracle with quantified noise; keep planted and real strata apart; report both.
  6. Report precision at real prevalence with a declared false-objection budget.
  7. Release per-item outputs so that pairwise error correlation can be recomputed by others.

3. The proposed study

3.1 Unit, data and strata

The unit is one signed radiology report for prevalence, precision and workload, and within it one candidate finding: a quoted line or line pair plus one of six text-internal error classes: laterality contradiction between findings and impression; numeric or unit inconsistency; negation contradiction; device or support-line contradiction; impression item with no supporting finding; meaning-changing homophone or dictation error. Diagnostic truth, perception and discordance between report and image are out of scope.

Data come from public corpora only, with no hospital data in 2026. The Open-i collection, whose page could not be reached on 5 November 2026, is the candidate public source; its per-item licence must be logged for every report used and tested for a no-derivatives term before any report enters an application-programming-interface seat or a shared packet. Until that page is readable, the 2026 runs use ReXErr-v1 on local seats and, for the application-programming-interface panel, a licence-cleared corpus still to be named. ReXErr-v1 [23] is published on PhysioNet as open access under the Open Data Commons Attribution License v1.0, with errors generated by GPT-4o and MIMIC-CXR as its parent project; PhysioNet's guidance of 24 September 2025 on MIMIC data [21] states that such data "must not be stored or retained by third-party LLM services" and to "use locally deployed LLMs", and that "if a service's data handling practices are unclear or cannot be fully verified, do not use the service." PhysioNet's earlier guidance of 18 April 2023 [22] named conditions under which particular vendor services could be used; the 2025 wording is the later and stricter of the two and governs this design. ReXErr-v1 therefore stays on locally run models until PhysioNet confirms whether the derivative's licence releases it from that guidance. The 1,024-variant set released by Akinci D'Antonoli and colleagues [8], radiologist-authored and publicly deposited, is a candidate planted stratum from a non-vendor source once its licence is read.

Three error strata, never pooled:

3.2 Oracle

For stratum (a) the sealed plant list settles every flag on a planted line, and a flag on a line the script did not touch is parked: it is not a hit and not a false objection, and it is carried to the stratum (c) readers. A false objection in stratum (a) is only a wrong class on a planted line, or a false presence that the report text refutes mechanically. A refutation is mechanical when the quoted span does not occur in the report, when it occurs without the token the finding claims, or when the claimed class contradicts a surface pattern in the quoted span; the rule is fixed before the first run. Parked flags do not enter the matched false-objection budget. For stratum (b) the dataset's planted-error labels settle flags on planted lines, as benchmark labels; the dataset's page warns that its synthetic reports "should never be used for clinical decision-making or treated as ground truth data", and no clinical claim rests on them. For stratum (c) two radiologists read every flagged report, capped by stratified sampling, plus a random sample of unflagged reports for miss discovery, with a third radiologist as arbiter; every read is logged with a timestamp before any model output is shown to the reader; disagreement is kept as a label and not flattened to consensus.

3.3 Arms

R: a deterministic rules checker for the six classes, tuned on the development split and frozen, after Weng's finding that a tuned rules checker beat its model hybrid on precision. S1: the strongest single-model workflow, selected on the development split and frozen, multi-pass allowed. S1xN: the same workflow run five times at a stated sampling temperature on identical packets, the within-model control. P-API: a cross-vendor panel of application-programming-interface models, each reading the same frozen packet once under the same brief, on licence-cleared material only; the panel's flag set is the union of its members' flags, and the matched false-objection budget is applied to that union. P-local: a panel of locally run open-weight models on material that may not leave the premises. W: a base model without instruction tuning, through a controlled wrapper, as a non-fluent witness. The drafting lineage of this proposal is excluded from every arm. No clinician arm exists in 2026: the clinician arm waits on stratum (c), which is not reachable in 2026, and on the unread clinician-anchoring literature. The companion paper's Study B names a qualified clinician as one of three possible added reads; this is stated here so that a reader of both records does not think an arm was dropped silently.

3.4 Measures

The companion paper's seven, on the union of candidate findings, per stratum and per panel, with recall against the oracle reported beside them as the kill rule's input: additional correct findings (confirmed findings in P not in S1, and in S1xN not in S1, at a matched false-objection budget); shared misses (planted or reference-confirmed errors missed by every member of P and by all five S1 runs, as counts and as pairwise joint-miss rates against the independence expectation, by class); false objections (as defined in section 3.2 for stratum (a); for stratum (b), flags refuted by the dataset's labels on planted lines; for stratum (c), flags refuted by the reference plus text); findings lost during filtering (every finding present in a raw return that the orchestrator's quote check or de-duplication dropped, logged with the reason and re-scored against the oracle at the end); adjudication effort (orchestrator and arbiter minutes per report and per confirmed finding, and tokens per read; in 2026 this is the hand adjudication of plant-stratum false-presence disputes); the packet each reader received, logged before any shared discussion; and agreement, recorded but never counted as confirmation. A severity axis is taken from a partner department's addendum policy when one exists, grading each finding as one that would trigger an addendum or a call to the referrer, a peer-learning item, or a typographical correction. Until a partner department's addendum policy is available, no separate severity grade is applied, and findings are reported as the six error classes in section 3.1.

3.5 Decorrelation, defined for free text

The fixed-option statistic of Kim, Garg, Peng and Garg has no form for finding sets. Decorrelation is therefore defined as the per-class rate at which two seats flag the same wrong line, meaning the same quoted span with a wrong class or a false presence, with the joint-miss rate against independence computed on the union of candidates, and undefined where misses approach zero. Pairs within P are compared with pairs within S1xN: if cross-vendor pairs are no less correlated than repeats of one model, vendor diversity bought nothing. That comparison does not itself remove the panel; removal is the additions test of section 3.6. Every batch carries one known-positive plant and one known-negative clean report; a seat that misses the known positive, or returns one label for every report, is dead for that batch rather than counted as a mediocre reader.

3.6 The kill rule, and what 2026 can show

The comparison of P's confirmed additions over S1 with S1xN's confirmed additions over S1 is a paired comparison of two proportions on the same items. The pre-registration states the exact paired test and the number of items at which "adds nothing" is distinguishable from "loses to S1" at the declared false-objection budget, computed from development-set miss rates before any confirmation run. Panel composition is fixed before that computation, and the item count is recomputed for each candidate panel. The panel is removed if its confirmed additions over S1 do not exceed those of S1xN at matched false objections, per panel and per stratum, never pooled. The research line is killed if the rules checker matches or beats S1 on recall and false objections for the six classes on strata (b) and (c); on stratum (a) the rules checker is a ceiling check only, since the plant script and the checker share the classes. The rules-checker kill cannot be applied in 2026, because it requires stratum (c). The packet-variation contrast of section 2.5, item 3, is not implemented in the 2026 form either: every 2026 arm reads the same frozen packet, and the contrast is deferred with stratum (c), whose independently obtained evidence is a hospital's record.

The 2026 form of this study, on strata (a) and (b), is a planted-stratum ceiling check and a methods check. It cannot measure the panel's precision at real prevalence, which needs stratum (c) and the radiologists of 2027, and it is declared as such.

3.7 Intended purpose and what is not claimed

Intended purpose, to be written into the pre-registration: a research instrument that measures the detection performance and inter-reader dependence of independent automated reads of de-identified, already-signed radiology report text, and in 2026 of script-altered copies and a synthetic derivative of such text, in a retrospective documentation-quality study. It produces no output for any patient's care; no clinician treating a patient receives it; it processes no image. On that intended purpose the author's reading is that qualification as a medical device under Regulation (EU) 2017/745 is not reached, following MDCG 2019-11 Rev.1 [24], and that the research exemption of Article 2(6) of Regulation (EU) 2024/1689 applies while the instrument is wired to no product [25]. Neither text was checked against its source in this work; the sentences are the author's reading, carried from the companion paper's assessment, not a sourced claim and not a legal opinion.

Not claimed: that any arm detects errors of diagnosis or perception, or discordance between report and image; that fewer errors reach patients; that performance on a public corpus predicts performance on a hospital's stream; that planted-error recall predicts real-error recall; that the two-radiologist reference is truth; that the panel is safe to show to a reporting radiologist; that this is a device, a second opinion or a clinical service; that chest radiograph results transfer to CT, MRI, pathology or discharge summaries.

3.8 Cost

Model usage: on the order of 1,800 reports at roughly 2,300 tokens per report per seat across six seats, plus five repeats of S1 and three development iterations, on the order of 45 million tokens before development iterations, about 58 million with three development iterations at one seat-pass each, and more once panel sizes are fixed. Orchestration: about forty hours in 2026 for the mechanised quote check and the hand adjudication of plant-stratum disputes. Stratum (c): two radiologists at up to 450 reads each and an arbiter, in 2027, priced as co-authorship or at a department's internal rate. Bench time: two to three weeks of windows on a contained bench that already runs the seats.

4. What must happen before pre-registration

The NeurIPS workshop paper read in full. The clinician-anchoring papers read, because they decide whether a 2027 clinician arm can be captured after model output at all. The Open-i licence page read and the per-item test run. PhysioNet's answer on ReXErr-v1 obtained. The licence of the Akinci D'Antonoli set read. The paired test and the item count computed from development-set miss rates. A plan for releasing per-item outputs. A partner department identified for stratum (c), with its addendum policy as the severity axis.

5. Relation to the companion paper and to the studies this one is not

This record is a companion to Correction Across Clinical Handovers and a second domain for its Study B, not a change to it; that paper's Study B keeps its unit, the simulated handover episode, and its ordering after Study A. Two further studies were considered and are not this record. A second-read study on clinical research reports, where a trial report is checked against its pre-start registration and the oracle is a public document, is a separate record under its own title. A diagnostic-level second read on case vignettes with gold diagnoses, the study closest to the author's original phrase, is named here as the one that would test that phrase and has not been designed.

6. Limitations

No study has been conducted. The screen is one lineage's reading of sixteen papers, two outside model reviewers having read the screen's notes but not the papers; its quotations were checked by a script with a planted control, and the misses the script reported were classified by hand as table rows and extraction artefacts. The design is one angle's, judged by three reviewers of the same lineage and revised on two outside reviewers' objections. The data-governance statements rest on pages read on one day and on a policy that says it may change without notice. The literature gap is provisional as stated in section 2.4. The six error classes have no published prevalence of their own; the base rates in section 2.3 are for the error definitions of the papers that report them.

Author declarations

AI assistance. Claude (Anthropic) drafted this proposal, conducted the literature screen through delegated readers, and applied every revision. Codex (OpenAI) wrote the draft from which the companion paper was cut and contributed to its revisions. Kimi K2.7 (Moonshot) and Grok (xAI) reviewed the screen note, the design note, the deposit decision, the critic's appraisal and the extracts in two rounds on 5 November 2026, closing most of their own rows and leaving the rest open or deferred, and reviewed this manuscript at v0.1, at v0.2 and, as the deposit candidate, at v0.4. DeepSeek (DeepSeek AI) read v0.2 fresh as a third lineage and returned ADOPT-WITH-CHANGES, a first attempt having returned no answer and being recorded as invalid, and read v0.4 as the deposit candidate; DeepSeek and GLM (Z.ai) then read v0.5, the first release text, as a release read. None of them read the papers. The six reference entries left incomplete at v0.3 were completed on 5 November from the Crossref, arXiv and neurips.cc records, every name checked against the paper texts fetched by other routes; one journal version named in earlier drafts was dropped because its identifier resolves to a different article. The three reviewers then read v0.4 as the deposit candidate and checked the completed entries against the records, and two reviewers read v0.5 after the author's additions; their verdicts are in the supplement. v0.6 differs from v0.5 by the changes the supplement lists; no reviewer read v0.6. A credit limit on the drafting lineage's provider interrupted the screen's own verification layer mid-run; the orchestrator's script-based quotation check replaced it, as the supplement records. These reviews were not peer review or clinical validation. The author made three decisions recorded in the supplement: that the model and not a human constructs links and disposes challenges in the tested arm of the companion paper; that this proposal reads "a second opinion on a diagnosis report" as text-internal error detection; and the title. Responsibility for the argument, sources, contents, licence and decision to publish remains with the human author.

Competing interests. The author has a commercial interest in CounterProof, a practice of Clavestra Capital Limited, whose review and construction activities could benefit from adoption of the proposed methods. This manuscript provides no independent evaluation of CounterProof's effectiveness.

Funding. No external funding supported this work.

Data and ethics. No patient records or participant data were processed for this manuscript. No participant study was performed. The proposed study uses de-identified public corpora under their licences; future work on a hospital's reports requires its own governance and approvals, none claimed here.

Licence. Creative Commons Attribution 4.0 International (CC BY 4.0).

References

Author lists for entries 10, 11, 12, 15, 19 and 20 were taken on 5 November 2026 from the Crossref record, the arXiv record or the neurips.cc page and checked against the paper texts; of these, entries with up to six authors list all of them and the others the first three. Every other element was read from the article page, the arXiv record or the database record named in the supplement, except where an entry says a page was not reached.

  1. Soons LJ. Correction Across Clinical Handovers: An evaluation design for agent-assisted learning health systems. Preprint v0.15, 5 November 2026. Zenodo. Version DOI 10.5281/zenodo.23974125; concept DOI 10.5281/zenodo.23974124.
  2. Soons LJ. Agentic Institutions: Organisational Memory, Correction and Continuity Across Successive AI Agents. Preprint v0.3, 28 October 2026. DOI 10.5281/zenodo.23727633.
  3. Soons LJ. Agentic Representation: Evidence, Selection and Warrant in AI-Assisted Reports. Conceptual preprint v0.3, 28 October 2026. DOI 10.5281/zenodo.23727617.
  4. Soons LJ. Agentic Stemmatics: Collation, Provenance, and Conservation for an Emerging Machine Textual Tradition. Preprint v1.41, 18 October 2026. DOI 10.5281/zenodo.23458161.
  5. Wang L, et al. Large Language Model-Assisted Radiology Report Quality Assurance: Real-World Multicenter Evaluation. Journal of Medical Internet Research. 2026;28:e106250. DOI 10.2196/106250.
  6. Kim S, Kim D, Shin HJ, et al. Large-Scale Validation of the Feasibility of GPT-4 as a Proofreading Tool for Head CT Reports. Radiology. 2025;314(1):e240701. DOI 10.1148/radiol.240701.
  7. Gertz RJ, Dratsch T, Bunck AC, et al. Potential of GPT-4 for Detecting Errors in Radiology Reports: Implications for Reporting Accuracy. Radiology. 2024;311(1):e232714. DOI 10.1148/radiol.232714.
  8. Akinci D'Antonoli T, Adams LC, Lübberstedt J, et al. GPT-4.1 and Llama 3.3 70 fail to detect clinically relevant errors in radiology reports in zero-shot evaluation. European Radiology. 2026;36(11):8149-8159. DOI 10.1007/s00330-026-12697-z. Dataset: DOI 10.5281/zenodo.17529445.
  9. Schmidt RA, Seah JCY, Cao K, Lim L, Lim W, Yeung J. Generative Large Language Models for Detection of Speech Recognition Errors in Radiology Reports. Radiology: Artificial Intelligence. 2024;6(2):e230205. DOI 10.1148/ryai.230205.
  10. Weng KH, Chou YC, Kuo YT, Hsieh TJ, Liu CF. Detecting Laterality Errors in Combined Radiographic Studies by Enhancing the Traditional Approach With GPT-4o: Algorithm Development and Multisite Internal Validation. JMIR Formative Research. 2025;9:e76384. DOI 10.2196/76384.
  11. Kim S, Lee S, Lee SY, Kim J, Kan K, Yoon D. A Multi-Pass Large Language Model Framework for Precise and Efficient Radiology Report Error Detection. arXiv 2506.20112 (25 June 2025), the version read. Its journal version, with a seventh author and a changed title, is Kim S, Lee S, Lee SY, Kim J, Kan K, Lee H, Yoon D. Improving Radiology Report Error Detection Using a Multipass Large Language Model: Framework Development and Validation. JMIR Medical Informatics. 2026;14:e87368. DOI 10.2196/87368; its Crossref record was read, its article page was not, and no figure in this manuscript is taken from it.
  12. Flanders AE, Peng Y, Prevedello L, et al. A multi-agent large language model framework to automatically assess performance of a clinical AI Triage tool. npj Health Systems. 2026;3(1):35. DOI 10.1038/s44401-026-00100-4.
  13. Wachinger C, et al. Whole-body CT attenuation and volume charts from routine clinical scans via evidence-grounded LLM report filtering. arXiv 2605.05933 (7 May 2026).
  14. Farajiamiri M, Sopa J, Afza S, et al. Agentic retrieval-augmented reasoning reshapes collective reliability under model variability in radiology question answering. arXiv 2603.06271 (6 March 2026), whose record lists the journal version as Patterns 2026, 101639, DOI 10.1016/j.patter.2026.101639; the journal page was not opened in this work.
  15. Zöller N, Berger J, Lin I, et al. Human-AI collectives most accurately diagnose clinical vignettes. Proceedings of the National Academy of Sciences. 2025;122(24):e2426153122. DOI 10.1073/pnas.2426153122.
  16. Kim EM, Garg A, Peng K, Garg N. Correlated Errors in Large Language Models. Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:30038-30066, 2025. arXiv 2506.07962.
  17. Bean AM, et al. Do Large Language Models have Shared Weaknesses in Medical Question Answering? arXiv 2310.07225 (v3, 11 October 2024).
  18. Zhu Y, He Z, Hu H, et al. MedAgentBoard: Benchmarking Multi-Agent Collaboration with Conventional Methods for Diverse Medical Tasks. NeurIPS 2025 Datasets and Benchmarks Track. arXiv 2505.12371 (v2, 30 October 2025).
  19. Li C, Akhtar Z, Kwak M, et al. A Scoping Review of LLM-as-a-Judge in Healthcare and the MedJUDGE Framework. arXiv 2604.25933 (3 April 2026).
  20. Bondarenko M, Brar SS, Sakiyama K, et al. Post-Deployment Reliability Monitoring of Clinically Deployed LLM-Generated Radiology Indications. NeurIPS 2026, Third Workshop on GenAI for Health: Agentic Systems, Clinical Trust, and Future Potential. OpenReview a9FqM2hQRz. Abstract only; authors as listed on the neurips.cc page.
  21. PhysioNet. Use of MIMIC Data with Large Language Models and Online Services. 24 September 2025. https://physionet.org/news/post/llm-responsible-use/
  22. PhysioNet. Responsible use of MIMIC data with online services like GPT. 18 April 2023. https://physionet.org/news/post/gpt-responsible-use
  23. Rao VM, Zhang S, Acosta JN, Adithan S, Rajpurkar P. ReXErr-v1: Clinically Meaningful Chest X-Ray Report Errors Derived from MIMIC-CXR (version 1.0.0). PhysioNet, 19 March 2025. DOI 10.13026/9dns-vd94. Paper: arXiv 2409.10829.
  24. Medical Device Coordination Group. MDCG 2019-11 Rev.1, Guidance on Qualification and Classification of Software in Regulation (EU) 2017/745 and Regulation (EU) 2017/746. European Commission.
  25. Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence, Article 2(6).
  26. Soons LJ. Collation and the microscope: how agentic stemmatics and interpretability could check each other. Research note, version 1.0, 16 October 2026. Zenodo. Version DOI 10.5281/zenodo.23424154; concept DOI 10.5281/zenodo.23424153.