DocuDebug

Editing paragraph 1 to say bulls don't hate red updates the semantic graph: paragraph 1 now contradicts its cited source, and paragraph 3, which argued 'because bulls hate red', loses its premise. The editor shows a red Broken Evidence error on paragraph 1 and a yellow Logic Leap warning on paragraph 3. — **Edit, update the graph, debug the document.** The user edits paragraph 1 (“Bulls **don’t** hate red”). DocuDebug updates its semantic graph: paragraph 1 now contradicts its own source, and paragraph 3 has lost its premise. The editor surfaces both as diagnostics on the text, a red **Broken Evidence** error and a yellow **Logic Leap** warning.

Abstract

While large language models (LLMs) accelerate report generation, maintaining logical consistency during revisions increases the cognitive load, and delegating this task to LLMs often results in unpredictable global overwrites. To address this, we present DocuDebug, an interactive editor that reframes document verification as a software engineering task. The system models a document’s logical structure and citations as an explicit dependency graph. Through a linter-like interface, users receive real-time warnings for logical conflicts and propagate localized edits with guided precision. In a within-subjects study with 12 professional writers, DocuDebug guided precise, necessary corrections, improving precision over threefold and reducing over-editing from over 40% to under 5%. Furthermore, objective logs revealed that users avoided blind acceptance, actively deliberating over each warning for nearly three minutes. This work demonstrates how structured visual diagnostics can shift AI-assisted writing from opaque global generation to systematic, user-controlled co-creation.

Video

One edit, many breaks

Six hours before a deadline, a co-author writes: “Our main result is no longer significant.” Fixing the Results section takes a minute. But that one change breaks paragraphs pages away, in the abstract, the introduction, the discussion and the conclusion, and some of them never mention the result directly.

Ctrl+F finds words, not dependencies, so it misses claims that rely on the result without sharing a keyword. Asking an AI to update the paper rewrites paragraphs everywhere, including ones that were fine, and now all of it needs review. In our study, writers using Google Docs with Gemini edited 40.8% of the paragraphs that needed no change.

The idea: debug text like code

Delete an import and the IDE flags every line that uses it. Documents work the same way.

analysis.pyIDE

~~import study_v1~~ # removed

faster = study_v1.significant

def abstract(): return faster

def conclusion(): return faster

error 'study_v1' is not defined (2 dependents)

report.mdDocuDebug

source~~Analysis v1~~ disabled

claim“Our tool was significantly faster [1].”

¶ AbstractWe show the tool speeds up writing…

¶ ConclusionWriters finished sooner with it…

Broken Evidence cited source was disabled

import→citation

variable→definition

function→paragraph

DocuDebug builds a semantic dependency graph, the document’s equivalent of an abstract syntax tree, and runs a logic linter over it. Both dependent paragraphs are flagged, even the one that never says “significant.”

Building the graph

Nodes are sources and paragraphs, the smallest unit that fully states a claim. Edges follow the taxonomy of computational argumentation: Supports (paragraph → source), Depends on a term or on another paragraph’s conclusion, and Contradicts. To avoid asking a large model about every pair of paragraphs, cheap stages narrow the candidates first.

Pipeline: paragraphs pass through a lexical filter and an embedding-based semantic filter, whose results are merged and checked by a small LLM candidate generator; the remaining pairs go to the edge generator LLM. A term extractor feeds term context to the edge generator. Sources are linked to paragraphs by a rule-based edge generator. The output is a semantic graph with paragraph-source, term-paragraph, and paragraph-paragraph edges. — **Graph construction pipeline.** Orange traces one paragraph from the text to its node in the semantic graph.

01 · RULES

Citation edges

Citation markers like “[1]” link a paragraph to its source with no LLM call.

02 · LLM

Term extraction

Concepts each paragraph defines or uses, which also powers “Go to Definition.”

03 · FILTERS

Cheap candidates

Rare-term overlap and embedding similarity in parallel, then a small LLM’s yes/no check.

04 · LLM

Edge labels

The main LLM labels each surviving pair and writes the explanation shown in the editor.

Logic Slicing: recompute only what an action touches

Rebuilding the whole graph on every edit would freeze the editor. A typo fix stops after a quick change classification. A real edit compares only the edited paragraph with the others and re-verifies its existing edges, which are kept on purpose so a broken premise surfaces as an error instead of silently disappearing. Disabling a source re-checks only the paragraphs that cite it.

~80% fewer LLM calls than checking every pair 0.1–2 s to re-check after toggling a source 90% edge recall for the initial graph

The logic linter

Diagnostics appear as squiggles on the text, like in an IDE. DocuDebug keeps two kinds strictly apart. Every diagnostic is a suggestion: the author makes the final decision.

Errors: deterministic

Found by graph traversal and regular expressions, with no LLM: a cited source was disabled, or a paragraph that defines a term or premise was deleted.

Broken EvidenceBroken LogicBroken Context

Warnings: probabilistic

An LLM reads the paragraph with its linked premises and sources from the graph and checks whether the argument still holds.

Weak EvidenceLogic LeapOvergeneralizationContext Mismatch

Disable a source
Find Citation
Edit a paragraph
Weaken Claim

The Evolution of Sports✓ 0 diagnostics

Para 4

Ancient Greece formalized athletic competition: the first recorded Olympic Games were held in 776 BC [1].

Para 5

HistoryEditDelete✦ Reinforce with AI

✏️ Editing💾 Save✖ Cancel

Sports grew out of ritual, hunting and military training long before the modern era [2]. Early contests were local and informal.

Para 6

Organized leagues with written rules first appeared in 19th-century Britain [1].

Sources

Enabled

[1] The Ancient Olympic GamesA

Enabled

[2] The Evolution of Sports and Their Societal ImpactC

A scripted re-creation of the prototype’s interface. In the real system the graph update and the LLM check take a few seconds.

DocuDebug main interface: paragraphs as blocks on the left with History, Edit, Delete and Reinforce with AI buttons; on the right, the reference list with source credibility ranks and enable toggles for each source. — **Main interface.** Paragraph blocks on the left; the Source Panel on the right ranks each source’s credibility (S–D) and lets the author toggle it.

Reinforce with AI: a before-and-after view of paragraph 5, two new sources to be added, and a Diagnostic Preview saying all warnings in the target paragraph will be resolved and no new issues appear in 4 downstream paragraphs. — **Reinforce with AI and Diagnostic Preview.** Before anything changes, the author sees the diff, any new sources, and the predicted linter result for the paragraph and everything downstream.

Weaken Claim

Adds hedging that matches the evidence the graph links to.

Find Citation

Searches for missing evidence, appends the citation and a Supports edge.

Approve As-Is

Suppresses the diagnostic without touching the text or the graph.

Reinforce with AI

An optional, larger rewrite within the paragraph’s graph context, always previewed first.

User study

12 professional writers and editors (3–20 years of experience) used both DocuDebug and Google Docs with Gemini in a counterbalanced within-subjects design. Modeled on think-tank reports, the designated task asked them to disable an outdated source, insert a new premise, and resolve every resulting conflict with minimal edits; a free-form task asked them to change the report’s tone while keeping it consistent. Edits were scored against a reference set of paragraphs that truly needed changes.

40.8% → 2.8%

of unaffected paragraphs edited anyway, Baseline → DocuDebug (p = .002)

94%

of 84 diagnostics resolved within the 30-minute limit

169 s

median review before suppressing a warning; none were dismissed within 10 s

Edits stayed where they belonged

Mean per participant. Wilcoxon signed-rank, BH-corrected.

Google Docs + GeminiDocuDebug

Writers resolved warnings, not clicked them away

Final resolution of the 84 diagnostics (52 errors, 32 warnings).

The one-click Approve As-Is, the easiest way to clear a squiggle, ended only 9 of 84. Reinforce with AI was never the final resolution: writers preferred local, transparent fixes.

Workload (NASA-TLX) was similar for both tools (64.0 vs 61.3, n.s.), so the visual diagnostics added no measurable mental demand. DocuDebug scored clearly higher in hedonic quality (UEQ-S, p < .01), while pragmatic quality and felt accountability were somewhat lower. Precision and source accuracy roughly doubled, but only as trends after correction; recall went down.

What we learned

Constrain AI with structure

Writers don’t need more generation under a deadline; they need to see what one change breaks. Build data structures that keep AI edits local and predictable.

A “compiler for text”

Participants felt less accountable (I lack confidence in whether I am truly ‘managing’ the document myself. P04), yet the logs show careful verification. Offloading dependency tracking is fine while the writer stays the decision-maker.

Squiggles anchor attention

Recall fell from 63.5% to 36.5%: unflagged meant “assumed fine.” My attention was completely taken by clearing the AI’s judgments. (P10) Next: ambient cues, on-demand validation, editable graph edges.