# Agent-first Writing Editor
> A layered writing system that turns dense agent output into prose people can absorb without erasing the author's voice
- HTML version: https://robbiepalmer.me/projects/agent-first-writing
- Status: in_progress
- Started: 2026-09-04
- Updated: 2026-09-30
- Technologies: Python, PyTorch, Hugging Face, DVC, Vale, jsdiff, ModelPack
- Built on: Governance, Base, Data, Python
- Platform technologies: GitHub, Mise, Copier, Python, Claude Code, Codex, CodeRabbit, Greptile, Agentic Code Review, GitHub Actions, Renovate, actionlint, zizmor, Trivy, OpenSSF Scorecard, Gitleaks, SonarQube, typos, Doppler, uv, Ruff, pytest, Pydantic, pytest-cases, DVC
- Platform policies: Public source, AGPL-3.0, Shared personal-project monorepo
- Platform overrides: Python for project.primary-language ([agent-first-writing:009-primary-language-python](https://robbiepalmer.me/projects/agent-first-writing/adrs/009-primary-language-python.md))
Read the shorter [project pitch deck](https://robbiepalmer.me/projects/agent-first-writing/deck.md).
# Vision
Make the final writing artifact as strong as the research and thinking that
produced it.
Long agent sessions excel at research, discovery, criticism, and working through
half-formed ideas. Their raw output often proves miserable to
share. It repeats itself, flattens the author's voice, over-explains simple
points, and asks readers to spend more energy than the ideas deserve. People who
read a lot of AI writing become overwhelmed and demoralized by it.
Writing should teach, motivate, and leave the reader with energy. Rewriting the
whole thing by hand should not be the only reliable way to get there.
# Product thesis
Grammarly combines different levels of intervention. It can
flag a local error, suggest a sentence, rewrite a passage, or let the writer
ignore it. An agent-first editor should do the same while accepting much larger
and messier inputs, including research sessions, plans, ADRs, and drafts built
by multiple models.
The system should build quality in layers:
1. Fast deterministic rules catch known problems and give exact feedback.
2. Lightweight models classify style problems and propose local corrections.
3. More capable models handle changes that need wider context.
4. A human accepts individual suggestions, reviews a readable diff, or approves
a confident rewrite according to the risk and type of artifact.
This follows the same pattern as agentic code review. Cheap checks handle what
they can prove. Expensive judgment is reserved for the remaining uncertainty.
# Current state
The Personal Knowledge Graph runs Vale against changed prose. Its custom rules target the
patterns that make agent writing tiring to read. This makes agent-authored work
more bearable and catches problems before review.
I tested GECToR-2024 by running ADRs through it. It found only insubstantial
corrections and did not close the gap between a technically correct draft and
something I wanted to publish. That negative result narrows the problem. Generic
grammatical error correction is not enough.
The shared `writing-editor-domain` package defines records for findings,
suggestions, grouped proposals, and editorial decisions. Every record refers to
hashed source bytes through a half-open UTF-8 span. Stable IDs retain producer
and revision provenance as records pass through review and evaluation.
The DVC evaluation pipeline extracts pinned source and published revisions,
freezes train, validation, and holdout membership, and runs Vale through the
shared finding contract. It aligns findings with observed published edits while
keeping boundary-crossing cases separate for manual adjudication. Git can show
that prose changed, but it cannot turn a detection-only finding into an
accepted suggestion. Scoring therefore waits for explicit editorial outcomes
and a reproducible rewrite baseline.
Grammarly remains the practical baseline for the final mile, but it does not
provide the API needed for an agentic publishing system. Its published research
and descriptions of its correction approach provide useful prior work without
making its private product reproducible.
# Product behavior
The editor should support three levels of control:
* Inline suggestions for local problems, each with a source span, proposed
replacement, reason, and confidence
* A proposed rewrite shown as a diff when changes depend on one another
* Automatic rewriting for narrow, high-confidence rules, followed by final
review when the artifact will be published under somebody's name
The user should be able to move between them. A deterministic punctuation fix
does not need the same ceremony as restructuring an argument.
## Editorial data flow
The implemented path runs repository text through Vale and stores its
detection-only findings against hashed source bytes. The evaluation pipeline
can compare those findings with published edits, but an observed edit is not an
acceptance decision. The suggestion, rewrite, review, and outcome path remains
planned. Solid blue nodes and lines are implemented. Dashed orange nodes and
lines are planned.
```mermaid
flowchart LR
Source["Source text
[Implemented]
Revision and SHA-256 hash"]
Rules["Deterministic checks
[Implemented]
Vale rules"]
Findings["Deterministic findings
[Implemented]
UTF-8 spans and provenance"]
Local["Local suggestions
[Planned]
Replacement and confidence"]
Rewrite["Passage rewrites
[Planned]
Grouped proposal and diff"]
Review["Human review
[Planned editor stage]
Accept, reject, or change"]
Evidence["Evaluation evidence
[Implemented foundation]
Published-edit alignment"]
Outcomes["Decision-backed evidence
[Planned]
Quality and review-time metrics"]
Source -->|"exact source bytes"| Rules
Rules -->|"detection-only records"| Findings
Findings -->|"finding IDs and published diffs"| Evidence
Source -.->|"bounded source spans"| Local
Source -.->|"wider document context"| Rewrite
Local -.->|"one-suggestion proposals"| Review
Rewrite -.->|"multi-edit proposals"| Review
Review -.->|"recorded editorial decisions"| Outcomes
Evidence -.->|"baseline comparisons"| Outcomes
classDef implemented fill:#dbeafe,stroke:#2563eb,color:#1e3a5f,stroke-width:2px
classDef planned fill:#ffedd5,stroke:#ea580c,color:#7c2d12,stroke-width:2px,stroke-dasharray:5 5
class Source,Rules,Findings,Evidence implemented
class Local,Rewrite,Review,Outcomes planned
linkStyle 3,4,5,6,7,8 stroke:#ea580c,stroke-width:2px,stroke-dasharray:5 5
```
Read left to right, with two different records leaving analysis. A rule that
can only locate a problem produces a finding and goes straight to evaluation.
A producer that can supply a replacement will create a local suggestion or a
grouped passage rewrite for review. The editor will preserve the proposal and
the person's accept, reject, or change decision as evaluation evidence. No
planned stage may publish in the author's name without that review.
# Evaluation data
The useful dataset should capture more than a generic collection of
grammatically correct text. It records the decisions made while turning rough
work into published work:
* raw notes and agent output;
* early drafts and later revisions in git;
* Vale findings;
* Grammarly suggestions where they can be captured legitimately;
* accepted and rejected edits; and
* the final published artifact.
This can test whether an edit improves clarity while preserving facts, intent,
terminology, and voice. Acceptance rate supplies evidence. A system that
makes timid edits could achieve a high acceptance rate while adding little
value.
# First useful release
The first release should work on content in this repository before becoming a
general editor:
1. Define a versioned suggestion format with source spans, replacements,
categories, reasons, confidence, and model or rule provenance.
2. Build an evaluation set from ADR and project-page revisions, including the
GECToR-2024 result as a baseline.
3. Measure factual preservation, unwanted voice changes, accepted edits, time
saved, and regressions caught by deterministic rules.
4. Run Vale and other deterministic checks through the shared finding format.
5. Add a lightweight model only for failure classes that the evaluation set
proves Vale cannot handle.
6. Add passage rewrites with diffs after local suggestions are reliable.
7. Capture accept and reject decisions so the evaluation set grows through use.
8. Extract the repository-specific implementation into a general product only
after it improves real published pages.
# Relationship to the knowledge graph
The knowledge graph supplies the context that generic writing tools lack. It
can tell the editor which project a draft describes, which ADR contains the
source decision, what terminology the author uses, and which claims need fresh
evidence. The editor returns proposed public artifacts and structured edit
history. It does not decide what is true or publish in the author's name without
the authority to do so.
# Success
The editor succeeds when people prefer reading its output to the raw agent
draft, authors keep their voice, and publishing takes less manual rewriting.
It fails if it merely produces more polished AI prose with the same cognitive
load underneath.
# Non-goals
* Replacing research, fact checking, or editorial judgment
* Training one universal style into every author
* Treating grammatical correctness as the whole measure of quality
* Automatically publishing essays under a person's name
* Adding a large model to problems a deterministic rule can solve
## Initiatives
- [Semi-autonomous Software Development](https://robbiepalmer.me/initiatives/semi-autonomous-software-development.md): A connected system that keeps useful work moving, asks for human judgment when it matters, and learns how to need less intervention next time
## Architecture Decision Records
- [ADR 000: Vale for deterministic prose checks](https://robbiepalmer.me/projects/agent-first-writing/adrs/000-vale-for-deterministic-prose-checks.md) — Accepted, 2026-09-04
- [ADR 001: Generic grammar correction with GECToR-2024](https://robbiepalmer.me/projects/agent-first-writing/adrs/001-generic-grammar-correction-with-gector.md) — Rejected, 2026-09-04
- [ADR 002: Grammarly as an automated editing backend](https://robbiepalmer.me/projects/agent-first-writing/adrs/002-grammarly-as-an-automated-backend.md) — Rejected, 2026-09-04
- [ADR 003: Versioned editor record contract](https://robbiepalmer.me/projects/agent-first-writing/adrs/003-versioned-editor-record-contract.md) — Accepted, 2026-09-04
- [ADR 004: DVC writing evaluation pipeline](https://robbiepalmer.me/projects/agent-first-writing/adrs/004-dvc-writing-evaluation-pipeline.md) — Proposed, 2026-09-06
- [ADR 005: Repository-first, evidence-gated release](https://robbiepalmer.me/projects/agent-first-writing/adrs/005-repository-first-evidence-gated-release.md) — Proposed, 2026-09-04
- [ADR 006: Knowledge graph context with bounded authority](https://robbiepalmer.me/projects/agent-first-writing/adrs/006-knowledge-graph-context-with-bounded-authority.md) — Proposed, 2026-09-04
- [ADR 007: Playwright-assisted Grammarly handoff](https://robbiepalmer.me/projects/agent-first-writing/adrs/007-playwright-assisted-grammarly-handoff.md) — Proposed, 2026-09-06
- [ADR 008: jsdiff source alignment without outcome inference](https://robbiepalmer.me/projects/agent-first-writing/adrs/008-jsdiff-source-alignment-without-outcome-inference.md) — Accepted, 2026-09-11
- [ADR 009: Python as the primary language](https://robbiepalmer.me/projects/agent-first-writing/adrs/009-primary-language-python.md) — Accepted, 2026-09-12
- [ADR 010: ModelPack-compatible model metadata](https://robbiepalmer.me/projects/agent-first-writing/adrs/010-modelpack-compatible-model-metadata.md) — Accepted, 2026-09-14
- [ADR 011: DVC model storage with portable package boundaries](https://robbiepalmer.me/projects/agent-first-writing/adrs/011-dvc-model-storage-with-portable-package-boundaries.md) — Accepted, 2026-09-14
---
Markdown index of this site: https://robbiepalmer.me/llms.txt