Read The Whole Page.

01The problem

Two bad options, and the one nobody ships

A page is rendered for eyes. An agent gets handed either the source it was built from or a picture of the result, and neither is the thing it needs. The gap between those two columns is the entire product.

02The read pipeline

Five stages, one document out

A page read halfway through rendering is the most common source of garbage output, so settling is explicit rather than a fixed sleep. After that the pipeline is a funnel: take everything the DOM can prove, work out what is left over, and pay for pixels only there. Cover is the stage that makes the rest cheap.

03Cover, the stage that decides

OCR only what the DOM cannot explain

Running OCR over a whole page is slow, and worse than slow: an OCR guess would overwrite text the DOM already knows exactly. So Troy screenshots the settled page and asks one question of every painted region: can the DOM account for this? A region becomes a gap when it is painted, big enough to matter, and is a canvas, an image or video with no accessible text, an embedded PDF, a cross-origin iframe, or an element with real painted area whose computed text is empty and whose crop has high edge density. That last test is what keeps photographs and flat panels out of the OCR queue. Pick a page below and step through it.

04The engines

One interface, three backends, no choice to make

Every backend answers the same two questions: are you available, and what lines do you see in this PNG. Troy picks the best one present on the host, so a script written on a Mac runs unchanged in Linux CI. The one exception is --deep, and that is a request for comprehension rather than a request for a particular vendor.

interface OcrEngine {
  name: string
  available(): Promise<boolean>
  recognize(png: Buffer): Promise<OcrLine[]>
}

read in 1.34 s · 3 regions OCR'd · AppleVision

05The action layer

Nothing is assumed to have worked

Reading makes acting better: a block's selector comes out of the same extraction pass, so troy read --json and troy click talk about the same page in the same terms. Every action then checks itself, because "the call returned" and "the page changed" are different claims.

troy fill "#po_number" "PO-4471"
  wrote · blurred · read back "PO-4471"   ok

troy fill "#po_number" "PO-4471-EXTENDED-REFERENCE"
  refused: 26 chars over maxlength 24

troy click "button[type=submit]"
  refused: submit control

troy fill "#password" "..."
  refused: password field

06A correctness rule

Text a human cannot see is not page content

Extraction marks a block invisible when it is display:none, zero sized, visibility:hidden, fully transparent, or clipped out of view. Those blocks are still extracted, because knowing they exist is useful, and they are kept out of the rendered document. Hidden text is a known way to poison a scraper, and printing it as page content would be a bug with real consequences the moment an agent acts on what it read.

<p style="clip-path:inset(50%)">
  Settled in full, no balance due.
</p>

Balance due 4,932.40          ocr

1 block excluded: visually hidden

07Sessions

Zero setup by default, logged in when you need it

One Session interface, three ways to get one. Everything downstream is mode agnostic, so reading and acting behave identically whether Troy launched the browser or merely joined it.

08State of the build

Two milestones down, six to go

Listing the plan with honest markers is more useful than a progress bar that starts at forty percent, so here is the order the work lands in and where it has actually got to.

• M1Scaffold. Repo, TypeScript project, the Cdp port headless plus attach, CI on macos, windows and ubuntu.done
• B1The browser. A real Chromium window under Troy's own chrome: tabs, an omnibox that refuses javascript: and friends in code, failure pages that keep the address you asked for, crash recovery, a new tab page, and twenty-seven cases driving the actual app on macOS, Windows and Linux. Packaged as a DMG and an EXE.done
• M2Extract and render. troy read works end to end on DOM-only pages.next
• M3Engines. The OCR interface, the Apple Vision helper, the Tesseract backend.not started
• M4Cover and fuse. The differentiator lands and the fixture suite passes.not started
• M5Actions. The existing action layer ported to TypeScript, behaviour unchanged, its tests with it.not started
• M6Polish. --deep, docs, the Claude Code plugin, npm publish.not started
• B2The bridge. The read pipeline wired into the live tab, so an agent drives the window you are already signed into.not started

Out of scope for v1, deliberately

• File upload. Reading and driving first; handing files to a page is a v1.1 candidate.
• Session recording and replay, and resuming a multi-step flow after a crash.
• More site targets. Two exist, ycombinator and generic, and neither grows for v1.
• Any browser other than Chromium. One engine, done properly.

09Where it lands

What you can do about it today