All technical writing

06 / UNDER THE HOOD

Let Code Shortlist, Let the Model Pick

We have an internal app that keeps track of documents and money: contracts, payment requests, vendor invoices, bank statements. A lot of its work is matching. Which invoice does this bank line pay? Is this the same invoice the email talked about? Which vendor is this row in the spreadsheet?

Animated: a bank statement with a few highlighted lines, a short card of invoice choices with one ticked and a none box, a robot hand ticking the first choice and a human hand holding a rubber stamp.
The model ticks one box. A person still holds the stamp.

The first version did this with rules. A payment reference in the memo, an exact amount, a vendor name spelled the way we expected. Rules work until a memo looks a bit different. Then nothing matches, and the fix is me adding another rule. Rules can’t learn, so I kept stepping in. That’s annoying, because I’m consistently out of tokens.

Then OpenAI shipped the Decisions API, and I kept thinking about how to make use of it. I think it’s a good fit for “system one”: the fast, gut answer to a closed question, not the slow reasoning. This demo is what got me thinking:

So now the rules still run first, and for what’s left we ask a decision model.

A closed question, not extraction

POST /v1/decisions takes one input and up to 400 questions about it. Each question is a choice question with its own list of allowed answers. The answer is the picked value and a confidence between 0 and 1.

That shape is the whole point. The model can’t invent an invoice. It can only pick one we put on the list, or pick none. Here’s one bank line, with made-up data:

One bank line as one closed questionSample data. A bank line on 14 Nov 2026, money out 15,840,000 VND, memo TT HOA DON 563 CTY AN BINH. Code shortlists three invoices: invoice 563 from An Binh Trading for 15,840,000 dated 10 Nov, invoice 0091 from Song Hong Supplies for 15,900,000 dated 2 Nov, invoice 1207 from An Binh Trading for 15,600,000 dated 28 Oct, plus none. The model answers invoice 563 with confidence 0.93, which is saved as a suggestion for a person.THE BANK LINE · SAMPLE DATA14 Nov 2026 · money out · 15,840,000 VND"TT HOA DON 563 CTY AN BINH"CHOICES · CODE BUILT THIS LISTInvoice 563 · An Binh Trading15,840,000 · 10 Nov 2026✓Invoice 0091 · Song Hong Supplies15,900,000 · 2 Nov 2026Invoice 1207 · An Binh Trading15,600,000 · 28 Oct 2026None of theseTHE ANSWERchoice: "invoice-563"confidence: 0.93≥ 0.6, so it's saved as a suggestion. A person accepts it.
Made-up data. The model can only pick from the list, or say none.

Code builds the list: same direction of money, an amount close enough, dated around the line. The model reads the memo, which is often Vietnamese without accents and cut short, and picks.

An answer below 0.6 confidence is thrown away. I’d love to say 0.6 came from testing. It didn’t, I picked it. What keeps that honest is that a pick at 0.6 or above still isn’t a link. It’s saved as a suggestion, and a person accepts or rejects it.

The Workflow around it

Every document that comes in gets its own Cloudflare Workflow instance, DocumentIntake. Emails reach it through Email Routing, which hands each one to a Worker. Uploads from the app start one too.

Animated diagram of the DocumentIntake Workflow. An email or upload goes to R2 and gets a row in Postgres. A matching ignore rule ignores it with no AI call. A known format is read by an exact reader or Learned mapping with no model; an unknown one goes to the Decisions API to say what kind it is, and a model reads it and drafts a Learned mapping. E-invoice links are followed with Browser Rendering, then the Decisions API cross-checks it. If sure, it is filed. If not, the Workflow waits for a person, who saves a Learned mapping, and the document is read again.
A sure answer gets filed. An unsure one waits for a person, and their answer loops back in.

One more detail: some Vietnamese e-invoice emails don’t attach the XML, they link to a portal. For portals that only work in a browser, Browser Rendering opens the page and clicks the download button.

The waiting is where the rules learn. When the XML is in a format no reader knows, the model drafts a Learned mapping and the Workflow calls step.waitForEvent for learned-mapping-saved, for up to 7 days. A person checks the draft and saves it. The event comes in, the Workflow reads the document again with that mapping, and settles. The next invoice in that format gets read by the mapping, no model and no me.

A saved Learned mapping for an e-invoice XML format, with made-up data. On the left, each invoice field (series, number, date, amount including VAT, VAT, seller, seller tax code) points at an XML path, with a sample value. On the right, the original XML with the mapped lines highlighted and labelled.
A Learned mapping, with made-up data. Each field points at a path in the XML, and the original shows where it came from.

While it waits it doesn’t run anything. And each box is a step.do, so if the Decisions call fails, the Workflow retries that step. It doesn’t download the file again or write a second row.

Bank statements get the same idea in a BankMatch Workflow: exact references in SQL, then the Decisions call from the section above, then it emails the reviewers and waits for review-done, reminding them every 3 days, at most 3 times.

Checking a reading against every copy

An invoice often arrives as more than one copy: the email body, the PDF, and the XML from Vietnam’s e-invoice system. When we read values out of one of them, we want to know if the others agree. With made-up data:

Value read XML Email PDF
Invoice number 563 same same same
Date 10/11/2026 same absent same
Total 15,840,000 same different (15,480,000) same

Each copy goes in as its own message, and each value gets one question per copy: same, different, or not shown? With two copies or more there’s one more question: are these the same document?

It’s not one API call per field. The whole table is one Decisions call per document: each copy is a message, and every value and copy pair is its own question in the same request. For the table above that’s 9 questions plus one:

input:     [task], [copy 1: XML], [copy 2: email], [copy 3: PDF]
questions: c1_invoice_number   same | different | absent
           c2_invoice_number   same | different | absent
           c3_invoice_number   same | different | absent
           c1_date ... c3_total
           same_document       yes | no

One request can hold up to 400 questions, so a normal invoice never needs a second call. Clef takes 64 per call, so on the fallback a big one is split.

Before, it was one yes/no question per value with all the copies joined together. The model could say “no”, but we couldn’t tell which copy disagreed.

The confidence for a value comes from checkFromCopies:

const best = (says) => Math.max(0, ...sources.filter((s) => s.says === says).map((s) => s.confidence));
return { ...d, confidence: best("same") * (1 - best("different")), sources };

The surest copy that shows the value, times one minus the surest copy that shows another one. A copy that doesn’t show it is neutral, because an email often doesn’t name an amount. In the table above, the total would go to a person.

One case: the XML had invoice number 00000563. Another copy wrote 563. The model said “different”. Now each question names the other ways a value can be written, and says leading zeros, a day-first date and thousands separators don’t make it another value.

Clef as the fallback

When the OpenAI call fails, or we’re out of credits, the same questions go to Clef, Cloudflare’s decision model on Workers AI. I added it to save money, not after a benchmark. The adapter in decisions.ts joins the messages into one text, sends 64 questions per call, and records the model on each answer, so a person can see which one picked.