README.md

2.3 Names list: facts as a file (English edition)

English twin of experiments/2_3_names/. Same design, English corpus.

How to run

op run --env-file experiments/aib.env -- .venv/bin/python experiments_en/2_3_names/run.py   # calls the API, skips existing out_*.json
.venv/bin/python experiments_en/2_3_names/run.py --score                                     # re-score only, no API

API: AI Builder Space. Models: deepseek-v4.1-flash and gemini-3.8-flash, temperature 0, one run each (as in zh). Input: corpus/transcripts_grok.json (Grok ASR, language=en, no vocabulary). Each of the 38 memos is corrected in its own fresh call. Two conditions:

• no_list: "Below is an automatic transcript of a voice memo. It may contain recognition errors. Fix the recognition errors and output only the corrected full text, with no explanation."
• with_list: the same, plus "These are the people and terms I mention often. Wherever the transcript sounds like one of them, use the spelling here:" and the contents of names.md (9 people with roles, then one line of 12 common terms).

Scoring uses corpus/score_names.py over the 28 W+L memos: 31 name mentions, 22 term mentions, case-insensitive whole words, correct count per memo = min(script, output). Case-sensitive term counts are reported alongside.

Files: names.md (the list used), out_<model>_<cond>.json (raw outputs), diffs_<model>_<cond>.md (every word-level change the model made, transcript → output), results.json (scores, plus a hand_check entry, see below).

Results (2026-10-14)

Hand check on deepseek with list. The scorer counts mentions per memo. In W07 deepseek left "Source says the Koyeb bill…" as is and turned the later "I asked her" into "I asked Saoirse", so the scorer gives it credit. The spoken mention is still wrong, so the strict count is 24/31. A pronoun-count check over every output found no other case. Use 24.

Without a list, no model can recover the spelling variants

Jaxon (heard as Jackson, 5 mentions) and Kaitlynn (Caitlin or Kaitlyn, 3 mentions) are real, common names spelled a less common way. Neither model changed any of the 8 mentions without the list. With the list, both models fixed all 8.

Opaque names: gemini knows some Irish names, and guesses wrong just as confidently

Without the list, gemini's 7 correct names are Priya ×4 ("Prea" → Priya in W01; the other 3 were already right), Aoife ×2 ("Oifi" → Aoife in W10 and W13) and Saoirse ×1 ("Sorsa" → Saoirse in W01). In the same run it also wrote "Eva asked me today" (W03), "Sarah says the Koyeb bill" (W07), "So, Sheetal sent over the new design mocks" (W05), "Mara's coming over this weekend" (L01) and "remind the illustrator" (W20). deepseek fixed no names at all without the list.

Confident wrong substitutions without the list (verbatim)

• deepseek-v4.1-flash, W01. Transcript: "the service is sitting on Coyab right now … spin up ValM on our own box … with Prea first". Output: "I mean, the service is sitting on Kubernetes right now, and she says the cold start is way too slow, so she wants to spin up a VM on our own box. I said, let's not rush it—next Wednesday we'll look at the KV cache hit rate with Prometheus first and then decide." Koyeb → Kubernetes is the English counterpart of the zh Koyeb → K8s. The partner engineer Priya also became Prometheus, the monitoring tool.
• deepseek-v4.1-flash, W07: "Source says the Coyab bill went up" → "The Cloud bill went up a lot this month" (the speaker is deleted and Koyeb becomes "Cloud").
• deepseek-v4.1-flash: W02 "training Elora" → "training a model"; W13 "Oifi asked" → "Yifei asked"; W10 "Oifi got her transcription script working" → "If I got her transcription script working"; W04 and W12 Kevi/Kevish → "Kevin"; W20 "remind the solicitor" → "remind the developer".
• gemini-3.8-flash, W01: "sitting on Coyab" → "the service is sitting on Colab right now". In W07 it did fix Coyab → Koyeb.
• gemini-3.8-flash, W05: "the AI builder space brand color" → "the AI Builder's base brand color".

With the list, both models wrote Koyeb, vLLM and LoRA correctly everywhere (22/22, case-sensitive too).

What the list doesn't fix (deepseek with list)

The strict misses are the ASR forms furthest from the name, or forms that are ordinary English words or names: W03 "If you ask me today" (Aoife disappears into the sentence), W07 "Source", W14 "Sochadel", W18 "Ify's", L02 "Ted", L04 "Merritt". One more is wrong in a new way: W20 "remind the solicitor" → "remind Saoirse" (the right list, the wrong person; it is the designer, Xochitl). gemini fixed every one, including W03 ("Aoife asked me today") and W20 ("remind Xochitl").

Side effect worth knowing: names.md lists "terminal" among the common terms (the 2.6 seed, mirroring Apple in the zh list), and deepseek with the list changed "kick off both from the shell" to "from the terminal" in W03.

Cost

$0.42 (152 chat calls), logged in experiments_en/COSTS.md.