# 2.6 homonym pilot (English): which word replaces 苹果?

Date: 2026-10-14. Script: `pilot_homonym.py`, raw outputs in `pilot_results.json`. Text only, no audio. Models: `gemini-3.8-flash` and `deepseek-v4.1-flash`. Total cost about $0.52 (see `../COSTS.md`).

## What was tested

The Chinese 2.6 needs a word with a work sense the author restyles (苹果 → "Apple" for the company) and an everyday sense where the same restyle is absurd (the fruit). Two cells were run for each English candidate, mirroring `experiments/2_5_overgeneralize/run.py`.

- (a) Single-edit distillation. The model sees one work transcript before and after the author's single edit, and is asked to turn it into one rule for the cleanup rule file. Temperature 1, 3 runs per model in round 1 and 5 in round 2. The question is whether the rule is scoped or global. The automatic scope flag in the JSON is a keyword heuristic; the counts below were checked by hand.
- (b) Downstream damage. Two work sentences and three life sentences are cleaned up at temperature 0 under three conditions: no rule, a global rule ("always write X as Y"), and a scoped rule. The question is whether the global rule converts the life mentions.

Candidates: Amazon → AWS (three wordings of the edited memo), shell → zsh, shell → terminal, Python → python3, and target → deploy target.

## Results

Unscoped single-edit rules, counted by hand:

| candidate | gemini unscoped | deepseek unscoped |
|---|---|---|
| Amazon → AWS (3 wordings, 13 runs per model) | 0/13 | 0/13 |
| shell → zsh | 0/3 | 2/3 ("Replace "shell" with "zsh".") |
| shell → terminal | 0/5 | 2/5 ("Replace "shell" with "terminal".", "Replace "the shell" with "the terminal".") |
| Python → python3 | 0/3 | 0/3 (one run produced an unrelated capitalization rule) |
| target → deploy target | 0/3 | 0/3 (all six over-narrow: "Replace "staging target" with "staging deploy target"") |

Downstream damage under the global rule. The table counts life sentences wrongly converted, out of 3. With the scoped rule, every candidate converts 2/2 work sentences and 0/3 life sentences, except Python, where deepseek converted 1/3.

| candidate | gemini | deepseek |
|---|---|---|
| Amazon → AWS | 3/3 ("grab the AWS package from the front desk") | 3/3 (one other run refused to edit) |
| shell → zsh | 2/3 ("fill up at the zsh") | 3/3 ("a hundred zsh at the beach") |
| shell → terminal | 2/3 ("Gas at the terminal by the school") | 2/3 ("The kids collected like a hundred terminals at the beach"; one run refused, saying the rule conflicts with the content) |
| Python → python3 | 1/3 | 3/3 |
| target → deploy target | 3/3 | 3/3 |

## Decision: shell → terminal

**Amazon** gives the cleanest damage numbers: the global rule converts 3/3 life sentences for both models. Its problem is cell (a). Neither model ever wrote an unscoped rule from the single edit, in 26 runs across three wordings. Both always wrote "when it refers to cloud services". AWS-vs-Amazon is common knowledge, so the models scope it on their own, and the "a single edit becomes a global rule" beat would not reproduce. The other single-word candidates (Python, target) fail cell (a) the same way.

**shell → terminal** is the only candidate where cell (a) reproduces the Chinese pattern. deepseek writes a bare global rule in about half of its runs (4/8 across the zsh and terminal variants), while gemini scopes 8/8. That matches the Chinese result: deepseek 0/5 scoped, gemini 5/5 scoped. Cell (b) also reproduces for both models ("fill up at the terminal", "a hundred terminals at the beach"). The everyday senses are varied and natural enough to fill 9 life mentions: the Shell gas station, eggshell, taco shell, seashell, a hermit crab's shell, pistachios in the shell, and shell pasta. The author's style edit also has a plausible reason: readers see the word "Terminal" on their screen, so the book says "terminal".

## Caveats for the full 2.6 run

- **The strong model's damage is partial.** gemini spared the plural "shells" (2/3), because the rule says "shell". In the corpus, 9 of the 9 life mentions are the singular "shell" or "Shell" to keep the test clean. Expect a capital-S "Shell" (the gas station) to be converted as readily as a lowercase one: both models did.
- **deepseek sometimes refuses.** Instead of applying the absurd rule, it explains the conflict. That counts as "not converted". Report it as resistance, not as a scoped success.
- **Single-edit rules also fail by being too narrow.** Examples are deepseek's "when it refers to running a script outside the IDE" and every "staging target" rule. This is the mirror failure, a rule that misses the next legitimate case (W13: "comfortable working in the shell"). It is worth one sentence in 2.6: a single edit underdetermines the boundary in both directions.
- **Gateway quirk.** Some plain-string Gemini prompts fail deterministically with HTTP 400 "Requests ending with a model turn are not supported". Any prompt containing "Found a gotcha today." or "from now on" triggers it. Sending the same text as a list of content parts works. `experiments_en/common.chat` does this for every gemini call.