README.md

2.5 Perspectives: a Skill decides how to write, Perspectives decide what to judge

English edition of the Chinese experiment experiments/2_9_neutralize/. The design, the models, the counts and the cross-vendor judge are the same; the skill, the 10 perspectives, the 10 proposals and the judge prompt are written natively in English, with the English cast (Jaxon, Saoirse, Kaitlynn, Aoife, Priya).

How to run

op run --env-file experiments/aib.env -- .venv/bin/python experiments_en/2_5_perspectives/run.py          # main experiment -> results.json
op run --env-file experiments/aib.env -- .venv/bin/python experiments_en/2_5_perspectives/run.py rep      # identical replicate -> results_rep.json
op run --env-file experiments/aib.env -- .venv/bin/python experiments_en/2_5_perspectives/run_hedge.py    # hedge pilot -> results_hedge.json
op run --env-file experiments/aib.env -- .venv/bin/python experiments_en/2_5_perspectives/run_hedge.py attack_rep   # 2nd run of the early "attack step" wording
.venv/bin/python experiments_en/2_5_perspectives/summarize.py   # summary block in results.json + chart data

All calls go through the AI Builder Space API. Models: deepseek-v4.1-flash and gemini-3.8-flash. Each run appends its cost to experiments_en/COSTS.md. Total for this folder: about $0.93.

Main experiment: skill only vs skill plus perspectives.md

Both conditions get the exact same writing skill (# Skill: evaluation note: verdict first, 2–3 sentences of reasons, 1–2 next steps, under 120 words). The only change is whether perspectives.md is in the context. That file holds 10 principles distilled from the idea memos I01–I10.

The test set is 10 proposals (Q01–Q10). Each follows the mainstream industry approach and collides head-on with one of the 10 principles: a framework survey, a prompt rewrite, a four-role agent pipeline, chunked vector RAG over 150 Markdown specs, agent-written TDD, LoRA fine-tuning for company knowledge, silent automatic memory, per-step approval, a month more of features, and choosing inference by decode tok/s.

Each writer runs at temperature 0 and writes one note per proposal per condition. The other vendor's model judges each note against the target principle: deepseek is judged by gemini, and gemini by deepseek. The judge answers two questions. endorsed_mainstream: did the note go along with the mainstream direction, including "agree with conditions"? hit_perspective: did it reject the core premise with the author's reason and alternative?

The replicate run (results_rep.json, identical code) gave deepseek 4/10 and 5/10, and gemini 4/10 and 3/10, for skill only. With perspectives.md it gave 0/10 and 10/10 for both models. The skill-only numbers move by a point between runs even at temperature 0, and the proposals that flip are different ones. The with-perspectives result does not move.

Across the 4 skill-only notes per proposal (2 models × 2 runs), the proposals split into two groups:

• Q09 (stop building and find users) and Q10 (prefill, not decode tok/s) got the non-consensus call 4/4 even without perspectives.md. Q01 (framework survey) got it 3/4 and Q08 (per-step approval) 2/4. In English these positions are already common advice.
• Q04 (chunked vector RAG) was endorsed 4/4: always "agree with conditions" plus hybrid BM25, header-aware chunking and reranking. Q03 (role-play pipeline), Q05 (agent-written TDD) and Q07 (silent memory) were endorsed 3/4 each.

Hedge pilot: turning an idea memo into a note

Mirrors experiments/2_9_neutralize/run_v2_neutralize.py. The input is the grok transcripts of I01–I10 (experiments_en/corpus/transcripts_grok.json).

• default: "Turn this voice memo into a note." Run twice.
• perspective: the same ask plus the idea-note rules: keep the stance and its strength, add no qualifiers or counterpoints I didn't say, critique is a separate step, and this step writes only the note.
• attack_v1: the early wording, whose last rule is "Where I'm wrong, leave it for a separate attack step." Run twice.

The cross-vendor judge counts the qualifiers and counterpoints each note adds (hedges) and whether the note weakens the claim (weakened).

By default, deepseek also pulled in outside research in 7 of 10 notes both times: "Supporting evidence" and "Sources" sections with links. It added none under the rules.

The 1 hedge and 2 "weakened" flags in deepseek's perspective condition are judge false positives. Those notes (I01, I08) are the transcript almost verbatim, with "I think" in front, which the rules asked for.

Section headings deepseek added by default (English twins of the Chinese "待验证 / 补充" and "边界条件（重要，别一刀切）"):

• I02 (prompt engineering is dead):
  • run 1: ## Where this holds up — and where it's overstated, with a bold **Overstated:** list opening "'Dead' is the wrong word — 'reclassified' is closer." It also says "The consensus framing is that prompt engineering is a subset of context engineering, not a rival."
  • run 2: ## Where the claim is too strong and ## Caveats.
• I04 (naive RAG is useless for agents):
  • run 1: ## Where the take needs a caveat, opening "The 'RAG is dead' framing is overstated — it's really 'RAG is the wrong default for agents over structured/small corpora.'" It then adds five counterpoints and ends with "Hybrid is the pragmatic answer."
  • run 2: ## Caveats / Nuance.
• Others: ## The counterargument (worth engaging with) (I05), **Caveats / where the memo overstates:** (I06), ### Counterpoint worth holding (I09), ## Caveats worth keeping in view (I03), ## Where the claim needs nuance (I06), ## One caveat worth keeping (I10).

The "separate attack step" wording

In the Chinese v1 run, the rule 我说得不对的地方，留给单独的攻击环节 made gemini append an "攻击环节" rebuttal section to all 10 notes (the judge counted 27 added hedges). Deepseek wrote no rebuttal, but in 5 of 10 notes it listed points "for the attack step later". In English this mostly does not reproduce. Over 2 runs × 10 memos × 2 models (40 notes), one note has an in-note rebuttal: gemini, run 1, I06, ### Attack Step (Critiques & Counterpoints), with three counterpoints for fine-tuning. The judge scored that note 0 hedges, which it missed. Deepseek once acknowledged the step without writing it: "Ready for the attack step whenever you are." (run 2, I05).

Files

• run.py, run_hedge.py, summarize.py: the scripts.
• results.json: main run, every note with its judge verdict, plus a summary block (both runs, per-proposal stability, the hedge pilot).
• results_rep.json: the replicate of the main run.
• results_hedge.json: the hedge pilot, all conditions, with the headings of each note.
• *.log: console output.
• CONTRACT.md: the exact facts, numbers and quotes for the English text.
• Chart data: ../chart_data/ch02_05_neutralize.json.