chopshopr

Reasoning Quality Lab

Human methods make better Agent Skills.

Proof by inversion, pigeonhole routing, Six Sigma defect control, checklists, threat models, and deliberate practice become package moves: triggers, references, scripts, evals, and receipts.

By Chopshopr 9 min read
Opinions Human methods make better Agent Skills.
Join newsletter

The Agent Skills spec is intentionally small: a folder, a `SKILL.md`, and optional scripts, references, assets, resources, and evals. That simplicity gives us a place to encode the best human operating methods without turning the context window into a dump.

The lab rule:

Every reasoning method must change the skill package, or it does not count.

What transfers cleanly

01

Proof systems become gates.

Inversion creates negative evals. Contradiction catches impossible rules. Invariants become audit checks. Pigeonhole reasoning exposes routing collisions.

02

Factory quality becomes control.

Six Sigma DMAIC turns recurring defects into measured loops. FMEA ranks failure modes. Poka-yoke makes predictable mistakes hard to perform.

03

Operations become state discipline.

OODA forces fresh observation. SRE error budgets define stop rules. Incident reviews convert failures into scripts, tests, and smaller surfaces.

04

Learning science becomes eval design.

Worked examples show the pattern. Deliberate practice creates harder reps. Spaced review keeps useful conventions from evaporating.

The SQLite-backed demo

The new lab skill carries a committed SQLite database at `skills/reasoning-quality-lab/assets/methods-lab.sqlite`. The database is generated by the skill script, audited by the same package, and exported into a static JSON snapshot for the public browser lab.

  • 51 methods across proof, algorithms, science, quality, operations, safety, law, medicine, learning, strategy, and control.
  • Each method has a failure mode, Agent Skills package move, and evidence gate.
  • The script can search, inspect, score a draft `SKILL.md`, export public lab data, and audit the package.
  • The public lab exposes `window.render_reasoning_quality_lab_to_text()` for automation.

A concrete example

Suppose a skill says it improves quality but has no proof gate. The lab maps that defect to DMAIC, burden of proof, replication, and checklists. The package moves are concrete: define the defect, add a measurement, write an audit command, add a replay receipt, and block completion when the audit fails.

Why it matters

Skills are small software packages for agent behavior. The best ones should inherit the methods humans already use when failure is expensive: proof, checklists, controlled experiments, defect control, adversarial review, and practice loops.