The most expensive test in a software organization is not a slow integration suite. It is a person rereading the same shape of failure every week because the system never learned to check its own constraints. The reviewer becomes the sensor, the control chart, the rollback plan, and the defect classifier. That is not engineering rigor. That is human variance wearing a process badge.
Constraints are system properties. Contract tests, property tests, complexity limits, and review packets are different ways of making those properties executable. The harness should layer those checks, then optimize correctness and quality together. If it cannot inspect them, the human reviewer becomes the harness.
Constraints are not paperwork
A constraint is a statement about what the system must keep true while it changes. "The id is unique" is a constraint. "Every artifact is versioned" is a constraint. "Every agent event links back to the job it served" is a constraint. Once you state those rules, you have more than documentation. You have raw material for a test, tracker, review packet, and production control.
Suppose a job owns many agent actions, artifacts, and decisions. If each artifact id is
unique, each revision is versioned, and each event links back to the job, the tracker
can use a composite handle like job_id:artifact_id:vNN. That handle is not
cosmetic. It lets the review packet answer: which job produced this object, which
version is under review, which agent touched it, and which property failed.
Contract test
Turns an integration promise into executable examples: this consumer expects this provider message to keep its shape.
Property test
Turns an invariant into generated pressure: this rule should remain true across many inputs, orders, ids, and edge cases.
Complexity limit
Turns maintainability into a budget: this path cannot keep gaining branches without forcing a design review.
Review packet
Turns a human review into a bounded artifact: here are the ids, properties, contracts, deltas, risks, and proof links.
Do not make one test carry the whole loop
The trap is to collapse the process into one giant code check and call that rigor. A single layer can tell you whether one slice passed, but it cannot tell you whether the system is still understandable, reviewable, compatible, recoverable, and worth keeping in this shape. The loop needs layers because each layer measures a different failure mode.
Treat the harness like an objective function with two families of signals. Correctness asks whether the system does what it promised: examples pass, contracts still match, generated properties hold, simulated pressure stays bounded, and live receipts prove the deployed path. Quality asks whether the system should be allowed to keep its current shape: complexity stays bounded, names explain intent, overload behavior is explicit, review packets are complete, and the next operator can resume without folklore.
Simulation catches pressure, not just shape
Contracts prove that two sides still agree on the shape of a message. They do not prove the system behaves when every valid message arrives too fast, the provider slows down, the queue fills, or the caller retries at the worst possible moment. A perfectly valid request can still create a production-only failure if the harness never applies pressure.
Backpressure simulation belongs in the loop because it turns "what happens when the caller keeps going?" into executable evidence. Inject bursty users, slow providers, saturated queues, retry storms, partial outages, and cancellations. Then check what is rejected, what is deferred, what is retried, what receives a receipt, and what the operator can see while the system is under load.
This layer spots issues contracts are blind to: work accepted after cancellation, orphaned jobs, stale receipts, a retry loop amplifying an outage, one large task starving the rest of the queue, or a UI that says "done" while the backlog is still growing. The goal is not theater-grade load testing. The goal is to make overload behavior reviewable before production traffic becomes the reviewer.
The toaster test
Imagine a toaster factory with no closed loop. The machine pushes bread through heat, the tray fills up, and quality control discovers the burnt pieces afterward. The process can still produce toast, but the control method is late, sensory, and human. Someone has to notice the defect after value has already been damaged.
That is how a lot of AI and agent review works. The system emits a patch, note, answer, or workflow result. Then a person tastes the toast. They ask whether the ids line up, whether the version is stale, whether the job reference exists, whether the tool call had an owner, whether complexity crept up, and whether the evidence supports the conclusion. The questions are valid. The timing is the problem.
Six Sigma's DMAIC loop is useful here because it refuses to stop at taste. Define the defect, measure the current process, analyze the cause, improve the mechanism, and control the future process so the same defect does not keep returning. In software, the control phase becomes the harness: contract checks, property generators, complexity gates, and review packets that keep the process from drifting back into human inspection theater.
Contract tests ask: do we still agree?
Contract testing is useful because it narrows the question. Instead of deploying the whole world to see whether two applications still communicate, a contract test checks whether the messages exchanged by a consumer and provider still match a shared understanding. Pact describes this as testing each side of an integration point in isolation against a contract.
The deeper lesson is not limited to HTTP. Every tool call, review packet, background job, local model server, and handoff artifact has a consumer and provider. The agent provides a job receipt. The reviewer consumes it. The deploy script provides a revision. The live verification consumes it. The contract is the smallest shared shape both sides can depend on.
Property tests ask: what must never stop being true?
Property-based testing moves the center of gravity from example to invariant. Instead of checking only one hand-picked case, the test describes a rule and lets the generator search the input space. Hypothesis frames this as writing tests that should pass for all inputs in the described range while the tool chooses cases, including edge cases the author may not have imagined.
That maps cleanly to agent systems. The interesting properties are rarely exotic: every artifact has one owning job, every submitted review packet has a version, no destructive tool call lacks an owner, no approval record points at a deleted object, and no "done" state appears without proof. Those rules are not just tests. They are the product's ethics, recovery model, and audit spine in executable form.
job_id:artifact_id:vNN appears in packet and logs.
Complexity constraints are also properties
Complexity budgets often get treated as style preferences. They are stronger than that. A complexity cap says the system should not keep hiding more decision paths inside one unit of code. ESLint's complexity rule makes this literal by warning when cyclomatic complexity crosses a configured threshold.
That is a property. It says the human reviewer should not be asked to simulate an unbounded decision tree during review. When a path crosses the budget, the harness can force the right conversation: split the handler, move the policy into a table, add contract coverage, or attach a review packet explaining why the complexity is deliberate.
A review packet is the harness snapshot
Review packets matter because they convert scattered proof into a bounded object. A good packet does not ask the reviewer to become a detective. It tells them which job ran, which artifacts changed, which contracts were touched, which properties passed, which complexity budgets moved, and which risks remain open.
- Identifiers: job id, artifact ids, versions, owners, and timestamps.
- Contracts: consumer-provider expectations and compatibility checks.
- Properties: invariants checked across generated or enumerated cases.
- Complexity: branch, file, route, or decision-table budgets that changed.
- Evidence: commands, routes, logs, screenshots, source links, and live receipts.
- Open-loop risk: anything still depending on human inspection instead of a gate.
The harness should reduce reviewer imagination
A reviewer should still use judgment. They should inspect intent, product fit, and the shape of risk. But they should not have to imagine whether ids are unique, whether the version moved forward, whether the job link exists, whether the contract still matches, or whether the branch count exploded. The harness should answer those questions before review begins.
This is the practical bridge between testing and operations. Contract tests protect agreements. Property tests protect invariants. Complexity constraints protect human comprehension. Review packets protect handoff. Layered together, they turn "please review this" into "here is the bounded system state, here are the constraints, here is the correctness evidence, here is the quality evidence, and here is what still needs human judgment."
Every repeated review question is a missing property. Name the question, turn it into a constraint, attach it to the job id, and make the next packet prove it. If a layer keeps asking for the same human judgment, promote that judgment into the next objective-function signal.
Related routes and sources
- Pact docs on contract testing
- Hypothesis docs on property-based testing
- ESLint complexity rule
- DMAIC methodology as a quality-improvement roadmap
- Nothing left inside the tool call.
- If the first operator disappears at minute 17, what does your agent leave behind?
- Make AI boring before you make it magical.