fix: harden engine, enrich failure diagnostics, close adoption gaps

- P0: CLI verify now honors  test budget with seeded multi-sample
- P0: Observe sampling enforced via Math.random() gate in hook-validator
- P1: Remove misleading undici-mock-agent isolation option
- P1: Qualify reuses shared discoverRouteDetails() with warnings
- P1: Chaos/scenario config exposed via preset schema
- P1: README/docs limitations updated to current state
- P2: Nested response annotations prefer 2xx deterministically
- P2: --changed documented as heuristic in verify.md

- Add observe sink tests (sampling 0/1, sink failure non-interference)
- Add verify runs regression tests (scale, determinism, variants)
- Add configured-scenario qualify test (independent of OAuth fixture)
- Add coverageBreakdown to qualify artifacts (per-gate route coverage)
- Add production-style observe example with real sink in docs/observe.md
- Add nightly/staging vs PR gating guidance to docs/qualify.md

- Enrich VerifyFailure with formula-aware diagnostics:
  status:201 => 'HTTP 200', body field checks => actual values
- Remove stale observe CLI activation message
- Document outbound mocks as process-global in getting-started.md
- Refresh APOPHIS_ADOPTION_AUDIT.md with current state

903 tests pass, build clean, typecheck clean.
This commit is contained in:
John Dvorak
2026-05-21 20:39:36 -07:00
parent 55b0262799
commit d0523fcc2d
128 changed files with 4004 additions and 3631 deletions
+6 -3
View File
@@ -1,5 +1,5 @@
import Fastify from 'fastify'
import apophisPlugin from 'apophis-fastify'
import apophisPlugin from '@apophis/fastify'
import crypto from 'crypto'
const fastify = Fastify()
@@ -20,8 +20,11 @@ fastify.post('/users', {
schema: {
'x-category': 'constructor',
'x-ensures': [
// Round-trip: the server returns exactly what we sent (no mutation, no drops)
'response_body(this) == request_body(this)',
// The response must have an id assigned by the server
'response_body(this).id != null',
// The echoed fields must match the request
'response_body(this).email == request_body(this).email',
'response_body(this).name == request_body(this).name',
// Cross-route: the created user must be retrievable
'response_code(GET /users/{response_body(this).id}) == 200',
],