fix: harden engine, enrich failure diagnostics, close adoption gaps

- P0: CLI verify now honors  test budget with seeded multi-sample
- P0: Observe sampling enforced via Math.random() gate in hook-validator
- P1: Remove misleading undici-mock-agent isolation option
- P1: Qualify reuses shared discoverRouteDetails() with warnings
- P1: Chaos/scenario config exposed via preset schema
- P1: README/docs limitations updated to current state
- P2: Nested response annotations prefer 2xx deterministically
- P2: --changed documented as heuristic in verify.md

- Add observe sink tests (sampling 0/1, sink failure non-interference)
- Add verify runs regression tests (scale, determinism, variants)
- Add configured-scenario qualify test (independent of OAuth fixture)
- Add coverageBreakdown to qualify artifacts (per-gate route coverage)
- Add production-style observe example with real sink in docs/observe.md
- Add nightly/staging vs PR gating guidance to docs/qualify.md

- Enrich VerifyFailure with formula-aware diagnostics:
  status:201 => 'HTTP 200', body field checks => actual values
- Remove stale observe CLI activation message
- Document outbound mocks as process-global in getting-started.md
- Refresh APOPHIS_ADOPTION_AUDIT.md with current state

903 tests pass, build clean, typecheck clean.
This commit is contained in:
John Dvorak
2026-05-21 20:39:36 -07:00
parent 55b0262799
commit d0523fcc2d
128 changed files with 4004 additions and 3631 deletions
+5 -5
View File
@@ -1,13 +1,13 @@
---
name: apophis-fastify
name: @apophis/fastify
description: Use this skill when adding or improving APOPHIS contract-driven testing for Fastify APIs. This tool finds real implementation bugs—resources that appear to create but cannot be retrieved, updates that silently fail to persist, deletions that leave data visible, cross-tenant leakage, and broken state transitions. Use it to encode intended behavior as executable contracts and verify them continuously, not to paper over failures.
---
# apophis-fastify
# @apophis/fastify
APOPHIS finds real API behavior bugs that schema validation misses. It verifies that operations cause the state changes, isolation guarantees, and dependency interactions the service actually depends on.
Inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): encode intended behavior as executable contracts, then verify them with property-based and stateful testing.
Inspired by research on invariant-driven testing: encode intended behavior as executable contracts, then verify them with property-based and stateful testing.
## Core Purpose
@@ -97,7 +97,7 @@ Treat context as a finite budget.
When entering a Fastify codebase:
1. Locate app construction and route registration.
2. Confirm `@fastify/swagger` is registered before `apophis-fastify`.
2. Confirm `@fastify/swagger` is registered before `@apophis/fastify`.
3. Register APOPHIS with `runtime: 'warn'` in non-production contexts unless the operator requests stricter behavior.
4. Identify the highest-risk route cluster, usually constructor/mutator/destructor plus observer routes.
5. Ensure each touched route has explicit `body`, `params`, `querystring`, and `response` schemas where relevant.
@@ -112,7 +112,7 @@ When entering a Fastify codebase:
```javascript
import Fastify from 'fastify'
import swagger from '@fastify/swagger'
import apophis from 'apophis-fastify'
import apophis from '@apophis/fastify'
import crypto from 'crypto'
const app = Fastify()