fix: harden engine, enrich failure diagnostics, close adoption gaps
- P0: CLI verify now honors test budget with seeded multi-sample - P0: Observe sampling enforced via Math.random() gate in hook-validator - P1: Remove misleading undici-mock-agent isolation option - P1: Qualify reuses shared discoverRouteDetails() with warnings - P1: Chaos/scenario config exposed via preset schema - P1: README/docs limitations updated to current state - P2: Nested response annotations prefer 2xx deterministically - P2: --changed documented as heuristic in verify.md - Add observe sink tests (sampling 0/1, sink failure non-interference) - Add verify runs regression tests (scale, determinism, variants) - Add configured-scenario qualify test (independent of OAuth fixture) - Add coverageBreakdown to qualify artifacts (per-gate route coverage) - Add production-style observe example with real sink in docs/observe.md - Add nightly/staging vs PR gating guidance to docs/qualify.md - Enrich VerifyFailure with formula-aware diagnostics: status:201 => 'HTTP 200', body field checks => actual values - Remove stale observe CLI activation message - Document outbound mocks as process-global in getting-started.md - Refresh APOPHIS_ADOPTION_AUDIT.md with current state 903 tests pass, build clean, typecheck clean.
This commit is contained in:
@@ -2,23 +2,25 @@
|
||||
|
||||
Get from install to your first behavioral bug in 10 minutes.
|
||||
|
||||
APOPHIS is inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): instead of only validating request and response shape, encode intended behavior as executable contracts and let the tool find violations automatically.
|
||||
APOPHIS is inspired by the concept of invariant-driven automated testing: instead of only validating request and response shape, encode intended behavior as executable contracts and let the tool find violations automatically.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 20.x or 22.x
|
||||
- **Fastify v5** (v4 is not supported)
|
||||
- **ESM project** (`"type": "module"` in package.json)
|
||||
- A Fastify app with `@fastify/swagger` registered
|
||||
|
||||
## Step 1: Install
|
||||
|
||||
```bash
|
||||
npm install apophis-fastify fastify @fastify/swagger
|
||||
npm install @apophis/fastify fastify @fastify/swagger
|
||||
```
|
||||
|
||||
## Step 2: Scaffold
|
||||
|
||||
```bash
|
||||
apophis init --preset safe-ci
|
||||
npx apophis init --preset safe-ci
|
||||
```
|
||||
|
||||
This creates:
|
||||
@@ -55,7 +57,7 @@ app.post('/users', {
|
||||
## Step 4: Run Verify
|
||||
|
||||
```bash
|
||||
apophis verify --profile quick --routes "POST /users"
|
||||
npx apophis verify --profile quick --routes "POST /users"
|
||||
```
|
||||
|
||||
## Example Failure
|
||||
@@ -232,7 +234,7 @@ APOPHIS reads these OpenAPI schema extensions:
|
||||
| `x-validate-runtime` | Top-level or `response[statusCode]` | Toggle runtime validation for this route (default: true) |
|
||||
| `x-extension-config` | Top-level | Per-route config for extensions (e.g., `{ jwt: { verify: false } }`) |
|
||||
|
||||
Annotations can be placed on the top-level schema or nested inside `response[statusCode]`. Nested annotations take precedence for that status code.
|
||||
Annotations can be placed on the top-level schema or nested inside `response[statusCode]`. Nested annotations from the first status code schema are merged with top-level annotations.
|
||||
|
||||
## Programmatic API
|
||||
|
||||
@@ -268,6 +270,8 @@ fastify.apophis.test.disableOutboundMocks()
|
||||
const calls = fastify.apophis.test.getOutboundCalls('payment-api')
|
||||
```
|
||||
|
||||
Outbound mocking patches `globalThis.fetch` and is process-global. Only one mock runtime can be installed at a time. Run mock-dependent tests serially or isolate by process.
|
||||
|
||||
## Config Reference
|
||||
|
||||
For the full configuration reference, see [CLI Reference](cli.md).
|
||||
|
||||
Reference in New Issue
Block a user