- P0: CLI verify now honors test budget with seeded multi-sample - P0: Observe sampling enforced via Math.random() gate in hook-validator - P1: Remove misleading undici-mock-agent isolation option - P1: Qualify reuses shared discoverRouteDetails() with warnings - P1: Chaos/scenario config exposed via preset schema - P1: README/docs limitations updated to current state - P2: Nested response annotations prefer 2xx deterministically - P2: --changed documented as heuristic in verify.md - Add observe sink tests (sampling 0/1, sink failure non-interference) - Add verify runs regression tests (scale, determinism, variants) - Add configured-scenario qualify test (independent of OAuth fixture) - Add coverageBreakdown to qualify artifacts (per-gate route coverage) - Add production-style observe example with real sink in docs/observe.md - Add nightly/staging vs PR gating guidance to docs/qualify.md - Enrich VerifyFailure with formula-aware diagnostics: status:201 => 'HTTP 200', body field checks => actual values - Remove stale observe CLI activation message - Document outbound mocks as process-global in getting-started.md - Refresh APOPHIS_ADOPTION_AUDIT.md with current state 903 tests pass, build clean, typecheck clean.
7.1 KiB
APOPHIS
Behavioral confidence for Fastify services.
APOPHIS checks whether route behavior holds across operations, states, and protocol flows.
Inspired by the concept of invariant-driven automated testing: instead of only checking payload shape, APOPHIS encodes intended behavior as executable contracts and verifies them with property-based and stateful testing.
Supported Node.js versions: >=20.18.1 (20.x) and 22.x.
npm install @apophis/fastify fastify @fastify/swagger
npx apophis init --preset safe-ci
npx apophis verify --profile quick --routes "POST /users"
x-ensures is an OpenAPI schema extension for behavioral contracts — statements about what a route must guarantee.
Cross-Route Failure Example
Add one behavioral contract next to a route schema. APOPHIS can verify cross-route behavior, such as whether a resource created by one route is retrievable through another.
Route:
import crypto from 'crypto';
app.post('/users', {
schema: {
'x-category': 'constructor',
'x-ensures': [
// BEHAVIORAL: Creating a user must make it retrievable
'response_code(GET /users/{response_body(this).id}) == 200'
]
}
}, async (request, reply) => {
const { name } = request.body;
const id = `usr-${crypto.createHash('sha256').update(name).digest('hex').slice(0, 8)}`;
reply.status(201);
return { id, name };
});
APOPHIS output:
Contract violation
POST /users
Profile: quick
Seed: 42
Expected
response_code(GET /users/{response_body(this).id}) == 200
Observed
GET /users/usr-7d865e returned 404
Why this matters
The resource created by POST /users is not retrievable.
Replay
apophis replay --artifact reports/apophis/failure-2026-04-28T12-30-22Z.json
Next
Check the create/read consistency for POST /users and GET /users/{id}.
JSON Schema cannot express this relationship. APOPHIS turns it into an executable check.
Three Modes
| Mode | Purpose | Default Environments |
|---|---|---|
verify |
Deterministic CI and local contract verification | local, test, CI |
observe |
Runtime visibility and drift detection without blocking | staging, prod |
qualify |
Exercise scenarios, stateful flows, and configured chaos checks before release | local, test, staging |
Quickstart: 3 Commands
# 1. Install
npm install @apophis/fastify fastify @fastify/swagger
# 2. Scaffold
npx apophis init --preset safe-ci
# 3. Verify
npx apophis verify --profile quick --routes "POST /users"
# 4. Doctor
npx apophis doctor
See docs/getting-started.md for the full walkthrough.
Trust and Safety
- Deterministic replay: Every failure includes a seed and a one-command replay.
- Explicit test budget: Control how many tests run with
runs: 10in your preset. - CI-safe default path:
verifyis deterministic and safe for CI pipelines. - Machine-readable output:
--format json-summaryand--format ndjson-summaryfor CI dashboards. - Production-safe observe path:
observeis non-blocking by default. Blocking behavior requires explicit break-glass policy. - Qualify path gated away from prod:
qualifyis blocked in production by default. - Monorepo workspace support:
--workspacefans outverifyanddoctoracross all packages. - Explicit environment boundaries: Config rejects unknown keys and unsafe environment mixes.
LLM-Safe
APOPHIS gives coding agents a constrained, repeatable way to encode and verify behavior:
- Official scaffolds (
safe-ci,llm-safe,platform-observe,protocol-lab) apophis doctorchecks for missing dependencies, malformed config, and unsafe modes- CI policy guards catch unknown keys, unsafe environments, and missing seeds
- Generated code follows the same pattern in every repo
See docs/llm-safe-adoption.md for templates and CI policy.
Full Documentation
- Getting Started — First route, first verify run, first replay
- CLI Reference — All 7 commands, global flags, exit codes
- Verify Mode — Deterministic contract verification
- Observe Mode — Runtime visibility and drift detection
- Qualify Mode — Scenarios, stateful testing, chaos
- Quality Engines — Chaos injection, flake detection, mutation testing
- Performance — Repeatable benchmarks and CPU profiling
- LLM-Safe Adoption — Scaffolds and CI guards
- Protocol Extensions — JWT, X.509, SPIFFE, WIMSE
Recommended Integration
New projects: Use createFastify() to ensure route discovery is installed before any routes are registered.
import { createFastify } from '@apophis/fastify'
const app = await createFastify({
logger: true,
apophis: { runtime: process.env.NODE_ENV === 'test' ? 'error' : 'off' },
})
// Register swagger, auth, plugins, and routes after app creation.
Existing projects: Register APOPHIS or install route discovery before routes. Run apophis doctor to verify routes are discovered with full schema metadata.
Schema-less fallback: If APOPHIS is registered after routes, printRoutes() can recover paths but not route schemas or behavioral contracts. apophis doctor and apophis verify will warn when discovery is schema-less.
Current Limitations
These reflect current implementation behavior. All are actively tracked for improvement.
- Route discovery requires ordering. If the APOPHIS plugin or route discovery hook is not installed before routes are registered, behavioral contract annotations (x-ensures, x-requires, x-outbound, x-variants, x-timeout) cannot be recovered. Use
createFastify()for new projects or register APOPHIS early. - Observe has two faces. The runtime plugin supports non-blocking sink emission via
observe.enabled+observe.sinkswhen registered programmatically.apophis observeCLI validates config readiness; it does not activate a long-running runtime observer. Seedocs/observe.mdfor the distinction between programmatic runtime observation and CLI config validation. - CLI verify samples once per contract by default. Set
runsin your preset to increase the number of property-based test samples per route. The programmaticfastify.apophis.contract()API supports the samerunsconfiguration. - Outbound mocks are process-global. The mock runtime patches
globalThis.fetch. Only one mock runtime can be installed at a time. Run mock-dependent tests serially or isolate by process. UndiciMockAgentintegration is not yet implemented. - Qualify coverage depends on profile configuration. Qualify runs scenario, stateful, and chaos checks based on profile gates. Chaos route selection uses the configured strategy (one/all/sample/routes).
Compatibility
- Fastify v5 only. Fastify v4 and earlier are not supported.
- ESM only. This package is
"type": "module"and does not provide a CommonJS build. Useimportsyntax. - Node.js
>=20.18.1 <21 || >=22 <23. @fastify/swaggermust be registered before routes (APOPHIS auto-registers it if missing).
License
MIT