apophis-fastify/APOPHIS_ADOPTION_AUDIT.md

APOPHIS Adoption Audit

Date: 2026-05-21

Scope: current working tree for @apophis/fastify v2.7.0, assessed as a developer deciding whether to use APOPHIS in a real Fastify v5 ESM service and whether to recommend it as a team standard.

This audit is based on code inspection plus command verification, not documentation claims alone.

Executive Summary

APOPHIS has real product value. It is not just a schema wrapper: it gives Fastify teams a way to express and verify behavioral API promises that OpenAPI/JSON Schema cannot cover, especially cross-route invariants such as create/read consistency, delete semantics, auth/session flows, state transitions, idempotency, outbound dependency expectations, and replayable counterexamples.

I would adopt APOPHIS today as a focused behavioral verification tool for Fastify v5 ESM services. I would start with CI verify and a small number of high-value contracts, then expand into qualify and runtime observation once the team has clear operating guidance.

I would not yet treat it as a complete production observability platform or a turnkey organization-wide release gate. The core implementation is strong, but the remaining value gap is mostly around operational maturity: standalone observe activation, deeper tests around recent CLI behavior, richer scenario authoring, and clearer release-gate recommendations.

Adoption verdict: strong team pilot candidate, credible standardization candidate after the remaining gaps below are addressed.

Verification Performed

Commands run successfully against the current working tree:

npm run typecheck
npm run build
npm run test:src
npm run test:cli
npm run test:docs

Observed results:

Area	Result
Typecheck	pass
Build	pass
Source tests	587 pass, 0 fail
CLI tests	311 pass, 0 fail
Docs smoke tests	4 pass, 0 fail
Total tests	902 pass, 0 fail

The working tree contains many broader project changes unrelated to this audit. This document evaluates the current working tree state.

Does It Do What It Says On The Tin?

Mostly yes for behavioral verification. Partially for production observation and broad release qualification.

Product Promise	Current Assessment
Behavioral contracts for Fastify	Yes. The plugin captures route schemas, extracts APOPHIS annotations, evaluates APOSTL formulas, and exposes programmatic runners.
Deterministic CI verification	Yes, materially. CLI verify now honors configured runs, uses seeded request generation, emits artifacts, supports route filters, replay metadata, and machine-readable output.
Cross-route behavior	Yes for supported formula operations and route-call semantics. This is the most differentiated value.
Runtime validation	Yes when the plugin is explicitly configured outside production. Production enforcement is intentionally blocked.
Runtime observation	Partially. Programmatic plugin observation exists and emits non-blocking sink events with sampling. The CLI validates/report readiness but does not attach to or run a service.
Stateful/scenario/chaos qualification	Partially. The runner and artifacts are useful, route discovery is now shared with verify, and config supports scenarios/chaos knobs. Scenario authoring is still young and needs more real-world examples/tests.
Outbound dependency mocking	Useful but intentionally process-global. The misleading scoped undici-mock-agent option has been removed. Teams still need careful test isolation.
Team-safe onboarding	Good. The package has CLI help, init/doctor/replay/verify/qualify/observe, config validation, machine output, docs smoke tests, packaging tests, and production safety checks.

What Has Real Value

1. Behavioral contracts fill a real Fastify testing gap.

JSON Schema validates shape. APOPHIS validates behavior: whether one operation changes another operation's result, whether an auth flow preserves a token property, whether cleanup restores state, or whether a dependency call follows a declared contract.

Relevant code: src/formula/parser.ts, src/formula/evaluator.ts, src/formula/runtime.ts, src/domain/contract.ts, src/domain/contract-validation.ts.

2. Fastify integration is natural.

The package uses a real Fastify plugin, fastify.inject(), onRoute capture, a decorated fastify.apophis API, and a createFastify() helper for discovery ordering.

Relevant code: src/plugin/index.ts, src/plugin/builders.ts, src/domain/discovery.ts, src/fastify-factory.ts.

3. CLI verification now has credible depth.

verifyCommand() resolves preset/profile run configuration and passes it into runVerify(). The runner generates seeded per-run requests and executes each contract for contractRuns. This better matches the documented property-testing story than the earlier single-sample behavior.

Relevant code: src/cli/commands/verify/index.ts, src/cli/commands/verify/runner.ts, src/quality/petit-runner.ts.

4. Discovery diagnostics are meaningfully useful.

Shared discovery reports whether routes came from captured Fastify metadata, legacy app.routes, or schema-less printRoutes() fallback. This matters because fallback discovery cannot recover APOPHIS annotations.

Relevant code: src/domain/discovery.ts, src/plugin/builders.ts, src/cli/commands/verify/runner.ts, src/cli/commands/qualify/index.ts.

5. Runtime safety is treated seriously.

Runtime validation is production-gated, qualify has policy checks, observe is non-blocking, and config validation rejects unknown APOPHIS-owned keys.

Relevant code: src/infrastructure/production-safety.ts, src/infrastructure/hook-validator.ts, src/cli/core/policy-engine.ts, src/cli/core/config-loader.ts.

6. Packaging confidence is high.

The package has ESM exports, Fastify peer boundaries, a CLI bin, npm-pack tests, temp-consumer import tests, and TypeScript consumer tests.

Relevant code: package.json, src/test/cli/packaging.test.ts.

Improvements Already Confirmed In Code

The following earlier adoption risks have been addressed in the current working tree:

Area	Confirmed Current State
CLI verify runs	VerifyRunnerDeps accepts runs; verifyCommand() passes resolved config; runVerify() executes contracts for contractRuns.
Observe sampling	hook-validator.ts gates sink emission using opts.observe.sampling before emitting pass/violation/error events.
Observe CLI honesty	observe output now says the CLI validates readiness and programmatic plugin registration activates runtime observation.
Outbound mock isolation	The misleading undici-mock-agent isolation option has been removed; the runtime treats fetch mocking as process-global.
Qualify discovery	qualify uses shared discoverRouteDetails() and includes discovery warnings in artifacts.
Qualify config	Config schema now accepts scenario definitions and chaos strategy/sample controls.
Nested response annotations	Contract extraction now prefers deterministic 2xx response schemas instead of relying on object-value order.
--changed	Documentation identifies it as a heuristic convenience, not a strict CI release gate.

Remaining Adoption Gaps

P0: Observation Is Programmatic, Not A Standalone Production Observer

The implementation supports runtime observation only when the application explicitly registers APOPHIS with observe options. The CLI command validates configuration and readiness. It does not start an app, attach to a running Fastify process, or deploy a collector.

Completed:

• Docs are explicit that CLI observe is validation/readiness only.
• Production-style TypeScript example with real ObserveSink implementation added to docs/observe.md.
• Integration tests prove sink sync failures and async rejections never change route responses.
• Integration tests prove sampling: 0 suppresses all events; sampling: 1 emits expected contract.pass/contract.violation events.

Still open: A future apophis observe --app ./app.ts mode that activates a running service observer.

P1: Recent verify Runs Behavior Now Has Regression Tests

Completed:

• Regression test proves runs: 1 produces single execution per contract.
• Regression test proves runs: 5 scales multiplicatively from runs: 1.
• Regression test proves runs: 10 is deterministic at the same seed.

Still open: Variant-aware runs test (verifying run budget is per-variant or shared).

P1: Qualify Product Shape Improved

Completed:

• docs/qualify.md now includes full config-defined scenario examples (idempotency, pagination).
• Configured-scenario qualify test added (independent of OAuth fixture routes).
• coverageBreakdown field added to qualify artifacts: per-gate routes covered, steps/tests/runs passed.

Still open: Clear guidance for nightly/staging use versus pull-request gating in qualify docs.

P1: Outbound Mocks Process-Global, Honestly Documented

Completed:

• Misleading undici-mock-agent isolation option removed.
• README and docs/getting-started.md explicitly state outbound mocking is process-global.
• Serial test guidance added.

Still open: True scoped mocking (undici dispatcher) remains future work, gated on whether concurrent in-process dependency tests become a core promise.

P2: Fastify Discovery Ordering Still Matters

Completed:

• createFastify() recommended as the pattern for new services.
• doctor output is explicit about schema-less fallback detection.
• Migration examples exist for existing apps with plugin-order constraints.

Still open: Automatic reordering or lazy discovery is not yet implemented — teams must still register discovery before routes.

P2: --changed Documented As Heuristic

Completed:

• docs/verify.md states --changed is a heuristic and not precise enough for strict CI gating.
• README recommends explicit route filters or full verify for release gates.

Still open: Route ownership metadata or generated route-to-file maps for future precision.

Fastify Team Adoption Guidance

Recommended starting pattern for new services:

import { createFastify } from '@apophis/fastify'

const app = await createFastify({
  logger: true,
  apophis: {
    runtime: process.env.NODE_ENV === 'test' ? 'warn' : 'off',
  },
})

// Register swagger, auth, plugins, and routes after app creation.

Recommended adoption path:

1. Run apophis doctor and confirm route discovery includes schema metadata.
2. Add 3 to 5 contracts for routes where schemas cannot express the behavioral promise.
3. Run apophis verify --profile quick in pull requests.
4. Use fixed seeds and replay artifacts for triage.
5. Use full verify or explicit route filters for release gates.
6. Treat qualify as staging/nightly until scenario coverage is well defined.
7. Treat observe as programmatic non-blocking runtime hooks, not standalone CLI monitoring.

High-value first contracts:

• POST /resource followed by GET /resource/{id} returns the created resource.
• DELETE /resource/{id} makes subsequent GET return 404 or equivalent domain response.
• Auth token/session claims remain valid across protected calls.
• Idempotency keys prevent duplicate side effects.
• Outbound dependency requests carry required headers and retry-safe behavior.

Adoption Scorecard

Dimension	Score	Reason
Core idea/value	9/10	Behavioral contracts are genuinely valuable and differentiated.
Fastify fit	8/10	Strong plugin/inject/decorator alignment; discovery order still matters.
Programmatic API	8/10	Useful contract/stateful/scenario/check API with meaningful tests.
CLI verify	8/10	Now honors run budgets with regression tests; good artifacts and determinism.
Observe	7/10	Runtime sink primitives, sampling, and sink-failure-resilience exist with tests. Production-style docs added. Standalone operational story not complete.
Qualify	7/10	Improved discovery/config/scenarios. Coverage breakdown in artifacts. Needs richer scenario examples and gating guidance.
Outbound mocking	7/10	Useful and honest about process-global behavior. Docs and README explicit. True scoped mocking remains future work.
Docs	8/10	Broad and increasingly precise. Observe and qualify docs expanded with real code examples.
Packaging	9/10	Strong for a Node/Fastify package.
Team readiness	8/10	Ready for pilot and selective CI use with regression-locked verification behavior.

Overall: 8/10 for real team pilot use. Potential 9/10 if observe gains a clearer production story and qualify gets first-class CI workflow guidance.

Highest-Impact Next Work

1. ✅ CLI verify runs honoring verified — regression tests added proving execution count scales with runs.
2. ✅ Observe sampling enforced in runtime hooks with dedicated tests for sampling: 0, sampling: 1, and sink failure non-interference.
3. ✅ Outbound mock docs explicitly say process-global — README and getting-started.md updated.
4. ✅ Qualify scenario config documented with full examples in qualify.md.
5. ✅ Configured-scenario qualify test added (does not depend on OAuth fixture routes).
6. Add full production-style observe example with a real collector sink implementation.
7. Improve qualify artifact coverage summaries to distinguish route-contract, scenario, stateful, and chaos coverage more clearly.
8. Consider true scoped outbound mocking (undici dispatcher) only if concurrent in-process dependency tests become a core promise.

Bottom Line

APOPHIS does what its core idea promises: it lets Fastify teams encode behavioral API guarantees and verify them with deterministic tooling. That is valuable, and the implementation is substantial enough to use in a real repository.

The remaining work is not about proving the idea. The remaining work is about product maturity: locking down recent fixes with regression tests, clarifying observe as programmatic runtime support rather than standalone monitoring, and making qualify scenarios feel like a first-class team workflow.

I would recommend APOPHIS for a Fastify team pilot today. I would recommend it as a default team standard after the highest-impact next work above is complete.