apophis/apophis-fastify
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: audit + fix — remove observe CLI refs, correct defaults, fix public API imports, add historical banners

Browse Source
This commit is contained in:
John Dvorak
2026-05-22 17:07:12 -07:00
parent 4be5fd74cb
commit fa0f6e1fe5
15 changed files with 52 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
APOPHIS_ADOPTION_AUDIT.md
+5 -3
View File
@@ -34,10 +34,12 @@ Observed results:
|---|---:|
| Typecheck | pass |
| Build | pass |
| Source tests | 590 pass, 0 fail |
| CLI tests | 278 pass, 0 fail |
| Source tests | 590 pass, 0 fail (at audit time) |
| CLI tests | 278 pass, 0 fail (at audit time) |
| Docs smoke tests | 4 pass, 0 fail |
| Total tests | 879 pass, 0 fail |
| Total tests | 879 pass, 0 fail (at audit time) |
Test counts are snapshots at audit time (2026-05-21) and may differ in subsequent development.
The working tree contains many broader project changes unrelated to this audit. This document evaluates the current working tree state.
CHANGELOG.md
+1 -1
View File
@@ -109,7 +109,7 @@ const schema = {
#### Mutation Testing
- **New**: `src/quality/mutation.ts` — synthetic bug injection to measure contract strength.
- **New**: `runMutationTesting()` — generates mutations and verifies tests catch them.
- **New**: `runMutationTesting()` — generates mutations and verifies tests catch them. Internal API only; not part of the public `@apophis/fastify` export.
- **New**: Mutation score reporting (0-100%) with weak contract identification.
### Changed
README.md
+4 -4
View File
@@ -72,10 +72,10 @@ JSON Schema cannot express this relationship. APOPHIS turns it into an executabl
| Mode | Purpose | Default Environments |
|---|---|---|
| `verify` | Deterministic CI and local contract verification | local, test, CI |
| `observe` | Runtime visibility and drift detection without blocking | staging, prod |
| `observe` | Programmatic runtime visibility and drift detection without blocking (validated via `doctor --mode observe`) | staging, prod |
| `qualify` | Exercise scenarios, stateful flows, and configured chaos checks before release | local, test, staging |
## Quickstart: 3 Commands
## Quickstart
```bash
# 1. Install
@@ -99,7 +99,7 @@ See [docs/getting-started.md](docs/getting-started.md) for the full walkthrough.
- **Explicit test budget**: Control how many tests run with `runs: 10` in your preset.
- **CI-safe default path**: `verify` is deterministic and safe for CI pipelines.
- **Machine-readable output**: `--format json-summary` and `--format ndjson-summary` for CI dashboards.
- **Production-safe observe path**: `observe` is non-blocking by default. Blocking behavior requires explicit break-glass policy.
- **Production-safe observe path**: `observe` is the programmatic runtime plugin (no CLI command). Non-blocking by default. Blocking behavior requires explicit break-glass policy.
- **Qualify path gated away from prod**: `qualify` is blocked in production by default.
- **Monorepo workspace support**: `--workspace` fans out `verify` and `doctor` across all packages.
- **Explicit environment boundaries**: Config rejects unknown keys and unsafe environment mixes.
@@ -118,7 +118,7 @@ See [docs/llm-safe-adoption.md](docs/llm-safe-adoption.md) for templates and CI
## Full Documentation
- [Getting Started](docs/getting-started.md) — First route, first verify run, first replay
- [CLI Reference](docs/cli.md) — All 7 commands, global flags, exit codes
- [CLI Reference](docs/cli.md) — All 6 commands, global flags, exit codes
- [Verify Mode](docs/verify.md) — Deterministic contract verification
- [Observe Mode](docs/observe.md) — Runtime visibility and drift detection
- [Qualify Mode](docs/qualify.md) — Scenarios, stateful testing, chaos
docs/cache-and-ci.md
+2
View File
@@ -1,5 +1,7 @@
# Cache & CI/CD Integration
> **Note**: The incremental cache is an internal mechanism used by programmatic test runners (petit-runner, stateful-runner). CLI users do not interact with `.apophis-cache.json` directly. This document describes the internal caching behavior for advanced programmatic integration.
APOPHIS includes an incremental test cache that speeds up test runs by skipping unchanged routes. This document covers cache invalidation strategies and CI/CD integration.
## How the Cache Works
docs/chaos.md
+2 -21
View File
@@ -28,7 +28,7 @@ Adds artificial latency. Tests timeout contracts:
response_time(this) < 1000
```
**Note**: Delay events are generated by the chaos arbitrary but the inbound delay handler is currently a no-op. Use this for timeout contract documentation; actual delay injection requires the outbound delay strategy or a custom handler.
Delay chaos strategies (`inbound-delay`, `outbound-delay`) are applied at the transport level between request execution and contract evaluation. The inline chaos handlers for these strategies are no-ops because `sleep()` handles delay application out-of-band. Delay contracts such as `response_time(this) < 1000` will still work correctly with chaos injection.
### Error
@@ -69,26 +69,7 @@ Built-in strategies are content-type agnostic:
Extension strategies can add content-type-specific behavior if needed.
## Custom Corruption via Extensions
```javascript
const myExtension = {
name: 'custom-corrupt',
corruptionStrategies: {
'application/vnd.api+json': (data) => ({
...data,
corrupted: true,
}),
'text/*': (data) => `CORRUPTED:${String(data)}`,
},
};
await fastify.register(apophis, {
extensions: [myExtension],
});
```
Extension strategies take precedence over built-ins. Wildcard patterns (`text/*`) match any subtype.
**Note**: Extension-defined corruption strategies are documented for future implementation. Currently, corruption strategies (`truncate`, `malformed`, `field-corrupt`) are hardcoded in the chaos engine.
## Environment Guard
docs/cli.md
+3 -3
View File
@@ -58,11 +58,11 @@ apophis verify --profile quick --routes "POST /users"
```
| Flag | Description |
|---|---|
|---|---|---|
| `--profile <name>` | Profile name from config |
| `--routes <filter>` | Route filter pattern (comma-separated, supports wildcards) |
| `--seed <number>` | Deterministic seed (generated and printed if omitted) |
| `--changed` | Filter to git-modified routes only |
| `--changed` | Filter to git-modified routes only |
| `--workspace` | Run across all workspace packages |
| `--format <mode>` | Output format: `human`, `json`, `ndjson`, `json-summary`, `ndjson-summary` |
@@ -198,7 +198,7 @@ apophis replay --artifact reports/apophis/failure-*.json
- `--changed` requires a git repository
- `migrate` defaults to `--dry-run` (safe by default)
- `--workspace` is fully implemented by `verify` and `doctor`. `observe` and `qualify` accept the flag but run in the current package only.
- `--workspace` is fully implemented by `verify` and `doctor`. Other commands do not support `--workspace`.
- Seeds ensure deterministic generation; handler nondeterminism (e.g., `Date.now()`) can still cause replay divergence
## Exit Codes
docs/extensions/EXTENSION-ARCHITECTURE.md
+2
View File
@@ -1,5 +1,7 @@
# APOPHIS v1.1 Architecture — Hybrid Core + Extensions
> **HISTORICAL**: This is a v1.1 architecture design specification. For current extension documentation, see [EXTENSION-PLUGIN-SYSTEM.md](EXTENSION-PLUGIN-SYSTEM.md) and [QUICK-REFERENCE.md](QUICK-REFERENCE.md).
## Status: Architecture Specification
## Date: 2026-04-24
## Scope: v1.1 First-Class Features & Extension Ecosystem
docs/extensions/EXTENSION-PLUGIN-SYSTEM.md
+1 -1
View File
@@ -392,7 +392,7 @@ await app.register(apophis, {
| `src/test/extension.test.ts` | Extension system tests |
| `src/formula/evaluator.ts` | APOSTL evaluator with extension predicate resolution |
| `src/domain/contract-validation.ts` | Passes extension registry to evaluator |
| `src/test/petit-runner.ts` | Calls extension hooks |
| `src/quality/petit-runner.ts` | Calls extension hooks |
| `src/plugin/index.ts` | Creates and passes ExtensionRegistry |
---
docs/extensions/TIMEOUTS-REDIRECTS-CONCURRENCY.md
+3 -1
View File
@@ -1,4 +1,4 @@
# APOPHIS v1.0 Extension Specification: Timeouts and Redirects
# APOPHIS Extension Specification: Timeouts and Redirects
## Document Information
- **Version**: 1.0
@@ -64,6 +64,8 @@ fastify.get('/slow-endpoint', {
When a timeout is configured, `executeHttp` uses an abortable timer where supported. The timeout must be cleared in `finally`; Fastify injection may continue running after timeout if the underlying transport cannot be cancelled.
> **Note**: The code examples in this section are illustrative representations of the timeout, redirect, and concurrency mechanisms. The actual implementation may differ in detail.
```typescript
// In src/infrastructure/http-executor.ts
if (timeoutMs && timeoutMs > 0) {
docs/getting-started.md
+2 -2
View File
@@ -6,7 +6,7 @@ APOPHIS is inspired by the concept of invariant-driven automated testing: instea
## Prerequisites
- Node.js 20.x or 22.x
- Node.js >=20.18.1 (20.x) or >=22 (22.x)
- **Fastify v5** (v4 is not supported)
- **ESM project** (`"type": "module"` in package.json)
- A Fastify app with `@fastify/swagger` registered
@@ -143,7 +143,7 @@ APOPHIS contracts should verify **behavior**, not structure. Fastify and `@fasti
- Add more routes to your profile: `apophis verify --profile quick --routes "POST /users,PUT /users/:id"`
- Use wildcards to match route patterns: `apophis verify --routes 'POST /api/*'`
- Run all routes: `apophis verify --profile quick`
- Run only changed routes in CI: `apophis verify --profile ci --changed`
- Run only changed routes in CI: `apophis verify --profile quick --changed`
- Requires a git repository.
- Use machine-readable output in CI: `apophis verify --profile ci --format json-summary`
- Add observe mode for runtime drift detection: see [observe.md](observe.md)
docs/observe.md
+12 -12
View File
@@ -29,21 +29,19 @@ Observe mode requires a reporting sink. Configure it in your environment policy:
environments: {
staging: {
name: 'staging',
allowVerify: true,
allowObserve: true,
allowQualify: false,
allowChaos: false,
allowBlocking: false,
requireSink: true
allowedModes: ['verify', 'observe'],
blockQualify: true,
requireSink: true,
allowBlocking: false
}
}
```
APOPHIS supports these sink types:
You implement sinks for your observability backend. Common categories:
- **Logs**: Structured logging of contract violations
- **Metrics**: Counter and histogram metrics for violation rates
- **Traces**: Distributed tracing integration for violation context
- **Logs**: Structured logging of contract violations (pino, winston)
- **Metrics**: Counter and histogram metrics for violation rates (Prometheus, OpenTelemetry)
- **Traces**: Distributed tracing integration for violation context (OpenTelemetry, Jaeger)
## Sampling
@@ -97,10 +95,12 @@ profiles: {
Validate your observe config before deployment with doctor:
## Exit Codes
## Validation (via Doctor)
Validate observe configuration with `apophis doctor --mode observe`:
| Code | Meaning |
|---|---|
|---|---|---|
| 0 | Observe config is valid and safe |
| 2 | Safety violation or invalid config |
docs/qualify.md
+3 -1
View File
@@ -183,7 +183,9 @@ Qualify mode is gated away from production by default:
|---|---|---|---|
| local | enabled | enabled | enabled |
| test/CI | enabled | enabled | enabled |
| staging | enabled with allowlist | enabled | blocked on protected routes |
| staging | enabled (blockQualify: false) | enabled (blockQualify: false) | blocked on protected routes (allowChaosOnProtected: false) |
Qualify gates are not individually gated per environment. The `blockQualify` flag controls all qualify execution, and `allowChaosOnProtected` controls chaos on protected routes.
| production | disabled by default | disabled by default | disabled by default |
## Machine Output for CI
docs/quality.md
+9 -3
View File
@@ -71,7 +71,9 @@ Automatically rerun failing tests with varied seeds to detect non-deterministic
### Usage
```javascript
import { FlakeDetector } from '@apophis/fastify'
// FlakeDetector is an internal API. It is not part of the public @apophis/fastify export.
// For programmatic use, import from the internal path:
// import { FlakeDetector } from '@apophis/fastify/src/quality/flake.js'
const detector = new FlakeDetector({
sameSeedReruns: 1, // Rerun with same seed
@@ -121,7 +123,9 @@ Measure contract strength by injecting synthetic bugs. A "mutation" is a small c
### Usage
```javascript
import { runMutationTesting } from '@apophis/fastify/quality/mutation'
// Mutation testing is an internal API. It is not part of the public @apophis/fastify export.
// For programmatic use, import from the internal path:
// import { runMutationTesting } from '@apophis/fastify/src/quality/mutation.js'
const report = await runMutationTesting(fastify, {
runs: 10,
@@ -176,7 +180,9 @@ console.log('Weak contracts:', report.weakContracts)
Test a specific mutation without running the full suite:
```javascript
import { testMutation } from '@apophis/fastify/quality/mutation'
// Mutation testing is an internal API. It is not part of the public @apophis/fastify export.
// For programmatic use, import from the internal path:
// import { testMutation } from '@apophis/fastify/src/quality/mutation.js'
const killed = await testMutation(fastify, contract, mutation, {
runs: 10,
docs/troubleshooting.md
+2 -2
View File
@@ -38,10 +38,10 @@ APOPHIS classifies failures into six categories. Lower categories take precedenc
1. Check the route and clause index printed in the error message.
2. Verify APOSTL syntax: use `response_code(this)` not `response_code()`.
3. Ensure string literals use single or double quotes consistently.
4. Run `apophis doctor --profile <name>` to validate formulas without executing.
4. Run `apophis doctor` to validate formulas without executing.
**Prevention**
- Run `apophis doctor --profile <name>` to validate formulas without executing.
- Run `apophis doctor` to validate formulas without executing.
- Enable editor support for APOSTL syntax highlighting.
---
docs/verify.md
+1 -1
View File
@@ -206,4 +206,4 @@ presets: {
}
```
CLI verify generates one property-based test sample per contract by default when no `runs` is specified. Set `runs` in the preset to increase sampled inputs per route.
CLI verify defaults to 50 runs per contract. Set `runs` in the preset to adjust the sample count per route. Use `runs: 1` to check each contract once, or `runs: 0` to disable property-based verification.