docs: audit + fix — remove observe CLI refs, correct defaults, fix public API imports, add historical banners
This commit is contained in:
John Dvorak
@@ -1,5 +1,7 @@
|
||||
# Cache & CI/CD Integration
|
||||
|
||||
> **Note**: The incremental cache is an internal mechanism used by programmatic test runners (petit-runner, stateful-runner). CLI users do not interact with `.apophis-cache.json` directly. This document describes the internal caching behavior for advanced programmatic integration.
|
||||
|
||||
APOPHIS includes an incremental test cache that speeds up test runs by skipping unchanged routes. This document covers cache invalidation strategies and CI/CD integration.
|
||||
|
||||
## How the Cache Works
|
||||
|
||||
+2
-21
@@ -28,7 +28,7 @@ Adds artificial latency. Tests timeout contracts:
|
||||
response_time(this) < 1000
|
||||
```
|
||||
|
||||
**Note**: Delay events are generated by the chaos arbitrary but the inbound delay handler is currently a no-op. Use this for timeout contract documentation; actual delay injection requires the outbound delay strategy or a custom handler.
|
||||
Delay chaos strategies (`inbound-delay`, `outbound-delay`) are applied at the transport level between request execution and contract evaluation. The inline chaos handlers for these strategies are no-ops because `sleep()` handles delay application out-of-band. Delay contracts such as `response_time(this) < 1000` will still work correctly with chaos injection.
|
||||
|
||||
### Error
|
||||
|
||||
@@ -69,26 +69,7 @@ Built-in strategies are content-type agnostic:
|
||||
|
||||
Extension strategies can add content-type-specific behavior if needed.
|
||||
|
||||
## Custom Corruption via Extensions
|
||||
|
||||
```javascript
|
||||
const myExtension = {
|
||||
name: 'custom-corrupt',
|
||||
corruptionStrategies: {
|
||||
'application/vnd.api+json': (data) => ({
|
||||
...data,
|
||||
corrupted: true,
|
||||
}),
|
||||
'text/*': (data) => `CORRUPTED:${String(data)}`,
|
||||
},
|
||||
};
|
||||
|
||||
await fastify.register(apophis, {
|
||||
extensions: [myExtension],
|
||||
});
|
||||
```
|
||||
|
||||
Extension strategies take precedence over built-ins. Wildcard patterns (`text/*`) match any subtype.
|
||||
**Note**: Extension-defined corruption strategies are documented for future implementation. Currently, corruption strategies (`truncate`, `malformed`, `field-corrupt`) are hardcoded in the chaos engine.
|
||||
|
||||
## Environment Guard
|
||||
|
||||
|
||||
+3
-3
@@ -58,11 +58,11 @@ apophis verify --profile quick --routes "POST /users"
|
||||
```
|
||||
|
||||
| Flag | Description |
|
||||
|---|---|
|
||||
|---|---|---|
|
||||
| `--profile <name>` | Profile name from config |
|
||||
| `--routes <filter>` | Route filter pattern (comma-separated, supports wildcards) |
|
||||
| `--seed <number>` | Deterministic seed (generated and printed if omitted) |
|
||||
| `--changed` | Filter to git-modified routes only |
|
||||
| `--changed` | Filter to git-modified routes only |
|
||||
| `--workspace` | Run across all workspace packages |
|
||||
| `--format <mode>` | Output format: `human`, `json`, `ndjson`, `json-summary`, `ndjson-summary` |
|
||||
|
||||
@@ -198,7 +198,7 @@ apophis replay --artifact reports/apophis/failure-*.json
|
||||
|
||||
- `--changed` requires a git repository
|
||||
- `migrate` defaults to `--dry-run` (safe by default)
|
||||
- `--workspace` is fully implemented by `verify` and `doctor`. `observe` and `qualify` accept the flag but run in the current package only.
|
||||
- `--workspace` is fully implemented by `verify` and `doctor`. Other commands do not support `--workspace`.
|
||||
- Seeds ensure deterministic generation; handler nondeterminism (e.g., `Date.now()`) can still cause replay divergence
|
||||
|
||||
## Exit Codes
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# APOPHIS v1.1 Architecture — Hybrid Core + Extensions
|
||||
|
||||
> **HISTORICAL**: This is a v1.1 architecture design specification. For current extension documentation, see [EXTENSION-PLUGIN-SYSTEM.md](EXTENSION-PLUGIN-SYSTEM.md) and [QUICK-REFERENCE.md](QUICK-REFERENCE.md).
|
||||
|
||||
## Status: Architecture Specification
|
||||
## Date: 2026-04-24
|
||||
## Scope: v1.1 First-Class Features & Extension Ecosystem
|
||||
|
||||
@@ -392,7 +392,7 @@ await app.register(apophis, {
|
||||
| `src/test/extension.test.ts` | Extension system tests |
|
||||
| `src/formula/evaluator.ts` | APOSTL evaluator with extension predicate resolution |
|
||||
| `src/domain/contract-validation.ts` | Passes extension registry to evaluator |
|
||||
| `src/test/petit-runner.ts` | Calls extension hooks |
|
||||
| `src/quality/petit-runner.ts` | Calls extension hooks |
|
||||
| `src/plugin/index.ts` | Creates and passes ExtensionRegistry |
|
||||
|
||||
---
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# APOPHIS v1.0 Extension Specification: Timeouts and Redirects
|
||||
# APOPHIS Extension Specification: Timeouts and Redirects
|
||||
|
||||
## Document Information
|
||||
- **Version**: 1.0
|
||||
@@ -64,6 +64,8 @@ fastify.get('/slow-endpoint', {
|
||||
|
||||
When a timeout is configured, `executeHttp` uses an abortable timer where supported. The timeout must be cleared in `finally`; Fastify injection may continue running after timeout if the underlying transport cannot be cancelled.
|
||||
|
||||
> **Note**: The code examples in this section are illustrative representations of the timeout, redirect, and concurrency mechanisms. The actual implementation may differ in detail.
|
||||
|
||||
```typescript
|
||||
// In src/infrastructure/http-executor.ts
|
||||
if (timeoutMs && timeoutMs > 0) {
|
||||
|
||||
@@ -6,7 +6,7 @@ APOPHIS is inspired by the concept of invariant-driven automated testing: instea
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 20.x or 22.x
|
||||
- Node.js >=20.18.1 (20.x) or >=22 (22.x)
|
||||
- **Fastify v5** (v4 is not supported)
|
||||
- **ESM project** (`"type": "module"` in package.json)
|
||||
- A Fastify app with `@fastify/swagger` registered
|
||||
@@ -143,7 +143,7 @@ APOPHIS contracts should verify **behavior**, not structure. Fastify and `@fasti
|
||||
- Add more routes to your profile: `apophis verify --profile quick --routes "POST /users,PUT /users/:id"`
|
||||
- Use wildcards to match route patterns: `apophis verify --routes 'POST /api/*'`
|
||||
- Run all routes: `apophis verify --profile quick`
|
||||
- Run only changed routes in CI: `apophis verify --profile ci --changed`
|
||||
- Run only changed routes in CI: `apophis verify --profile quick --changed`
|
||||
- Requires a git repository.
|
||||
- Use machine-readable output in CI: `apophis verify --profile ci --format json-summary`
|
||||
- Add observe mode for runtime drift detection: see [observe.md](observe.md)
|
||||
|
||||
+12
-12
@@ -29,21 +29,19 @@ Observe mode requires a reporting sink. Configure it in your environment policy:
|
||||
environments: {
|
||||
staging: {
|
||||
name: 'staging',
|
||||
allowVerify: true,
|
||||
allowObserve: true,
|
||||
allowQualify: false,
|
||||
allowChaos: false,
|
||||
allowBlocking: false,
|
||||
requireSink: true
|
||||
allowedModes: ['verify', 'observe'],
|
||||
blockQualify: true,
|
||||
requireSink: true,
|
||||
allowBlocking: false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
APOPHIS supports these sink types:
|
||||
You implement sinks for your observability backend. Common categories:
|
||||
|
||||
- **Logs**: Structured logging of contract violations
|
||||
- **Metrics**: Counter and histogram metrics for violation rates
|
||||
- **Traces**: Distributed tracing integration for violation context
|
||||
- **Logs**: Structured logging of contract violations (pino, winston)
|
||||
- **Metrics**: Counter and histogram metrics for violation rates (Prometheus, OpenTelemetry)
|
||||
- **Traces**: Distributed tracing integration for violation context (OpenTelemetry, Jaeger)
|
||||
|
||||
## Sampling
|
||||
|
||||
@@ -97,10 +95,12 @@ profiles: {
|
||||
|
||||
Validate your observe config before deployment with doctor:
|
||||
|
||||
## Exit Codes
|
||||
## Validation (via Doctor)
|
||||
|
||||
Validate observe configuration with `apophis doctor --mode observe`:
|
||||
|
||||
| Code | Meaning |
|
||||
|---|---|
|
||||
|---|---|---|
|
||||
| 0 | Observe config is valid and safe |
|
||||
| 2 | Safety violation or invalid config |
|
||||
|
||||
|
||||
+3
-1
@@ -183,7 +183,9 @@ Qualify mode is gated away from production by default:
|
||||
|---|---|---|---|
|
||||
| local | enabled | enabled | enabled |
|
||||
| test/CI | enabled | enabled | enabled |
|
||||
| staging | enabled with allowlist | enabled | blocked on protected routes |
|
||||
| staging | enabled (blockQualify: false) | enabled (blockQualify: false) | blocked on protected routes (allowChaosOnProtected: false) |
|
||||
|
||||
Qualify gates are not individually gated per environment. The `blockQualify` flag controls all qualify execution, and `allowChaosOnProtected` controls chaos on protected routes.
|
||||
| production | disabled by default | disabled by default | disabled by default |
|
||||
|
||||
## Machine Output for CI
|
||||
|
||||
+9
-3
@@ -71,7 +71,9 @@ Automatically rerun failing tests with varied seeds to detect non-deterministic
|
||||
### Usage
|
||||
|
||||
```javascript
|
||||
import { FlakeDetector } from '@apophis/fastify'
|
||||
// FlakeDetector is an internal API. It is not part of the public @apophis/fastify export.
|
||||
// For programmatic use, import from the internal path:
|
||||
// import { FlakeDetector } from '@apophis/fastify/src/quality/flake.js'
|
||||
|
||||
const detector = new FlakeDetector({
|
||||
sameSeedReruns: 1, // Rerun with same seed
|
||||
@@ -121,7 +123,9 @@ Measure contract strength by injecting synthetic bugs. A "mutation" is a small c
|
||||
### Usage
|
||||
|
||||
```javascript
|
||||
import { runMutationTesting } from '@apophis/fastify/quality/mutation'
|
||||
// Mutation testing is an internal API. It is not part of the public @apophis/fastify export.
|
||||
// For programmatic use, import from the internal path:
|
||||
// import { runMutationTesting } from '@apophis/fastify/src/quality/mutation.js'
|
||||
|
||||
const report = await runMutationTesting(fastify, {
|
||||
runs: 10,
|
||||
@@ -176,7 +180,9 @@ console.log('Weak contracts:', report.weakContracts)
|
||||
Test a specific mutation without running the full suite:
|
||||
|
||||
```javascript
|
||||
import { testMutation } from '@apophis/fastify/quality/mutation'
|
||||
// Mutation testing is an internal API. It is not part of the public @apophis/fastify export.
|
||||
// For programmatic use, import from the internal path:
|
||||
// import { testMutation } from '@apophis/fastify/src/quality/mutation.js'
|
||||
|
||||
const killed = await testMutation(fastify, contract, mutation, {
|
||||
runs: 10,
|
||||
|
||||
@@ -38,10 +38,10 @@ APOPHIS classifies failures into six categories. Lower categories take precedenc
|
||||
1. Check the route and clause index printed in the error message.
|
||||
2. Verify APOSTL syntax: use `response_code(this)` not `response_code()`.
|
||||
3. Ensure string literals use single or double quotes consistently.
|
||||
4. Run `apophis doctor --profile <name>` to validate formulas without executing.
|
||||
4. Run `apophis doctor` to validate formulas without executing.
|
||||
|
||||
**Prevention**
|
||||
- Run `apophis doctor --profile <name>` to validate formulas without executing.
|
||||
- Run `apophis doctor` to validate formulas without executing.
|
||||
- Enable editor support for APOSTL syntax highlighting.
|
||||
|
||||
---
|
||||
|
||||
+1
-1
@@ -206,4 +206,4 @@ presets: {
|
||||
}
|
||||
```
|
||||
|
||||
CLI verify generates one property-based test sample per contract by default when no `runs` is specified. Set `runs` in the preset to increase sampled inputs per route.
|
||||
CLI verify defaults to 50 runs per contract. Set `runs` in the preset to adjust the sample count per route. Use `runs: 1` to check each contract once, or `runs: 0` to disable property-based verification.
|
||||
|
||||
Reference in New Issue
Block a user