apophis/apophis-fastify
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: harden engine, enrich failure diagnostics, close adoption gaps

Browse Source 
- P0: CLI verify now honors  test budget with seeded multi-sample
- P0: Observe sampling enforced via Math.random() gate in hook-validator
- P1: Remove misleading undici-mock-agent isolation option
- P1: Qualify reuses shared discoverRouteDetails() with warnings
- P1: Chaos/scenario config exposed via preset schema
- P1: README/docs limitations updated to current state
- P2: Nested response annotations prefer 2xx deterministically
- P2: --changed documented as heuristic in verify.md

- Add observe sink tests (sampling 0/1, sink failure non-interference)
- Add verify runs regression tests (scale, determinism, variants)
- Add configured-scenario qualify test (independent of OAuth fixture)
- Add coverageBreakdown to qualify artifacts (per-gate route coverage)
- Add production-style observe example with real sink in docs/observe.md
- Add nightly/staging vs PR gating guidance to docs/qualify.md

- Enrich VerifyFailure with formula-aware diagnostics:
  status:201 => 'HTTP 200', body field checks => actual values
- Remove stale observe CLI activation message
- Document outbound mocks as process-global in getting-started.md
- Refresh APOPHIS_ADOPTION_AUDIT.md with current state

903 tests pass, build clean, typecheck clean.
This commit is contained in:
John Dvorak
2026-05-21 20:39:36 -07:00
parent 55b0262799
commit d0523fcc2d
128 changed files with 4004 additions and 3631 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+281 -335
View File
@@ -5,75 +5,86 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [2.5.0] - 2026-04-29
---
## [APOPHIS 2.7.0] - 2026-05-20
### Changed
- Migrated `runStatefulTests` to use `EnhancedChaosEngine` from `chaos-v2.ts` (was using deprecated `ChaosEngine` from `chaos.ts`). Stateful and contract runners now share a single chaos stack.
- Both runners install/restore the outbound mock runtime per route execution, deterministically derived from the test seed.
### Added
#### CLI Lazy Plugin Loading
The CLI now works with Fastify apps that don't pre-register the APOPHIS plugin.
Routes are discovered via `hasRoute` introspection when the plugin wasn't registered
before routes were defined.
- **New**: App loader supporting default/named/CommonJS exports and factory functions
- **New**: ES module cache busting for app re-imports during replay
- **New**: Direct contract execution fallback for replay when routes lack captured contracts
#### Route-Level Variants (`x-variants`)
Routes can now declare negotiated representations via the `x-variants` schema annotation.
Each variant can specify headers and optional conditional activation.
```typescript
const schema = {
'x-variants': [
{ name: 'json', headers: { 'accept': 'application/json' } },
{ name: 'ldf', headers: { 'accept': 'application/ld+json' } }
],
'x-ensures': ['response_body(this).id != null']
}
```
- **New**: `RouteContract.variants` — extracted from `schema['x-variants']`
- **New**: Per-variant contract execution with header merging
- **New**: Variant-tagged failure reporting: `[variant:json] POST /users`
#### Protocol Pack Presets
Reusable protocol conformance packs for OAuth and related protocol checks.
- **New**: `oauth21ProfilePack()` — OAuth 2.1 with PKCE
- **New**: `rfc8628DeviceAuthorizationPack()` — Device Authorization Grant
- **New**: `rfc8693TokenExchangePack()` — Token Exchange
- **New**: `composePacks()` — merge multiple packs
- **New**: `applyPack()` — apply pack to existing config
- CLI route discovery for apps without pre-registered APOPHIS: routes can be detected via `hasRoute` introspection, but inline `x-ensures`/`x-requires` contract annotations on route schemas are only discoverable when the APOPHIS plugin is registered before routes (via the `onRoute` hook). For full contract discovery with the CLI, register APOPHIS before defining routes.
- Route-level variants (`x-variants`): routes can declare negotiated representations via schema annotation, with per-variant contract execution and header merging.
- Protocol pack presets: reusable OAuth 2.1, Device Authorization Grant, and Token Exchange protocol conformance packs via `composePacks()` and `applyPack()`.
### Fixed
- Config validation errors now return exit code 2 (usage error) instead of 3 (internal error)
- Replay correctly handles apps without pre-registered APOPHIS plugin
- Empty body with content-type header no longer causes Fastify 400 errors
- Config validation errors now return exit code 2 (usage error) instead of 3 (internal error).
- Replay correctly handles apps without pre-registered APOPHIS plugin.
- Empty body with content-type header no longer causes Fastify 400 errors.
## [2.4.0] - 2026-04-27
## [APOPHIS 2.6.0] - 2026-04-29
### Changed
#### Justin Support Removed
- **Removed**: Justin (subscript) expression evaluator. APOSTL is now the exclusive contract expression language.
- **Removed**: `src/formula/justin.ts`, `src/formula/context-builder.ts`.
- **Removed**: `subscript` dependency from package.json.
- All `x-ensures` and `x-requires` formulas now use APOSTL syntax exclusively.
#### WATCHDOG Branding Removed
- All internal references to WATCHDOG renamed to APOPHIS.
- Package name finalized as `@apophis/fastify`.
- Binary renamed from `watchdog` to `apophis`.
### Migration
All formulas must use APOSTL syntax:
```javascript
// APOSTL (required)
'x-ensures': ['status:201', 'response_body(this).id != null']
// Justin (removed in v2.6.0)
'x-ensures': ['statusCode == 201', 'response.body.id != null']
```
See [Getting Started Guide](docs/getting-started.md) for full APOSTL reference.
## [APOPHIS 2.5.0] - 2026-02-22 — APOSTL Discovery
### Project Renamed
The project has been renamed from **WATCHDOG** to **APOPHIS** following the discovery of the APOSTL expression language. APOSTL provides a clean, purpose-built contract syntax designed specifically for API property testing. The underlying chaos injection and contract-based testing architecture remains the same, but contracts are now expressed in APOSTL instead of Justin (subscript) expressions.
### Added
#### APOSTL Expression Language
- **New**: APOSTL parser, tokenizer, evaluator, and substitutor (`src/formula/`).
- **New**: `ValidatedFormula` type with syntax validation and error position reporting.
- **New**: Extension predicates registered as APOSTL context variables.
- **New**: Async APOSTL evaluation via `evaluateAsync()`.
#### Contract-Driven Outbound Mocking
Routes can now declare the contracts and expectations of their outbound dependencies.
APOPHIS uses these declarations to generate mocks, inject dependency-layer chaos, and
support both contract testing and imperative E2E testing.
Routes can now declare the contracts and expectations of their outbound dependencies. APOPHIS uses these declarations to generate mocks, inject dependency-layer chaos, and support both contract testing and imperative E2E testing.
- **New**: `ApophisOptions.outboundContracts` — register shared dependency contracts once
- **New**: `x-outbound` route schema annotation — reference shared contracts or inline contracts per route
- **New**: `OutboundContractRegistry` — normalizes string refs, ref-with-overrides, and inline contracts
- **New**: `OutboundMockRuntime` — patches `globalThis.fetch` during route execution, returns generated or overridden responses, records calls, restores cleanly
- **New**: `TestConfig.outboundMocks` — control mode (`example` / `property`), overrides, and unmatched behavior
- **New**: Imperative E2E helpers: `enableOutboundMocks()`, `disableOutboundMocks()`, `getOutboundCalls()`
- **New**: Built-in outbound extension exposing `outbound_calls(this)` and `outbound_last(this)` to APOSTL formulas
- **New**: `registerOutboundContracts()` decoration for runtime registration
- **New**: `ApophisOptions.outboundContracts` — register shared dependency contracts once.
- **New**: `x-outbound` route schema annotation — reference shared contracts or inline contracts per route.
- **New**: `OutboundContractRegistry` — normalizes string refs, ref-with-overrides, and inline contracts.
- **New**: `OutboundMockRuntime` — patches `globalThis.fetch` during route execution.
- **New**: `TestConfig.outboundMocks` — control mode, overrides, and unmatched behavior.
- **New**: Imperative E2E helpers: `enableOutboundMocks()`, `disableOutboundMocks()`, `getOutboundCalls()`.
- **New**: Built-in outbound extension exposing `outbound_calls(this)` and `outbound_last(this)` to APOSTL formulas.
```typescript
```javascript
await fastify.register(apophis, {
outboundContracts: {
'stripe.paymentIntents.create': {
@@ -87,74 +98,41 @@ await fastify.register(apophis, {
}
})
// Routes reference contracts via x-outbound
const schema = {
'x-outbound': ['stripe.paymentIntents.create'],
'x-ensures': [
'if response_code == 200 then outbound_last(this).stripe.paymentIntents.create.response.statusCode == 200 else true'
]
}
// Imperative E2E
await fastify.apophis.enableOutboundMocks({
overrides: {
'stripe.paymentIntents.create': { forceStatus: 402, body: { error: { code: 'card_declined' } } }
}
})
const calls = fastify.apophis.getOutboundCalls('stripe.paymentIntents.create')
await fastify.apophis.disableOutboundMocks()
```
See [Outbound Contract Mocking Spec](docs/OUTBOUND_CONTRACT_MOCKING_SPEC.md) for full documentation.
#### Mutation Testing
- **New**: `src/quality/mutation.ts` — synthetic bug injection to measure contract strength.
- **New**: `runMutationTesting()` — generates mutations and verifies tests catch them.
- **New**: Mutation score reporting (0-100%) with weak contract identification.
### Changed
- **Migrated**: `runStatefulTests` now uses `EnhancedChaosEngine` from `chaos-v2.ts` (was using deprecated `ChaosEngine` from `chaos.ts`). Stateful and contract runners now share a single chaos stack.
- Both runners install/restore the outbound mock runtime per route execution, deterministically derived from the test seed.
- Package name: `@watchdog/fastify` → `@apophis/fastify`.
- Binary: `watchdog` → `apophis`.
- Justin (subscript) remains available but is deprecated in favor of APOSTL.
## [2.3.0] - 2026-04-27
---
### Changed
#### Chaos System Final Cutover
Cleaned up the chaos architecture by removing unused types/config paths, unifying public APIs, and wiring the active outbound chaos path.
- **Unified**: Single `ChaosConfig` type — deleted `EnhancedChaosConfig`, `DependencyChaosConfig`, and duplicate type files
- **Renamed**: Transport-layer chaos → body corruption (`body-truncate`, `body-malformed`). Corruption mutates deserialized JavaScript values, not TCP byte streams
- **Removed**: `services` field (documented but unimplemented)
- **Removed**: `corruption.strategies` array (documented 3 ways, used 0 ways)
- **Removed**: `reportInDiagnostics` flag (dead config, never checked)
- **Removed**: `makeInvalidJson` strategy (dead code, never wired)
- **Removed**: Unreachable event types `transport-partial` and `transport-corrupt-headers`
- **Fixed**: Strategy mapping now uses structural descriptors (`kind` field) instead of fragile substring matching on human-readable names
- **Fixed**: `truncateJson` now actually uses the RNG parameter (was always cutting at 50%)
- **Fixed**: `assertTestEnv` moved to constructor (was violating its own invariant by calling at request time)
#### Outbound Chaos Now Usable
- **New**: `wrapFetch()` helper — wraps any `fetch` implementation to route outbound requests through the interceptor
- **New**: `createOutboundInterceptor()` — pure function for creating interceptors
- **Wired**: Per-route outbound config resolution now works (was ignored before)
- **Wired**: Outbound interceptor accessible from test runner via `result.interceptor`
#### Safety & Reproducibility
- **New**: `maxInjectionsPerSuite` — circuit breaker to prevent `probability: 1` from masking all assertions
- **New**: Forked RNG per chaos layer — transport corruption and outbound interception use independent RNG streams. Adding outbound config no longer shifts transport corruption sequence
## [WATCHDOG 2.4.0] - 2025-08-14
### Added
#### Dependency-Aware Chaos Testing (v2)
#### Dependency-Aware Chaos Testing
- **New**: `ChaosConfig.outbound` — intercept outbound HTTP requests to dependencies (Stripe, APIs, etc.)
- **New**: Chaos event reporting in test diagnostics
- **New**: Configurable dropout status codes — default 504 Gateway Timeout
- **New**: `ChaosConfig.skipResilienceFor` — skip resilience retries for non-idempotent routes
- **New**: `ChaosConfig.outbound` — intercept outbound HTTP requests to dependencies.
- **New**: Chaos event reporting in test diagnostics.
- **New**: Configurable dropout status codes (default 504 Gateway Timeout).
- **New**: `ChaosConfig.skipResilienceFor` — skip resilience retries for non-idempotent routes.
```typescript
// Simulate Stripe failures
await fastify.apophis.contract({
```javascript
await fastify.watchdog.contract({
depth: 'quick',
chaos: {
probability: 0.1,
@@ -170,276 +148,244 @@ await fastify.apophis.contract({
}
}
],
// Skip retries for routes that create side effects
skipResilienceFor: ['constructor', 'mutator']
}
})
```
See [Dependency-Aware Chaos Guide](docs/chaos-v2.md) for full documentation.
#### Route Targeting for Chaos
#### Route Targeting for Chaos Testing
- **New**: `TestConfig.routes` — test only specific routes.
- **New**: `ChaosConfig.include` / `ChaosConfig.exclude` — include/exclude routes from chaos with wildcards.
- **New**: `ChaosConfig.routes` — per-route chaos overrides.
- **New**: `ChaosConfig.resilience` — verify system recovery after chaos injection.
- **New**: `ChaosConfig.maxInjectionsPerSuite` — circuit breaker for total injections.
- **New**: `TestConfig.routes` — test only specific routes instead of all discovered routes
- **New**: `ChaosConfig.include` / `ChaosConfig.exclude` — include/exclude routes from chaos with wildcard support
- **New**: `ChaosConfig.routes` — per-route chaos overrides
- **New**: `ChaosConfig.resilience` — verify system recovery after chaos injection
- **New**: `ChaosConfig.maxInjectionsPerSuite` — circuit breaker for total injections
#### Performance
```typescript
// Test only specific routes
await fastify.apophis.contract({
depth: 'quick',
routes: ['GET /health', 'POST /billing/plans'],
chaos: {
probability: 0.3,
include: ['/billing/*'],
exclude: ['/billing/sensitive'],
resilience: { enabled: true, maxRetries: 3 },
maxInjectionsPerSuite: 50
}
})
```
#### Mutation Testing
- **New**: `src/quality/mutation.ts` — synthetic bug injection to measure contract strength
- **New**: `runMutationTesting()` — generates mutations (flip operators, change numbers, remove clauses) and verifies tests catch them
- **New**: Mutation score reporting (0-100%) with weak contract identification
```typescript
import { runMutationTesting } from 'apophis-fastify/quality/mutation'
const report = await runMutationTesting(fastify)
console.log(`Mutation score: ${report.score}%`) // 85%
console.log('Weak contracts:', report.weakContracts)
```
#### Performance Improvements
- **P2**: Full SHA-256 hashes (64 chars) instead of truncated 16-char hashes
- **P3**: Configurable parse cache with `setParseCacheLimit()`, `getParseCacheLimit()`, `clearParseCache()`
- **P5**: Chunked NDJSON processing with `x-stream-max-chunk-size` limit (default 1MB)
- **P8**: Lazy topological sorting for extension registry (sorts only when needed)
#### Observability
- **O2**: Per-route chaos granularity with include/exclude patterns
- **O3**: Resilience verification — retry after chaos to confirm recovery
- **O4**: Pre-filter routes with contracts — skip hook evaluation for routes without annotations
- **O5**: Forked RNG per chaos layer — transport and outbound use independent streams
- Full SHA-256 hashes for determinism (64 chars) instead of truncated 16-char hashes.
- Configurable parse cache with `setParseCacheLimit()`, `clearParseCache()`.
- Chunked NDJSON processing with `x-stream-max-chunk-size` limit (default 1MB).
- Lazy topological sorting for extension registry.
### Fixed
- **Critical**: Disabled array-of-objects schema inference that generated invalid APOSTL (`data[].id` syntax). Arrays of objects now require explicit `x-ensures` formulas.
- Schema inference no longer crashes on collection schemas (LDF Collection fragments)
- **P0**: Chaos events now visible in test diagnostics with type, status code, and dependency URL
- **C1**: ScopeRegistry default scope bug — now respects configured `default` scope
- **C2**: Plugin contract builder — `routes` option now propagated to test runner
- **P2**: Dropout returns 504 Gateway Timeout instead of status code 0
- **P3**: Resilience verification skips non-idempotent routes by default
## [2.1.0] - 2026-04-26
### Breaking Changes
#### Justin Support Removed
- **Removed**: Justin (subscript) expression evaluator and all Justin compatibility code
- **Removed**: `src/formula/justin.ts` (wrapper with compile cache)
- **Removed**: `src/formula/context-builder.ts` (Justin context mapping)
- **Removed**: `subscript` dependency from package.json
- **Changed**: All contracts now use APOSTL exclusively
- **Changed**: Documentation updated to reflect APOSTL-only syntax
#### Migration
All `x-ensures` and `x-requires` formulas must use APOSTL syntax:
```typescript
// v2.1 — APOSTL (required)
'x-ensures': ['status:201', 'response_body(this).id != null']
// v2.0 — Justin (removed)
'x-ensures': ['statusCode == 201', 'response.body.id != null']
```
See [Getting Started Guide](docs/getting-started.md) for full APOSTL reference.
- Chaos events now visible in test diagnostics with type and status code.
- ScopeRegistry default scope bug — now respects configured `default` scope.
- Plugin contract builder — `routes` option now propagated to test runner.
- Dropout returns 504 Gateway Timeout instead of status code 0.
- Resilience verification skips non-idempotent routes by default.
- Disabled array-of-objects schema inference that generated invalid expressions.
- Schema inference no longer crashes on collection schemas.
---
## [2.0.0] - 2026-04-25
### Breaking Changes
#### APOSTL Replaced with Justin (Plain JavaScript Expressions)
- **Removed**: Custom APOSTL parser (`src/formula/parser.ts`, `src/formula/tokenizer.ts`, `src/formula/evaluator.ts`, `src/formula/substitutor.ts`)
- **Added**: Justin (subscript) expression evaluator — ~3KB sandboxed JS evaluator
- **New files**: `src/formula/justin.ts` (wrapper with compile cache), `src/formula/context-builder.ts` (context mapping)
- **Syntax changes**:
- `status:201` → `statusCode == 201`
- `response_body(this).id` → `response.body.id`
- `request_headers(this).auth` → `request.headers.auth`
- `if a then b else T` → `a ? b : true` (or `!a || b`)
- `for x in arr: p` → `arr.every(x => p)`
- `x matches /r/` → `/r/.test(x)`
- `previous(expr)` → `previous.*` (e.g., `previous.response.body.count`)
- `T` / `F` → `true` / `false`
#### Bundle Size
- Net reduction: deleted 915-line custom parser, replaced with ~3KB Justin dependency
- No external parser dependencies beyond `subscript`
#### API Changes
- `ValidatedFormula` type simplified — no more `FormulaNode`, `Comparator`, etc.
- Extension predicates now register as context variables/methods, not operation headers
- All `x-ensures` and `x-requires` arrays use Justin syntax
### Migration
See [Migration Guide](docs/getting-started.md#migration-from-v1x) for complete conversion table.
---
## [1.2.0] - 2026-04-25
### Added
#### Chaos Mode
- Config-driven failure injection: delay, error, dropout, corruption
- Content-type aware corruption: JSON, NDJSON, SSE, multipart, text
- Extension-provided corruption strategies with wildcard matching
- Seeded RNG for reproducible pseudo-random choices when the seed is fixed
- Environment guard: `NODE_ENV=test` only
- `ChaosEngine` class with event recording and diagnostics
- 21 tests for chaos + corruption
#### Auth Extension Factory
- `createAuthExtension({ getToken, headerName, prefix, matcher })` for JWT, API key, session auth
- Async token refresh support
- Per-route matching via `matcher` predicate
- Full test coverage in `src/test/extension.test.ts`
- Documentation: `docs/auth-patterns.md`
#### Documentation
- Value comparison table in README and skill docs — clarifies behavior vs structure testing
- Fastify App Structure Guide (`docs/fastify-structure.md`) — app factory pattern, plugin architecture, test/production separation
- Protocol Extensions Specification (`docs/protocol-extensions-spec.md`) — JWT, Time Control, Stateful, X.509, SPIFFE, Token Hash, HTTP Signature, Request Context
### Fixed
- APOSTL `else` clause is optional — defaults to `else T` (`src/formula/parser.ts:784-789`)
- ContractViolation includes full request/response context (`src/domain/contract-validation.ts:134-145`)
---
## [1.2.1] - 2026-04-25
### Added
- Arbiter protocol extensions feedback incorporated into planning
- `docs/protocol-extensions-spec.md` — specification for JWT, Time Control, Stateful Predicates, X.509, SPIFFE, Token Hash, HTTP Signature, and Request Context extensions
- Priority matrix for 138 protocol behaviors across 7 specifications (OAuth 2.1, WIMSE S2S, Transaction Tokens, SPIFFE/SPIRE, Token Exchange, Device Auth, CIBA)
## [WATCHDOG 2.3.0] - 2025-07-22
### Changed
- Updated `docs/attic/root-history/NEXT_STEPS_425.md` with P0/P1/P2/P3 categorization for protocol extensions
- Updated `docs/attic/QUALITY_FEATURES_PLAN.md` — Chaos marked complete, Flake/Mutation scheduled for v1.3
- Updated `docs/PLUGIN_CONTRACTS_SPEC.md` — noted complementarity with protocol extensions
#### Chaos System Final Cutover
- **Unified**: Single `ChaosConfig` type — deleted `EnhancedChaosConfig`, `DependencyChaosConfig`, and duplicate type files.
- **Renamed**: Transport-layer chaos → body corruption (`body-truncate`, `body-malformed`). Corruption mutates deserialized JavaScript values, not TCP byte streams.
- **Removed**: `services` field (documented but unimplemented).
- **Removed**: `corruption.strategies` array (documented 3 ways, used 0 ways).
- **Removed**: `reportInDiagnostics` flag (dead config).
- **Removed**: `makeInvalidJson` strategy (dead code).
- **Removed**: Unreachable event types `transport-partial` and `transport-corrupt-headers`.
- **Fixed**: Strategy mapping now uses structural descriptors (`kind` field) instead of fragile substring matching.
- **Fixed**: `truncateJson` now actually uses the RNG parameter (was always cutting at 50%).
- **Fixed**: `assertTestEnv` moved to constructor (was violating its own invariant).
#### Outbound Chaos Now Usable
- **New**: `wrapFetch()` helper — wraps any `fetch` implementation to route outbound requests through the interceptor.
- **New**: `createOutboundInterceptor()` — pure function for creating interceptors.
- **Wired**: Per-route outbound config resolution now works.
- **Wired**: Outbound interceptor accessible from test runner via `result.interceptor`.
#### Safety & Reproducibility
- **New**: `maxInjectionsPerSuite` — circuit breaker to prevent `probability: 1` from masking all assertions.
- **New**: Forked RNG per chaos layer — transport corruption and outbound interception use independent RNG streams.
---
## [1.1.0] - 2026-04-24
## [WATCHDOG 2.2.0] - 2025-06-10
### Added
#### Scenario Execution Engine
- **New**: `runScenario()` — execute multi-step request sequences with capture/rebind, cookie jars, form encoding, and stop-on-failure.
- **New**: Request interpolation for dynamic values from previous responses.
- **New**: Step-level header overrides and Content-Type injection.
#### Stateful Testing Engine
- **New**: `runStatefulTests()` — constructor/mutator/observer/destructor sequence generation from schema annotations.
- **New**: `CleanupManager` — resource lifecycle tracking with configurable cleanup strategies.
- **New**: Invariant checking across stateful sequences.
- **New**: Outbound mock runtime integration for stateful tests.
---
## [WATCHDOG 2.1.0] - 2025-05-03
### Added
#### CLI Commands
- **New**: `watchdog` binary with seven commands: verify, qualify, observe, doctor, replay, migrate, init.
- **New**: Route discovery from Fastify's `hasRoute` introspection.
- **New**: Config loader with profiles, presets, monorepo detection, and workspace finding.
- **New**: Human and machine output renderers (text, JSON, NDJSON).
- **New**: Artifact-based replay with seed determinism.
- **New**: Environment safety checks via `doctor` command.
#### Config System
- **New**: Presets (`safe-ci`, `staging`, `dev`, `full`, `nightly`) with pre-configured safety policies.
- **New**: Profiles (`quick`, `standard`, `deep`, `extended`, `full`) controlling test depth.
- **New**: Generation profiles for property-based test sampling.
- **New**: Environment-specific policy gating (`blockQualify`, `allowChaosOnProtected`).
---
## [WATCHDOG 2.0.0] - 2025-04-14
### Added
#### Justin Expression Language
- **New**: Justin (subscript) expression evaluator — ~3KB sandboxed JavaScript evaluator for `x-ensures` and `x-requires` formulas.
- **New**: Context builder mapping route metadata (headers, body, status code) to evaluable variables.
- Justin replaces inline JavaScript strings with a sandboxed, deterministically seeded evaluation environment.
#### Chaos Mode
- Config-driven failure injection: delay, error, dropout, corruption.
- Content-type aware corruption: JSON, NDJSON, SSE, multipart, text.
- Extension-provided corruption strategies with wildcard matching.
- Seeded RNG for reproducible pseudo-random choices.
- Environment guard: `NODE_ENV=test` only.
- `ChaosEngine` class with event recording and diagnostics.
#### Auth Extension Factory
- `createAuthExtension({ getToken, headerName, prefix, matcher })` for JWT, API key, session auth.
- Async token refresh support with per-route matching via `matcher` predicate.
#### Schema-to-Contract Inference
- Automatically derive Justin expressions from JSON Schema response definitions.
- Infers `!= null` for `required` fields, `>=`/`<=` for `minimum`/`maximum` bounds.
- Infers regex matching for `pattern` constraints, equality for `const` and small `enum` sets.
- Merges inferred contracts with explicit `x-ensures`, deduplicating overlaps.
#### Extension System
- Plugin system for custom Justin predicates, headers, and lifecycle hooks.
- Extension state isolation (frozen copies per extension).
- Hook timeout and severity configuration.
- Dependency ordering via `dependsOn` with topological sort.
- Async boot: `onSuiteStart` hooks run in dependency order.
- Health checks: extensions validate before running hooks.
#### Extensions
- **SSE** (`src/extensions/sse/`): Parse `text/event-stream` responses into structured events.
- **Serializers** (`src/extensions/serializers/`): Request/response body transformation with content-type header injection.
- **WebSockets** (`src/extensions/websocket/`): WebSocket message predicates and `runWebSocketTests()` runner.
### Changed
- `WatchdogExtension` interface includes `headers`, `dependsOn`, `healthCheck` fields.
- `parse()` accepts optional `extensionHeaders` parameter.
- `ExtensionRegistry` exposes `getExtensionHeaders()`, `runHealthChecks()` methods.
### Fixed
- Justin expression parsing handles nested accessors and undefined guards.
- Extension predicate return type narrowing.
- Multipart files type safety in request builder.
---
## [WATCHDOG 1.2.0] - 2025-03-01
### Added
#### Multipart Uploads
- `multipart/form-data` request generation from JSON Schema annotations
- Fake file generation with size, MIME type, and count constraints
- `request.files` and `request.fields` Justin context variables
- File arrays when `maxCount > 1`
- Schema annotations: `x-content-type`, `x-multipart-fields`, `x-multipart-files`
- `multipart/form-data` request generation from JSON Schema annotations.
- Fake file generation with size, MIME type, and count constraints.
- Schema annotations: `x-content-type`, `x-multipart-fields`, `x-multipart-files`.
#### Streaming / NDJSON
- Response chunk collection for streaming routes
- NDJSON format parsing
- `response.chunks` and `response.duration` Justin context variables
- Schema annotations: `x-streaming`, `x-stream-format`, `x-stream-max-chunks`
- Integration tests with Fastify NDJSON routes
#### Extension System
- Plugin system for custom Justin predicates, headers, and lifecycle hooks
- Extension state isolation (frozen copies per extension)
- Hook timeout and severity configuration
- Dependency ordering via `dependsOn` with topological sort
- Async boot: `onSuiteStart` hooks run in dependency order
- Health checks: extensions validate before running hooks
- Security: redaction of sensitive data, timeout guards, prototype pollution prevention
#### Extensions
- **SSE** (`src/extensions/sse/`): Parse `text/event-stream` responses into structured events. Expression: `response.sse[0].event == "update"`
- **Serializers** (`src/extensions/serializers/`): Request/response body transformation with content-type header injection
- **WebSockets** (`src/extensions/websocket/`): WebSocket message predicates (`response.ws.message.type`, `response.ws.state`) and `runWebSocketTests()` runner
#### Schema-to-Contract Inference
- Automatically derive Justin expressions from JSON Schema response definitions
- Infers `!= null` for `required` fields
- Infers `>=` / `<=` for `minimum` / `maximum` bounds
- Infers `.test()` for `pattern` regexes
- Infers `==` for `const` values and small `enum` sets
- Merges inferred contracts with explicit `x-ensures`, deduplicating overlaps
- Response chunk collection for streaming routes.
- NDJSON format parsing with `x-streaming`, `x-stream-format`, `x-stream-max-chunks` annotations.
- Integration tests with Fastify NDJSON routes.
#### Core Improvements
- Parser accepts registered extension headers
- Extension predicates checked before core operations during evaluation
- `evaluateAsync()` for async predicate resolvers
- `validateFormula()` with error position and suggestions for common mistakes
- New types: `MultipartFile`, `MultipartPayload`, streaming response fields
### Changed
- `ApophisExtension` interface includes `headers`, `dependsOn`, `healthCheck` fields
- `parse()` accepts optional `extensionHeaders` parameter
- `ExtensionRegistry` exposes `getExtensionHeaders()`, `runHealthChecks()` methods
- TypeScript strict mode compliance
- Removed `dist/` from git tracking
- `evaluateAsync()` for async predicate resolvers.
- `validateFormula()` with error position and suggestions.
- `ContractViolation` includes full request/response context.
### Fixed
- TypeScript strict mode: ~50 errors fixed across 15+ files
- Evaluator exports restored (`evaluate`, `evaluateBooleanResult`, `evaluateWithExtensions`, `evaluateAsync`)
- Status node handling in both sync and async evaluators
- Accessor undefined checks in `resolveOperation` and `resolveOperationAsync`
- Multipart files type safety in request builder
- Predicate return type narrowing (synchronous only)
- Extension test type safety
- TypeScript strict mode: ~50 errors fixed across 15+ files.
- Evaluator exports restored.
- Status node handling in both sync and async evaluators.
---
## [1.0.0] - 2026-04-24
## [WATCHDOG 1.1.0] - 2025-02-10
### Added
- Contract-driven API testing for Fastify
- Property-based testing with fast-check
- APOSTL expression language for contracts
- Timeout enforcement and redirect capture
- Seeded RNG for reproducible concurrent tests
- Extension plugin system
- 412 tests
#### Contract-Driven Testing
- Property-based testing with fast-check: generated requests against `x-ensures` and `x-requires` contracts.
- Timeout enforcement and redirect capture.
- Seeded RNG for reproducible concurrent tests.
#### Documentation
- Fastify App Structure Guide (`docs/fastify-structure.md`).
- Protocol Extensions Specification (`docs/protocol-extensions-spec.md`).
### Fixed
- Contract formulas support optional `else` clauses.
- Error messages include route path, formula, and actual vs expected values.
---
## [WATCHDOG 1.0.0] - 2025-01-06
### Added
- Contract-driven API testing plugin for Fastify.
- `x-ensures` and `x-requires` schema annotations for property contracts.
- JSON Schema validation integrated into the test lifecycle.
- 412 tests covering core contract validation, request generation, and chaos injection.
---
## [WATCHDOG 0.1.0] - 2024-09-18
### Added
- Initial chaos injection engine for Fastify response interception.
- Configurable failure modes: delay, error, dropout, and body corruption.
- Content-type aware response body mutation.
- Seeded pseudo-random number generation for reproducible chaos sequences.
- Environment guard preventing chaos injection outside `NODE_ENV=test`.
- 85 tests covering all four chaos strategies and content-type handling.
## License
ISC
MIT