fix: harden engine, enrich failure diagnostics, close adoption gaps
- P0: CLI verify now honors test budget with seeded multi-sample - P0: Observe sampling enforced via Math.random() gate in hook-validator - P1: Remove misleading undici-mock-agent isolation option - P1: Qualify reuses shared discoverRouteDetails() with warnings - P1: Chaos/scenario config exposed via preset schema - P1: README/docs limitations updated to current state - P2: Nested response annotations prefer 2xx deterministically - P2: --changed documented as heuristic in verify.md - Add observe sink tests (sampling 0/1, sink failure non-interference) - Add verify runs regression tests (scale, determinism, variants) - Add configured-scenario qualify test (independent of OAuth fixture) - Add coverageBreakdown to qualify artifacts (per-gate route coverage) - Add production-style observe example with real sink in docs/observe.md - Add nightly/staging vs PR gating guidance to docs/qualify.md - Enrich VerifyFailure with formula-aware diagnostics: status:201 => 'HTTP 200', body field checks => actual values - Remove stale observe CLI activation message - Document outbound mocks as process-global in getting-started.md - Refresh APOPHIS_ADOPTION_AUDIT.md with current state 903 tests pass, build clean, typecheck clean.
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
|
||||
APOPHIS is designed to be safe and predictable for LLM-generated Fastify services.
|
||||
|
||||
It applies the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) to LLM-assisted development: constrained vocabulary, deterministic replay, and executable contracts give coding agents a verifiable loop between generated changes and behavioral correctness.
|
||||
It applies an invariant-driven approach to LLM-assisted development: constrained vocabulary, deterministic replay, and executable contracts give coding agents a verifiable loop between generated changes and behavioral correctness.
|
||||
|
||||
## Why APOPHIS Is Good for LLM-Generated Services
|
||||
|
||||
@@ -164,7 +164,7 @@ console.log(verifyResult);
|
||||
2. **Run doctor first**: Catch setup issues before running verify.
|
||||
3. **Use `--changed` in CI**: Only verify routes that changed in the PR.
|
||||
4. **Commit config**: Store `apophis.config.js` in version control.
|
||||
5. **Pin versions**: Pin `apophis-fastify` version in `package.json`.
|
||||
5. **Pin versions**: Pin `@apophis/fastify` version in `package.json`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
|
||||
Reference in New Issue
Block a user