apophis-fastify/docs/quality.md

# Quality Engines

APOPHIS includes three quality engines for advanced testing: chaos injection, flake detection, and mutation testing. All require `NODE_ENV=test`.

## Chaos Injection

Inject controlled failures into contract tests to validate resilience guarantees. Chaos events are generated by fast-check alongside test data, making them shrinkable — when a test fails, fast-check finds the minimal chaos event that causes the failure.

### Usage

```javascript
const result = await fastify.apophis.contract({
  runs: 50,
  chaos: {
    delay: { probability: 0.1, minMs: 100, maxMs: 500 },
    error: { probability: 0.1, statusCode: 503 },
    dropout: { probability: 0.05 },
    corruption: { probability: 0.1 },
  },
})
```

### Event Types

| Type | Effect | Tests |
|------|--------|-------|
| `delay` | Artificial latency | `response_time(this) < 1000` |
| `error` | Forces HTTP status code | Error-handling contracts |
| `dropout` | Network failure (status 0 or 504) | Fallback contracts |
| `corruption` | Mutates response bodies | Parsing robustness |

### Corruption Strategies

| Strategy | Effect |
|----------|--------|
| `truncate` | Cuts response body in half |
| `malformed` | Returns invalid JSON (`{"broken":`) |
| `field-corrupt` | Sets a random field to `null` |

### Programmatic API

```javascript
import {
  applyChaosToExecution,
  createChaosEventArbitrary,
  formatChaosEvents,
} from '@apophis/fastify'

// Apply pre-generated chaos events to a context
const result = applyChaosToExecution(ctx, events)

// Generate deterministic chaos events
const arb = createChaosEventArbitrary(config, contractNames)
const events = fc.sample(arb, { numRuns: 1, seed: 42 })[0]

// Format for diagnostics
console.log(formatChaosEvents(events))
```

### Best Practices

1. Start small: `probability: 0.05` (5% of requests)
2. Test one failure mode at a time
3. Verify contracts handle chaos: `if status:503 then response_code(GET /health) == 200`
4. Use seeds for reproducibility: `seed: 42`

## Flake Detection

Automatically rerun failing tests with varied seeds to detect non-deterministic contracts. A "flake" is a test that fails on one run but passes on another with the same or different seed.

### Usage

```javascript
import { FlakeDetector } from '@apophis/fastify'

const detector = new FlakeDetector({
  sameSeedReruns: 1,    // Rerun with same seed
  seedVariations: 3,    // Try 3 additional seeds
})

const report = await detector.detectFlake(
  originalFailingResult,
  async (seed) => {
    const suite = await fastify.apophis.contract({ seed })
    return { passed: suite.summary.failed === 0 }
  },
  originalSeed
)

if (report.isFlaky) {
  console.log(`Flaky with ${report.confidence} confidence`)
  console.log('Reruns:', report.reruns)
}
```

### Report Structure

```javascript
{
  isFlaky: true,
  confidence: 'high', // 'high' | 'medium' | 'low'
  reruns: [
    { seed: 42, passed: false },
    { seed: 43, passed: true },
  ]
}
```

### Confidence Scoring

| Pass Rate | Confidence |
|-----------|------------|
| 0% pass | `high` (deterministic failure) |
| < 50% pass | `medium` |
| >= 50% pass | `low` (likely flaky) |

## Mutation Testing

Measure contract strength by injecting synthetic bugs. A "mutation" is a small change to a contract (e.g., flip `==` to `!=`). If the test suite catches the mutation (fails), the mutation is "killed". If it passes, the mutation "survives" — indicating weak coverage.

### Usage

```javascript
import { runMutationTesting } from '@apophis/fastify/quality/mutation'

const report = await runMutationTesting(fastify, {
  runs: 10,
  seed: 42,
  maxMutationsPerContract: 5,
  routes: ['/items'], // Optional: only test these routes
})

console.log(`Mutation score: ${report.score}%`)
console.log(`Killed: ${report.killed}, Survived: ${report.survived}`)
console.log('Weak contracts:', report.weakContracts)
```

### Mutation Operators

| Type | Example |
|------|---------|
| `flip-operator` | `== 201` → `!= 201` |
| `change-number` | `== 200` → `== 201` |
| `remove-clause` | `A && B` → `A` |
| `negate-boolean` | `== true` → `== false` |
| `swap-variable` | `response_body` → `request_body` |
| `remove-ensures` | Remove one ensures clause entirely |

### Report Structure

```javascript
{
  score: 85,              // 0-100
  killed: 17,
  survived: 3,
  durationMs: 4500,
  weakContracts: ['POST /items'], // Routes where no mutations were killed
  mutations: [
    {
      mutation: {
        id: 'm0',
        route: 'POST /items',
        original: 'response_code(this) == 201',
        mutated: 'response_code(this) != 201',
        type: 'flip-operator',
      },
      killed: true,
      durationMs: 120,
    }
  ]
}
```

### Single Mutation Test

Test a specific mutation without running the full suite:

```javascript
import { testMutation } from '@apophis/fastify/quality/mutation'

const killed = await testMutation(fastify, contract, mutation, {
  runs: 10,
  seed: 42,
})
```

## Environment Guard

All quality engines require `NODE_ENV=test`:

```
Error: chaos is only available in test environment.
Set NODE_ENV=test to enable quality features.
```

This prevents accidental execution in production or development.

## Integration Example

Run all three engines in a CI pipeline:

```javascript
// 1. Standard contract tests
const suite = await fastify.apophis.contract({ runs: 50, seed: 42 })

// 2. Chaos tests
const chaosSuite = await fastify.apophis.contract({
  runs: 50,
  seed: 42,
  chaos: { error: { probability: 0.1, statusCode: 503 } },
})

// 3. Flake detection on failures
for (const test of suite.tests.filter(t => !t.ok)) {
  const report = await detector.detectFlake(test, rerunFn, 42)
  if (report.isFlaky) {
    console.warn(`Flaky test detected: ${test.name}`)
  }
}

// 4. Mutation testing
const mutationReport = await runMutationTesting(fastify, { runs: 10 })
if (mutationReport.score < 80) {
  console.warn(`Low mutation score: ${mutationReport.score}%`)
}
```