apophis-fastify/docs/observe.md

# Observe Mode

Runtime visibility and drift detection without blocking by default.

APOPHIS observe is **programmatic only**. Register the APOPHIS plugin with `observe.enabled: true` and `observe.sinks` to emit contract pass/violation/error events from live traffic without blocking responses. Sampling controls the fraction of observed requests.

Use `apophis doctor --mode observe` to validate your observe configuration (sinks, sampling, blocking, production safety) before deploying.

## When to Use It

- **Staging**: Validate observe config before promoting to production
- **Production**: Monitor contract drift without affecting requests
- **Platform teams**: Centralized visibility across services

## Safety Boundaries

Observe mode is non-blocking by default:

- **Non-blocking by default**: Contract violations are logged, not thrown
- **No request failures in non-blocking mode**: Violations are reported instead of thrown
- **Explicit opt-in for blocking**: Requires `allowBlocking: true` in environment policy
- **Production gating**: Blocking behavior is blocked in production by default

## Sink Configuration

Observe mode requires a reporting sink. Configure it in your environment policy:

```javascript
environments: {
  staging: {
    name: 'staging',
    allowVerify: true,
    allowObserve: true,
    allowQualify: false,
    allowChaos: false,
    allowBlocking: false,
    requireSink: true
  }
}
```

APOPHIS supports these sink types:

- **Logs**: Structured logging of contract violations
- **Metrics**: Counter and histogram metrics for violation rates
- **Traces**: Distributed tracing integration for violation context

## Sampling

Control observation overhead with sampling:

```javascript
profiles: {
  'staging-observe': {
    name: 'staging-observe',
    mode: 'observe',
    preset: 'platform-observe',
    routes: []
  }
}
```

The `platform-observe` preset enables sampling. Configure the rate explicitly:

```javascript
profiles: {
  'staging-observe': {
    mode: 'observe',
    preset: 'platform-observe',
    routes: [],
    sampling: 1.0  // 100% of requests observed
  }
}
```

## Staging vs Production

| Environment | Blocking | Sampling | Sink Required |
|---|---|---|---|
| Staging | No (default) | 100% | Yes |
| Production | No (default) | 100% | Yes |

Default is `1.0` (100%). Configure lower rates for production explicitly:

```javascript
profiles: {
  'prod-observe': {
    mode: 'observe',
    preset: 'platform-observe',
    routes: [],
    sampling: 0.1  // 10% of requests observed
  }
}
```

## Config Validation

Validate your observe config before deployment with doctor:

## Exit Codes

| Code | Meaning |
|---|---|
| 0 | Observe config is valid and safe |
| 2 | Safety violation or invalid config |

## Config Example

```javascript
// apophis.config.js
export default {
  mode: 'observe',
  profile: 'staging-observe',
  profiles: {
    'staging-observe': {
      name: 'staging-observe',
      mode: 'observe',
      preset: 'platform-observe',
      routes: []
    }
  },
  presets: {
    'platform-observe': {
      name: 'platform-observe',
      timeout: 10000,
      parallel: true,
      chaos: false,
      observe: true
    }
  },
  environments: {
    staging: {
      name: 'staging',
      allowVerify: true,
      allowObserve: true,
      allowQualify: false,
      allowChaos: false,
      allowBlocking: false,
      requireSink: true
    },
    production: {
      name: 'production',
      allowVerify: true,
      allowObserve: true,
      allowQualify: false,
      allowChaos: false,
      allowBlocking: false,
      requireSink: true
    }
  }
};
```

## Programmatic Runtime Activation

Register APOPHIS with observe options in your application before routes are registered.

```typescript
import Fastify from 'fastify'
import apophisPlugin from '@apophis/fastify'
import type { ObserveSink, ObserveEvent } from '@apophis/fastify'

const app = Fastify({ logger: true })

// Implement the ObserveSink interface.
// Capture events to your preferred observability backend.
const metricsSink: ObserveSink = {
  emit(event: ObserveEvent) {
    // Emit a counter for each contract evaluation
    myMetrics.increment(`apophis.contract.${event.type}`, {
      route: event.route,
      formula: event.formula,
    })

    // Record duration as a histogram
    myMetrics.histogram('apophis.contract.duration_ms', event.durationMs, {
      route: event.route,
    })

    // Log high-signal violations for immediate triage
    if (event.type === 'contract.violation') {
      logger.warn({ event }, 'APOPHIS contract violation')
    }
  },
}

// Register APOPHIS with observe enabled.
// This emits non-blocking contract pass/violation/error events
// for every covered request, gated by sampling.
await app.register(apophisPlugin, {
  runtime: process.env.NODE_ENV === 'test' ? 'error' : 'off',
  observe: {
    enabled: true,
    sampling: 0.1,               // observe 10% of requests
    sinks: [metricsSink],
  },
})
```

For new services, `createFastify()` wires discovery and APOPHIS before your
routes, which avoids the most common ordering mistake:

```typescript
import { createFastify } from '@apophis/fastify'

const app = await createFastify({
  logger: true,
  apophis: {
    runtime: process.env.NODE_ENV === 'test' ? 'error' : 'off',
    observe: {
      enabled: true,
      sampling: 0.1,
      sinks: [metricsSink],
    },
  },
})
```

Key constraints:
- Sink `emit()` can be sync or async (returns `void | Promise<void>`).
- Sink rejections and thrown errors are silently caught — they never affect the route response or status code.
- In production, observe hooks still run when `observe.enabled` and `observe.sinks` are configured; blocking runtime validation does not.
- Sampling is applied per-formula evaluation via `Math.random() < sampling`.
  At `sampling: 1` every formula is emitted. At `sampling: 0` nothing is emitted.
- Only routes with APOPHIS annotations (`x-ensures`, `x-requires`) produce events.
  Routes without annotations are not evaluated in observe mode.

## Sink Implementations

APOPHIS does not ship with built-in sinks. The `ObserveSink` interface lets you
plug in any backend.

### OpenTelemetry

Export contract evaluations as OTel custom metrics alongside your existing spans and traces:

```typescript
import type { ObserveSink, ObserveEvent } from '@apophis/fastify'
import { metrics } from '@opentelemetry/api'

const meter = metrics.getMeter('apophis')

const contractCounter = meter.createCounter('apophis.contract.evaluations', {
  description: 'Number of contract evaluations',
})
const violationCounter = meter.createCounter('apophis.contract.violations', {
  description: 'Number of contract violations',
})
const durationHistogram = meter.createHistogram('apophis.contract.duration_ms', {
  description: 'Contract evaluation duration',
})

export const otelSink: ObserveSink = {
  emit(event: ObserveEvent) {
    const labels = { route: event.route, formula: event.formula }
    contractCounter.add(1, labels)
    if (event.type === 'contract.violation') {
      violationCounter.add(1, labels)
    }
    durationHistogram.record(event.durationMs, labels)
  },
}
```

Register it with your Fastify app:

```typescript
await app.register(apophisPlugin, {
  observe: { enabled: true, sinks: [otelSink] },
})
```

APOPHIS violations now appear in your OTel metrics backend (Prometheus, Datadog, New Relic, etc.) alongside latency and error data — no custom collector needed.

### Console (development / debugging)

```typescript
export const consoleSink: ObserveSink = {
  emit(event: ObserveEvent) {
    if (event.type === 'contract.violation') {
      console.warn('[apophis]', event.route, event.formula, event.observed)
    }
  },
}
```

### pino logger

```typescript
export function createPinoSink(logger: pino.Logger): ObserveSink {
  return {
    emit(event: ObserveEvent) {
      logger[event.type === 'contract.violation' ? 'warn' : 'info'](event)
    },
  }
}
```

## Monorepo Validation

For monorepos, use `apophis doctor --mode observe --workspace` to validate observe configuration across all workspace packages.

## Mode Mismatch

Profiles configured for `verify` mode will be rejected at runtime. Only profiles with `mode: 'observe'` are valid for observe mode.
```