Failure Scenarios

Practice answering with truth source, operation behavior, user-facing state, and diagnostics.

Sleeping App Receives Payment Webhook

A free-tier app is sleeping. Stripe sends a POST webhook. No hot allocation exists and the route has no idempotency store binding.

Wake and replay

Do not silently replay after wake

Return controlled retryable response or hold only within explicit policy

Surface missing idempotency posture as promotion/readiness risk

Two promotion operations try to update production from route epoch 42. One commits epoch 43. The other reports success late.

Route ownership

Compare-and-swap protects route owner

Late worker report is rejected

Operation state/audit explains losing promotion

An app uses Vango global live state. Rhone starts a second allocation without broadcast backend.

Runtime services

Block horizontal scale or mark global fanout unsupported

Do not claim one-big-server semantics

Require runtime service binding before readiness

Operators need to upgrade a runner host, but all remaining hosts are at admission limits and the host has active sessions.

Operations

Cordon is not enough

Drain is blocked until runner headroom exists

Capacity/launch policy must preserve one-host drain margin

Production migration removed a column. The previous app artifact expects that column. User clicks rollback.

Deploy/data

Route rollback is blocked or dangerous

Database compatibility decides safety

UI must explain forward-fix vs data restore