Skip to main content

Module recovery

Module recovery 

Source
Expand description

Recovery ownership and replacement attempts for one book.

  • BookRecoveryState admits one running recovery per book and cancels obsolete work.
  • BookRecovery runs replacement attempts until the adapter accepts a snapshot or cancels the episode. A book never ends in a terminal failure state.
  • BookRecoveryOutcome carries pending, accepted, or rejected results from the adapter’s frame handling to the recovery runner.

§Recovery Lifecycle

The adapter claims an episode before starting work and supplies the replacement operation, retry classification, and error construction. Each attempt receives a child cancellation token and a closed snapshot gate. The adapter opens the gate after the intended connection confirms the write, then accepts a valid snapshot through BookRecovery::accept. Write completion alone never completes recovery.

Attempts back off within a retry budget of eight attempts in 180 seconds. An exhausted budget, or an error the adapter classifies as not retryable, moves the episode to attempts at a jittered interval that doubles from one minute to fifteen minutes, which bounds subscription traffic for a book that keeps failing. Each such attempt is bounded by max(snapshot_timeout, 60 seconds), so a stalled write or a disabled snapshot deadline cannot end the retries. The runner logs the transition as an error, each later failed attempt as a warning, and completion as info.

§Adapters

The adapter serializes claims and snapshot acceptance under its state lock or owning task. A reconnect keeps a running episode, whose replacement may be in flight, so the reconnect can neither replenish its budget nor abandon a write halfway; it wakes only a wait between attempts after the budget, as BookRecoveryState::reset_on_reconnect describes. Removing or resetting a BookRecoveryState cancels its episode; dropping an attempt cancels that attempt’s child token.

Task spawning, subscription correlation, and book cache updates remain adapter-owned. Claim an episode only once the replacement has what it needs, and cancel the episode if its task cannot start: a running episode without a task keeps its book owned with nothing running.

Structs§

BookRecovery
One recovery episode, running until a snapshot is accepted or the episode is cancelled.
BookRecoveryState
Owns a book’s recovery episode under its adapter’s state lock.

Enums§

BookRecoveryOutcome
Outcome published by an adapter’s snapshot and rejection handling.