mechanoid · v0.3.2

Durable Timeouts

Why durable

Fiber timeouts are fast and local. If the node dies while an FSM sits in a timed state, that fiber is gone. Durable timeouts store deadlines in a TimeoutStore so another node's sweeper can fire them.

dies

TimeoutStrategy

StrategyLayerSurvives restart
FiberTimeoutStrategy.fiber[Id]No
DurableTimeoutStrategy.durable[Id] (+ TimeoutStore)Yes

Schedule with durable strategy, then send the timeout event the sweeper would fire (DocSpecs use a live clock; unit tests can TestClock.adjust fiber timeouts instead):

{
  enum OrderState derives Finite:
    case Pending, Started, Done, Cancelled

  enum OrderEvent derives Finite:
    case StartPayment, Complete, PaymentTimeout

  import OrderState.*, OrderEvent.*

  val machine = Machine(
    assembly[OrderState, OrderEvent](
      (Pending via StartPayment to Started) @@ Aspect.timeout(1.hour, PaymentTimeout),
      Started via Complete to Done,
      Started via PaymentTimeout to Cancelled,
    )
  )

  val orderId: OrderId = "order-timeout-1"

  ZIO
    .scoped {
      for
        fsm   <- FSMRuntime(orderId, machine, Pending)
        _     <- fsm.send(StartPayment)
        _     <- fsm.send(PaymentTimeout)
        state <- fsm.currentState
      yield state
    }
    .provide(
      InMemoryEventStore.layer[OrderId, OrderState, OrderEvent],
      ZLayer.fromZIO(InMemoryTimeoutStore.make[OrderId]),
      TimeoutStrategy.durable[OrderId],
      LockingStrategy.optimistic[OrderId],
    )
    .asDoc
}
Cancelled

TimeoutSweeper

A background sweeper:

  1. Queries expired, unclaimed timeouts

  2. Claims each timeout

  3. Validates (stateHash, sequenceNr) so stale timeouts do not fire

  4. Looks up the timeout event via Machine.timeoutEvents and runtime.sends it

  5. Marks complete

Use TimeoutSweeperConfig for interval, jitter, batch size, claim duration, and nodeId. Optional leader election via LeaseStore keeps a single active sweeper to reduce DB load.

See examples/heartbeat for a full sweeper alongside FSMRuntime, and Testing for the DocSpec vs TestClock choice.

Next: Distributed Coordination.