Replay a projection
Rebuild a table projection from the start of the event log by stopping its processor, resetting its token and clearing the table in one transaction.
By the end of this guide you can rebuild the inventory table from the first event in the log, without losing a row and without sending a receipt twice. You need the projection from Project into your own tables: an event handler that writes the inventory table through ctx.sql(), in a processor named inventory-view that runs on postgresTokenStore and postgresUnitOfWork.
Replaying is how you fix a projection after you change what its handlers write, add a column, or find a bug in them. The log is not touched. The processor starts reading it again from a position you choose, and the handlers write the table anew.
Stop, reset, start
A replay has three calls on the running processor, which is in app.processors under its name. stop() ends delivery. resetTokens(startPosition, reset) moves the stored position to startPosition, where 0n is the start of the log. start() begins delivery from there, and the processor reads the log from startPosition onward.
export async function replayInventory(processor: RunningProcessor, from = 0n): Promise<void> {
await processor.stop()
await processor.resetTokens(from, truncateInventory)
await processor.start()
}resetTokens throws if the processor is still running, and fails with an error naming the segment when another node holds one. Stop the processor on every node before you call it.
Clear the table in the same transaction
The second argument of resetTokens is a function that runs in the transaction that stores the new token. Anything it writes through postgresTransaction(uow) commits with the token, and if it throws, the old rows and the old token both stay.
export async function truncateInventory(uow: UnitOfWork): Promise<void> {
const tx = await postgresTransaction(uow)
await tx.query(`TRUNCATE inventory`)
}Without this, the replay would write onto the old rows, and a row for a tool that no handler now produces would stay. With it, the table is empty from the moment the reset commits and fills as the processor reads the log, so a query during the replay returns a partial table. resetTokens(1000n, reset) replays from position 1000 instead, when the table is correct up to there and reset removes only the later rows.
Tell a replay from live delivery
A replay delivers events the handlers have already handled once. ctx.isReplay() is true for events up to the position the processor had reached when you reset it, and false after that.
A handler with a side effect that must not happen twice checks ctx.isReplay(). The ToolRented handler sends a receipt, and skips it during a replay.
export const recordRentedTool = eventHandler(
ToolRented,
async ({ payload }, ctx: EventHandlerContext & PostgresCapability) => {
await ctx.sql().query(`UPDATE inventory SET available = false WHERE tool_id = $1`, [payload.toolId])
if (!ctx.isReplay()) sendReceipt(payload.customerId, payload.toolId)
},
)The table writes in the handler run in both cases, so they must tolerate rows that already exist, as described under Idempotency.
Replay from an admin endpoint
An endpoint starts a replay and reports the processor's status. adminRoutes is a function from a Request to a Response, which Bun.serve, Hono and similar servers take as their fetch handler. POST /admin/processors/inventory-view/replay runs the three calls and GET /admin/processors/inventory-view returns the processor's status.
const route = /^\/admin\/processors\/([^/]+)(\/replay)?$/processor.status() returns whether the processor is running, its position, whether it has caught up with the log, and whether it is replaying. The position is a bigint, so the endpoint sends it as a string.
const { running, position, caughtUp, replaying } = processor.status()
return Response.json({ running, position: String(position), caughtUp, replaying })The replay branch rejects every processor except inventory-view, because the reset callback truncates only that processor's table. A failed reset returns 409 and leaves the table and the token as they were.
try {
await replayInventory(processor)
} catch (error) {
return Response.json({ error: String(error) }, { status: 409 })
}
return Response.json({ replaying: processor.replaying }, { status: 202 })The endpoint returns 202 once the processor is started again, not when the replay has finished. A client polls the status until replaying is false. Put the route behind your own authentication: it deletes data.
Checking it works
app.ts adds two tools, rents one, then calls the endpoint function directly instead of over HTTP. It waits for the replay to finish and prints the inventory again:
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/toolstore bun run app.tsreceipt sent: ada rented t-1
before replay [
{
toolId: "t-1",
name: "Cordless drill",
available: false,
}, {
toolId: "t-2",
name: "Tile saw",
available: true,
}
]
POST replay 202 {
replaying: true,
}
GET status {
running: true,
position: "3",
caughtUp: true,
replaying: false,
}
after replay [
{
toolId: "t-1",
name: "Cordless drill",
available: false,
}, {
toolId: "t-2",
name: "Tile saw",
available: true,
}
]
receipts sent 1The receipt line appears once, before the replay: during the replay ctx.isReplay() was true for all three events. To check that the table is rebuilt from the log, insert a row of your own with psql before the replay runs, such as insert into inventory values ('junk', 'junk', true); the reset removes that row, because it is not in the log.
Related
- Event processors describes tokens, replay tokens and segments.
- Postgres describes
postgresTransaction. - Project into your own tables builds the projection that this guide replays.