Queues

Durable async work: declare a queue, produce from app code, consume with a handler or an agent, and handle retries and the dead-letter queue. Plus how to get strict per-key ordering when you need it.

The Spikefrost Team30 Jul 20264 min read

A queue is the right shape for work that must happen reliably but not right now: processing an order, syncing an external system, generating a report, handling a burst of webhooks.

Declaring one

// app/queues.json
{
  "queues": [
    {
      "name": "orders",
      "maxConcurrency": 5,
      "maxRetries": 5,
      "backoff": { "baseSeconds": 10, "maxSeconds": 3600 },
      "target": { "kind": "handler" }
    }
  ]
}

sf deploy provisions the queue and a dead-letter twin, and binds it for producing. Like the database and KV, each environment gets its own queue — staging's backlog is staging's.

Limit Value
Queues per app 10
maxConcurrency ≤ 100
maxRetries ≤ 20
backoff.maxSeconds ≤ 43200 (12h)

Removing a queue from the file destroys its backlog, so the deploy requires "confirmDelete": ["orders"] in the same file.

Producing

The binding is QUEUE_<NAME>, with dashes as underscores — order-events becomes QUEUE_ORDER_EVENTS:

await c.env.QUEUE_ORDERS.send({ orderId: 42 });

External producers don't get the binding — expose your own authenticated route that validates the request and then calls send(). That keeps the auth decision in your code.

Consuming with a handler

Export a queues object from the app entry, keyed by queue name:

// app/src/index.tsx
export const queues = {
  orders: async (msg, env, ctx) => {
    const { orderId } = msg.body;

    // Idempotent: at-least-once means this may run twice.
    const already = await env.DB
      .prepare('SELECT 1 FROM processed WHERE message_id = ?')
      .bind(msg.messageId)
      .first();
    if (already) return;                     // ack, nothing to do

    await processOrder(env, orderId);

    await env.DB
      .prepare('INSERT INTO processed (message_id) VALUES (?) ON CONFLICT DO NOTHING')
      .bind(msg.messageId)
      .run();
  },
};

The message is { queue, messageId, attempts, body }.

Return normally to acknowledge. Throw to retry. Retries back off exponentially per your backoff settings, and after maxRetries the message goes to the dead-letter queue.

Two rules that decide whether this works in production:

  • Handlers must be idempotent on messageId. At-least-once delivery is not an edge case — redelivery happens. Charging a card twice because a handler assumed exactly-once is the classic version of this bug.
  • maxConcurrency is how many messages are in flight at once. Raise it for throughput; keep it low when the work hits something rate-limited or writes to contended rows.

Consuming with an agent

"target": { "kind": "agent", "agent": "OrderProcessor" }

Each message becomes a turn on that agent. Useful when handling a message needs judgment rather than a deterministic transform.

But note: agent consumption is serialized per queue — one conversation object handles them in sequence, regardless of maxConcurrency. That's judgment work, not throughput work. If you need parallelism, use a handler and let it call an agent only for the messages that actually need one.

Operating a queue

sf queues list                          # queues per environment
sf queues send orders --body '{"orderId":42}'   # inject a test message
sf queues dlq peek orders               # what failed, and why
sf queues dlq redrive orders            # replay the dead-letter queue

Check the dead-letter queue deliberately. A DLQ nobody looks at is a silent data-loss channel — messages arrive, fail five times, and disappear from anyone's attention. Peek at it after any deploy that touches a consumer, and consider an alert on its depth.

Before redriving, fix the cause. Redriving into the same bug just re-fills the DLQ.

Strict ordering

Managed queues are unordered and parallel. There is no FIFO flavor.

When you need strict order for one key — all events for one order, one customer, one document — route them to a single Durable Object instance:

const id = c.env.ORDER_PROCESSOR.idFromName(`order-${orderId}`);
await c.env.ORDER_PROCESSOR.get(id).fetch(request);

A Durable Object is single-threaded, so per-key ordering is free. Deduplicate with INSERT ... ON CONFLICT DO NOTHING on a business key in its own storage, and use alarms for retries and delays. See Durable Objects.

The choice in one line: queues for throughput, Durable Objects for order.

One deploy gotcha

Adding queues.json makes the deploy build the platform wrapper — even in an app with no agents. That app needs the agent dependency set in package.json and a regenerated lock file, or the build fails with unresolved platform packages.

If you see a build failure naming platform modules right after adding your first queue, that's this. The deploy self-heals its own build copy and logs what it added; add the same pins locally so your editor and typechecker agree.

Choosing a shape

Need Use
Reliable background work, parallel A queue with a handler
Reliable background work needing judgment A queue targeting an agent
Strict per-key order A Durable Object
Fire-and-forget within one request ctx.waitUntil()
Time-triggered work A schedule or routine

Next

Frequently asked questions

How do I add a queue?

Declare it in app/queues.json and deploy. The platform provisions the queue plus a dead-letter twin and binds env.QUEUE_<NAME> for producing.

Are messages delivered exactly once?

No — at-least-once. A handler can see the same message twice, so it must be idempotent on messageId. Treat that as a requirement, not a nicety.

How do retries work?

Return normally to acknowledge; throw to retry. Retries back off exponentially up to your configured limit, then the message lands in the dead-letter queue where you can inspect or replay it.

Can a queue wake an agent instead of a handler?

Yes, by targeting an agent — each message becomes a turn. But agent consumption is serialized per queue regardless of concurrency settings, so it's for judgment work, not throughput.

How do I guarantee ordering for one customer or one order?

Not with a managed queue — they're unordered. Route all work for one key to a single Durable Object instance, which is single-threaded, so per-key order comes for free.