You can't shed a payment, only delay it

Dropping the excess request isn't an option for a payment instruction. What's left is deferral — which turns a throughput problem into a prioritization problem with a deadline attached.

I’ve been sketching the limiter that sits between a service and a payment provider, and I keep tripping over the same thing. The release valve I reach for first isn’t there. Queue’s full, return a 503, drop the request, move on — fine for a search query, not for a payout. A payment someone is owed doesn’t stop being owed because your token bucket ran dry.

There are honest ways to make the pipe wider: ask for a higher limit, batch what batches, poll less, take webhooks instead of polling where you can stomach what webhooks cost you. Do all of it. It still runs out, and what’s left when it does is deferral — hold it, send it later. Even the textbook answer, slow the producer down, is deferral wearing a different hat. The work doesn’t disappear, it just waits somewhere upstream where you can’t see it.

That sounds like a smaller problem than it is. Deferral moves work in time, which quietly turns a throughput problem into a prioritization problem. The budget is fixed and usually published: Stripe documents 100 requests per second on a live account, 25 per second on an individual endpoint (opens in new tab), a 429 when you go over, and advice to run a client-side token bucket so you don’t get there. Everything you want to send this second is competing for the same tokens.

So: a 50,000-row payout batch starts at the same moment a customer is sitting at a checkout waiting on an authorization. One global FIFO queue, and the batch wins — it arrived first and it has fifty thousand turns to take. At 25 requests a second, that’s over half an hour with the pipe full. Nothing fails. No alert fires. You’ve just parked a live human behind a batch that had no deadline at all, and the queue was perfectly fair the entire time. Fairness was the wrong goal.

Which is why I don’t think you get one bucket. You get lanes with per-class quotas: interactive traffic gets a floor it never has to compete for, batch work runs on the remainder and is explicitly allowed to starve. At that point it isn’t a rate limiter, it’s a scheduler — and all the design work has moved into the traffic you’re not sending.

The lane that gets forgotten is retries. When a provider goes slow rather than down, latency drifts past your client timeout, every in-flight request fails at roughly the same instant, and every worker does the sensible thing. Your request rate multiplies at exactly the moment there’s least capacity to absorb it, and the new checkout is the one that eats the 429. Per-request backoff doesn’t bound that — it only decides when one client tries again. The control has to be aggregate. Google’s SRE book has each client track the ratio of its own requests that are retries and only retry while that ratio stays below 10% (opens in new tab), on top of a three-attempt-per-request cap. Retries get a budget, like every other lane.

Here’s the part I’m less comfortable with. Deferral is only cheap if the deferred thing keeps its value, and in payments it mostly doesn’t, because everything is on a clock. A card authorization expires — Stripe’s table gives card-not-present holds roughly five to seven days depending on the network (opens in new tab). Same Day ACH has three input deadlines: 10:30 a.m., 2:45 p.m. and 4:45 p.m. ET (opens in new tab). Miss 4:45 and “later today” is now tomorrow. Fedwire closes at 7:00 p.m. ET with a 6:45 p.m. customer cutoff (opens in new tab). An FX quote has a validity window, and when it lapses you aren’t retrying, you’re re-pricing — possibly worse, and someone eats the difference.

So the thing I’d argue with, including when I’m the one saying it, is “we’ll just queue it.” That’s half an answer. The other half is what happens when the queue outlives the deadline. Every deferred item needs a deadline attached and a decision for when it passes: roll to the next window, re-quote, or hand it back to a human who can make a call. Pick one, write it down. And whatever finally goes out carries the idempotency key it was born with, not a fresh one.

Without that, the queue isn’t backpressure. It’s a slower, quieter way to lose the payment — and the 503 was at least honest about it.

Archie