Skip to content

Waterfall & Fallback

What walks down the candidate list, what stops it, and what takes a route out.

The waterfall is one pass down the ranked candidate list, inside the POST /payments/{id} call, while your request is still open. It is not a retry policy and not a background job. When it ends, the HTTP response is already on its way back to you.

There is no fallback chain to configure. The chain is the candidate list that Orchestration builds, and everything on this page is about how far down it the engine walks before it gives up.

What moves the ladder on

Only one thing does: the provider failed in a way that says nothing about the payer. In practice that is a driver throwing on the network, or a driver returning a failure it marked as worth retrying elsewhere.

What the connection didWhat happens next
Threw before an answer came back: refused connection, DNS failure, socket timeoutThe next candidate is tried
Answered 5xx, 408 or 429The next candidate is tried
Answered in a way its own driver marks as worth retrying elsewhereThe next candidate is tried

Each of these marks the attempt failed, records the failure against that route, and moves down one line. The payer is not told anything, because from their side nothing has happened yet.

The last row is the provider's own judgement, and it varies: a provider that answers without a reference the charge could be tracked by is unusable for this payment but fine for the next one, so its driver hands the payment to the next connection rather than failing it.

What stops it

Four things, and only one of them is good news.

A provider accepted. That includes accepting as pending and as requires_action. The provider owns the payment from that moment and nothing else is tried.

A provider refused for a reason that would repeat. An empty wallet, an unregistered number, an operator ceiling, a declined card, a prompt the payer let expire. The engine returns 402 on the spot and does not touch the next candidate, because asking a second provider would ask the same person the same question.

A connection of yours is misconfigured. Missing credentials, credentials the provider rejected, a required field absent for that channel. These also stop the ladder rather than skipping to the next connection, so a single broken connection at the top of your list can cost you every payment routed through it. This is the case worth catching early, and the routing log names it on the attempt row.

Three providers have been tried. orchestration.executor.max_attempts is 3, platform wide, not configurable per account. A candidate list ten deep is still walked three lines at most, and the request ends with 402 even though eligible connections were left untried.

Every attempt leaves a row

One attempt is one Processing record on the payment, created before the provider is called and updated with whatever came back. A payment that cascaded twice carries three of them, in order, each with the provider that was tried, the gateway code and message it returned, the latency, and its rank in the decision.

You read them in the Dashboard on the payment itself, under the timeline, and in Konsole next to the decision that produced them. They are the only place the attempt history exists: the payment object returned by the API carries none of it.

One payment, two providers

  1. attempt 1status: failed
    12:31:40

    Rank 1 in the decision, priority 100, score 2550. The operator did not answer inside the timeout, so the failure counted against this route.

    provider
    cinetpay
    gateway_response_code
    timeout
    latency
    8.0 s
  2. attempt 2status: processing
    12:31:49

    Rank 2, same priority tier. The operator agreed to prompt the payer, which is an acceptance, and the ladder ended here.

    provider
    flutterwave
    gateway_response_code
    pending
    latency
    0.9 s

The second row is where most of the confusion about the cascade comes from.

What the failure classification decides

Every failure is mapped to a canonical class, from the provider's own error code first and from keywords in its message otherwise. That class does not decide whether the ladder moves on. It decides two other things, and both of them are about the route's reputation rather than about this payment.

ClassSideCounts against the route
operator_timeoutPathYes
operator_unavailablePathYes
path_auth_errorPathYes
float_exhaustedPathYes
unknownUnclassifiedYes
insufficient_fundsCustomerNo
invalid_numberCustomerNo
limit_exceededCustomerNo
customer_action_timeoutCustomerNo
pending_unresolvedLifecycleCircuit breaker only
duplicateLifecycleCircuit breaker only

The rule behind the column is the one worth remembering: a failure only counts against a route when it reflects that route's reliability. A payer with an empty wallet is not evidence that MTN is having a bad day, and counting it would slowly push your traffic away from a perfectly healthy connection. The circuit breaker skips the four customer-side classes entirely, and the live success rate is computed from the path-side ones alone.

The circuit breaker

A route that keeps failing is taken out of rotation before it can cost you more payments. A route here is the triple of your account, one channel, and one provider, so a connection going bad on cm.mtn leaves the same connection serving cm.orange untouched, and leaves every other merchant untouched too.

Opens after5 failures inside 60 seconds, customer-side ones excluded
Stays open for5 minutes
Thenone probe attempt is let through
If the probe succeedsthe counters clear and the route returns to normal rotation
If the probe failsthe open period restarts, doubling each time up to 16 times the base

An open route is dropped before ranking, which is why a connection you expected to be tried can be absent from a decision entirely. It is not dropped silently: the decision records it with the reason, and Konsole shows it on its own row.

Routing decision · cm.mtn · 25 000 XAF

One route out of rotation, two ranked, the first one carried it.

  1. 1
    flutterwave
    accepted0.9 s
    priority 100 · score 250
  2. 2
    pawapay
    never called
    priority 100 · score 0
  3. cinetpay

    Skipped before ranking. Its circuit is open on this channel after repeated path failures.

If that route was your preferred one, the effect is visible and confusing without this page: a rule pointing at CinetPay looks ignored for five minutes. It was not ignored, the candidate was never in the list to be scored.

Two providers can never both charge

Within one call, the executor returns on the first acceptance, so at most one attempt succeeds. The part that needs care is the call itself being repeated.

Send an Idempotency-Key header on POST /payments/{id} and the response is cached for 24 hours against your account, the key and a hash of the payload, and a repeat of the same call returns the same response without running a second waterfall. Without the header there is no such protection, and a client that retries a request whose response it never saw can open a second cascade on a payment that already succeeded.

One key per payment attempt

Derive the key from something stable in your own system, such as the order reference plus the attempt number. The key is validated as up to 128 characters of A-Z a-z 0-9 . _ : -, and a different payload under the same key is treated as a different request.

When the list was empty to begin with

A 422 on the channel key means no candidate survived the filters, so nothing was tried and nothing can be retried elsewhere. It is an account configuration problem, not a payer problem: the channel, the currency, a missing driver or the circuit breaker removed every connection before ranking started.

Response · 422 Unprocessable Content
{
"code": 422,
"status": "Unprocessable Content",
"message": "No eligible payment provider found for this transaction.",
"errors": {
"channel": [
"No eligible payment provider found for this transaction."
]
}
}

The fix is never in your request. Open the routing log, find the decision, and read which filter emptied the list.

What did you think of this content?