> ## Documentation Index
> Fetch the complete documentation index at: https://docs.saasybill.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Delivery and retries

> How Saasybill delivers events, retries failures, and switches off an endpoint that never works

Saasybill delivers each event **at least once**. If your server doesn't acknowledge it, Saasybill tries again for about 20 hours.

***

## What counts as success

| Your response | Result |
| - | - |
| Any `2xx` within 10 seconds | <Badge color="green">Delivered</Badge> |
| Any other status | Failure. The event is retried. |
| No response within 10 seconds | Failure. The event is retried. |
| A connection error | Failure. The event is retried. |
| A redirect (`3xx`) | Failure. Saasybill doesn't follow redirects. |

Saasybill reads at most 500 characters of your response body, to show you in the delivery log.

***

## Retry schedule

A failed delivery is tried again after a delay that grows each time. There are seven attempts in about 20 hours.

| Attempt | Sent |
| - | - |
| 1 | Straight away |
| 2 | 1 minute after attempt 1 failed |
| 3 | 5 minutes after attempt 2 failed |
| 4 | 30 minutes after attempt 3 failed |
| 5 | 2 hours after attempt 4 failed |
| 6 | 6 hours after attempt 5 failed |
| 7 | 12 hours after attempt 6 failed |

Up to 20% extra delay is added to each wait, so one recovering server isn't hit by every queued event at the same second. Retries run on a one-minute cycle, so a delay means "no sooner than".

After the seventh failure the delivery is <Badge color="red">Failed</Badge>. It stays in the delivery log for 30 days.

The `Saasybill-Delivery-Attempt` header tells you which attempt a request is.

***

## Duplicates and order

Because delivery is at-least-once, the same event can arrive more than once. It also can arrive after a newer event.

* **Skip repeats.** Store the `Saasybill-Event-Id` of each event you have processed. If you see it again, reply `200` and do nothing.
* **Don't rely on order.** If two events about the same object arrive in the wrong order, act on the object's current state. Fetch it from the API.
* **Keep handlers idempotent.** Processing an event twice should have the same effect as processing it once.

Retries send the same body, so the signature stays valid. The `t` in the signature header is refreshed on every attempt.

***

## The delivery log

Every attempt is recorded. Open **Developer Settings**, then **Webhooks**, then **Recent Deliveries**.

Each delivery shows its event type, the endpoint's host, and a status.

| Status | Meaning |
| - | - |
| <Badge color="green">Delivered</Badge> | Your server replied `2xx`. |
| <Badge color="yellow">Waiting</Badge> | A retry is due. The time of the next attempt is shown. |
| <Badge color="red">Failed</Badge> | Every attempt failed. |

Click a delivery to see each attempt: the status code (or **No response**), how long it took, any error, and the start of your response body.

### Send an event again

Click <Badge>Send Again</Badge> on a delivery to send the same event to the endpoint straight away. Use it after you've fixed a problem, to catch up on an event you missed.

***

## Errors you might see

Saasybill describes a failure in plain terms, without exposing anything about your network.

| Message | What it means |
| - | - |
| The endpoint answered 500. | Your server replied, but not with a `2xx`. The number is the status it sent. |
| The endpoint answered 301. Redirects are not followed. | Your URL redirects elsewhere. Save the final URL instead. |
| Timed out after 10 seconds. | Your server took too long to reply. Acknowledge first, and do slow work afterwards. |
| The host name could not be found. | The domain in your URL doesn't resolve. |
| The connection was refused. | Nothing is listening at that address. |
| The connection was closed before a response. | Your server dropped the connection. |
| The TLS certificate has expired. | Renew the certificate. |
| The TLS certificate could not be trusted. | Use a certificate from a public authority. Self-signed certificates are refused. |
| The TLS certificate does not match the host name. | The certificate is for a different domain. |
| Blocked: the address is not publicly routable. | The host resolves to a private or reserved address. |

***

## An endpoint that never works is switched off

If **5 events in a row** can't be delivered at all, with no success between them, Saasybill switches the endpoint off. Every attempt at each of those events must have failed.

* The **Webhooks** page shows a banner and says why on the endpoint.
* **Nobody is emailed.** Check the page from time to time, or monitor your endpoint yourself.
* Click <Badge>Turn On</Badge> once the problem is fixed. This doesn't replay what was missed.
* To catch up, use <Badge>Send Again</Badge> on each missed delivery, or fetch the objects from the API.

<Warning>
  Events that happen while an endpoint is off are not sent to it later. If your endpoint has been off, reconcile from the API: list recent subscriptions and invoices and compare them with your records.
</Warning>

***

## Design for reliability

<CardGroup cols={2}>
  <Card title="Acknowledge, then work" icon="bolt">
    Reply `200` first. Put the event on a queue and process it in the background.
  </Card>

  <Card title="Store the event id" icon="database">
    Keep the ids you've processed so a repeat does nothing.
  </Card>

  <Card title="Fetch, don't trust" icon="cloud-arrow-down">
    Read the object's current state from the API. Don't depend on the order events arrive in.
  </Card>

  <Card title="Reconcile now and then" icon="scale-balanced">
    Run a periodic job that lists recent objects and repairs anything you missed.
  </Card>
</CardGroup>
