What Your Integration Code Is Missing: Delivery Guarantees for Product Events

Most hand-rolled retry code gets at least one of these wrong. If you want the full picture on retry implementation, webhook retry logic done right covers the backoff math and failure classification in detail.

2. No idempotency protection

Here's the thing about that HubSpot timeout: the first call might have actually gone through. HubSpot received the request and updated the contact, but the response never made it back before your client gave up. Your retry succeeds too — and now HubSpot has processed the same event twice. The contact's lifecycle stage was updated once (fine), but your custom event log shows two subscription.started entries, and the workflow triggered by that event fires duplicate emails.

Idempotency protection means tagging every event with a unique key and ensuring that the destination only processes it once, even if it arrives multiple times. Some APIs support this natively (Stripe's Idempotency-Key header, for example). For destinations that don't, you need deduplication logic on your side — and that means tracking which events have been delivered, which adds state, storage, and cleanup concerns.

3. No dead letter handling

Let's say HubSpot's bad day gets worse. Your retries exhausted, the subscription.started event to HubSpot is simply dropped. A log line gets written — maybe — and the event is gone. Three weeks later, a sales rep notices the customer's lifecycle stage never updated. There's no way to recover the original event, no way to replay it. The data is wrong now — and fixing it is manual.

Retries have a limit. After three, five, or ten attempts, some events will still fail. What happens to them?

In most codebases, the answer is: they disappear.

A dead letter queue captures events that have exhausted all retry attempts. It gives your team visibility into what failed, why it failed, and the ability to replay those events once the underlying issue is fixed. Without one, event loss is silent and permanent.

4. No per-destination isolation

Remember, the subscription.started event needed to reach three destinations: HubSpot, Salesforce, and Slack. Your code sends them sequentially. HubSpot is timing out, your retry logic blocks for 30 seconds, and now the Salesforce opportunity and the Slack notification are sitting in a queue behind a destination that can't receive. The sales team doesn't find out about the upgrade for another hour. HubSpot's bad day became everyone's bad day.

When you send the same event to multiple destinations, a failure to one shouldn't affect the others. But in most integration code, it does.

Destination isolation means each delivery path has its own retry budget, its own failure state, and its own dead letter queue. A degraded HubSpot API doesn't slow down Slack notifications. An expired Salesforce token doesn't prevent Intercom messages.

This is the fan-out problem: one event in, many deliveries out, each with independent lifecycle management. It sounds simple. The implementation is not.

5. No delivery visibility

A week later, someone asks: "Did customer X's upgrade event actually reach Salesforce?" Your team checks application logs. There's a generic INFO: event sent line, but no record of the response code, no record of how many attempts were made, no record of whether HubSpot got it on the first try or the third. The investigation takes two hours and ends with "probably?"

If someone asks "did the subscription.started event from customer X reach Salesforce last Tuesday?" — can your team answer that question?

Most integration code has no audit trail. Events are fire-and-forget by default. When they succeed, nobody asks questions. When they fail, there's nothing to investigate.

Delivery visibility means logging every attempt — not just successes and final failures, but each retry, each response code, each timeout. It means being able to trace a single event from emission through every delivery attempt to every destination.

Without this, debugging integration issues means reading application logs, correlating timestamps, and guessing. With it, you can answer "what happened to this event?" in seconds.

Reliable product-event delivery with Meshes

Meshes brings retries, deduplication, fan-out, dead letters, and delivery history together for customer-facing integrations. Each destination has its own delivery outcome, so a failure in one connection does not block the others.

Your customers configure their connections and rules in isolated workspaces. Your team can inspect delivery attempts and retry failed events after resolving the cause.

Reliable delivery for every customer destination. Join Meshes and get retries, deduplication, fan-out, dead letter queues, and delivery logging for your product events out of the box.