Acceptable Failure Rates

It is not uncommon for prospective customers and existing clients to ask us to provide more specific information about our tolerance for message failures and bounces. This question comes up because the SocketLabs Acceptable Use Policy states:

Your email list must be valid and not generate excessive failures as determined by the sole discretion of SocketLabs.

There is not an exact percentage point at which we believe a stream of email messages becomes in violation of our AUP. The hard failure rate is only a single point of measurement about a stream of email messages. We must analyze every aspect of a stream of messages including type (marketing vs. transactional vs. person-to-person), content, volume, complaint rate, engagement rate (opens, clicks, and unsubscribes), and other factors, before we can determine what we find acceptable.

For example, a high (5%+) failure rate may not be an issue for a stream of welcome messages. This is because a welcome message is usually a type of mail that performs very well in other metrics like engagement rates. A high rate of engagement may be able to offset the negative reputation effects of a higher than normal rate of failures.

On the contrary, it is also important to understand that having a low bounce or failure rate is not indicative of an acceptable stream of email messages. This is especially true for addresses or lists that have been processed through a list verification service. Verification services can significantly reduce bounce rates for a given mail stream. A verification service may validate an email address exists, but it does not validate opt-in consent or desirability of a message or campaign. Other metrics like complaint rate, or poor engagement can quickly over take any positive reputation points earned for sending messages with a low failure rate.

The best way to answer the question of what our tolerance for failures is is it is a dynamic scale as shown below. As failure rates exceed 3% to 5%, the likelihood our automated abuse detection systems, or compliance department will take action regarding the status of an account increases significantly.

Failure rates over a few percentage points indicate issues with the addresses being contacted.

Failure rates over a few percentage points indicate issues with the addresses being contacted.

We can also look at aggregate data from our most successful customers to gauge on average how well they perform in regards to hard failures. Over the last 15 days our top 1000 customers are averaging just a 1.04% hard failure rate. Only 22 of these 1000 customers who we consider to be top performers had hard failure rates above 5%.