“Does my list have too many bounces? Will I get banned by my email sending program?” Before sending a newsletter to a new email list, you might be worried about the bounces. Fortunately, there are a number of email cleaning apps in the market for us to clean our lists. Some are “award-winning”, some are “used by tens of thousands of companies”, and some are “trusted”.
The question is: can we trust their results?
To this end, we compare 3 popular email cleaning apps, NeverBounce, ZeroBounce, and EmailMarker.
| NeverBounce | https://neverbounce.com/ |
| ZeroBounce | https://www.zerobounce.net/ |
| EmailMarker | https://www.emailmarker.com/ |
We have a list of 1,000 academic emails. Since we recently sent a newsletter to these contacts, we know exactly which email are good and which are bounced: 956 emails are good, and 44 are bounced. The bounce rate is 4.4%. We call this “Ground Truth”.
Then we submit these 1,000 emails to the 3 email cleaning apps. We want to find out if they can classify our emails accurately.
First, let’s see the how the 3 apps classify the 956 good emails.
| Classification Status | Groud Truth | NeverBounce | ZeroBounce | EmailMarker |
| Valid | 100% | 72% | 68% | 42% |
| Not sure | 0% | 28% | 31% | 55% |
| Invalid (bounced) | 0% | 1% | 1% | 3% |
We immediately see a big problem here – instead of just classifying the emails to “good” and “bad”, there is a big chunk of “Not sure”. One reason why some emails are in this “not sure” category is that the email servers of many organizations refuse to give a response of the existence of the emails.
If we only use the emails where these apps are certain to be good emails, then we are throwing anywhere from 30% to 60% good emails to trash! Doing so will keep us from contacting some organizations by email and thus losing opportunities. Can we afford to do so?
If we instead include the “Not sure” emails, we can see that we are using 97%-99% of the good emails. This is good news. But we are running a risk of raising our bounce rate, which we will address below.
Next, let’s see how the 3 apps classify the 44 bounced emails. Since our main purpose to use an email cleaning tool is to remove bad emails, we expect they should pick up the bad emails accurately.
| Classification Status | Groud Truth | NeverBounce | ZeroBounce | EmailMarker |
| Valid | 0% | 9% | 16% | 11% |
| Not sure | 0% | 27% | 5% | 18% |
| Invalid (bounced) | 100% | 64% | 80% | 70% |
We can see from the table that they pick up 60%-80% of bad emails, which is an OK job. If we considered “not sure” as “bad emails” too, then we can remove ~90% bad emails.
Here we are facing a decision: should we use the “Not sure” emails? Let’s compare the two options. What we will calculate is the number of usable emails and the corresponding bounce rate.
Option 1: We only use the emails that these apps are certain to be good:
| NeverBounce | ZeroBounce | EmailMarker | |
| Number of usable Emails | 689 | 655 | 405 |
| Bounce rate | 0.6% | 1.1% | 1.2% |
Option 2: We use the emails that these apps are certain to be good, plus the “Not sure” emails:
| NeverBounce | ZeroBounce | EmailMarker | |
| Number of usable Emails | 966 | 955 | 938 |
| Bounce rate | 1.7% | 0.9% | 1.4% |
To me, the decision is obviously to use both “good” and “Not sure” emails. With this option, our bounce rate is still very low (less than 2%), and more importantly, we are able to use ~95% of our original emails. If we choose “good” emails only, we are wasting 40%-60% our valuable emails!
Based on the results above, we can see that the more important decision is whether to include the “Not sure” emails – which I think yes. Then out of these 3 apps which is best? They are not much different to me. We personally use EmailMarker and have been enjoying it.
Lesson learned:
1. If we only use the “cleaned” emails, we are wasting 40%-60% of good emails.