How we improved email deliverability at Paystack
A detailed breakdown of how we significantly improved our email deliverability and email sender reputation
In late 2018, a friend sent me a screenshot of a warning he received from Gmail when he opened an email broadcast from Paystack - a large yellow banner carrying the words “Be careful with this message.”
Gmail was warning our email subscribers that our emails might be harmful, and it was immediately clear that our open rates and email deliverability could suffer as a result.
In this article, I’m going to share in detail everything we learned while fixing this. If you're ever in a similar situation, this post should help you resolve it for yourself.
Understanding Email Sender Reputation
I did several Google searches to understand the problem, and quickly realised that:
- It was hard to find a single guide that explained everything I needed to do from A to Z
- Email Sender Reputation significantly influences email deliverability
An Email Sender Reputation is a score, usually between 0 - 100, that Email Service Providers (ESPs) such as Gmail, Outlook, and Yahoo Mail assign to organisations that send out emails. The higher the score, the more likely it is that ESPs will deliver emails to subscribers’ inboxes.
The lower the score, the more likely it is that when you send broadcast emails, many of those emails will either get caught in subscribers’ spam folders, or not get delivered at all.
We monitor our reputation score with Sender Score's look-up tool. Check your email sender reputation.
A lot of factors contribute to the reputation score, but the important ones all have to do with how subscribers engage with your emails. ESPs listen for both positive and negative engagement signals to increase or reduce a sender’s reputation score.
Examples of positive engagement signals include:
- a high open rate - your open rate is the percentage of people that open your email compared to the number of people who received the email
- a high click-through rate - the click-through rate is the percentage of people who click on a link in your email, compared to the number of people who received the email
- subscribers forwarding your emails to other recipients
Examples of negative engagement signals include:
- a very low open rate
- subscribers reporting your emails as spam
- a high unsubscribe rate
- inclusion of your domain name in email blacklists (more on email blacklists later)
- a high hard bounce rate - the percentage of emails that don’t get delivered because they were sent to invalid email addresses
- a high soft bounce rate - the percentage of emails that don’t get delivered because they were sent to subscribers with full inboxes
A checklist for improving your email deliverability and sender reputation
Here’re the actions we took to improve our Email Sender Reputation, and by extension, our deliverability. We'll dive into each one in detail.
- Authenticating Paystack’s email sending domain
- Removing Paystack’s domain from credible email blacklists
- Periodically pruning our email list of hard bounces
- Monitoring key email deliverability metrics
Authenticating our email sending domain
Email authentication helps an organisation prove that emails claiming to come from them are actually authorised by the organisation.
Before we authenticated our emails, our subscribers' Email Service Providers - sometimes called Mail Transfer Agents (MTA) - couldn't tell for sure that emails claiming to come from a paystack.com email address were actually coming from Paystack. This is one of the reasons why Gmail was sending our subscribers the warning on the large yellow banner.
After we authenticated our email sending domains, our subscribers’ ESPs could verify authorship of Paystack emails on our subscribers’ behalf, and help us get the emails delivered into the right mailboxes.
On a high level, this is how email authentication works: we add a bit of information to all our emails, and then add a corresponding bit of information to a publicly available directory that everyone can see, but only someone from Paystack can access. Now, when we send out emails, our subscribers' ESPs can verify the authenticity of the emails by checking the bit of information in the email against the publicly available directory for a match.
The main ways to authenticate emails
There are several ways to authenticate emails, but the 3 most important ones, and the ones we implemented, are:
- Sender Policy Framework (SPF) - an authentication method that lets our subscribers' ESPs verify that an email claiming to come from paystack.com is indeed sent by an IP address, or domain authorised by us
- Domain Keys Identified Mail (DKIM) - an email authentication method that lets our subscribers' ESPs verify that the server claiming to have sent the email is indeed who they claim they are. Since modern email servers use secure email sending protocols, this also implies that the content of our emails didn't change in transit and that we take credit for the content of the emails. As a pair, SPF and DKIM are effective authentication methods that complement each other. SPF only allows us to declare authorised senders, while DKIM helps to verify the identity of senders claiming to be authorised by us.
- Domain-based Message Authentication, Reporting and Conformance (DMARC) - this authentication method allows us to indicate that our emails are protected by SPF and DKIM. It also instructs ESPs on actions to take when an email claiming to come from us fails one or both of SPF and DKIM - should our subscribers' ESPs accept, reject, or quarantine these emails? At Paystack, we set our DMARC record to quarantine emails that fail SPF, DKIM, or both. Lastly, DMARC lets us set up an email where reports of emails that pass or fail our email authentication are sent to. This allows us to review and modify our policies while identifying false positives and false negatives.
Setting up email authentication
The two things I needed to be able to authenticate our emails were:
- Access to the DNS Server for our email sending domain
- Access to Drip, the tool we use to send broadcast emails
Authenticating Paystack’s email domain required some technical work, so I worked with our engineering team to make it happen. Here’s a breakdown of the steps we took:
STEP 1: We copied Drip's SPF record, which is:v=spf1 include:sendgrid.net ~all
Important: your email marketing tool will likely have a different SPF record
STEP 2: Our marketing automation tool, Drip, sends emails using Sendgrid, so we needed to get the DKIM record from Sendgrid, which is something like:v=DKIM1; k=rsa; p=really-long-public-key
Important: your email marketing tool might have a different SPF record. Please refer to your email marketing tool's Help Desk for help on how to get the specific SPF and DKIM records for that tool. If you use Drip, you can find detailed instructions here.
STEP 3: The engineering team updated our DNS with our SPF and DKIM records
Important: If your organisation sends out emails using different tools, please include the different SPF and DKIM records of all those tools in one record and not as separate records. Sometimes, some ESPs are not smart enough to check for other SPF and DKIM records after they see the first ones. This may lead to some of your emails not getting delivered.
For example, here's the preferred SPF record format:
v=spf1 include:sendgrid.net include: domain1.net include: domain2.net ~all
It contains the multiple SPF records for sendgrid.net, domain1.net and domain2.net all in one line.
Here's an example of an SPF record that some ESPs might have trouble with: v=spf1 include:sendgrid.net ~allv=spf1 include:domain1.net ~allv=spf1 include:domain2.net ~all
This format contains different SPF records in multiple lines, and some ESPs might not know to check lines 2 & 3 causing some of your emails to not deliver.
STEP 4: I shared our preference for how ESPs should treat emails that fail one or both of SPF and DKIM with our engineering team, and they translated it into a DMARC record that looks like this: v=DMARC1; p=quarantine; rua=mailto:dreports[at]paystack[dot]com; pct=100
The DMARC record above instructs all ESPs to quarantine ("p=quarantine") 100% (pct=100) of emails that fail one or both of SPF and DKIM, and send pass or fail reports to the email "dreports[at]paystack.com (rua=mailto:dreports[at]paystack.com)
STEP 5: The engineering team updated our DNS record with our DMARC record
Authenticating Paystack’s email domains contributed significantly to our improved email sender reputation, and ultimately, our improved email deliverability.
Sometimes, we use Global Cyber Alliance's free lookup tool to check the status of our email authentication. Try Global Cyber Alliance's free lookup tool for yourself here.
Removing Paystack’s domain name from email blacklists
An email blacklist is a real-time database of domains that are believed to be sending emails considered to be spam or fraudulent. There are more than 100 known blacklists, and ESPs routinely check for domain names on some of the lists to update their reputation scores for the domains. I discovered that our paystack.com domain had somehow been listed on 20 email blacklists.
(I should mention immediately that we handle customers' emails with the highest standards, and as I describe later, simply authenticating our domain immediately removed us from the blacklists.)
📝 Important: Please note that not all blacklists are credible - a few are fraudulent. When I was searching through blacklists, I found one that listed Paystack’s sending domain, and asked for payment to delist our domain. No credible email blacklist will ask you to pay to get delisted. Instead, they’ll provide a clear list of action items designed to have you following email sending best practices.
How do blacklists work?
Here’re the 2 most popular ways blacklists add domains to their list:
- Spam traps - blacklists often operate a large number of active emails that have never subscribed to an email list before and inactive emails that should have stopped receiving any emails. If your organisation sends large amounts of emails to these trap emails, you risk being added to a blacklist
- Feedback from subscribers - some blacklists also listen for feedback from subscribers. If subscribers keep reporting you for sending unwanted emails, or not honouring unsubscribe requests, you also risk being added to a blacklist
In order to find the blacklists where we were listed, I first searched through aggregated blacklist lookup platforms (tools that search about a 100 blacklists at a time). Here’re the ones I used:
Next, I checked our domain against some very popular blacklists by hand:
Removing Paystack’s domain from the blacklist
I found that by simply authenticating Paystack’s email sending domain, we were excluded from all the blacklists that previously listed us. In your case, however, your email sending domain may have been included on a blacklist for other reasons, and email authentication might not get you off the hook.
Here’s a general workflow for delisting your email domain from an email blacklist:
- Get help from your technical team to ensure that your email server is correctly configured, and your email domain is authenticated
- Follow the recommended blacklist removal process - some blacklists will tell you why you’ve been listed and give specific instructions that’ll get you delisted
- Check to see if the blacklist allows self-removal. If it does, be sure to fix the identified issues, and then remove your organisation’s sending domain. It’s pretty straight forward the first time, but it gets more difficult if you get blacklisted subsequently
- Some other blacklists give clear instructions that’ll get you delisted, and then routinely do sweep checks to list new domains or delist some existing ones
Removing Paystack’s sending domain from blacklists was an important step in improving our email deliverability.
Pruning our email list
After a while, it's natural for a percentage of your email subscribers to stop opening your emails. When this happens, the email list is said to have partly decayed. We've created an email pruning workflow that removes our least engaged subscribers from our email list.
Why do we prune our email list?
You might be asking "It takes a long time to grow a large email list. Why on earth would you willingly delete email addresses from your list?!" There’re 2 main reasons why we prune our list:
- To improve our sender reputation - When a significant percentage of our subscribers don’t open our emails, it drives down our engagement numbers which in turn affects our email sender reputation. Since a percentage of subscribers will inevitably churn, if you don't prune your list, those inactive subscribers start to account for a larger and larger percentage of your entire email list. For example, if 20 out of 100 subscribers open our emails - that’s an open rate of 20%. When we prune our email list to say, 80 emails - 20/80 would net an improved 25% open rate, which would slightly improve our reputation score. Removing subscribers who're no longer engaged, therefore, ultimately improves our email deliverability.
- To reduce cost - like many email marketing tools, Drip's cost is determined by the number of emails in our account. The more the emails in our list, the more expensive it gets. Without realizing it, our email list had grown very large with many inactive addresses. So proactively identifying and removing inactive subscribers saves us a lot of money.
How we prune our email list
The longer it’s been since a subscriber opens an email from you, the lower the chances of them ever opening another email. This is an inevitable "law" of email marketing. Depending on the industry, what qualifies as an engaged user varies. At Paystack, we default to a conservative 90 day period of inactivity before drawing subscribers into a re-engagement campaign.
Here’s how I found our “90 days” number:
- I queried our email list with this condition: “show me the number of subscribers who haven't opened our email in the last 10 days”
- I then increased the inactivity period by 10 days at a time till I got to 150 days
At first, the corresponding number of subscribers dropped sharply as the inactivity period increased but the drop started to steady and I found that after a point, the number of inactive subscribers remained flat.
Let's break this down with sample numbers and a simple graph.
(Please note that these numbers are not real - they're sample numbers for the purpose of this demonstration.)
From this graph, it’s clear that after 60 days of inactivity, a subscriber that doesn’t open an email from the sender will likely never open one again. This is a safe method for determining the cut off point for email lists.
Now to the actual “how” - how exactly did we prune our list? At Paystack, we use Drip to automate our email marketing efforts, so I designed and activated a “pruning workflow.” Here’s what the workflow looks like:
For this workflow, we only bring in subscribers who haven’t read the past 10 emails or opened a Paystack email in the past 90 days. We apply a "quarantine" tag, which we use to exclude these email address from subsequent broadcasts (in case they're simply tired of hearing from us), and we wait for 30 days, after which we draw them into a re-engagement campaign that encourages them to choose their email subscription preferences.
If they open one of the campaign emails and select their preferred email types, we keep them on our list and only send them the exact types of emails they opted into. Otherwise, we remove them from the Paystack email list.
Pruning our email list required some initial effort, but its benefits are long term and require little to no subsequent input. The pruning exercise improves open rates, as well as email deliverability.
Monitoring our email sender reputation metrics
We monitor key email metrics which help us detect fluctuations in our email deliverability, and take appropriate actions. Here’re the most important metrics we track:
- Spam rate - This is a ratio of emails that subscribers mark as spam compared to the number of emails that were sent out
- IP reputation - This gives a sense of how likely it is that an email client’s spam filter will mark our email as spam or not
- Authentication - This shows the ratio of emails that pass SPF, DKIM & DMARC checks compared to the number of emails that attempted the authentication
There’re a number of tools that can help you monitor one or more of these key metrics. I’ve, however, found Google Postmaster Tools to be one of the most user friendly tools to use.
Like you can see from the image above. we now score 100% across the three metrics - DKIM success rate, SPF success rate, and DMARC success rate.
📝 Important: Google Postmaster Tool won't provide data on emails that weren’t sent to a Gmail client, and that might be a downside for you if a significant percentage of your subscribers don't use Gmail.
Summary
I hope this post has provided a good understanding of everything we did to go from yellow banner to improved email deliverability!
To sum it up, here’re the action items that worked for us:
- Authenticating Paystack’s email sending domain by setting up SPF, DKIM, and DMARC
- Removing Paystack’s email sending domain from credible email blacklists
- Proactively pruning unengaged subscribers from our email list
- Monitoring key email metrics
If you’ve got questions, or ideas on better ways to improve email deliverability, please hit me up at [email protected] and address the email to Abdulrahman Jogbojogbo; I’d love to hear from you!
