Are open rates a good measure of campaign success?
The answer might surprise you once you understand how open rates are calculated and the technical complexities affecting those calculations.
In this article, I cover these topics:
What is an open rate, really?
On the surface, the answer is simple. Somebody goes to their inbox and opens a message. That action counts as one "open event."
To calculate the open rate, take the total number of open events divided by the total number of delivered messages. Voila! If your open rate is between 20 and 28%, your campaign's average performance is.
But wait ...
Oh, here's where the math gets messy.
Tracking an open event depends on an invisible image added to every message (an industry standard). The image is tiny (just one pixel), so nobody can see it, and it does not affect your message design.
Whenever somebody opens an email message, our server gets a call to load the email content into the recipient's email client. That call includes the tracking pixel.
Every tracking pixel in every message has a unique URL. That means we can track who opens the message and count the number of open events.
With this new information, let's change our math calculation a bit. An open rate is the number of tracking pixel calls divided by the total number of delivered messages.
Total vs unique open rates
The tinyEmail campaign dashboard shows two types of open rates.
Total Opens takes all opens divided by total delivered messages. This includes messages opened multiple times by a recipient.
Unique Opens only counts the first open event and ignores multiple opens.
Are pixel calls a good measure?
In a word, no. Multiple factors can overstate or understate Total Opens and Unique Opens. Here's why.
Pre-loading
Some email clients (e.g. Gmail and Apple Mail) pre-load email images, including the tracking pixel. That means the email message sends a call to the server even though the recipient did not open the message. Why is that important? Pre-loading can inflate open rates.
Here's proof. I ran a small test campaign and sent messages to test accounts with different email clients. All the messages sent to my Gmail test accounts were recorded as opens even though I didn't open a single message.
Blocked images
Email clients can also block images from loading, which includes the tracking pixel. If blocked, the message won't send a call for the tracking pixel. That means a message opened by a subscriber won't get recorded as an open event. In particular, the privacy features in Apple's iOS 15 (and beyond) specifically prevent email tracking. The result is underreported open rates.
Let me add one caveat here. If a pixel call is blocked, but the reader clicks a link in the message, tinyEmail counts that as an open event.
Text-only messages
Every email you send in a campaign has two versions: HTML and text-only. The text-only version has limited formatting and looks like it was created with a typewriter. Almost every email client reads the HTML version, and few people choose the text-only version if that option is even available. Even still, it's worth noting that text-only messages remove the tracking pixel (along with all images).
That means the only way to track reader engagement (with text-only messages) is by tracking the number of links clicked in the message, which I discuss below.
Counting clicks
Given the unreliable nature of open rates, a better measure of email performance might be the click rate (sometimes called the click-through rate or CTR).
tinyEmail counts the number of times a subscriber clicks a link in your messages and then divides by the total number of delivered messages. We track both unique clicks and total clicks.
Here's an example. From our test campaign, I opened one message in a test account and clicked links. The tinyEmail dashboard shows the results (see below): one unique click and four total clicks. The percentages are based on the total number of delivered messages (n = 14).
Wrap up
A Greek philosopher thousands of years ago said change is the only constant in life. That's an accurate description of technology in general and email marketing in particular.
Technological change has made open rates less accurate. Once the yardstick of email marketing performance, it's now a legacy measure with some value, though it may be diluted.
Email marketers look at a basket of indicators when they want a global understanding of campaign performance. The basket includes include bounce rate, delivery rate, and unsubscribe rate.
I'd argue we should keep open rates in the basket, though more attention should shift to click rates if we want an accurate measurement of reader engagement.