Holistic Email Metrics Matrix: Are you seeing the whole picture?

Email analytics are powerful… and also dangerous. Used correctly, analytics can inform both wise strategic decisions and smart tactical improvements. Used incorrectly, analytics give you complete confidence to make bad decisions.

Unfortunately, there are lots of ways to reach the wrong conclusions with analytics, including:

• Not reaching statistical significance and making other mistakes with your A/B tests
• Using the wrong metric to measure success, like unwisely using open rates to determine a winning subject line
• Not recognizing the limitations of your ability to accurately measure a particular metric, especially complex ones like email marketing ROI, which only about 35% of marketers can measure very well
• Being overly focused on campaign metrics like opens and clicks, and failing to look at performance over time
• Not recognizing the influence of your email marketing efforts on the performance of other channels like direct mail and social media
• Focusing too much on company-centric metrics like email marketing revenue and not enough on metrics that indicate how subscribers and inbox providers feel about your emails

Those last three are symptoms of not seeing the big picture when it comes to email marketing metrics. Our Holistic Email Metrics Matrix provides that perspective.

It shows that the impact of an email campaign ripples outward, affecting:

1. The business directly in terms of revenue and leads generated, for instance
2. The business indirectly as part of omnichannel campaigns or by influencing the performance of other channels more generally
3. Inbox providers and how they perceive the business as a sender of email
4. Subscribers and how they perceive the business and its emails

Email marketing campaigns can be felt individually in the short-term—as well as cumulatively over the long-term as part of your overall email program. For instance, a particular email send could do serious damage to your sender reputation, but you might not see the damage manifest in terms of blocks or a blacklisting until you’ve sent several more subpar emails.

Let’s break down each quadrant of the Holistic Email Metrics Matrix and which short- and long-term metrics are in each…

Direct Business Impact

These metrics help brands measure how their email campaigns and their email program are directly affecting their sales, profits, web traffic, and other important business key performance indicators.

In the short-term, email metrics like conversion rates and email campaign revenue demonstrate the direct value of having sent a campaign. While the goal of one email may be to generate registrations of a webinar (lead generation) and another may be to create sales (conversions), these are generally the top metrics for determining the success of an email campaign.

Email Campaign Success

• Duration of engagement
• Clicks & click rate
• Post-click behavior (browsing, carting, etc.)
• Conversions & conversion rate
• Email campaign revenue & profit
• Lead generation

In the long-term, email metrics like email program revenue show the overall success that your email channel is having. For a more subscriber-centric view of revenue, look at revenue per subscriber and subscriber lifetime value.

 

Email Channel Success

• Email program revenue & profit
• Email revenue per email
• Email revenue per subscriber
• Subscriber lifetime value
• Subscriber RFM (recency, frequency, monetary)
• Email marketing return on investment (ROI)
• Lead pipeline value, etc.

Email metrics to watch: The duration of engagement, or time spent engaging with an email, is becoming an important way to measure the impact of interactive emails, which require marketers to look at email engagement in new ways because they effectively pull landing page content into an email.

Interactive emails will also likely require marketers to differentiate between an email open and an email interaction. The former is the firing of a tracking pixel within the non-interactive portion of an email, and tells you the subscriber viewed the email. The latter is the firing of a tracking pixel that’s placed inside the interactive portion of an email, such as inside an email carousel or hamburger menu, and tells you the subscribe engaged with the interactive element.

Indirect Business Impact

Subscribers don’t always follow the highly measurable golden path that we want them to. Sometimes they see an email, don’t open, but then go to your store or call your call center to convert because of it. Other times they open an email and then open their browser and type in your URL directly and convert.

Because of these and other common behaviors, marketers can significantly under-measure email marketing’s impact if they’re not connecting the dots.

In the short-term, looking at the web traffic, mobile app activity, social media activity, store sales, and other activity that your brand experiences during the 48 hours after an email is sent can give you a good idea of the omnichannel influence of a particular email campaign. Keep in mind that your goals may be changing from email to email, and also that one of your goals might be to reduce activity in another channel, such as call volume to your call center.

Email Campaign’s Omnichannel Influence

• Email campaign’s indirect effect on web traffic & sales, mobile app activity, store traffic & sales, social media activity, direct mail effectiveness, call center volume, etc.

In the long-term, looking at metrics like the web traffic, mobile app activity, social media activity, store sales, and other activity that your brand experiences during the 48 hours after you send an email versus days that you don’t can reveal the omnichannel influence of your email program.

Email Program’s Omnichannel Influence

• Email program’s indirect effect on web traffic & sales, mobile app activity, store traffic & sales, social media activity, direct mail effectiveness, call center volume, etc.
• Customer lifetime value of subscribers vs. non-subscribers
• Customer RFM (recency, frequency, monetary) of subscribers vs. non-subscribers

Email metric to watch: Comparing the customer value of subscribers compared to non-subscribers is a great way of measuring some of the otherwise hard-to-measure impact of email campaigns on customer behavior. For example, if your customers who subscribe to your emails spend $147 per year on average and your customers who aren’t subscribers only spend $100, then that difference speaks to the impact of your email program, regardless of the level of email engagement of those customers.

Subscriber Relationship

Email marketing is a give-and-take channel. In order for brands to win with email, subscribers have to feel like they’re winning as well. Maintaining a healthy subscriber relationship shows that your emails are giving your subscribers value.

In the short-term, email metrics like opens and clicks indicate positive subscriber reactions, while unsubscribes and complaints indicate negative subscriber reactions.

Subscriber Reaction

• Opens & open rate
• Clicks & click rate
• Click-to-open rate
• Unsubscribes & unsubscribe rate
• Spam complaints & complaint rate

In the long-term, the reactions to those individual campaigns affect the size of your list and how engaged that list is, which are key signs of the overall health of your email marketing list.

Email List Health

• Active list size
• List growth & churn rate
• Inactivity rate
• Length of inactivity
• Open reach (percent of subscribers who have opened at least one email in past X days)
• Click reach (percent of subscribers who have clicked at least one email in past X days)
• Tracking metrics by acquisition source

Email metrics to watch: Often, a good portion of the negative reaction to your emails will come from one or two acquisition sources—such as email list rental and co-registration, which are among the five most problematic subscriber acquisition sources. Tracking email performance by acquisition source can reveal which ones are causing outsized problems and are in need of revamping or being shut down.

Inbox Provider Relationship

In addition to pleasing subscribers, email marketers also have to please inbox providers, each of which can have different expectations for senders. Inbox providers look at how your subscribers react to your emails—including tracking many actions that marketers can’t see, such as foldering and deletes without opens.

However, they also look at how you treat their platform. For instance, if you generate a lot of hard bounces by trying to send email to accounts that don’t exist on an inbox provider’s platform, that tells them that you may be buying email addresses, guessing at email addresses, not removing addresses that previously bounced, or not doing enough to avoid typos and transcription errors. Similarly, if you’re hitting old inactive email accounts that the inbox provider has recycled into spam traps, then it tells them you’re not paying attention to list hygiene.

In the short-term, inbox providers look at behaviors that demonstrate that subscribers are engaged with your emails. Unfortunately, some of these behaviors—such as foldering, deletes without opens, and spam trap hits—are impossible for marketers to see. Other behaviors, such as forwards and duration of engagement, can’t been seen without special analytics. Even spam complaints can be tricky, since Gmail doesn’t report them through a traditional feedback loop.

Email Engagement

• Opens & open rate
• Duration of engagement
• Deletes without opens
• Replies
• Forwards
• Social shares
• Foldering/archiving
• Soft and hard bounce rates
• Recycled spam trap hits
• Spam complaints & complaint rate

In the long-term, how users of a particular inbox engage with your emails and whether you abuse their platform adds up, potentially leading to deliverability problems. Inbox providers look at engagement, bounce rates, and spam complaints over a period of time (often 30 days). So if one campaign has a spike in negative metrics, that probably won’t cause deliverability issues. But if you have a string of such campaigns, then you’re surely in trouble with one or more inbox providers.

Deliverability

• Delivered rate
• Inbox placement rate
• Blocks
• Blacklistings
• Tracking metrics by inbox provider

Email metrics to watch: Inbox problems have different tolerances for certain behaviors, which is due in a large part to the different demographics and attitudes of their user bases. So you might experience problems with, say, Gmail and not AOL. You’ll only know if you track email performance by inbox provider.

See the Whole Picture

Many email programs are focused almost entirely on the direct business impact of their email efforts. That’s due in part to the fact that that’s how they’re measured by executives.

That’s unfortunate because in many organizations up to half of email’s impact will be felt in other channels. Proving those effects and getting credit for them could have a massive impact on email marketing budgets.

Similarly, only focusing on company-centric metrics exposes brands to growing subscriber inactivity and deliverability issues. To have a well-balanced, high-performance email program, brands have to have a broad, four-quadrants approach to email metrics.

Want more resources like this?

Subscribe to Litmus Weekly to get your weekly dose of email inspiration, tips, and tricks, right in your inbox.

Get on the list →

The post Holistic Email Metrics Matrix: Are you seeing the whole picture? appeared first on Litmus Software, Inc..