The Reality of Reporting: What’s Behind Open and Click Rates?

4 min.

671

A Reporting Challenge

The tryandreply.email newsletter launched in October 2024 features a multi-email welcome series designed to nurture and educate new participants about the email experiment.

Here’s how a 150% Open Rate can teach marketers and stakeholders that email stats aren’t always what they seem.


When you check an email campaign’s open and click-through rates, you’re only seeing part of the story. Email platforms offer sleek reports with polished charts and summaries, but without attention to detail, those numbers can be misleading.

Behind every open and click metric, there’s a reality check waiting to be questioned.

Pulling Back the Curtain on Open & Click Rates

Open rates aren’t as reliable as they used to be.

Senders need to navigate inflated open rates caused by Apple’s Mail Privacy Protection and corporate scanning that automatically open emails. These privacy features are meant to protect user data but can create false-positive engagement.

Open rates still offer value:

  • Great for A/B testing subject lines
  • Helpful for tracking engagement with large sample sizes
  • Provide insight into audience interactions

But they shouldn’t be your only metric.

Understanding What Counts (And What Doesn’t)

To assess the success of the tryandreply.email welcome journey, I dug deeper than just open and click rates. With such a small audience, there is a need for more context and more KPIs to measure what success looks like.

What you need to know:

  • Open Rate: Measures how many recipients opened an email. It’s tracked by an invisible pixel that loads when the email is opened.

    The Issue: Automated opens, like those from Apple’s privacy feature or corporate firewalls, inflate these numbers.
  • Click Rate: The percentage of recipients who clicked a link in the email. More reliable, as it requires actual action.

    Important: Pay attention to the Click-to-Open Rate (CTOR). It shows how well the content resonates with those who opened the email. If an email campaign has high opens but low clicks, it might signal a disconnect between the subject line and CTA.

The 150% Open Rate

After the first batch of subscribers completed the journey, I was ready to report when I noticed something that didn’t pass the smell test: the open rates for the final email in the series were 150%.

Is that even possible?

It turns out this particular welcome journey became an edge case that even stumped Mailchimp support.

After troubleshooting with the support team, it was confirmed that while the overall dashboard metrics were inaccurate, the individual email reports were correct.

The lesson? Inaccurate dashboards make it difficult to measure engagement effectively.

Until the issue is resolved, I’ll rely on manual data tabulation.

Open and Click Rates: Best Practices for 2025

To navigate these reporting challenges, here are some best practices for a more accurate view of a campaign’s performance:

  • Use Larger Test Groups: Larger samples help counteract inflated open rates from privacy tools, giving you more reliable data.
  • Track Subscriber Engagement Over Time: Don’t just focus on immediate open rates. Monitor long-term engagement to see how your audience interacts with emails over time.
  • Monitor Subscriber Behavior: Pay attention to when subscribers engage most. What time of day are they opening emails? When are they clicking the most? This helps optimize timing and content.
  • Use Click-Through Rates or Reply Rates for Real Action: Click rates and CTORs are more reliable indicators of engagement. These metrics show how your audience responds to CTAs and give insight into content effectiveness.

Pay no attention to the marketer behind the curtain

As marketers, we need to pull back the curtain on the data and question the numbers we’re being given. Opens and clicks aren’t the be-all and end-all. It’s crucial to look beyond the surface, use additional metrics, and understand the factors that can skew the data.

If you’re curious to see the real-time reporting or want to dive deeper into the Welcome Journey results that got us here, sign up for the newsletter and join the experiment.

I try new tools and tactics in email and you reply with your thoughts.
Your input drives the email experiment.

Behind every open and click metric, there’s a reality check waiting to be questioned. Here’s how a Welcome Journey stumped Mailchimp Support—and why email marketers and stakeholders must understand email stats aren’t always what they seem.


From Idea to Inbox: 4-Step Blueprint to Launch Your Newsletter


B2B Cold Email That Got an Open and Why (Gloria, Part 1)