Skip to content
Open Engine

Home Topics Email Marketing A/B testing

Email Marketing · A/B testing

Email A/B testing not working? Troubleshooting guide

MailMaid Academy · · 1 min read DMARC pass
Short answer

When email A/B testing fails, check in this order: the DNS or platform configuration is published and resolves; the sending system is actually using it; the domain in headers aligns with what you published; and the receiver's verdict in Authentication-Results or bounce messages.

Symptoms#

  • Messages landing in spam despite previously good placement
  • Bounces mentioning policy, authentication, or reputation
  • Provider dashboards showing a drop in reputation or authentication pass rate

Diagnosis steps#

  1. Confirm the configuration is live: query DNS from an external resolver and check the sending platform's settings page.
  2. Send to a seed mailbox and read the full headers, especially Authentication-Results.
  3. Compare the domains in From, Return-Path, and DKIM d= for alignment.
  4. Check provider dashboards (Google Postmaster Tools, Microsoft SNDS) for reputation and error rates.
  5. Review recent changes: new vendors, DNS edits, list imports, volume spikes.

Likely causes#

  • Testing several changes at once.
  • Declaring winners on open rate after Mail Privacy Protection.

Fix and re-verify#

  1. Choose one variable per test and one primary metric (clicks or revenue, not opens).
  2. Size samples so results are statistically meaningful; small lists need bigger splits.
  3. Run long enough to capture late openers, typically 4 to 24 hours.
  4. Log results so learnings persist beyond one campaign.

Frequently asked questions#

What should I A/B test first in email?

Subject lines, then the primary call to action, then send time. These have the largest measurable effect.

Keep reading on A/B testing