A/B testing sends two or more variants of a message to random samples and measures which performs better on a chosen metric before sending the winner to the rest.
What good looks like#
- Done: Choose one variable per test and one primary metric (clicks or revenue, not opens).
- Done: Size samples so results are statistically meaningful; small lists need bigger splits.
- Done: Run long enough to capture late openers, typically 4 to 24 hours.
- Done: Log results so learnings persist beyond one campaign.
What bad looks like#
- Seen in audits: Testing several changes at once.
- Seen in audits: Declaring winners on open rate after Mail Privacy Protection.
How to move from bad to good#
Work through the good list in order and re-verify after each change. Most teams find one or two items from the bad list already present; fixing those usually produces the largest improvement.
Frequently asked questions#
What should I A/B test first in email?
Subject lines, then the primary call to action, then send time. These have the largest measurable effect.