Agencies send on behalf of many client domains and must isolate reputation per client.
What email A/B testing is#
A/B testing sends two or more variants of a message to random samples and measures which performs better on a chosen metric before sending the winner to the rest.
Why it matters#
Opinions about subject lines and layouts are unreliable; tests turn them into evidence and compound small gains over time.
Implementation plan for marketing agencies#
- Choose one variable per test and one primary metric (clicks or revenue, not opens).
- Size samples so results are statistically meaningful; small lists need bigger splits.
- Run long enough to capture late openers, typically 4 to 24 hours.
- Log results so learnings persist beyond one campaign.
Priorities specific to marketing agencies#
Agencies send on behalf of many client domains and must isolate reputation per client. Weight your effort toward the steps above that address this constraint first, and measure with metrics that match how marketing agencies generate value from email.
Common mistakes#
- Testing several changes at once.
- Declaring winners on open rate after Mail Privacy Protection.
Frequently asked questions#
What should I A/B test first in email?
Subject lines, then the primary call to action, then send time. These have the largest measurable effect.