What email A/B testing is#
A/B testing sends two or more variants of a message to random samples and measures which performs better on a chosen metric before sending the winner to the rest.
Why it matters#
Opinions about subject lines and layouts are unreliable; tests turn them into evidence and compound small gains over time.
What Mailchimp handles#
Most platforms, including Mailchimp, generate the records or settings you need under a domain authentication or sending settings page, and expose bounce and complaint data in reports. Confirm the exact location in Mailchimp's current documentation.
What you still own#
- Choose one variable per test and one primary metric (clicks or revenue, not opens).
- Size samples so results are statistically meaningful; small lists need bigger splits.
- Run long enough to capture late openers, typically 4 to 24 hours.
- Log results so learnings persist beyond one campaign.
Common mistakes#
- Testing several changes at once.
- Declaring winners on open rate after Mail Privacy Protection.
Frequently asked questions#
What should I A/B test first in email?
Subject lines, then the primary call to action, then send time. These have the largest measurable effect.
Does Mailchimp set up email A/B testing automatically?
Mailchimp provides the values and some checks, but publishing DNS, aligning domains, and monitoring results remain your responsibility.