A/B Email Comparison Tests
What is an A/B comparison Test?
A Comparison Test is a method of gauging the effectiveness of the content of a message while a campaign is in progress. By delivering two different messages, equally distributed amongst a percentage of your subscribers, we can determine which group received, opened or clicked more of the links in your message and send the remaining readers the better option.
The greater the size of your mailing list, the more effective comparison tests will be. We suggest utilizing comparison tests on lists with more than a few thousand subscribers.
The image to the right shows the comparison test options available. Optimum settings depend on your list size, spread and whether or not you're tracking links. Read on for more detailed descriptions for each option.
Which field would you like to compare?
For each campaign, you may compare one Subject against another, or one From name against another, or one Message against another. Which ever you select, you will then be asked to enter 2 of them. If you were to select Subject, then we'd send Subject A with every odd numbered message, and Subject B with every even numbered message.
How should we decide on a winner?
If you send the test messages to your first 1,000 Subscribers using the Subject test condition, then the 500 subscribers would have received Subject A and 500 subscribers would have received Subject B. If you select Most opens as the deciding factor, and your subscribers opened more of the messages with Subject A, then the remaining subscribers in your mailing list would be sent Subject A, as that was more likely to be opened and read by your subscribers. Or you could have the winner decided by Most clicks. That is, whether more clicks were recorded for messages using Subject A or Subject B.
Hours we should wait for statistics?
Once your test group of messages have been delivered, the remaining messages can be put on hold while we gather statistics. Not everyone is instantly aware of messages waiting in their inbox. You may choose to wait several hours so a greater number of recipients have had a chance to check their email. Once the specified number of hours has lapsed, we'll look at which message had more success and deliver that message to the non test group.
Percentage of Mailing List to be tested?
This is the test group. For example, you may decide to apply the A/B split comparison test on the first 20% of your subscriber database. If you have a list of 10,000 subscribers, then 1,000 would receive Test A, and 1,000 would receive Test B. The remaining 80% of subscribers would be sent the better performing message, based on which message garnered either the Most opens or the Most clicks.