Split tests, also known as A/B testing, simply test one variable in an email send against another variable. An example of this would be if you wanted to see whether an email with a green Click Here button receives more clicks than an email with a blue Click Here button.
Here, we are sharing some helpful tips you can use to get the most out of your next split test in ClickDimensions.
1. Change only one variable.
To run a split test, you need to create two different versions of an email send. You will send out these two emails to two separate groups of recipients, and analyze which email send performs better.
As you begin testing your email sends, you may find that there are many variables you want to test. However, we recommend that you choose only one of these variables per split test so you can be sure which of the variables causes a change in performance.
Keep in mind that simple things such as the image you choose, your subject line, sender name or the color of your Click Here button can lead to big changes in your email’s performance.
2. Determine your sample size.
If you are split testing an email, you will probably want to send the two versions of your test to a smaller sub-section of your recipient list. Once the split test has finished running, you can pick a winner manually or have the ClickDimensions split test wizard choose the winner for you based on clicks or opens, and then send the winning version to the remainder of your list.
3. Split your test groups equally.
When sending emails, you have control over your recipients, which means you can create equal test groups. Splitting lists evenly helps ensure that your results are reflective of the entire email list.
4. Test both versions at once.
Timing can make a big difference in your email send’s results. The time of day, day of the week and even month of the year can have major impacts on your email send statistics. If you were to send out Version A on Tuesday and Version B on Friday, you would not be able to conclude whether the change in opens was due to the email’s different subject line or the day of the week the email was received. For this reason, when you run split tests, we recommend that you run the two versions at the same time.
There will be situations where you want to use the split test wizard to test the email send timing itself to find the best time to send your email. In this scenario, you will just want to make sure that all other variables are the same.
5. Run the test long enough to get significant results.
Depending on your business and how you set up the split test, getting significant results could take hours, days or sometimes weeks. When determining how long your split test should run before selecting a winning version, you will want to consider the email events and email statistics of your other email sends. If you receive opens and clicks rather quickly for most of your email sends, you may want to select a few hours as your winning version indicator. However, if there is typically a lag in your recipients interacting with your emails, you will want to increase the time it takes before a winning version is selected.
6. Act on your results.
If one version of your split test performed better than the other version, congratulations! You have a winner! Complete your test by choosing your split test winning version manually or allow the split test wizard to choose the winner for you.
If neither version performed better, you can conclude that the variable you chose to test did not impact your results. If this is the case, you could either choose one of the versions as the winner or run another split test using a new variable to test performance.
Split tests help you view results on a send-by-send basis. You can, however, apply your learnings from each test to future email sends. For example, if you’ve conducted multiple split tests and have found that using numbers in email subject lines generates more clicks, or that sending out your emails on Thursdays gives you the most opens, you might want to consider using those methods in your future email sends.
7. Plan your next test.
The split test you just finished may have helped you discover a new way to make your email sends more effective. Just keep in mind that there is always more room for enhancing your email sends. You could even try conducting a split test on another aspect of the same email you just ran a test on. For example, if you just tested the subject line of your email send, you could also do a new test on the email’s color scheme or images.