Home | New Releases | ClickDimensions Split (A/B) Testing for emails

ClickDimensions Split (A/B) Testing for emails

We are happy to announce that we have released our email A/B testing feature. With this new feature marketers can test two versions of their email with varying subject lines, content (i.e. template) and/or ‘from’ address. The two versions are referred to as the A version and the B version. You will determine what percentage of your recipients should receive the A and B versions until a winning version is chosen automatically by our system or manually by you. The winning email version, A or B, is then sent to the remainder of the recipients. For example, let’s say you have a marketing list of 100,000 recipients and you want to test how an email performs with two versions of the subject line. You will create an A and B version with the two subject lines and then you might decide to send these versions to 1,000 of the 100,000 recipients and let the system determine the winner after a period of 2 hours. At the end of the 2 hours the system will choose a winner based on either unique open rate or unique click rate (your choice). The winning version, A or B, will then be sent to the remaining 99,000 recipients. The benefit is that the vast majority of your recipients will receive the better performing email version and you will learn what caused the increased engagement. Let’s take a look…

We’ll start by creating an Email Send just as we normally would. We’ll select a template, subject line, ‘from’ name/address, marketing lists, etc. Notice the new ‘Split Test’ button on the ribbon as well as the ‘Split Test Preferences’ section on the left navigation…

To perform a split test on this email we’ll click the ‘Split Test’ button on the ribbon. We are now able to set the content options for the B group (the content options for the A group were set when we created our Email Send record). In the screen below we changed both the subject line and the ‘from’ name and ‘from’ email address. As you can see below we could also even select an entirely different email template if we wanted to test a new design format or content option…

The next step in the process presents the email template for the B version. Here we would be able to make changes to it if we wanted it to differ from the A version. Note that whatever ClickDimensions editor you used to create the template (i.e. our block editor, free style or custom HTML editor) will be shown here…

The next screen lets us set our testing variables. We’ll start by moving the slider bar to change the size of the A and B test groups. These will be randomly selected recipients from our marketing lists and the groups will always have an equal number of recipients. Here we can also select whether we want to pick the winner based on how many people opened it (unique open rate) or clicked on links in it (unique click rate). We will also set how long the test should run before the winning version is sent to the remaining 4,834 recipients. Alternatively we can tell it to not automatically select a winner and we can check back periodically and pick the winner manually based on the click or open rate at that point…

Once we click ‘Save and Send’ the email will be sent to the A and B groups and the test has begun. In the Email Sends view you will notice a new System View called Split Test where the Email Send record will reside until it a winner has been selected. At that time, the remainder of the emails will be sent and the Email Send record will then appear in the Outbox view briefly and then the Sent view as per the normal process…

As the test is underway, we will be able to see the results of the A and B version unique open and click rates in the statistics are of the Email Send record…

In addition, the content options for the B version are recorded on the Email Send record so you can be reminded of what made that version work better or worse for you…

If you are looking at an individual Sent Email email record you will also notice that an A or B will appear in the ‘Split Strategy’ field if this recipient was in either of the test groups. This would allow you to further segment and see if perhaps one type of recipient (i.e. mail vs. female) responded better to one version or the other…

And, believe it or not, we even record the A/B version at the email event level (i.e. open, click, etc.) should you want to get extra deep in your analysis.

We hope you like this new feature as much as we do. Go do some testing and improve your engagement rates. Happy Marketing!

2017-05-16T18:19:22+00:00 By |New Releases|2 Comments

About the Author:

mm
The ClickDimensions marketing team is committed to helping marketers using Microsoft Dynamics 365 achieve their goals.

2 Comments

  1. Yeequon December 3, 2012 at 12:06 pm - Reply

    How can you view the stats of each test set? We have successfully delivered a split test but I have no way to view detailed analytics such as (link performance, who clicked on what, who opened) by test email. I can’t find a report that only reports on A or B. Hope that makes sense.

  2. ClickDimensions March 15, 2013 at 8:56 pm - Reply

    Not sure that is possible. Would have to ask Microsoft.

Leave A Comment