A/B Testing for Email Simplified; Excuses Squashed.

by Jim Cota

Marketers inherently understand the benefits of using A/B testing to help improve their communications, but few take advantage of it. The reasons are many, but they fall into a couple of basic categories: It’s too difficult and it takes too much time. Now, as far as email is concerned, both of these issues have been solved.

Our clients are now enjoying the ability to easily perform their own A/B testing for their email campaigns. They’re testing subject lines, delivery days and times, and “From” addresses from within their Exact Target accounts. As one of the first Agency Partners to have access to this new feature, we’re already seeing some very interesting results, which we’ll discuss in a later post. Today, though, I wanted you to see how easy it is and eliminate all your excuses for not doing it. (Sorry about that!)

The new feature lies beneath this simple button in the interface:

A/B Testing Button in Exact Target Application

When you create a new A/B test, it’s an easy four-step process. It may look a little complicated, but you’ll just have to trust me; it’s very, very easy to do…

Step 1: First, just pick the email and then determine what you’d like to test.

A/B Testing Step One

This example shows a standard test of a promotional email where we’re going to see which subject line is most effective.

Step 2: Pick the recipients and set testing thresholds

During step two, you’ll select the list you want to send to and the thresholds for the test. This is pretty cool, because it allows you to set some very specific guidelines for the test. In this case, the overall list contains 168,000 subscribers, so we’re going to split the test among the first 10% of the total. About 8,500 people will receive the email with one of the two subject lines.

A/B Testing Step Two

Step 3: Set the criteria to determine the winner

Step three is the magic. Here we decide what criteria we’ll use to determine the winner. In some cases, you may only be concerned with open rates (how many people open the email). Other times, you may want to know how many people are clicking through to your site (especially important for e-commerce applications). In this case, I’m testing open rates and I’ve set the time frame to just one hour. With smaller lists, you might need to extend the length of the test to a couple hours– or even a day– depending on the normal behavior of your subscribers.

A/B Testing Step Three

Step 4: Set the actions for what happens after the winner is declared

At the last stage, you’re setting preferences for what happens after a winner is declared. You can choose to have the winning email combination sent as soon as a winner is determined, or you can send it at a scheduled time. You can also set up triggered notifications to tell you how the test is going; these emails will tell you when the winning conditions are met and when the winner is delivered.

A/B Testing Step Four

That’s it! Now just hit that “Finish” button, wait for the results to roll in, and garner the accolades of everyone you’re working to impress.

You're done!

Post a Comment

Leave a Reply

Your email address will not be published. Required fields are marked *

*