I think we can all agree on the fact that split testing is an effective method to find out what works best and get more out of your existing traffic. It’s extremely useful since it can be applied to a number of different things: subject lines and content of emails, landing pages, home pages, creatives for ads and the list goes on.
Also, you can find articles, case studies on split testing for almost anything, except drip campaigns. It’s because experimenting with automated emails takes a lot of time and preparation.
It is nearly impossible and too technical with most marketing automation tools. In this article, you can find an easy to understand explanation of AI-powered optimization with an example email course where the subject lines are tested and optimized. Increase your open rate with machine learning in your drip campaigns!
The difference between Split Testing and the Multi-Armed Bandit algorithm
When you split test something you have an A, a B, and a C version of a website, a landing page, an email or ad.
You send 33% of your traffic to A, 33% to the B, and 33% to the C variation. You run this test until you reach a significant result: when the sample size is big enough to prove that the result is not accidental. For example, 97% significance means that we can be 97% sure that our results are valid. So there’s just a 3% chance that our results are accidental. This is the exploration or learning period.
In split testing, you run the experiment long enough to have a winner of the variations that is significant.
After you have a winner, the exploitation (earning) part starts: you send 100% of your contacts to the variation that won.
So you have a significant sample, you learn from it. Then you implement the things you learned – this is the process of split testing.
The problem with this is that the whole process has two distinct stages. Generally, the exploration period is so long that makes it hard and time-consuming to adapt to possible changes in campaigns. Also, the changes made might not be relevant because it’s based on data collected over a long time.
Multi-Armed Bandit Testing
While split testing uses the pure exploration and exploitation phases that are distinct, the multi-armed bandit algorithm mixes these phases. It results in adaptive changes because it is continuously mixing exploration and exploitation.
The two main advantages that multi-armed bandit testing has over split testing is that:
- the transition between exploration and exploitation is smoother
- and the data collection (exploration) wastes too much time and resources in split testing.
You will earn and learn. As Matt Gershoff said, during the exploration phase, you learn but it has a cost: you lose opportunities. But if you can decrease the cost of exploration by immediate implementation of your learnings into exploitation, you’ll have higher ROI.
So the purpose of multi-armed bandit algorithms, in this case, can be summed up in one goal: to reduce the possibility of sending traffic to lower performing variations as fast as possible. This way the cost of experimentation will decrease.
Multi-Armed Bandit Testing + Machine Learning = high volume of automated split tests in your drip campaigns
The multi-armed bandit algorithm mixed with machine learning opens up a world of new possibilities: automated drip campaign optimization that saves times and helps marketers have higher open, click and goal conversion rates.
This type of testing can’t be performed using traditional split testing. Because humans simply can’t conduct multi-armed bandit tests as it requires countless repetitive phases of learning and exploiting.
In a nutshell: the AI-powered optimization in drip campaigns means that the machine learning algorithm is continuously testing and optimizing your drip campaigns to achieve a higher conversion rate based on the goal you previously picked So all you have to do is:
- set up a drip campaign,
- create A/B/N versions of your emails,
- choose the time period you want your emails to be sent
- and a goal.
From this point on the system will test your drip campaign as more and more leads go through it. Without any the need for any manual intervention. Conversion goals can be:
- basic email metrics like: open and click rates,
- form submissions on a landing page,
- custom events like registration for trial or payment for service
- and so on.
Now that we covered what you need to know about how the AI works let’s have a look at the email course to have higher open rate with machine learning:
Higher open rate for your email course
Email courses are one of the best ways to generate quality leads. You provide educational content, therefore the leads will be more qualified and higher engaged after they finish your course. Not to mention that it is quite easy to set up an email course.
For a successful course, you’ll need a well-constructed landing page with proper wording and a value proposition. The order of the emails must be well thought out, and each email should have a clear structure.
Start with a summary of the course. Then continue with the actual lessons, and the last lesson should include either a special offer for the participants or just your sales pitch.
It’s crucial that the email course should be related to your product. Therefore, it should be able to solve the challenges your subscribers seeking answers to.
With a well-designed, problem-focused strategy it’s easier to convert a course subscriber to a trial user and a trial user to a paying customer at the end of the day.
But as nothing in life, it can’t always be applicable: a longer sales cycle, for example, might require some additional lead nurturing process too.
Let’s look at the example: a 6-day email course consisting of 6 emails. For each email, 3 different subject lines are tested. The topic of the course is digital marketing fundamentals, and it contains the following lessons:
- Welcome email and summary of the email course
- Introduction to digital marketing
- SEO Fundamentals
- Email Marketing Basics
- Social Media Marketing Essentials
- Analytical Foundations
People can sign up for the course on a landing page where they have to fill out a form with their name and email address. After opting in they receive a welcome email. In this email, you thank them for signing up and give them an outline of the course so they know what to expect. The next day they receive the first lesson, the following day the second, and so on.
Now let’s have a look at some possible subject lines for the 6 emails in the campaign:
However, the problem with email courses is that the open rates of the emails almost always decrease as the course goes on. But since you provide valuable educational content in all of these emails you want your recipients to open them. Otherwise, you wasted time and effort putting together the course, if barely anyone opens the last emails in the campaign which usually include your sales pitch.
So, in this case, the goal is to increase the overall open rate throughout your drip campaign. You’ve set up a killer drip campaign for your email course where the trigger is a form subscription. If you want to have a higher open rate, you have to conduct split tests for the email subject lines in your drip campaign.
First, let’s have a look how to conduct split tests for all these subject lines manually.
The first problem is that it takes a lot of time until enough people go through a certain drip campaign for the results of split testing to be significant. So the exploration period is very long.
The second challenge is executing split testing on 6 emails with 3 subject line variations for each. Can you imagine how long would it take? Theoretically, it’s possible to do it, but it consumes way too much time. And even if you end up with results, the amount of time and resources you put into testing just isn’t worth it. Especially, because there’s an easier way.
That easier way is through testing with a multi-armed bandit solution that is supercharged with machine learning algorithm.
All you have to do is write the emails, set up the subject line variations. After that, it’s time to let the algorithm do its job. In this case that is to determine which combination of subject lines and sending times result in the highest possible open rate. The algorithm starts out with a sending weight of 33.3% percent for each subject line per email. But as the subscribers move through the drip campaign it adjusts the sending weights based on the open rates. This process continues to ensure the best possible message sequence. Here’s a short tutorial video on how to set this up:
So there you go, with this algorithm multiple days of work can be done in about an hour. In addition, it will continue to optimize your campaign based on an endless cycle of testing. So increase your email open rate with machine learning and scale your business!
More to come…
This article is the first of a three-part series where you can learn about the potential use cases of the AI-powered workflow optimization. In the next one, you can take a look at how the multi-armed bandit can increase your conversions during user onboarding. After that we show you how to build a more loyal customer base and increase upsells by testing the body copies of your lifecycle emails.
Send emails at the right time, to the right people with the right message. Get started now!
He graduated from the Technical University of Budapest, with master’s degree in Software Engineering, Economics and Management.
As a serial entrepreneur he has 10+ failed projects and one big winner one: growing Protopmail email marketing service from zero to a market leader position in Hungary.
He has been working more than 20 years in software development, more than 10 years in IT project management and also having the serious of 10+ years experience in email marketing and marketing automation.
Aside from running a successful business, Gabor is also an email marketing and marketing automation consultant and works for clients like GE Money, DHL, Bridgestone and for several banks and insurance companies.
He also appears at multiple online marketing colleges as a guest speaker, and a well-known speaker at online marketing related conferences.
Gabor speaks 4 languages, has three kids and he is a serious golf addict.