Airplanes, Coca Cola’s Marketing, the great wall of China, French gastronomy, and many more examples are all undeniable achievements of the human mind. The Wrights brothers, who invented the first airplane ever, didn’t build the perfect flying engine on their first attempt. There were countless types of wings, stabilizers, and fuselages. They have built many hypotheses, theories, and tests, but only one thing was for sure: They didn’t want to die in the process. So they started experimenting.
Similarly, in any business, the only escape from unsuccess and deception is experimentation. You don’t want to spend millions on a failed campaign and wrap it up with hopes and prayers. You have to be confident that your next move will be a massive success because you know that it’s built on experiments and a testified hypotheses.
However, before you start experimenting, you should know that you can’t run the same test for a product and a Marketing strategy and you can’t understand customer insights and comprehend the human behavior all by following the same process. Having one option is not an option. We surely don’t want this to discourage you from running experiments and so to help you out we have made the Experiment Design Matrix. The Matrix regroupes over 15 types of experiments, classified according to your hypotheses.
Why you need to classify your experiments:
- Firstly, You need to know what options are available to you. By using the matrix and canvas to ideate on different experimental models and setups, you will be able to see what channels are of relevance to perform your tests on. Example: Fashion brands can run Pinterest ads, B2B companies can use Facebook or LinkedIn, etc. Yet, don’t let the canvas hold you back from thinking about your own experiments.
- Mapping out experiments gives you an overview of the flow of an experiment.
- Categorising experiments will help you in the selection process.
- Depending on the situation, you will know which questions you should answer and which departments you should include.
What are customers trying to accomplish?
Why are people showing certain behavior?
Does the UX and UI align for this service?
Which features of the product should we add/ remove?
- You will know which currency to ask for: Email, open questions, rating, suggestions, etc.
The 4 categories of business experiments
A popular tool in business, but why now? The answer has a lot to do with Digital Marketing, there are 2 special things in it that are driving experimentation. The first one is that in Digital Marketing, we can see more of the shopping and purchase process than we can on other mediums. We can easily “measure”. While in a TV ad, for instance, you can’t get exact data of how many people stepped into a store to buy a certain brand of soap.
In Digital Marketing, we can send different digital messages to different and targeted customers and see what is how they react to it individually, which gives us control. So when you combine measurement and control, you set a solid ground for experimentation.
If Marketing experiments were near to impossible to do 20 years ago, today we have no excuses not to run them and that’s why we are seeing an explosion of marketing experiments in the business world.
The purpose of this type of experiments is to examine the relationship between variables and is used when you want to prove to the world that you are right.
It is often based on numbers and statistics and hard facts. It involves fewer insights and is easier to analyze.
when you have the control and measurement as a key ingredient, you proceed by dividing the customers into multiple groups, then expose each group to a different advertisement, measure the outcome for each group and you compare them (ABCDE testing) to end up with more information and more data.
Examples: Emails ads
You can test endless things in an email ad: subject line, colors, graphics, time and place, call to action, discounts and promotion and what you get also is a set of measurable outcomes as open rate, what product customer view, how much time they spend on site, the unsubscription rate, etc.
Regardless if we are talking about a physical or a digital product, There’s no way to absolutely know how a customer is going to react to something new. The human behavior is completely unpredictable. The philosophy here is: you can’t move it if you don’t measure it. You can skip all the experimentation phases and decide to put your product out there, but how can you be sure the design is understandable? How can you know if the User Experience aligns with the User Interface? How can you know that your product responds perfectly to your target audience needs? What if your product is missing one or two characteristics?
It is indeed fun to think of the reaction that your product will receive once it is unleashed upon the world, and it’s even more fun to daydream about spending all of the theoretical money that your product or service will generate. But don’t forget that the devil is in the details and that experimentation is your only ticket to success.
The purpose of this kind of experiments is to explore the meaning of customers insights. Data here is mostly unstructured, anecdotal, revealing and hard to aggregate, but often too positive and reassuring.
It is used to gain an understanding of underlying reasons, opinions, and motivations. It provides insights into the problem or helps to develop ideas or hypotheses for potential quantitative research.
Example: In the intersection of product and qualitative experiment: Concierge experiment
Is an experience, of creating a hypothesis of who you early adopters may be and another one of what the value proposition would be. Once you have both hypotheses, running the experiment will help you decide of their validity. Even if your ultimate goal is to create a technology or a physical product, you should start by crafting an experience. Once you approach the early adopters and gather feedback, you can focus effort on what actually should be built.
Give a taste of the product and get feedback and insight from them. Establish a close relation with early adopters. And reduce iteration circle.
Example: Before starting a project of Online teaching, you can start by creating a one- page ad listing the product benefits. See how people react to it and what additional information do they require. This way, you can easily adjust to early adopters.
To conclude, you should absolutely keep in mind two main things: no great product, service or strategy had ever existed without a background of experimentation. Don’t fool yourself and people around you thinking “let’s just launch it and hope for the best!”. Second thing to remember is that people before you have worked so hard to come up with different types of experiments in different fields and domains, for you to be able to validate your hypothesis. All that’s left for you to do is download the Experiment Design Matrix.