Experiments

​

What is Experiments in Brew?

• Subject lines - Test different approaches to see what resonates with your audience
• Email content - Compare different messaging, tone, or value propositions
• Design elements - Compare layouts, images, or visual styles
• Call-to-action buttons - Test different CTA text, colors, or placement
• Many more elements - Really anything you can edit in an email, you can test…

​

Creating an Experiment

1

Create Your Experiment

• With AI
• Manually

Simply state in the left-side chat that you’d like to create an experiment

Create an experiment to test different subject lines for the “Unleash Your Creative Potential with AI” email

Brew will sometimes proactively create Experiments for you.

2

Configure Your Audience Split

Set your experiment’s audience split by adjusting the percentage slider in the Experiment node. This determines how many contacts will receive your test variants versus your control.

Understanding Experiment BranchesExperiments consist of two types of branches:

• Variant branches - Contain the emails you’re testing. These are sent to the percentage of contacts you specify. Your variants automatically split the test audience equally. For example, with a 60% test size and three variants plus a control:
  • Each variant receives 20% of contacts (60% ÷ 3)
  • The control receives the remaining 40%
• Control branch - Contains your baseline emails sent to all contacts not in the test group. Controls are optional:
  • We recommend always having a control branch for accurate comparison
  • Without a control, only your selected variant sample size will enter the experiment
  • Example: With three variants at 75% sample size and no control, the remaining 25% of contacts will exit the automation without receiving emails

Before starting: Ensure you have at least one variant or control branch with at least one email in each branch.Changing branch type: Convert a variant to control by clicking the variant node and toggling Use as control.

3

Create Your Test Variations

Now it’s time to make the changes to each of your emails that will be tested. You’ll need to apply these changes to each email one-by-one:

1. Click on the email node for each variant
2. Enter the editor for each email
3. Describe the changes you want to make in the chat or make manual changes

• Subject Line Testing
• Content Testing
• Design Testing

“Change the subject line to focus on benefits instead of features”

“Make the subject line more urgent and action-oriented”

“Create a subject line that uses personalization with the contact’s name”

4

Launch Your Experiment

To run your experiment, set your automation to Live in the top right corner. Before doing so, make sure:

• At least one variant or control branch connected to your experiment node
• At least one email in each branch

If these conditions are not met, the automation cannot be set live.

5

Crown Your Winner

Once your experiment is running and contacts have flowed through the automation, visit the Metrics tab in the automation to analyze performance. Here you can compare variants and view key metrics for sends, opens, and clicks.After sufficient testing, you’ll want to send all future contacts through your best-performing variant. Here’s how to do so while preserving your test data:

1. Convert your winner to the control: Click on your best-performing variant and toggle Use as control
2. Route all traffic to the winner: Set the Experiment node sample size to 0%. This automatically changes the Control size to 100%, sending all future contacts through your winning variant

This approach maintains all your experiment metrics for future reference while ensuring new contacts receive your proven best-performer.

​

Editing Live Experiments

• Adjust the sample size percentage
• Edit variant and control branches
• Update email content
• Add or remove variants

​

Best Practices

​

Need Help?