How to A/B test your referral rewards
Guesswork is the most expensive way to set a reward. A discount too small never spreads, one too generous quietly eats margin, and from the outside the two look identical. A/B testing replaces the guess with evidence: run two reward setups side by side and let conversions decide.
Bloop splits new participants roughly 50/50. Control (A) is your current rewards, untouched. Variation (B) starts as an exact copy that you edit to test one idea. When you end the test you pick which side to keep, and Bloop applies it to the live campaign. Run tests from A/B Testing on the campaign — one active test per campaign at a time.
An A/B test belongs to a campaign. Open Referral → Campaigns, open the campaign you want to test, then A/B Testing under Others.
What you can test
Section titled “What you can test”The test covers the referrer reward and the referee reward, and within each you can change anything the reward editor offers:
| What you change on B | Why it’s worth testing |
|---|---|
| Reward value | The heaviest lever — a few points of discount changes who bothers to share |
| Reward type | Coupon, store credit, cash or a free product read very differently to customers |
| The give-and-get split | Moving generosity to the referee side often beats raising both |
| Minimum requirement | A guardrail more than a growth lever, but worth knowing its cost |
| Expiry | Whether urgency helps or just annoys |
Write your hypothesis as one sentence before you start, and change exactly one variable. A win from two simultaneous changes tells you nothing about which one caused it.
While a test is running
Section titled “While a test is running”The campaign’s reward settings are read-only. Changing a reward mid-test would invalidate the result you’re paying traffic to collect, so Bloop blocks it — end the test to edit rewards again. For the same reason you can’t create a second test while one is active.
Participants keep their assignment for the life of the test, so switching device or logging in doesn’t move anyone between sides. Referrers who were already active before the test started are excluded entirely.
Run a test
Section titled “Run a test”
A/B tests and FlexiTiers can’t run together. If the campaign has two or more reward tiers, remove them first (delete tiers down to one to return to a flat reward), then create the test.
- Open the campaign’s A/B Testing tab and Create test.
- Name it. The audience split is fixed at 50/50.
- Under Variable, the referrer and referee rewards appear side by side as Control and Variation. Edit reward on the Variation column and change your one thing.
- Start test. The test shows as Active with its start date.
- Leave it alone until both sides have real volume.
A test you started configuring but didn’t launch stays in the list with Continue setup.
Read the results
Section titled “Read the results”The detail view shows how many referrers enrolled on Control A and Variation B, with all six metrics side by side. There is no metric to pick. The test’s duration appears once the test has ended.
| Metric | What it counts |
|---|---|
| Referral revenue | Revenue from referees’ first orders |
| Referral orders | First orders placed by referees |
| Referral invites | Referral sharing links generated |
| Referee store visits | Visits from referred customers |
| Referee coupon claims | Coupons claimed by referees |
| Referral conversion rate | First orders ÷ referral invites |
Bloop shows the gap as “X more than A/B” so you can see the direction at a glance, and warns you when a test is too young to read.
Referral conversion rate is usually the metric to judge on. Raw revenue rewards whichever variant happened to catch a big order; conversion rate asks the question you actually care about — of the people who shared, how many produced a sale?
Then check the cost. A variant can win on referrals and still lose money because it gives away more per order. Read the metric alongside what each reward costs you.
End the test
Section titled “End the test”Ending asks you to pick Control A or Variation B. The winner’s settings are applied to your campaign automatically — including when you pick A, which simply restores what you already had.
A test can’t be restarted once ended, so don’t end one to “have a look”. The results stay in the list for reference.
How long to run it
Section titled “How long to run it”Run until both variants have enough conversions to trust the gap. That’s about volume, not the calendar:
- Don’t conclude from a handful of conversions, and don’t stop the moment one side pulls ahead. Early leads swap back and forth, and peeking crowns false winners.
- A stable gap after real volume is a real result — including “no difference”, which tells you to keep the cheaper option.
- If your volume is too thin to reach a trustworthy read in a reasonable time, grow referral traffic first. A/B testing isn’t the right tool yet.
A worked example
Section titled “A worked example”Illustrative, not measured Bloop results. Your live offer gives referees 10% off; you think 15% would spread further. Bloop copies your rewards into Variation B, you change only the referee discount, and you leave it alone. B converts at 12% against A’s 9% — a stable gap that still covers the deeper discount on margin — so you end the test on Variation B and Bloop applies it. Had B only edged ahead while costing far more, the right call was Control A.
Common mistakes to avoid
Section titled “Common mistakes to avoid”- Changing more than one thing. You won’t know what caused the win.
- Stopping too early. A few conversions each side is noise.
- Judging on revenue alone. Conversion rate and cost per referral are what you can act on.
- Ending a test to peek. It can’t be restarted.
Next steps
Section titled “Next steps”- Set up the rewards you’ll test in Referrer and referee rewards.
- Test the wording as well as the value with reward display.
- Track the outcome in Referral analytics.