> For the complete documentation index, see [llms.txt](https://docs.bloop.plus/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bloop.plus/referral-program/ab-testing.md).

# A/B Testing

Guesswork is the most expensive way to set a reward — a discount too small never spreads, one too generous quietly eats margin, and from the outside the two look the same. A/B testing replaces the guess with evidence: run two reward setups side by side and let conversions decide.

Bloop splits new participants into two groups. **Variant A** is your current rewards, untouched. **Variant B** is a copy you edit to test one idea. When you end the test and declare a winner, B's settings (if it wins) are promoted to your live campaign automatically. Run tests from **A/B testing** on the **Campaign** tab — only one test can be active per campaign.

> **Best practice:** Test the reward value and reward type first — they move conversions more than wording or layout. Change one thing per test, run until both sides have real volume, and bank each winner before starting the next.

### What you can test

| What you can test             | Change on Variant B                                |
| ----------------------------- | -------------------------------------------------- |
| **Reward value**              | Raise or lower the discount/credit/cash amount     |
| **Reward type**               | Switch between a percentage, store credit, or cash |
| **Referee vs referrer split** | Move generosity to one side of the give-and-get    |
| **Minimum requirement**       | Add, remove, or change a minimum spend             |

Write your hypothesis as one sentence before you start, and change exactly one variable per test. Reward value and type are the heavy levers — test those first. A minimum requirement is more of a guardrail; it rarely lifts conversions.

### What tends to win

Useful as a starting hypothesis, not a result to skip testing — your own data is the tiebreaker:

* A two-sided **give-and-get** reward outperforms a referrer-only offer, and a lower incentive sometimes wins on total economics.
* On creative: a **white background**, **product-focused imagery**, **no human photos**, and **visible social proof** tend to win.

### How long to run a test

Run until both variants have **enough conversions to trust the gap** — that's about volume, not the calendar. A few rules of thumb:

* **Don't conclude from a handful of conversions**, and don't stop the moment one side pulls ahead — early leads bounce back and forth ("peeking" crowns false winners).
* **A stable gap after real volume is a real result** — even a "no difference" tells you to keep the cheaper option.
* **Watch cost, not just count.** A variant can win on raw referrals and still lose money if it gives away more per order.
* If your volume is too thin to reach a trustworthy read in a reasonable time, grow referral traffic first — A/B testing isn't the right tool yet.

### A worked example

*Illustrative, not measured Bloop results.* Your live offer gives referees 10% off; you hypothesize 15% would spread further. Bloop copies your rewards into Variant B, you change only the referee discount, and let it run untouched until both sides gather real volume. Say B converts at 12% versus A's 9% — a stable gap that covers the deeper discount on margin too — so you declare **B** the winner, and Bloop copies its setting onto your live campaign. Had B only marginally beaten A while costing far more, the right call would have been to keep **A**.

### How it works

* **Variant A** is your current rewards, untouched. **Variant B** starts as an exact copy, so the only difference is whatever you change on B.
* Participants are assigned **roughly 50/50** and keep their assignment for the life of the test — switching devices or logging in doesn't move them. Referrers active **before** the test started are excluded.

### Run a test

1. Open **Campaign → A/B testing** and create a new test.
2. Edit **Variant B** — change the reward value, type, or a minimum requirement. Variant A stays untouched.
3. Name the test and start it — it moves from **draft** to **active**.
4. Let it run until both variants have real volume, then review the enrolled counts and metrics.
5. Declare the winner — **A** or **B** — to end the test.

| Winner    | What happens                                                                |
| --------- | --------------------------------------------------------------------------- |
| Variant A | Your campaign keeps its current rewards; Variant B is discarded             |
| Variant B | Bloop copies B's settings onto your live campaign, becoming the new default |

Bloop stores a snapshot of both variants' results, so you can reopen a finished test to review it later.

### Common mistakes to avoid

* **Changing more than one thing at a time.** A win tells you nothing about which change caused it.
* **Stopping too early.** A few conversions on each side is noise — wait for real, comparable volume.
* **Ignoring margin.** A variant that wins on raw referrals can still lose money if it gives away more per order.

### Next steps

* Set up the rewards you'll test in [Referrer and referee rewards](/referral-program/rewards.md).
* Track the outcome of a winning test in [Referral analytics](/referral-program/analytics/analytics.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.bloop.plus/referral-program/ab-testing.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.