> For the complete documentation index, see [llms.txt](https://docs.bloop.plus/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bloop.plus/referral-program/ab-testing.md). # A/B Testing Test two reward offers against each other and let the data set your reward. Guesswork is the most expensive way to set a reward. A discount too small never spreads, one too generous quietly eats margin, and from the outside the two look identical. A/B testing replaces the guess with evidence: run two reward setups side by side and let conversions decide. Bloop splits new participants roughly 50/50. **Control (A)** is your current rewards, untouched. **Variation (B)** starts as an exact copy that you edit to test one idea. When you end the test you pick which side to keep, and Bloop applies it to the live campaign. Run tests from **A/B Testing** on the campaign — one active test per campaign at a time. An A/B test belongs to a campaign. Open **Referral → Campaigns**, open the campaign you want to test, then **A/B Testing** under **Others**. > **Best practice:** Test the reward value and reward type first — they move conversions more than wording or layout. Change one thing per test, run until both sides have real volume, and bank each winner before starting the next. ### What you can test The test covers the **referrer reward** and the **referee reward**, and within each you can change anything the reward editor offers: | What you change on B | Why it's worth testing | | ---------------------- | ------------------------------------------------------------------------------- | | Reward value | The heaviest lever — a few points of discount changes who bothers to share | | Reward type | Coupon, store credit, cash or a free product read very differently to customers | | The give-and-get split | Moving generosity to the referee side often beats raising both | | Minimum requirement | A guardrail more than a growth lever, but worth knowing its cost | | Expiry | Whether urgency helps or just annoys | Write your hypothesis as one sentence before you start, and change exactly one variable. A win from two simultaneous changes tells you nothing about which one caused it. ### While a test is running The campaign's reward settings are **read-only**. Changing a reward mid-test would invalidate the result you're paying traffic to collect, so Bloop blocks it — end the test to edit rewards again. For the same reason you can't create a second test while one is active. Participants keep their assignment for the life of the test, so switching device or logging in doesn't move anyone between sides. Referrers who were already active before the test started are excluded entirely. ### Run a test 1. Open the campaign's **A/B Testing** tab and **Create test**. 2. Name it. The audience split is fixed at 50/50. 3. Under **Variable**, the referrer and referee rewards appear side by side as **Control** and **Variation**. **Edit reward** on the Variation column and change your one thing. 4. **Start test.** The test shows as **Active** with its start date. 5. Leave it alone until both sides have real volume. A test you started configuring but didn't launch stays in the list with **Continue setup**. ### Read the results The detail view shows how many referrers enrolled on each side, the test's duration so far, and a metric you choose: | Metric | What it counts | | ------------------------ | ----------------------------------- | | Referral revenue | Revenue from referees' first orders | | Referral orders | First orders placed by referees | | Referral invites | Referral sharing links generated | | Referee store visits | Visits from referred customers | | Referee coupon claims | Coupons claimed by referees | | Referral conversion rate | First orders ÷ referral invites | Bloop shows the gap as "X more than A/B" so you can see the direction at a glance, and warns you when a test is too young to read. **Referral conversion rate is usually the metric to judge on.** Raw revenue rewards whichever variant happened to catch a big order; conversion rate asks the question you actually care about — of the people who shared, how many produced a sale? Then check the cost. A variant can win on referrals and still lose money because it gives away more per order. Read the metric alongside what each reward costs you. ### End the test Ending asks you to pick **Control A** or **Variation B**. The winner's settings are applied to your campaign automatically — including when you pick A, which simply restores what you already had. A test can't be restarted once ended, so don't end one to "have a look". The results stay in the list for reference. ### How long to run it Run until both variants have enough conversions to trust the gap. That's about volume, not the calendar: * **Don't conclude from a handful of conversions**, and don't stop the moment one side pulls ahead. Early leads swap back and forth, and peeking crowns false winners. * **A stable gap after real volume is a real result** — including "no difference", which tells you to keep the cheaper option. * **If your volume is too thin** to reach a trustworthy read in a reasonable time, grow referral traffic first. A/B testing isn't the right tool yet. ### A worked example *Illustrative, not measured Bloop results.* Your live offer gives referees 10% off; you think 15% would spread further. Bloop copies your rewards into Variation B, you change only the referee discount, and you leave it alone. B converts at 12% against A's 9% — a stable gap that still covers the deeper discount on margin — so you end the test on **Variation B** and Bloop applies it. Had B only edged ahead while costing far more, the right call was **Control A**. ### Common mistakes to avoid * **Changing more than one thing.** You won't know what caused the win. * **Stopping too early.** A few conversions each side is noise. * **Judging on revenue alone.** Conversion rate and cost per referral are what you can act on. * **Ending a test to peek.** It can't be restarted. ### Next steps * Set up the rewards you'll test in [Referrer and referee rewards](/referral-program/rewards.md). * Test the wording as well as the value with [reward display](/referral-program/reward-display.md). * Track the outcome in [Referral analytics](/referral-program/analytics.md). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.bloop.plus/referral-program/ab-testing.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.