Installation and Setup
-
1
Download the Plugin
Download the latest version of the plugin below as a zip file. -
2
Install the Plugin in WordPress
- Login to your WordPress Admin Dashboard.
- Navigate to Plugins -> Add New.
- Click on Upload Plugin at the top of the page.
- Choose the zip file downloaded from GitHub.
- Click Install Now.
-
3
Activate The Plugin
- Go to Installed Plugins.
- Activate: Captchify Simple A/B.
-
4
Sign up For Captchify
- Login or Create a Captchify Account
- If you see "Failed to fetch account data." Refresh a couple times as the initial setup takes some time
- Wait until you see this page.
-
5
Choose a Billing Plan
- Click on view our pricing plans.
- Choose the free plan for now.
-
6
Enter your Billing Email
- Enter an email for your billing profile it can be the same or different then the email you signed up for.
-
7
Wait for Completed Billing Setup
- Your Billing Status will change in a couple minutes as our system processes your subscription. You will know when the setup is complete if you account page on the Portal looks like the image
-
8
Go to the Simple A/B Portal and Generate an API Key
- Go to the Portal
- Go to API Keys
- Click add Application
- Type in a name and click submit
-
9
Add API Key to your Wordpress Site
- Login to your WordPress Admin Dashboard.
- Navigate to Settings -> Captchify Simple A/B.
- Copy and paste the API Key in the portal to the API Key field and save settings
-
10
Create your First Experiment
- Go to the Portal and Click Experiments
- Select your Application and click Add Experiment
- Add a Prefix in this case we chose demo
- Add a description
- click submit
- After your create the experiment click on its name to navigate to the setup
-
11
Add Dimensions
- After Navigating to our experiment lets add Dimensions. More info about segmentation and dimensions can be found here. In short, its how you want to segment your users on your site.
- Lets keep it simple, click Add Dimension and select Global and All devices for now and click Save Dimension
-
12
Add Treatments
- Now lets add a second treatment. This is what you will use to determine what to show to a segment and in what proportion. Note: Control is already populated by default.
- Click on Add Treatment and write a description. In my case I work demo. and click Save Treatment.
-
13
Add your Dimension to a Stage
- Now that we have treatments and dimensions lets do to the Prod tab which is meant to represents production.
- Click on Add Dimension and select GLO-all (this is the identifier string for Global - All). Click Add Dimension.
-
14
Allocate your Treatments
- This is the step where we will set the ratio that a treatment is shown to this specific segment. Check off the box next to Global - All to select it.
- Click Set Allocations.
- Type 100 in the C box and click Apply Allocations.
- This now set Global - All to show C 100% of the time in production, but we are not finished yet!
-
15
Set your Exposure
- Now, we need to configure the exposure for the dimension. This allows you to allocate 50% to C and 50% to T1, while limiting the experiment to only 10% of your user base. In our case we are going to set it to 100% for simplicity.
- Click Set Expsoure.
- Drag the dongle or type 100 to set it to 100%
- Click Apply Exposure
- Now Global - All will determine treatment allocations for all your user base.
-
16
Enable your Experiment
- We now have Global - All configured to show 100% of treatment C to 100% of our populace. Thought there is one extra step to do before it will work. We have toggle the Enable switch. This switch is there so you can quickly turn off an experiment.
- Click Enabled.
-
17
Set up your short code
- Congratulations, we are now ready to integrate this experiment into your WordPress site.
- In this demo we will create a button that will simulate a check with a revenue amount. C will give 100 dollars and T1 will give use 200 dollars.
- Copy the below code, note you should type it out out as it is formatted for display and is non functional if you copy it.
[simpleab_test experiment_id="<your-experiment-id>" stage="Prod"]
<button data-simpleab-metric="revenue" data-simpleab-aggregation="sum" data-simpleab-value="100" data-simpleab-events="click,focus"> Click Me - C! </button>
||
<button data-simpleab-metric="revenue" data-simpleab-aggregation="sum" data-simpleab-value="200" data-simpleab-events="click,focus"> Click Me - T1! </button>
[/simpleab_test] - In this demo we created a dummy post with the above short code block
-
18
Click the Control Button a Bunch
- Lets collect some data, click the control button a bunch to submit metrics for the experiment.
-
19
Change the allocation to T1
- Go Back to your experiment and set the allocation to 100% T1.
-
20
Reload the Demo Page and Click the T1 Button
- Lets collect some data, click the T1 button a bunch to submit metrics for the experiment.
-
21
Reviewing The Experiment Results
- We will start seeing real time results from our metric results as soon as we get enough data.
- Lets Review the metric results by going to the Results Tab
- Choose the Metric Name, Stage and Dimension from the drop downs
- Choose if you want to have the probability calculation be for maximizing the results or minimizing the metric
- Select the Date Range to analyze for and click Get Results
- This table presents the results of an A/B test with two treatments, C (Control) and T1 (Treatment 1). Here’s what each column represents:
- Treatment: Identifies the group or variant in the experiment. "C" is typically the control group (baseline), while "T1" is the treatment group being tested against the control.
- Probability To Be Best: Indicates the likelihood that each treatment is the most effective. In this case, Treatment T1 has a 100% probability of being the best, suggesting it performs better than the control across the observed data. The control (C) has a 0% probability, indicating it is not the best performer.
- Relative Difference: Shows the percentage difference in effectiveness between the treatment and control. Treatment T1 has a 38.96% relative improvement over the control group. This value represents the extent of T1’s performance gain compared to C.
- Credible Interval: Provides a range within which the true mean effect of each treatment is likely to fall. For the control (C), this interval is [122.98, 167.13], meaning there is high confidence that the true mean for C falls within this range. T1 has a single value of 201.57, which indicates a very tight credible interval with high certainty.
- Mean: The average observed outcome for each treatment. For the control (C), the mean is 145.0567, while for T1, it is 201.5743. These values suggest that on average, T1 performs substantially better than C.
