🔬 Free Planner

Split Test Experiment Planner

Plan every A/B test before you launch it. Define your hypothesis, set up variants, configure tracking, and build the decision framework - so every test produces an actionable result.

Download PDF Book a Funnel Experiment Sprint →

Hypothesis

Every good test starts with a clear, falsifiable prediction

5 items

State exactly what you're testing in one sentence

Example: "Changing the hero headline from benefit-focused to pain-focused will increase opt-in rate." Be specific.

Define your expected outcome and direction (increase, decrease, or no change)

Write it as: "I believe [change] will [increase/decrease] [metric] by approximately [X%]." Forces clarity.

Choose your primary success metric (one metric only)

Opt-in rate, click-through rate, booking rate, or purchase rate. Never test for multiple primary metrics simultaneously.

Calculate the minimum sample size needed for statistical significance

Use a sample size calculator (Evan Miller's is free). For a 5% baseline conversion rate and 20% MDE, you need ~1,500 visitors per variant.

Confirm this test is worth running - will the result change what you do?

If you'd keep the same page regardless of the result, don't waste traffic on the test. Only test decisions you'll actually act on.

Variant Setup

One change per test - otherwise you can't attribute the result

6 items

Document the control (version A) - what the page looks like today

Screenshot the full page, note the headline, CTA text, layout, and any key elements. This is your baseline.

Document the variant (version B) - the single change you're making

Screenshot the variant. Clearly mark what's different from the control. Only one element should change.

Verify only ONE variable differs between control and variant

If you change the headline AND the button color, you can't know which caused the result. Isolate one variable.

Build the variant in GHL and verify it renders correctly on mobile and desktop

Use GHL's A/B test feature or create a duplicate funnel step. Test on real devices, not just the builder preview.

Verify the variant page loads at the same speed as the control

Run both through PageSpeed Insights. A slower variant will lose due to speed, not the change you're testing.

Save screenshots of both versions in a shared folder for future reference

Tests get overwritten. Screenshots + notes in a "Test Archive" folder prevent you from re-testing the same thing.

Tracking & Attribution

Bad tracking = worthless test - verify everything fires

5 items

Set up UTM parameters to distinguish control vs variant traffic in analytics

Example: utm_content=control vs utm_content=variant-headline-pain. Use consistent naming across all tests.

Define the conversion event and verify it fires on both variants

Form submit, button click, page view of thank-you page. Test the event in GTM Preview mode on BOTH versions.

Verify GA4 tracking is active on both control and variant pages

Check GA4 Realtime report while visiting each page. Both should appear as active users with correct page paths.

Verify GHL tracking and pipeline attribution work on both variants

Submit a test lead on each variant. Confirm the contact appears in GHL with the correct source/medium/campaign.

Confirm ad platform pixels (Meta, Google Ads) fire on both variants

Use Meta Pixel Helper and Google Tag Assistant. A pixel that only fires on one variant will skew your ad reporting.

Test Parameters

Set the rules before the test starts - never mid-test

5 items

Set traffic split percentage (50/50 is standard, 80/20 for high-risk changes)

50/50 reaches significance fastest. Use 80/20 if the variant is risky and you want to limit exposure.

Set minimum test duration (at least 7 days to capture day-of-week variation)

Weekend vs weekday traffic behaves differently. Running less than 7 days gives you biased results.

Calculate minimum conversions needed per variant (typically 100+ each)

With fewer than 100 conversions per variant, results are unreliable. At 50 or fewer, they're essentially random.

Set statistical significance threshold (95% is standard, 90% acceptable for low-risk)

95% means there's only a 5% chance the observed difference is due to random chance. Never call a winner below 90%.

Commit to not peeking at results until the minimum sample size is reached

Checking results daily and stopping early when one variant "looks better" is the #1 cause of false-positive test results.

Decision Framework

Know what you'll do with every possible outcome before it happens

5 items

Define winner criteria: what metric value and significance level declares a winner?

Example: "Variant wins if opt-in rate is ≥15% higher than control at 95% significance." Write this down before launching.

Plan for inconclusive results - what do you do if neither variant wins?

Options: keep the control (safer), extend the test duration, or design a new test with a bigger change.

Plan the rollout: how will you implement the winner across all traffic?

In GHL: set the winner to 100% traffic, remove the losing variant, and update any linked ads or email sequences.

Document learnings regardless of outcome - what did this test teach you?

Even "no difference" is valuable data. Record: test name, dates, variants, result, and one-sentence insight.

Plan the next test based on what you learned from this one

Each test should inform the next hypothesis. Build a testing roadmap, not one-off experiments.

Want ATJ to design and run your split tests?

We set up the experiments, configure tracking, monitor results, and roll out winners - so every test produces a measurable lift in conversions.

Book a Funnel Experiment Sprint → See all GHL services →