Facebook Conversion A/B Test: Count Successful People

Use the A/B Test Results Analyzer with Facebook context after preparing your own aggregate counts. The fictional Cedar Workshop team is comparing two explanations of a home-planning consultation. Its example follows eligible people through a valid request, then checks whether more requests also mean more useful appointments. The Facebook selector labels the project; it does not connect an account, retrieve reporting, or allocate an audience.

Decide whether the question starts before or after the click

Start by naming the decision. Cedar could compare two invitation messages, two form explanations, or two landing-page layouts. These are not interchangeable tests. A message comparison might ask whether an eligible assigned person eventually requests a consultation. A page comparison might ask whether an eligible visitor assigned on arrival completes the page form. Write the starting point into the metric description so a later reader knows exactly where the comparison begins.

If assignment happens before the click, counting only visitors who clicked can discard part of the message effect. People who ignore the invitation still belong to the assigned cohort when the planned outcome is a completed request per assigned person. By contrast, assignment that genuinely happens at the landing page can support a page-level question. Its answer concerns arriving visitors, not every person who might have encountered a Facebook post.

A form opener can be a useful denominator only for a question that starts at that stage and uses a trustworthy unique-person count. Do not divide one variant’s completed requests by form openers and the other’s by link visitors. Likewise, comparing a form route with a separate website route changes the experience as a package. Even with a sound experiment, the result would not isolate the effect of one sentence if the route also changed.

Cedar chooses a narrow landing-page question: among eligible visitors assigned when they arrive, does a practical room-planning explanation produce more valid consultation requests than a general service explanation? Both pages use the same form, offer, appointment availability, and completion rule. Facebook describes the campaign context, while the actual allocation and measurement are handled outside the analyzer. This keeps the eventual claim aligned with the people and experience actually studied.

Turn form records into unique eligible and successful people

Create a working record outside the calculator with a stable person reference, original group assignment, eligibility status, assignment date, and completion status. Use a consistent identity rule that your records can genuinely support. A visitor identifier is not automatically a verified person across devices. If unresolved duplication is substantial, do not present device or session totals as known independent people. Document the measurement gap before choosing an analysis.

Keep eligibility separate from success. Cedar defines eligible visitors before assignment: new consultation prospects in the served area, with no existing open request. A person who sees the page but does not submit remains eligible and unsuccessful for this outcome. Removing them because they were unresponsive would change the denominator after observing behavior. A failed validation on a submitted form also does not automatically remove an otherwise eligible person from the cohort.

Deduplicate successful people as well as the denominator. One prospect may submit twice after refreshing, correct a phone number, or ask about a second room. Those records can matter operationally, but the binary question gives that prospect at most one success. Apply the same completion rule to both groups. Decide beforehand whether a valid request needs working contact details and an in-area project, rather than tightening the rule only for the busier variant.

Reconcile the source records in a small audit. Record raw submission rows, repeat rows, invalid requests, and final successful people, with mutually exclusive reasons where you subtract categories. Keep the individual records in your own system; the analyzer needs only aggregate totals. It does not deduplicate identities or inspect form answers. If successful people exceed eligible people, stop and fix the record linkage instead of increasing the denominator until the input is accepted.

Set the comparison rules before the first result arrives

For a controlled experiment, arrange random assignment through a suitable process outside this tool and keep each eligible person in one group. Repeated visits should retain the original assignment rather than create new observations. Cedar changes only the page explanation. If staff send different follow-up messages to the two groups during the outcome window, the comparison may include those messages too. The written plan should make that boundary visible.

Choose both an enrollment period and an outcome deadline. For example, visitors may enter during two planned weeks and receive seven days from assignment to complete a valid request. The analysis then waits until the final visitor has that full opportunity. A calendar end date that leaves the last group members only a few hours is not the same rule. Put the cohort definition and follow-up allowance into the project window or concise notes.

Make the comparison independent at the person level. A household sharing a planning decision can create related responses even if two contacts look separate in a database. If the design assigns households, businesses, or other clusters, this person-level calculator does not adjust for that grouping. Seek an analysis that matches the assignment unit. Do not turn a cluster experiment into an apparently larger person experiment by counting every contact separately.

Set a practical target before reviewing the totals. Cedar would consider a two-percentage-point improvement worth discussing if request quality and handling effort remain acceptable. That target is a business choice, not a significance setting in the analyzer. Keep the planned end fixed even if an early rate looks exciting. Repeatedly checking and stopping on a favorable p-value is a different monitoring design from the fixed-horizon comparison supported here.

Calculate the request rates from reconciled counts

In Cedar’s fictional completed comparison, group A contains 2,400 unique eligible people and group B contains 2,500. A has 244 raw submission rows: remove 16 repeat rows and 12 invalid requests to leave 216 successful people. B has 304 rows: remove 19 repeat rows and 10 invalid requests to leave 275. These subtraction categories do not overlap, and the records link each remaining success to a person in the correct eligible cohort.

Enter 2,400 people and 216 outcomes for A, then 2,500 people and 275 outcomes for B. The observed rates are 216 / 2,400 = 9% and 275 / 2,500 = 11%. Non-successes are 2,184 in A and 2,225 in B. Those counts include eligible people who never completed the form, not just people whose submitted requests failed validation. Unequal group sizes do not prevent calculating comparable person-based rates.

The absolute difference is B minus A: 11% – 9% = 2 percentage points. Relative lift uses A as the baseline, so it is 2 divided by 9, or approximately 22.22%. Saying conversion rose by 22.22 percentage points would be incorrect. The observed difference corresponds to two additional successful people per hundred eligible people at these rates; it is not a guaranteed future increase for every hundred visitors.

Raw totals alone would show 59 more successful people in B, but B also contains 100 more eligible people. Rates separate the observed outcome frequency from that volume difference. Cedar reports both counts and rates, names the valid-request rule, and labels the example as fictional. If A had zero successes, relative lift would be undefined rather than an infinite improvement; the tool can still show the rates and their percentage-point difference.

Read statistical uncertainty without declaring a guaranteed winner

Start in descriptive mode when the design is uncertain. Valid completed counts still give rates, an absolute difference, and relative lift where defined. Randomized mode adds inference only when all four design confirmations are true: random assignment, independent groups, a fixed stopping plan, and the same binary outcome with a common follow-up rule. Selecting the mode or checking the boxes cannot repair a design that did not meet those conditions.

The analyzer also requires at least ten successes and ten non-successes in each group before displaying inference. Cedar’s reconciled example clears that count guard. Clearing it does not prove good assignment, resolve duplicate people, or guarantee that a normal approximation is accurate in every setting. If a smaller campaign has nine successful people in one group, the tool withholds inferential results rather than offering a more impressive label for fragile data.

For eligible comparisons, the fixed method uses a two-sided pooled two-proportion z-test at the 5% level. It also shows 95% Wilson intervals for individual rates and a Newcombe interval for B minus A, without continuity correction. A p-value is not the probability that B is better or the chance that the result will repeat. The difference interval describes uncertainty under the stated design assumptions, not an assured range for next month’s campaign.

Read the interval alongside the two-point observed gap and Cedar’s practical target. A result that leaves room for a small or negative difference calls for a more cautious decision than the central estimate alone suggests. Near a threshold, the test and interval can disagree because they use different methods. Do not force agreement by rounding, swapping methods, or reading overlap between the individual rate intervals as the formal test of the difference.

Check appointment quality and workload as separate outcomes

More valid requests can still create a weaker business result. In a separate quality review, suppose Cedar records 54 people with completed appointments from A and 55 from B within a fully matured, predefined appointment window. Using the original eligible cohorts gives 54 divided by 2,400, or 2.25%, and 55 divided by 2,500, or 2.20%. The observed appointment-rate difference is minus 0.05 percentage points, even though B’s request rate was higher.

Among successful requesters, those appointment shares are 54 divided by 216, or 25%, and 55 divided by 275, or 20%. That is useful funnel context, but it compares selected subsets created after the page experience. It should not replace the original-cohort appointment comparison when assessing the overall outcome. Keep request completion and appointment completion in separate projects or records, each with its own definition and window. Do not add them together as successes.

Suppose the associated campaign costs are 1,800 currency units for A and 2,250 for B. Cost per valid request is approximately 8.33 and 8.18 units, respectively. These simple cost calculations are separate from the analyzer’s binary proportion method. Spending is not a count of eligible people, and revenue is not a success count. Record whether costs cover the same activities before treating the slightly lower cost per request as an operational advantage.

Discuss handling capacity as well. An extra request needs review, scheduling, and possibly clarification. If B attracts more people asking for an unavailable service, that learning belongs in the next offer revision, even when the requests technically satisfy the agreed rule. Do not rewrite the completed experiment’s primary definition to hide the problem. Preserve the original result, label the quality review separately, and decide what a future experiment should measure more directly.

Keep organic post comparisons descriptive and properly scoped

Two Facebook posts published on different days are not automatically A and B in a randomized test. Their audiences may differ, the same person may encounter both, and news, timing, comments, or distribution may change the response. Matching the image dimensions or campaign budget does not establish random assignment. A later post with more requests can suggest a useful direction while leaving the reason for the difference unresolved.

If your own records identify unique eligible people in two separate observational cohorts, you can describe their binary rates. For example, one period might have 160 valid requesters among 2,000 eligible people, while another has 132 among 1,200. The rates are 8% and 11%: a three-percentage-point gap and 37.5% relative lift against the first period. The later period produced fewer successful people despite the higher rate, showing why both volume and rate matter.

That calculation does not adjust for audience differences or establish that the later wording caused the gap. Keep the comparison descriptive and record the changed conditions. If the cohorts overlap or you cannot establish unique eligible people, do not force those totals into an independent person-based result. Use a narrative campaign review or a more suitable analysis. Native impressions, link clicks, reach, and total form events are not interchangeable substitutes for this tool’s required counts.

Separate different funnel stages in that review. An engaging conversation post may be good at attracting attention but poor at explaining eligibility for a consultation. A quieter explanatory post may send fewer but more suitable visitors. Use that pattern to choose a specific next question, such as whether clearer service-area wording improves valid requests among arriving prospects. The next controlled test can investigate that question without claiming that the organic comparison already answered it.

Leave a decision record that the next operator can use

Before sharing the result, write a short conclusion with the outcome, cohorts, observation window, rates, difference, and design limits. Cedar’s descriptive summary could say that valid requests were observed at 9% in A and 11% in B among the defined landing-page cohorts. Add the separate appointment finding and the chosen next action. Avoid a bare B won headline that conceals the qualification rule or suggests that future performance is settled.

Use the A/B test calculator to copy the review or export the prepared comparison. JSON preserves an editable tool project, while CSV keeps the supplied values in a data format. The PDF is a rasterized review, so retain CSV or JSON when selectable raw values matter. Save files deliberately in your own workflow; this local calculator does not provide account storage or autosave, and a native reporting file is not its project format.

Once the team has approved the next public explanation, the Facebook automation hub explains SMTasker’s supported publishing, likes, and configured comments. Those routines can help keep approved material and account activity organized with schedules, limits, and logs. They do not set up ad experiments, assign the landing-page cohorts, or collect these conversion outcomes. Keep the measurement plan and the publishing routine as separate responsibilities.

A useful next step is to configure one approved publishing routine through SMTasker setup, using an Android phone and the Windows or macOS client. Add the prepared post material yourself and inspect the first activity before expanding the schedule. Return to the free tools directory for other preparation work. An analyzer export is a decision handoff, not an automatic transfer into the app or an instruction to run a new test.

Questions about Facebook Conversion Tests

Continue with another comparison guide

Review your two prepared groups in the A/B Test Results Analyzer, or choose a focused guide below. More browser-based helpers are in the free tools directory.