How it works
- When a campaign starts, Horizon sets aside a random share of the matching accounts as the holdout. You choose the share, up to 20%.
- Their CSMs keep working them as they do today. They just don't get Horizon outreach.
- Every campaign has a goal with a time window, for example "renewal call booked within 30 days".
- Horizon reports the share that met the goal with outreach and in the holdout, and the difference between them.
What the report shows
- Goal attainment with outreach and in the holdout, and the lift between them. When there are too few accounts to conclude anything, it says so.
- Replies from people (auto-replies are left out), meetings, and clicks. Opens are shown as a rough guide only, because privacy features such as Apple Mail open emails automatically.
- The funnel from drafted to sent to replied to goal met, dismissal reasons, how much CSMs edited the drafts, and time to review.
- Breakdowns by goal, campaign, automation and sender.
Questions
What is a holdout group?
A small random set of at-risk accounts that Horizon leaves on your normal process when a campaign starts. Comparing their results with the accounts that got outreach shows what the outreach changed.
How big is the holdout?
You choose the share when the campaign starts, up to 20%. When there are too few accounts to conclude anything, the report says so.
Do holdout accounts get ignored?
No. Their CSMs keep working them as they do today. They just don't get Horizon outreach for that campaign.
What counts as a result?
Every campaign has a goal with a time window, for example "renewal call booked within 30 days". The report shows the share that met the goal with outreach and in the holdout, and the lift between them, with breakdowns by goal, campaign, automation and sender.
Why are email opens shown as a rough guide only?
Privacy features such as Apple Mail open emails automatically, so opens overcount. Horizon counts replies from people (auto-replies are left out), meetings and clicks instead.
Is holdout measurement on the Free plan?
No. It comes with Professional and Enterprise. To see the math before you start, try the retention lift calculator.
More on this topic: renewals, definitions, guides and tools.
Updated November 3, 2026