Product · Holdout measurement

Show your CFO what your outreach changed.

A holdout group (or control group) is a small random set of at-risk accounts that Horizon leaves on your normal process, so you can compare their results with the accounts that got outreach. The difference between the two groups is what your outreach changed.

How it works

  1. When a campaign starts, Horizon sets aside a random share of the matching accounts as the holdout. You choose the share, up to 20%.
  2. Their CSMs keep working them as they do today. They just don't get Horizon outreach.
  3. Every campaign has a goal with a time window, for example "renewal call booked within 30 days".
  4. Horizon reports the share that met the goal with outreach and in the holdout, and the difference between them.

What the report shows

Is a holdout risky? It's a small random slice, and those accounts still get your team's normal attention. That's what makes the result credible to finance.

Questions

What is a holdout group?

A small random set of at-risk accounts that Horizon leaves on your normal process when a campaign starts. Comparing their results with the accounts that got outreach shows what the outreach changed.

How big is the holdout?

You choose the share when the campaign starts, up to 20%. When there are too few accounts to conclude anything, the report says so.

Do holdout accounts get ignored?

No. Their CSMs keep working them as they do today. They just don't get Horizon outreach for that campaign.

What counts as a result?

Every campaign has a goal with a time window, for example "renewal call booked within 30 days". The report shows the share that met the goal with outreach and in the holdout, and the lift between them, with breakdowns by goal, campaign, automation and sender.

Why are email opens shown as a rough guide only?

Privacy features such as Apple Mail open emails automatically, so opens overcount. Horizon counts replies from people (auto-replies are left out), meetings and clicks instead.

Is holdout measurement on the Free plan?

No. It comes with Professional and Enterprise. To see the math before you start, try the retention lift calculator.

Updated November 3, 2026

See which of your renewals are at risk.

A 30-minute call on a demo book, or on your own top 25 accounts.

Book a call