---
title: "Holdout group: how to measure customer success impact"
url: https://renewalfix.com/glossary/holdout-group/
description: "What a holdout (control) group is, how Customer Success teams use one to prove that outreach saved renewals, and how to keep it safe."
updated: 2026-11-03
---

Glossary

# Holdout group

A holdout group (or control group) is a random set of accounts deliberately left out of an intervention, so you can measure what the intervention changed by comparing outcomes.

## Why use one

Without a comparison, a saved renewal could have renewed anyway. A holdout gives you the comparison: the same kind of accounts, at the same time, without the outreach.

## Worked example

40 at-risk accounts get a playbook or outreach and 31 renew (77.5%). 16 similar accounts in the holdout stay on the normal process and 9 renew (56.3%). The difference, about 21 points, is the estimated effect of the outreach. (Example numbers. With groups this small, the result could still be chance.)

## Doing it safely

- Keep the holdout small and random.
- Holdout accounts still get your team's normal attention; they just don't get the new outreach.
- Say when the sample is too small to conclude anything.

See [holdout measurement in Horizon](https://renewalfix.com/product/holdout-measurement/).

More on this topic: [renewals, definitions, guides and tools](https://renewalfix.com/topics/renewals/).

---

Horizon is renewal risk software for Customer Success teams. Book a call: https://calendar.chilipiper.com/book/me/serban-goanta
