The ICP You Can Prove
Works with
Implementation kit
- The fit definition, with four verdicts and two profiles
- The blind labelling guide for the golden set
- The verdict prompt, with the output check
- The agreement and stability check, with the run record
- The monthly outcomes readout and the widen or pull back rule
- Ten sample companies with the answer key
Most fit models get one test, a backtest on deals already won. This playbook tests the verdict your routing runs on today, gives undecidable accounts a queue of their own, and reads outcomes by tier to decide when to widen.
The full method is below. One verdict field, two checks before it routes, and one monthly readout after.
A fit score that was only ever tested on history routes every new account as if it were proven.
The failure this playbook fixes
How it works
01Write the ICP as one verdict field
Routing reads one field per account, so write your customer definition as that field. It takes four values. Fit means every checkable rule passes with a source. Marginal means near the rules. No_fit means a rule plainly fails.
Unknown covers a missing source, two sources that disagree, or a verdict that flipped between runs.
Each rule names the page that checks it, so a stranger applying it gets your answer. Fit and intent stay in separate fields, because mixing them makes both untestable.
Watch for the missing fourth value. Without unknown, an account with thin data gets forced into a tier, and the guess routes like a fact.
02Label a golden set blind
Pick twenty to thirty accounts from the same sources your routing draws on. Mix in clear no_fits and the hard cases on purpose, like a subsidiary on its parent's domain or a company at the edge of your size band.
A person labels each one against the written definition before seeing any agent output. Unknown is a valid label. A blank cell is an error, because a skipped account quietly shrinks the number the agreement is measured against.
03An agent returns one sourced verdict
Only the company name and domain go in. The agent finds each rule's value on the web, with the page and its date, and returns a fixed JSON shape.
A short check runs before your CRM reads anything. A fit missing a source drops to unknown, and so does evidence older than your cutoff.
If you score two profiles, run two separate calls. One call that judges both lets the first verdict pull the second.
04Check agreement against the labels
Score the agent on the golden set and read every disagreement by hand. Fix the prompt, the rule, or the label, and write down which. When a run looks right, accept it. Its agreement becomes the baseline, and the threshold sits about 0.15 under it.
A run that scores fewer accounts than the minimum is unreadable, and unreadable fails. A small run that happens to agree proves nothing.
Watch for a threshold set at the baseline. It fails on ordinary run-to-run noise and gets ignored within a month.
05Check that the verdict holds still
A backtest runs once, so it never sees a verdict change. Run the golden set three times, then once on a second model.
A verdict that flips goes to unknown with the reason "unstable", even when most runs agreed. Majority voting hides the accounts your rules do not cover.
The flipped accounts usually share one rule. Rewrite that rule, then run the set again. A failed check holds the routing until a person signs off.
06Route by tier, with unknown as its own queue
Code maps verdicts to routing, and the model never picks a channel. Fit goes to a person. Marginal goes to an automated sequence. No_fit gets nothing, with its reason kept. Unknown goes to its own queue with a named owner.
Every verdict is appended to a dated log. The readout in the next step needs the verdict that was in force, and a re-score overwrites the current one.
07Read outcomes by the tier that routed them
Each month, credit every meeting, opportunity, win and loss to the verdict in force before the month opened. Under ten accounts, the cell prints a refusal in place of a rate.
A 0% from a month nobody entered looks exactly like a month where nothing happened.
Write the widen, hold, and pull-back rule before the first readout. Tier 2 converting like Tier 1 for two months means widen. Tier 1 converting no better than Tier 2 sends the definition back to step 01.
If you hold a declared profile and an evidenced one, score both until the readout picks.
Test the verdict your routing runs on, every time it changes.
The implementation kit above packages the definition, the verdict prompt, both checks, the readout, and the answer key.