Check a PostHog experiment's sample size and metrics

PostHog keeps an experiment's hypothesis, variants, and metrics together, and its own sample size estimate is often the piece the plan is missing.

How it works

  1. Connect PostHog's MCP server at https://mcp.posthog.com/mcp to your AI assistant or coding agent, then sign in to the project holding your experiments.
  2. Install the ExpLint MCP at https://explint.com/mcp to enable your agent to audit tests.
  3. Run the prompt below to audit these plans before the work starts.

How the check runs

  1. It reads the experiment's hypothesis, flag, variants, metrics, and any sample size or run time note, then adds the PostHog estimate in plain words so the checker can judge whether the plan shows its working.
  2. It sends that text to ExpLint with explint__lint, which returns errors, warnings, and passes, each finding naming the rule behind it.
  3. You update PostHog and ask for a fresh read, and the assistant separates the findings that need a change in the flag from the ones that need a better sentence.

Copy and paste this prompt

Change the words in square brackets to name your own project, board, or test.

Prompt for your assistant
Use PostHog MCP tools to read the experiment [experiment name], including any sample size or run time estimate. Build one text from its hypothesis, variants and each change, primary, secondary and guardrail metrics, traffic split, and run time. Lint it with explint.com against [your ExpLint project name]. Show each error and warning with its PostHog field. Do not launch or edit it. Suggest fixes here, and re-check after I update PostHog.

What ExpLint looks for

ExpLint ships over 24 rules, and a project switches each one on or off or rewrites it in plain English. These are the rules the plans this guide reads break most often. An error means the test cannot answer its question as written, and a warning means the result will be hard to judge when it comes in.

RuleLevelIn plain words
hypothesis/is-falsifiableerrorA PostHog experiment that only promises a lift gives the team nothing a finished result could contradict.
metrics/has-one-primary-metricerrorThe experiment has to name the one metric that decides the result, since secondary and guardrail metrics are read against it rather than each other.
power/has-a-sample-size-calculationerrorPostHog can estimate the sample size, and the plan reads as complete only when that estimate appears in the text rather than staying in the interface.
power/states-the-minimum-detectable-effecterrorThe plan should name the smallest lift worth acting on, because the sample size rests on it and a reader cannot check one without the other.

Read more

Portrait of Kameron Tanseli

Kameron Tanseli

Founder, ExpLint

Kameron Tanseli is the founder of ExpLint and a growth engineer who has run more than 1,500 experiments across a nine-year career. As the founding growth hire at Fyxer he launched 540 experiments in a year while the company grew from $1M to $35M in annual revenue, and he now runs growth engineering at FLORA, where a one-person team ships six to eight experiments a week.

Don’t let a bad test make it to production

Install the MCP and start flagging ticket slop before it reaches your users