A primary metric tells you whether the intended behaviour changed. A guardrail tells you whether the change caused unacceptable harm somewhere else. Both are necessary, but they do different jobs.
Start with the plausible failure mode
Do not copy a generic set of metrics into every test. Ask what this feature could realistically damage. A faster onboarding flow might increase activation while producing incomplete profiles. A new recommendation module might raise clicks while reducing add-to-cart quality. Name the failure mode first, then select a metric that can detect it.
Choose guardrails that can move in the test window
A twelve-month retention metric cannot protect a two-week experiment. Use an observable leading indicator and document the longer-term outcome it represents. This keeps the decision timely without pretending the proxy is the final business result.
Set thresholds in advance
Agree on the maximum acceptable deterioration before looking at results. A threshold should reflect commercial reality, customer risk and measurement noise. It should not be adjusted after an inconvenient result appears.
Keep the set small
Too many guardrails create contradictory signals and inflate the chance of a random alert. Prioritise severe, plausible harms. Monitor broader diagnostics, but reserve formal decision status for the few metrics that should genuinely stop a rollout.