OL-04
False Improvement Pattern
Outliers
Default severity: medium
MetricsOutputQualityRisk
What it detects
Rolling average drops significantly below the configured dip threshold then rebounds above the configured recovery threshold within the validation window — indicates temporary batch release, not genuine process change.
Detection formula
IF rolling_avg \< overall_avg * config.outliers.false_dip_threshold // default: 0.7 THEN WATCH for config.outliers.validation_window_days // default: 30d IF rolling_avg recovers > overall_avg * config.outliers.false_recovery_threshold // default: 0.85 THEN RECLASSIFY as false_improvementExamples in practice
- Individual items or windows fall outside statistically expected cycle time bounds.
- Outlier density increases compared with the prior analysis window.
- A team shows rolling average drops significantly below the configured dip threshold then rebounds above the configured recovery threshold within the validation window while false dip threshold is set to 0.7.
Suggested response
Validate whether outliers reflect real incidents or data and process issues before changing targets.
Coaching playbook
Symptom
Rolling average drops significantly below the configured dip threshold then rebounds above the configured recovery threshold within the validation window — indicates temporary batch release, not genuine process change.
Why it matters
When "False Improvement Pattern" keeps appearing, the team is signalling a repeatable process gap. Left unexamined, the pattern hides where work really stalls and makes improvement metrics harder to trust.
What you can achieve
Validate whether outliers reflect real incidents or data and process issues before changing targets.
Facilitation questions
- What system change would stop "False Improvement Pattern" from firing again?
- What do the cited issues have in common — same root cause or same workaround?
- Who owns the two-week experiment and how will we verify on the next import?
Run this rule against your own tracker data with Flow Analyzer.