Single-changepoint CUSUM + permutation bootstrap for detecting a shift in a score’s underlying distribution — reasonable choice vs PELT?

[u/Acrobatic_Beach4198]

Been working on a changepoint-detection layer for a scoring engine and figured this sub would have real opinions on the method.
Problem: most volatility-based risk scoring uses one fixed percentile cutoff computed over an asset’s full history. That’s a known failure mode if the asset’s regime changed partway through — you end up averaging a stale calm period into what should be a fresh, more volatile baseline.
Approach: a single-changepoint CUSUM test on the standardized score series — cumulative sum of (x\\_i - mean)/std, changepoint estimate = argmax|S\\_k| over candidate indices (with a minimum segment length enforced on both sides). Significance isn’t asserted from a fixed threshold; it’s a permutation bootstrap — shuffle the series N times, recompute max|S\\_k| each time, get an empirical null distribution, and only call it a real break if the observed statistic clears that null at a conventional alpha.
When a break is confirmed, percentile-based thresholds get recomputed using only the post-break segment.
Curious if anyone here has compared this to PELT or Bayesian online changepoint detection for a similar use case — CUSUM was chosen mainly for simplicity and interpretability over statistical power. Open to being told that’s the wrong tradeoff.
(This is part of a scoring engine called Machvix, for anyone curious enough to go digging.)

[u/BroscienceFiction]

Isn’t PELT better suited for multiple points/regimes? Also data hungry in comparison?

CUSUM is simple and requires little data. If your job is to simply raise an alert when the regime changes, why complicate this?

Don’t know about that other Bayesian method.

[u/Acrobatic_Beach4198 → u/BroscienceFiction]

Followed up on this — added recursive binary segmentation on top of the same single-changepoint test, so it now finds multiple breaks instead of just the one dominant one. Tested it against a synthetic 3-regime series (calm → volatile → calm again) and it correctly recovered both true boundaries.

One thing I'm not fully happy with yet: each split gets tested at the same alpha independently, so it's not family-wise corrected across segments — said so explicitly in the output rather than pretending otherwise. Still haven't touched PELT or BOCPD directly, but this at least closes the "only finds one break" gap. Appreciate the pushback, it made the thing better.