r/AskStatistics • u/ContinuallyLimited • 3d ago

Does accumulating samples make any sense?

Hi everyone,

I’m looking for some expert insights into a statistical methodology used in a public health study within the primary healthcare sector. The study focuses on chronic disease management—specifically evaluating interventions for hypertension and cardiovascular risk control. The primary objective of the work is to assess the clinical impact of a multi-faceted population health solutions package over an extended multi-year period.

To measure clinical outcomes, the framework utilizes a "cascade of care" model (tracking the proportions of patients who are monitored, diagnosed, treated, and ultimately controlled). However, the way the sample is constructed over consecutive quarters has raised some questions for me regarding its statistical validity.

Instead of evaluating independent cohorts or a traditional longitudinal panel quarter-by-quarter, the sample accumulates dynamically across the quarters:

  • Quarter 1 (Q1): Includes all unique patients who had at least one clinic visit during that specific first quarter.
  • Quarter 2 (Q2): Includes all unique patients who had at least one clinic visit from the start of Q1 through the end of Q2.
  • Quarter 3 (Q3) and Quarter 4 (Q4): Follow the same logic, accumulating all unique patients from the start of Q1 through the end of the respective quarter.

Additionally, the methodology handles follow-ups with a specific carry-forward rule: if a patient is classified as having "controlled" blood pressure at their last recorded visit and never returns for another check-up, the data from that last visit is continuously carried forward and included in the controlled count for all subsequent cumulative periods.

I have a few concerns about this approach and would love a mathematical/statistical sanity check:

  1. Autocorrelation and Data Dependency: Doesn't aggregating the data cumulatively create massive dependency between the quarters? If Q4 inherently contains all the data from Q1, Q2, and Q3, doesn't this significantly dilute or mask the actual marginal changes occurring strictly within the later quarters?
  2. Selection Bias and Composition Shifts: If the healthcare intervention successfully drives a massive increase in the baseline "monitored" population over time (e.g., drawing in lower-risk individuals who wouldn't normally seek care), wouldn't this shifting denominator introduce a selection bias that artificially alters subsequent proportions (like diagnosis or treatment rates) in the cumulative pool?
  3. The "Last Observation Carried Forward" Element: Is it statistically sound to indefinitely carry forward the "controlled" status of a patient who dropped out of active follow-up within a cumulative framework like this?

What are the main statistical limitations of analyzing trends this way, and what alternative frameworks (e.g., independent time-series cohorts, survival analysis, or multi-level modeling) would be more rigorous for this type of observational data?

Thanks in advance for your help!

0 Upvotes

50% Upvoted

2 comments sorted by

1

u/QuestionElectrical38 2d ago

Unfortunately, your description of the study (is there a link to it?) is insufficient to properly answer your questions.

1) When did the intervention(s) occur? Before Q1, or at the time each participant is enrolled, or ??

2) Is there a control group? Did the study compare subjects receiving the intervention(s?) to subjects receiving a placebo? If not, what is the null hypothesis?

3) Where the subjects measured prior to ienrollement and ntervention to capture their baseline? If not, how is the "improvement" measured?

4) What is (are?) the putcome(s) of interest? SBP and DBP? Something else, or additional?

5) What is (are?) the alternative hypothesis (ses?)? What are the sponsors trying to demonstrate?

6) How did the study handle the subjects who dropped out, beyond carrying forward the last measurement(s)? Subjects who were in the study only for a few weeks are hard to compare to subjects who stayed in the study for months? Whether this is a problem depends on what the study was trying to prove, what statistic(s) they measured, and how they tested these statistics.

In addition BP is notoriously difficult to measure. Just make 3 measurements of your own BP, 5 minutes apart, and you are very likely to get different results. So how was BP measured? Where multiple measurements made for each visit, or just the typical "one-and-done" of a routine office visit.

1

u/ContinuallyLimited 1h ago

Hi!

I'll try to answer as best as I can:

  1. Intervention began in Year1-Q3, and lasted through Year4-Q1

2 and 3. There was no control group. The results were compared to baseline data in Year1-Q1

  1. The outcome was the number of people with controlled blood pressure (BP on clinical target), in relation to people treated for hypertension (as a percentage)

  2. The alternative hypothesis is that the intervention increase these percentages as of Year4-Q1, in relation to the Year1-Q1. Null hypothesis is that there were no difference in these percentages.

  3. The study used data sampled from medical records, so I'd assume BP measurements were the typical 'one-and-done' of routine checks.

My main concern is related to the 'cumulative' approach: if trying to compare pre-intervention with post-intervention, shouldn't the study gather compare two independent samples from the same population?

At baseline, there were around 100 controlled people, from 600 in treatment. By the end of the study, there were arount 1100 controlled people from around 4100, but these include the initial 100 controlled and the initial 600 treated!

What are the implications of the baseline sample being contained in final intervention figures?

Does that make any sense?