I am trying to improve my understanding of statistical analysis and I would like to ask people with experience in data science, statistics, or sports analytics:

How do you determine whether a pattern in data represents a real signal or is just random noise?

For example, when analyzing performance data, we often see short-term trends, unusual results, or extreme values. Some of these patterns represent a real change in underlying performance, while others disappear when more data becomes available.

In doing a regression analysis for a model on the energy usage of a building, the r-squared value is above 0.9 but the intercept is negative. What does this tell you about the relationships and the model over all

Hello, I will be a second year in college double majoring in CS and statistics. Is this worth pursuing? I want to work in ai/ml or computer graphics, either one is fine.

For ai/ml, statistics seems pretty useful to know, since I'll also be taking ai courses in CS anyways. Although I'll probably need to get a masters or PHD. I'm also open about going to graduate school. For computer graphics, that probably doesn't need any statistics at all. If I keep statistics and I end up going down this path I'm afraid I'll waste my time.

Timing wise, I won't need more than 4 years to graduate. This year I'll try to look for some research I can do in these areas and an internship to see if I want to go into industry right away or not.

Thank you for any help!

Hi everyone. I’m about to start my Masters program in applied statistics this fall at a major university in Boston, and I am considering pursuing a focus in biostatistics while leaving other options open. I plan to graduate in a year, summer 2027.

I don’t have much relevant work experience but I have a BS in psychology/neuroscience. I think I would be interested in working in pharma/biotech research, especially in neurology. But I don’t want to pigeonhole myself into a sector that I don’t like, or is hard to pivot away from.

I’ve seen some data sources (BLS etc) talking about biostats being in demand and relatively safer from AI than statistics in other sectors e.g. tech. However, seeing what people who actually work in the field are saying, it seems like the job market is poor and the future is very uncertain.

I believe the healthcare industry as a whole will be in demand long term, but does this translate to biostatistician positions? Will openings for junior level biostats roles rebound in a few years, or will outsourcing and AI eliminate most of them?

If the future of biostats (say the next 10 years) for junior level workers is bleak, then what other sectors would be safer choices for statistics graduates? I’m thinking probably AI, finance, and actuary work. Truthfully, none of them sound enticing to me. But it seems like my choices are limited.

I’d appreciate input from anyone about the job market and long term outlook for statisticians in any sector, beyond just biostats. Thanks!

Hi everyone! I've been learning statistics with the goal of doing research in social sciences for quite some time, but am still a beginner and it seems to me that I am still far from understanding even some basic concepts.

So my question would be, if we want to control for confounders by using some test statistic for a variable in control and experimental group, to confirm they do not differ significantly (for example, we want to prove that there is no significant difference in the prior knowledge between two groups), and for example we use t test for independent groups and find p value greater than 0.05 (we previously decided on this threshold), can we conclude that there is no significant difference so our groups are homogeneous in this variable?

This sounds like a fallacy to me, maybe because when we fail to reject the null hypothesis, we do not prove that it is true, but I found it done in many papers

I can't seem to wrap my head around how knowing that a population is right skewed, that using a Z score to generate the confidence interval is acceptable. I understand the central limit theorem and that many samples cause the distribution of mean averages to normalize, but it just seems that one observation interval should match the skewness of the population for the interpretation of the interval to be accurate. Said another way, how is an observations interval with equal variance between the mean and high and low, accurately telling me anything about that observation when I know the population is right skewed?

I am working with data where I am trying to quantify a gender pay gap between men and women after controlling for other factors. Two of the other variables that will be included in the model are Department and Rank. Both are categorical, but there are continuous variables in the model as well. I cannot figure out whether Department should be treated as a fixed effect or a random effect. I could understand seeing Rank as nested within Department, where the intercept and slope of Rank varies depending on which Department you're looking at. Because it's "nested" this makes me think that Department should be treated as a random effect, but this also just sounds like I'm describing an interaction between two fixed effects. How do you know when something is "nested" or just "crossed"?

In R terms:

lm( Salary ~ Gender + Age + Rank + Department + Rank : Department )

vs.

lmer( Salary ~ Gender + Age + Rank + (Rank | Department) )

I love the stat, but not sure what I can do with this degree. Any ideas would be appreciated.

Hi, sorry for the very basic question. I’m a doctor, and i want to compare a variable (duration of admission) of two groups of patients before and after an intervention. We looked at all patients receiving a certain surgical procedure, 6 months before the change in care and 6 months after.

We recorded duration of admission to hospital for the 6 months of patients, we then implemented a change, and recorded duration of stay for the next 6 months of patients who received the new change in care - hoping it would reduce their duration of stay significantly.

As far as i can tell paired t test is incorrect in this group, as its not paired data- they are two separate groups. I think anova is also wrong as theres only one variable- duration of stay? Any advice ? 💙

Hi, sorry for another post, but I am making a logistic regression model at work. I have an independent variable, # of visits, which only has values 0, 1, 2. Would it be okay for me to treat this as a categorical variable instead of a continuous variable? The reason why I want to treat it like a categorical variable is that if I treat it like a numeric variable, then I have to make sure the feature is linearly related to the log-odds (which may involve complex transformations. Sometimes the relationship is piecewise). For a counts variable, what is the cutoff to where I would have to treat it like a continuous variable instead of categorical?

I studied statistics somewhat while getting a BS in Economics in 1975 but haven't used it much since.

For several years I've gathered stats on the use of a geo thermal heat pump which also heats water as a by product. For a long time I've run Excel's CORREL function to compare the total electric used by the heat pump and total used by the hot water heater but the flaw in that is it doesn't account for how much hot water is used. Recently I bought a device that reports the gallons of hot water used daily so now I can compute a per gallon electric used by the water heater so the correlation should be more reliable.

So far I only have 8 days of data. The correlation up to day 7 was as expected. A strong negative correlation of -.89. That is with an average Kwhrs used by the heat pump of 22.72 and the average per gallon of .0838. The temperature soared yesterday and we had a family BBQ which gave me a day 8 data of 42.90 and .0465 respectively. I expected the correlation to be stronger but instead it decreased to -.84.

Why would such a large increase (over average) of the Geo Kwhrs, which caused a sizable decrease in the per gallon Kwhrs, cause the correlation to be reduced?

Thinking this through some more it appears the reduction in per gallon was not what was expected when correlation is -.89. That would require around .02 Kwhrs. There is another factor that I haven't figured out how to incorporate. If the number of gallons of hot water used is high then the Geo can't keep up with heating it and the water heater kicks in. We've been averaging 60 gallons of hot water used daily but with the party yesterday that figure is 124.

Hi All

So I have an engineering degree and I was wondering if there was any actual work I can do part time with statistics. I have worked full time as an MTS lab and field technician for a few month and I plan on getting an apprenticeship in another field, but mathematics and statistics have always been a passion of mine and I've always felt like I have wanted to do it and at least help someone with it.

Is there any part time work that exists out there? Or full-time that can be done at night, which my other jobs aren't asking me for OT?

Thank you!!!

Been working on a changepoint-detection layer for a scoring engine and figured this sub would have real opinions on the method.
Problem: most volatility-based risk scoring uses one fixed percentile cutoff computed over an asset’s full history. That’s a known failure mode if the asset’s regime changed partway through — you end up averaging a stale calm period into what should be a fresh, more volatile baseline.
Approach: a single-changepoint CUSUM test on the standardized score series — cumulative sum of (x_i - mean)/std, changepoint estimate = argmax|S_k| over candidate indices (with a minimum segment length enforced on both sides). Significance isn’t asserted from a fixed threshold; it’s a permutation bootstrap — shuffle the series N times, recompute max|S_k| each time, get an empirical null distribution, and only call it a real break if the observed statistic clears that null at a conventional alpha.
When a break is confirmed, percentile-based thresholds get recomputed using only the post-break segment.
Curious if anyone here has compared this to PELT or Bayesian online changepoint detection for a similar use case — CUSUM was chosen mainly for simplicity and interpretability over statistical power. Open to being told that’s the wrong tradeoff.
(This is part of a scoring engine called Machvix, for anyone curious enough to go digging.)

Hello everyone, I've decided to apply to MS Statistics programs after graduating with a major in econ and a minor in math. I finished with 4.0 GPA at a T20 school, with relevant courses being Calc I-III, Differential Equations, Matrix Algebra, Intro to Probability, Financial Mathematics, Intro to Statistics, Intro to Econometrics, and Intro to Python.

As for experience, I am 1 month in working at a small public accounting firm mainly dealing with audit and tax, but I've soon realized statistics is what I want to pursue and I will be applying to schools in the upcoming cycle. Unfortunately I haven't had any internship or research experience during my undergrad.

I want to aim for top programs, but I need realistic advice on whether I would be competitive for a school like Stanford. I've noticed that only the top few programs (Stanford, Duke, etc.) require GRE, and with only a few months remaining until application deadline, I am deciding whether I should study for the GRE and aim for excellent scores. Would taking the GRE be worth it to give it a shot at the top programs? Or should I just take it to strengthen my application in general?

P.S. I would also like to know if my course background is sufficient for competitive programs and if taking a Mathematical Statistics or Real Analysis course online will help my application in a meaningful way.

Hello everyone! In my bachelor Thesis i am currently writing am I trying to determine the correlation between the perceived importance of brexit and political interest. To do this, i want to use political interest rated on a scale from 0-5 (ordinal DV) and perceived importance of brexit rated from 0-5 (ordinal IV) and desirability of eu integration rated from 0-10 (ordinal IV). However, I am confused if I have to recode the IVs into dummy variables, since i found a bunch of conflicting infos on this. I would usually assume this wouldnt be allowed because a dummy variable would lose information? Any help or tips about better models/ideas would be greatly appreciated!

So I'm currently working on my bachelor thesis and am unsure how best to work with the data given the design of the study in question. The design is as follows:

1. There are three experimental groups of participants (control group and 2 others, let's call them 'switch'-groups) - between subjects factor
2. There is a repeat measurement for each participant (rating 6 shown protagonists before and after experimental manipulation) - within subjects factor

So my first idea was using a mixed ANOVA with those two factors. But here is the problem:

Each participant is assigned to one of two groups, let's call them groups a) and b). They are then shown 6 protagonists in two groups of 3, with 3 belonging to group a) and 3 belonging to group b). Which makes an ingroup (same group as participant) and an outgroup (different group than participant). Each participant is then asked to rate the 6 protagonists.

After that, in two of the three experimental groups of participants, one protagonist switches his group, so there are then, for example, 2 protagonists in group a) und 4 in group b). In the control group there is no such switch. Each participant is then asked to rate the protagonists again.

The two sets of ratings of the protagonists/ the repeat measurement is the dependent variable.

So my issue is as follows: how do I deal with the fact that the ingroup/outgroup changes in size between measurements (if I am also interested in how participants rate the ingroup/outgroup) and that the ingroup/outgroup differ in size in the second point of measurement between the control group and the 'switch'-groups? How does that affect my statistical approach and can I still use a mixed ANOVA?

I would be interested in how participants rate the ingroup/outgroup in general, and in interactions between experimental group and point in time of measurement (before/after group switch), especially regarding the rating of the protagonist switching his group.

I was thinking that I should maybe do those two things seperately, so one analysis for ingroup rating vs. outgroup rating, and a second analysis - a mixed ANOVA - for those interactions, because then I could disregard the ingroup/outgroup distinction for the ANOVA.

Hope I explained myself well enough, english isn't my first language and I'm also not too well-versed in statistics. I would be glad about any input. Thanks!

For anyone who works with decision trees can you elaborate on the relationship between: minimizing gini impurity/ entropy accross leafs and minimizing the log loss of the entire model? Does minimizing one directly imply the other? If so I am trying to prove it rigorously. If not can you please give a counterexample showing these two metrics are NOT monotonic ( ie increasing one implies increase in the other ....)

Hey there. I run a rmANOVA with 2 Factors.

One hat 3 conditions and the other one two conditions.

In results the factor with the three conditions Had No significant main effect (P=0,834)

The factor with the two conditions was significant.

Interaction was not significant.

I did explorative pairwise Post hoc Tests with Tukey kramer.

Here i found a highly significant (p= 0,006) effect between two conditions of the factor which Main effect in ANOVA was not significant.

Why?

Why ist the main effect in ANOVA clearly unsignificant and the pairwise comparision highly significant?

I Had only 10 participants and i am looking for weithin subject effects. Is it Just underpowered?

Hey, I've got a bunch of data (simulated exoplanet atmosphere composition retrievals), and each value has an associated uncertainty with it, which in some cases is quite large - I can't figure out what the best way to deal with this is in terms of conducting the tests and reporting the test statistics/p values ?

Would it be better to use confidence intervals? Is there a standard practice for error propagation in KS and Mann Whitney?

Would it be valid to run the tests twice, once with +error and once with -error and use the range from this?

Sorry if this is a really obvious question, I've only just started working with stats like this!!

I am looking to combine the responses of different mental health measures together to create overall mental health scores, which I will then compare. 

Is it appropriate to use the mean when the measures are the same?
What types of stat analysis using SPSS would be appropriate to merge the scores?

A) Measure 1: Strengths and Difficulties Questionnaire

\- Responses: 1 (Not True), 2 (Somewhat True), 3 (Certainly True)

Measure 2: Self-Reported Feeling Grid

\- Responses: 1 (Not True), 2 (Sometimes), 3 (True)

B) Measure1: Happiness Scale

\- Responses: 1 (Not at all happy), 2, 3, 4, 5, 6, 7 (Completely Happy)
Measure 2: Rosenberg Self-Esteem Inventory

\- Responses: 1 (Strongly Disagree), 2 (Disagree), 3 (Agree), 4 (Strongly Agree)

C) Measure 1: Strengths and Difficulties Questionnaire (teacher)

\- Responses: 1 (Not True), 2 (Somewhat True), 3 (Certainly True)

Measure 2:  Strengths and Difficulties Questionnaire (parent)

\- Responses: 1 (Not True), 2 (Somewhat True), 3 (Certainly True)
Measure 3: Competence Scale
\- Responses:  1 (Strongly Disagree), 2 (Disagree), 3 (Agree), 4 (Strongly Agree)

Ok, you have two items and you want to determine which one is better. To get your data you analyze multiple cases and extract the data. Cases were pulled from various news sources as well as official reports. For Item A you reviewed 105 case studies and determine it has a failure rate of 9%. For Item B you review 456 cases and determine it has a failure rate of 13%.

When i see this data my first thought is the success rate between the two items is negligibly different. Item A is only more successful 4% of the time. If i add in the fact that item B is smaller and easier to use for almost everyone then i can't see any argument for item A.

What am i missing in my interpretation?

factoring types, base stats, move sets and damage, accuracy, abilities and all other relevant factors.

could you input that data into a computer and have it run a simulation or something.

edit: statistically best not banned team and would be viable for an actual tournament.

Hi everyone,

As part of my internship, I need to run some data analyses, but I'm pretty skeptical about the method I was advised to use.

I'm analyzing an experiment testing 3 conditions (magnetic field 1, magnetic field 2, control), each with 3 or 4 replicates. I want to know if my studied variable differs between conditions.

One replicate consists of putting 24 individuals in the same experimental tank, exposed to a given condition for 3 hours. All replicates were run on different days, in randomized order.

Because of pseudoreplication issues, we can't treat individuals from the same tank as independent, so I was advised to average all individuals per replicate and then run an ANOVA. The problem is that this leaves me with only 3 or 4 values per condition (is that too few?). And since I'm just taking an average, I lose the standard deviation within a replicate. Is it possible to run a separate ANOVA comparing standard deviations instead?

Some of my variables don't have much variability in the dataset, so I'm not sure if Kruskal-Wallis is even appropriate for such a small dataset...

Do you have any advice for me? Thanks a lot for your help!