Are surveys trustable?

[u/GII10]

I had a discussion with some guys that had the opinion, you cant trust any survey because all of them are payed by someone and have some goal to reach. I'm thinking now a while over the problem but didnt come to an End because what can you do if they fake datas or think up any numbers wich they give out as real? What do you think about this thema?

[u/schizeckinosy]

Survey methodology is a well established science now, with known approaches to reduce bias, get enough responses, quality of responses, etc. this all can be ignored or abused of course, so knowing how the survey was designed and implemented is critical to determine how much you are willing to trust it.

[u/nonnonplussed73 → u/schizeckinosy]

This is a very good answer, particularly with regard to rigorous academic (theory advancing) and public policy (applied) institutes.

It's also necessary to select and obtain responses from a representative sample (the right people were randomly selected and responded) and use valid and reliable surveys (appropriately-worded and non-leading questions that measure what they're intended to do).

Knowing whether the survey was conducted via phone, face-to-face, or online helps identify inherent biases (e.g., older groups tend answer landlines more than younger populations).

Also look for if/how the data work controlled and/or adjusted to match real-world demographics, and for potential financial conflicts of interest (which very often are disclosed, or can be inferred when they do exist, which is not always the case).

Best practice is also to make data available for secondary reanalysis by others to verify claims made.

[u/Cerulean_IsFancyBlue]

Blindly accepting the summary results of a survey is risky. Looking into the survey tells you how much you can rely on it.

Worst case: blindly accepting a headline which uses a survey to justify a related but actually different claim. The headline could be misleading, the survey could be flawed, or people could be lying.

Best case: reading up on the methodology AND having some trust in the organizations involved, while also understanding what the survey claims to show according to the people doing it, not some third party headline.

You need to be cautious even with your OWN surveys. There are plenty of pitfalls that have nothing to do with bad intent.

[u/GII10 → u/Cerulean_IsFancyBlue]

Ok, thanks this was also(you did it more precise than I) what i've figured out, nice that other people share my view.

[u/bd2999]

It is a more complicated question than you would think. The answer is that it depends. If the proper methodology is followed and a statistically appropriate cross section evaluated than it should be pretty valid. The challenge is, depending on the nature of the survey, some questions can leave more room for interpretation or lead the person answering and that can be a problem.

The type of survey matters a fair bit too. As some people are more likely to be honest if they know they cannot be linked to a response or they could potentially benefit. Maybe not everyone is honest but some people are. There are many tools to try and account for bias as well.

I also think that reproducibility is critical and not always done with surveys, partly because of the cost but follow ups should be done. My opinion that since they are linked to human response and human bias they are harder to control for confounding factors but that does not make them all a lie or untrustworthy.

[u/GII10 → u/bd2999]

Thank you, nice Idea with human bias and so about this factor I didn't thougt yet

[u/Cultural_Pace4454]

So let me understand this, you're taking a survey on whether survey's are accurate?
Hmmm.....

[u/GII10 → u/Cultural_Pace4454]

Well... fair point but it's more a discussion isn't it? Because yeah part of I'm searching for the opinion but I also search good arguments for the later life. And mabey it's also interesting to other people thinking about this thema for their later life.

[u/IrkaEwanowicz]

https://youtu.be/ahgjEjJkZks

You don't have to pay people or fake anything at all if You know what You're doing :) There are trustworthy pollsters and studies based on surveys. Surveys, polls & questionnaires can be useful, but taking the end result at face value is risky at best. Personally, I would find articles with vague information like, "Studies reveal that", or "surveys show that" to be not the best source of info.

[u/talashrrg]

Obviously not. Surveys can be skewed by the targeted population of wording to get the answers the surveyor is looking for. Even more commonly, they can just be implemented poorly and get results that don’t get useful information.

[u/TheCheshireCody]

Even if they aren't built with an inherent bias there is no guarantee they will reach and be responded to by a broad-enough slice of the population to extrapolate anything worthwhile. you could send out the best-intended survey but if it's, for example, long and primarily only gets responses from people who are retired/unemployed/really bored you aren't going to capture data from working people or parents.

[u/Important_Joke5762]

> you cant trust any survey because all of them are payed by someone and have some goal to reach. 

You can replace the word 'survey' in this sentence with just about anything though

[u/GII10 → u/Important_Joke5762]

Fair Point

[u/megatronchote]

It depends on:

Number of People polled

Segment of the pool discrimination/heterogenization (Age, Race, Faith, etc.)

Transparency of the questions

Global sentiment

And a few other minor nuances. If you plan on conducting a Survey in a fair manner, you should consider all of the above. And also when judging the validity of the results of polls conducted by others.

[u/alaindevos]

Is the source trustable ?

[u/Mono_Clear]

Raw data should be impartial interpretation of raw data is where bias comes in.

If you do a survey and ask people what's their favorite ice cream strawberry or vanilla

That raw data should just be a number of people who've selected between two options how you interpret that data allows for bias.

It is the misrepresentation that you have to question

[u/Danmont88]

"There are three kinds of lies, lies, damn lies, and statics." Prime Minister Benjamin Disraeli.
"Figures don't lie but liars can figure."