Seven Reasons to Stop Making & Taking Surveys | SAFETY DOC PODCAST 68 with Dr. David Perrodin

PODCAST-“Sometimes the questions are complicated and the answers are simple” (Dr. Seuss). Dr. Perrodin centers this episode to the seven reasons he loathes surveys and lays out a vastly better alternative for gathering relevant input to inform decision making. 

DIRECT LINK to MP3 of this Episode:  https://tinyurl.com/SDP68

WHAT IS A SURVEY – THE NUTS AND BOLTS

Fundamentally, a survey is a method of gathering information from a sample of people, traditionally with the intention of generalizing the results to a larger population. Surveys provide a critical source of data and insights for nearly everyone engaged in the information economy, from businesses and the media to government and academics. There are several steps to creating surveys, but the pillars of any surveys are the constructs. In the context of a survey, a construct is the abstract idea or underlying theme. Constructs can be direct or indirect. Direct is easily and reliably measured, such as height and weight. These also will have a very high level of inter-rater reliability. On the other hand, indirect constructs, such as happiness, frustration and satisfaction are more challenging to assess and splattered with issues of inter-rater reliability as happiness will hold a meaning unique to the individual.

THE 7 FLAWS OF SURVEYS

1. Surveys aggregate, or combine, data from many survey takers. For example, one might deduct from collective findings of a youth risk survey of 300 high school students that “15% of youth surveyed had suicidal thoughts at least twice during the past year”. At first glance, this appears like sound, useful information. It isn’t. In fact, it’s largely useless. Why? Who are the 15% of students that have considered suicide? How does one exactly recall at least 2 events within the past 365 days. We already know the forgetting curve wreaks havoc in the courtroom.
2. Response rates – how many surveys do you receive and actually complete? People dislike filling our long forms and response rates for emailed surveys might hover at 2%. Almost everyone will ignore a survey.
3. Positionality of person(s) writing the questions. Positionality is the practice of a researcher delineating his or her own position in relation to the study, with the implication that this position may influence aspect of the study, such as the data collected or the way in which it is interpreted. The survey will mirror the mind of its author – and that often can lead to bias. I mean, how many human resources want to craft a survey that will reveal the true level of disdain simmering within employees?
4. Survey takers are not able to ask seek clarification of the questions. Surveys are one-way. Good luck!
5. It’s not easy to ask “good” questions. Formulating constructs takes thought and operational definitions. The most effective surveys have the fewest questions, like 20, and they must pepper construct-aligned questions throughout the survey instead of clumping them into sections.
6. Survey question vernacular, register or reading level are all ways to point out that most surveys are written with industry jargon and won’t be understood by all survey takers. When considering youth and adults with disabilities or language barriers, comprehension plummets and you largely end up with people guessing as they simply don’t understand the questions. This might even be a form of passive discrimination. Nobody does a readability check on a survey – even though such a task can be done online for free in a 5 minutes.
7. People generalize survey findings beyond the unit of measure. For example, a survey from, and applied to, one elementary school is much more useful than taking surveys from 100 elementary schools, combining the findings and then distilling blanket conclusions. Contexts and situations are lost under such common practices and the more you aggregate findings the less you lose of the “unit of measure.”

(Below – much to consider when developing a survey)

THE BETTER OPTION – FOCUS GROUPS

Dr. Perrodin regularly conducts focus groups – in fact, he has facilitated eight focus groups of 6-8 people within the past 2 months! In this episode, he talks in detail about how to carry out a focus group. Advantages of focus groups. The main advantages of focus groups are: they are useful to obtain detailed information about personal and group feelings, perceptions and opinions. You can record them, code them, field or ask immediate clarifying questions and be open to un-anticipated divergent input!

FOLLOW

• PodBean MP3  https://tinyurl.com/SDP68
• Apple Podcasts   http://tinyurl.com/SafetyDocApplePodcasts
• SUBSCRIBE to “The Safety Doc” YouTube channel  https://tinyurl.com/SDP68-VIDEO
• The 405 Media  http://the405media.com/the-safety-doc/
• SAFETY DOC WEBSITE & BLOG  safetyphd.com
• Follow David & The Safety Doc Podcast on Twitter   @SafetyPhD
• Email Dr. Perrodin   [email protected] 

Looking for Dr. Timothy Ludwig, PhD?

Dr. Perrodin’s “Safety Doc Podcast” negotiates school and community safety. To be informed about industrial safety, please contact Appalachian State University Professor Dr. Timothy Ludwig, PhD, at www.safety-doc.com

Endorsement:

Learn About the Nation’s Leading Bullying & School Safety Reporting System – SPRIGEO  www.sprigeo.com