Back in the day, we could learn many things from telephone surveys.
But then computers and cell phones were born and people started dropping their land lines. Not surprisingly those that still have land lines are different than those who do not. Think less educated, less wealthy and older.
So clearly land line only surveys are not going to be projectable.
Instead research companies have begun to combine cell phone and computer data with the information received from land lines. Weighting the responses has proved to be tricky.
Perhaps more troubling are the results from a recent study by Pew Research to determine the validity of combined responses. Pew found that answers from phones versus computers varied by anywhere from 0 to 18 percentage points. That's a big difference. And companies are likely to make decisions based on any skew greater than 10%. Possibly wrong decisions.
Interestingly the disparities had to do with intensity - so they aren't contradictory. That's a relief.
Apparently phone respondents were more likely to give extreme responses. They are also more likely to favor the last choice they hear, while online respondents are more likely to favor the first choice.
Discrepancies were especially large when respondents were asked to access their quality of family and social life. Telephone respondents were more likely to be satisfied than online respondents. They were also more likely to say gay, lesbian, Black and Hispanics face a lot of discrimination.
Meanwhile, online respondents were more likely to rate political figures negatively.
Researchers noted that it could be possible that telephone respondents were more inclined to give socially responsible answers. (McGinty, 2021)
So where does that leave us?
Can we use research gathered exclusively online? Or should people look for combined data? Should we verify research before we use it? How would we do that? Is the the best option for people to do their own research?
Have you participated in a phone survey or live research? Did you have a desire to please the interviewer? Have you noticed any pattern to your answers when you participate in online research?
McGinty, J. (2021, April 30) What Happens When Opinion Polls Move Online? wsj.com. Retrieved January 24, 2022, from https://www.wsj.com/articles/what-happens-when-opinion-polls-move-online-11619775001
4 comments:
I believe that there are many benefits to surveys conducted exclusively online. Firstly, there is no one who needs to read off every question which may be quite tedious. Also, people may feel more comfortable to share information online versus speaking it out loud over the phone if it is uncomfortable for them. As years go by, busy schedules may be a lot more conducive to some sort of mobile-friendly survey option online that individuals can take at any time versus having to answer the phone when called.
Speaking on my own experiences, I have found that I spend more time and go into more detail with online research and feel a lot more comfortable being open and honest. I find myself not needing to “please” the interviewer in any way that sometimes I feel pressure to over the phone with live interviews. On the other hand, sometimes the tone of voice in which an interviewer asks a question may provide different, more extreme answers which aligns with the data from McGinty.
I believe that we should look for a variety of research types in order to come up with the most accurate results. Every person has different instincts and habits when answering questions in different formats, and it is important for that to be acknowledged. If there is not enough data to pull from and any of it seems quite biased, it may be the best option for one to do their own research.
-Joe
Research conducted online has several advantages. It is easily accessible, people could participate anytime and anywhere at their convenience. Also, it is more time and cost-efficient. Moreover, as Joe mentioned, people would feel more comfortable. People could avoid uncomfortable questions, could be more open, and confidentiality would be more ensured.
Personally, I have never participated in a phone survey or live research but I believe that I would not have the desire to please the interviewer. However, I might feel pressured since I would not have much time to think about the questions than when I do it online.
I believe that the way the interviewer conducts the research could affect participants' answers and research conducted exclusively online could prevent this. On the other hand, not all people feel comfortable or feel they can easily access online research. Thus, it might not be available to gather information from specific target audiences which means that it would not be able to be generalized and may not reflect the publics’ opinion. Therefore, I believe that to have more accurate information from a wide variety of people, people should look for combined data.
Combined data would provide more accurate and diverse survey results than results obtained exclusively from online or one specific medium. Collecting and analyzing a wider range of results will allow for research to be looked at from different perspectives and potentially provide more thorough and accurate conclusions. Research can be verified by checking if credible sources were used, however, I believe we can only benefit ourselves by doing additional research on our own to determine what is truly subjective and objective data, as well as what is factually correct or incorrect.
I have rarely participated in phone surveys or live research, and I have always had the tendency to avoid these surveys as I feel many do today to spare ourselves time. However, whenever I have been obligated to respond to an online survey or if I had been stopped to be surveyed in person, I would take the time to respond as thoroughly as possible. When responding to survey questions in person, I do feel that I would try to please the interviewer and I also have done this in the past. In contrast, my survey responses when submitted online have been less about pleasing the surveyor, but more about me being honest in my responses. I notice I am more truthful and straightforward in my survey responses when not interacting with someone in a real-time scenario.
This further confirms to me that survey results vary depending on which medium people respond through. With an increased dependency on comfort and efficiency today, I believe surveys should be made accessible to people when and on which they are most likely to respond most truthfully and thoroughly.
- Nomi Q.
I don't think researchers can rely solely on data gathered online because they will be missing out on some key demographics that may not use or have access to the internet. I believe to have a more rounded sense they should use a combined data collection process. I believe the use of online forms, in-person focus groups, and cellphone and landline calls would create a better sense of how people feel about products and issues.
I have done a few phone surveys but never in any in-person. I did not have the desire to please the interviewer. I have not ever noticed a pattern in my answers when doing online forms, but I tend to take part in many in the first place. mainly because it's easier to just close a web window than it is to hang up on a person that is doing their job.
Post a Comment