How Asking Gun Control Questions Differently Dramatically Changes the Results: Comparing NPR/PBS versus the Crime Prevention Research Center
A survey in May for NPR/PBS NewsHour by Marist had the headline: “Most Americans say curbing gun violence is more important than gun rights.” The Survey from May 15th through the 18th claimed: “6 in 10 say controlling gun violence is more important than protecting gun rights.” The survey asked: “Do you think it is more important to: Protect Gun Rights or Control Gun Violence.” The survey received massive news attention, extensive coverage on public radio and television affiliates nationwide, and other mainstream media such as USA Today and The Hill. The headlines in these other outlets echoed the headlines provided by public radio and television.
But here is the problem, despite gun control advocates being labeled as supporting “safety” and wanting to “reduce violence,” that is something that people on both sides of the debate believe. The way the NPR/PBS framed the question, it isn’t surprising that people respond as they do. To try to make the survey more balanced, we reworded the question to ask:
“Which of the following methods do you think is more effective in reducing violent crime?: Enacting stricter gun laws or Allowing people to protect themselves with guns.”
There was another difference in our surveys. The NPR/PBS NewsHour survey by Marist talked to 1,166 registered voters, but our survey looked at a more selective 1,000 likely general election voters with a 95% confidence interval of 3.1%. While the NPR/PBS survey found a large statistically significant gap, with many more people favoring reducing violence than protecting gun rights, our survey showed a statistical tie between the two methods of reducing violent crime.
We re-examined two other questions in the NPR/PBS NewsHour by Marist.
They asked whether people approve of Stand Your Ground laws. Their question read: “Some states have enacted legislation known as ‘stand your ground laws.’ Under those laws, people who are in a public place and believe that their life or safety is in danger are allowed to kill or injure the person who they think is threatening them. Do you approve or disapprove of those laws?” One key problem with their question is the phrase: “are allowed to kill or injure the person who they think is threatening them.” That is incorrect. People just aren’t allowed to kill or injure someone who they think is threatening them. There is actually a “reasonable person” standard where a reasonable person who believied that they were at serious risk of death or injury would be allowed to use force proportional to the harm without having to retreat as far as possible. That is quite a difference. The NPR/PBS NewsHour survey incorrectly makes it look like a person has a much discretion to arbitrarily shot someone and can use much more force than is proportional.
While the NPR/PBS NewsHour survery shows that people support Stand Your Ground laws by a 58%-to-40%, we believe that they have underestimated support. We instead asked: “Some states have enacted legislation known as ‘stand your ground laws.’ Under these laws, a person who is in a public place and is in a situation where a ‘reasonable person’ would believe that they are at serious risk of death or injury, is allowed to use necessary force that is proportional to the harm they face without having to retreat as far as possible. Do you approve or disapprove of these laws?” With that question, we now found that the gap increases from the 18 percentage points that NPR/PBS obtained to 43 percentage points that we showed. The 66%-to-23% margin in our survey showed that all demographic groups supported Stand Your Ground laws, with even Democrats supporting it by 52%-to-35% and liberals by 54%-to-39%.
Finally, we redid the NPR/PBS survey question on mass shootings. They asked: “When you hear about a mass shooting in the U.S., is your first reaction: More people need to carry a gun or This country needs stricter gun laws.” And they found that people supported stricter laws by a 62%-to-35% margin. We made a small change in the responses: “To allow people to protect themselves with guns or To enact stricter gun laws.” While we still found more people supported stricter gun laws, the margin was must smaller. Their gap was 27 percentage points, but ours was just 10 (41%-to-51%), with the standard divided response between Republicans/Democrats, liberals/conservatives, and urban/rural. While our results show a much smaller margin, it is still statistically significant.
