Leading Questions - Yes Prime Minister
LimeXd4 HN recommenders · 2012–2025
1.6M views · comedy · evergreen · Uploaded 2012-01-15
A clip from Yes Prime Minister in which Sir Humphrey shows how leading questions can make a survey support or oppose national service. It is a short satirical scene.
HN commenters bring it up whenever a poll or survey result looks suspect, as the classic illustration of biased question framing.
19:09A Better Default Colormap for Matplotlib | SciPy 2015 | Nathaniel Smith and Stéfan van der Walt
Enthought
58:51DON'T PANIC — Hans Rosling showing the facts about population
Gapminder Foundation
51:22USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?
USENIX
2:58The Loving Trap
Ben Woodhams
12:22Is Most Published Research Wrong?
Veritasium
19:08NaN Gates and Flip FLOPS
suckerpinch
19:10How not to be ignorant about the world | Hans and Ola Rosling
TED
9:55Erlang The Movie II: The Sequel
gar1t
20 comments from Hacker News
This is different from the classic phenomenon explored in Yes, Prime Minister [1]. It's not that they manipulated the questions in advance -- it's that they reversed the meaning of the questions after the respondent had answered them . Even under these conditions, the majority of respondents were then prepared to support the position which they had, ostensibly, only a few minutes ago opposed and vice versa . Remarkable. [1] [VIDEO]
"voters say 45 - 40 percent the government's anti-terrorism efforts go too far restricting civil liberties, a reversal from a January 10, 2010, ... when voters said 63 - 25 percent that such activities didn't go far enough to adequately protect the country." The fact that the statement has reversed ("not far enough" -> "too far") rather than the numbers (ie. it's not "63 - 25" -> "40 -> 45") brings to mind this scene from Yes Minister [VIDEO]
The issue with psychology is that double blind experiments are prone to observer effect. The behaviour often have nothing to do with reality - when measured, everything tends towards the ideal. And the experiments can often result in the results you want if the double blinds are removed. I always fall back onto the Survey from "Yes Minister" to explain how that works [VIDEO]
Taking that humorous example further, you could go down both questions, still have a have a final "Yes" to both questions, but then have a final question to ask which one of the two a particular voter is worried about. e.g. [VIDEO] has a comment with the transscript of the skit HUMPHREY: No. So she starts asking you some questions. Mr Woolley, are you worried about the number of young people without jobs? BERNARD: Yes. HUMPHREY: Are you worried about the rise in crime among teenagers? BERNARD: Yes. HUMPHREY: Do you think there's a lack of discipline in our comprehensive schools?] BERNARD: Yes. HUMPHREY: Do you think young people welcome some authority and leadership in their lives? BERNARD: Yes. HUMPHREY: Do you think they respond to a challenge? BERNARD: Yes. HUMPHREY: Would you be in favour of reintroducing national service? BERNARD: ...Oh. Well, I suppose I might be. HUMPHREY: Yes or no? BERNARD: Yes. ... HUMPHREY: Mr Woolley, are you worried about the danger of war? BERNARD: Yes. HUMPHREY: Are you worried about the growth of armaments? BERNARD: Yes. HUMPHREY: Do you think there's a danger in giving young people guns and teaching them how to kill? BERNARD: Yes. HUMPHREY: Do you think it's wrong to force people to take up arms against their will? BERNARD: Yes. HUMPHREY: Would you oppose the reintroduction of national service? BERNARD: Yes. ... FINAL QUESTION: Are you more worried about the rise in crime among teenagers or about the growth of armaments? POTENTIAL FINAL ANSWE
Polls aren't always about matching reality. They're often used as ammunition in an argument - "Look, the majority supports my position!" The classic example is [VIDEO]
I was finally able to get access to the full questions asked in the poll. These make clear that the survey was a fishing expedition designed to elicit viral stats. The questions asked in the poll include “Is Corona related to the coronavirus?” and “In light of the coronavirus, do you plan to stop drinking Corona?” Those presumably didn't get interesting answers, so they omitted them from the report, and instead published on the question of whether people "would not buy Corona under any circumstances now" and made it sound like the first questions. Good lord, it really is like the "Yes, Prime Minister" clip: [VIDEO]
Assuming it's similar polling to the UK there are all sorts of weighting and balancing to get truly representative samples within a margin of error. Where the system is weakest is in the questions asked, as explained by Yes Prime Minister [VIDEO] I can't see the tables gallup produced to make their claim, without those I can't see why it's of any interest.
> the questions can shape the response. An excellent illustration of this: [VIDEO] Although in this case, it doesn't look like they had bad intentions and they seem to have dug at least a little below the surface and drawn what seem like sensible conclusions. I wonder if more detailed results and the full list of questions are available somewhere.
Yes. The culture war is to a great extent a US import. The US is extremely polarised, possibly uniquely so for a country with a single national language and mostly-shared ethnic background (compare Belgium, which is highly polarised but along language lines). The UK has its own natural fragmentation lines (north/south, class system, four nations, catholic/protestant) but those are not the lines on which the culture war is run. While there has been incursion of talk radio (LBC/Farage), there is no counterpart to the hyper-polarised TV of Fox News yet. Sky News is comparatively normal. The right-wing disinformation comes in via the press and various "client journalists" who repeat things they've heard from "Downing street sources" who they refuse to hold accountable. > It found that most voters balanced competing political concerns and ideas. Its polling found that 73% believe hate speech is a problem, while 72% believe political correctness is an issue. Some 60% believe many are too sensitive about race, but 60% also recognise issues around “white privilege”. This is just the combination of leading questions and people responding to words without actually thinking about the underlying concepts. See, always, Yes Minister: [VIDEO]
> As ever with polls it depends how you frame the question. Obligatory 'Yes, Prime Minister' clip: [VIDEO]
The most typical reason people don't like to answer that question is because they believe it is as setup for a followup: "Then why aren't you doing X?!" where X in this case could be "Fire this person", "Setup a committee overseeing all hires", "Commit to quotas", "Donate money to this cause." Also reminds me of this "Yes Prime Minister" skit about leading questions: [VIDEO]
"We asked 100 people who were interested enough to write to their MEP about digital privacy and 80% of them said digital privacy was important" People who are motivated enough to know and comment about these proposals are not a representitive sample of the population. If you were to stand outside Old Trafford and ask people who the best football team in the UK is, you'll get the vast majority saying Manchester United. Do the same thing in London and you'll get people saying it's Arsenal. Aside from both being wrong, you can see the bias. Accredited polling companies adjust results to minimise sample bias, but even then have to be careful to avoid leading questions as shown in this 'documentary': [VIDEO]
Can someone post an English translation of the "theses" or questions in the Wahl-o-mat? When I hear about such initiatives I am always wary of this kind of polling manipulation: [VIDEO]
Link for the curious: [VIDEO] Starts at about 1:00. One of my favorite shows Edit: updated link, thanks @dvngnt_
I'm saying that polls can be twisted however you want, if you're willing to put in the effort. [0] How the question is framed is a large part of the effort, and what the quoted doctor was complaining about. (Specifically, he was complaining that things were not sufficiently twisted in the direction they found favorable) Disaggregating the question based on trimesters is one form of twisting. Asking only a single blanket "are you for or against the right of a woman to choose" is also twisting. 0: [VIDEO]
This is probably a good time to recall that polling in politics can be highly misleading . I recall an excellent segment on Yes, Minister that explains it [0, 1]. We really need to know the other questions they were being asked at the time to see if they were primed in some way. A near majority being uncomfortable with white people would be outrageous, but Americans are usually pretty tolerant so it is likely that Rasmussen is playing games here. [0] [VIDEO] [1] [LINK] - For people who like text, search for "Have another opinion poll done"
The tool would be useful as a QA step to test for leading questions in survey design. See Yes Minister's[1] explanation for how they can work. A simulation to see if the questions get the same response irrespective of the order they were asked in could improve survey quality. Obviously, the tool could be used in the opposite way too, to help design surveys that say exactly what the company/govt/charity wants it to. [1] [VIDEO]
Polls may or may not be carried out in a neutral way. A good many are deliberately biased, even more have known flaws that are glossed over. A humorous take on this I love (from the 80s BBC comedy Yes Minister, which is often scarily realistic): [VIDEO]
It depends on how you word the survey. ;-) [VIDEO] (Yes Minster: the unofficial video on-boarding course for UK bureaucrats)
You are probably talking about the YouGov poll. The poll asked a clearly leading question IMO. You can get any result you want by asking leading questions on polling. This was of course satirised by Yes Minister. [VIDEO] I can counter any of the iffy polls by simple point to the official online petitions service. There were a huge number of signatures to revoke OSA and two million signatures to abolish the plans for the Digital ID. While the Digital ID is technically a separate issue, many of the same privacy concerns are present. [LINK] [LINK] The number of people that signed these petitions is far more representative than any polling. On top of that, recently I've seen reportrs of both the Liberal Democrats and Reform (the two largest parties after the main two) recongising the OSA as unpopular and are likely to suggest reforming/removing it. On top of that. The labour government and the conservative government that proceeded it which created the OSA were/are both deeply unpopular. So any notion that there is a popular mandate for this is nonsense.