Install

Get the latest updates as we post them — right on your browser

. Last Updated: 07/27/2016

The Truth About Polls




Every weekend, the political analysis shows on television serve up bits and pieces from public opinion surveys. Sometimes, they offer jarringly conflicting views of Russian political reality; and almost always the results are meaningless anyway because important information - who was polled, for example, or what was asked - gets left out.


On Moscow's TV6, Stanislav Kucher's program, "Obozrevatel," offers poll data without a word about how the data was collected. TV6 was purchased in June by famous businessman and now State Duma deputy candidate Boris Berezovsky, who made clear upon purchasing it that he wanted politically loyal news coverage.


On ORT - the nation's most-watched television station - anchorman Pavel Sheremet on Nov. 6 cited a poll by the Fund for Public Opinion that he said covers "every region in the Russian Federation." The poll itself actually surveyed people in 29 of the nation's 89 regions.


The next evening, Sheremet's colleague, ORT anchorman Sergei Dorenko, offered up a set of polls he said had a statistical margin of error of no more than 2.5 percent; actually, according to the Fund for Public Opinion, which conducted the poll, it had an error margin of 3.6 percent.


Either way, the margin of error rendered even more meaningless a dubious argument Dorenko offered: Emergency Situations Minister Sergei Shoigu, who has been hastily put at the head of the pro-Kremlin Duma bloc Unity, is suddenly "the only person who has still not fully realized himself as a politician." (See sidebar for details.)


Sheremet and Dorenko have both been on record frankly stating they serve the interests of ORT shareholder Berezovsky. Unity was initially set up by Berezovsky, though he now says he has nothing to do with it.


And then there is NTV television, an independent station set up by Berezovsky's foe, Vladimir Gusinsky, the oligarch who founded MOST-Bank. Until about three weeks ago, "Itogi" - the station's signature news program, under anchorman Yevgeny Kiselyov - was enthusiastically pushing polls by the Fund for Public Opinion showing a meteoric popularity rise for Vladimir Putin, Russia's war-time prime minister, and musing over other ups and downs of the week - even though sociologists insist many of the changes seized upon are too slight to be meaningful.


"When Yevgeny Kiselyov said with excitement that the rating of one politician rose by 1 percent, while that of another dropped by 1 percent, this is manipulation of the public - because those changes lie within the statistical margins of error and Kiselyov knows this perfectly well," said Sergei Tumanov, head of the Center for Political Opinion Studies at Moscow State University.


Recently, "Itogi" has switched to call-in surveys. Kiselyov frequently notes that these polls cannot be considered scientific - but he rarely explains how thoroughly one-sided they are, as they only represent the views of those educated, politically active Muscovites willing to watch a two-hour politics show like "Itogi," and then be bothered to pick up the telephone and participate.


At least the phone-in polls on "Itogi" are honest on one point: Since they happen live, the date the poll is conducted is clear. In nearly all cases, television anchors citing polls neglect to mention that the results are a week old.


Opinions for Sale


Under Josef Stalin, sociology was derided as a pseudoscience. With Soviet leader Nikita Khrushchev's thaw, it made tentative gains, only to be rolled back when Leonid Brezhnev sent tanks into Czechoslovakia in 1968. Until the perestroika era, public opinion polls usually amounted to announcements that the entire Soviet nation supported, say, the war in Afghanistan.


Such ham-handed manufacturing of data is rarer now, but Tumanov recalled that it was still around as late as 1992, when the government issued every citizen a privatization voucher - a check for use in mass privatization auctions. The face value of the voucher was 10,000 rubles, but its real value was to be dictated by market action.


"Once in 1992, we got a call from one client who said they were prepared to pay well for the results of the coming Sunday's public opinion poll," Tumanov said.


"The condition was that for the answer to the question, 'What will be the price of a voucher by the end of the year?' we must say '20,000 rubles.' We refused.


"But that Sunday, we heard on several television channels that, according to the results of a poll conducted by some other company, the price of a voucher will reach 22,000 rubles by year's end. In fact, they stayed at the level of 10,000 rubles. But the immediate results of trading with vouchers that Monday and Tuesday were great for those who ordered the poll - prices rose for a couple of days, and then dramatically dropped. Someone made a lot of money on that."


Do those sorts of things happen with national political polls? Tumanov said he can't "completely rule that out."


"For instance, I can't quite understand how it is that Fatherland's rating by the Fund for Public Opinion fell at the beginning of October in just two weeks from 29 to 18 percent - when there was apparently no grounds for such a change," he said.


That poll was cited on "Itogi," but other polling agencies tracked no such dramatic drop. Polls from the VTsIOM public research center, conducted monthly, indicated a fall from 22 percent in mid-September to just 21 percent in mid-October. According to polls done by ROMIR public opinion research center, also conducted monthly, Fatherland's rating fell from 20 percent in mid-September to 18.2 percent in mid-October.


In other words, two national polls found no change of statistical significance - over an entire month, Fatherland-All Russia pretty much held stable in the public eye. Yet a third national poll recorded the party plummeting, losing nearly half of its public support in just two weeks' time.


Alexander Oslon, head of the Fund for Public Opinion, said that the polls on Fatherland-All Russia were jumping all over the place then because of the decision of Moscow Mayor Yury Luzhkov and former Prime Minister Yevgeny Primakov to link up and form the bloc.


Some polls suffer at their fringes from laziness or low-level corruption among underpaid interviewers gathering the hard data.


"There are cases of scandalous unscrupulousness among interviewers, even of those who are working for the well-known sociological services, which are not paying enough attention to the control for their activities," said Yelena Bashkirova, general director of ROMIR.


"Some of them fill in the question lists themselves without asking the respondent, or ask only a part of the questions."


Reputable public opinion companies check up on their interviewers. The Fund for Public Opinion double-checks about 10 percent of its questionnaires through controllers, who phone or visit respondents to check the validity of the results.


"Recently, we stopped cooperation with three regional offices that were found to fake results," said Olga Menshchikova, the head of one of the fund's research departments. The fund pays 27 rubles (about $1) per completed interview - and fines a researcher 50 rubles for a forged one. Menshchikova said some students in need of money forge interviews despite the potential for a fine.


Apt. 433, Babushka, Rich


Different polling agencies take the national pulse in different ways. Most interview at least 1,500 people - or about one person for every 100,000 in Russia. To be representative of Russia, these 1,500 must have the same demographic breakdown - by age, sex, education level and geographic location - as the nation itself. Yet, they must also be identified at random.


The approach of the Fund for Public Opinion is typical of the major polling companies: Interviewers poll respondents at 160 points across Russia - among them Moscow, St. Petersburg and other cities and villages. Chechnya has been judged inaccessible and so taken off the fund's studied regions, as have some of the more remote regions of the Far North.


"The biggest task here is to provide a random list of respondents and to arrange the poll in a way that any Russian citizen may participate in it," Menshchikova said.


In Moscow, for example, the fund must poll 93 people - about 10 people from each of the city's nine districts. A computer generates street names in a district at random, and interviewers are told to start on a street at, say, building No. 2. If there is no apartment building at No. 2, they move down the same side of the street to No. 4, No. 6 and so on.


Say they have identified building No. 10 as an apartment building - but there is a security code at the front door. The interviewer does not know the code - but the rules require that he or she wait, however long it takes, until someone coming into or out of the building permits them to enter.


Once inside, the interviewer's next stop is an apartment. The number of that apartment is also randomly selected by a computer.


What happens if the computer has chosen, say, apartment 928 - but the building only has 450 apartments? Interviewers are instructed to subtract by units of 100 - so 828, 728, 628, 528 and so on - until they reach an existing apartment - in this case 428.


What if no one is home at apartment 428, or if they don't want to be interviewed? The interviewer climbs steadily up, to apartment 429, 430 and so on.


Say a resident in apartment 433 agrees to be interviewed. Success? Far from it: Researchers working for the fund are also handed demographic quotas. It's not just enough to find 10 people in a Moscow district; those 10 people must be demographically specific. Each interviewer will have different quotas, and in the end, the 1,500 polled must reflect a mini-portrait - in age, education, sex and location - of the electorate.


So a researcher may be told that three of those 10 people must have higher education, and the 10 must also include: one woman aged 18 to 24, three men and three women all aged from 25 to 54 and one man and two women older than 55.


"Once, when I needed to get an interview from a woman older than 55, I ended up in a family dormitory," recounted Yulia, one of the fund's footsoldiers. "I had to go through more than 200 apartments looking for a person [fitting this mold] whom I might interview, but only young families were there. I was hoping someone's mother would be living with a young couple, but there weren't any. It took a while."


Yulia said she is sometimes afraid for her safety when entering an unfamiliar apartment for an interview.


"Thank God, nothing has ever happened so far. But once, the questions were being answered by a drug addict. I understood this after about 10 minutes, when I realized he was forgetting questions immediately after they had been asked."


Did she leave?


"No, I didn't leave him. He is a citizen, just like all the others," she said.


A Babel of Opinions


Here was the political portrait of Russia offered Nov. 7 by TV6's Kucher. Citing a poll by the Agency for Regional Political Research, or ARPI, Kucher reported that the Communist Party was enjoying a dominating lead of 33 percent in public opinion. Turning then to Fatherland-All Russia - the bloc headed by Moscow Mayor Yury Luzhkov and former Prime Minister Yevgeny Primakov - Kucher said it had sunk to 22 percent, a showing he said "should cause concern."


He did not say who should be concerned, or why. Nor did he say by how much Fatherland-All Russia had dropped in the polls - there's was a drop of 4 percent from the previous week's poll, but given the margin of error of 3.6 percent, that is barely significant. What's more, according to ARPI, that bloc's ratings have gone like this in the past four weeks: 24 , 26 , 22 (the drop Kucher found concerning) to 24 percent in polling conducted from Nov. 5 to Nov. 7. In other words, there is no pattern whatsoever, and on the day Kucher was speaking, the polls were actually tracking an insignificant rise.


Kucher left out other key information. He did not say the poll considered only those people who say they intend to vote in Duma elections - a population sample that sociologists said usually ranges from 55 to 59 percent of eligible voters.


When assembling political ratings, the MGU center, ROMIR and the Fund for Public Opinion track the entire electorate; VTsIOM and ARPI usually track only those who intend to vote.


"[Looking just at those who say they'll vote] reveals a truer picture of the potential results of the elections," said Alexei Grazhdankin, deputy head of VTsIOM. But Oslon, of the Fund for Public Opinion, said that many people who say they won't vote now will change their mind, while many others who insist they will vote won't.


ARPI and the MGU sociology center usually ask open-ended questions. VTsIOM, ROMIR and the Fund for Public Opinion all pose multiple choice questions. Each approach provides its own insights and advantages.


"If a person who is not involved with politics is asked an open question, he or she feels a bit of a problem," Tumanov said. "But when a list is given, a name in the back of one's mind is easy to recall. Actually, this way of presenting questions - which mirrors the situation of filling in a ballot - is the most appropriate for finding out electoral ratings."


Such nuances are never discussed in media here, and pollsters complain that even when journalists are well-meaning and honest, they leave out or misunderstand so many key details that they usually get it wrong anyway.


"It is a big problem that every journalist - this is their professional duty - wants to find some sensation. They draw conclusions out of what actually means nothing," said Alexei Grazhdankin, deputy head of VTsIOM. "One should watch over a fairly long period, and compare results found by several [polling] companies, in order to talk about trends."


"In our view, there is only a slim chance that mass media will soon become a reliable partner [of sociologists]," Tumanov wrote in one of his scholarly articles.


When Polls Skew Opinions


A given medium is, however, usually a reliable partner for the political intrigues pursued by its parent oligarch. And polls can be a powerful weapon for winning new votes in such intrigues.


According to MGU's Tumanov, about 12 to 15 percent of the Russian population are "conformists" - people who try to answer poll questions (and cast ballots) the way they believe most people would do.


"This is not a small group and its votes are worth fighting for. And this group consumes poll results avidly," he said.


Indeed, many people seem to decide who to vote for based on what polls say others are doing. VTsIOM's Grazhdankin ticked off some of the different voter profiles: those who see their party sinking in the polls, thus bringing its in doubt, and so desert it, unwilling to waste a vote; those who planned not to vote, but change their minds to come to the aid of a sinking party they like; those who see a party soaring and want to jump on the winning train; and those who see their favored party safely in the Duma and so desert it to vote for a second favorite in more need of help.


Those interested in seeing raw polls can find VTsIOM polls at www.wciom.ru or at www.polit.ru; ROMIR polls at www.romir.ru; ARPI polls at www.monitoring.ru; and Fund for Public Opinion polls at www.fom.ru.