Wednesday, May 20, 2015

Electoral fraud and Benford's Law


Allegations of electoral fraud are at least as old as elections themselves. Granted that such a fraud exists in a given situation, it may not be that simple to prove it. Some said that the motivation for electoral frauds is helped by the fact that once you get elected, it is almost impossible to get removed.

There could be barefaced ways of electoral fraud as well as finer ways. It may be tempting to think that the blatant ways of fraud would be easier to prove, yet only the powerful people would be likely to choose such methods because they don't care. And so also, the proofs would be harder to get. Yet finer ways of fraud would be able to bring about the desired results, especially in closely fought elections and would then be almost impossible to point your finger to. Anyway, to get started with electoral fraud, you can look for it in Wikipedia, or look for electoral abuses in Encyclopedia Britannica or anything similar.

Are electoral frauds widespread in space and time, in our world? We don't know. You may assume that for any convicted electoral fraud there may have been multiple cases that escaped conviction or just assume it as an isolated case. It depends on whether you are with the camp, like the case in U. S., that argues that election fraud is rampant and elections are completely corrupt, or with the other camp which dismisses all claims of election fraud as partisan and instead argue that election fraud is nonexistent in U.S. elections. While it seems easy enough to have a gut-feeling that such abuses exist anywhere, in general it is harder to detect them, and still harder to convince the courts or the election authorities about the irregularities.

Are there statistical methods available for detecting electoral fraud? Are there digital signatures differentiating uncertainty in data collection, process errors, and methodological biases from downright vote rigging? In this connection, the attempt to develop statistical methods to verify whether elections results are accurate has come to be known as election forensics in the field of social science.

An obviously appealing approach for detecting election irregularities seems to be to compare official voting reports with the exit poll results. However the exit polls are very much imprecise as we have seen, for example, for the very recent UK 2015 General Elections. You may think that intensive polling independent of those by media groups would do the trick. In fact, this idea is nothing news and also unworkable. David W. Moore, Senior Gallup Poll Editor has refuted this in Exit Polls Probably Ineffective Against Vote Fraud (2004). He quoted Warren Mitofsky, the inventor of the exit poll, who said it would not work unless the size of the error in any single polling place is very large because small errors will be undetectable. We could also note that additionally the voters have to say truthfully who they voted for. Well, it is not always the case. In the UK 2015 election it was acknowledged that "Shy Tories", or those who voted for Conservatives but reluctant to say they did, really existed and that was one major factor for the downfall of the exit poll and all other pre-election polls.

If a less intrusive method such as exit poll would not work, let's consider the most intrusive, and the most direct way of investigating the existence or otherwise of electoral abuse. Assume that you get past the intricacies of handling electoral fraud complaints by the electoral authority or the court, what could you do when ballot papers don't exist anymore? It was this latter kind of situation the researchers Leeman and Boschler faced with, "In a Swiss referendum in 2011, one in twelve municipalities irregularly destroyed the ballots, rendering a recount impossible. We do not know whether this happened due to sloppiness, or to cover possible fraudulent actions." (A systematic approach to study electoral fraud, Electoral Studies, 2014), and so:

    How can we detect electoral fraud? The answer to this question depends on the type of committed fraud. Lacking access to the proof (the ballot papers), researchers have started to develop statistical methods to detect irregularities in the reported election results, which might be due to illegitimate manipulations. ...
    In general, there are two ways to go about detecting electoral fraud. We focus on the returns at the lowest levels possible and we try to compare outcomes with expectations. The origins of these expectations distinguish the two instruments we have. First, we may rely on ecological information. Knowing the political structure of a village may allow us to predict the voting pattern we should observe (Alvarez and Boehmke, 2008). This approach relies on regression style models based on a subsample where we can (with large confidence) outrule fraud.
    Second, we can focus solely on the return sheets (the reported numbers). We compare these figures not with other returns but with a theoretical distribution of digits. ... The basic idea is that when someone makes up numbers they fail to produce numbers that are truly random in the way they would be in a truly fair election or vote.

This second technique is based on the Benford's Law.
Looking for an explanation of this law for dummies (and myself), I couldn't exactly find it.  The following from RonJoniak.org (The Mysteries of Benford’s Law, June 7, 2014) without math, without probabilities, may be good enough to give some sense of this strange law to everyone, though much simplified and incomplete.

Benford’s Law, in the most elementary form of understanding, states that the number “1” transpires as the leading digit 30% of the time compared to higher digits such as 9 which occurs 5% of the time. This occurs for all kinds of data sets ranging from electricity bills, street addresses, stock prices, to even physical and mathematical constants. Yes, that’s right, the physical and mathematical constants of the universe follow this mysterious law.


That is Benford's Law stated for the first significant digit, whereas a more complete definition would give for other digits also. Actually, electoral forensics researchers favor looking at the second, third, or fourth digit, and particularly the second digit instead of the first one.

A nice presentation of introduction to Benford's Law is by John D. Barrow (Benford's Very Strange Law). The following is slide number 28:

While Benford's Law is increasingly used for election forensics in recent times, some believe that it is suitable only for cases where vote counts are altered which is considered quite unlikely (Leeman and Boschler, cited earlier), while some think it is outright unsuitable (The Irrelevance of Benford’s Law for Detecting Fraud in Elections, Deckert et al, 2010). Walter Mebane who has been investigating Benford's Law in his election forensic research, finds it inconclusive (Using Vote Counts’ Digits to Diagnose Strategies and Frauds: Russia, Walter Mebane, 2013):

Both the second-digit Benford’s-like Law (2BL) and the idea that the last digits should be uniformly distributed have been proposed as standards for clean elections. Many claim that election fraud is rampant in recent Russian federal elections (since 2004), so Russia should be a good setting in which to see whether the digit tests add any diagnostic power. Using precinct-level data from Russia, ... The digit tests produce surprising and on balance implausible results. ...The usefulness of simple and direct application of either kind of digit tests for fraud detection seems questionable, although in connection with more nuanced interpretations they may be useful.

Pointing out that the while the Second-Digit Benford's Law (2BL)-test is getting popular with researchers, it has mostly been applied to fraud suspect elections, Shikano and Mack tried applying the test to the 2009 German Federal Parliamentary Election against which no serious allegation of fraud has been raised (When Does the Second-Digit Benford’s Law-Test Signal an Election Fraud? Facts or Misleading Test Results, Shikano and Mack., 2011). According to them:

Surprisingly, the test results indicate that there should be electoral fraud in a number of constituencies. These counterintuitive resultsmight be due to the naive application of the 2BL-test which is based on the conventional v2 distribution. If we use an alternative distribution based on simulated election data, the 2BLtest indicates no significant deviation. Using the simulated election data, we also identified under which circumstances the naive application of the 2BL-test is inappropriate. Accordingly, constituencies with homogeneous precincts and a specific range of vote counts tend to have a higher value for the 2BL statistic.

Mack and Shikano, April 2013, and summarized that:

Concentration of precincts votes in a certain range can boost the BL statistic. Some argue that the use of second digits instead of first digits would solve this problem. This is, however, no solution since concentrated precincts votes appear in certain circumstances which can affect the distribution of even second digits. In this paper, we apply 2BL and an alternative distribution systematically to different institutional settings. More specifically, we investigate the latest parliamentary and presidential elections of France (both 2012), with no suspicion of fraud, and Russia (2011 and 2012), with strong suspicion of fraud. Finally, we replicate another detection method for the purpose of cross validation and compute simple fraud scenarios to assess the performance and mechanisms of 2BL. We can identify a circumstance when 2BL gives misleading signals and have to conclude that 2BL is inappropriate for fraud detection.

On the other hand, Pericchi and Torres who were the first to suggest using the 2BL-test support the Benford's Law tests with a number of variants (Quick Anomaly Detection by the
the USA, Puerto Rico and Venezuela , Statistical Science, 2011):

The test examines the frequencies of digits on voting counts and rests on the First (NBL1) and Second Digit Newcomb–Benford Law (NBL2), and in a novel generalization of the law under restrictions of the maximum number of voters per unit (RNBL2). We apply the test to the 2004 USA presidential elections, the Puerto Rico (1996, 2000 and 2004) governor elections, the 2004 Venezuelan presidential recall referendum (RRP) and the previous 2000 Venezuelan Presidential election. ... The adequacy of the law is assessed through Bayes Factors (and corrections of p-values) instead of significance testing, since for large sample sizes and fixed α levels the null hypothesis is over rejected. Our tests are extremely simple and can become a standard screening that a fair electoral process should pass.

The appeal of the Benford's Law for election forensics is that it is simple and doesn't need much more data than vote counts to apply it. However, research has shown that the view that it could be used as a model for generating fair voting pattern in all elections so that results for any election may be compared with this standard to judge irregularities couldn't be supported. However, that may not be the last word. Certain type of blatant electoral fraud may still be detectable with some adaption of Benford's Law or appropriately modified variants based on the investigation of digits. Some things are just too dear to throw away.

Mebane (2013) said:

A caveat is that the motivation for the digit tests presumes that those who commit election fraud are unsophisticated or careless, or that the mechanisms used to commit the frauds do not allow precise control of what the fraudulent outcomes are. Beber and Scacco (2012) argue that humans who fake vote counts simply by writing down numbers they happen to think of are subject to psychological limitations that produce nonuniform patterns in the results. Such human limitations would be easily overcome, say by using a random number generator to create the fake numbers: using well-known algorithms, the fake numbers can have any desired distribution. Vote counts that are 2BL-distributed are easy to simulate as well, which would tend to undercut the test advocated by Pericchi and Torres (2011). Indeed, Beber and Scacco (2012) point out that the simulated counts produced by one of the mechanisms in Mebane (2006) that was designed to produce counts that satisfy 2BL also have uniformly distributed last digits.

True, it is complicated. And yet it's all the more interesting for it.

No comments:

Post a Comment