Lifting the veil on some issues in the ‘Burka debate’

30 Mar

For the unfamiliar, the BBC guide to Muslim veils.

The somewhat polemical:
Assuming that God has recommended that women wear the burka, assuming that burka has no impact on a woman’s ability to communicate or quality of life, as has been suggested by its supporters, then here’s a suggestion—to all men, who haven’t been ordered by God to wear burka, and who don’t see a downside to wearing it—why not voluntarily commit to wearing the burka, since no law opposes such a voluntary act, to show solidarity with the women. My sense is that even the French would come to support the burka if Muslim men en masse chose to wear it.

More considered:
‘The interior ministry says only 1,900 women wear full veils in France, home to Europe’s biggest Muslim minority’ (BBC). If the problem is interpreted solely in terms of women wearing the veil, then it is much smaller than the dust in its wake.

There are three competing concerns at the heart of the debate: Protecting rights of women who voluntarily want to wear it, protecting rights of women who are forced to wear it, and protecting (French) ‘culture.’ Setting aside cultural concerns for the moment, let’s focus on the first two claims.

People are incredulous of the claim that women will voluntarily choose to wear something so straightforwardly unpleasant. Even when confronted with a woman who claims to comply voluntarily, they fear coercion, or something akin to brainwashing at play. There is merit to the thought. However, there is much evidence that women subject themselves to many unpleasant things voluntarily, such as wearing high heels (which I understand are uncomfortable to wear). So it is very likely indeed that there is ‘voluntary compliance’ by some women.

Assuming there exist both, voluntary compliers, and those forced to wear the niqab, wouldn’t it be pleasant if we could ensure the rights of both? In fact, doesn’t the extant legal framework provide for such a privilege already? Yes and no, mostly no. While it is true that women forced to wear the niqab can petition the police, it is unlikely to happen for a variety of reasons. Going to the police would mean going against the family, which may mean doing something painful, and risking financial and physical well-being. Additionally, the laws governing such ‘coercion’ are likely to carry modest penalties, and unlikely to redress the numerous correlated issues including inadequate financial, and educational opportunities. Many of the issues raised here would seem familiar to people working with domestic abuse, and they are, and the modern state hasn’t (tried to) found a good solution.

Perhaps both camps will agree that wearing a niqab does dramatically limit the career opportunities for women. Of course people in one of the camps may be happy that there are limits to such opportunities but let’s assume that they would be happy if the women had the same opportunities. Part of the problem here then is the norms of dressing in business environments in the West. Entrepreneurs in Saudi Arabia recently brought to air a television talk show in which both of the hosts wore the niqab. The entire effect was disturbing. However, that isn’t the point. The point is that there may be ways to not reduce career opportunities for women based on the dress code, which after all seems ‘coercive.’

Time considerations mean a fuller consideration on the issue will have to wait. One last point – One of the problems cited about the burka is that it poses a security threat, which has some merit, given its long history in being used a method of escape, including by militant clerics.

The paradox(es) of Condorcet’s jury theorem when applied to democracy

18 Mar

Majority preferences were seen by Rousseau (The Social Contract) as expression of general will. With Condorcet, the ‘general will’ has also been imbued with the notion of ‘correctness’. As contradictory evidence is ample, it is time to remove to the false comfort of any epistemic benefit accruing from such aggregation.

Condorcet Jury Theorem, formalized by Duncan Black based on the elliptical essays of Marquis de Condorcet, runs roughly as follows –

If –

  1. Jury has to decide between two options using simple majority
  2. If each juror’s probability of being correct is greater than half (~ competence)
  3. Each juror has an equal probability of being correct (~ homogeneity)
  4. Each juror votes independently (~ independence)

Then –

  1. Any jury of odd juror is more likely to arrive at the correct answer than any single juror
  2. As n increases, probability of arriving at the correct answer approaches 1

(The above summarization of the key points is paraphrased from ‘Aggregation of Correlated Votes and CJT’ by Serguei Kaniovski)

There have been multiple attempts at ‘generalizing’ Condorcet – mostly by showing that violations to one or more of the assumptions doesn’t automatically doom the possibility of achieving an ‘epistemically’ superior outcome. One of the generalizations, offered by Christian List and Bob Goodin, is that the result still holds if people are posed with k options, and they have higher than 1/k chance of being correct.

Suppose there are k options and that each voter/juror has independent probabilities p1, p2, …, pk of voting for options 1, 2, …, k, respectively, where the probability, pi, of voting for the “correct” outcome, i, exceeds each of the probabilities, pj, of voting for any of the “wrong” outcomes, j ≠ i. Then the “correct” option is more likely than any other option to be the plurality winner. As the number of voters/jurors tends to infinity, the probability of the “correct” option being the plurality winner converges to 1.

Other ‘generalizations’ outline modified versions of the theorem if independence doesn’t hold, or when competence is unevenly distributed, etc.

One way to summarize the theorems is that math works to the extent the assumptions hold. Assumptions are at best poorly realized, and at worst inapplicable when we transpose CJT to democracy.

Problems of Applying Condorcet’s Jury theorem to Electoral Democracy


To apply CJT to democracy – we must assume citizenry to be a jury, and the decision task in front of it as choosing the “right” party or candidate.

The word ‘jury’ is saddled with association with courts in the current American context, and it is important to disambiguate how the citizenry differs from the ‘jury’ of citizens summoned by court for disambiguation will allow us to cover central issues that affect the epistemic utility of any “aggregations” of human beings. In the court system, a jury is (randomly ~ within certain guidelines) selected from the community, generally subject to a battery of ‘voir dire’ questions so as to assess their independence, lack of conflict of interest, biases etc., sworn to render a “rational”, and “impartial” verdict, instructed in applicable law (including evidentiary law), asked not to learn about case from any other source other than what is presented within the court (which itself is subjected to reasonably stringent evidentiary guidelines), guarded from undue influence (for example – bribes by interested parties), made to at least sit through presentation of extensive presentations from ‘both sides’, and their rebuttals, and generally asked to deliberate the evidence (among what is generally a ‘diverse’ pool) before reaching a verdict etc. On the other hand, citizenry that comes to vote is a self-selected sample (roughly half of the total body), highly and admissibly ‘non-independent’ in how they look at the evidence, generally sworn to ‘parties’, unconstrained by law on what evidence to look at, and how to look at it, generally extensively manipulated by interested ‘parties’, rarely informed about the ‘basis’, rarely arriving at decision after learning about arguments by ‘both sides’, and rarely ever deliberating etc.

The comparison provides a rough template for arguing against positive comparisons between the epistemic competence of juries and that of the citizenry. However Condorcet’s argument is a bit different – though many of the above lessons apply – and hinges on the enormous n in a democracy. The only other assumption that one then needs is each juror having more than ½ chance of having it right, or some variation thereof. The central contentions that can be made against Condorcet can come from two sources – theorization of the sources and extent of violation of the assumptions, for example – independence, and competence; inapplicability due to incongruence etc. The various contentions – emerging from the two sources – are covered below (in no particular order).

Rational voting, Sincere voting

While it is one of the weaker cases against applying Condorcet – mostly because the counterargument imagines a ‘rational’ voter – the argument deserves some attention – mostly because of its salience in the political science literature. One of the axioms of political science, since Downs, has been that information acquisition is costly. Hence it follows that as the decision making body becomes larger, and as the chance to be a ‘pivotal voter’ goes down, the incentives to shirk (free-ride) increase.

Austen-Smith and Banks, among others, have shown that ‘sincere voting’ – voting the best choice based on information signal – is not ‘equilibrium behavior’ as rational voter votes not only based on the signal but also on the chance of being pivotal. Feddersen and Pesendorfer (1998, APSR), taking the claim (perils of strategic voting) to its logical extreme – and applying it to ‘unanimity rule’ (not majority rule though similar less stark contentions apply – which they note), have shown that as jury size increases, the probability of convicting an innocent increases.

Extreme Non-independence

Given p > half is a ‘reasonably high’ threshold – jurors performing better than random – especially in circumstances of misinformation, problems can arise quickly.

In the current state, about 90% of the voters exhibit high forms of non-independence emerging from apathy and partisanship. (It also reasons that reduction in either one will lead to higher probability of citizenry choosing the ‘better choice’ on offer, and arguably better choices on offer.) Partisanship also means that people have different utilities that they intend to maximize. The other 10% err on average on the side of manipulation.

The flawed choice task

To the extent there are two inferior choices to choose from, one can imagine that in the best case the polity will choose the slightly better one among the two inferior choices. Condorcet offers no comfort for what kind of choices are on offer – perhaps the central and pivotal role of any normative conception of democracy. In fact, it is likely that the quality of choices on offer (‘correctness of choices’) is likely to be a function of probability with which a body politic knows about the ‘optimal correct choice’, and probability that it chooses the ‘optimally correct’ choice (which is likely to be collinear with odds of picking the ‘better choice’).

Policy Choices

Policy choices are an array of infinite counterfactuals. To choose the ‘most correct’ one would mean a population informed enough to disinter the right choice with a higher probability than any other wrong choice. Given infinite choices, the bar set for each citizen is very high, and chances of citizenry constituted as such crossing that bar – non-existent.

The well known paradox from 3 or more choices

The manipulability of system offering more than 2 choices is well documented and filed alternately as Condorcet’s Paradox and Arrow’s impossibility theorem. Much work has been done to show that propensity of cycles in democracy is not great. (For example, Gerry Mackie, ‘Democracy Defended’) One contention however remains unanswered for the binary choice version -American Democracy often reduces larger sets into two options. One can imagine that the preference order for citizens will depend on unoffered choices. Depending how multiple choices are reduced to two choices, one can think of ways ‘cycling’ can work even in the offered binary choices. (David Austen Smith) More succinctly – all binary decisions in democratic politics can be thought to come from larger option sets, and the threat of cycling hence is omnipresent.

‘Correct decisions’

CJT – a trivial result from probability -when applied to voting with two choices is just that an electorate is most likely to arrive at the more likely choice of each of its members. The probability of achieving that comes close to 1 as n increases.

If we assume that electoral democracy is a competition between interests, then we just get majoritarian opinions, not ‘correct’ answers. As in there is no ‘common’ utility function but different set of utilities for different groups – so people look at a common information signal and split based on their group interests. In that case, the ‘correctness’ of the decision really reduces to the ‘winning’ decision.

Median Voter – Condorcet in reverse

Applying Condorcet to democracy is in many ways applying things in reverse. We know that politicians create policies that appeal to the ‘median voter’ (not to be confused with median citizen, or anything to do with ‘correctness’). Politicians work to cobble together a ‘majority’ such that the p of the majority picking them is the greatest. Significantly – policy preferences that can be sold to the majority have no similar claims as made by CJT.  Another important conclusion that can be drawn from the above is that since the options on offer can manipulate the population, it is likely that the errors are not at random.

Democratic errors don’t cancel

Benjamin Page and Bob Shapiro, in ‘The Rational Public’, argue that one of the benefits of aggregation is that errors cancel out. Errors may be seen to cancel if they are at ‘random’ but if they are heteroskedastic, and strongly predicted by sociodemographics, they are likely to have political consequences. For example, we know then that such ‘errors’ will reduce the likelihood certain constituencies from making a demand, or from coalescing into raising political demands in line with their interests.

Formation of preferences, aggregation of preferences

Applying CJT to democracy, we can roughly proxy that preferences emerge from available data. Assuming people have perfect lens to the hazy data, the “probability that the correct alternative will win under majority voting converges to the probability that the body of evidence is not misleading.” (Franz Dietrich, and Christian List, ‘A Model of Jury Decisions Where All Jurors Have the Same Evidence’)

While even the probability calculated thence is optimistic – as we know that evidence isn’t same for all jurors, and the lens of most jurors is foggy – it is a good start to thinking about the – what data is available to the jurors, and how it is used by the jurors (citizens), and what are the consequences of different information and ‘analytic lens’ distributions.

Letting experts speak

If our interest is limited to getting the ‘correct outcome’, then we ought to do better (in terms of likelihood of arriving at correct decision) by polling people with higher probabilities of getting it right.  We will also save on resources. Another version of the idea would be to do a weighted poll, with weights proportional to probability of being correct.  The optimal strategy is to have weights proportional to log p(correct)/p(incorrect).  (Nitzan and Paroush, 1982; Shapley and Grofman, 1984)

It isn’t as much a contention as a prelude to the following conclusion – Any serious engagement with epistemic worthiness as a prime motive in governance will probably mean serious adjustments to the shape and nature of democracy, and in all likelihood abandonment of mass democracy.

When 60% differs from 51%

The key consideration in CJT is choosing the ‘right’ option from the two on offer. Under this system, 51% doesn’t quite differ from 60% or 90% for all yield the same ‘right choice’. Politics works differently – presidents tout and base their policy agendas on ‘mandates’, Congress and Senate have a slew of procedural and legislative rule that buckle under larger numbers. Thinking about Congress and Senate brings new complications, and here’s why – while election of each member may be justified by CJT, the benefit produced by elected representatives needs another round of aggregation – without some of the large n benefits of mass democracy. Here again we may note – as McCarty and Poole have relentlessly shown – that the ‘jury’ is extremely ‘non-independent’, prone to systematic biases, etc. In addition, no longer is choice limited to two – though each choice task can be broken down into a series of Boolean decisions (arriving at the ‘right decision’ in this kind of linear aggregation over choice spectrum will follow a complex function of p(correct choice) for each binary decision.

In Summary

Conjectures about epistemic utility of electoral democracy are particularly rife with problems when seen through the lens of Condorcet. This isn’t to say that no such benefits exist but that alternate frameworks are needed to understand those benefits.

On representation in a democracy

3 Dec

Theory of Representation

“Representation means the making present of something that is nevertheless not literally present.” (Pitkin, 1967) In a representative democracy, which, minimally construed, involves a mediating assembly for political decision making, representation, as defined by Hanna Pitkin, implies then an attempt to find the people in assembly’s political decision making.

But what exactly do we mean by that? Should we look for people’s values, thoughts, (current or stable, weak or strong) opinions (“phantom”, as Philip Converse puts it – though they may be?) in governance, and where exactly should we look — should it be policy, or institutional design, or process, or in the race and gender of representatives? Not to say that all of this rests upon the idea that these things (say opinions) can be coherently expressed by people (in aggregate), and cognizable in institutions, policy outcomes, etc. Other than these seemingly intractable questions of measurement, we also have substantive questions — who is represented, and to what degree, and why? And we must struggle with some normative questions that lie adjacent to the empirically posed question above — who/what should be represented, and in what degree?

Origins of our thinking about representation

There are two intersecting facets of how representation is understood, and perhaps should be — one is highlighting the cultural construction of concern with representation, and the other is historical understanding of representation.

Partly, idealized notions of representation are built against the inequalities manifest in the economic processes. The need for political equality/representation is a necessary counterpart to the society that has salient economic inequalities built of mythology of everything is possible for everyone.

Some of our understanding of representatives and control by people, constrained as they are by social norms of the Congress or bargaining between President and Congress, ought to be shaped by historical and normative conception. Historical foundations of current form of representation (in US) can be traced to at least Madison. As is commonly surmised, elite deliberation as a model for representation, was developed against the fear of the mob. True, but there was much positive thought guiding Madison’s idea of a modern democracy, and representation. It wasn’t just that unconstrained mass democracy is unsuitable, or the larger logistics based argument that mass democracies are untenable, Madison’s claim was that a desired effect of (elite) political representation is “to refine and enlarge the public views, by passing them through the medium of a chosen body of citizens, whose wisdom may best discern the true interest of their country.” Of course to what degree he succeeded in that ideal is open to conjecture, if not open derision.

Partly we also get an understanding of what is to be represented by the prominent instruments that key institutions provide people to express or control their representatives. If representation implies the extent to which political leader acts in accordance with wants and need and demands on public, then we ought to look into how public can express its needs, and how those are funneled in the political process. Lets take for example, vote. We know for a fact that vote itself is a poor instrument for expressing multifaceted preferences. Vote is binary, or at best trichotomous. So typically the role of a citizen was conceived to be relatively minimal, at least on a per capita basis.

But by constraining ourselves to discussion about voting, arguably the single most potent symbol of democracy, we fail to fully understand the ability and opportunity provided by democratic governance systems.

More simply, not all representation is via representatives. Democratic governance systems provide multiple ways to shape the public’s agenda, shape public opinion on the agenda, and how it is fed into governance. It provides multiple modes (lobbying, media, etc.), multiple institutional entry points (courts, legislature, public hearings of executive branches etc.), multiple temporal entry points (at the crafting of law, or as its failings are exposed, or in restricting its application –prerogative of the executive branch etc.), through communication of dissent, and consent, and hence allows for representation in many different ways within the many different institutional frameworks.

Measurement: Who is represented?

One way to assess who is represented is to merely track the economic well being of various groups over time. Another would be to correlate opinion/policy of representatives with that of the opinion of the constituency. Given politicians often actively shape opinion, and the problems with using correlation (as highlighted by Christopher Achen), the measure is largely doomed. In addition – any such measure ought to incorporate – the agendas of people (problematic to measure), and their opinion on those agenda items. In other words, we ought to measure two things – are issues considered important by people/constituents considered similarly important by the representatives, and the ‘correlation’ in opinion on those issues. In absence of similar agenda priorities, the question about agenda would be hard to measure. And certainly concerns about strategic/manipulative agenda setting by politicians (Page and Shapiro recently came out with a book – Politicians don’t pander that gives this worry some legs) would be of import here as well.

What should be represented?

The answer to the question is murky. Clearly multiple things need to be represented. For example: say a policy has a disproportional negative impact on a small group of people – their concerns perhaps ought to be represented. It is inarguable that the representation structure somehow constrains what is to be represented, depending on how widespread the cognition of its impact is – and to what salience. Part of our answer to what ought to be represented depends on our conception of democracy. So if governance is at heart about allocation of meager resources, and it is certainly at least about that, then does ‘representation’ of one’s interests (hard to define) at the bargaining table as ‘interest groups’ (or mobilized segments of society) present their cases the ideal?

If we minimally understand people’s wants as interest in ‘better’ outcomes, and assuming that ‘better outcomes’ emerge from good information, we can perhaps then focus on representation of (all) information – be it differential impact of certain policies, or some innovative technique.

Role of a representative

Heinz Eulau et al. present two models of thinking about the role of a representative – Who is being represented (district/state) – this needs to be further disinterred; and how (?) (trustee, delegate, politico or hybrid). These axes are a small but essential kernel of a theory of a representative. Yet it would do us disservice if we think that representatives do one or the other, on any of the dimensions. To a very significant degree, the institutional mechanisms have evolved to dole out the pork (district) and deal with say national issues as well, and many a times getting the former accomplished seamlessly as part of the latter. Alternately phrased – it does us disservice to think of district and state orientation as polar opposites of a continuum. A broad set of policies accommodate both. Similarly, trusteeship needn’t automatically contradict the role of a delegate. The theorization of the role of a representative ought to take into account the ‘fact’ that given that over a large set of policy issues, population has minimal (if not ‘phantom’) opinions, what is his/her role and responsibilities? Is it opinion leadership or manipulation (again the reference to Page and Shapiro – Politicians don’t pander)? The Page and Shapiro version is considerably closer to the dystopic version outlined by Pitkin – mass democracy inevitably fades into “fascist” manipulation. The argument, differently expressed elsewhere, goes like this – representation in a democracy is best understood not in terms of accurate correspondence between pre-existing citizen preferences and subsequent government decision but rather as a “constructive” (if ideally working) process that shapes the very same preferences and perspectives that are represented.

Hanna Pitkin. The Concept of Representation. (1967)
Heinz Eulau et al. The Role of the Representative: Some Empirical Observations on the Theory of Edmund Burke. (1959)
Christopher Achen. Measuring Representation. (1978)

Stability and Democracy

16 Oct

How does one ‘democratically’ govern a heterogeneous population with immense plurality of interests – perceived or real? In fact, how does one keep pressures stemming from economic, ethnic, racial, religious, regional, identities back? How does one avoid centrifugal forces from building up, and cleaving? We build an institutional system that only rewards broad coalitions. There is a nice corollary to the system that demands ‘broad coalitions’ for governance –one that opens up the opportunity for ‘change’: As the coalitions becomes broader, and more unwieldy, the opportunity beckons for the smaller party(ies) to expand their base by appealing to under served segments of that coalition, and perhaps win enough over to get a chance to govern.

Then – if it was the threat of ‘factions’ that led to the institutional design of American democracy, we have succeeded, almost entirely. The American political system has become a stable duopoly, with ‘factions’ –even troublesome ones like 1968 McCarthy supporters – now residing largely within the parties, mostly quietly.

But to discuss success of institutional systems that reward ‘broad coalitions’ in American context is to not fully discuss them at all. While it is true that in the American context, the ‘first past the post’ electoral system (if indeed the kind of electoral system ‘predicts’ the number of parties) has produced a largely stable two-party system (with occasional bouts of third-parties, the latest being Ross Perot in 1992; and the longest lasting being the ‘left-wing’ parties in the Teddy Roosevelt era), the system has had much less success in India, which boasts of thirty plus parties, with each ploughing its own furrow.

So clearly, there are limits to what institutional design can achieve. A closer inspection may reveal that some of the fault lines are visible even in US. One may argue that the term ‘broad coalitions’ is a misnomer– especially in the American context – where a significant number don’t vote, and where you can win an election by appealing to the ‘median evangelist’ or ‘median racist’, in Republican Party’s case. Similarly, one must question why significant ‘third parties’ like the Socialist party came to be important players, given the ‘logic’ of wasted votes. But overall, the system has worked well.

Party on

Democracy is perhaps best understood as a Schumpeterian ideal of mass public choosing from competing elites. Parties emerge as natural coalitional vehicles in a democracy to allow elites to stand on ideas, and not as elites. They allow provide the more ambitious members of the public to gain power, in exchange of co-option, partial indoctrination, and work. And furthermore, they allow for only people who aver by the dogma to rise to the top. But reality impedes. More so now, when media have made possible for politicians to come to the fore with only limited help from the party machinery.

Stable Coalitions

If factionalized political systems amplify every segment’s sane and insane demands, political systems that demand ‘broad coalitions’ are, by design, tethered to broad dysfunctions within a society.

At the heart of it – there is nothing seemingly ‘stable’ or even vaguely comprehensible about the ‘broad coalition’ that the Republican Party commands – it is a coalition of the rich, and the poor, the ‘fiscal conservatives’, and the taxation-averse (sometimes both), the social conservatives who elect Larry Craig, the libertarians who want government to legislate marriage (and more), etc. The subtext of this coalition, its glue, is of course race.

To keep ‘broad coalitions’ from heeding to their worst instincts, one needs an informed, civic and liberal minded citizenry. Failing which, while democracy with a relatively free press may prevent famines, it may not always prevent slavery or foreign occupation, if that is a ‘broad coalition’ supports it.

Inequality that barks, and dogs that don’t

6 Oct

Jennifer Hochschild, professor of Political Science at Harvard, begins her 1981 book, ‘What’s Fair: American Beliefs about Distributive Justice’, with an excerpt from Arthur Conan Doyle’s story, Silver Blaze, featuring the popular fictional detective Sherlock Holmes. In that excerpt, Holmes remarks on the ‘curious’ fact that the dog didn’t bark even when the ‘evidence suggested that it should have’. Hochschild uses this analogy to remark on the ‘curious’ fact about the American polity that it has steadfastly shied away from socialism (even socialist rhetoric) despite rising inequality and the large pool of likely beneficiaries of socialist policies. Politics has also been largely absent of demands of more income redistribution. I carry Hochschild’s analogy further – to somewhat disastrous extremes – all in service of conveying something simple. After all, all is fair in love, war, and blogging. Even dogs.

Before we investigate, ‘why the dog doesn’t bark’, it is incumbent upon us to identify who the dog is, why it should bark, and when, and how loudly? And does it bite? And we must investigate whether the implicit and naive assumption – that barking will result in anything – is actually correct. Only after we answer these, will we tackle some version of Hochschild’s question.

The Dog

The definition of the ‘dog’ depends heavily on the counter-factual that we want to use. For example, is it the bottom 95% of the income earners, or the lowest two quintiles of the income distribution, or the group below median income, or the minority of the federally defined ‘poor’? All of these ‘groups’ can in sense coalesce together to demand more redistribution of income taxes, certainly a ‘progressive’ income tax with substantially higher marginal tax on incomes above their own. But theoretical counter-factuals base their premise of group formation on automatic group formation on basis of economic interests. Such counter-factuals ignore things like extant cross-cutting social cleavages (for example race – disingenuously captured as ‘South/Non-South Dummy’, the Baptist/Southern Baptist dummy etc. in Political Science literature) that come in way of ‘class consciousness’, atomistic drives of the new labor and consumption regimes, apathy, ‘political culture’, historical narratives, and the near absolute dispersion of legitimizing discourses of inequality offered by the ‘society’. These reasons damn the existence of a dog to only those instances when political entrepreneurship meets economic realities powerfully enough to overcome the centrifugal forces mentioned above. So perhaps then the problem really is that the dog doesn’t bark because mostly there is no dog.

However, political coalitions around class do form, and if evidence presented by scholars is anything to go by – they are most salient and most persistent among the rich end of the spectrum. Larry Bartels has recently shown that policy choices reflect elite opinion much more so than mass opinion. “In almost every instance, senators appear to be considerably more responsive to the opinions of affluent constituents than to the opinions of middle-class constituents, while the opinions of constituents in the bottom third of the income distribution have no apparent statistical effect on their senators’ roll call votes. Disparities in representation are especially pronounced for Republican senators, who were more than twice as responsive as Democratic senators to the ideological views of affluent constituents.” (Bartels, 2005 – Economic Inequality and Political Representation).

Hochschild, relying on census data from 1929 to 1977, puts forth the fact that while the shares received by poorest two quintiles has changed little between these years, the largest change has been transfer of money from the richest quintile to the third and fourth quintile. Hoschschild’s story is about the ‘Director’s law’ (after economist Aaron Director), which goes something like this –‘Government has coercive power, which allows it to engage in acts (above all, the taking of resources) which could not be performed by voluntary agreement of the members of a society. Any portion of the society which can secure control of the state’s machinery will employ the machinery to improve its own position. Under a set of conditions to be discussed below, this dominant group will be the middle income classes.’ (George Stigler’s summary).

So perhaps there is a dog – a rich and a middle class dog, just not a ‘poor dog’. And that is in itself a ‘mystery’.

To bark or not to bark? And when to bark?

Should it on ‘perceiving’ the narrowing of the opportunities to move to the next class bracket? Or should it be on coming in contact with ‘increased’ inequality between ‘class peers’, as is so nicely documented in a recent series of articles in New York Times –the chronicler of the anxieties of the rich – that show that inequality is the greatest (and gallingly so) in the top 1% with top .001% earning far more than the .01%, which in turn earns substantially less than the next .1%. Or should it be the expansion of difference between the 25th percentile and the 75th percentile? Or having dramatically lower income than say our parents? A lot depends on how we define the dog, and what the dog sees. Both the dog and dog’s vision, if it wasn’t clear from the discussion above, is as much politically constructed as socially (if the two can be pried apart). Perhaps the answer is best approached via historical examples – times when we can be reasonably sure the dog did something or it looked like the dog did something.

If we go back to 1870s, the era of ‘Robber Barons’ and the original ‘gilded age’, the post-reconstruction era of lavish wealth, and even more gratuitous displays, we are at a point of history with indisputable and egregious inequalities. This era with its early stages of thuggish capitalists bought not only the rise of labor but also the trust busting presidency of Teddy Roosevelt. Perhaps there the dog did whimper. Similarly, there is a period again starting 1933 when there is a precipitous climb in the marginal income tax rate, partly brought upon by the war, and by FDR. The top marginal rates as recently as Eisenhower era were over 90%, and now top off at a miserly 35%. It is relatively unclear –except perhaps for rise of communist parties in US and a response to the depression, why we saw such a rise in redistribution of income. But it seems that that was the last time the dog whimpered.

Data from Piketty and Saez, among others, suggests the oncoming of a new gilded age. For example, according to census data, in 1967, households at 95th percentile had six times more income than ones at 25th percentile, the ratio in 2005 had grown to 8.6. It may yet be that the dog rises again, albeit slowly and feebly. And it may gnaw at immigrants, when it rises, before it gnaws at ‘greedy’ Wall Street guys. Oh, it’s already happening.

The Curious Incident: Why the dog doesn’t bark sooner, or bite?

Sven Steinmo, the clear eyed analyst attributes it to the ‘Political culture’ – the pull yourself up the bootstraps entrepreneurial anti-statist immigrant culture, constitution – the deliberately ‘anti-democratic’ (in words of American historian Gordon Wood) fragmentary government structure, weak parties, weak labor, weak government, and the fact that elites play a critical role in shaping people’s preferences. For Hochschild it is the lack of feudal history, the rapid rise of petit bourgeoisie, people being better off than their parents – at least much of the 19th and 20th century as the vast natural resources of US were exploited to carve out wealth, the fact that people have ‘chosen’ Capitalism (gain) over distributive ideas, deliberate fragmented structure of the government, the fact that poor limit their dreams, the fact that poor don’t demand absence of difference but just end of ‘unjust’ differences, and that the fact that people just want an ‘equal opportunity to be unequal’. For Gunnar Myrdal, the Swedish political scientist and economist and Nobel Prize Laureate and most significantly author of the Carnegie (who wanted someone from outside US for objectivity) funded ‘An American Dilemma: The Negro Problem and Modern Democracy’, it is the ‘American creed’. And perhaps because things haven’t been that bad, mostly. We never ask if things can be better for democracy isn’t about that. It is just mostly avoiding famines. (Amartya Sen)

Analyzing epistemic gains, and robustness of prediction markets

9 Mar

Since at least James Suroweicki’s “Wisdom of the crowds”, a multitude of scholars involved in the field of epistemic democracy have taken to theorizing epistemic utility of tools like “Prediction Markets”, and even the “Wikipedia” model. Cass Sunstein, a Professor of Law at University of Chicago, has in particular been effective in advocating the idea through a stylized analysis that cherry picks successfully working corporate prediction markets and ignores problems like the current morass of InTrade. Here below I analyze the conditions under which predictions markets can deliver their theorized epistemic gains, and test their robustness to violation of optimal conditions. I however start with analyzing a comparison that political scientist Josiah Ober makes between Ostracism and Prediction Markets and in doing so lay out some of the essential features of markets.

Josiah Ober and Learning from Athens

Ober has been a keen exponent of the idea that ancient Athens had institutions that ably aggregated information from citizens, and fostered “considered” judgments. In his Boston Review article, he strangely argues that the decision to build hundreds of warships, prodded by Oracle (!) and now known deliberate misinformation by Themistocles, led to an ultimately ‘right’ decision by the assembly to build warships and not say distribute the windfall from the silver mines to the average citizen. There are two problems here – one is epistemological with its reliance on Oracles for signs, and the other is use of manipulative information ala Cheneyesque. It is impossible to answer whether Persia attacked because they felt inklings of a threat due to the huge armada of ships that Athens had built.

At another place, Ober has compared the first step of Ostracism proceedings – the Demos taking a vote to determine whether to hold ostracism or not – with Prediction Markets. He argues that the vote to hold Ostracism or not aggregated individual level information or predictions about whether there is “someone” whose presence is pernicious enough so as to merit Ostracism. There are three pitfalls to such comparisons and I will deal with them individually. Firstly, Ostracism didn’t provide people with direct private economic incentives to reveal private information or seek “correct” information, and something which economists believe is essential (it is also born out in experiments). To counteract this argument, Dr. Ober argues that the manifest threat of making a “wrong” decision was large enough to impel citizens to gather the best information. There are two problems with this argument – penalties for making wrong decision fall on a continuum and are rarely either prosperity or annihilation (certainly the case in Themistocles and Persia), and secondly even in presence of immanent threats (something not quiet true in this case as the threat is defined vaguely as “wrong decision”- which is a little different from the most informed decision) to groups “collective action problem” prevails – albeit in an extenuated form.

In ancient Athens, decision to hold Ostracism or not was made through a vote. Vote is a deeply impoverished aggregator of private information for with each dip you get only a yes or a no. This impoverished information sharing also exerts enormous pressure on distribution of “right” information among the population for a small minority of “right” voters can easily be silenced by a misinformed majority. The only way in fact a Voting system can reliably aggregate information (if choice is binary) is if each dip –on average -has more than 50% chance of being correct. (Condorcet’s insight) Markets on the other hand provide for information to be expressed much more precisely through price. (We will come to the violations of this tenet in markets later.)

Unlike in voting, markets deter information (and misinformation unless strategically) sharing although price does send cues (information) to the market. (Of course strategic players fudge investments so as monetize their investment maximally) Suffice it is to say however that voting systems are more prone to aggregating disinformation, than market systems where incentives for gaining “right information” increase in tandem with people investing with “wrong information”.

Markets, Betting Markets

I will deal with some other issues including assumptions about distribution of private information later in the article. Let me briefly stop here to provide an overview of markets and betting markets in particular.

Markets, when working optimally, are institutions that aggregate all hidden and manifest information and preferences and express it in a one-dimensional optimally defined parameter, price. Since all individual preferences are single-peaked with reference to price, markets are always single-peaked, avoiding aggregation issues and Condorcet’s paradox. Markets aggregate not only information about demand, and supply but also the utility afforded by the commodity to each individual consumer, and such aggregation optimizes the “allocative efficiency”. And apparently all this is done magically – and in Adam Smith’s coinage at the beckoning of the famous “invisible hand”.

Prediction markets – also known under the guises of “information markets” or “idea futures” among others – tie economic gains to fulfillment of some prediction. The premise is that possibility of economic gain will provide people to reveal hidden information – or more precisely bet optimally without revealing information. Prediction markets are quiet different from regular markets for trading is centralized against a bookmaker that decides the odds after aggregating bets. This type of architecture puts significant constraints on the market than say the architecture of a share market, which is essentially decentralized. I will come to the nature of the constraints later but suffice it is to say that it avoids some of the “variances” and “excesses” and “excess variances” of the decentralized system – the kinds which made Robert J. Shiller turn to behavioral economics from playing with math and monkey wrench models.

Expanding on the nature of prediction markets – “A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election.” (Prediction Markets in Theory and Practice -2005 Draft, Justine Wolfers and Eric Zitzewitz) A more robust description is perhaps necessary to explain how bookies come to know about these odds. In much of sports betting, bookies commence betting by arriving at consensus that reflects expert opinions of a small group of professional forecasters. “If new information on the relative strengths of opposing teams (e.g., a player injury) is announced during that week, the bookie may adjust the spread, particularly if the volume of behavior favors one of the teams. In addition, since the identity of the bettors is known, bookies may also change the spread if professional gamblers place bets disproportionately on one team. To make these adjustments, the bookie moves the spread against the team attracting most of the bets to shift the flow of bets toward its opponent. Shortly before game time, the bookie stops taking bets at the ‘closing’ point spread. Like securities prices at the end of trading, closing spreads are assumed to reflect an up-to-date aggregation the information and, perhaps, biases of the market participants.” (Golec and Tamarkin, Degree of inefficiency in the football betting market, 1991, Journal of Financial Economics: 30). There are other ways through which a similar arrangement can be executed. For example, computerized software now continually adjust the odds depending on bets. The danger is that you can quickly short the system if you solely rely on anonymous betting data. I will come back to his later. One additional point to finish the description – Given the nature of the commodities or assets traded, we can only get results on questions that have binary answers, and not say discovery questions unless discovery questions can be split into innumerable binary questions.

Before we analyze the betting market efficiency, I would like to present a short list of the previously theorized (and proven) betting market failures or “instances where the operation of the market delivers outcomes that do not maximize collective welfare.” There are several forms of market failure:

  • Imperfect competition – where there is unequal bargaining power between market participants;
  • Externalities – where the costs of a particular activity are external to the individual or business and imposed on others (e.g. Assassination Markets);
  • Public goods – where there are goods for which property rights cannot be applied; and
  • Imperfect information – where market participants are not equally informed.

* Taken from Objectives of Betting and Racing Legislation (doc)

*As always, penalties follow some function of the extent of violation. Most effects are non-linear.

Let’s analyze the epistemic dimension of the market as in its capability to deliver information that is somehow better. The supposition in a prediction market is that people aren’t revealing (or finding information) for they have inadequate incentives to do so. So betting is merely a way to incentivize the discovery process. It is important to note that merely the fact it assumes that people have private reserves of information (generally amounting to knowledge that other people aren’t smart) severely limits the role of the prediction markets in areas where there isn’t such knowledge. Certainly I can’t think of a lot of public policy arena where it is the case. (It is also important to keep in mind that most policy decisions have a normative dimension aside from some fully informed preference dimension.) Otherwise betting markets merely try to aggregate – and don’t do so well – public information cues. Simon Jackman in his forthcoming paper that analyzes betting behavior in political markets in Australia has found that betting markets essentially move following the cues of opinion polling results. There is no information source outside of what is already publicly accessible that people rely on to make their bets. So the idea that somehow prediction markets will deliver better results even where privately held reserves of information are low or zero is ludicrous and easily empirically disproved.

More importantly, betting markets – even sophisticated ones like the sports betting markets – are incurably biased – proven statistically multiple times over – they underestimate home field advantage, and all too often “go with the winners”. The bias is supported by two intertwining psychological biases – “safe betting” and “betting on favorites” – and it is a bias found in nearly all betting markets.

Betting markets behave best if there is complete adversarial betting – which is never the case for most of the price is set by investment by small players following the elite herd. This has defined by Sushil Bikhchandani, David Hirshleifer and Ivo Welch, in a classic 1992 article, as “information cascades” that can lead people into serious error. Shiller recently wrote about this while explaining how the housing bubble (essentially banks betting on loans) stayed under the radar so long. He quotes the paper at length –

“Mr. Bikhchandani and his co-authors present this example: Suppose that a group of individuals must make an important decision, based on useful but incomplete information. Each one of them has … information…, but the information is incomplete and “noisy” and does not always point to the right conclusion.

Let’s update the example…: The individuals in the group must each decide whether real estate is a terrific investment… Suppose that there is a 60 percent probability that any one person’s information will lead to the right decision. …
Each person makes decisions individually, sequentially, and reveals … decisions through actions — in this case, by entering the housing market and bidding up home prices.

Suppose houses are really of low investment value, but the first person to make a decision reaches the wrong conclusion (which happens, as we have assumed, 40 percent of the time). The first person, A, pays a high price for a home, thus signaling to others that houses are a good investment.

The second person, B, has no problem if his own data seem to confirm the information provided by A’s willingness to pay a high price. But B faces a quandary if his own information seems to contradict A’s judgment. In that case, B would conclude that he has no worthwhile information, and so he must make an arbitrary decision — say, by flipping a coin to decide whether to buy a house.
The result is that even if houses are of low investment value, we may now have two people who make purchasing decisions that reveal their conclusion that houses are a good investment.

As others make purchases at rising prices, more and more people will conclude that these buyers’ information about the market outweighs their own.

Mr. Bikhchandani and his co-authors worked out this rational herding story carefully, and their results show that the probability of the cascade leading to an incorrect assumption is 37 percent. … Thus, we should expect to see cascades driving our thinking from time to time, even when everyone is absolutely rational and calculating.

This theory poses a major challenge to the “efficient markets” view of the world… The efficient-markets view holds that the market is wiser than any individual: in aggregate, the market will come to the correct decision. But the theory is flawed because it does not recognize that people must rely on the judgments of others. …

It is clear that just such an information cascade helped to create the housing bubble. And it is now possible that a downward cascade will develop — in which rational individuals become excessively pessimistic as they see others bidding down home prices to abnormally low levels. “

Betting markets like all other markets are “sequential” with each investor trying to parse tea leaves and motives of prior investors. The impulse to do original research is counterveiled by the costs, and by the fear that others know something that they don’t.

The other intersecting psychological factor that complicates markets is complete blind betting. Time and again even as information and probabilities converge, some bettors hold out for a miracle.

It is also important to keep in mind the following tenet that governs prediction market behavior– “Garbage in, garbage out… Intelligence in, intelligence out…” So prediction markets – to the extent that they rely on speculation are remarkably likely to follow any information that is likely to give them a leg up. While misinformation theoretically incentivizes procurement of good information, it never pans out empirically for major investment by another is seen as an informational cue, more powerful than whatever access you may have. This is an important point – for the competitor has no way of knowing your information for all s/he has access to is the investment that you make on it, and the space for conjecture about the veracity of the competitor’s information is immense. This is a market based on never revealing information, and that diminishes the efficiency considerably.

Betting markets merely rely on the fact that you are less misinformed than others, and that gradient can be built through strategically spreading misinformation (quiet common in betting circles) or through some theorized virtuous cycle of increasingly good information.

Betting markets can be easily shot by someone willing to lose some money. Asymmetry in finances can hobble the incentives for betting market and information discovery process.

Lastly, laws against insider trading limit the kind of information bettors have access to. They limit information discovery process severely. Relatedly, information – for it to be monetizable – has to be brought into the system privately so bettors may try to sabotage release of public information. Not only that, they have to be strategic in how they send cues to the market so that they earn the most money from their bets. If done en masse or rashly, it will almost certainly short their bets. So not only do betting markets have only one way of expressing information – price/investment- bettors go to great lengths to hide that cue especially if they know how the cues are being aggregated.

In summary, the above list of problems with betting markets underscores the analytical and empirical evidence against the naïve ill-substantiated unbridled faith in the epistemic prowess of the betting markets.

Democracy and information: The case for epistemic controls

14 Feb

First Amendment mandates that “Congress shall make no law … abridging the freedom of speech …” Courts have over the years construed the clause in a way that privileges political speech in its various manifestations while ruling to limit the protections afforded to commercial speech. Hence – however limitedly – different regulatory frameworks have emerged to control commercial speech in a variety of arenas. So for example, FDA through its Division of Drug Marketing, Advertising, and Communications sets standards for all drug advertisements. For example, it mandates that “all drug advertisements contain (among other things) information in brief summary relating to side effects, contraindications, and effectiveness.” Similarly any company listed publicly has to comply with a host of disclosure laws about its finances and business practices that dramatically restrict the kind of claims companies can make – at least about their accounts. The principle behind these mandates is the understanding that public good (in case of stock market – investor good) is served when we limit or penalize disinformation. The further pretext for such laws is that such laws are necessary where supply of information is essentially a monopoly of organizations or people with explicit incentives to lie or strategically misrepresent information.

Arguments for setting epistemic standards for speech in the political arena have been criticized in America on a variety of grounds including that such a law will be unduly restrictive and hard to implement, that free speech is necessary for democracy, free speech is a “fundamental” “human right” (absent of its necessity or utility) – it is part of Article 19 of UN Declaration of Human Rights, and free speech promotes search for truth, etc. More broadly, there are two major defenses for freedom of speech – a deontological conception of the sanctity of freedom of speech (expression – more broadly), and an instrumental defense of its merits – its necessity for preserving democratic values and ideals etc. It is hard to contest either of the claims: the claim for normative supremacy is hard to make in face of axiomatic judgments about the “fundamental” nature of these “rights”, and instrumental supremacy is hard to argue given “democratic values and ideals” are so loosely defined that they can be spun every which way, and an incommensurable value attached to any of those parts. But let’s hold this chain of thought for now.

Black’s Law Dictionary, 5th ed., by Henry Campbell Black, West Publishing Co., St. Paul, Minnesota, 1979, defines fraud as, “All multifarious means which human ingenuity can devise, and which are resorted to by one individual to get an advantage over another by false suggestions or suppression of the truth. It includes all surprises, tricks, cunning or dissembling, and any unfair way which another is cheated.”

It is an unsurprisingly vague definition cognizant of the artfulness of the subtlety with which fraud is perpetrated. The law relies on the skill of the prosecutor, and in company’s case – the defense attorney, and the perspicacity of the judge (or) jury to come up with a judgment as to whether fraud was perpetrated. One can come up with a more restrictive definition of the “law” but it would most likely be counterproductive for it would channel effort in perpetrating “fraud” that is still technically legal – much like compliance with tax law through using tax havens – and by taking away discretion from the judges, make it impossible to award judgments against patent cases of fraud. In the political arena, people are justifiably skeptical of arriving at limiting definitions, and at submitting speech to be analyzed by a body with fairly large discretionary limits on interpretation of the “letter of the law”. More nefarious motives undoubtedly exist for such protestations – least of them perhaps include worries that such a law would jeopardize their chances at electoral success. Then there is empirical evidence to suggest that politicians are very skillful at being honest without ever telling the truth. A stylistic narrow minded honesty that includes choice picking of their own life and opposition’s words is currently in vogue, and very hard to guard against. Not to mention the perverse ability to bludgeon the voter with the inconsequential, or ability to strategically shift focus on issues. For example – the 2000 campaign was essentially fought on the grounds of “whom would one like to have beer with?”, or the “Willie Horton” ad used in 1988 election featuring real life person and story – though of limited evidential value – to skillfully convey race and crime cues to the disadvantage of the Democratic challenger, Michael Dukakis.

Quiet apart from the justifications for “freedom of speech” is the claim that we should let the market dictate roughly the proportion of what is heard at what volume, and what isn’t. The supposition is that market will somehow tune up the volume of the speech in proportion to its appeal to people – with no claims made about its epistemic worth. The ancillary argument is that “marketplace of ideas” will sift through ideas in a way that the “best” ideas and opinions rise to the top, and additionally even if someone buys more airtime it doesn’t quiet matter for the public decides whether the idea is in its interest and hence dictates its adoption (popularity). Understandably it is a hopelessly unsupported proposition lacking any merit whatsoever.

The argument that competition alone would be enough to bring the “product” with the “best utility” (economic utility) to come to the top is doubtful at best, and predicated on a host of bizarre wholly empirically unsupported assumptions. The argument is particularly inapplicable to the domain of “public goods” where people have limited incentives to gather enough information, and in systems where information distribution is asymmetric. Anthony Downs expressed worries about “asymmetric information” in his 1957 opus, An Economic Theory of Democracy. There is indeed a steadily increasing penalty for disinformation and the resulting sub-optimal decisions, but given the low efficacy that people feel (among other things), their interest and motivation remains in making themselves more informed low. Given such conditions, we need – more strongly than a regulatory framework governing information dispersal in private economic choices – a similar mechanism so as to mitigate some of the most severe problems.

One of the ways – and one that would be appealing to all political parties and free speech advocates – through which we can mitigate some of the problems in the current information regime would be to mandate release of government information. Transparency has been found to increase attendance of teachers in rural schools, and distribution of funds to people in rural employment scheme in India, attendance of legislators in Uganda etc. FOIA works to a great degree in US and it is arguable that the gravest trespasses occur where the reach of the statute is the most limited – “national defense”. Transparency is effective because it creates opportunities for accountability. It however also foments strategic compliance. For example, as I mentioned earlier – we now have fairly truthful ads that systematically misrepresent issues and positions of opposition. Graver still is the issue that transparency only works to a limited degree in a country where people are apathetic, and media absconding. For example, while America has inarguably the largest trove of publicly available data including statistics on economics, labor etc. – they are hardly ever part of the public discourse. The corrective solution flows directly from the way I describe the problem –creating an aware media that works to highlight these facts and put issues and positions in context.

Let me take a moment to argue that given mass media is currently the dominant way through which people get information, any proposal for instituting epistemic controls has to cover media. Similarly, any proposal for epistemic controls has to not only ensure epistemic superiority of the resulting commentary, but also regulate how it is presented so as to be useful to the masses. In other words, the framework has to take into account the state of the masses, and how they process information. Given the psycho-cognitive research by Kahneman and others, we know that people regularly ignore base rate information in favor of illustrative anecdotes – the way news is traditionally offered. We also know enough through the priming literature that repeatedly bringing up an issue –regardless of context – increases its salience in decision making.

Given the difficulty in mounting such controls, perhaps the best epistemic intervention that we can provide is an educative one. There is ample evidence that if we improve people’s understanding of what constitutes valid and pertinent evidence, and even help inculcate simple skills like numeric literary – we can have a tangible impact on the way people make decisions and how they respond to appeals.