Complimentary spam

August 5, 2010 in General

Those who don’t read this blog are its biggest fans. No surprise there. Here below is a selection of comments left by spam-bots. Spelling and grammatical ‘errors’, likely deliberately put in by spammers, have been left untouched. The word ‘errors’ has bunny ears because you can’t really call a deliberate strategy an error. These ‘errors’ also bring up the question that in the era of computers, perhaps errors will be the one indication that the work is produced by a human (directly or indirectly).

Good job!

I love your site. Logan.

Pretty good post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon.

I was actually looking for this resource a few weeks back. Thanks for sharing with us your wisdom.This will absolutely going to help me in my projects.

Extremely helpful. I love the way you write. Do you provide an RSS feed?

What theme are you utilizing here ? I like the layout. Many thanks for this posting.

This is all very new to me and this article really opened my eyes.Thanks for sharing with us your wisdom.

I’m impressed!! Really informative blog post here my friend. I just wanted to comment & say keep up the quality work. I’ve bookmarked your blog just now and I’ll be back to read more in the future my friend! Also nice colors on the layout, it’s really easy on the eyes.

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! I’m sure you had fun writing this article. Thank you

Wow I have read your article and by the way I found you website on AOL and I think after I read particularpost on you website especially this one I have my own opinion about what should I say on the next hang out with my boy friend, maybe today I will tell my familyabout this one and get debate.

Super-Duper web site! I’m loving it!! Will come back again – taking you feeds also, Thanks.

thanks for the inspiration I was stressed by work but i learnt that life is about living to the fullest and enjoying every moment.Thanks a million

Hello. Undoubtedly good job. I did not expect this on a Wednesday. This is a very good story. Thanks!

Great information! I’ve been looking for something like this for a while now. Thanks!

Really great informative blog post here and I just wanted to comment & thank you for posting this. I’ve bookmarked youi blog and I’ll be back to read more in the future my friend! Also nice colors on the layout, it’s really easy on the eyes.

nice site thanks for the link, nice monitor :) great design, well not bad service will take a look closer, well not bad site, not so many programs listed but looks quite great.

Monetizing websites using Mechanical Turk

July 14, 2010 in Science and Technology

Mechanical Turk, a system for crowd-sourcing complex ‘human intelligence tasks’ through parceling a large task and putting the parcels up on a marketplace, has seen a recent resurgence, partly on back of success of easier to work with clones like crowdflower.com.

Given that on the Internet people find it easier to part with labor than with money, one way to monetize websites may be to request users to help pay for the site by doing a task on Mechanical Turk, made easily available as part of the website. Websites can ask users to fill out the kind of tasks they would like to work on as part of their profiles.

For example, a news website may also provide its users a choice between watching ads and doing a small number of tasks for every X number of articles that they read. Similarly, a news website may ask its users to proofread a sentence or two of its own articles, thereby reducing costs of its production.

Some notable ideas

June 30, 2010 in General

Education of a senator

While top colleges have been known to buck their admissions criteria for the wrong reasons, they do on average admit smarter students, and the academic training they provide is easily above average. So it may be instructive to know what percentage of Senators, the most elite of the political brass, have graced the hallowed corridors of top academic institutions, and if at all the proportion attending top colleges varies by party.

Whereas 5 of the 42 Republican senators have attended top colleges (in the top 20), 22 of the 57 Democratic Senators have done the same. The skew in numbers may be due to the fact that New England is home to both top schools and a good many Democratic Senators. But privilege accorded by accident of geography is no less consequential than one afforded by some more equitable regime. As in if people of fundamentally similar caliber are there in Senate in either party, a belief cursory viewing of CSPAN would absolve one off, some due to happenstance of geography have been trained better than others.

Discounting elite education, one may simply want to look at extent to education. There one finds that whereas 73% of the Democratic Senators have an advanced degree, only 64% of Republican Senators have the same. While again it is likely that going to top schools increases admission to good advanced degree programs, thereby making advanced education more attractive perhaps, there again one can conjecture that whatever the reason – some groups of people, due to mere privilege perhaps, have better skills than others.

Why do telephones have numbers?
Unique alphabetical addresses could always be uniquely coded using numbers or translated as signals. However technological limitations meant that such coding wasn’t widely practiced. This meant people were forced to memorize multiple long strings of numbers, a skill most people never prided themselves with, and when forced to learn it, they took to it like fish to land. Freedom from such bondage would surely be welcome by many. And now, unsparing hands of technological change which have made such coding ubiquitous hope to deliver comeuppance to telephone numbers. Providing each phone with an alphabetical ID that maps to its IP address would be one smart way of implementing this change.

Misinformation and Agenda Setting
People actively learn from the environment, though they do so inexpertly. Given an average American consumes over 60 hrs of media each week, one prominent place from they learn is the media. Say if one were to watch a great many crime shows – merely as a result of the great many crime shows on television – one may impute wrongly that crime rate has been increasing. A similar, though more potent, example of the this may be the 70% of Americans who believe breast cancer is the number one reason for female mortality.

Proposal for a program in Political Communication

June 4, 2010 in Research

Forces that govern people’s behavior in politics are diverse, a diversity not always appreciated by scientists stuck in disciplinary bunkers. While occasional interdisciplinary philandering, by scientists otherwise faithful to their disciplines, has contributed enormously to our understanding of the topic by fruitfully leveraging knowledge across disciplines, formalizing such interdisciplinary training may prove to be a good catalyst for increasing this salutary (though non-virtuous) behavior.

Training for academia should include – logic, ethics (broadly philosophy), methodology, writing, teaching, skill in using tools for statistics (R), and typesetting (Latex), and other miscellaneous but important skills like project management, how to present, etc. In addition, a student needs training in the specific specialization.

Given the extent of training needed, a student needs both time, and strong mentoring.

Here below, I expand upon three particular aspects of training –

Methods Training
Methods training in Political Science, Communication, and Psychology (the three parent disciplines of Political Communication) is generally unsatisfactory, hobbled by incompetent teaching, if not incompetence. The only compensatory aspect is that the courses are fairly applied in nature. For firmer foundations in methodology, required for scholarship, training in Statistics Department is a sine-qua-non.
Courses: statistical inference, modeling and causal inference, stochastic methods, parametric and non-parametric analysis, bayesian analysis; Applied: time series analysis, data mining, sampling, programming, and optimization.

Statistics covers one part of methodology. A separate important part of methods include courses on measurement – what to measure and how best to measure it. Recommended courses: survey design, and psychological measurement.

A course devoted to content analysis may prove useful as well.

Content
There exist at least three fields that directly relate to Political Communication – Psychology, Communication, and Political Science.
Psychology: group psychology, social psychology, cognition, neuropsychology, evolutionary psychology.
Communication: News and Politics, Political Communication (an assimilative course), Political Economy of Media, Media and Communication.
Political Science: historical, institutional, theoretical, and behavioral aspects of politics.

Courses in law and sociology would be useful as well.

Mentoring
Regardless of the efforts to the contrary, there is still considerable variation in the students admitted. Students vary in their level of mental maturity, specific skills that they may excel in, etc. A proper and early assessment of weaknesses and strength of a student can allow the faculty to develop a specific plan crafted to address each. Directed reading courses in initial year(s) with one’s advisor provides an excellent opportunity for the student to learn, and for an advisor to address concerns above and beyond those discovered in reading.

Seminar series provide excellent places to learn from others – care in thinking, presentation skills, research questions, etc.

Personal milestone campaigners

May 24, 2010 in General

We have all read about beautiful women undressing to raise awareness about cruelty to animals (less handsome women just don’t care about animals), men and women strutting on runways wearing fashionable clothes to raise awareness about global warming, intrepid souls sailing across vast oceans to raise awareness about waste plastic in the sea, mountaineers climbing to raise awareness about cancer, and global warming, and the swarms of marathoners who run many times an year to raise awareness about breast cancer and of late, St. Jude’s Children Research Hospital. In this pantheon of heroes, now I humbly submit the ‘human polar bear’, who recently swam in a glacial lake in the Himalayas to raise awareness about global warming [BBC].

I often wonder how many things I would have remained in dark about had it not been for these heroic men and women, and the camera crews in toe. And then I mull over the contributions they have made to the world. It is work by such serious people and cameras following them that has led 76% of the people to believe that breast cancer is the single most deadly disease afflicting women (it is the fourth largest).

Not only do these men and women raise awareness, they raise money. The ‘acts’ they perform are now sponsored by corporations; the boosters have boosters. And all of these incredible personal achievements, which will only incidentally flood trophy cases and Facebook walls, all of these have been accumulated selflessly in service of some worthy cause (applause). It also reasons that the money and hours that these selfless volunteers spend on training, and recovering from training, for registering for races, and organizing events, pales in comparison to the money they raise.

The myth of knowledge

April 14, 2010 in Books, Epistemology, General

People frequently over-estimate how much they know. They also confidently share things they don’t know.

There are likely many reasons behind these tendencies. One of the reasons may be that often times lay conjecture, more colorfully known as ‘pulling something out of your ass’, in face of ignorance suffices as a substitute for knowledge; in fact such conjecture works well when narrated amongst uninformed co-partisans. More loosely, people make up stuff when they assess that there is little chance of getting caught. Formally, propensity to make up ‘something’ depends on the commenter’s assessment of audience’s information level, and its partisan affiliations about the topic under discussion, and the exact taint of the ‘fact’. Now as any professor will tell you – many students very confidently and happily make things up on the spot even though they are in presence of ‘experts’. So the rate of decline in propensity to make up stuff in presence of experts is low, and the absolute level of propensity to make up stuff high. It is also likely that the confidence with which people typically say such ‘lies’ is an attempt to cover up their ignorance for confidence is likely seen by others as a sign of surety about facts.

Thus never seriously threatened by their own ignorance, people build a somewhat more positive assessment of how much they know. People themselves rarely stop themselves from such ‘lying’. But why? ‘Lying’ may be an attempt to please, simply keep up one’s reputation, or show off, be seen as intelligent among peers, etc. Can it be that people think that lying about their own ignorance is only a minor transgression? Do people have this innate belief (which rarely gets challenged) that somehow when they speak, they will be able to be right; some sort of a ‘God bias’ – that they will be the exception to making sense without knowing?

We have hitherto given a negative account of motivation for making up facts which runs something like- if there is no police, people shall steal. This presupposes that people have an innate motivation to in fact make up stuff, and that the propensity to do so depends on lack of a monitor. But why is there this desire to appear make up facts? One reason simply is social desirability – motivation to appear knowledgeable. Another is that conversations are carried not typically for any epistemological purposes but for emotional and social purposes (Muhchyun Tang, personal communication). So we make up things because truth is unimportant and at times a hindrance. For example, envision a topical conversation where both parties profess their ignorance. Such conversation simply cannot happen. So fencing oneself within one’s realm of knowledge will typically disallow a variety of conversations about salient topics.

Modern era of knowledge production has brought its own challenges. As rate of knowledge production has exploded, so has the complexity of knowledge production. Increasingly substantive discussions about important areas of human activity (public policy more broadly but say health, fiscal policy etc.) need more sophisticated thought, and deeper immersion in the wealth of knowledge that has been produced. None of this should lead to an increasing tendency to make up stuff but conjecture based on partial and imperfect knowledge is perhaps increasing.

There are many likely consequences to such habits. One such is – once said, repeated many times, some of those ‘conjectures’, often wrong, in time, become ‘facts’.

It is often said, many a times without surprise, something of the order – ‘the more you know, more you realize how much you don’t know’. But why would that be? Are limits of knowledge so obscure to be not known by the proverbial (and now literal) average Joe? It is in fact easy to deduce one’s ignorance and comes down to honestly assessing our ignorance.

For example, we are surrounded by phenomenon we can’t describe well, much less explain. To infer our aggregate levels – one may do the following – recognize the fact that even in areas where we claim expertise we often fall short, hence we must really know very little about the things we don’t spend time learning. But it seems we shy away from taking account for such a task would easily and unambiguously reveal the limits of our knowledge.

The point about limits of our knowledge is broader and not there to malign the average Joe. There are real limits to what any human can achieve. Think about the following – If one were to read a book a week for the next 50 years, one would end up reading 2500 books. If the smallness of the number surprises you, then let it be a lesson in humility. This point allows me to segue into the next one and that is – ‘the myth of being well-read’.

The myth of being well-read

Many a times, people confuse being well-read to mean reading a few bad books poorly. To be well-read, one must satisfy three criteria – 1) read at the least 100-150 books; 2) a substantial majority, if not all, of which ought to be ‘good’ – literature or non-fiction; 3) and they ought to ‘read well’.

The 7 step program to correctly self-classifying oneself as ‘well-read’ or not ‘well-read’

  • Only a few people (<< 1%) are well-read. Do you think you are in such elite company? And then again, do you want to be in that company of ‘nerds’ and ‘geeks’?
  • Force yourself to list as many books that you have read in the past year (or life). If that number is less than 10, you may not be well-read. Apply this to specific areas as needed. For example, one may ask whether one is well read in Political Science.
  • Alternately, think hard about how many books you bought or checked out from library last year. If you haven’t spent more than $100 on books in the past year, and/or haven’t checked out more than 10 books from the library, you are unlikely to have read much.
  • If you catch yourself citing one book repeatedly, whenever the topic of books comes up during conversations, you haven’t read much. In U.S. it is typically ‘Catcher in the Rye’, though that may be changing – Harry Potter, Dan Brown, Twilight series, etc. may be the new ‘go to’ books; for a colleague of mine, it is ‘White Noise’; in some circles it may be some Malcolm Gladwell or Thomas Friedman book.
  • Relatedly, if Harry Potter, Blink, Atlas Shrugged, Catcher in the Rye etc., are among your favorite books, it is unlikely that you are well-read.
  • If you are younger than 21 (or typically 25), you couldn’t have read much. There are few exceptions. They help make the rule.
  • Count the books you own. If you don’t own more than 50 books, you are unlikely to have read much.

From the news

April 14, 2010 in General

Someone just got a bonus

Abercrombie and Fitch is to pay its Chief Executive $4m to limit reimbursed use of private jet to $200k/year till Feb. 2014, when the CEO’s contract expires [BBC].

CEO’s typical expenditure/year ~ 850k/year
Expected cost ~ $4.25m
Or
Max(previous years) = $1.1m
Expected cost ~ $5.5m

CEO’s reimbursed expenditure/year ~ 200k/year = .8m
Total cash ~ $4m* + .8m = $4.8m
* CEO makes money on the interest by getting the money upfront. Total income at about 5%/year compounded annually ~ $1.1m

Profits ~ 4.8 + 1 – 4.25 = 1.55m
(or)
5.8 – 5.5 = .3m*

*Very likely a low estimate as CEO now has the incentive to fly the plane less.

Science about the opinions towards climate science
Some plausible explanations (not exculpations) for forces shaping public opinion about climate science -

  • “When it comes to climate, academic scientists are jigsaw types, dissenters from their view house-of-cards-ists.”(The Economist). Same article -”People often assume that data are simple, graspable and trustworthy, whereas theory is complex, recondite and slippery, and so give the former priority. In the case of climate change, as in much of science, the reverse is at least as fair a picture. Data are vexatious; theory is quite straightforward.”
  • ’Weather is not climate’ (NY Times)

Lifting the veil on some issues in the ‘Burka debate’

March 30, 2010 in Islam, Politics

For the unfamiliar, the BBC guide to Muslim veils.

The somewhat polemical:
Assuming ‘God’ has recommended or even ordered that women wear burka, assuming that burka has no impact on a woman’s ability to communicate or quality of life, as has been suggested by its supporters, then here’s a suggestion – to all men, who haven’t been ordered by ‘God’ to wear burka, and who don’t see a downside to wearing it, why not voluntarily commit to wearing the burka, since no law opposes such a voluntary act, to show solidarity with the women. My sense is that even the French would come to support the burka if men en masse chose to wear it.

More considered:
“The interior ministry says only 1,900 women wear full veils in France, home to Europe’s biggest Muslim minority” (BBC). If the problem is interpreted solely in terms of women wearing the veil, then it is much smaller than the dust in its wake.

There are three competing concerns at the heart of the debate – Protecting rights of women who voluntarily want to wear it, protecting rights of women who are forced to wear it, and protecting (French) ‘culture’. Setting aside ‘cultural’ concerns for the moment, let’s focus on the first two claims.

People are incredulous of the claim that women will voluntarily choose to wear something so straight forwardly unpleasant. Even when confronted with a woman who claims to comply voluntarily, they fear coercion, or something akin to brainwashing at play. There is merit to the thought. However there is much evidence that people do many unpleasant things voluntarily – such as wear high-heels (which may also be seen as ‘coercive’). So it is very likely indeed that there is ‘voluntary compliance’ by some women.

Assuming there exist voluntary compliers, and ones forced to wear the niqab, wouldn’t it be pleasant if we could ensure the rights of both? In fact, doesn’t the extant legal framework provide for such a privilege already? Yes and No; mostly no. While it is true that women forced to wear the niqab can petition the police, it is unlikely to happen for a variety of reasons – going to the police would mean going against the family, which may mean – doing something painful, and risking financial and physical well-being. Additionally the laws governing such ‘coercion’ are likely to carry modest penalties, and unlikely to redress the numerous correlated issues including inadequate financial, and educational opportunities. Many of the issues raised here would seem familiar to people working with domestic abuse, and they are, and the modern state hasn’t (tried to) found a good solution.

Perhaps both camps will agree that wearing a niqab does dramatically limit the career opportunities for women. Of course people in one of the camps may be happy that there are limits to such opportunities but let’s assume that they would be happy if the women had the same opportunities. Part of the problem here then is the norms of dressing in business environments in the West. Entrepreneurs in Saudi Arabia recently brought to air a television talk show in which both of the hosts wore the niqab. The entire effect was disturbing. However that isn’t the point. The point is that there may be ways to not reduce career opportunities for women based on the dress code, which after all is somewhat pointless.

Time considerations mean a fuller consideration on the issue will have to wait. One last point – One of the problems cited about the burka is that it poses a security threat, which has some merit, given its long history in being used a method of escape, including by militant clerics.

Opening interfaces to websites and software

March 20, 2010 in General, Science and Technology

Imagine the following scenario – You go to NYTimes.com, and are offered a choice between variety of interfaces – not just ‘skins’ or font adjustments – built for a variety of purposes by whoever wants to put in the effort. You get to pick the interface that is right for you – carries the stories you like, presented in the fashion you prefer. Wouldn’t that be something?

Currently, we are made to choose between using weak customization options, or build some ad hoc ‘interface’ using RSS readers. What is missing is open-sourced or internally produced selection of interfaces that cater to diverse needs, and wants of the public.

We need to separate data from the interface. Applying it to the example at hand – NY Times is in the data business, not the interface business. So like Google Maps, it can make data available, with some stipulations, including monetization requirements, and let others take charge of creatively thinking of ways that data can be presented. If this seems too revolutionary, more middle of the road options exist. NY Times can allow people to build interfaces which are then made available on the New York Times site. For monetization, NY Times can reserve areas of real estate, or charge users for using some ad free interfaces.

This trick can be replicated across websites, and easily extended to software. For example, MS-Excel can have a variety of interfaces, all easily searchable, downloadable, and deployable, that cater to specific needs of say, Chemical Engineers, or Microbiologists, or programmers. The logic remains the same – MS needn’t be in the interface business, or more limitedly, doesn’t need to control it completely or inefficiently (for it does allow tedious customization), but can be a platform on which people can build, and share, innovative ways to exploit the underlying engine.

An adjacent broader and more useful idea is to come up with a rich interface development toolkit that provides access to processed open data.

Polling our theories: Learning from the pollsters

March 4, 2010 in Research

For all those who cast aspersions on Social Science’s ability to produce valid replicable findings, they need look no further than pollsters (election polling) in US, who have near perfected the ability to produce accurate results, except of course Rasmussen, which has perfected the art of producing reliably Republican findings.

But how are pollsters able to do that with samples of 1000, samples which if collected naively, vary in the estimates of truth enough to render the estimates pointless? The short answer is that they utilize knowledge about how people behave, and how they are distributed in the population, and adjust the results based on that knowledge. Given their ability to do so with great accuracy, it is likely that pollsters know more about how people vote, why they vote, etc. than political scientists. It is also likely however that social science will remain somewhat unaware of the results as most of the knowledge will be proprietary.

Using these techniques in ‘hypothesis testing’
The traditional (frequentist) ‘model’ of hypothesis testing has shied away from utilizing knowledge about the population. Typically, multiple parameters are estimated simultaneously, using ‘regression’ or any of its sibling methods. Bayesian of course embrace the concept of prior knowledge, though typically shy away from utilizing it fully. One modest and somewhat defensible way to test theories would be to ‘fix’ relation of variables with others, using prior knowledge. So in the domain of voting, one can get away from vagaries of sampling, and directly ‘fix’ black respondents from voting 90% for Democrats with modest decreasing propensity given income. This shrinkage of variance from modeling part of data, or theory, would allow for other parameters from being estimated more reliably.