What is the trend over the last X months? One estimate of the ‘trend’ over the last k time periods is what I call the ‘hold up the ends’ method. Look at t_k and t_0, get the difference between the two, and divide by the number of time periods. If t_k > t_0, you say that things are going up. If t_k < t_0, you say things are going down. And if they are the same, then you say that things are flat. But this method can elide over important non-linearity. For instance, say unemployment went down in the first 9 months and then went up over the last 3 but ended with t_k < t_0. What is the trend? If by trend, we mean average slope over the last t time periods, and if there is no measurement error, then 'hold up the ends' method is reasonable. If there is measurement error, we would want to smooth the time series first before we hold up the ends. Often people care about 'consistency' in the trend. One estimate of consistency is the following: the proportion of times we get a number of the same sign when we do pairwise comparison of any two time consecutive time periods. Often people also care more about later time periods than earlier time periods. And one could build on that intuition by weighting later changes more.
Say you make k products and have a list of potential customers. Assume that you can show each of these potential customers one ad. And that showing them an ad costs nothing. Assume also that you get profits p_1, …, p_k from each of the k products. What is the optimal targeting strategy?
The goal is to maximize profit. If you didn’t know the probability each customer will buy a product if shown an ad or assume it to be the same across products, then the strategy reduces to showing the ad for the most profitable product to everyone.
If we can estimate the probability the customer will buy each product if they are shown an ad for the product well, we can do better. (We assume that customers won’t buy the product if they don’t see the ad for it.)
When k = 1, the optimal strategy is to target everyone. Now let’s solve it for when k = 2. A customer can either buy k_0 or k_1 or buy nothing at all.
If everyone is shown k_0, the profits are p_0*prob_i_0_true where prob_i_0_true gives the true probability of the ith person buying k_0 when shown an ad for it. And if everyone is shown k_1 ad, the profits are p_1*prob_i_1_true. The net utility calc. for each customer = p_1*prob_i_1 – p_0*prob_i_0. (We generally won’t know prob_i but would estimate it from data and hence the subscript true is not there.) You can recover two numbers from this. One is how to target—pick whichever number is bigger for each customer. Another is in what order to target in: sort by absolute value.
Calculating Benefits of Targeting
If you don’t target, the best case scenario is that you earn the bigger of the two numbers: p_1*prob_i_1_true, p_0*prob_i_0_true
If you target well (estimate probabilities well), you will recover something that is as good or better than that.
Since we generally won’t have the true probabilities, the best way to estimate the benefit of targeting is via A/B testing. But if you can’t do that, one estimate =
No Targeting estimate = Larger of p_1*prob_i_1, p_0*prob_i_0
Targeting estimate = Sum over all i
p_1*prob_i_1 if p_1*prob_i_1 > p_0*prob_i_0
p_0*prob_i_0 if p_1*prob_i_1 < p_0*prob_i_0
Similar calculations can be done for deriving estimates of the value of better targeting.
“What is crucial for you as the writer is to express your opinion firmly,” writes William Zinsser in “On Writing Well: An Informal Guide to Writing Nonfiction.” To emphasize the point, Bill repeats the point at the end of the paragraph, ending with, “Take your stand with conviction.”
This advice is not for all writers—Bill particularly wants editorial writers to write with a clear point of view.
When Bill was an editorial writer for the New York Herald Tribune, he attended a daily editorial meeting to “discuss what editorials … to write for the next day and what position …[to] take.” Bill recollects,
“Frequently [they] weren’t quite sure, especially the writer who was an expert on Latin America.
“What about that coup in Uruguay?” the editor would ask. “It could represent progress for the economy,” the writer would reply, “or then again it might destabilize the whole political situation. I suppose I could mention the possible benefits and then—”
The editor would admonish such uncertainty with a curt “let’s not go peeing down both legs.”
Bill approves of taking a side. He likes what the editor is saying if not the language. He calls it the best advice he has received on writing columns. I don’t. Certainty should only come from one source: conviction born from thoughtful consideration of facts and arguments. Don’t feign certainty. Don’t discuss concerns in a perfunctory manner. And don’t discuss concerns at the end.
Surprisingly, Bill agrees with the last bit about not discussing concerns in a perfunctory manner at the end. But for a different reason. He thinks that “last-minute evasions and escapes [cancel strength].”
Don’t be a mug. If there are serious concerns, don’t wait until the end to note them. Note them as they come up.
In 2010, Google estimated that approximately 130M books had been published.
As a species, we still know very little about the world. But what we know already far exceeds what any of us can learn in a lifetime.
Scientists are acutely aware of the point. They must specialize, as chances of learning all the key facts about anything but the narrowest of the domains are slim. They must also resort to shorthand to communicate what is known and what is new. The shorthand that they use is—citations. However, this vital building block of science is often rife with problems. The three key problems with how scientists cite are:
1. Cite in an imprecise manner. This broad claim is supported by X. Or, our results are consistent with XYZ. (Our results are consistent with is consistent with directional thinking than thinking in terms of effect size. That means all sorts of effects are consistent, even those 10x as large.) For an example of how I think work should be cited, see Table 1 of this paper.
2. Do not carefully read what they cite. This includes misstating key claims and citing retracted articles approvingly (see here). The corollary is that scientists do not closely scrutinize papers they cite, with the extent of scrutiny explained by how much they agree with the results (see the next point). For a provocative example, see here.)
3. Cite in a motivated manner. Scientists ‘up’ the thesis of articles they agree with, for instance, misstating correlation as causation. And they blow up minor methodological points with articles whose results their paper’s result is ‘inconsistent’ with. (A brief note on motivated citations: here).
We often make causal claims based on fallible heuristics. Some of the heuristics that we commonly use to make causal claims are:
- Selecting on the dependent variable. How often have you seen a magazine article with a title like “Five Habits of Successful People”? The implicit message in such articles is that if you were to develop these habits, you would be successful too. The articles never discuss how many unsuccessful people have the same habits or all the other dimensions on which successful and unsuccessful people differ.
- Believing that correlation implies causation. A common example goes like this: children who watch more television are more violent. From this data, people deduce that watching television causes children to be violent. It is possible, but there are other potential explanations.
- Believing that events that happen in a sequence are causally related. B follows A so A must cause B. Often there isn’t just one A, but lots of As. And the B doesn’t instantaneously follow A.
Beyond this, people also tend to interpret vague claims such as X causes Y as X causes large changes in Y. (There is likely some motivated aspect to how this interpretation happens.)
Why do many people think that people on the other side are not well motivated? It could be because they think that the other side is less moral than them. And since opprobrium toward the morally defective is the bedrock of society, thinking that the people in the other group are less moral naturally leads people to censure the other group.
But it can’t be that two groups simultaneously have better morals than the other. It can only be that people in the groups think they are better. This much logic dictates. So, there has to be a self-serving aspect to moral standards. And this is what often leads people to think that the other side is less moral. Accepting this is not the same as accepting moral relativism. For even if we accept that some things are objectively more moral—not being sexist or racist say—some groups—those that espouse that a certain sex is superior or certain races are better—will still think that they are better.
But how do people come to know of other people’s morals? Some people infer morals from political aims. And that is a perfectly reasonable thing to do as political aims reflect what we value. For instance, a Republican who values ‘life’ may think that Democrats are morally inferior because they support the right to abortion. But the inference is fraught with error. As matters stand, Democrats would also like women to not go through the painful decision of aborting a fetus. They just want there to be an easy and safe way for women should they need to.
Sometimes people infer morals from policies. But support for different policies can stem from having different information or beliefs about causal claims. For instance, Democrats may support a carbon tax because they believe (correctly) the world is warming and because they think that the carbon tax is what will help reduce global warming the best and protect American interests. Republicans may dispute any part of that chain of logic. The point isn’t what is being disputed per se, but what people will infer about others if they just had information about the policies they support. Hanlon’s razor is often a good rule.
‘Why do people (re)-elect bad leaders?’ used to be a question that people only asked of third-world countries. No more. The recent election of unfit people to prominent positions in the U.S. and elsewhere has finally woken some American political scientists from their mildly racist reverie—the dream that they are somehow different.
So why do people (re)-elect bad leaders? One explanation that is often given is that people prefer leaders that share their ethnicity. The conventional explanation for preferring co-ethnics is that people expect co-ethnics (everyone) to do better under a co-ethnic leader. But often enough, the expectation seems more like wishful thinking than anything else. After all, the unsuitability of some leaders is pretty clear.
If it is wishful thinking, then how do we expose it? More importantly, how do we fix it? Let’s for the moment assume that people care about everyone. And if they were to learn that the co-ethnic leader is much worse than someone else, they may switch votes. But what if people care about the welfare of co-ethnics more than others? The ‘good’ thing about bad leaders is that they are generally bad for everyone. So, if they knew better, they would still switch their vote.
You can verify these points using a behavioral trust game where people observe allocators of different ethnicities and different competence, and also observe welfare of both co-ethnics and others. You can also use the game to study some of the deepest concerns about ‘negative party ID’—that people will harm themselves to spite others.
It has been nearly 25 years since the publication of Benjamin Barber’s Jihad Vs. McWorld. So how does the search volume for McDonald’s and jihad compare in Pakistan? Look here for the answer. (I know Barber’s McWorld != McDonald’s.)
For fun, I also looked into search volume for sex and jihad in Pakistan.
It has been nearly five years since the publication of Affect, Not Ideology: A Social Identity Perspective on Polarization. In that time, the paper has accumulated over 450 citations according to Google Scholar. (Citation counts on Google Scholar tend to be a bit optimistic.) So how does the paper hold up? Some reflections:
- Disagreement over policy conditional on aims should not mean that you think that people you disagree with are not well motivated. But regrettably, it often does.
- Lack of real differences doesn’t mean a lack of perceived differences. See here, here, here, and here.
- The presence of real differences is no bar to liking another person or group. Nor does a lack of real differences come in the way of disliking another person or group. History of racial and ethnic hatred will attest to the point. In fact, why small differences often serve as durable justifications for hatred is one of the oldest and deepest questions in all of social science. (Paraphrasing from Affectively Polarized?.) Evidence on the point:
- Sort of sorted but definitely polarized
- Assume partisan identity is slow moving as Green, Palmquist, and Schickler (2002) among others show. And then add to it the fact people still like their ‘own’ party a fair bit—thermometer ratings are a toasty 80 and haven’t budged. See the original paper.
- People like ideologically extreme elites of the party they identify with a fair bit (see here).
- It may seem surprising to some that people can be so angry when they spend so little time on politics and know next to nothing about it. But it shouldn’t be. Information generally gets in the way of anger. Again,
the history of racial bigotry is a good example.
- The title of the paper is off in two ways. First, partisan affect can be caused by ideology. Not much of partisan affect may be founded in ideological differences, but at least some of it is. (I always thought so.) Secondly, the paper does not offer a social identity perspective on polarization.
- The effect that campaigns have on increasing partisan animus is still to be studied carefully. Certainly, ads play but a small role in it.
- Evidence on the key take-home point—that partisans dislike each other a fair bit—continues to mount. The great thing is that people have measured partisan affect in many different ways, including using IAT and trust games. Evidence that IAT is pretty unreliable is reasonably strong, but trust games seem reasonable. Also see my 2011 note on measuring partisan affect coldly.
- Interpreting over-time changes is hard. That was always clear to us. But see Figure 1 here that controls for a bunch of socio-demographic variables, and note that the paper also has over-time cross-country to clarify inferences further.
- If you assume that people learn about partisans from elites, reasoning what kinds of people would support this ideological extremist or another, it is easy to understand why people may like the opposing party less over time (though trends among independents should be parallel). The more curious thing is that people still like the party they identify with and approve of ideologically extreme elites of their party (see here).