The Internet is for porn (Avenue Q). So it makes sense to measure things on the Internet in porn units.
I jest, just a bit.
In Everybody Lies, Seth Stephens Davidowitz points out that people search for porn more than weather on GOOG. Data from Google Trends for the disbelievers.
But how do searches for news fare? Surprisingly well. And it seems the new president is causing interest in news to outstrip interest in porn. Worrying, if you take Posner’s point that people’s disinterest in politics is a sign that they think the system is working reasonably well. The last time searches for news > porn was when another Republican was in the White House!
How is the search for porn affected by Ramadan? For answer, we turn to Google Trends from Pakistan. But you may say that the trend is expected given Ramadan is seen as a period for ritual purification. And that is a reasonable point. But you see the same thing with Eid-ul-Fitr and porn.
Remote teams live and die by the phone meeting. So how can we prevent death? Channel Rumsfeld. Begin with known knowns of conducting a successful meeting: having a clear agenda, a discussion leader, and ending with a summary. Do those well. Then try out some ideas to address well-known challenges: disengagement, social friction, exclusion, time wastage, and inability to follow what’s going on.
People on the phone can’t always tell between the two uses of brief silence—a brief pause, and signal for opening the floor for discussion. The speaker can address the issue by signaling the end of speech with a phrase such as ‘I am done.’ A speaker may start the speech by noting: ‘at the end of what I have to say, I will formally open up the floor and go around alphabetically among those whose speakers are unmuted.’
Having ‘too many’ people = wasting people’s time + disinterested participants. How many is too many? The maximum number of people who can productively engage when everyone is expected to contribute is probably as low as 4–5. What do you do when you have a large team? Divide and conquer. Split people into small teams and share notes.
Prevent or Cure Rambling as side effects are the same as above—time wastage and disinterest. If people are having trouble articulating, the discussion leader should take on the responsibility to energetically understand the point people are trying to get at. The discussion leader may also refer the person to the shared document to sketch out the idea and try again.
Stuff in advance that everyone actually reads is important. Just tell people if you didn’t find time to read + independently think, just opt out (semi-private opt-outs with emails to meeting organizers should be allowed). The job of a meeting is not spoon feeding.
Keeping people on the same page:
Visual aids, e.g. slides, are useful in bringing people on the same page.
Taking notes on a shared screen can also help see people that progress is being made. The document can be shared and that allows others to contribute and organize simultaneously.
Most importantly, avoid meetings when you can. If the aim of meeting = transferring information, it only makes sense to have a meeting < 10% of the times. Alternatives = write out a document, or create a slideshow or a video and send it along.
A lot of people have their lives cut short because they eat too much and exercise too little. Worse, the quality of their shortened lives is typically much lower as a result of avoidable' illnesses that stem frombad behavior.’ And that isn’t all. People who are not feeling great are unlikely to be as productive as those who are. Ill-health also imposes a significant psychological cost on loved ones. The net social cost is likely enormous.
One way to reduce such costly avoidable misery is to invest upfront. Teach people good habits and psychological skills early on, and they will be less likely to self-harm.
So why do we invest so little up front? Especially when we know that people are ill-informed (about the consequences of their actions) and myopic.
Part of the answer is that there are few incentives for anyone else to care. Health insurance companies don’t make their profits by caring. They make them by investing wisely. And by minimizing ‘avoidable’ short-term costs. If a member is unlikely to stick with a health plan for life, why invest in their long-term welfare? Or work to minimize negative externalities that may affect the next generation?
One way to make health insurance care is to rate them on estimated quality-adjusted years saved due to interventions they sponsored. That needs good interventions and good data science. And that is an opportunity. Another way is to get the government to invest heavily early on to address this market failure. Another version would be to get the government to subsidize care that reduces long-term costs.
Dissimilarity index is a measure of segregation. It runs as follows:
is population of in the ith area
is population of in the larger area
from which dissimilarity is being measured against
The measure suffers from a couple of issues:
Concerns about lumpiness. Even in a small area, are black people at one end, white people at another?
Choice of baseline. If the larger area (say a state) is 95\% white (Iowa is 91.3% White), dissimilarity is naturally likely to be small.
One way to address the concern about lumpiness is to provide an estimate of the spatial variance of the quantity of interest. But to measure variance, you need local measures of the quantity of interest. One way to arrive at local measures is as follows:
Create a distance matrix across all addresses. Get latitude and longitude. And start with Euclidean distances, though smart measures that take account of physical features are a natural next step. (For those worried about computing super huge matrices, the good news is that computation can be parallelized.)
For each address, find n closest addresses and estimate the quantity of interest. Where multiple houses are similar distance apart, sample randomly or include all. One advantage of n closest rather than addresses in a particular area is that it naturally accounts for variations in density.
But once you have arrived at the local measure, why just report variance? Why not report means of compelling common-sense metrics, like the proportion of addresses (people) for whom the closest house has people of another race?
As for baseline numbers (generally just a couple of numbers): they are there to help you interpret. They can be brought in later.
When strapped for time, some resort to wishful thinking, others to lashing out. Both are illogical. If you are strapped for time, it is either because you scheduled poorly or because you were a victim of unanticipated obligations. Both are understandable, but neither justify ‘acting out.’ So don’t.
Whatever the reason, being strapped for time means either that some things won’t get done on time, or that you will have to work harder, or that you will need more resources (yours or someone else’s), or all three. And the only things to do are:
Ask for help, and
Communicate effectively to those affected
If you have landed in soup because of poor scheduling, for instance, by not budgeting any time to deal with things you haven’t scheduled, make a note. And improve.
And since it is never rational to worry—it is at best unproductive, and at worst corrosive—avoid it like plague.
How can fallible creatures like us know something? The scientific method is about answering that question well. To answer the question well, we have made at least three big innovations:
1. Empiricism. But no privileged observer. What you observe should be reproducible by all others.
2. Open to criticism: If you are not convinced about the method of observation, the claims being made, criticize. Offer reason or proof.
3. Mathematical Foundations: Reliance on math or formal logic to deduce what claims can be made if certain conditions are met.
These innovations along with two more innovations have allowed us to ‘scale.’ Foremost among the innovations that allow us to scale is our ability to work together. And our ability to preserve information on stone, paper, electrons, allows us to collaborate with and build on the work done by people who are now dead. The same principle that allows us to build as gargantuan a structure as the Hoover Dam and entire cities allows us to learn about complex phenomenon. And that takes us to the final principle of science.
Peers are equals, except as reviewers, when they are more like capricious dictators. (Or when they are members of a peerage.)
We review our peers’ work because we know that we are all fallible. And because we know that the single best way we can overcome our own limitations is by relying on well-motivated, informed, others. We review to catch what our peers may have missed, to flag important methodological issues, to provide suggestions for clarifying and improving the presentation of results, among other such things. But given a disappointingly long history of capricious reviews, authors need assurance. So consider including in the next review a version of the following note:
Reviewers are fallible too. So this review doesn’t come with the implied contract to follow all ill-advised things or suffer. If you disagree with something, I would appreciate a small note. But rejecting a bad proposal is as important as accepting a good one.
Talk to customers, analyze usage data, talk to peers, managers, think, hold competitions, raffles,… For each idea:
Define the idea
Write out the idea for clarity — at least 5–10 sentences. Do some due diligence to see what else is there. Learn and revise (including abandon).
Does the idea make business sense? Does it make sense to the developers (if the idea is mechanistic or implementation related)?
Ideally, peer review. But at the minimum: Talk about your idea with the manager(s) and the developers. If both manager(s) and developer(s) agree (for some mechanistic things, only developers need to agree), move to the next step.
This is optional, and for major, complex innovations that can be easily prototyped only. Write code. Produce Results. Does it make sense? Peer review, if needed. If not, abandon. If it does, move to the next step
Write the specifications
Spend no less than 20% of the entire development time on writing the specs, including proposed functions, options, unit tests, concerns, implications. Run the specifications by developers, get this peer reviewed, improve, and finalize.
Set Priority and Release Target
Talk to the manager about the priority order of the change, and assign it to a particular release cycle.
Who Does What by When?
Create JIRA ticket(s)
General cycle = code, test -> peer review -> code, test -> peer review …
MVP of Expected Code or Aspects of good code: ‘Code your documentation’ (well-documented), modular, organized, tested, nice style, profiled. Do it once, do it well.
In answering a question, scientists sometimes collect data that answers a different, sometimes yet more important question. And when that happens, scientists sometimes overlook the easter egg. This recently happened to me, or so I think.
Kabir and I recently investigated the extent to which estimates of motivated factual learning are biased (see here). As part of our investigation, we measured numeracy. We asked American adults to answer five very simple questions (the items were taken from Weller et al. 2002):
If we roll a fair, six-sided die 1,000 times, on average, how many times would the die come up as an even number? — 500
There is a 1% chance of winning a $10 prize in the Megabucks Lottery. On average, how many people would win the $10 prize if 1,000 people each bought a single ticket? — 10
If the chance of getting a disease is 20 out of 100, this would be the same as having a % chance of getting the disease. — 20
If there is a 10% chance of winning a concert ticket, how many people out of 1,000 would be expected to win the ticket? — 100
In the PCH Sweepstakes, the chances of winning a car are 1 in a 1,000. What percent of PCH Sweepstakes tickets win a car? — .1%
The average score was about 57%, and the standard deviation was about 30%. Nearly 80% (!) of the people couldn’t answer that 1 in a 1000 chance is .1% (see below). Nearly 38% couldn’t answer that a fair die would turn up, on average, an even number 500 times every 1000 rolls. 36% couldn’t calculate how many people out of a 1,000 would win if each had a 1% chance. And 34% couldn’t answer that 20 out of 100 means 20%.
If people have trouble answering these questions, it is likely that they struggle to grasp some of the numbers behind how the budget is allocated, or for that matter, how to craft their own family’s budget. The low scores also amply illustrate that the education system fails Americans.
Given the importance of numeracy in a wide variety of domains, it is vital that we pay greater attention to improving it. The problem is also tractable — with the advent of good self-learning tools, it is possible to intervene at scale. Solving it is also liable to be good business. Given numeracy is liable to improve people’s capacity to count calories, make better financial decisions, among other things, health insurance companies could lower premiums in lieu of people becoming more numerate, and lending companies could lower interest rates in exchange for increases in numeracy.