This year my whole country was very excited when the Royal Swedish Academy of Sciences announced the Nobel Prize winners, because a Hungarian biochemist, Katalin Karikó was also listed among the possible laureates, moreover in two categories, in medicine and in chemistry. Even if she didn’t win this year, we are very proud of the Hungarian developer of mRNA vaccine, and according to the president of the Hungarian Academy of Sciences, we have no reason to feel disappointed, because in the field of medicine the evaluation of the Swedish Academy always takes more time.
As an evaluator, an economist and a sociologist, however I still had a reason for being happy, because in 2021 the three laureates in Economic Sciences were scholars, whose remarkable work is very important for empirical researchers. David Card, Joshua Angrist and Guido Imbens received the Nobel Prize for building the foundations of “natural experiments” in economic research. The official announcement says that the Royal Swedish Academy of Sciences awarded the prize with one half to David Card “for his empirical contributions to labour economics” and the other half jointly to Joshua D. Angrist and Guido W. Imbens “for their methodological contributions to the analysis of causal relationships”.
Let’s see in more detail, what they achieved, and why their work is so important for economists, sociologists and program evaluators.
In order to understand the greatness of natural experiments, we have to understand the conception of randomized control trial (RCT), or true experiment that are considered the gold standard of uncovering causal relationships in many fields. In this research design, the participants are assigned randomly to the experimental and the control groups, and the researcher has high control over the circumstances and over the extraneous and independent variables. The advantage of RCT is that the procedure is easily replicable and randomization solves serious statistical issues, like handling alternative explanations and lurking variables. Randomization is critical in isolating the treatment effect since it makes the experimental and control groups similar in terms of any other factors. This interventional design (e.g. laboratory experiment) is prevalent in psychological sciences however cannot be always used for investigating an intervention due to ethical, financial or practical reasons (randomization is not always possible). One way to avoid these issues is to perform a quasi-experiment, however, in lack of randomization, we cannot be sure, that any difference in the outcome occurs due to the treatment, because the treatment and control groups may have systematic deviations at the baseline. Another way is to collect observational data without controlled experimental variation, however in this case a fundamental problem arises: the underlying cause of any correlation remains unclear.
And here comes the revolutionary approach: this year’s laureates have shown that it is possible to unveil cause-effect relationships using natural experiments. In this research design, the units of analysis are distributed based on randomization caused by nature, institutions, or policy changes, making it possible to have “clean identification” of causal mechanisms. Referring to one of the papers of Joshua Angrist, let me mention an example, that I also bring to my program evaluation seminars:
An important question in economic research is what determines earnings. Angrist (1990) evaluated the effects of military service on lifetime earnings.
In 1969, the government of the United States conducted two draft lotteries to determine the order of call to military service in the Vietnam War for men born between 1944 and 1950. Angrist capitalized on the approximate random assignment of the lottery, and used it as an instrumental variable associated with eligibility (or non-eligibility) for military service. The draft lottery gave an opportunity to a natural experiment, and those drafted into the military could be compared to those not drafted. Angrist found that veterans earned 15 percentage points less on average compared to non-veterans.
We can see, that although natural experiment is a rare option, it can be very fruitful. Unlike laboratory experiment, behavior in a natural experiment is more likely to reflect real life because of its natural setting, thus it results in very high ecological validity. In addition, demand characteristics don’t create bias, as participants are not aware they are being studied, and it can be used in situations in which it would be ethically unacceptable to manipulate the independent variable. The key is here to use situations in which chance events or policy changes result in similar groups of people being treated differently. Card and Krueger (1994) used natural experiment, with data from two neighboring states in the US — one in which the minimum wage was increased. In an other study, Angrist and Krueger (1991) compared people born in the first and fourth quarters of the year, and saw that the first group had, on average, spent less time in education. Because chance decides exactly when a person is born, Angrist and Krueger (now deceased) were able to use this natural experiment to establish a causal relationship showing that more education leads to higher earnings.
Joshua Angrist and Guido Imbens showed how natural experiments can be used to find clear cause-effect relationships in complex social settings, while, David Card used this approach to analyze the effect of minimum wage, immigration and education on labor markets. According to the Swedish Academy, his studies from the early 1990s challenged conventional wisdom, leading to new analyses and additional insights. Among other things, his results showed, that increasing the minimum wage does not necessarily lead to fewer jobs.
Literature:
Angrist, Joshua D. (1990). “Lifetime Earnings and the Vietnam Draft Lottery: Evidence from Social Security Administrative Records”. American Economic Review. 80 (3): 313–336. JSTOR 2006669.
Angrist, J.D. and A.B. Krueger (1991). “Does compulsory schooling attendance affect schooling and earnings?” Quarterly Journal of Economics, 106: 976-1014.
Card, D. and A.B. Krueger (1994). “Minimum wages and employment: A case study of the fastfood industry in New Jersey and Pennsylvania.” American Economic Review, 84: 772-784.