During the 2008 Democratic primary, when Barack Obama and Hillary Clinton were in a heated, neck-and-neck contest for their party’s nomination, Professors Leonard Lee and Michel Pham, along with Andrew Stephen, PhD ’09, now of the University of Pittsburgh, were working on a project related to emotional decision making. They wondered, if people were to simply trust their feelings, to what extent would they be able to predict the outcome of the primaries?
Researchers have almost always approached forecasting and making predictions as a rational, systematic process of studying past trends and data and extrapolating into the future. “So looking at how emotions contribute to forecasting represents a real divergence from the typical approach,” Lee says, “even as more and more evidence shows that people do rely quite a lot on their feelings in making day-to-day decisions and judgments and that this may be a good thing.” Pham, for instance, has shown that trusting feelings when negotiating can mean coming out on the winning side more often. Lee has shown that people are more consistent in choosing among products when they rely more on their feelings to guide their decision.
The researchers conducted a series of experiments in which their participants were asked to predict (a) the 2008 US Democratic presidential nominee, (b) the box-office success of different movies, (c) the winner of American Idol, (d) movements of the Dow Jones Index, (e) the winner of a championship football game, and (f) the weather.
Despite the range of events to be predicted — from the winner of the presidential nomination to movements of the Dow Jones and even the weather — the results across all experiments were remarkably consistent: people who were prompted to have high trust in their feelings were more likely to correctly predict the eventual outcomes than control participants and those who were prompted to trust their feelings less. In the case of the Clinton-Obama contest, high-trust respondents predicted correctly for Obama about 72 percent of the time compared with low-trust respondents, who predicted correctly for Obama about 64 percent of the time — a particularly striking result because at the time of the experiment the eventual winner was far from certain: all major polls reflected a very tight race between the two candidates. For the winner of American Idol, the difference was 41 percent of high-trust respondents compared to 24 percent for low-trust respondents. When individuals were asked to predict what the weather in their own zip code would be in two days, those with high trust in their feelings did so correctly 54 percent of the time, compared with 21 percent of the time for individuals with low trust in their feelings.
One of the most notable results came when the researchers asked participants to predict future levels of the Dow Jones, an area where rational thinking — rather than emotion —is highly favored. High-trust individuals made predictions that were roughly 30 percent closer to the levels the Dow Jones actually moved than low-trust individuals. This was true for both March 2009, a volatile period in the US economy, and during the fall of 2010, when the US economy had entered a recovery phase.
The results were also consistent under two very different methods that the researchers used to influence or measure how much or how little individuals relied on their feelings to make their predictions. With some participants, the researchers used an increasingly standard trust-in-feelings manipulation originally developed by Pham and Tamar Avnet, PhD ’04, of Yeshiva University, based on a methodology by Norbert Schwarz of the University of Michigan and his colleagues: participants are asked to think of two instances in which they trusted their feelings in making a decision that had a good outcome — a relatively easy task that typically prompts people to trust their feelings. Alternately, participants are asked to think of 10 such instances — a much harder task that typically undermines the degree to which people trust their feelings. The second technique was much more straightforward: after asking for predictions, the researchers simply asked participants how much they typically rely on their feelings in general when making predictions. Regardless of the method used, participants who trusted their feelings in general or were experimentally induced to trust their feelings were more accurate in their predictions compared to participants with lower trust in their in their feelings and participants in a control group.
At the start of the experiments, the researchers considered two different explanations for the accuracy of the predictions. “When it came to the presidential nominations and the winner of American Idol, we thought perhaps that when people trust their own feelings they are more attuned to the feelings of others and this increased the chance they would make a more accurate prediction — we call this the social attunement hypothesis,” Lee explains.
But as they continued their experiments, they began to lean toward the explanation they call the privileged window hypothesis. Pham elaborates on the hypothesis: “When we rely on our feelings, what feels ‘right’ or ‘wrong’ summarizes all the knowledge and information that we have acquired consciously and unconsciously about the world around us. It is this cumulative knowledge, which our feelings summarize for us, that allows us make better predictions. In a sense, our feelings give us access to a privileged window of knowledge and information — a window that a more analytical form of reasoning blocks us from.”
“The weather study in particular shows that social attunement can’t explain people’s ability to predict the weather when they trust their feelings because weather is an act of nature and not something that human beings can directly control,” Lee notes. “People were more accurate at predicting weather conditions in their own zip codes — but not in Beijing or Melbourne. They don’t possess a knowledge base that would help them to make those predictions.” Further bolstering the privileged window hypothesis, only participants who had some background knowledge about football benefited from trust in feelings in predicting the winner of the 2011 national college football Bowl Championship Series.
Because some amount of relevant knowledge appears to be required to more accurately forecast the future, it is not always the case that trusting feelings will lead to better predictions. High trust in feelings does not help predict the unpredictable, such as the roll of a die. But many people might have a decent shot at predicting how the Dow Jones will move in the immediate future because they are likely to have garnered information about the market from the press.
“Given that there are so many professions dedicated to predicting what is going to happen tomorrow — whether it is which stocks will rise and fall, what the weather will be, or who is going to be the next president,” Lee says, “our findings show that sometimes feelings can be a good compass to guide us to what the future will bring.”
Leonard Lee is associate professor of marketing at Columbia Business School.
Michel Pham is the Kravis Professor of Business in the Marketing Division at Columbia Business School.