Absract
The study evaluates a very common question designed to measure happiness: “Taken all together, how would you say things are these days–would you say that you are very happy, pretty happy, or not too happy?” Through five representative survey experiments, we show that (1) this survey item underestimates the level of happiness with one’s life; (2) this is because the measure is more likely to reflect satisfaction with the state of the world rather than personal life; (3) this measures is more susceptible to priming; (4) the addition of three words “in your life” to the item greatly reduces priming and question order effects; and (5) the addition of these three words produces results that are very similar to life satisfaction measures that include “in your life” and are more positively associated with income. These results provide evidence that a simple correction better measures personal happiness. Furthermore, our findings reassess the foundation of a considerable volume of scholarship about how politics and income is associated happiness.
Similar content being viewed by others
Notes
http://www.federalreserve.gov/newsevents/speech/bernanke20100508a.htm---It should be noted that in this speech Bernanke specifically cites the survey item we evaluate in this study.
The advantage of using external evaluations of happiness is that one can avoid problems stemming from common-source bias. Researchers have observed that when two self-reported measures are obtained from the same source, such as a single survey, the association between the two measures is often inflated (Favero and Bullock 2015; Meier and O’Toole 2010).
Additionally, Smith (1979) investigated the slight difference between the National Opinion Research Center (NORC) wording and the Survey Research Center, University of Michigan (SRC) wording of a single item happiness question and finds noticeable differences in responses but suggests that much of this may be due to question ordering in the two surveys.
GSS variable name HAPPY. A Google Scholar search of the exact question revealed 628 results. Document counts accessed on 9/19/2015.
Mutz (2011) argues that “population-based survey experiments provide a means of establishing causality that is unmatched by any large-scale survey data collection effort, no matter how extensive.”
Details of each survey is presented in Table 5 in the Online Supporting Materials.
See Nicholson-Crotty and Meier (2002) for a defense of single-state studies.
The results were also consistent across survey mode. In an online, web-based survey we found that the modified happiness question produced happier responses than the standard question wording.
To assess accuracy of coding, we coded responses with multiple coders. Intercoder reliability was 0.71. Other scales could be used for this coding.
We estimate a parsimonious model. However, results are consistent with additional right-hand side variables.
A content analysis of the February 2013 open-ended responses to the questions asking why a respondent was happy or unhappy found that 8 percent of respondents mentioned "government" when "in your life" wasn't in the wording. This is compared to only 1 percent when "in your life" was included in question. In addition, only 3 respondents mentioned the president or his name in the "in your life" survey item, while 14 referenced the president when those words were dropped from the question.
For example, the World Values Survey; European Social Survey, and European Quality of Life Survey use such a measure.
Although Fry and Stutzer (2005) question the causal direction of this relationship and find that for some age groups happy people are more likely to get married.
We replicated the analysis with each of the four telephone surveys containing the 4 category income measure and found in every case our modified happiness measure was more strongly correlated with income than the standard happiness measure.
We estimate a parsimonious model. However, results are consistent with additional right-hand side variables.
References
Abdel-Khalek, A. M. (2006). Measuring happiness with a single-item scale. Social Behavior and Personality: An International Journal, 34(2), 139–150.
Alvarez-Diaz, A., González, L., & Radcliff, B. (2010). The politics of happiness: On the political determinants of quality of life in the American states. The Journal of Politics, 72(03), 894–905.
Andrews, F. M., & Robinson, J. P. (1991). Measures of subjective well-being. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 61–114). San Diego: Academic Press.
Brickman, P., Coates, D., & Janoff-Bulman, R. (1978). Lottery winners and accident victims: Is happiness relative? Journal of Personality and Social Psychology, 36(8), 917.
Cheung, Felix, & Lucas, Richard E. (2014). Assessing the validity of single-item life satisfaction measures: results from three large samples. Quality of Life Research, 23(10), 2809–2818.
Connolly, M. (2013). Some like it mild and not too wet: The influence of weather on subjective well-being. Journal of Happiness Studies, 14(2), 457–473.
Cummins, R. A. (1995). On the trail of the gold standard for subjective well-being. Social Indicators Research, 35(2), 179–200.
Di Tella, R., & MacCulloch, R. (2005). Partisan social happiness. The Review of Economic Studies, 72(2), 367–393.
Diener, E. (2000). Subjective well-being: The science of happiness and a proposal for a national index. American Psychologist, 55(1), 34.
Diener, E., Inglehart, R., & Tay, L. (2012). Theory and validity of life satisfaction scales. Social Indicators Research, 112(3), 497–527.
Easterlin, R. A. (1995). Will raising the incomes of all increase the happiness of all? Journal of Economic Behavior & Organization, 27(1), 35–47.
Easterlin, R. A. (2013). Happiness, growth, and public policy†. Economic Inquiry, 51(1), 1–15.
Favero, N., & Bullock, J. B. (2015). How (not) to solve the problem: An evaluation of scholarly responses to common source bias. Journal of Public Administration and Theory, 25(1), 285–308.
Ferrer-i-Carbonell, A., & Frijters, P. (2004). How important is methodology for the estimates of the determinants of happiness? The Economic Journal, 114(497), 641–659.
Flavin, P., Pacek, A., & Radcliff, B. (2014). Size of government and human well-being: Assessing the impact of the size and scope of government on human well-being. Social Forces, 92(4), 1241–1258.
Fowler, F. J. (1995). Improving survey questions: Design and evaluation. Thousand Oaks, CA: Sage.
Frey, B. S., & Stutzer, A. (2000). Happiness, economy and institutions. The Economic Journal, 110(466), 918–938.
Frey, B. S., & Stutzer, A. (2002). What can economists learn from happiness research? Journal of Economic literature., 40, 402–435.
Frey, B. S., & Stutzer, A. (2010). Happiness and public choice. Public Choice, 144(3–4), 557–573.
Gerber, A. S., & Huber, G. A. (2010). Partisanship, political control, and economic assessments. American Journal of Political Science, 54(1), 153–173.
Gerber, A. S., Huber, G. A., Doherty, D., & Dowling, C. M. (2013). Assessing the stability of psychological and political survey measures. American Politics Research, 41(1), 54–75.
Graham, J. M. (2011). Measuring love in romantic relationships: A meta-analysis. Journal of Social and Personal Relationships, 28(6), 748–771.
Graham, C. (2012). The pursuit of happiness: An economy of well-being. Washington, DC: Brookings Institution Press.
Haller, M., & Hadler, M. (2006). How social relations and structures can produce happiness and unhappiness: An international comparative analysis. Social Indicators Research, 75(2), 169–216.
Helliwell, J. F. (2003). How’s life? Combining individual and national variables to explain subjective well-being. Economic Modelling, 20(2), 331–360.
Helliwell, J. F., & Huang, H. (2008). How’s your government? International evidence linking good government and well-being. British Journal of Political Science, 38(04), 595–619.
Hills, P., & Argyle, M. (2002). The Oxford Happiness Questionnaire: A compact scale for the measurement of psychological well-being. Personality and Individual Differences, 33(7), 1073–1082.
Howell, R. T., Rodzon, K. S., Kurai, M., & Sanchez, A. H. (2010). A validation of well-being and happiness surveys for administration via the Internet. Behavior Research Methods, 42(3), 775–784.
Huang, P. H. (2010). Happiness studies and legal policy. Annual Review of Law and Social Science, 6, 405–432.
Huppert, F. A. (2009). Psychological Well-being: Evidence Regarding its Causes and Consequences†. Applied Psychology: Health and Well-Being, 1(2), 137–164.
Kalmijn, W. M., Arends, L. R., & Veenhoven, R. (2011). Happiness scale interval study. Methodological considerations. Social Indicators Research, 102(3), 497–515.
Krueger, A. B., & Schkade, D. A. (2008). The reliability of subjective well-being measures. Journal of public economics, 92(8), 1833–1845.
Lane, R. E. (2000). The loss of happiness in market democracies. CT, New Haven: Yale University Press.
Layard, R. (2005). Happiness: Lessons from a New Science. London: Allen Lane.
Lucas, R. E., & Donnellan, M. B. (2012). Estimating the reliability of single-item life satisfaction measures: Results from four national panel studies. Social Indicators Research, 105(3), 323–331.
Lucas, Richard E., & Schimmack, Ulrich. (2009). Income and well-being: How big is the gap between the rich and the poor? Journal of Research in Personality, 43(1), 75–78.
Lyubomirsky, S., & Lepper, H. S. (1999). A measure of subjective happiness: Preliminary reliability and construct validation. Social Indicators Research, 46(2), 137–155.
MacKerron, G. (2012). Happiness economics from 35,000 feet. Journal of Economic Surveys, 26(4), 705–735.
McClendon, M. J., & O’Brien, D. J. (1988). Question-order effects on the determinants of subjective well-being. Public Opinion Quarterly, 52(3), 351–364.
Meier, K. J., & O’Toole, L. J., Jr. (2010). Subjective organizational performance and measurement error: common source bias and spurious relationships. Journal of Public Administration Research and Theory, 23(2), 429–445.
Moskowitz, D. S. (1986). Comparison of self-reports, reports by knowledgeable informants, and behavioral observation data. Journal of Personality, 54(1), 294–317.
Mutz, D. C. (2011). Population-based survey experiments. Pinceton, NJ: Princeton University Press.
Nicholson-Crotty, S., & Meier, K. J. (2002). Size doesn’t matter: In defense of single-state studies. State Politics & Policy Quarterly, 2(4), 411–422.
Orviska, M., Caplanova, A., & Hudson, J. (2014). The impact of democracy on well-being. Social Indicators Research, 115(1), 493–508.
Oswald, A. J. (1997). Happiness and economic performance. The Economic Journal, 107(445), 1815–1831.
Pacek, A., & Radcliff, B. (2008a). Assessing the welfare state: The politics of happiness. Perspectives on Politics, 6(02), 267–277.
Pacek, A. C., & Radcliff, B. (2008b). Welfare policy and subjective well-being across nations: An individual-level assessment. Social Indicators Research, 89(1), 179–191.
Pavot, W., & Diener, E. (1993). Review of the satisfaction with life scale. Psychological Assessment, 5(2), 164.
Porta, P. L., & Scazzieri, R. (2007). 5 Public happiness and civil society. In L. Bruni & P. L. Porta (Eds.), Handbook on the economics of happiness (pp. 95–109). Northhampton, MA: Edward Elgar.
Rehdanz, K., & Maddison, D. (2005). Climate and happiness. Ecological Economics, 52(1), 111–125.
Rodríguez-Pose, A., & Maslauskaite, K. (2012). Can policy make us happier? Individual characteristics, socio-economic factors and life satisfaction in Central and Eastern Europe. Cambridge Journal of Regions, Economy and Society, 5(1), 77–96.
Ross, M., Eyman, A., & Kishchuk, N. (1986). Determinants of subjective well-being. In J. M. Olson, C. P. Herman, & M. P. Zanna (Eds.), Relative deprivation and social comparison: The Ontario symposium (pp. 79–93). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Røysamb, E., Nes, R. B., & Vittersø, J. (2014). Well-being: Heritable and changeable. In K. M. Sheldon & R. E. Lucas (Eds.), Stability of happiness: Theories and evidence on whether happiness Can change (pp. 9–36). London: Academic Press.
Ryff, C. D. (1989). Happiness is everything, or is it? Explorations on the meaning of psychological well-being. Journal of Personality and Social Psychology, 57(6), 1069.
Sandvik, E., Diener, E., & Seidlitz, L. (1993). Subjective well-being: The convergence and stability of self-report and non-self-report measures. Journal of Personality, 61(3), 317–342.
Schimmack, U., Diener, E., & Oishi, S. (2002). Life-satisfaction is a momentary judgment and a stable personality characteristic: The use of chronically accessible and stable sources. Journal of Personality, 70(3), 345–384.
Schlenker, B. R., Chambers, J. R., & Le, B. M. (2012). Conservatives are happier than liberals, but why? Political ideology, personality, and life satisfaction. Journal of Research in Personality, 46(2), 127–146.
Schwarz, N., & Strack, F. (1991). Evaluating one’s life: A judgment model of subjective well-being. Subjective well-being: An interdisciplinary perspective, 21, 27–47.
Schwarz, N., & Strack, F. (1999). Reports of subjective well-being: Judgmental processes and their methodological implications. In D. Kahneman, E. Diener, & N. Schwarz (Eds.), Well-being: The foundations of hedonic psychology. New York: Russell Sage Foundation.
Schwarz, N., Strack, F., & Mai, H. P. (1991). Assimilation and contrast effects in part-whole question sequences: A conversational logic analysis. Public Opinion Quarterly, 55(1), 3–23.
Scott, W. A. (1958). Research definitions of mental health and mental illness. Psychological Bulletin, 55(1), 29.
Smith, T. W. (1979). Happiness: Time trends, seasonal variations, intersurvey differences, and other mysteries. Social Psychology Quarterly, 42(1), 18–30.
Smith, T. W. (1990). Timely artifacts: A review of measurement variation in the 1972-1989 GSS. GSS Methodological Report 56.
Stones, M. J., Hadjistavropoulos, T., Tuuko, H., & Kozma, A. (1995). Happiness has traitlike and statelike properties: A reply to Veenhoven. Social Indicators Research, 36(2), 129–144.
Strack, F., Martin, L. L., & Schwarz, N. (1988). Priming and communication: Social determinants of information use in judgments of life satisfaction. European journal of social psychology, 18(5), 429–442.
Studer, R. (2012). Does it matter how happiness is measured? Evidence from a randomized controlled experiment. Journal of Economic and Social Measurement, 37(4), 317–336.
Tavits, M. (2008). Representation, corruption, and subjective well-being. Comparative Political Studies, 41(12), 1607–1630.
Tourangeau, R., Rasinski, K. A., & Bradburn, N. (1991). Measuring happiness in surveys: A test of the subtraction hypothesis. Public Opinion Quarterly, 55(2), 255–266.
Van Praag, B. M., et al. (2003). The anatomy of subjective well-being. Journal of Economic Behavior & Organization, 51(1), 29–49.
Veenhoven, R. (1991). Is happiness relative? Social Indicators Research, 24(1), 1–34.
Veenhoven, R. (1995). The cross-national pattern of happiness: Test of predictions implied in three theories of happiness. Social Indicators Research, 34(1), 33–68.
Wanous, J. P., & Reichers, A. E. (1996). Estimating the reliability of a single-item measure. Psychological Reports, 78(2), 631–634.
Weiss, A., et al. (2008). Happiness is a personal (ity) thing: The genetics of personality and well-being in a representative sample. Psychological Science, 19(3), 205–210.
Weitz-Shapiro, R., & Winters, M. S. (2011). The link between voting and life satisfaction in Latin America. Latin American Politics and Society, 53(4), 101–126.
Wills-Herrera, E., Orozco, L. E., Forero-Pineda, C., Pardo, O., & Andonova, V. (2011). The relationship between perceptions of insecurity, social capital and subjective well-being: Empirical evidences from areas of rural conflict in Colombia. The Journal of Socio-Economics, 40(1), 88–96.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Husser, J.A., Fernandez, K.E. We Are Happier than We Realize: Underestimation and Conflation in Measuring Happiness. J Happiness Stud 19, 587–606 (2018). https://doi.org/10.1007/s10902-016-9831-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10902-016-9831-0