Measuring income volatility: why the estimates disagree

Household income is usually reported as an annual figure. That convention is convenient for tax systems and for survey design, and it conceals a great deal. Two households with identical annual totals can experience entirely different years: one arriving in twelve even instalments, the other in a handful of uneven ones with gaps in between. The literature on income volatility attempts to measure that second dimension, and the estimates it produces vary considerably depending on the data source.

Two families of measurement

The first family uses administrative financial records. The JPMorgan Chase Institute, in work led by Diana Farrell and Fiona Greig, analysed de-identified checking account data to observe month-to-month inflows and outflows directly rather than through recall. Their reports describe substantial monthly fluctuation in both income and spending for a large share of account holders, and note that the two do not move in step — spending does not smoothly absorb an income dip, which is what a simple consumption-smoothing model would predict.

The second family uses survey and earnings-record panels. James Ziliak, Bradley Hardy and Christopher Bollinger, working with matched Current Population Survey records, examined how earnings volatility in the United States changed over several decades and found the trend differs by sex and by earnings level. Related work by Molly Dahl, Thomas DeLeire and Jonathan Schwabish, using Social Security Administration earnings records, produced flatter trends than several survey-based studies had reported.

Why the numbers diverge

Four differences account for most of the gap.

Population. Bank account data covers people who hold accounts with that institution and use them as a primary account. That population is not a national sample, and it under-represents unbanked and cash-dependent households, whose income is likely to be the most irregular.

Definition. Account data observes deposits. Deposits are not earnings: they include transfers between the household's own accounts, reimbursements, loan proceeds and gifts. Survey data observes reported earnings, which excludes those items but is subject to recall error and to systematic under-reporting of irregular and informal work.

Frequency. Monthly measurement finds volatility that annual measurement cannot see by construction. Comparing a monthly estimate with an annual one is a comparison of two different quantities.

Zeroes. How a study treats months with no recorded income has a large effect on the resulting variance, and different papers treat them differently. Some exclude them, some floor them, some model them separately.

What follows

None of this makes the volatility literature unreliable. It makes single headline figures unreliable when they are quoted without their source and definition attached. A statement that some share of households experiences income swings above a given threshold is only interpretable alongside the answers to four questions: which households, which definition of income, measured over what interval, and with zero-income periods handled how.

The broader methodological point is not specific to this field. Where two credible research programmes report different magnitudes for the same phenomenon, the disagreement is often located in measurement rather than in the phenomenon, and the more informative reading is the one that examines the instruments.

Sources

• Farrell, D., & Greig, F. (2015). Weathering Volatility: Big Data on the Financial Ups and Downs of U.S. Individuals. JPMorgan Chase Institute.
• Ziliak, J. P., Hardy, B., & Bollinger, C. (2011). Earnings volatility in America: Evidence from matched CPS. Labour Economics, 18(6).
• Dahl, M., DeLeire, T., & Schwabish, J. (2011). Estimates of Year-to-Year Variability in Worker Earnings and in Household Incomes from Administrative, Survey, and Matched Data. Journal of Human Resources, 46(4).