Family Support Topics

• Strengthening Families
• Focusing on Youth
• Understanding TANF
• Clients

Assessing the Quality of Income Data Across Surveys

Because income is a critical variable in policy analysis, many federal surveys collect at least some data on income. Yet income is difficult to measure well in a household survey. Income questions produce some of the highest item nonresponse rates recorded in surveys, and comparisons of survey estimates with benchmarks developed from administrative records provide evidence of significant underreporting for many sources.

Mathematica provided a systematic and comprehensive assessment of the collection and processing of income data in eight surveys either sponsored by the U.S. Department of Health and Human Services (DHHS) or used for policy research and evaluation within DHHS. The DHHS-sponsored surveys are the Medical Expenditure Panel Survey (MEPS), the National Health Interview Survey (NHIS), the Medicare Current Beneficiary Survey (MCBS), the Health and Retirement Study (HRS), and the Panel Study of Income Dynamics (PSID). The three additional surveys—all designed and conducted by the U.S. Census Bureau—are the Survey of Income and Program Participation (SIPP), the Annual Social and Economic (ASEC) Supplement to the Current Population Survey (CPS), and the American Community Survey (ACS).

The assessment included both descriptive and empirical components. The descriptive component compiled extensive information on survey design and methodology, in addition to the measurement of income and poverty, and presented these data in a side-by-side format. The empirical component generated comparative tabulations of the distribution of income and poverty status for a range of personal characteristics for a common universe, income concept, and family definition, to the extent that this was feasible. Additional analysis focused on the implications of specific survey design choices. An annotated bibliography documented relevant findings in the literature on survey income data extending back to the 1960s.

Key findings from the comparative empirical analysis include estimates of aggregate income and its distribution by quintile of family income; poverty status; earned versus unearned income; the proportion of income allocated because of nonresponse; and the frequency of rounding. Key results of the methodological analysis include the impact of the family definition and the proximity of measured family composition to the income reference period on estimated poverty and its distribution, the relationship between the interview month and the frequency of allocation, and the impact of imputation methodology on rounding.

A technical advisory group, consisting of representatives of the eight surveys and a separate group of data users, reviewed the analysis plan, the annotated bibliography, and the final report.

Publications

"Income Data for Policy Analysis: A Comparative Assessment of Eight Surveys" (December 2008)