Effective Sampling from Social Media Sites and Search Engines for Web Surveys: Demographic and Data Quality Differences in Surveys of Google and Facebook Users

With proliferation of web surveys, the relative affordability of recruitment, and increasing nonresponse in other survey modes, nonprobability methods are increasingly being considered by researchers and government offices alike. However, research needs to more fully understand how the demographic characteristics of respondents may depend heavily on the source of sample, mode of recruitment, and context of the survey experience. As a first step in exploring the potential implications of recruitment source on response quality, we use data from a web survey fielded in 2013 to compare data quality indicators in survey data from the two recruitment platforms (Google and Facebook advertisements). In so doing, taking into account demographic differences that may arise from various steps in the recruitment process, we explore the effect of demographics, device and technology usage, incentives, and recruitment platform on data quality and response strategy. Our results show differences between platforms in comparability to national benchmarks, breakoffs, completion time, nonsubstantive answers, and numeric response strategies. Importantly, some variation in substantive responses was explained by demographic differences related to mobile device usage, which varied by recruitment platform. With the use of nonprobability samples on the rise, future work should build from these results to more directly assess the role of recruitment source in data quality.