Issue: The Pros and Cons of Questions Containing Cue Lists
Key Words: cues, data quality, mode of administration, questionnaire design, recall error and response alternatives
Contact for further discussion:
Office of Survey Methods Research, PSB 1950
Bureau of Labor Statistics
2 Massachusetts Avenue NE
Washington, DC 20212
Telephone: (202) 691-7530
Fax: (202) 691-7426
A challenging research issue is that of questions which contain non-exhaustive cue lists (response alternatives). Such lists help shape and align respondents' interpretation of the question with that of the survey designer by providing respondents with subsets of acceptable (included) and unacceptable (excluded) answers. Without cue lists respondents are forced to devise their own inclusion criterion and as a result may not provide acceptable answers (Dashen and Fricker, 2001).
Questions containing cue lists can be seen in a variety of surveys, including establishment surveys (Annual Refiling survey) and household survey (Consumer Expenditure Survey). The table below provides examples of questions containing cue lists.
|Survey & Respondent Expectations||Example of Questions with Cue Lists|
|In the Annual Refiling Survey, respondents are expected to verify whether their establishment fits an industry or not.||Child day care services of infants or children. Generally care is for preschool children, but may include caring for older children when they are not in school and may also offer prekindergarten educational programs. Examples include, but are not limited to:|
|Child day care services of infants or children. Generally care is for preschool children, but may include caring for older children when they are not in school and may also offer prekindergarten educational programs. Examples include, but are not limited to:|
|* Child day care babysitting services||* Nursery schools|
|* Child or infant day care centers||* Preschool centers|
|DOES NOT INCLUDE kindergarten educational programs provided in elementary and secondary schools.|
|In the Consumer Expenditure- Diary, respondents are expected to record their expenses for 7-days.||Food and Drinks Away from Home (selected cues)|
|Breakfast buffet||pizza delivery||beer at happy hour||croissant from café|
|Carry-out lunch||Chinese takeout||pretzels at ballgame||ice cream from truck|
|Dinner/cocktails at restaurant||child's school lunch||wine at tavern||wedding reception caterer|
Cue lists are often non-exhaustive because of space limitations and other reasons. To signify this non-exhaustive status, survey designers often use phrases like "illustrative," "e.g.," or "not limited to." Survey designers often assume that the cues on the list will aid in the recollection of related items, thereby assisting the respondent in generating items. For example, Consumer Expenditure (Diary) respondents might remember taking out Mexican food after having read the Chinese Takeout cue. The number of items in a cue list can vary greatly and can be provided across all modes of administration. Cues can be either presented in a list or a sentence-like format. Cues can also be presented in a general manner (e.g., fresh produce) or a specific manner (e.g., bananas, oranges, and lettuce).
Brown (1968). Reciprocal facilitation and impairment in free recall. Psychonomic Science, 10, 41-42.
Dashen, M. & Fricker, S. (2001). Understanding the Cognitive Processes of Open-Ended Categorical Questions and their Effects on Data Quality. Journal of Official Statistics. 17(4), 457-77.
Krosnick, J. & Alwin, D. (1987). An evaluation of a cognitive theory of response order effects in survey measurement. Public Opinion Quarterly, 51, 201-19.
Marsh E.J., Dolan, P.O. Balota, D.A., and Roediger, H.L. (2004) Part-set cuing effects in younger and older adults. Psychological Aging. 19(1), 134-44.
Nickerson (1984). Retrieval inhibition from part-set cuing: a persisting enigma in memory research. Memory and Cognition. 12(6), 531-52.
Schober, M.F. & Conrad, F.C. (1997). Does Conversational Interviewing Reduce Survey Measurement Error? Public Opinion Quarterly. 61(4), 576-602.
Tucker, C. (1992). The Estimation of Instrument Effects on Data Quality in the Consumer Expenditure Diary Survey. Journal of Official Statistics, 8(1), 41-61.
Last Modified Date: January 06, 2006
Last Modified Date: July 19, 2008