Frederick Conrad, Norman Brown, and Monica Dashen (1999) "Estimating the Frequency of Events from Unnatural Categories", Proceedings of the Section on Survey Research Methods, American Statistical Association.
"How often do you do light or moderate activities for at least 10 minutes that cause only light sweating or a slight to moderate increase in breathing or heart rate?" This is a hard question because it asks about an event category ("light or moderate activities .") which seems to be at odds with the way most respondents think about events. We argue that the danger in asking respondents about the frequency of such categories - we call them unnatural categories - is that relevant episodes may not come to mind, leading to underreporting. We explored this in two studies. In both, participants studied a list of words and were then asked to estimate the number that were members of either unnatural or natural categories. In the first experiment, the unnatural category group was asked how many of the words they had studied contained particular properties (e.g. shiny, smelly, round); the group tested on more natural categories was asked how many words were members of taxonomic categories (e.g. furniture, mammals, fruit). Both groups underestimated actual frequency but it was far more extreme for those answering about properties. In the second experiment, both groups were tested on properties but one of these groups, the instance + property group, studied the properties along with the individual words (e.g. milk-white) to see if by promoting encoding of the properties we could render them more natural. Response-time patterns indicated that this was the case: instance-only estimates took three times as long as instance + property estimates and involved far more underestimation of actual frequency. We recommend decomposing unnatural categories into their natural parts and asking separate questions about each.
Last Modified Date: July 19, 2008