Background: Cognitive interviewing uses theories of cognitive psychology and survey methodology to understand how respondents perceive and interpret survey questions. There have been few examples of cognitive interviewing in developing discrete choice experiments (DCEs). Our aim was to use cognitive interviewing to explore two aspects of decision making: a cheap-talk script to reduce hypothetical bias, and the reality and complexity of opt-out options vs forced choices.
Methods: We conducted ‘think aloud’ cognitive interviews with 17 prostate and colorectal cancer survivors completing a rigorously developed DCE of preferences for cancer follow-up services. The DCE contained 8 choice sets each with 8 attributes, including cost. The survey also included: a cheap-talk script, both forced choice and opt-out choice sets, and debriefing questions for comprehension, engagement and attribute non-attendance (ANA). An iterative analysis process was completed by two researchers using a framework approach, assessing decision making within and across participants.
Results: Preliminary analysis suggests the cheap-talk script has little impact on individuals’ hypothetical bias, primarily because the cost attribute was often ignored. The realism of the opt-out was affected by respondents’ prior experiences of cancer follow-up. It emerged that many debriefing question responses, particularly ANA, were contradictory to the decision making within choice sets.
Conclusion: Our study tested aspects of decision making that have not been explored using cognitive interviewing, and demonstrates the insight gained from pre-testing. In cancer survivors considering follow-up options, the cheap-talk script did not have the anticipated effect, costs were ignored, and responses to debriefing questions may be misleading.
Note: This poster was awarded 'Best Poster Presentation' at the HESG Winter 2016 meeting.