Behind the Scenes: A Qualitative Investigation of Interviewers' Performance in EQ-5D Valuation Studies

Abstract

Background: The EuroQol Valuation Technology (EQ-VT) protocol is currently employed by the valuation studies of the EQ family of instruments worldwide. To date, all the evidence in support of the quality control (QC) originates from quantitative indicators. Objective: We aimed to explore interviewers’ conversational patterns in EQ-VT interviews, beyond quantitative QC indicators, and to provide a preliminary exploration of how the interaction between interviewer and respondent impacts data quality. Methods: Two researchers transcribed and independently coded 24 video-recorded interviews from the Italian EQ-5D-5L valuation study, adopting the conversational analysis framework. The analysis identified positive and negative ‘patterns’ of conversational practice. These were categorized into themes and sub-themes and were used to score a random sample of 42 video-recorded interviews conducted at different time points by seven interviewers. Results: The conversational analysis identified 20 positive and 14 negative interview patterns, which were grouped into two main themes (i.e., task execution and communication skills). Positive items included appending questions that stimulated respondents’ engagement, providing different explanations for an unclear aspect, supporting the participant with useful information for completing the tasks, and increasing the interview’s coherence by confirming the respondent answers. Negative patterns included moving forward in the exercise without making sure that the respondent understood, trying to force an answer from the respondent, speaking too fast, and providing incomplete or incorrect explanations of the task. Most interviewers exhibited a moderate increase in positive patterns or a decrease in negative patterns over time. A certain degree of consistency between the quantitative QC results and the qualitative scoring deriving from the interviewer-respondent interaction was observed, with the best and worst performers of the qualitative scoring showing good and bad scores on key QC items, respectively. Conclusions: The identified positive and negative patterns may be useful to inform the training material of EQ-VT studies worldwide and complement the existing QC process.