TOWARDS QUANTIFYING INTERVIEWS: COMPARING TECHNIQUES TO EVALUATE THE QUALITY OF DESIGN INTERVIEWS

Exploratory interviewing for front-end design research is a widely used practice, but unstandardized process. There are documented best practices for conducting “good” interviews, and these skills can be used to develop metrics to evaluate interview quality. A standard practice for verifying interview quality would increase confidence in studies that utilize student or novice interviewers, who often have less interviewing experience than expert researchers. Additionally, metrics to evaluate interviews could be used for individualized feedback to help novices improve over time, and feel confident and ready to complete field interviews for design and research projects. In this study, 12 undergraduate students were trained in design ethnography techniques and participated in field interviewing for a design project related to developing a service model for dialysis treatment in End Stage Kidney Disease in Zimbabwe. The quality of the students’ practice interviews from training and from field interviews during the design project were measured using a calculated scoring method that was developed based on interviewing best practices. To test the validity of the scoring method, this study compared the method to two accepted practices for evaluating interview quality: experts and information gained from interviews. The scoring method was found to not have a strong relationship to either accepted practice. Therefore, the metrics and formula used in the calculated scoring method were reexamined in light of the findings from the practice and field interviews to produce recommendations for future iterations.