Quantitative vs. Qualitative: A Comparison of Methods For Improved Usability Research
Summary
One of the most common ways to perform usability research is by direct observation and questioning. However, there are various psychological and social factors that can influence participants’ behavior when usability research contains explicit self-reporting. It would therefore be very useful to circumvent the subjectivity of traditional usability research by measuring actual behavior, instead of relying on the verbal report of participants. The purpose of this research was, therefore, to determine whether or not quantitative usability research methods provide different, or better insights into the usability of a product than traditional qualitative usability research methods. To explore this, quantitative research was conducted using eye tracking, mouse metrics, EEG, facial expressions and the System Usability Scale (n = 48), as well as qualitative research using observation and questioning (n = 8). Results showed that facial expression analysis and EEG were not particularly suitable for usability purposes and did therefore not provide useful insights. However, eye tracking in combination with mouse metrics identified a larger amount of usability problems (which are also more specific and detailed) compared to traditional qualitative research. On the other hand, qualitative analysis revealed the reasons behind usability problems: something which quantitative analysis could only speculate about. By combining both quantitative and qualitative approaches, the results from qualitative usability research could be an excellent starting point for further in-depth usability research with eye tracking and mouse metrics.