There seems to be some confusion within the industry as to the role of qualitative research, as some consultants are introducing quantitive reporting within qualitative research. Qualitative research runs the risk of being discredited as a result.

I wrote this piece to help explain clarify the difference between quantitative and qualitative research.

-----------------------------------

This article examines the relative merits of qualitative and quantitative research methods. The aim is to identify their strengths and weaknesses in order to help User Experience Consultants to employ the most appropriate method.

As the audience for this paper mightn't be familiar with qualitative research the focus of this article is on explaining the merits of qualitative research.

This method uses data based on a large numbers of participants to yield insight into preferences and behaviour. Roughly speaking, this tells you WHAT people are doing or what they say they might do. This type of research is carried out using surveys of large samples yielding statistics which are often represented as percentages.

This method users small numbers of people to uncover WHY people act in a certain way. This yields in-depth understanding of human behaviour and the reasons that govern such behaviour. This research usually involves one-on-one interviews, ethnography and group discussions.

Our experience from traditional marketing research involves hundreds, if not thousands, of participants. Only interviewing a handful of people seems to be counter intuitive if we went to garner what most people think or do.

Some typical concerns are: 

Horses for courses; What measurement system is relevant?

What research method you should employ boils down to what question you are trying to answer. Each method has it's strengths and weaknesses. Knowing which scale to choose is critical because it relates to the types of statistics you can use to analyse the data.

Here are the main systems of measurement...

As we can see two of these measurements involve subjective measurement, which lends itself to qualitative interpretation and is impossible to empirically measure. Even if you wanted to use quantitative research it would prove inadequate as it is unable to measure certain subjective attributes.

We have established that there are important factors that quantitative research is unable to measure. In addition there are other weaknesses to quantitative research:

We have established that you don't necessarily need large numbers, in fact some things absolutely can't be measured using quantities. So how many people should you include for a qualitative study?

There is no straight forward answer to that questions as it depends on several factors such as what you are researching, the quality of your participants, the ability of your facilitator and some luck.

One method developed to solve this problem is called grounded theory. This method involves open investigation where interviews over a period of time yields patterns. These patterns form the basis of a hypothesis that can then be tested with further research. With this method you continue until patterns begin to repeat, therefore there isn't a specific number of users, but the sample is likely to increase with the complexity of the subject.

Jacob Nielsen advocates testing 3-5 users as his statistical model indicated that with 5 users a typical usability test will uncover 90% of the findings. This is highly contingent upon the subject and audience for the product being tested.

As a rule of thumb it is usually recommend to test with 20 users when collecting quantitative usability metrics, which gives a margin of error of +/- 19%.

It may surprise you to hear that the answer could be that the observation of one person can overturn the opinions of the majority; For example, the dominant logic for 1500 years was that the earth was the centre of the universe (geocentrism). The evidence seemed to support this hypothesis and the majority of people were adherents to this received logic. It wasn't until one person Copernicus' published De revolutionibus orbium coelestium in 1543 that overturned the old model. This is called falsifiability, when a claim can be proved wrong by one observation.

We could envisage the scenario that a quant study of 100,000 people found that users thought a banks' website was secure. However, it would only take one person to discover a security flaw to overturn the opinions of everyone else.

Qualitative methods aim to derive insights from statistics this method can only infer from questions people were asked. This limits the insights to the data set, who's basis is founded in a fixed set of questions. Whereas, qualitative research can uncover unknowns as the researcher is free to explore anything a participant may wish to speak about. In short, quantitative is limited and qualitative is unlimited.

Qualitative research  gives us a very accurate picture of peoples' EXPERIENCE. After all people understand life in the form of an emotional narrative. Our first and foremost reaction to the outside world is emotional, not rational. Therefore, the most compelling insights are those that understand these drives.

For example, A quant study may tell us that 70% of people are able to purchase an application via the Nokia Ovi store. This might seem, on the surface, to be a complete success. However, until we talk to some of those people we don’t know if they struggled through every step (a white knuckle experience) and found the whole process to be tedious or even horrible. As soon as they can, users will switch to a more pleasant service.

In summary, here is  a list of the main advantages to using qualitative research:

As we have established, there are strengths and weaknesses to both research methodologies. It is, therefore, wise to offset the weaknesses of one with the strengths of the other. Properly implemented, each method can work together to give you insights that form a complete picture.

One theoretical scenario would be to use qualitative interviews with users to uncover needs, behaviours, goals, mental models and their emotional experiences in connection with a particular product. To ascertain if these aspects were borne out in the population as a whole, it would be prudent to run a larger study with a statistically relevant sample of the population that would 'qualify' the smaller sample.