How many qualitative interviews is enough?
This is a question that I think all qualitative researchers struggle with - how many interviews do you need to produce adequate, meaningful evidence? As a transcription company, we've dealt with projects with five interviews, and others with hundreds. There are certainly a lot of questions to consider, and Sarah Baker and Rosalind Edwards' very useful working paper aims to address these. Rosalind was interviewed for an NCRM podcast about this paper, which we've transcribed below for you.
Transcript
Welcome to the podcast series from the National Centre for Research Methods at the University of Southampton. In today’s podcast, Professor Rosalind Edwards talks about her hugely popular working paper, addressing the question of how many interviews is enough in qualitative research.
Rosalind Edwards: Well I think really, the whole paper is a response to student concerns. My colleague, Sarah, and I were chatting about the way that undergraduates, masters, and PhD students who are doing qualitative research often say, “Well how many interviews do I need to do.?" We could give our own answers, but there was no resource that we could actually point them towards. A lot of the advice on how many is enough was dotted around. There was no central place that anyone could go to. We thought, well okay, it’s a question we keep getting asked. Let’s answer it.
Interviewer: What was your approach to this? How did you go about addressing that question?
Rosalind: We thought, we won’t just put forward our own perspectives here because we’ve got particular ideas. Why don’t we go to the people who usually provide advice about research methods, experts in the field, and ask them to answer the question. So we drew up a list of experts whose work we admired, who’d written on research methods, and we looked to see that there was a variety of disciplines and different geographical areas and different substantive topics that they researched on.
Then we simply sent them an email, saying, “If you had to answer this question, if a student asked you how many qualitative interviews are enough, how would you answer it?” But we thought, as well as experts, we also needed some early career researchers who had just gone through this process, for them to give their ideas. So we did actually also a similar process of approaching early career researchers by email, sending them the questions, say, “What advice would you give to other students?”
Interviewer: You got a very good response. One of those responses really did make you stop and pause for thought. Tell us a bit about that particular response, what it was, and why it made you think again.
Rosalind: That was from Harry Alcott, who unfortunately died back in November last year. He was a professor of anthropology of education at Oregon. He was unable to give us a full written response, but he did email back to us and sort of said in a nutshell what he thought were the key issues. He said, “Whatever you do, it should be transparent. That’s the key issue. Make that transparent.”
Then he asked us this question, which was a bit of a left field question for us, “How many interviews did you decide was enough?” Then we thought, “Oh my goodness! How are we deciding how many people to approach?” It really made us think about the process. What did we reckon were enough interviews? Because we hadn’t thought of the emails as interviews, but then we started to and thought, “How many interviews is enough for this?”
We decided that actually, we thought that we would look for an estimate of data saturation, and that’s how many we decided upon, and so we told him that. Then he emailed back and said, “Yes, but how many?” So we thought, okay, 15 is what we said. We thought that 15 experts would be enough to gain all perspectives on the issue. We ended up with only 14 experts, but we thought that we did see recurring themes in what they were saying. Then when the five early researchers, that was 19 responses in all.
Interviewer: So a pretty good response, and it’s difficult perhaps to give an overview of everything, but if you could perhaps pick out some of the key responses and the ones that you then reported in your research.
Rosalind: We read them as they came in, and all the experts and indeed the early career researchers, we sort of nodded away as we read their contributions. Some of the key points that came out for me was that in quantitative methods, it’s really fairly simple. It’s how much is enough for statistical validity. But in qualitative, what you’re dealing with is you’re trying to get at meanings; you’re trying to get at processes about complexity; you might be looking for commonalities; you might be looking for differences.
Also, quite often researchers are working in qualitative methods where they’re sort of doing this combination of sampling, collecting the data, analysing that, maybe going back again, doing more sampling, collecting and analysing. So the overall answer that came out after all of these experts, is it depends. Our task, we felt, was to draw out what it depends on that would be useful for people reading the paper.
We identified three areas:
One key area was around the epistemology and the methodology. What is the nature of the research? What’s its purpose? Is it trying to find out commonalities? Is it trying to find out differences between the sample? Is it looking for uniqueness? Is your research looking for complexity? Are you trying to compare different groups within your research or different instances within your research? The key issue there in terms of epistemology, or your understanding of how the social world works and methodology or understanding of how you should go about researching it, the key issue there is that you can build a convincing narrative, a convincing story about what it is that you’re doing. So you need to think about your sample in those terms.
The second way to think about your sample is actually in practical things, in terms of how many. What sort of degree are you going for? How much time have you got available? Have you got a three year project or a three month project? What resources, what money have you got available? Can you go travelling around the country or across the world? Or are you rooted in a particular place? What about the committees that you have to go through for your studies? There are ethics committees. Is there anything that you need to satisfy them? There are upgrade committees, if you’re a PhD student.
Then the third theme that came out that you need to think about in terms of how many qualitative interviews is enough is what we call the epistemic community, which is actually the people who are going to be judging your work. They’re going to be your mentors or your peers, or the people who are going to read your work or examine your work. What would they judge to be enough?
So the answer actually can be ‘one’- one unique case. It can be 20 cases. It can be 50 cases, depending on judging those three points and thinking about what it is that you need to know how many is enough. Or actually, it could be no cases, because some of our respondents, the experts, they actually said, “No, why go and interview people. Just observe particular cases or instances.” So no interviews in that case.
Interviewer: Lots of lessons to be learned from that, but what were the principal take aways? Is it all to do with the preparation and the planning? That’s what it sounds like, for students who are starting to think about this.
Rosalind: Yes, it’s preparing and planning, but also being flexible later on in the research process. It’s not always completely cut and dried right from the start. But what you need to think about as a student in deciding how many qualitative interviews is enough is, what is my research about? How many people do I need to form my research topic? What resources have I got – time and money and so on? And who is it for? Who is going to be reading this?
If I was to point to anything, I would point to the issue of the epistemic community, because that cuts across the issue of your epistemology, how you think the social world works, and also there is practical resource issues. It holds those two together. Who’s going to be reading this? Who’s judging me? And go with that one, if anything.
Interviewer: The paper has been designed as a resource for students. It’s been incredibly popular, thousands of downloads already. You must feel very satisfied by that as a starting point, but it demonstrated that there’s a clear need for this sort of steer, this sort of advice, this sort of help. What plans then to take it further or develop it further?
Rosalind: Actually, Sarah and I have been absolutely stunned. Within a year, well over 20,000 downloads. If only we had royalties. But yes, it does show that actually that was a really needed resource. It’s been tweeted and retweeted and everything. What I have done to take this forward, because Sarah is in New Zealand now and so she’ll take things forward in her own way over there, but what I’ve done is with another colleague I’ve written a book called, ‘What is Qualitative Interviewing?’ which is going to be published by Bloomsbury Academic soon. That’s a short advice book, about half the length of a usual academic book. So that’s another of these sorts of advice books.
The other thing that I think in order to take this forward is to listen to the recurring concerns of our students and to identify what working papers would be useful in that way.
Audio ends
We were founded by a qualitative researcher who remains on hand for advice if anyone needs assistance. We strive to give more help than a standard transcription company who do not employ specialists beyond quick typers.
Get in touch, we’re here to make things easier!
