The Method Section, Part II: Establishing the Trustworthiness of Qualitative Research

In my last post, I reviewed a series of considerations for writing the method section—points you might want to include to establish the reasonableness of your findings. While some of these considerations also apply to qualitative data collection and analysis, qualitative researchers attend to other issues in describing their method. Newman and colleagues (2011) identified 20 different legitimation techniques that qualitative researchers may choose to address while designing and reporting on their research. Rather than describe all 20 here, I’ve identified three overarching themes that can help writers establish the trustworthiness of qualitative research studies.

1. Demonstrate that you have adequate insight into the phenomenon under study. Journal reviewers and others will want to know that you have enough supporting materials—observations, interviews, etc.—to support any inferences you make. You might do this by demonstrating sustained engagement over a period of time that allowed you to capture the entire phenomenon understudy. Prolonged engagement also enables you to determine whether specific observations represent one-offs or a more consistent pattern of behavior.

As in quantitative studies, describing decisions made about the sample selection is essential for establishing the reasonableness of your findings. Sample sizes are likely to be much smaller in qualitative studies. As such, it becomes important to demonstrate that the inclusion criteria for the study and recruitment techniques were likely to yield participants (or settings, texts, etc.) that could shed light on the phenomenon under investigation or the primary research question(s). Qualitative researchers often use theoretical sampling—targeted data collection—to support the development of an emerging theory based on initial data analysis. This approach might involve recruiting additional participants, collecting additional data from participants, or re-analyzing data to find additional support for inferences. At other times, researchers will suspend data collection when they feel they have reached a point where no new themes will arise—referred to as saturation. You should document such decisions and the process of making them in the method section.

2. Describe practices used to check your assumptions. In quantitative methodologies, researchers describe steps taken to control for and report on the probability of statistical error. Qualitative researchers can similarly introduce practices that increase the reliability of inferences made throughout data collection and analysis. One such technique, peer debriefing, involves consulting with other professionals not involved in the research. These knowledgeable outsiders confirm the reasonableness of the researcher’s perceptions of the phenomenon under study. Researchers can also engage in member checking, which may involve having participants review transcripts for accuracy or read and comment on the manuscript draft. Qualitative researchers often engage in reflexive journal writing or memoing throughout data collection and analysis. These records of the research experience help document how and why certain perceptions emerge. Finally, qualitative researchers frequently describe their position (i.e., positionality or reflexivity) vis à vis the participants or research question. Such statements typically point to how the researcher’s identity(ies) may have shaped the kinds of data collected and how they interpreted it.

3. Help readers understand the potential relevance of the findings to other contexts. Frequently, authors list a lack of generalizability as a limitation of qualitative studies. In fact, qualitative research is not designed to be generalizable. Instead, you can demonstrate your findings' potential transferability or applicability to other contexts. One way to do this is by providing a detailed description of data or phenomenon (i.e., thick description) so that reviewers and readers can determine whether findings might be applicable to other contexts or the larger population from which a sample was drawn, for example.

The legitimation techniques described here support the trustworthiness of qualitative manuscripts. You can learn more about these techniques and how to assess their usefulness for your own research through my online course, Writing About Your Research Methods. The four-module course addresses strategies for writing the method section for quantitative, qualitative, and mixed-methods studies. The accompanying workbook includes templates and writing prompts to help you develop and revise the method section.

Reference

Newman, I., Newman, D., & Newman, C. (2011). Writing research articles using mixed methods: Methodological considerations to help you get published. In T. S. Rocco, T. Hatcher, & Associates, The handbook of scholarly writing and publishing (pp. 191-208). Jossey-Bass.