case study vs research methods

Distinguishing case study as a research method from case reports as a publication type

• Kristine M. Alpi William R. Kenan, Jr. Library of Veterinary Medicine, North Carolina State University, Raleigh, NC http://orcid.org/0000-0002-4521-3523
• John Jamal Evans North Carolina Community College System, Raleigh, NC

Author Biography

Kristine m. alpi, william r. kenan, jr. library of veterinary medicine, north carolina state university, raleigh, nc.

Akers KG, Amos K. Publishing case studies in health sciences librarianship [editorial]. J Med Libr Assoc. 2017 Apr;105(2):115–8. DOI: http://dx.doi.org/10.5195/jmla.2017.212 .

Creswell JW. Qualitative inquiry & research design: choosing among five approaches. Los Angeles, CA: SAGE; 2018.

Yin RK. Case study research: design and methods. 4th ed. Los Angeles, CA: SAGE; 2009.

Creswell JW. Research design: qualitative, quantitative and mixed methods approaches. 4th ed. Thousand Oaks, CA: SAGE; 2014.

Yin RK. Case study research and applications: design and methods. 6th ed. Thousand Oaks, CA: SAGE; 2018.

Stake RE. The art of case study research. Thousand Oaks, CA: SAGE Publications; 1995.

Merriam SB. Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass; 1998.

Yazan B. Three approaches to case study methods in education: Yin, Merriam, and Stake. Qual Rep. 2015;20(2):134–52.

Bartlett L, Vavrus F. Rethinking case study research: a comparative approach. New York, NY: Routledge; 2017.

Walsh RW. Exploring the case study method as a tool for teaching public administration in a cross-national context: pedagogy in theory and practice. European Group of Public Administration Conference, International Institute of Administrative Sciences; 2006.

National Library of Medicine. Case reports: MeSH descriptor data 2018 [Internet]. The Library [cited 1 Sep 2018]. < https://meshb.nlm.nih.gov/record/ui?ui=D002363 >.

National Library of Medicine. Organizational case studies: MeSH descriptor data 2018 [Internet]. The Library [cited 26 Oct 2018]. < https://meshb.nlm.nih.gov/record/ui?ui=D019982 >.

American Psychological Association. APA databases methodology field values [Internet]. The Association; 2016 [cited 1 Sep 2018]. < http://www.apa.org/pubs/databases/training/method-values.aspx >.

ERIC. Case studies [Internet]. ERIC [cited 1 Sep 2018]. < https://eric.ed.gov/?ti=Case+Studies >.

Janke R, Rush K. The academic librarian as co-investigator on an interprofessional primary research team: a case study. Health Inf Libr J. 2014;31(2):116–22.

Clairoux N, Desbiens S, Clar M, Dupont P, St. Jean M. Integrating information literacy in health sciences curricula: a case study from Québec. Health Inf Libr J. 2013;30(3):201–11.

Federer L. The librarian as research informationist: a case study. J Med Libr Assoc. 2013 Oct;101(4):298–302. DOI: http://dx.doi.org/10.3163/1536-5050.101.4.011 .

Medical Library Association. Journal of the Medical Library Association author guidelines: submission categories and format guidelines [Internet]. The Association [cited 1 Sep 2018]. < http://jmla.mlanet.org/ojs/jmla/about/submissions >.

Martin ER. Team effectiveness in academic medical libraries: a multiple case study. J Med Libr Assoc. 2006 Jul;94(3):271–8.

Hancock DR, Algozzine B. Doing case study research: a practical guide for beginning researchers. New York, NY: Teachers College Press; 2017.

Current Issue

ISSN 1558-9439 (Online)

Distinguishing case study as a research method from case reports as a publication type

Affiliations.

• 1 OHSU Library, Oregon Health & Science University, Portland, OR, [email protected].
• 2 North Carolina Community College System, Raleigh, NC, [email protected].
• PMID: 30598643
• PMCID: PMC6300237
• DOI: 10.5195/jmla.2019.615

The purpose of this editorial is to distinguish between case reports and case studies. In health, case reports are familiar ways of sharing events or efforts of intervening with single patients with previously unreported features. As a qualitative methodology, case study research encompasses a great deal more complexity than a typical case report and often incorporates multiple streams of data combined in creative ways. The depth and richness of case study description helps readers understand the case and whether findings might be applicable beyond that setting.

Publication types

• Comparative Study
• Biomedical Research / methods*
• Organizational Case Studies / standards*
• Publications / standards*
• Qualitative Research
• Research Design / standards*

Case study research: opening up research opportunities

RAUSP Management Journal

ISSN : 2531-0488

Article publication date: 30 December 2019

Issue publication date: 3 March 2020

The case study approach has been widely used in management studies and the social sciences more generally. However, there are still doubts about when and how case studies should be used. This paper aims to discuss this approach, its various uses and applications, in light of epistemological principles, as well as the criteria for rigor and validity.

Design/methodology/approach

This paper discusses the various concepts of case and case studies in the methods literature and addresses the different uses of cases in relation to epistemological principles and criteria for rigor and validity.

The use of this research approach can be based on several epistemologies, provided the researcher attends to the internal coherence between method and epistemology, or what the authors call “alignment.”

Originality/value

This study offers a number of implications for the practice of management research, as it shows how the case study approach does not commit the researcher to particular data collection or interpretation methods. Furthermore, the use of cases can be justified according to multiple epistemological orientations.

• Epistemology

Takahashi, A.R.W. and Araujo, L. (2020), "Case study research: opening up research opportunities", RAUSP Management Journal , Vol. 55 No. 1, pp. 100-111. https://doi.org/10.1108/RAUSP-05-2019-0109

Emerald Publishing Limited

Copyright © 2019, Adriana Roseli Wünsch Takahashi and Luis Araujo.

Published in RAUSP Management Journal . Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

1. Introduction

The case study as a research method or strategy brings us to question the very term “case”: after all, what is a case? A case-based approach places accords the case a central role in the research process ( Ragin, 1992 ). However, doubts still remain about the status of cases according to different epistemologies and types of research designs.

Despite these doubts, the case study is ever present in the management literature and represents the main method of management research in Brazil ( Coraiola, Sander, Maccali, & Bulgacov, 2013 ). Between 2001 and 2010, 2,407 articles (83.14 per cent of qualitative research) were published in conferences and management journals as case studies (Takahashi & Semprebom, 2013 ). A search on Spell.org.br for the term “case study” under title, abstract or keywords, for the period ranging from January 2010 to July 2019, yielded 3,040 articles published in the management field. Doing research using case studies, allows the researcher to immerse him/herself in the context and gain intensive knowledge of a phenomenon, which in turn demands suitable methodological principles ( Freitas et al. , 2017 ).

Our objective in this paper is to discuss notions of what constitutes a case and its various applications, considering epistemological positions as well as criteria for rigor and validity. The alignment between these dimensions is put forward as a principle advocating coherence among all phases of the research process.

This article makes two contributions. First, we suggest that there are several epistemological justifications for using case studies. Second, we show that the quality and rigor of academic research with case studies are directly related to the alignment between epistemology and research design rather than to choices of specific forms of data collection or analysis. The article is structured as follows: the following four sections discuss concepts of what is a case, its uses, epistemological grounding as well as rigor and quality criteria. The brief conclusions summarize the debate and invite the reader to delve into the literature on the case study method as a way of furthering our understanding of contemporary management phenomena.

2. What is a case study?

The debate over what constitutes a case in social science is a long-standing one. In 1988, Howard Becker and Charles Ragin organized a workshop to discuss the status of the case as a social science method. As the discussion was inconclusive, they posed the question “What is a case?” to a select group of eight social scientists in 1989, and later to participants in a symposium on the subject. Participants were unable to come up with a consensual answer. Since then, we have witnessed that further debates and different answers have emerged. The original question led to an even broader issue: “How do we, as social scientists, produce results and seem to know what we know?” ( Ragin, 1992 , p. 16).

An important step that may help us start a reflection on what is a case is to consider the phenomena we are looking at. To do that, we must know something about what we want to understand and how we might study it. The answer may be a causal explanation, a description of what was observed or a narrative of what has been experienced. In any case, there will always be a story to be told, as the choice of the case study method demands an answer to what the case is about.

A case may be defined ex ante , prior to the start of the research process, as in Yin’s (2015) classical definition. But, there is no compelling reason as to why cases must be defined ex ante . Ragin (1992 , p. 217) proposed the notion of “casing,” to indicate that what the case is emerges from the research process:

Rather than attempt to delineate the many different meanings of the term “case” in a formal taxonomy, in this essay I offer instead a view of cases that follows from the idea implicit in many of the contributions – that concocting cases is a varied but routine social scientific activity. […] The approach of this essay is that this activity, which I call “casing”, should be viewed in practical terms as a research tactic. It is selectively invoked at many different junctures in the research process, usually to resolve difficult issues in linking ideas and evidence.

In other words, “casing” is tied to the researcher’s practice, to the way he/she delimits or declares a case as a significant outcome of a process. In 2013, Ragin revisited the 1992 concept of “casing” and explored its multiple possibilities of use, paying particular attention to “negative cases.”

According to Ragin (1992) , a case can be centered on a phenomenon or a population. In the first scenario, cases are representative of a phenomenon, and are selected based on what can be empirically observed. The process highlights different aspects of cases and obscures others according to the research design, and allows for the complexity, specificity and context of the phenomenon to be explored. In the alternative, population-focused scenario, the selection of cases precedes the research. Both positive and negative cases are considered in exploring a phenomenon, with the definition of the set of cases dependent on theory and the central objective to build generalizations. As a passing note, it is worth mentioning here that a study of multiple cases requires a definition of the unit of analysis a priori . Otherwise, it will not be possible to make cross-case comparisons.

These two approaches entail differences that go beyond the mere opposition of quantitative and qualitative data, as a case often includes both types of data. Thus, the confusion about how to conceive cases is associated with Ragin’s (1992) notion of “small vs large N,” or McKeown’s (1999) “statistical worldview” – the notion that relevant findings are only those that can be made about a population based on the analysis of representative samples. In the same vein, Byrne (2013) argues that we cannot generate nomothetic laws that apply in all circumstances, periods and locations, and that no social science method can claim to generate invariant laws. According to the same author, case studies can help us understand that there is more than one ideographic variety and help make social science useful. Generalizations still matter, but they should be understood as part of defining the research scope, and that scope points to the limitations of knowledge produced and consumed in concrete time and space.

Thus, what defines the orientation and the use of cases is not the mere choice of type of data, whether quantitative or qualitative, but the orientation of the study. A statistical worldview sees cases as data units ( Byrne, 2013 ). Put differently, there is a clear distinction between statistical and qualitative worldviews; the use of quantitative data does not by itself means that the research is (quasi) statistical, or uses a deductive logic:

Case-based methods are useful, and represent, among other things, a way of moving beyond a useless and destructive tradition in the social sciences that have set quantitative and qualitative modes of exploration, interpretation, and explanation against each other ( Byrne, 2013 , p. 9).

Other authors advocate different understandings of what a case study is. To some, it is a research method, to others it is a research strategy ( Creswell, 1998 ). Sharan Merrian and Robert Yin, among others, began to write about case study research as a methodology in the 1980s (Merrian, 2009), while authors such as Eisenhardt (1989) called it a research strategy. Stake (2003) sees the case study not as a method, but as a choice of what to be studied, the unit of study. Regardless of their differences, these authors agree that case studies should be restricted to a particular context as they aim to provide an in-depth knowledge of a given phenomenon: “A case study is an in-depth description and analysis of a bounded system” (Merrian, 2009, p. 40). According to Merrian, a qualitative case study can be defined by the process through which the research is carried out, by the unit of analysis or the final product, as the choice ultimately depends on what the researcher wants to know. As a product of research, it involves the analysis of a given entity, phenomenon or social unit.

Thus, whether it is an organization, an individual, a context or a phenomenon, single or multiple, one must delimit it, and also choose between possible types and configurations (Merrian, 2009; Yin, 2015 ). A case study may be descriptive, exploratory, explanatory, single or multiple ( Yin, 2015 ); intrinsic, instrumental or collective ( Stake, 2003 ); and confirm or build theory ( Eisenhardt, 1989 ).

both went through the same process of implementing computer labs intended for the use of information and communication technologies in 2007;

both took part in the same regional program (Paraná Digital); and

they shared similar characteristics regarding location (operation in the same neighborhood of a city), number of students, number of teachers and technicians and laboratory sizes.

However, the two institutions differed in the number of hours of program use, with one of them displaying a significant number of hours/use while the other showed a modest number, according to secondary data for the period 2007-2013. Despite the context being similar and the procedures for implementing the technology being the same, the mechanisms of social integration – an idiosyncratic factor of each institution – were different in each case. This explained differences in their use of resource, processes of organizational learning and capacity to absorb new knowledge.

On the other hand, multiple case studies seek evidence in different contexts and do not necessarily require direct comparisons ( Stake, 2003 ). Rather, there is a search for patterns of convergence and divergence that permeate all the cases, as the same issues are explored in every case. Cases can be added progressively until theoretical saturation is achieved. An example is of a study that investigated how entrepreneurial opportunity and management skills were developed through entrepreneurial learning ( Zampier & Takahashi, 2014 ). The authors conducted nine case studies, based on primary and secondary data, with each one analyzed separately, so a search for patterns could be undertaken. The convergence aspects found were: the predominant way of transforming experience into knowledge was exploitation; managerial skills were developed through by taking advantages of opportunities; and career orientation encompassed more than one style. As for divergence patterns: the experience of success and failure influenced entrepreneurs differently; the prevailing rationality logic of influence was different; and the combination of styles in career orientation was diverse.

A full discussion of choice of case study design is outside the scope of this article. For the sake of illustration, we make a brief mention to other selection criteria such as the purpose of the research, the state of the art of the research theme, the time and resources involved and the preferred epistemological position of the researcher. In the next section, we look at the possibilities of carrying out case studies in line with various epistemological traditions, as the answers to the “what is a case?” question reveal varied methodological commitments as well as diverse epistemological and ontological positions ( Ragin, 2013 ).

3. Epistemological positioning of case study research

Ontology and epistemology are like skin, not a garment to be occasionally worn ( Marsh & Furlong, 2002 ). According to these authors, ontology and epistemology guide the choice of theory and method because they cannot or should not be worn as a garment. Hence, one must practice philosophical “self-knowledge” to recognize one’s vision of what the world is and of how knowledge of that world is accessed and validated. Ontological and epistemological positions are relevant in that they involve the positioning of the researcher in social science and the phenomena he or she chooses to study. These positions do not tend to vary from one project to another although they can certainly change over time for a single researcher.

Ontology is the starting point from which the epistemological and methodological positions of the research arise ( Grix, 2002 ). Ontology expresses a view of the world, what constitutes reality, nature and the image one has of social reality; it is a theory of being ( Marsh & Furlong, 2002 ). The central question is the nature of the world out there regardless of our ability to access it. An essentialist or foundationalist ontology acknowledges that there are differences that persist over time and these differences are what underpin the construction of social life. An opposing, anti-foundationalist position presumes that the differences found are socially constructed and may vary – i.e. they are not essential but specific to a given culture at a given time ( Marsh & Furlong, 2002 ).

Epistemology is centered around a theory of knowledge, focusing on the process of acquiring and validating knowledge ( Grix, 2002 ). Positivists look at social phenomena as a world of causal relations where there is a single truth to be accessed and confirmed. In this tradition, case studies test hypotheses and rely on deductive approaches and quantitative data collection and analysis techniques. Scholars in the field of anthropology and observation-based qualitative studies proposed alternative epistemologies based on notions of the social world as a set of manifold and ever-changing processes. In management studies since the 1970s, the gradual acceptance of qualitative research has generated a diverse range of research methods and conceptions of the individual and society ( Godoy, 1995 ).

The interpretative tradition, in direct opposition to positivism, argues that there is no single objective truth to be discovered about the social world. The social world and our knowledge of it are the product of social constructions. Thus, the social world is constituted by interactions, and our knowledge is hermeneutic as the world does not exist independent of our knowledge ( Marsh & Furlong, 2002 ). The implication is that it is not possible to access social phenomena through objective, detached methods. Instead, the interaction mechanisms and relationships that make up social constructions have to be studied. Deductive approaches, hypothesis testing and quantitative methods are not relevant here. Hermeneutics, on the other hand, is highly relevant as it allows the analysis of the individual’s interpretation, of sayings, texts and actions, even though interpretation is always the “truth” of a subject. Methods such as ethnographic case studies, interviews and observations as data collection techniques should feed research designs according to interpretivism. It is worth pointing out that we are to a large extent, caricaturing polar opposites rather characterizing a range of epistemological alternatives, such as realism, conventionalism and symbolic interactionism.

If diverse ontologies and epistemologies serve as a guide to research approaches, including data collection and analysis methods, and if they should be regarded as skin rather than clothing, how does one make choices regarding case studies? What are case studies, what type of knowledge they provide and so on? The views of case study authors are not always explicit on this point, so we must delve into their texts to glean what their positions might be.

Two of the cited authors in case study research are Robert Yin and Kathleen Eisenhardt. Eisenhardt (1989) argues that a case study can serve to provide a description, test or generate a theory, the latter being the most relevant in contributing to the advancement of knowledge in a given area. She uses terms such as populations and samples, control variables, hypotheses and generalization of findings and even suggests an ideal number of case studies to allow for theory construction through replication. Although Eisenhardt includes observation and interview among her recommended data collection techniques, the approach is firmly anchored in a positivist epistemology:

Third, particularly in comparison with Strauss (1987) and Van Maanen (1988), the process described here adopts a positivist view of research. That is, the process is directed toward the development of testable hypotheses and theory which are generalizable across settings. In contrast, authors like Strauss and Van Maanen are more concerned that a rich, complex description of the specific cases under study evolve and they appear less concerned with development of generalizable theory ( Eisenhardt, 1989 , p. 546).

This position attracted a fair amount of criticism. Dyer & Wilkins (1991) in a critique of Eisenhardt’s (1989) article focused on the following aspects: there is no relevant justification for the number of cases recommended; it is the depth and not the number of cases that provides an actual contribution to theory; and the researcher’s purpose should be to get closer to the setting and interpret it. According to the same authors, discrepancies from prior expectations are also important as they lead researchers to reflect on existing theories. Eisenhardt & Graebner (2007 , p. 25) revisit the argument for the construction of a theory from multiple cases:

A major reason for the popularity and relevance of theory building from case studies is that it is one of the best (if not the best) of the bridges from rich qualitative evidence to mainstream deductive research.

Although they recognize the importance of single-case research to explore phenomena under unique or rare circumstances, they reaffirm the strength of multiple case designs as it is through them that better accuracy and generalization can be reached.

Likewise, Robert Yin emphasizes the importance of variables, triangulation in the search for “truth” and generalizable theoretical propositions. Yin (2015 , p. 18) suggests that the case study method may be appropriate for different epistemological orientations, although much of his work seems to invoke a realist epistemology. Authors such as Merrian (2009) and Stake (2003) suggest an interpretative version of case studies. Stake (2003) looks at cases as a qualitative option, where the most relevant criterion of case selection should be the opportunity to learn and understand a phenomenon. A case is not just a research method or strategy; it is a researcher’s choice about what will be studied:

Even if my definition of case study was agreed upon, and it is not, the term case and study defy full specification (Kemmis, 1980). A case study is both a process of inquiry about the case and the product of that inquiry ( Stake, 2003 , p. 136).

Later, Stake (2003 , p. 156) argues that:

[…] the purpose of a case report is not to represent the world, but to represent the case. […] The utility of case research to practitioners and policy makers is in its extension of experience.

Still according to Stake (2003 , pp. 140-141), to do justice to complex views of social phenomena, it is necessary to analyze the context and relate it to the case, to look for what is peculiar rather than common in cases to delimit their boundaries, to plan the data collection looking for what is common and unusual about facts, what could be valuable whether it is unique or common:

Reflecting upon the pertinent literature, I find case study methodology written largely by people who presume that the research should contribute to scientific generalization. The bulk of case study work, however, is done by individuals who have intrinsic interest in the case and little interest in the advance of science. Their designs aim the inquiry toward understanding of what is important about that case within its own world, which is seldom the same as the worlds of researchers and theorists. Those designs develop what is perceived to be the case’s own issues, contexts, and interpretations, its thick descriptions . In contrast, the methods of instrumental case study draw the researcher toward illustrating how the concerns of researchers and theorists are manifest in the case. Because the critical issues are more likely to be know in advance and following disciplinary expectations, such a design can take greater advantage of already developed instruments and preconceived coding schemes.

The aforementioned authors were listed to illustrate differences and sometimes opposing positions on case research. These differences are not restricted to a choice between positivism and interpretivism. It is worth noting that Ragin’s (2013 , p. 523) approach to “casing” is compatible with the realistic research perspective:

In essence, to posit cases is to engage in ontological speculation regarding what is obdurately real but only partially and indirectly accessible through social science. Bringing a realist perspective to the case question deepens and enriches the dialogue, clarifying some key issues while sweeping others aside.

cases are actual entities, reflecting their operations of real causal mechanism and process patterns;

case studies are interactive processes and are open to revisions and refinements; and

social phenomena are complex, contingent and context-specific.

Ragin (2013 , p. 532) concludes:

Lurking behind my discussion of negative case, populations, and possibility analysis is the implication that treating cases as members of given (and fixed) populations and seeking to infer the properties of populations may be a largely illusory exercise. While demographers have made good use of the concept of population, and continue to do so, it is not clear how much the utility of the concept extends beyond their domain. In case-oriented work, the notion of fixed populations of cases (observations) has much less analytic utility than simply “the set of relevant cases,” a grouping that must be specified or constructed by the researcher. The demarcation of this set, as the work of case-oriented researchers illustrates, is always tentative, fluid, and open to debate. It is only by casing social phenomena that social scientists perceive the homogeneity that allows analysis to proceed.

In summary, case studies are relevant and potentially compatible with a range of different epistemologies. Researchers’ ontological and epistemological positions will guide their choice of theory, methodologies and research techniques, as well as their research practices. The same applies to the choice of authors describing the research method and this choice should be coherent. We call this research alignment , an attribute that must be judged on the internal coherence of the author of a study, and not necessarily its evaluator. The following figure illustrates the interrelationship between the elements of a study necessary for an alignment ( Figure 1 ).

In addition to this broader aspect of the research as a whole, other factors should be part of the researcher’s concern, such as the rigor and quality of case studies. We will look into these in the next section taking into account their relevance to the different epistemologies.

4. Rigor and quality in case studies

Traditionally, at least in positivist studies, validity and reliability are the relevant quality criteria to judge research. Validity can be understood as external, internal and construct. External validity means identifying whether the findings of a study are generalizable to other studies using the logic of replication in multiple case studies. Internal validity may be established through the theoretical underpinning of existing relationships and it involves the use of protocols for the development and execution of case studies. Construct validity implies defining the operational measurement criteria to establish a chain of evidence, such as the use of multiple sources of evidence ( Eisenhardt, 1989 ; Yin, 2015 ). Reliability implies conducting other case studies, instead of just replicating results, to minimize the errors and bias of a study through case study protocols and the development of a case database ( Yin, 2015 ).

Several criticisms have been directed toward case studies, such as lack of rigor, lack of generalization potential, external validity and researcher bias. Case studies are often deemed to be unreliable because of a lack of rigor ( Seuring, 2008 ). Flyvbjerg (2006 , p. 219) addresses five misunderstandings about case-study research, and concludes that:

[…] a scientific discipline without a large number of thoroughly executed case studies is a discipline without systematic production of exemplars, and a discipline without exemplars is an ineffective one.

theoretical knowledge is more valuable than concrete, practical knowledge;

the case study cannot contribute to scientific development because it is not possible to generalize on the basis of an individual case;

the case study is more useful for generating rather than testing hypotheses;

the case study contains a tendency to confirm the researcher’s preconceived notions; and

it is difficult to summarize and develop general propositions and theories based on case studies.

These criticisms question the validity of the case study as a scientific method and should be corrected.

The critique of case studies is often framed from the standpoint of what Ragin (2000) labeled large-N research. The logic of small-N research, to which case studies belong, is different. Cases benefit from depth rather than breadth as they: provide theoretical and empirical knowledge; contribute to theory through propositions; serve not only to confirm knowledge, but also to challenge and overturn preconceived notions; and the difficulty in summarizing their conclusions is because of the complexity of the phenomena studies and not an intrinsic limitation of the method.

Thus, case studies do not seek large-scale generalizations as that is not their purpose. And yet, this is a limitation from a positivist perspective as there is an external reality to be “apprehended” and valid conclusions to be extracted for an entire population. If positivism is the epistemology of choice, the rigor of a case study can be demonstrated by detailing the criteria used for internal and external validity, construct validity and reliability ( Gibbert & Ruigrok, 2010 ; Gibbert, Ruigrok, & Wicki, 2008 ). An example can be seen in case studies in the area of information systems, where there is a predominant orientation of positivist approaches to this method ( Pozzebon & Freitas, 1998 ). In this area, rigor also involves the definition of a unit of analysis, type of research, number of cases, selection of sites, definition of data collection and analysis procedures, definition of the research protocol and writing a final report. Creswell (1998) presents a checklist for researchers to assess whether the study was well written, if it has reliability and validity and if it followed methodological protocols.

In case studies with a non-positivist orientation, rigor can be achieved through careful alignment (coherence among ontology, epistemology, theory and method). Moreover, the concepts of validity can be understood as concern and care in formulating research, research development and research results ( Ollaik & Ziller, 2012 ), and to achieve internal coherence ( Gibbert et al. , 2008 ). The consistency between data collection and interpretation, and the observed reality also help these studies meet coherence and rigor criteria. Siggelkow (2007) argues that a case study should be persuasive and that even a single case study may be a powerful example to contest a widely held view. To him, the value of a single case study or studies with few cases can be attained by their potential to provide conceptual insights and coherence to the internal logic of conceptual arguments: “[…] a paper should allow a reader to see the world, and not just the literature, in a new way” ( Siggelkow, 2007 , p. 23).

Interpretative studies should not be justified by criteria derived from positivism as they are based on a different ontology and epistemology ( Sandberg, 2005 ). The rejection of an interpretive epistemology leads to the rejection of an objective reality: “As Bengtsson points out, the life-world is the subjects’ experience of reality, at the same time as it is objective in the sense that it is an intersubjective world” ( Sandberg, 2005 , p. 47). In this event, how can one demonstrate what positivists call validity and reliability? What would be the criteria to justify knowledge as truth, produced by research in this epistemology? Sandberg (2005 , p. 62) suggests an answer based on phenomenology:

This was demonstrated first by explicating life-world and intentionality as the basic assumptions underlying the interpretative research tradition. Second, based on those assumptions, truth as intentional fulfillment, consisting of perceived fulfillment, fulfillment in practice, and indeterminate fulfillment, was proposed. Third, based on the proposed truth constellation, communicative, pragmatic, and transgressive validity and reliability as interpretative awareness were presented as the most appropriate criteria for justifying knowledge produced within interpretative approach. Finally, the phenomenological epoché was suggested as a strategy for achieving these criteria.

From this standpoint, the research site must be chosen according to its uniqueness so that one can obtain relevant insights that no other site could provide ( Siggelkow, 2007 ). Furthermore, the view of what is being studied is at the center of the researcher’s attention to understand its “truth,” inserted in a given context.

The case researcher is someone who can reduce the probability of misinterpretations by analyzing multiple perceptions, searches for data triangulation to check for the reliability of interpretations ( Stake, 2003 ). It is worth pointing out that this is not an option for studies that specifically seek the individual’s experience in relation to organizational phenomena.

In short, there are different ways of seeking rigor and quality in case studies, depending on the researcher’s worldview. These different forms pervade everything from the research design, the choice of research questions, the theory or theories to look at a phenomenon, research methods, the data collection and analysis techniques, to the type and style of research report produced. Validity can also take on different forms. While positivism is concerned with validity of the research question and results, interpretivism emphasizes research processes without neglecting the importance of the articulation of pertinent research questions and the sound interpretation of results ( Ollaik & Ziller, 2012 ). The means to achieve this can be diverse, such as triangulation (of multiple theories, multiple methods, multiple data sources or multiple investigators), pre-tests of data collection instrument, pilot case, study protocol, detailed description of procedures such as field diary in observations, researcher positioning (reflexivity), theoretical-empirical consistency, thick description and transferability.

5. Conclusions

The central objective of this article was to discuss concepts of case study research, their potential and various uses, taking into account different epistemologies as well as criteria of rigor and validity. Although the literature on methodology in general and on case studies in particular, is voluminous, it is not easy to relate this approach to epistemology. In addition, method manuals often focus on the details of various case study approaches which confuse things further.

Faced with this scenario, we have tried to address some central points in this debate and present various ways of using case studies according to the preferred epistemology of the researcher. We emphasize that this understanding depends on how a case is defined and the particular epistemological orientation that underpins that conceptualization. We have argued that whatever the epistemological orientation is, it is possible to meet appropriate criteria of research rigor and quality provided there is an alignment among the different elements of the research process. Furthermore, multiple data collection techniques can be used in in single or multiple case study designs. Data collection techniques or the type of data collected do not define the method or whether cases should be used for theory-building or theory-testing.

Finally, we encourage researchers to consider case study research as one way to foster immersion in phenomena and their contexts, stressing that the approach does not imply a commitment to a particular epistemology or type of research, such as qualitative or quantitative. Case study research allows for numerous possibilities, and should be celebrated for that diversity rather than pigeon-holed as a monolithic research method.

The interrelationship between the building blocks of research

Byrne , D. ( 2013 ). Case-based methods: Why We need them; what they are; how to do them . Byrne D. In D Byrne. and C.C Ragin (Eds.), The SAGE handbooks of Case-Based methods , pp. 1 – 10 . London : SAGE Publications Inc .

Creswell , J. W. ( 1998 ). Qualitative inquiry and research design: choosing among five traditions , London : Sage Publications .

Coraiola , D. M. , Sander , J. A. , Maccali , N. & Bulgacov , S. ( 2013 ). Estudo de caso . In A. R. W. Takahashi , (Ed.), Pesquisa qualitativa em administração: Fundamentos, métodos e usos no Brasil , pp. 307 – 341 . São Paulo : Atlas .

Dyer , W. G. , & Wilkins , A. L. ( 1991 ). Better stories, not better constructs, to generate better theory: a rejoinder to Eisenhardt . The Academy of Management Review , 16 , 613 – 627 .

Eisenhardt , K. ( 1989 ). Building theory from case study research . Academy of Management Review , 14 , 532 – 550 .

Eisenhardt , K. M. , & Graebner , M. E. ( 2007 ). Theory building from cases: Opportunities and challenges . Academy of Management Journal , 50 , 25 – 32 .

Flyvbjerg , B. ( 2006 ). Five misunderstandings about case-study research . Qualitative Inquiry , 12 , 219 – 245 .

Freitas , J. S. , Ferreira , J. C. A. , Campos , A. A. R. , Melo , J. C. F. , Cheng , L. C. , & Gonçalves , C. A. ( 2017 ). Methodological roadmapping: a study of centering resonance analysis . RAUSP Management Journal , 53 , 459 – 475 .

Gibbert , M. , Ruigrok , W. , & Wicki , B. ( 2008 ). What passes as a rigorous case study? . Strategic Management Journal , 29 , 1465 – 1474 .

Gibbert , M. , & Ruigrok , W. ( 2010 ). The “what” and “how” of case study rigor: Three strategies based on published work . Organizational Research Methods , 13 , 710 – 737 .

Godoy , A. S. ( 1995 ). Introdução à pesquisa qualitativa e suas possibilidades . Revista de Administração de Empresas , 35 , 57 – 63 .

Grix , J. ( 2002 ). Introducing students to the generic terminology of social research . Politics , 22 , 175 – 186 .

Marsh , D. , & Furlong , P. ( 2002 ). A skin, not a sweater: ontology and epistemology in political science . In D Marsh. , & G Stoker , (Eds.), Theory and Methods in Political Science , New York, NY : Palgrave McMillan , pp. 17 – 41 .

McKeown , T. J. ( 1999 ). Case studies and the statistical worldview: Review of King, Keohane, and Verba’s designing social inquiry: Scientific inference in qualitative research . International Organization , 53 , 161 – 190 .

Merriam , S. B. ( 2009 ). Qualitative research: a guide to design and implementation .

Ollaik , L. G. , & Ziller , H. ( 2012 ). Distintas concepções de validade em pesquisas qualitativas . Educação e Pesquisa , 38 , 229 – 241 .

Picoli , F. R. , & Takahashi , A. R. W. ( 2016 ). Capacidade de absorção, aprendizagem organizacional e mecanismos de integração social . Revista de Administração Contemporânea , 20 , 1 – 20 .

Pozzebon , M. , & Freitas , H. M. R. ( 1998 ). Pela aplicabilidade: com um maior rigor científico – dos estudos de caso em sistemas de informação . Revista de Administração Contemporânea , 2 , 143 – 170 .

Sandberg , J. ( 2005 ). How do we justify knowledge produced within interpretive approaches? . Organizational Research Methods , 8 , 41 – 68 .

Seuring , S. A. ( 2008 ). Assessing the rigor of case study research in supply chain management. Supply chain management . Supply Chain Management: An International Journal , 13 , 128 – 137 .

Siggelkow , N. ( 2007 ). Persuasion with case studies . Academy of Management Journal , 50 , 20 – 24 .

Stake , R. E. ( 2003 ). Case studies . In N. K. , Denzin , & Y. S. , Lincoln (Eds.). Strategies of Qualitative Inquiry , London : Sage Publications . pp. 134 – 164 .

Takahashi , A. R. W. , & Semprebom , E. ( 2013 ). Resultados gerais e desafios . In A. R. W. , Takahashi (Ed.), Pesquisa qualitativa em administração: Fundamentos, métodos e usos no brasil , pp. 343 – 354 . São Paulo : Atlas .

Ragin , C. C. ( 1992 ). Introduction: Cases of “what is a case? . In H. S. , Becker , & C. C. Ragin and (Eds). What is a case? Exploring the foundations of social inquiry , pp. 1 – 18 .

Ragin , C. C. ( 2013 ). Reflections on casing and Case-Oriented research . In D , Byrne. , & C. C. Ragin (Eds.), The SAGE handbooks of Case-Based methods , London : SAGE Publications , pp. 522 – 534 .

Yin , R. K. ( 2015 ). Estudo de caso: planejamento e métodos , Porto Alegre : Bookman .

Zampier , M. A. , & Takahashi , A. R. W. ( 2014 ). Aprendizagem e competências empreendedoras: Estudo de casos de micro e pequenas empresas do setor educacional . RGO Revista Gestão Organizacional , 6 , 1 – 18 .

Acknowledgements

Author contributions: Both authors contributed equally.

Corresponding author

Related articles, we’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Qualitative vs. Quantitative Research | Differences, Examples & Methods

Qualitative vs. Quantitative Research | Differences, Examples & Methods

Published on April 12, 2019 by Raimo Streefkerk . Revised on June 22, 2023.

When collecting and analyzing data, quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings. Both are important for gaining different kinds of knowledge.

Common quantitative methods include experiments, observations recorded as numbers, and surveys with closed-ended questions.

Quantitative research is at risk for research biases including information bias , omitted variable bias , sampling bias , or selection bias . Qualitative research Qualitative research is expressed in words . It is used to understand concepts, thoughts or experiences. This type of research enables you to gather in-depth insights on topics that are not well understood.

Common qualitative methods include interviews with open-ended questions, observations described in words, and literature reviews that explore concepts and theories.

Table of contents

The differences between quantitative and qualitative research, data collection methods, when to use qualitative vs. quantitative research, how to analyze qualitative and quantitative data, other interesting articles, frequently asked questions about qualitative and quantitative research.

Quantitative and qualitative research use different research methods to collect and analyze data, and they allow you to answer different kinds of research questions.

Quantitative and qualitative data can be collected using various methods. It is important to use a data collection method that will help answer your research question(s).

Many data collection methods can be either qualitative or quantitative. For example, in surveys, observational studies or case studies , your data can be represented as numbers (e.g., using rating scales or counting frequencies) or as words (e.g., with open-ended questions or descriptions of what you observe).

However, some methods are more commonly used in one type or the other.

Quantitative data collection methods

• Surveys :  List of closed or multiple choice questions that is distributed to a sample (online, in person, or over the phone).
• Experiments : Situation in which different types of variables are controlled and manipulated to establish cause-and-effect relationships.
• Observations : Observing subjects in a natural environment where variables can’t be controlled.

Qualitative data collection methods

• Interviews : Asking open-ended questions verbally to respondents.
• Focus groups : Discussion among a group of people about a topic to gather opinions that can be used for further research.
• Ethnography : Participating in a community or organization for an extended period of time to closely observe culture and behavior.
• Literature review : Survey of published works by other authors.

A rule of thumb for deciding whether to use qualitative or quantitative data is:

• Use quantitative research if you want to confirm or test something (a theory or hypothesis )
• Use qualitative research if you want to understand something (concepts, thoughts, experiences)

For most research topics you can choose a qualitative, quantitative or mixed methods approach . Which type you choose depends on, among other things, whether you’re taking an inductive vs. deductive research approach ; your research question(s) ; whether you’re doing experimental , correlational , or descriptive research ; and practical considerations such as time, money, availability of data, and access to respondents.

Quantitative research approach

You survey 300 students at your university and ask them questions such as: “on a scale from 1-5, how satisfied are your with your professors?”

You can perform statistical analysis on the data and draw conclusions such as: “on average students rated their professors 4.4”.

Qualitative research approach

You conduct in-depth interviews with 15 students and ask them open-ended questions such as: “How satisfied are you with your studies?”, “What is the most positive aspect of your study program?” and “What can be done to improve the study program?”

Based on the answers you get you can ask follow-up questions to clarify things. You transcribe all interviews using transcription software and try to find commonalities and patterns.

Mixed methods approach

You conduct interviews to find out how satisfied students are with their studies. Through open-ended questions you learn things you never thought about before and gain new insights. Later, you use a survey to test these insights on a larger scale.

It’s also possible to start with a survey to find out the overall trends, followed by interviews to better understand the reasons behind the trends.

Qualitative or quantitative data by itself can’t prove or demonstrate anything, but has to be analyzed to show its meaning in relation to the research questions. The method of analysis differs for each type of data.

Analyzing quantitative data

Quantitative data is based on numbers. Simple math or more advanced statistical analysis is used to discover commonalities or patterns in the data. The results are often reported in graphs and tables.

Applications such as Excel, SPSS, or R can be used to calculate things like:

• Average scores ( means )
• The number of times a particular answer was given
• The correlation or causation between two or more variables
• The reliability and validity of the results

Analyzing qualitative data

Qualitative data is more difficult to analyze than quantitative data. It consists of text, images or videos instead of numbers.

Some common approaches to analyzing qualitative data include:

• Qualitative content analysis : Tracking the occurrence, position and meaning of words or phrases
• Thematic analysis : Closely examining the data to identify the main themes and patterns
• Discourse analysis : Studying how communication works in social contexts

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts and meanings, use qualitative methods .
• If you want to analyze a large amount of readily-available data, use secondary data. If you want data specific to your purposes with control over how it is generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organize your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Streefkerk, R. (2023, June 22). Qualitative vs. Quantitative Research | Differences, Examples & Methods. Scribbr. Retrieved November 8, 2023, from https://www.scribbr.com/methodology/qualitative-quantitative-research/

Is this article helpful?

Raimo Streefkerk

Other students also liked, what is quantitative research | definition, uses & methods, what is qualitative research | methods & examples, mixed methods research | definition, guide & examples, what is your plagiarism score.

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

What the Case Study Method Really Teaches

• Nitin Nohria

Seven meta-skills that stick even if the cases fade from memory.

It’s been 100 years since Harvard Business School began using the case study method. Beyond teaching specific subject matter, the case study method excels in instilling meta-skills in students. This article explains the importance of seven such skills: preparation, discernment, bias recognition, judgement, collaboration, curiosity, and self-confidence.

During my decade as dean of Harvard Business School, I spent hundreds of hours talking with our alumni. To enliven these conversations, I relied on a favorite question: “What was the most important thing you learned from your time in our MBA program?”

• Nitin Nohria is a professor and former dean at Harvard Business School and the chairman of Thrive Capital, a venture capital firm based in New York.

Partner Center

• Privacy Policy
• SignUp/Login

Home » Case Study – Methods, Examples and Guide

Case Study – Methods, Examples and Guide

Table of Contents

A case study is a research method that involves an in-depth examination and analysis of a particular phenomenon or case, such as an individual, organization, community, event, or situation.

It is a qualitative research approach that aims to provide a detailed and comprehensive understanding of the case being studied. Case studies typically involve multiple sources of data, including interviews, observations, documents, and artifacts, which are analyzed using various techniques, such as content analysis, thematic analysis, and grounded theory. The findings of a case study are often used to develop theories, inform policy or practice, or generate new research questions.

Types of Case Study

Types and Methods of Case Study are as follows:

Single-Case Study

A single-case study is an in-depth analysis of a single case. This type of case study is useful when the researcher wants to understand a specific phenomenon in detail.

For Example , A researcher might conduct a single-case study on a particular individual to understand their experiences with a particular health condition or a specific organization to explore their management practices. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a single-case study are often used to generate new research questions, develop theories, or inform policy or practice.

Multiple-Case Study

A multiple-case study involves the analysis of several cases that are similar in nature. This type of case study is useful when the researcher wants to identify similarities and differences between the cases.

For Example, a researcher might conduct a multiple-case study on several companies to explore the factors that contribute to their success or failure. The researcher collects data from each case, compares and contrasts the findings, and uses various techniques to analyze the data, such as comparative analysis or pattern-matching. The findings of a multiple-case study can be used to develop theories, inform policy or practice, or generate new research questions.

Exploratory Case Study

An exploratory case study is used to explore a new or understudied phenomenon. This type of case study is useful when the researcher wants to generate hypotheses or theories about the phenomenon.

For Example, a researcher might conduct an exploratory case study on a new technology to understand its potential impact on society. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as grounded theory or content analysis. The findings of an exploratory case study can be used to generate new research questions, develop theories, or inform policy or practice.

Descriptive Case Study

A descriptive case study is used to describe a particular phenomenon in detail. This type of case study is useful when the researcher wants to provide a comprehensive account of the phenomenon.

For Example, a researcher might conduct a descriptive case study on a particular community to understand its social and economic characteristics. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a descriptive case study can be used to inform policy or practice or generate new research questions.

Instrumental Case Study

An instrumental case study is used to understand a particular phenomenon that is instrumental in achieving a particular goal. This type of case study is useful when the researcher wants to understand the role of the phenomenon in achieving the goal.

For Example, a researcher might conduct an instrumental case study on a particular policy to understand its impact on achieving a particular goal, such as reducing poverty. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of an instrumental case study can be used to inform policy or practice or generate new research questions.

Case Study Data Collection Methods

Here are some common data collection methods for case studies:

Interviews involve asking questions to individuals who have knowledge or experience relevant to the case study. Interviews can be structured (where the same questions are asked to all participants) or unstructured (where the interviewer follows up on the responses with further questions). Interviews can be conducted in person, over the phone, or through video conferencing.

Observations

Observations involve watching and recording the behavior and activities of individuals or groups relevant to the case study. Observations can be participant (where the researcher actively participates in the activities) or non-participant (where the researcher observes from a distance). Observations can be recorded using notes, audio or video recordings, or photographs.

Documents can be used as a source of information for case studies. Documents can include reports, memos, emails, letters, and other written materials related to the case study. Documents can be collected from the case study participants or from public sources.

Surveys involve asking a set of questions to a sample of individuals relevant to the case study. Surveys can be administered in person, over the phone, through mail or email, or online. Surveys can be used to gather information on attitudes, opinions, or behaviors related to the case study.

Artifacts are physical objects relevant to the case study. Artifacts can include tools, equipment, products, or other objects that provide insights into the case study phenomenon.

How to conduct Case Study Research

Conducting a case study research involves several steps that need to be followed to ensure the quality and rigor of the study. Here are the steps to conduct case study research:

• Define the research questions: The first step in conducting a case study research is to define the research questions. The research questions should be specific, measurable, and relevant to the case study phenomenon under investigation.
• Select the case: The next step is to select the case or cases to be studied. The case should be relevant to the research questions and should provide rich and diverse data that can be used to answer the research questions.
• Collect data: Data can be collected using various methods, such as interviews, observations, documents, surveys, and artifacts. The data collection method should be selected based on the research questions and the nature of the case study phenomenon.
• Analyze the data: The data collected from the case study should be analyzed using various techniques, such as content analysis, thematic analysis, or grounded theory. The analysis should be guided by the research questions and should aim to provide insights and conclusions relevant to the research questions.
• Draw conclusions: The conclusions drawn from the case study should be based on the data analysis and should be relevant to the research questions. The conclusions should be supported by evidence and should be clearly stated.
• Validate the findings: The findings of the case study should be validated by reviewing the data and the analysis with participants or other experts in the field. This helps to ensure the validity and reliability of the findings.
• Write the report: The final step is to write the report of the case study research. The report should provide a clear description of the case study phenomenon, the research questions, the data collection methods, the data analysis, the findings, and the conclusions. The report should be written in a clear and concise manner and should follow the guidelines for academic writing.

Examples of Case Study

Here are some examples of case study research:

• The Hawthorne Studies : Conducted between 1924 and 1932, the Hawthorne Studies were a series of case studies conducted by Elton Mayo and his colleagues to examine the impact of work environment on employee productivity. The studies were conducted at the Hawthorne Works plant of the Western Electric Company in Chicago and included interviews, observations, and experiments.
• The Stanford Prison Experiment: Conducted in 1971, the Stanford Prison Experiment was a case study conducted by Philip Zimbardo to examine the psychological effects of power and authority. The study involved simulating a prison environment and assigning participants to the role of guards or prisoners. The study was controversial due to the ethical issues it raised.
• The Challenger Disaster: The Challenger Disaster was a case study conducted to examine the causes of the Space Shuttle Challenger explosion in 1986. The study included interviews, observations, and analysis of data to identify the technical, organizational, and cultural factors that contributed to the disaster.
• The Enron Scandal: The Enron Scandal was a case study conducted to examine the causes of the Enron Corporation’s bankruptcy in 2001. The study included interviews, analysis of financial data, and review of documents to identify the accounting practices, corporate culture, and ethical issues that led to the company’s downfall.
• The Fukushima Nuclear Disaster : The Fukushima Nuclear Disaster was a case study conducted to examine the causes of the nuclear accident that occurred at the Fukushima Daiichi Nuclear Power Plant in Japan in 2011. The study included interviews, analysis of data, and review of documents to identify the technical, organizational, and cultural factors that contributed to the disaster.

Application of Case Study

Case studies have a wide range of applications across various fields and industries. Here are some examples:

Business and Management

Case studies are widely used in business and management to examine real-life situations and develop problem-solving skills. Case studies can help students and professionals to develop a deep understanding of business concepts, theories, and best practices.

Case studies are used in healthcare to examine patient care, treatment options, and outcomes. Case studies can help healthcare professionals to develop critical thinking skills, diagnose complex medical conditions, and develop effective treatment plans.

Case studies are used in education to examine teaching and learning practices. Case studies can help educators to develop effective teaching strategies, evaluate student progress, and identify areas for improvement.

Social Sciences

Case studies are widely used in social sciences to examine human behavior, social phenomena, and cultural practices. Case studies can help researchers to develop theories, test hypotheses, and gain insights into complex social issues.

Law and Ethics

Case studies are used in law and ethics to examine legal and ethical dilemmas. Case studies can help lawyers, policymakers, and ethical professionals to develop critical thinking skills, analyze complex cases, and make informed decisions.

Purpose of Case Study

The purpose of a case study is to provide a detailed analysis of a specific phenomenon, issue, or problem in its real-life context. A case study is a qualitative research method that involves the in-depth exploration and analysis of a particular case, which can be an individual, group, organization, event, or community.

The primary purpose of a case study is to generate a comprehensive and nuanced understanding of the case, including its history, context, and dynamics. Case studies can help researchers to identify and examine the underlying factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and detailed understanding of the case, which can inform future research, practice, or policy.

Case studies can also serve other purposes, including:

• Illustrating a theory or concept: Case studies can be used to illustrate and explain theoretical concepts and frameworks, providing concrete examples of how they can be applied in real-life situations.
• Developing hypotheses: Case studies can help to generate hypotheses about the causal relationships between different factors and outcomes, which can be tested through further research.
• Providing insight into complex issues: Case studies can provide insights into complex and multifaceted issues, which may be difficult to understand through other research methods.
• Informing practice or policy: Case studies can be used to inform practice or policy by identifying best practices, lessons learned, or areas for improvement.

Advantages of Case Study Research

There are several advantages of case study research, including:

• In-depth exploration: Case study research allows for a detailed exploration and analysis of a specific phenomenon, issue, or problem in its real-life context. This can provide a comprehensive understanding of the case and its dynamics, which may not be possible through other research methods.
• Rich data: Case study research can generate rich and detailed data, including qualitative data such as interviews, observations, and documents. This can provide a nuanced understanding of the case and its complexity.
• Holistic perspective: Case study research allows for a holistic perspective of the case, taking into account the various factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and comprehensive understanding of the case.
• Theory development: Case study research can help to develop and refine theories and concepts by providing empirical evidence and concrete examples of how they can be applied in real-life situations.
• Practical application: Case study research can inform practice or policy by identifying best practices, lessons learned, or areas for improvement.
• Contextualization: Case study research takes into account the specific context in which the case is situated, which can help to understand how the case is influenced by the social, cultural, and historical factors of its environment.

Limitations of Case Study Research

There are several limitations of case study research, including:

• Limited generalizability : Case studies are typically focused on a single case or a small number of cases, which limits the generalizability of the findings. The unique characteristics of the case may not be applicable to other contexts or populations, which may limit the external validity of the research.
• Biased sampling: Case studies may rely on purposive or convenience sampling, which can introduce bias into the sample selection process. This may limit the representativeness of the sample and the generalizability of the findings.
• Subjectivity: Case studies rely on the interpretation of the researcher, which can introduce subjectivity into the analysis. The researcher’s own biases, assumptions, and perspectives may influence the findings, which may limit the objectivity of the research.
• Limited control: Case studies are typically conducted in naturalistic settings, which limits the control that the researcher has over the environment and the variables being studied. This may limit the ability to establish causal relationships between variables.
• Time-consuming: Case studies can be time-consuming to conduct, as they typically involve a detailed exploration and analysis of a specific case. This may limit the feasibility of conducting multiple case studies or conducting case studies in a timely manner.
• Resource-intensive: Case studies may require significant resources, including time, funding, and expertise. This may limit the ability of researchers to conduct case studies in resource-constrained settings.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Qualitative Research – Methods, Analysis Types...

Focus Groups – Steps, Examples and Guide

Basic Research – Types, Methods and Examples

Ethnographic Research -Types, Methods and Guide

Qualitative Research Methods

Research Methods – Types, Examples and Guide

Research Methods

• Research Process
• Research Design & Method

Qualitative vs. Quantiative

Correlational vs. experimental, empirical vs. non-empirical.

• Survey Research
• Survey & Interview Data Analysis
• Resources for Research
• Ethical Considerations in Research

Qualitative Research gathers data about lived experiences, emotions or behaviors, and the meanings individuals attach to them. It assists in enabling researchers to gain a better understanding of complex concepts, social interactions or cultural phenomena. This type of research is useful in the exploration of how or why things have occurred, interpreting events and describing actions.

Quantitative Research gathers numerical data which can be ranked, measured or categorized through statistical analysis. It assists with uncovering patterns or relationships, and for making generalizations. This type of research is useful for finding out how many, how much, how often, or to what extent.

Correlational Research cannot determine causal relationships. Instead they examine relationships between variables.

Experimental Research can establish causal relationship and variables can be manipulated.

Empirical Studies are based on evidence. The data is collected through experimentation or observation.

Non-empirical Studies do not require researchers to collect first-hand data.

• << Previous: Research Design & Method
• Next: Survey Research >>
• Last Updated: Apr 5, 2023 4:19 PM
• URL: https://semo.libguides.com/ResearchMethods

Case Study vs. Experiment

What's the difference.

Case studies and experiments are both research methods used in various fields to gather data and draw conclusions. However, they differ in their approach and purpose. A case study involves in-depth analysis of a particular individual, group, or situation, aiming to provide a detailed understanding of a specific phenomenon. On the other hand, an experiment involves manipulating variables and observing the effects on a sample population, aiming to establish cause-and-effect relationships. While case studies provide rich qualitative data, experiments provide quantitative data that can be statistically analyzed. Ultimately, the choice between these methods depends on the research question and the desired outcomes.

Further Detail

Introduction.

When conducting research, there are various methods available to gather data and analyze phenomena. Two commonly used approaches are case study and experiment. While both methods aim to provide insights and answers to research questions, they differ in their design, implementation, and the type of data they generate. In this article, we will explore the attributes of case study and experiment, highlighting their strengths and limitations.

A case study is an in-depth investigation of a particular individual, group, or phenomenon. It involves collecting and analyzing detailed information from multiple sources, such as interviews, observations, documents, and archival records. Case studies are often used in social sciences, psychology, and business research to gain a deep understanding of complex and unique situations.

One of the key attributes of a case study is its ability to provide rich and detailed data. Researchers can gather a wide range of information, allowing for a comprehensive analysis of the case. This depth of data enables researchers to explore complex relationships, identify patterns, and generate new hypotheses.

Furthermore, case studies are particularly useful when studying rare or unique phenomena. Since they focus on specific cases, they can provide valuable insights into situations that are not easily replicated or observed in controlled experiments. This attribute makes case studies highly relevant in fields where generalizability is not the primary goal.

However, it is important to note that case studies have limitations. Due to their qualitative nature, the findings may lack generalizability to broader populations or contexts. The small sample size and the subjective interpretation of data can also introduce bias. Additionally, case studies are time-consuming and resource-intensive, requiring extensive data collection and analysis.

An experiment is a research method that involves manipulating variables and measuring their effects on outcomes. It aims to establish cause-and-effect relationships by controlling and manipulating independent variables while keeping other factors constant. Experiments are commonly used in natural sciences, psychology, and medicine to test hypotheses and determine the impact of specific interventions or treatments.

One of the key attributes of an experiment is its ability to establish causal relationships. By controlling variables and randomly assigning participants to different conditions, researchers can confidently attribute any observed effects to the manipulated variables. This attribute allows for strong internal validity, making experiments a powerful tool for drawing causal conclusions.

Moreover, experiments often provide quantitative data, allowing for statistical analysis and objective comparisons. This attribute enhances the precision and replicability of findings, enabling researchers to draw more robust conclusions. The ability to replicate experiments also contributes to the cumulative nature of scientific knowledge.

However, experiments also have limitations. They are often conducted in controlled laboratory settings, which may limit the generalizability of findings to real-world contexts. Ethical considerations may also restrict the manipulation of certain variables or the use of certain interventions. Additionally, experiments can be time-consuming and costly, especially when involving large sample sizes or long-term follow-ups.

While case studies and experiments have distinct attributes, they can complement each other in research. Case studies provide in-depth insights and a rich understanding of complex phenomena, while experiments offer controlled conditions and the ability to establish causal relationships. By combining these methods, researchers can gain a more comprehensive understanding of the research question at hand.

When deciding between case study and experiment, researchers should consider the nature of their research question, the available resources, and the desired level of control and generalizability. Case studies are particularly suitable when exploring unique or rare phenomena, aiming for depth rather than breadth, and when resources allow for extensive data collection and analysis. On the other hand, experiments are ideal for establishing causal relationships, testing specific hypotheses, and when control over variables is crucial.

In conclusion, case study and experiment are two valuable research methods with their own attributes and limitations. Both approaches contribute to the advancement of knowledge in various fields, and their selection depends on the research question, available resources, and desired outcomes. By understanding the strengths and weaknesses of each method, researchers can make informed decisions and conduct rigorous and impactful research.

Comparisons may contain inaccurate information about people, places, or facts. Please report any issues.

Home » Education » Difference Between Action Research and Case Study

Difference Between Action Research and Case Study

Main difference – action research vs case study.

Research is the careful study of a given field or problem in order to discover new facts or principles. Action research and case study are two types of research, which are mainly used in the field of social sciences and humanities. The main difference between action research and case study is their purpose; an action research study aims to solve an immediate problem whereas a case study aims to provide an in-depth analysis of a situation or case over a long period of time.

1. What is Action Research?      – Definition, Features, Purpose, Process

2. What is Case Study?      – Definition, Features, Purpose, Process

What is Action Research

Action research is a type of a research study that is initiated to solve an immediate problem. It may involve a variety of analytical, investigative and evaluative research methods designed to diagnose and solve problems. It has been defined as “a disciplined process of inquiry conducted by and for those taking the action. The primary reason for engaging in action research is to assist the “actor” in improving and/or refining his or her actions” (Sagor, 2000). This type of research is typically used in the field of education. Action research studies are generally conductors by educators, who also act as participants.

Here, an individual researcher or a group of researchers identify a problem, examine its causes and try to arrive at a solution to the problem. The action research process is as follows.

Action Research Process

• Identify a problem to research
• Clarify theories
• Identify research questions
• Collect data on the problem
• Organise, analyse, and interpret the data
• Create a plan to address the problem
• Implement the above-mentioned plan
• Evaluate the results of the actions taken

The above process will keep repeating. Action research is also known as cycle of inquiry or cycle of action since it follows a specific process that is repeated over time.

What is a Case Study

A case study is basically an in-depth examination of a particular event, situation or an individual. It is a type of research that is designed to explore and understand complex issues; however, it involves detailed contextual analysis of only a limited number of events or situations. It has been defined as “an empirical inquiry that investigates a contemporary phenomenon within its real-life context; when the boundaries between phenomenon and context are not clearly evident; and in which multiple sources of evidence are used.” (Yin, 1984)

Case studies are used in a variety of fields, but fields like sociology and education seem to use them the most. They can be used to probe into community-based problems such as illiteracy, unemployment, poverty, and drug addiction. 

Case studies involve both quantitative and qualitative data and allow the researchers to see beyond statistical results and understand human conditions. Furthermore, case studies can be classified into three categories, known as exploratory, descriptive and explanatory case studies.

However, case studies are also criticised since the study of a limited number of events or cases cannot easily establish generality or reliability of the findings. The process of a case study is generally as follows:

Case Study Process

• Identifying and defining the research questions
• Selecting the cases and deciding techniques for data collection and analysis
• Collecting data in the field
• Evaluating and analysing the data
• Preparing the report

Action Research : Action research is a type of a research study that is initiated to solve an immediate problem.

Case Study : Case study is an in-depth analysis of a particular event or case over a long period of time.                         

Action Research : Action research involves solving a problem.

Case Study : Case studies involve observing and analysing a situation.

Action Research : Action research studies are mainly used in the field of education.

Case Study : Case studies are used in many fields; they can be specially used with community problems such as unemployment, poverty, etc.

Action Research : Action research always involve providing a solution to a problem.

Case Study : Case studies do not provide a solution to a problem.

Participants

Action Research : Researchers can also act as participants of the research.

Case Study : Researchers generally don’t take part in the research study.

Zainal, Zaidah.  Case study as a research method . N.p.: n.p., 7 June 2007. PDF.

 Soy, Susan K. (1997).  The case study as a research method . Unpublished paper, University of Texas at Austin.

Sagor, Richard.  Guiding school improvement with action research . Ascd, 2000.

Image Courtesy: Pixabay

About the Author: Hasa

Hasanthi is a seasoned content writer and editor with over 8 years of experience. Armed with a BA degree in English and a knack for digital marketing, she explores her passions for literature, history, culture, and food through her engaging and informative writing.

​You May Also Like These

Leave a reply cancel reply.

• Open access
• Published: 31 October 2023

How to perform prespecified subgroup analyses when using propensity score methods in the case of imbalanced subgroups

• Florian Chatelet 1 , 2 ,
• Benjamin Verillaud 2 , 3 &
• Sylvie Chevret 1  

BMC Medical Research Methodology volume  23 , Article number:  255 ( 2023 ) Cite this article

210 Accesses

Metrics details

Looking for treatment-by-subset interaction on a right-censored outcome based on observational data using propensity-score (PS) modeling is of interest. However, there are still issues regarding its implementation, notably when the subsets are very imbalanced in terms of prognostic features and treatment prevalence.

We conducted a simulation study to compare two main PS estimation strategies, performed either once on the whole sample (“across subset”) or in each subset separately (“within subsets”). Several PS models and estimands are also investigated. We then illustrated those approaches on the motivating example, namely, evaluating the benefits of facial nerve resection in patients with parotid cancer in contact with the nerve, according to pretreatment facial palsy.

Our simulation study demonstrated that both strategies provide close results in terms of bias and variance of the estimated treatment effect, with a slight advantage for the “across subsets” strategy in very small samples, provided that interaction terms between the subset variable and other covariates influencing the choice of treatment are incorporated. PS matching without replacement resulted in biased estimates and should be avoided in the case of very imbalanced subsets.

Conclusions

When assessing heterogeneity in the treatment effect in small samples, the “across subsets” strategy of PS estimation is preferred. Then, either a PS matching with replacement or a weighting method must be used to estimate the average treatment effect in the treated or in the overlap population. In contrast, PS matching without replacement should be avoided in this setting.

Peer Review reports

Randomized controlled trials remain the gold standard for evaluating treatment effects. However, there are several situations where they are challenging to conduct for technical, ethical, or feasibility reasons [ 1 ]. This challenge is particularly evident in the surgical field, where comparative studies, often complex to design, face difficulties in inclusion, with patients and surgeons reluctant to randomize because of a strong prior belief in the superiority of one treatment over another [ 2 ]. Such a difficulty of randomization is similarly observed when evaluating drug effects in rare diseases such in oncology or in vulnerable populations - such as pregnant women, fetuses, neonates, children, prisoners, persons with physical handicaps or mental disabilities, and disadvantaged persons (“the Belmont report”) [ 3 ].

Thus, in these fields, observational studies are frequently used. However, they are subject to many sources of bias because the baseline characteristics of patients receiving the different therapeutic modalities may differ widely regarding important prognostic factors, illustrating the confounding-by-indication bias from nonrandom treatment allocation. These biases should be properly addressed to avoid biasing the treatment estimate [ 4 ]. Multivariable regression has been widely used to that end. However, it is at risk of overfitting in the case of insufficient observations relative to the number of covariates. To overcome these limitations, in 1983, Rosenbaum and Rubin proposed the use of a propensity score (PS), corresponding to the individual probability of receiving the treatment as a function of the measured confounders [ 5 ]. Samples are matched or weighted [ 6 ] to minimize the discrepancies in observed confounders between treatment groups; in other words, individuals are assigned “balancing” weights, derived from their PS, to under- or overrepresent the characteristics of their treatment group compared to the other group. Under the assumptions of consistency, exchangeability, positivity, no interference, and correct model specification, causal estimates of treatment effect can be provided [ 7 ]. Although other causal inference approaches such as g-computation, targeted maximum likelihood estimation, and/or a doubly robust estimator may outperform the propensity score-based approaches [ 8 ], the propensity score-based approaches are still the most popular ones in the medical literature. This is even more prominent in the surgical setting, where 83.8% of such studies have been reported to use PS matching [ 9 ].

Whichever the setting, clinicians and surgeons often have a strong belief regarding which subset of patients may benefit from which treatment. We considered the question of facial nerve resection in patients with parotid cancer as an illustrative example. Facial function weakness is often used as a surrogate of facial nerve involvement, resulting in the choice of nerve resection [ 10 , 11 ]. However, Park et al. recently demonstrated that approximately 1/3 of patients with preoperative facial weakness do not exhibit any perineural invasion on final pathologic examination [ 12 ], so facial nerve sparing could be considered even in this situation. Moreover, facial nerve sacrifice has been reported to significantly reduce the quality of life, despite facial nerve reconstruction [ 13 ]. Thus, whether the facial nerve should be resected in all patients with parotid tumors abutting the facial nerve or only in those with facial palsy is a matter of debate.

From a statistical point of view, this issue raises the concern of treatment-by-subset interaction when using propensity score approaches (where, in the example above facial nerve resection and no resection are the two “treatment” groups, and facial palsy or no facial palsy are the two “subsets”). One issue is whether the PS estimation should be performed once for the whole sample before performing any subset analyses (“across subsets” strategy) or within each subset separately (“within subsets” strategy). Indeed, while in theory, the true PS balances the distribution of covariates between subsets, in practice, this action occurs only with a large number of patients (reported above 1,000) and events [ 14 ]. Thus, the balance between covariates could be improved by estimating the PS in each subset, although this approach may increase the variance in the estimate, with potential numerical issues if there are few patients in one subset [ 15 ]. Otherwise, there are uncertainties concerning the extrapolation of these results when the subsets are very unbalanced, and few studies have considered right-censored outcomes [ 16 ].

To address these issues of estimating the PS before assessing treatment-by-subset interactions on a right-censored outcome on observational data, we conducted a simulation study for the case when the subsets are very imbalanced in terms of prognostic features and treatment prevalence. We then illustrated those approaches on the motivating example.

Motivating example

To illustrate the problem, we used data from an observational prospective multicenter cohort of a French national network, focusing on rare head and neck cancers, the Réseau d’expertise français sur les cancers ORL rares (REFCOR) database. Patients were included between 2009 and 2021 at the time of diagnosis and then followed prospectively. Inclusion was carried out by each center using a standardized questionnaire. In accordance with French law, their data were anonymized, and all patients signed an informed consent form.

Only patients diagnosed with a primary histologically proven parotid cancer who were surgically treated and included in the REFCOR database were included. To address the objectives of this work, we selected patients with a tumor that was in close contact with the nerve. Surgical reports were reviewed to assess the relationships between the facial nerve and the tumor, and close contact was defined as a contact with at least one of the following three criteria:

strong adhesion with the nerve described by the surgeon

peri-neural invasion described by the pathologist

inframillimeter surgical margin as defined by the pathologist.

Patients with a metastasis located in the parotid gland, patients without any follow-up data, and patients treated for recurrence were excluded. To resume the prognosis of each patient, we used a validated prognostic score for parotid cancers, developed by Vander Poorten et al. [ 17 , 18 ], classifying patients into 4 groups representing increased risk of poor survival.

Surgical treatment was performed according to local recommendations after discussion in a multidisciplinary tumor board meeting. For the current study, the treatment of interest was facial nerve resection, defined as resection of the facial nerve trunk or one of its main divisions, for carcinologic purposes.

The primary outcome was overall survival (OS), defined as the time from surgery to death or the last visit. The secondary outcome was disease-free survival (DFS), defined as the time from surgery to death or recurrence (local, regional, or distant) or to the last visit. Survival times longer than 5 years were right-censored.

A total of 707 patients from 21 centers were included in this study (see flow chart in Supplementary Fig. 1, Additional file 1 ). Among these patients, 300 had a tumor in contact with the nerve, including 178 who benefited from a facial nerve resection. Comparison of these 178 patients with facial nerve resection with those 122 patients who did not have any nerve resection revealed marked differences across groups in key prognostic factors, as measured by standardized mean differences (SMDs), of which all but one were above 10% (Table 1 ). Patients who underwent facial nerve resection had deleterious outcomes in terms of both OS and DFS (see Additional file 4 ). Two hundred (66.7%) patients with no FN paresis, compared to 87 (29%) who had pretreatment facial weakness, differed from most prognostic factors, with SMDs above 20% (Table 2 ). Therefore, estimating treatment-by-subset interaction required a propensity score approach to correct for such a potential confounding by indication bias.

• Simulation study

We aim to evaluate from observational data a subset-by-treatment interaction on right-censored data using propensity score methods. To specifically evaluate the empirical performances of the two “across” and “within” subsets strategies, we performed a Monte-Carlo simulation study. We generated data close to the REFCOR setting, where patients with facial palsy, the smaller group of the sample, received mostly (in 80% of cases) a facial nerve resection and may have benefit more to that resection than those without (the majority of the sample but who only received a nerves resection in 40% of cases).

We thus considered a population partitioned into two subsets \((S=1,2)\) of different sizes, with a potential heterogeneity in treatment effect across the subsets. Similarly to the REFCOR study, we considered the treatment effect possibly restricted to the smaller subset, but where the treatment has been widely preferred.

Data generation-generating mechanisms

We considered a population partitioned into two subsets \(S (=1,2)\) of potential differential influence on a right-censored outcome (where large times indicate improved outcomes), with a proportion of \(p(S=1)=0.25\) patients in the subset 1 (the smaller subset).

We simulated samples of n =3,000 patients, with a set of continuous ( \(X_1\) and \(X_2\) ) covariates using independent normal distributions of mean 0 and standard deviation of 1, and seven binary ( \(X_3, \ldots , X_{10}\) ) covariates using independent Bernoulli distributions, with parameter equal to 0.5. Covariates had a strong, moderate or no association, first with outcome, and second with treatment allocation (Table 3 ).

For subject \(i=1,\ldots ,n\) , we generated his(her) belonging to subset \(S=1\) , from a Bernoulli \(S_i \sim B(0.25)\) distribution, then generated the treatment group, \(Z_{i}\) \(\sim B(p_{i})\) with \(logit(p_{i})= \beta _{0|S} + \sum _{j=1}^{10}\beta _{j|S}.x_{j,i}\) , where \(\beta _{0|S=1}\) was set at 0.3 and \(\beta _{0|S=2} = -1.9\) to obtain \(P(Z_i=1|S=1) \approx 0.8\) and \(P(Z_i=1|S=2)\approx 0.4\) (close to the REFCOR proportions of treated patients in each subset); and \(\beta _{j|S}\) denote the different covariate effects on treatment allocation.

Each survival outcome \(T_{i}\) was then generated an exponential distribution with hazard depending on the treatment \(Z_i\) , covariates \((x_{ji},j=1,\ldots ,10)\) and subset \(S_i\) of the patient, given by \(\lambda _{i} = \lambda _0.exp [\theta _S.Z_{i} + \alpha _S.S_i + \sum _{j=1}^{10} \alpha _j.x_{j,i}]\) , where the baseline hazard, \(\lambda _0\) , was set to 0.005, and the conditional treatment effects in subsets 1 and 2 at \(\theta _{S=1}\) and \(\theta _{S=2}\) , respectively, while \(\alpha _S\) denote the effect of the subset on the outcome, and \(\alpha _j\) denote the covariates effect on the outcome (Table 3 ). Strong, moderate and no impact were set by parameter values of \(\log 2, \log 1.3\) and \(\log 1\) , respectively.

We simulated an independent censoring time for each patient using a uniform distribution \(U=[1,150]\) , where patients with a censoring time below the time-to-event, or above 60, were administratively right-censored.

Several scenarios were investigated, depending on the impact of the subsets on the outcome ( \(\alpha _{S=1}\) and \(\alpha _{S=2}\) ), that is without treatment effect and treatment-by-subset interaction (Table 4 ). We then assessed the influence of sample size ( n , from 500 to 5,000), proportions of patients in each subset ( \(p(S=1))\) , relative risks to be treated in each subset ( \(\beta _{0|S=1}\) and \(\beta _{0|S=2}\) ), and impact of covariates on the outcome ( \(\alpha _j\) ).

Estimand/target of analysis

True causal marginal treatment effect in the treated, as measured on the log HR scale, were computed for each scenario in each subset \(S=1,2\) , using a sample of 1,000,000 individuals.

In looking for treatment-by-subset interaction, two strategies of analysis regarding the PS estimation were considered and applied to each dataset. First, the “across subsets” strategy was used, which consisted of estimating a single PS from the whole sample but incorporating the subset indicator and potential interaction terms into the PS. Second, we applied the “within subsets” strategy, which consisted of estimating the PS in each subset separately.

Regardless of the strategy, several PS methods were applied, targeting the average treatment effect in the treated group (ATT) and then the average treatment effect in the overlap (ATO). Thus, we first performed PS matching without replacement using a 1-to-1 nearest neighbor matching algorithm, with a caliper set to 0.2, then to 0.1 standard deviations of the logit of the PS [ 19 ]. The hazard ratio (HR) of an event was then estimated from a Cox model with a robust estimator of the variance. In a second approach, we allowed replacement in the untreated group, calculating the variance in the estimator based on the Austin and Caufri estimator [ 20 ]. We also used a PS weighting approach, with standardized mortality ratio weights (SMRWs) [ 21 ] after stabilization [ 22 ] and with a bootstraped variance estimation [ 23 ], and then with overlap weights [ 24 ], with a robust estimation of the standard errors to account for weighting.

To evaluate the influence of the PS model, we used different models for PS estimation. “True PS” was defined as multivariable logistic regression including true confounders (variables affecting both treatment allocation and outcome) and interaction terms between the subset and variables with different effects on treatment allocation ( X 4 and X 9). We further included X 6 for the “PS with a prognostic variable”, X 1 for the “PS with an instrumental variable”, interaction terms with X 2, X 5, X 7 and X 10 for the “PS with all interaction terms”, and we omitted all interaction terms for the “PS without interaction term”.

Performance measures

To assess the performances of these methods, \(n_{sim}=\) 1,000 independent replications of each scenario were performed, corresponding to a \(< 1\%\) Monte Carlo standard error, for a coverage of 95% [ 25 ].

Over those replications, we computed the mean bias, defined as the average difference between the estimated treatment effect and the true marginal treatment effect, and the coverage of the 95% confidence interval, defined as the percent of time the true treatment effect was included in the 95% confidence interval. We also reported the 95% confidence interval of the bias estimation, using the Monte Carlo standard error, and recorded the frequency of non-convergence issues.

The simulation study and analyses for the applied example were performed in R version 4.1.3 using the “survival”, “survey”, “simsurv”, “ggplot2”, “survminer”, “tableone”, “mice”, “MatchIt”, “MatchThem”, “WeightIt”, “cobalt”, “boot”, “VIM” and “forestplot” packages.

We first considered samples of \(n=3,000\) individuals. In Scenario 1, where some treatment-by-subset interaction was introduced, both the “across subsets” and “within subsets” strategies yielded similar results, except using the “across subsets” approach when no interaction term was included in the PS model. Using this PS model, an important bias in the estimation of the treatment effect and impaired coverage in subset 1 were observed (Fig. 1 ). As expected, the inclusion of an instrumental variable in the PS model increased the variance in the estimation (Fig. 1 and Supplementary Fig. 3, Additional file 3 ). Bias in the estimated effect was higher in subset 1 than in subset 2 and was proportional to the treatment effect (Fig. 2 ).

Comparison of strategies according to the PS model in Scenario 1. Comparison of “across subsets” or “within subsets” strategy in terms of the mean absolute bias ( A ), variance ( B ) and the coverage of the 95% CI ( C ) according to the PS model. S1 = subset 1; S2 = subset 2

Bias according to the treatment effect in subset 1 (Scenario 2). Comparison of the “across subsets” or “within subsets” strategy in terms of the bias ( A ) variance ( B ) and the coverage of the 95% CI ( C ) for the estimation of the treatment effect in each subset

Using PS matching, caliper set at 0.1 gave similar results. We further show only results with the caliper set at 0.2 standard deviations of the logit of the PS. PS matching without replacement resulted in greater amounts of bias than the other approaches. This result can be explained by the discarding of treated patients because of the lack of comparative untreated patients. This bias was thus inversely proportional to the proportion of treated patients who could be matched in both strategies and inversely proportional to the relative number of comparative untreated patients (Fig. 3 ). When the relative risk to be treated increased in the small subset (subset 1), the “across subsets” strategy was significantly biased compared to the “within subsets” strategy using PS matching without replacement; however, this bias was controlled with replacement or PS-weighting methods (Supplementary Fig. 4B, Additional file 3 ). Given the importance of this bias, PS matching without replacement was not represented in the following simulations. The results of the simulations with this method can be found in Supplementary Fig. 5, Additional file 3 .

Bias in the estimation of the treatment effect under PS matching without replacement using the “across subsets” or the “within subsets” strategy, according to the treatment prevalence ( A ), the relative risk to be treated in subset 1 ( B ) and the treatment effect in subset 1 ( C ) (Scenario 1)

When sample size decreased down to \(n=300\) , convergence issues occurred, notably using SMRW, while variance inflated using other methods, especially with the “within subets” strategy, which also reflects a convergence problem even if an estimation of the treatment effect could be obtained (Supplementary Figs. 6-8, Additional file 3 ). When the sample size increased from 300 to 5,000, results were poorly affected, except that PS matching without replacement achieved a decrease in variance while the bias persisted, resulting in a lowered coverage probability of confidence interval (Supplementary Fig. 9, Additional file 3 ), while type I error rate slightly decreased (Supplementary Fig. 5, Additional file 3 ). Otherwise, results were not markedly impacted by the size of the subsets (Supplementary Fig. 10, Additional file 3 ), the prognostic value of the subsets (Supplementary Fig. 11, Additional file 3 ), or by the treatment prevalence (Supplementary Fig. 4, Additional file 3 ).

When a non-observed confounder was generated, all methods were biased, as expected. Bias was proportional to the impact of confounders on the outcome and inversely proportional to its correlation with an observed covariate (Fig. 4 ). This also resulted in a decrease of the coverage probability of the confidence interval, more pronounced with the “within subsets” approach. Overall, the overlap weighting and matching with replacement were slightly more robust than the SMRW weighting.

Simulations with an unknown confounder. Comparison of the “across subsets” or “within subsets” strategy in terms of bias ( A ), variance ( B ) and coverage ( C ) in the estimation of the treatment’s effect according to the presence of an unknown confounder (Scenario 1)

In the absence of treatment-by-subset interactions, whichever there was a treatment effect or not (Scenarios 3 and 4), type I error of the Gail & Simon interaction quantitative test was maintained (Supplementary Fig. 12, Additional file 3 ). However, the “across subsets” strategy appeared to be slightly more powerful for detecting an interaction in small samples (Fig. 5 C). Weighting methods (overlap weighting and SMRW weighting) also seemed to be more powerful than PS matching with replacement (Fig. 5 ).

Power of the interaction test. Comparison of the power of the Gail and Simon quantitative interaction test by the number of patients ( A ) (Scenario 1) or the treatment effect ( B and C ) (Scenario 2). The number of patients is set to n =3000 in B and n =300 in C (Scenario 1). Robust estimate of variance was used for SMRW weighting when \(n=300\) , rather than bootstrapping, due to the importance of convergence problems that made it impratical to compute it

Revisited motivating example

We applied similar methods as in the simulation study. The PS, defined as the probability of receiving a nerve resection, was estimated by a multivariable logistic regression model, including age at diagnosis, sex, tumor size (with log transformation), extraparenchymal invasion, skin or bone invasion, cN status, M stage, histological grade, histological type (adenoid cystic carcinoma or not), whether a total parotidectomy was performed, and whether a neck dissection was performed. These variables were chosen because of their known prognostic value and were measured before or at the time of the treatment choice. The T stage was not included in this main analysis because facial nerve invasion classifies the tumor as T4, thus almost consistently resulting in the resection of the facial nerve.

Regardless of the approach, in the matched or in the weighted pseudopopulations, the quality of the balance between the treatment groups was measured using the SMDs of potential confounders and of PS and based on the overlap coefficients (OVL) [ 26 ].

Interaction terms and/or quadratic terms were incorporated into the PS until a satisfactory balance was achieved. To display the SMDs in both subsets for each confounder, the connect-S plot proposed by Yang et al. [ 27 ] was used.

To address missing data, we performed multiple imputation with chained equations. We imputed 33 datasets, with 20 iterations, using an imputation model including important variables, the estimated cumulative baseline hazard based on the Nelson-Aalen estimator and interaction terms between the Nelson-Aalen estimator and covariates [ 28 ] (details are provided in Additional file 2 ). To account for multiple imputations, variances in estimated treatment effects were calculated by bootstrap [ 29 ], except for matching without replacement [ 30 ].

All PS models are described in Additional file 5 . For the “across subsets” strategy, we additionally included in the PS model multiple interaction terms between the subset of interest (pretreatment facial palsy) and the prognostic covariates, whose effect on treatment choice was potentially modified by the existence of preoperative facial palsy (i.e., tumor grade, adenoid cystic carcinoma, extraparenchymal invasion, and bone or skin invasion).

Balances of covariates across treatment groups in each subset were more easily achieved with the “across subsets” strategy (Fig. 6 and Supplementary Figs. 11 to 14, Additional file 5 ) than with the “within subsets” strategy (Fig. 7 and Supplementary Fig. 15 to 22). Compared to the “within subsets” strategy, in the facial palsy subset, the “across subsets” strategy allowed us to include 18-22% more patients with PS weighting methods (means of 201.5 and 17.2 for weighted patients with SMRW and overlap weights, respectively, vs. 164.8 and 14.6 for the “within subsets” strategy), 62% more patients with the PS matching method (means of 22.4 vs. 13.8 patients) and 85% more patients with PS matching with replacement (means of 152.8 vs. 82.4 patients) (Fig. 8 ).

REFCOR data: Connect-S-plot with the “across subsets” approach. Connect-S plot representing standardized mean differences (SMDs) between treatment groups in the “across subsets” approach, in the original dataset ("naive estimation") and according to the PS-based method

REFCOR data: Connect-S-plot with the “within subsets” approach. Connect-S-plot representing standardized mean differences (SMDs) between treatment groups in the “within subsets” approach in the original dataset (first two lines) and according to the PS-based method

The treatment effects in both subsets obtained with the different PS-based methods are summarized in Fig. 8 for OS and DFS. No treatment-by-subset interaction was found regardless of the PS estimation strategy and the PS method with the recommended methods, but the interaction was significant when using the biased “across” method without an interaction term, using PS weighting methods. We used previously simulated data to obtain further insights into these results. To demonstrate a difference in our illustrative example, we used the Gail & Simon interaction quantitative test, which showed that 1,300 to 2,000 patients would have been required to demonstrate an interaction between subset 1 with log(HR) = -0.7 and subset 2 with log(HR) = 0 on the outcome, with a power of 80%. Otherwise, a log (HR) of -2 to -3 in subset 1 was also needed, depending on methods, to demonstrate an interaction with only 300 patients (Fig. 5 ).

REFCOR data: Forest plots. Forest plots representing treatment effects in the subsets using the “across subsets” strategy with or without interaction terms and with the “within subsets” strategy. HR of death ( A ) and event ( B ) regarding facial nerve resection are represented. The first three lines refer to naive analyses performed on the original samples, ignoring potential confounding-by-indication bias. HR = hazard ratio. DFS = disease-free survival

In this study, we considered the issue of using propensity scores to estimate the heterogeneity in the treatment effect across baseline subsets. To address this issue, two strategies for estimating the propensity score were compared.

The first strategy consisted of estimating the propensity score on the whole sample, incorporating the subset variable, to create either a matched population or a pseudo population according to the PS-based method used. The treatment-by-subset interaction was then studied in the resulting whole matched or weighted sample. This strategy is theoretically valid because when the population is balanced on the true propensity score, the subsets are also theoretically balanced on treatment groups, as previously demonstrated [ 31 ]. However, in real life, the true propensity score is not known and must be estimated from the sample. This strategy can therefore lead to a poor balance between treatment groups in the covariates within subsets or even to a worsening of this imbalance [ 15 ]. In our illustrative case, this strategy afforded a good balance of covariates overall, although some imbalances persisted across treatment groups in the subsets (Figs. 6 and 7 ). Expectedly, most persisted differences were found in the small subset of patients with facial palsy (ranging from 93 original observations down to 17 in the overlap population with the “across subsets” strategy and 15 with the “within subsets” strategy).

The second strategy consisted of estimating the propensity score within the subsets, separately. The propensity scores were then used to create a matched population or a pseudo population in each subset, allowing the treatment effect to be evaluated in each subset separately. Then, treatment-by-subset interaction can be tested using the Gail and Simon statistics. This strategy, which should make it easier to obtain a balance in each subset, as previously demonstrated [ 15 ], did not work well in the case of our illustrative example. This result is likely because one subset had few patients, particularly in the case of PS matching without replacement, which suffered even more than the “across subsets” strategy from the limitations of adopting this approach for small samples [ 32 ].

Our simulation study showed that the two “across subsets” and “within subsets” strategies achieve similar results in terms of bias and variance, provided that interaction terms between the subset variable and other covariates influencing the choice of treatment are incorporated. Otherwise, the omission of these interaction terms based on the “across subsets” strategy induced an important bias, regardless of the PS-based method used, which confirms previous results [ 33 , 34 ]. This bias led to the identification of an interaction that was not found with the other two strategies in our illustrative example. Interestingly, the incorporation of interaction terms that do not exist did not induce bias and only slightly increased the variance. Thus, when using the “across subsets” strategy, these results encourage the nonparsimonious use of interaction terms with the subset of interest. The demonstration of an interaction was also slightly more powerful when using the “across subsets” strategy in the case of a very small sample. These results were confirmed in our illustrative example, in which we found similar treatment effect estimates between methods but with lower variances using the “across subsets” strategy.

Focusing on the covariates included in the PS model, we confirmed that the use of an instrumental variable is detrimental in terms of variance. In contrast, the incorporation of a prognostic variable had little impact on the estimation of the treatment effect. However, the omission of a confounder led to a bias. Our study demonstrated that this bias was less important when matching with replacement or when overlap weight methods were used than when SMRW weighting was used. The “within subsets” strategy was also slightly more robust than the “across subsets” strategy in this case. Although previous studies on this topic focused on PS matching without replacement [ 14 , 15 , 16 , 33 , 34 , 35 , 36 , 37 ], compared to the other methods, this method achieved a bias in the estimation of the treatment effect in our setting of large differences between subsets. This bias has been previously named the “unmatched patient bias” [ 38 ]. In the case of a small sample size, replacement has been demonstrated to reduce this bias [ 39 ]; we indeed found that this bias was proportional to the proportion of matched patients.

Our study has some limitations. First, we used propensity-score methods, while they could be outperformed by g-computation and/or doubly robust estimators [ 40 , 41 ]. Nevertheless, we were only concerned by examining two main issues (imbalanced subgroups, right-censored outcomes) when implementing pre-specified subgroup analyses in a causal inference framework using propensity score approaches. Actually, we placed ourselves in the most popular setting in the medical and surgical literature for evaluating causal effects in observational studies, that is, targeting the ATT. We first used propensity score matching, in line with recent works that used Monte Carlo simulations to evaluate propensity score matching with data from complex sample surveys [ 42 ], when dealing with clustered data [ 43 ], or when a confounder has missing data [ 44 ]. Other PS-based methods could have been used, such as the inverse probability treatment weighting (IPTW), which is commonly used in subset analyses [ 16 , 33 , 36 , 37 ]. This method has been reported to achieve better performance than PS matching in the case of right-censored outcomes [ 16 ]. In secondary analyses, we thus also used IPTW using either standardized mortality ratio weights or overlap weights [ 24 ]. Of note, the later targets another estimand, the ATO. Actually, the ATO targets an “artificial” and less defined population consisting of patients with the highest mutual overlap of PS between the 2 treatment groups. ATO can be considered as an intermediate between the average treatment effect (ATE) and the ATT. The population targeted by the ATO indeed consists of patients with a high probability of appearing in either of the 2 treatment groups, that could be interpreted as a population at clinical equipoise. This complicated interpretation is the main drawback of such overlap weighting. Nevertheless, the overlap weights facilitate a perfect and straight balance between groups and could therefore be largely used in this setting where it is difficult to obtain a satisfactory balance with other methods. Otherwise, overlap weighting has been shown to preserve a higher proportion of the sample with a reduction in bias [ 24 ] and to provide close performances to that of g-computation [ 8 ]. However, the overlap weighting method did not outperform other PS-based methods in our simulation study.

Second, we considered only interactions between the subset and covariates that affected treatment choice rather than the outcome. However, the omission of an interaction term when there is an interaction between the subset and prognostic covariate has already been reported to bias the treatment effect [ 33 ].

Third, we did not study other alternatives to the multivariable logistic model that have been proposed. These alternatives include the use of a generalized propensity score [ 45 ] or a balancing propensity score [ 31 ], extending the covariate balancing propensity score [ 46 ] for multiple subset analyses; however, given the small difference observed between the two abovementioned strategies, we did not evaluate them.

In conclusion, when aiming to evaluate the treatment effect in prespecified subsets from observational data using propensity score approaches, estimating the propensity score in the whole sample appears a valid option compared to the estimation of the propensity score within each subset, provided that interaction terms between the subsets and other covariates are included in the PS model. This “across subsets” strategy could be useful in small samples, especially when the samples are imbalanced in terms of the subsets. Indeed, in this setting, estimating the propensity score can lead to convergence issues in a small subset while preventing a satisfactory balance between treatment groups. Weighting methods appear to be more powerful for demonstrating a treatment-by-subset interaction. In the case of PS matching, the use of replacement appears to be preferred in this setup with a lack of comparable patients, regardless of the PS estimation strategy.

Availability of data and materials

The data that illustrate this study are available from the REFCOR but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the REFCOR by contacting the scientific committee ([email protected]).

Abbreviations

Average treatment effect

Average treatment effect in the overlap

Average treatment effect in the treated

Clinically involved lymph nodes

Disease-free survival

Facial nerve

Hazard-ratio

Inverse probability treatment weighting

Overall survival

Overlap coefficient

• Propensity score

Réseau d’expertise français sur les cancers ORL rares

Standard deviation

Standardized mean difference

Standardized mortality ratio weight

Ergina PL, Cook JA, Blazeby JM, Boutron I, Clavien PA, Reeves BC, et al. Challenges in evaluating surgical innovation. Lancet (London, England). 2009;374(9695):1097–104.

Article   PubMed   PubMed Central   Google Scholar  

Abraham NS, Young JM, Solomon MJ. A systematic review of reasons for nonentry of eligible patients into surgical randomized controlled trials. Surgery. 2006;139(4):469–83.

Vollmer SH, Howard G. Statistical power, the Belmont report, and the ethics of clinical trials. Sci Eng Ethics. 2010;16:675–91.

Article   PubMed   Google Scholar  

Stukel TA, Fisher ES, Wennberg DE, Alter DA, Gottlieb DJ, Vermeulen MJ. Analysis of observational studies in the presence of treatment selection bias: effects of invasive cardiac management on AMI survival using propensity score and instrumental variable methods. JAMA. 2007;297(3):278–85.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55.

Article   Google Scholar  

Austin PC, Schuster T. The performance of different propensity score methods for estimating absolute effects of treatments on survival outcomes: A simulation study. Stat Methods Med Res. 2016;25(5):2214–37.

Cole SR, Hernan MA. Constructing Inverse Probability Weights for Marginal Structural Models. Am J Epidemiol. 2008;168(6):656–64.

Ren J, Cislo P, Cappelleri JC, Hlavacek P, DiBonaventura M. Comparing g-computation, propensity score-based weighting, and targeted maximum likelihood estimation for analyzing externally controlled trials with both measured and unmeasured confounders: a simulation study. BMC Med Res Methodol. 2023;23(1):18.

Grose E, Wilson S, Barkun J, Bertens K, Martel G, Balaa F, et al. Use of Propensity Score Methodology in Contemporary High-Impact Surgical Literature. J Am Coll Surg. 2020;230(1):101-112.e2.

Spiro JD, Spiro RH. Cancer of the parotid gland: role of 7th nerve preservation. World J Surg. 2003;27(7):863–7.

Terhaard C, Lubsen H, Tan B, Merkx T, van der Laan B, Baatenburg de Jong R, et al. Facial nerve function in carcinoma of the parotid gland. Eur J Cancer (Oxford, England: 1990). 2006;42(16):2744–50.

Park W, Park J, Park SI, Kim H, Bae H, Cho J, et al. Clinical outcomes and management of facial nerve in patients with parotid gland cancer and pretreatment facial weakness. Oral Oncol. 2019;89:144–9.

Guntinas-Lichius O, Straesser A, Streppel M. Quality of life after facial nerve repair. Laryngoscope. 2007;117(3):421–6.

Rassen JA, Glynn RJ, Rothman KJ, Setoguchi S, Schneeweiss S. Applying propensity scores estimated in a full cohort to adjust for confounding in subgroup analyses. Pharmacoepidemiol Drug Saf. 2012;21(7):697–709.

Green KM, Stuart EA. Examining moderation analyses in propensity score methods: application to depression and substance use. J Consult Clin Psychol. 2014;82(5):773–83.

Izem R, Liao J, Hu M, Wei Y, Akhtar S, Wernecke M, et al. Comparison of propensity score methods for pre-specified subgroup analysis with survival data. J Biopharm Stat. 2020;30(4):734–51.

Vander Poorten VL, Balm AJ, Hilgers FJ, Tan IB, Loftus-Coll BM, Keus RB, et al. The development of a prognostic score for patients with parotid carcinoma. Cancer. 1999;85(9):2057–67.

Article   CAS   PubMed   Google Scholar  

Vander Poorten VLM, Hart AAM, van der Laan BFAM, Baatenburg de Jong RJ, Manni JJ, Marres HAM, et al. Prognostic index for patients with parotid carcinoma: external validation using the nationwide 1985-1994 Dutch Head and Neck Oncology Cooperative Group database. Cancer. 2003;97(6):1453–63.

Austin PC. Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharm Stat. 2011;10(2):150–61.

Austin PC, Cafri G. Variance estimation when using propensity-score matching with replacement with survival or time-to-event outcomes. Stat Med. 2020;39(11):1623–40.

Sato T, Matsuyama Y. Marginal structural models as a tool for standardization. Epidemiol (Cambridge, Mass). 2003;14(6):680–6.

Xu S, Ross C, Raebel MA, Shetterly S, Blanchette C, Smith D. Use of stabilized inverse propensity scores as weights to directly estimate relative risk and its confidence intervals. Value Health J Int Soc Pharmacoeconomics Outcome Res. 2010;13(2):273–7.

Austin PC. Variance estimation when using inverse probability of treatment weighting (IPTW) with survival analysis. Stat Med. 2016;35(30):5642–55.

Li F, Thomas LE. Addressing extreme propensity scores via the overlap weights. Am J Epidemiol. 2018;188(1):250–7.

Morris T, White I, Crowther M. Using simulation studies to evaluate statistical methods. Stat Med. 2019;38:2087–9.

Franklin JM, Rassen JA, Ackermann D, Bartels DB, Schneeweiss S. Metrics for covariate balance in cohort studies of causal effects. Stat Med. 2014;33(10):1685–99.

Yang S, Lorenzi E, Papadogeorgou G, Wojdyla DM, Li F, Thomas LE. Propensity score weighting for causal subgroup analysis. Stat Med. 2021;40(19):4294–309.

White IR, Royston P. Imputing missing covariate values for the Cox model. Stat Med. 2009;28(15):1982–98.

Ling A, Montez-Rath M, Mathur M, Kapphahn K, Desai M. How to Apply Multiple Imputation in Propensity Score Matching with Partially Observed Confounders: A Simulation Study and Practical Recommendations. J Mod Appl Stat Methods. 2021;19(1):1.

Abadie A, Imbens GW. On the failure of the bootstrap for matching estimators. Econometrica. 2008;76(6):1537–57.

Dong J, Zhang JL, Zeng S, Li F. Subgroup balancing propensity score. Stat Methods Med Res. 2020;29(3):659–76.

Andrillon A, Pirracchio R, Chevret S. Performance of propensity score matching to estimate causal effects in small samples. Stat Methods Med Res. 2020;29(3):644–58.

Marsden AM, Dixon WG, Dunn G, Emsley R. The impact of moderator by confounder interactions in the assessment of treatment effect modification: a simulation study. BMC Med Res Methodol. 2022;22(1):88.

Wang SV, Jin Y, Fireman B, Gruber S, He M, Wyss R, et al. Relative Performance of Propensity Score Matching Strategies for Subgroup Analyses. Am J Epidemiol. 2018;187(8):1799–807.

Girman CJ, Gokhale M, Kou TD, Brodovicz KG, Wyss R, Stürmer T. Assessing the impact of propensity score estimation and implementation on covariate balance and confounding control within and across important subgroups in comparative effectiveness research. Med Care. 2014;52(3):280–7.

Radice R, Ramsahai R, Grieve R, Kreif N, Sadique Z, Sekhon JS. Evaluating treatment effectiveness in patient subgroups: a comparison of propensity score methods with an automated matching approach. Int J Biostat. 2012;8(1):25.

Kreif N, Grieve R, Radice R, Sadique Z, Ramsahai R, Sekhon JS. Methods for estimating subgroup effects in cost-effectiveness analyses that use observational data. Med Dec Making Int J Soc Med Dec Making. 2012;32(6):750–63.

Rosenbaum PR, Rubin DB. Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. Am Stat. 1985;39(1):33–8.

Google Scholar  

Bottigliengo D, Baldi I, Lanera C, Lorenzoni G, Bejko J, Bottio T, et al. Oversampling and replacement strategies in propensity score matching: a critical review focused on small sample size in clinical settings. BMC Med Res Methodol. 2021;21(1):1–16.

Neugebauer R, van der Laan M. Why prefer double robust estimators in causal inference? J Stat Plan Infer. 2005;129(1–2):405–26.

Waernbaum I, Pazzagli L. Model misspecification and bias for inverse probability weighting estimators of average causal effects. Biom J. 2023;65(2):2100118.

Austin PC, Jembere N, Chiu M. Propensity score matching and complex surveys. Stat Methods Med Res. 2018;27(4):1240–57.

Langworthy B, Wu Y, Wang M. An overview of propensity score matching methods for clustered data. Stat Methods Med Res. 2023;32(4):641–55.

Ségalas C, Leyrat C, Carpenter JR, Williamson E. Propensity score matching after multiple imputation when a confounder has missing data. Stat Med. 2023;42(7):1082–95.

Eeren HV, Spreeuwenberg MD, Bartak A, de Rooij M, Busschbach JJV. Estimating subgroup effects using the propensity score method: a practical application in outcomes research. Med Care. 2015;53(4):366–73.

Imai K, Ratkovic M. Covariate balancing propensity score. J R Stat Soc Ser B Stat Methodol. 2014;76(1):243–63.

Download references

Acknowledgements

We thank the REFCOR for providing us with the data.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and affiliations.

ECSTRRA Team, INSERM U1153, Université Paris Cité, 1 avenue Claude Vellefaux, 75010, Paris, France

Florian Chatelet & Sylvie Chevret

ENT and head and neck surgery department, Lariboisiere hospital, 2 rue Ambroise Paré, 75010, Paris, France

Florian Chatelet & Benjamin Verillaud

INSERM U1141 “NeuroDiderot”, Université Paris Cité, 75010, Paris, France

Benjamin Verillaud

You can also search for this author in PubMed   Google Scholar

Contributions

FC designed and wrote the computer code, ran and analysed the simulations and the study. FC and BV get illustrative data. SC and BV directed the research. All authors wrote and reviewed the manuscript. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Florian Chatelet .

Ethics declarations

Ethics approval and consent to participate.

The study used for the motivating example was approved by ethical commitee Comité Consultatif sur le Traitement de l’Information en matière de Recherche dans le domaine de la Santé (CCTIRS #11.337). It also obtained the authorization of the national control for databases (CNIL DR 2012-070) as well as the favourable opinion of the Committee of Protection of the People of 09/06/2011, according to the French law. Informed consent was obtained from all participants included in the study. All methods were carried out in accordance with relevant guidelines and regulations complied with the Helsinki declaration.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1., additional file 2., additional file 3., additional file 4., additional file 5., additional file 6., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Cite this article.

Chatelet, F., Verillaud, B. & Chevret, S. How to perform prespecified subgroup analyses when using propensity score methods in the case of imbalanced subgroups. BMC Med Res Methodol 23 , 255 (2023). https://doi.org/10.1186/s12874-023-02071-8

Download citation

Received : 19 October 2022

Accepted : 16 October 2023

Published : 31 October 2023

DOI : https://doi.org/10.1186/s12874-023-02071-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Subset analyses
• Interaction

BMC Medical Research Methodology

ISSN: 1471-2288

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 02 November 2023

Effect of using a mobile drug management application on medication adherence and hospital readmission among elderly patients with polypharmacy: a randomized controlled trial

• Hossein Poorcheraghi 1 ,
• Reza Negarandeh 2 ,
• Shahzad Pashaeypoor 1 &
• Javad Jorian 3  

BMC Health Services Research volume  23 , Article number:  1192 ( 2023 ) Cite this article

153 Accesses

2 Altmetric

Metrics details

Adherence to complex drug regimens and polypharmacy are among the challenges of old age, which may negatively affect their motivation to continue drug therapy or lead to incorrect drug consumption. The present study was conducted to evaluate the effect of using a mobile drug management application on medication adherence and hospital readmission among polypharmacy older adults.

In this randomized controlled trial study conducted in 2022, with Trial Registration Number (IRCT20191231045966N1) (18/07/2021), 192 Iranian older adults with polypharmacy were selected according to the inclusion criteria and allocated to case and control groups using the block randomization method. The data collection tools included a demographic questionnaire, case report form, and Morisky Medication Adherence Scale. The intervention was done using a mobile drug management application. Drug adherence was measured at baseline and both with hospital readmission were measured after 8 weeks. The collected data were entered into the SPSS software version 22 and analyzed using descriptive (frequency, percentage, mean, standard deviation) and inferential (Chi-square, Fisher’s exact test, independent t-test) statistics.

The case and control groups were homogeneous in terms of demographic variables and drug adherence level before the intervention. A significant difference was found in the drug adherence level after using the app (p < 0.001). Moreover, a significant difference was found in adverse events, including re-hospitalization due to disease aggravation, re-hospitalization due to error in medication consumption, falling, hypo or hypertension, and hypo or hyperglycemia, and medication use accuracy between the groups after the intervention (p < 0.05).

The results showed that using a mobile drug management application that meets the specifications of older adults can improve drug adherence, reduce the adverse events and pave the way for a better disease period management.

Peer Review reports

Population ageing has turned into one of the most important public health challenges in recent years, a phenomenon that has affected Iran at a higher speed compared to other countries [ 1 ]. Ageing is associated with changes in different body organs. In this period of life, chronic diseases threaten the elderly person’s health so that the elderly population is the largest drug consumer in different societies [ 2 ]. Under these circumstances, the elderly should take complex drug regimens for their treatment process leading to a phenomenon known as polypharmacy. There are different definitions for polypharmacy; however, it most commonly refers to daily using five medications or more [ 3 ]. Polypharmacy is extensively prevalent in different societies, for example, its prevalence is about 36% in England where older adults above 65 years constitute one-fifth of the population [ 4 ]. The mean prevalence of polypharmacy is 23.1% in Iranian older adults [32.7% in women and 15.2% in men) [ 5 ]. This phenomenon is associated with adverse consequences, including; drug interactions, error in medication consumption, increased side effects, re-hospitalization, falling, functional and cognitive disorders, imposing financial burdens on the health care system, and finally disability and death. Polypharmacy has a direct relationship with reduced physical activity, motion disorder, decreased appetite, and depression in the elderly, and may seriously affect their medication adherence and quality of life [ 6 ].

Medication adherence is one of the challenges associated with polypharmacy in the elderly population. Medication adherence occurs when a patient takes their medications according to the prescribed dosage, time, frequency, and direction [ 7 ]. Effective medication adherence reduces treatment costs, accelerates the recovery process, stops disease progression, and prevents re-hospitalization [ 7 ]. The personal factors related to drug non-adherence are divided to two categories of intentional non-adherence including self-drug discontinuance and unintentional non-adherence including problems such as forgetfulness, visual impairment, and inability to move, among which forgetting is a very important cause of poor medication adherence [ 8 ]. Poor medication adherence has been reported in 26–59% in older adults, depending on the population and methods used to assess drug adherence [ 9 ]. Also a study on 24,000 Iranian elderly patients showed that 62% of them forgot to take their medications [ 10 ]. Poor drug adherence is associated with worsening of the elderly patients’ health condition, increased hospitalization period, and risk of disease progression, disability, and death [ 11 ].

Technology advances have opened new horizons for management of chronic diseases and improvement of health care services. Use of mobile phone facilitates monitoring of treatment process and health care providers-clients’ relationship [ 12 ]. Today, mobile health (mHealth) is one of the most up-to-date types of health care interventions that can play an effective role in promoting older adults’ health. Mobile health apps have features such as; messaging, alarming, and event reminder. These features can be used to overcome problems such as forgetfulness, so it’s one of the most popular method among technology-based strategies for drug use management [ 13 ].

However, many apps that are designed for the general population are not customized for using by the elderly, which comprise a large population with various health needs so little interaction is observed between them and these apps [ 14 ]. Most of older adults suffer from impaired vision and hearing and have tough problems for using smartphones, which are usually ignored in designing these apps [ 15 ]. Moreover, the apps designed for older adults are usually in languages other than Farsi, rendering them practically useless for Iranian older adults. Persian drug reminder apps only remind medication use time and have failed to meet elderly needs. Ease of use, interesting user interface, font adjustment capability, and use of appropriate warm colors to compensate any vision impairment are among the factors that should be considered in designing a suitable app for this age group. In other words, a drug management app should be designed in such a way that even those who are only able to read and write, can use it [ 16 ].

In this regard, the Medisafe, a reminder alarms app, was designed to monitor the older adults’ blood pressure and boost drug adherence level. The results showed that medication adherence improved and blood pressure preserved in the normal range in patients in intervention group. However, despite improving medication adherence among the elderly, the mobile application did not provide features such as information about the drug use instructors [ 17 ]. As another case, the AlerHTA app, an alarm reminder, aiming to increase drug adherence, drug literacy promotion and aiming reminding time of drug use for patients with hypertetion. The results showed higher medication adherence in the elderly patients in the intervention group. Considering medication adherence promotion in studied elderly, this application was not specifically for older adults and only served as a medication reminder [ 14 ]. Additionally, in some cases, the designed apps failed to achieve their objectives completely. For example, in one study, no significant difference was observed in blood pressure control between elderly patients in case and control groups after using an educational app [ 18 ].

Since the available apps do not meet the older adults’ needs for medication use management due to their special conditions, they need an app that has the highest congruence with their physical and mental conditions. A review of the literature suggests that the older adults needed items are not considered in the available apps. Therefore, it was decided to design an app that has the highest congruence with their needs. Furthermore, there are controversies data regarding the effectiveness of drug management apps in promoting drug adherence and its associated consequences. Insights into the potential benefits of an expert-designed mobile application in promoting drug adherence among older adults; this controlled trial study was conducted to evaluate the effect of using a mobile drug management application on medication adherence and hospital readmission among polypharmacy older adults.

Study design, sampling and data collection

A randomized controlled trial study was conducted in April-June 2022. The research population consisted of older adults presenting to a hospital in Tehran, Iran. This hospital is a prominent health center for the geriatric population and has geriatric-specific clinics and services specializing in both acute and chronic diseases. This hospital welcomes clients for routine and periodic health checkups based on their health needs and physician’s order in all days of week.

The inclusion criteria were age above 60 years, daily intake of more than 5 types of drugs, ability to read and write based on being able to fill out necessary forms in clinic, ability to communicate with, having smartphone ownership based on their self-report and a positive history of cardiovascular disease, diabetes, hypertension, or COPD. The exclusion criteria were a history of cognitive diseases, use of expensive and hard-to-find drugs, a positive history of special hard-to-treat diseases like cancer, use of injection drugs (e.g., insulin-dependent diabetic patients were excluded), not receiving clinic follow-up, unwillingness to participate in the study, and death. To calculate the sample size, for an intervention that could reduce poor adherence by 20% with a confidence interval of 95% and power of 80%, 86 participants were needed in each group [ 10 ]. Considering a loss to follow-up of 10%, 96 participants were needed for each group (Fig.  1 ) [ 19 ].

Sample size calculation formula

Recruitment process occurred among inpatient and outpatient clients referring to hospital. The researcher was present in hospital reception to assess the eligible elderly based on inclusion and exclusion criteria checklist. Among these, two hundred and fourteen older adults who were compatible with inclusion and exclusion criteria were selected with simple random method. Finally, 192 older adults volunteer to participate in this study were assigned to case and control groups using block randomization method with a block size of four.

The allocation sequence was generated using www.randomization.com [ 20 ]. An opaque envelope was used for allocation concealment. At the time of enrollment, according to the order by which the participants entered the study, one of the envelopes was opened in order and the allocation group was determined. The primary outcome was medication adherence and the secondary outcome was the adverse events experienced during the study. Medication adherence was measured at baseline and eight weeks after the intervention. The data collection tools were a demographic questionnaire, a researcher-made adverse events questionnaire, and the Morisky Medication Adherence Scale. The demographic questionnaire was used to collect data on age, sex, marital status, education level, income sufficiency, type of disease, list and number of chronic medications used by patients for confirming polypharmacy, method of information acquisition about the consumed drug, and the most used feature in mobile phone. The Morisky Medication Adherence Scale was developed to evaluate medication adherence by Morisky et al. in 2008 [ 21 ]. This scale was translated to Persian according to the Iranian culture and validated by Kooshyar et al. [ 22 ]. This scale contains eight questions. Response categories are yes/no for the first seven items and a 5-point Likert response from never to always for the last item. A score < 6 indicates low adherence, a score of 6 to < 8 shows moderate adherence, and a score of 8 represents high adherence [ 21 ]. In addition, pill count method was also used to assess medication adherence. Pill count method was taken for each intervention group participant. The control group only received medication for their prescriptions. In first visit and after having the application set for older adults in intervention group, the investigator confirmed patient enough pills supply until next visit. In second clinic visit, information was obtained on the number of pills returned and dispensed. The difference between the number of pills received at the previous visit and the number of pills returned represented the number of pills assumed used by pill count. This number was compared with the number of days that had elapsed between the previous visit and the current visit. Ratio ranges between 0 and 1 where the maximum value is 1. Medication correct use was defined as having value ≥ 0/85 − 1 [ 23 , 24 ]. A researcher-made adverse events questionnaire was used to measure the occurrence five complications; re-hospitalization due to disease recurrence, re-hospitalization due to error in medication consumption; determined by doctor, falling from the bed at home by patient report, hypo or hypertension and hypo or hyperglycemia measured by the patient at home or health care providers in hospitals in periodic visits. Accuracy and reliably of patient response to this case report form, was examined by their self-declaration and medical records.

This form was developed for this study and in order to assess the face and content validity of this case report form, a panel of six geriatric experts was asked to rate the relevancy, clarity, simplicity, and necessity of each question using Likert scale. All members of the panel had relevant knowledge in either usability evaluations or elderly needs. Subsequently, the Content Validity Ratio (CVR) and Content Validity Index (CVI) were calculated according to previous studies. Content validity was considered to be acceptable when CVI and CVR were at least 0.78. In order to confirm the reliability of the questionnaire, two methods of internal reliability and test-retest reliability were used. In this sense, Cronbach’s alpha was calculated as the measure of internal reliability. Cronbach’s alpha equal to and above 0.7 was considered as the minimum acceptable value. To measure the test-retest, a total of 20 older adults was asked to score the questionnaire twice with a two-week interval. Then, the Pearson correlation coefficient was calculated between the two sets of scores.

Intervention

In order to design the application used in this study, an in-depth review was conducted on the existing medication reminder application (both in Farsi and English) to identify their week points in order to solve these issues in new design. A drug management app compatible with android operating system with features such as ease of use, adjustable font and text size, use of proper colors in background and app item, saying the name of the drug and showing its picture while playing a reminder for its use, and using phrases like “Dear mother/father! It’s time for your drug” was designed for Iranian older adults in Persian language for the first time. The drugs name and picture was recorded when setting the reminding alarm. All educational content in this app were reviewed and confirmed by three geriatric experts. In order to protect the privacy and security of app users, the most up-to-date programing codes were used to design the app. As a pilot test and finding the possible flaws and ensuring its correct function on mobile phones, it was installed on the mobile phones of 10 elders. They were asked to report any problem and their ideas about app improvement and its ease of use. The comments of geriatric experts in this field and older adults were both used in design phase. After receiving the final correct operability confirmation and obtaining informed consent, the app was installed on the mobile phones of the participants in the intervention group. Face-to-face training on how to use the app was offered to each participant in a 60-minute session and medication use alarms were set. During the intervention, app users were contacted via phone calls or in routine hospital referrals to ask and check if there is any problem using the app to ensure to ensure intervention fidelity. Also the participants could contact the researcher through the phone number given to them to ask their questions. Number and therapeutic category of drugs were similar in the control and intervention groups. The participants in the control group received the routine care of the health center including periodical visits to evaluate the treatment process and required care. Considering the Covid-19 pandemic in Iran and the sensitive conditions of older adults, it was difficult to make the necessary arrangements for participation of the elders in the introduction session, which included various topic such as the objective of the study, using the app, and completing the questionnaire. To address this problem, the participants were grouped into different groups, and Covid-19 related protocols including social distancing, face mask use, and disinfection were strictly considered.

Data analysis

Data were entered into the SPSS software version 22 and analyzed according to per protocol using descriptive (frequency for number of chronic medications, mean for age, percentage and standard deviation for related categories) and inferential (Chi-square for sex, education level, income sufficiency, type of disease, adherence level before and after intervention, Fisher’s exact test for marital status method of information acquisition about the consumed drug and the most used feature in mobile phone and finally independent t-test for age) statistics. In inferential analysis, demographic data of control and intervention group and adherence level before and after intervention in both group were compared with each other at baseline and 8 weeks. The effect size was calculated using the Cramer’s statistic. P values less than 0.05 were considered significant.

The flow of participants in the study (CONSORT Flow Diagram)

In the present study, 214 patients were evaluated according to the inclusion criteria, of whom 13 were excluded due to not meeting the inclusion criteria and 9 due to unwillingness to participate in the study. Then, the remaining 192 patients were randomly assigned to intervention and control groups. During the intervention, 4 patients were excluded from the control group (3 due to death and 1 due to unwillingness to continue the study because of travelling); moreover, 4 patients were also excluded from the intervention group (2 due to death and 2 due to unwillingness to continue the study; one for not clear reason, the other one for having his phone broken). Finally, data of 92 cases and 92 controls were analyzed (Fig.  2 ).

Demographic findings showed that the participants mean age was 69 ± 5.6 and 68.9 ± 5.2 years in the control and intervention group respectively, indicating no significant difference (p = 0.926). Mean total number of medications taken by control group was 6.53 ± 2.3 and 6.67 ± 2.4 at baseline and in follow-up respectively which indicate no significant difference P = 0.839. Also mean total number of medications taken by intervention group was 6.45 ± 2.1 and 6.55 ± 2.3 at baseline and in follow-up respectively which indicate no significant difference P = 0.874. Moreover, the two groups were homogenous in terms of sex, marital status, education level, income sufficiency, cardiovascular disease, hypertension, diabetes, COPD, method of information acquisition about the consumed drug, and the most used feature in mobile phone (p > 0.05) (Tables  1 and 2 ).

Evaluation of drug adherence level showed no significant differences between two groups before the intervention (p = 0.919). However, after the intervention, the difference was significant in the intervention group, and the participants with high adherence increased from 12.5 to 44.56%, indicating a significant difference (p < 0.001). According to the Cramer’s statistic, the effect size of the intervention on drug adherence was above moderate [ 25 ] (Table  3 ).

Comparison of the adverse events between the two groups showed a significant difference in re-hospitalization due to disease aggravation and error in medication consumption, falling, hypo or hypertension, hypo or hyperglycemia, and drug use accuracy according to the prescriber’s order based on the pill count method between the two groups (p < 0.05). Drug use accuracy ranges between 0 and 1 where the maximum value is 1. Medication correct use was defined as having value ≥ 0/85 − 1 (Table  4 ).

The present study was conducted to evaluate the effect of using a drug management app on drug adherence and adverse events in polypharmacy adults. The results showed that using an app customized for the special conditions of the older adults improves medication adherence. In line with results of present study, Najafi et al. conducted a study to investigate the effect of using a mobile phone-based application on medication adherence in patients with heart failure during three months. The results showed a significant increase in the medication adherence score after using the app. Medication adherence changes were more significant in the intervention group compared to the control group, indicating improved medication adherence in people with heart failure after using the app [ 26 ]. The results of the above study were consistent with the results of the present study, suggesting the positive effect of drug management apps on improved medication adherence. Santo K conducted a study to determine the effect of medication reminder applications on drug adherence in patients with coronary heart disease in three months. The primary outcome was the medication adherence level and the secondary outcomes were the blood pressure and cholesterol level control. Mean score of medication adherence was significantly higher in app users compared to control group, which is similar to the present study [ 27 ]. In another study, Li et al. evaluated the effect of a smartphone application on medication adherence in 24 polypharmacy patients with a mean age of 59.5 years for one year in Australia. In this study, app users in intervention group received medication regimen and educational messages through the app. Participants were required to report each time they took a medication via the app. The control group received routine care including routine visits. Both groups were evaluated three times, including one, three, and twelve months after the intervention. The results showed a 4.37 times higher improvement in the medication adherence in the intervention group compared to the control group in the third assessment [ 28 ]. Although Li et al. did not exclusively focus on older adults and the participants age ranges 18–75 years, its results were consistent with the results of the present study, indicating improved medication adherence in drug management app users.

Baghei et al. studied the effect of a mobile educational application on medication adherence in hypertensive older adults. Comparison mean scores of medication adherence components showed significant difference in commitment to treatment and hesitation in implementation of treatment between two groups while no significant difference was observed in the blood pressure status between the intervention and control groups. However, in present study, blood pressure alterations reduced significantly in the intervention group compared to control group; therefore, the results are not consistent. This difference may be due to the follow-up duration. However, it can be concluded that mobile app education can improve medication adherence in older adult app users [ 18 ].

Habib et al. conducted a study to improve medication adherence following hospital discharge using a mobile application in Canada. This study was conducted on 49 patients with a mean age of 64.6 years assigned to two groups. The patients were followed for 30 days’ post-discharge. During the follow-up period, they were evaluated for medication adherence and re-hospitalization. At the time of discharge, the app was installed on the mobile phones of the patients in the intervention group. Drug management app named SAM, an alarm reminder app, with mission to improve medication adherence in older adults. According to the results, the medication adherence rate was 83.7% in the intervention group and 77.8% in the control group, indicating no significant difference [ 29 ]. Although it seems that the results of the above study and our study are not consistent, the reason may be the short follow-up time of this study. Moreover, in this study, only 65.2% of the patients in the intervention group used the app, which could be due to reasons such as failure to design an appropriate app for this group and lack of proper training for its use or not being user friendly.

The results of the present study showed that using the drug management app reduced adverse events including re-hospitalization due to disease aggravation and error in medication consumption, falling, hypo or hypertension, hypo or hyperglycemia, and promote medication use accuracy according to the pill count method. In this regard, Park et al. conducted a study to determine the effect of using digital health monitoring on readmission reduction in patients with health failure in the United States. This study was carried out on 58 patients and readmission rate was measured during 30 days. The results showed that overall 30-day readmission rate was 10% in these patients, while the mean readmission rate was 25% across the country indicating a significant reduction [ 30 ], which was consistent with results of present study. Findings suggest use of new technologies in health care system to have a significant reduction in the re-hospitalization rate. Furthermore, Sartori et al. conducted a study to evaluate the effect of educational intervention using the WhatsApp platform on medication adherence in hypertensive and diabetic patients. Intervention group participants received training in form of audio, image, or video messages with focus on increasing medication adherence via the WhatsApp while the control group received the routine care. Data analysis 16 weeks after initiating the intervention showed no significant difference between the two groups [ 31 ]. The results of this study are not consistent with the results of present study and the use of WhatsApp could not lead to a significant difference between two groups. Differences in structures of these apps, including ease of use and observance of delicate considerations required for older adults regarding using the app, which were carefully implemented in designing the app in the present study might be the reason of this discrepancy. Chandler et al. investigated the impact of mobile phone use on treatment adherence in hypertensive patients in the United States. Three months after using the app, treatment adherence improved markedly in the intervention group while no change was observed in control group. Findings showed a significant reduction in systolic and diastolic blood pressure in intervention group compared to control group [ 32 ]. The results of this study are consistent with the results of the present study; as mentioned earlier, the designed app reduced blood pressure alterations in the intervention group in the present study.

Finally, Athilingam et al. conducted a study to evaluate the effect of enhanced self-care using mobile technology on reducing readmission in patients with congestive heart failure. Results showed that patients in intervention group were not readmitted during the 30 days of study compared to control group. Preliminary results showed the potential effectiveness of app in reducing readmission and improving self-care in heart failure patients [ 33 ], which was similar to the results of the present study. Nonetheless, the sample size was much larger in the present study compared to above study, which improves the generalizability of the results.

Considering that one of the basic goals of the health care system is to prevent the increase of costs and reduce the economic burden that is created by each individual and group of society, investigations such as the present study will provide valuable information about what policies should be implemented to achieve the stated objective. According to the findings of this study, by improving medication adherence, many adverse events such as re-hospitalization, as the major one, will be reduced, so it could a precious data for policy makers to promote older adults’ quality of life and society welfare by familiarize the elderly with new technologies as much as possible in order to take advantage of it and employ them to manage their health condition effectively. Use of health mobile application designed according to the needs and capabilities of the target group improve the patients’ control over their disease and helps them prevent adverse events.

Limitations

There were some limitations in this study. Considering Morisky scale as a subjective tool to assess medication adherence, we used pill count method to avoid internal validity threat. This method was used as an objective and supplementary method, considering its practicality and simplicity. Lack of generalizability due to a single site of study was the other limitation. Also lifestyle-related factors with medicine adherence were seldom examined in this study. Larger studies covering wider areas and focusing more on the lifestyle or other risk factors should be carried out in future. Use of a case report form for assessing adverse events in this study was another limitation. Totally self-reported answers may be exaggerated; respondents may be too embarrassed to reveal private details or even forget needed data. Due to neutralize this effect, the investigator compared the answer with their medical records and physician confirmation. The possibility of cross-group contamination also could have affected our study. Although the investigators planned to have even introduction session with each group in different days to avoid participants’ connection.

Data availability

All data generated in this study are included in the manuscript. Datasets are available upon reasonable request from the corresponding author. Mrs. Pashaeypoor is available for data and materials availability. The available e-mail address is [email protected].

Aghamohamadi S, Hajinabi K, Jahangiri K, Asl IM, Dehnavieh R. Population and mortality profile in the Islamic Republic of Iran, 2006–2035. East Mediterr Health J. 2018;24(5):469–76.

Article   PubMed   Google Scholar  

Curtin D, O’mahony D, Gallagher P. Drug consumption and futile medication prescribing in the last year of life: an observational study. Age Ageing. 2018;47(5):749–53.

Article   CAS   PubMed   Google Scholar  

Poorcheraghi H, Valieiny N, Pashaeypoor S, Mirzadeh FS. Polypharmacy Management Strategies in older adults; a scoping review. Iran J Nurs Res. 2022;17(5):71–87.

Google Scholar  

Dhalwani NN, Fahami R, Sathanapally H, Seidu S, Davies MJ, Khunti K. Association between polypharmacy and falls in older adults: a longitudinal study from England. BMJ open. 2017;7(10):e016358.

Article   PubMed   PubMed Central   Google Scholar  

Hosseini SR, Zabihi A, Amiri SRJ, Bijani A. Polypharmacy among the Elderly. J mid-life Health. 2018;9(2):97.

Article   Google Scholar  

Eyigor S, Kutsal YG, Toraman F, Durmus B, Gokkaya KO, Aydeniz A, et al. Polypharmacy, physical and nutritional status, and depression in the elderly: do polypharmacy deserve some credits in these problems? Exp Aging Res. 2021;47(1):79–91.

Shapiro JS, McCoy RG, Takahashi PY, Thorsteinsdottir B, Peterson SM, Naessens JM, et al. Medication use leading to hospital readmission in frail elders. J Nurse Practitioners. 2017;13(10):708–15.

Banna GL, Urzia V, Benanti C, Pitrè A, Lipari H, Di Quattro R, et al. Adherence to abiraterone or enzalutamide in elderly metastatic castration-resistant Prostate cancer. Support Care Cancer. 2020;28(10):4687–95.

Irshaidat S, Gustafsson M, Norberg H. Self-reported medication adherence among older people admitted to hospital: a descriptive study. Drugs-real World Outcomes. 2023;10(1):23–9.

Mehdinia A, Loripoor M, Dehghan M, Heidari S. The Effect of Pillbox Use on Medication Adherence among Elderly patients: a Randomized Controlled Trial. Disease and Diagnosis. 2020;9(1):38–43.

Cheong V-L, Sowter J, Scally A, Hamilton N, Ali A, Silcock J. Medication-related risk factors and its association with repeated hospital admissions in frail elderly: a case control study. Res Social Administrative Pharm. 2020;16(9):1318–22.

Iancu I, Iancu B. Designing mobile technology for elderly. A theoretical overview. Technol Forecast Soc Chang. 2020;155:119977.

Pratiwi H, Kristina SA, Widayanti AW, Prabandari YS, Kusuma IY. A systematic review of compensation and technology-mediated strategies to maintain older adults’ Medication Adherence. Int J Environ Res Public Health. 2023;20(1):803.

Marquez Contreras E, Marquez Rivero S, Rodriguez Garcia E, López-García-Ramos L, Carlos Pastoriza Vilas J, Baldonedo Suarez A, et al. Specific Hypertension smartphone application to improve medication adherence in Hypertension: a cluster-randomized trial. Curr Med Res Opin. 2019;35(1):167–73.

Peng Y, Wang H, Fang Q, Xie L, Shu L, Sun W, et al. Effectiveness of mobile applications on medication adherence in adults with chronic Diseases: a systematic review and meta-analysis. J Managed care Specialty Pharm. 2020;26(4):550–61.

Conway CM, Kelechi TJ, Nemeth LS. Engaging older adults to inform Diabetes medication adherence mobile application selection. Healthy Aging Research. 2018;7(2):e20.

Morawski K, Ghazinouri R, Krumme A, Lauffenburger JC, Lu Z, Durfee E, et al. Association of a smartphone application with medication adherence and blood pressure control: the MedISAFE-BP randomized clinical trial. JAMA Intern Med. 2018;178(6):802–9.

Baghei D, Poortaghi S, Sahebi L, Amrollah MajdAbadi Z, Nikpeyma N. Effect of Mobile Educational Application on Adherence to Treatment in Elderly with Hypertension. J Mazandaran Univ Med Sci. 2022;32(209):114–24.

Abbasi Nazari M, Hosseinpanah F, Moradi O, Soleimani S. Evaluation of the pattern of adherence to Levothyroxine Use and Effect of Education via Pamphlet in Hypothyroidism patients. J Med Council Islamic Repub Iran. 2018;36(3):168–72.

Available from: www.randomization.com.

Morisky DE, Ang A, Krousel-Wood M, Ward HJ. Predictive validity of a medication adherence measure in an outpatient setting. J Clin Hypertens. 2008;10(5):348–54.

Kooshyar H, Shoorvazi M, Dalir Z, Hosseini M. Health Literacy and its relationship with Medical Adherence and Health-Related Quality of Life in Diabetic Community-Residing Elderly. J Mazandaran Univ Med Sci. 2013;22(1):134–43.

Hartman L, Cutolo M, Bos R, Opris-Belinski D, Kok MR, Griep-Wentink HR, et al. Medication adherence in older people with rheumatoid arthritis is lower according to electronic monitoring than according to pill count. Rheumatology. 2021;60(11):5239–46.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Sandarenu K, Dilhara K, Rupasinghe P, Nuwandika T, Abhayasinghe M, Fernando N. Influence of patient counseling (by the pharmacist) on adherence to medication in people with Mental illnesses. Int J Innovative Sci Res Technol. 2021;6(4).

Chinn S. A simple method for converting an odds ratio to effect size for use in meta-analysis. Stat Med. 2000;19(22):3127–31.

Vakilian F, THE IMPACT OF USING A SMARTPHONE-BASED APPLICATION, ON MEDICATION ADHERENCE IN PEOPLE WITH HEART FAILURE. Nurs Midwifery J. 2019;17(6):494–504.

Santo K, Singleton A, Rogers K, Thiagalingam A, Chalmers J, Chow CK, et al. Medication reminder applications to improve adherence in coronary Heart Disease: a randomised clinical trial. Heart. 2019;105(4):323–9.

Li A, Del Olmo MG, Fong M, Sim K, Lymer SJ, Cunich M, et al. Effect of a smartphone application (perx) on medication adherence and clinical outcomes: a 12-month randomised controlled trial. BMJ open. 2021;11(8):e047041.

Habib B, Buckeridge D, Bustillo M, Marquez SN, Thakur M, Tran T, et al. Smart about meds (SAM): a pilot randomized controlled trial of a mobile application to improve medication adherence following hospital discharge. JAMIA open. 2021;4(3):ooab050.

Park C, Otobo E, Ullman J, Rogers J, Fasihuddin F, Garg S, et al. Impact on readmission reduction among Heart Failure patients using digital health monitoring: feasibility and adoptability study. JMIR Med Inf. 2019;7(4):e13353.

Sartori AC, Rodrigues Lucena TF, Lopes CT, Picinin Bernuci M, Yamaguchi MU. Educational intervention using WhatsApp on medication adherence in Hypertension and Diabetes patients: a randomized clinical trial. Telemedicine and e-Health. 2020;26(12):1526–32.

Chandler J, Sox L, Diaz V, Kellam K, Neely A, Nemeth L, et al. Impact of 12-month smartphone breathing meditation program upon systolic blood pressure among non-medicated stage 1 hypertensive adults. Int J Environ Res Public Health. 2020;17(6):1955.

Athilingam P, Labrador M, Redding B. Enhanced Self Care with Mobile Technology to Reduce Readmission in Congestive Heart Failure: a pilot study. Cardiol Vasc Res. 2021;5(3):1–7.

Download references

Acknowledgements

The current research is the result of a master’s thesis in geriatric nursing. We appreciate professor Morisky for giving us permission to use the MMAS-8-Item instrument. The researchers consider it necessary to express their gratitude to the Research Vice-Chancellor of the Faculty of Nursing and Midwifery of Tehran University of Medical Sciences and also the authors wish to thank all of the participants in this study.

This research has been approved and supported by Tehran University of Medical Sciences and Health Services.

Author information

Authors and affiliations.

Dept. of Community Health and Geriatric Nursing, School of Nursing and Midwifery, Tehran University of Medical Sciences, Tehran, Iran

Hossein Poorcheraghi & Shahzad Pashaeypoor

Nursing and Midwifery Care Research Center, School of Nursing and Midwifery, Tehran University of Medical Sciences, Tehran, Iran

Reza Negarandeh

Dept. of AI, Faculty of Engineering, Mashhad Branch, Islamic Azad University, Mashhad, Iran

Javad Jorian

You can also search for this author in PubMed   Google Scholar

Contributions

HP: design of the study, implementation of study, drafting the manuscript; ShP: analysis and interpretation of data, drafting the manuscript; RN: design of the study, analysis and interpretation of the data, drafting the manuscript. JJ: designing the mobile application. All authors have read and approved the manuscript. HP and ShP are the guarantors of the manuscript.

Corresponding author

Correspondence to Shahzad Pashaeypoor .

Ethics declarations

Ethical approval and consent to participate.

The study was approved by the Ethics Committee of Tehran University of Medical Sciences (IR.TUMS.FNM.REC.1400.068). It was also registered in the Iranian Registry of Clinical Trials (code: IRCT20191231045966N1) (18/07/2021). The participants were assured of data confidentiality and informed written consent was obtained from them. Moreover, based on research ethics principles, the app was also installed on the mobile phones of the participants in the control group at the end of the study. It also should be notified all methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Cite this article.

Poorcheraghi, H., Negarandeh, R., Pashaeypoor, S. et al. Effect of using a mobile drug management application on medication adherence and hospital readmission among elderly patients with polypharmacy: a randomized controlled trial. BMC Health Serv Res 23 , 1192 (2023). https://doi.org/10.1186/s12913-023-10177-4

Download citation

Received : 06 June 2023

Accepted : 18 October 2023

Published : 02 November 2023

DOI : https://doi.org/10.1186/s12913-023-10177-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Polypharmacy
• Drug adherence
• Application
• Mobile phone

BMC Health Services Research

ISSN: 1472-6963

• Submission enquiries: [email protected]
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Advanced Search
• Journal List
• Int J Qual Stud Health Well-being

Methodology or method? A critical review of qualitative case study reports

Despite on-going debate about credibility, and reported limitations in comparison to other approaches, case study is an increasingly popular approach among qualitative researchers. We critically analysed the methodological descriptions of published case studies. Three high-impact qualitative methods journals were searched to locate case studies published in the past 5 years; 34 were selected for analysis. Articles were categorized as health and health services ( n= 12), social sciences and anthropology ( n= 7), or methods ( n= 15) case studies. The articles were reviewed using an adapted version of established criteria to determine whether adequate methodological justification was present, and if study aims, methods, and reported findings were consistent with a qualitative case study approach. Findings were grouped into five themes outlining key methodological issues: case study methodology or method, case of something particular and case selection, contextually bound case study, researcher and case interactions and triangulation, and study design inconsistent with methodology reported. Improved reporting of case studies by qualitative researchers will advance the methodology for the benefit of researchers and practitioners.

Case study research is an increasingly popular approach among qualitative researchers (Thomas, 2011 ). Several prominent authors have contributed to methodological developments, which has increased the popularity of case study approaches across disciplines (Creswell, 2013b ; Denzin & Lincoln, 2011b ; Merriam, 2009 ; Ragin & Becker, 1992 ; Stake, 1995 ; Yin, 2009 ). Current qualitative case study approaches are shaped by paradigm, study design, and selection of methods, and, as a result, case studies in the published literature vary. Differences between published case studies can make it difficult for researchers to define and understand case study as a methodology.

Experienced qualitative researchers have identified case study research as a stand-alone qualitative approach (Denzin & Lincoln, 2011b ). Case study research has a level of flexibility that is not readily offered by other qualitative approaches such as grounded theory or phenomenology. Case studies are designed to suit the case and research question and published case studies demonstrate wide diversity in study design. There are two popular case study approaches in qualitative research. The first, proposed by Stake ( 1995 ) and Merriam ( 2009 ), is situated in a social constructivist paradigm, whereas the second, by Yin ( 2012 ), Flyvbjerg ( 2011 ), and Eisenhardt ( 1989 ), approaches case study from a post-positivist viewpoint. Scholarship from both schools of inquiry has contributed to the popularity of case study and development of theoretical frameworks and principles that characterize the methodology.

The diversity of case studies reported in the published literature, and on-going debates about credibility and the use of case study in qualitative research practice, suggests that differences in perspectives on case study methodology may prevent researchers from developing a mutual understanding of practice and rigour. In addition, discussion about case study limitations has led some authors to query whether case study is indeed a methodology (Luck, Jackson, & Usher, 2006 ; Meyer, 2001 ; Thomas, 2010 ; Tight, 2010 ). Methodological discussion of qualitative case study research is timely, and a review is required to analyse and understand how this methodology is applied in the qualitative research literature. The aims of this study were to review methodological descriptions of published qualitative case studies, to review how the case study methodological approach was applied, and to identify issues that need to be addressed by researchers, editors, and reviewers. An outline of the current definitions of case study and an overview of the issues proposed in the qualitative methodological literature are provided to set the scene for the review.

Definitions of qualitative case study research

Case study research is an investigation and analysis of a single or collective case, intended to capture the complexity of the object of study (Stake, 1995 ). Qualitative case study research, as described by Stake ( 1995 ), draws together “naturalistic, holistic, ethnographic, phenomenological, and biographic research methods” in a bricoleur design, or in his words, “a palette of methods” (Stake, 1995 , pp. xi–xii). Case study methodology maintains deep connections to core values and intentions and is “particularistic, descriptive and heuristic” (Merriam, 2009 , p. 46).

As a study design, case study is defined by interest in individual cases rather than the methods of inquiry used. The selection of methods is informed by researcher and case intuition and makes use of naturally occurring sources of knowledge, such as people or observations of interactions that occur in the physical space (Stake, 1998 ). Thomas ( 2011 ) suggested that “analytical eclecticism” is a defining factor (p. 512). Multiple data collection and analysis methods are adopted to further develop and understand the case, shaped by context and emergent data (Stake, 1995 ). This qualitative approach “explores a real-life, contemporary bounded system (a case ) or multiple bounded systems (cases) over time, through detailed, in-depth data collection involving multiple sources of information … and reports a case description and case themes ” (Creswell, 2013b , p. 97). Case study research has been defined by the unit of analysis, the process of study, and the outcome or end product, all essentially the case (Merriam, 2009 ).

The case is an object to be studied for an identified reason that is peculiar or particular. Classification of the case and case selection procedures informs development of the study design and clarifies the research question. Stake ( 1995 ) proposed three types of cases and study design frameworks. These include the intrinsic case, the instrumental case, and the collective instrumental case. The intrinsic case is used to understand the particulars of a single case, rather than what it represents. An instrumental case study provides insight on an issue or is used to refine theory. The case is selected to advance understanding of the object of interest. A collective refers to an instrumental case which is studied as multiple, nested cases, observed in unison, parallel, or sequential order. More than one case can be simultaneously studied; however, each case study is a concentrated, single inquiry, studied holistically in its own entirety (Stake, 1995 , 1998 ).

Researchers who use case study are urged to seek out what is common and what is particular about the case. This involves careful and in-depth consideration of the nature of the case, historical background, physical setting, and other institutional and political contextual factors (Stake, 1998 ). An interpretive or social constructivist approach to qualitative case study research supports a transactional method of inquiry, where the researcher has a personal interaction with the case. The case is developed in a relationship between the researcher and informants, and presented to engage the reader, inviting them to join in this interaction and in case discovery (Stake, 1995 ). A postpositivist approach to case study involves developing a clear case study protocol with careful consideration of validity and potential bias, which might involve an exploratory or pilot phase, and ensures that all elements of the case are measured and adequately described (Yin, 2009 , 2012 ).

Current methodological issues in qualitative case study research

The future of qualitative research will be influenced and constructed by the way research is conducted, and by what is reviewed and published in academic journals (Morse, 2011 ). If case study research is to further develop as a principal qualitative methodological approach, and make a valued contribution to the field of qualitative inquiry, issues related to methodological credibility must be considered. Researchers are required to demonstrate rigour through adequate descriptions of methodological foundations. Case studies published without sufficient detail for the reader to understand the study design, and without rationale for key methodological decisions, may lead to research being interpreted as lacking in quality or credibility (Hallberg, 2013 ; Morse, 2011 ).

There is a level of artistic license that is embraced by qualitative researchers and distinguishes practice, which nurtures creativity, innovation, and reflexivity (Denzin & Lincoln, 2011b ; Morse, 2009 ). Qualitative research is “inherently multimethod” (Denzin & Lincoln, 2011a , p. 5); however, with this creative freedom, it is important for researchers to provide adequate description for methodological justification (Meyer, 2001 ). This includes paradigm and theoretical perspectives that have influenced study design. Without adequate description, study design might not be understood by the reader, and can appear to be dishonest or inaccurate. Reviewers and readers might be confused by the inconsistent or inappropriate terms used to describe case study research approach and methods, and be distracted from important study findings (Sandelowski, 2000 ). This issue extends beyond case study research, and others have noted inconsistencies in reporting of methodology and method by qualitative researchers. Sandelowski ( 2000 , 2010 ) argued for accurate identification of qualitative description as a research approach. She recommended that the selected methodology should be harmonious with the study design, and be reflected in methods and analysis techniques. Similarly, Webb and Kevern ( 2000 ) uncovered inconsistencies in qualitative nursing research with focus group methods, recommending that methodological procedures must cite seminal authors and be applied with respect to the selected theoretical framework. Incorrect labelling using case study might stem from the flexibility in case study design and non-directional character relative to other approaches (Rosenberg & Yates, 2007 ). Methodological integrity is required in design of qualitative studies, including case study, to ensure study rigour and to enhance credibility of the field (Morse, 2011 ).

Case study has been unnecessarily devalued by comparisons with statistical methods (Eisenhardt, 1989 ; Flyvbjerg, 2006 , 2011 ; Jensen & Rodgers, 2001 ; Piekkari, Welch, & Paavilainen, 2009 ; Tight, 2010 ; Yin, 1999 ). It is reputed to be the “the weak sibling” in comparison to other, more rigorous, approaches (Yin, 2009 , p. xiii). Case study is not an inherently comparative approach to research. The objective is not statistical research, and the aim is not to produce outcomes that are generalizable to all populations (Thomas, 2011 ). Comparisons between case study and statistical research do little to advance this qualitative approach, and fail to recognize its inherent value, which can be better understood from the interpretive or social constructionist viewpoint of other authors (Merriam, 2009 ; Stake, 1995 ). Building on discussions relating to “fuzzy” (Bassey, 2001 ), or naturalistic generalizations (Stake, 1978 ), or transference of concepts and theories (Ayres, Kavanaugh, & Knafl, 2003 ; Morse et al., 2011 ) would have more relevance.

Case study research has been used as a catch-all design to justify or add weight to fundamental qualitative descriptive studies that do not fit with other traditional frameworks (Merriam, 2009 ). A case study has been a “convenient label for our research—when we ‘can't think of anything ‘better”—in an attempt to give it [qualitative methodology] some added respectability” (Tight, 2010 , p. 337). Qualitative case study research is a pliable approach (Merriam, 2009 ; Meyer, 2001 ; Stake, 1995 ), and has been likened to a “curious methodological limbo” (Gerring, 2004 , p. 341) or “paradigmatic bridge” (Luck et al., 2006 , p. 104), that is on the borderline between postpositivist and constructionist interpretations. This has resulted in inconsistency in application, which indicates that flexibility comes with limitations (Meyer, 2001 ), and the open nature of case study research might be off-putting to novice researchers (Thomas, 2011 ). The development of a well-(in)formed theoretical framework to guide a case study should improve consistency, rigour, and trust in studies published in qualitative research journals (Meyer, 2001 ).

Assessment of rigour

The purpose of this study was to analyse the methodological descriptions of case studies published in qualitative methods journals. To do this we needed to develop a suitable framework, which used existing, established criteria for appraising qualitative case study research rigour (Creswell, 2013b ; Merriam, 2009 ; Stake, 1995 ). A number of qualitative authors have developed concepts and criteria that are used to determine whether a study is rigorous (Denzin & Lincoln, 2011b ; Lincoln, 1995 ; Sandelowski & Barroso, 2002 ). The criteria proposed by Stake ( 1995 ) provide a framework for readers and reviewers to make judgements regarding case study quality, and identify key characteristics essential for good methodological rigour. Although each of the factors listed in Stake's criteria could enhance the quality of a qualitative research report, in Table I we present an adapted criteria used in this study, which integrates more recent work by Merriam ( 2009 ) and Creswell ( 2013b ). Stake's ( 1995 ) original criteria were separated into two categories. The first list of general criteria is “relevant for all qualitative research.” The second list, “high relevance to qualitative case study research,” was the criteria that we decided had higher relevance to case study research. This second list was the main criteria used to assess the methodological descriptions of the case studies reviewed. The complete table has been preserved so that the reader can determine how the original criteria were adapted.

Framework for assessing quality in qualitative case study research.

Adapted from Stake ( 1995 , p. 131).

Study design

The critical review method described by Grant and Booth ( 2009 ) was used, which is appropriate for the assessment of research quality, and is used for literature analysis to inform research and practice. This type of review goes beyond the mapping and description of scoping or rapid reviews, to include “analysis and conceptual innovation” (Grant & Booth, 2009 , p. 93). A critical review is used to develop existing, or produce new, hypotheses or models. This is different to systematic reviews that answer clinical questions. It is used to evaluate existing research and competing ideas, to provide a “launch pad” for conceptual development and “subsequent testing” (Grant & Booth, 2009 , p. 93).

Qualitative methods journals were located by a search of the 2011 ISI Journal Citation Reports in Social Science, via the database Web of Knowledge (see m.webofknowledge.com). No “qualitative research methods” category existed in the citation reports; therefore, a search of all categories was performed using the term “qualitative.” In Table II , we present the qualitative methods journals located, ranked by impact factor. The highest ranked journals were selected for searching. We acknowledge that the impact factor ranking system might not be the best measure of journal quality (Cheek, Garnham, & Quan, 2006 ); however, this was the most appropriate and accessible method available.

International Journal of Qualitative Studies on Health and Well-being.

Search strategy

In March 2013, searches of the journals, Qualitative Health Research , Qualitative Research , and Qualitative Inquiry were completed to retrieve studies with “case study” in the abstract field. The search was limited to the past 5 years (1 January 2008 to 1 March 2013). The objective was to locate published qualitative case studies suitable for assessment using the adapted criterion. Viewpoints, commentaries, and other article types were excluded from review. Title and abstracts of the 45 retrieved articles were read by the first author, who identified 34 empirical case studies for review. All authors reviewed the 34 studies to confirm selection and categorization. In Table III , we present the 34 case studies grouped by journal, and categorized by research topic, including health sciences, social sciences and anthropology, and methods research. There was a discrepancy in categorization of one article on pedagogy and a new teaching method published in Qualitative Inquiry (Jorrín-Abellán, Rubia-Avi, Anguita-Martínez, Gómez-Sánchez, & Martínez-Mones, 2008 ). Consensus was to allocate to the methods category.

Outcomes of search of qualitative methods journals.

In Table III , the number of studies located, and final numbers selected for review have been reported. Qualitative Health Research published the most empirical case studies ( n= 16). In the health category, there were 12 case studies of health conditions, health services, and health policy issues, all published in Qualitative Health Research . Seven case studies were categorized as social sciences and anthropology research, which combined case study with biography and ethnography methodologies. All three journals published case studies on methods research to illustrate a data collection or analysis technique, methodological procedure, or related issue.

The methodological descriptions of 34 case studies were critically reviewed using the adapted criteria. All articles reviewed contained a description of study methods; however, the length, amount of detail, and position of the description in the article varied. Few studies provided an accurate description and rationale for using a qualitative case study approach. In the 34 case studies reviewed, three described a theoretical framework informed by Stake ( 1995 ), two by Yin ( 2009 ), and three provided a mixed framework informed by various authors, which might have included both Yin and Stake. Few studies described their case study design, or included a rationale that explained why they excluded or added further procedures, and whether this was to enhance the study design, or to better suit the research question. In 26 of the studies no reference was provided to principal case study authors. From reviewing the description of methods, few authors provided a description or justification of case study methodology that demonstrated how their study was informed by the methodological literature that exists on this approach.

The methodological descriptions of each study were reviewed using the adapted criteria, and the following issues were identified: case study methodology or method; case of something particular and case selection; contextually bound case study; researcher and case interactions and triangulation; and, study design inconsistent with methodology. An outline of how the issues were developed from the critical review is provided, followed by a discussion of how these relate to the current methodological literature.

Case study methodology or method

A third of the case studies reviewed appeared to use a case report method, not case study methodology as described by principal authors (Creswell, 2013b ; Merriam, 2009 ; Stake, 1995 ; Yin, 2009 ). Case studies were identified as a case report because of missing methodological detail and by review of the study aims and purpose. These reports presented data for small samples of no more than three people, places or phenomenon. Four studies, or “case reports” were single cases selected retrospectively from larger studies (Bronken, Kirkevold, Martinsen, & Kvigne, 2012 ; Coltart & Henwood, 2012 ; Hooghe, Neimeyer, & Rober, 2012 ; Roscigno et al., 2012 ). Case reports were not a case of something, instead were a case demonstration or an example presented in a report. These reports presented outcomes, and reported on how the case could be generalized. Descriptions focussed on the phenomena, rather than the case itself, and did not appear to study the case in its entirety.

Case reports had minimal in-text references to case study methodology, and were informed by other qualitative traditions or secondary sources (Adamson & Holloway, 2012 ; Buzzanell & D'Enbeau, 2009 ; Nagar-Ron & Motzafi-Haller, 2011 ). This does not suggest that case study methodology cannot be multimethod, however, methodology should be consistent in design, be clearly described (Meyer, 2001 ; Stake, 1995 ), and maintain focus on the case (Creswell, 2013b ).

To demonstrate how case reports were identified, three examples are provided. The first, Yeh ( 2013 ) described their study as, “the examination of the emergence of vegetarianism in Victorian England serves as a case study to reveal the relationships between boundaries and entities” (p. 306). The findings were a historical case report, which resulted from an ethnographic study of vegetarianism. Cunsolo Willox, Harper, Edge, ‘My Word’: Storytelling and Digital Media Lab, and Rigolet Inuit Community Government (2013) used “a case study that illustrates the usage of digital storytelling within an Inuit community” (p. 130). This case study reported how digital storytelling can be used with indigenous communities as a participatory method to illuminate the benefits of this method for other studies. This “case study was conducted in the Inuit community” but did not include the Inuit community in case analysis (Cunsolo Willox et al., 2013 , p. 130). Bronken et al. ( 2012 ) provided a single case report to demonstrate issues observed in a larger clinical study of aphasia and stroke, without adequate case description or analysis.

Case study of something particular and case selection

Case selection is a precursor to case analysis, which needs to be presented as a convincing argument (Merriam, 2009 ). Descriptions of the case were often not adequate to ascertain why the case was selected, or whether it was a particular exemplar or outlier (Thomas, 2011 ). In a number of case studies in the health and social science categories, it was not explicit whether the case was of something particular, or peculiar to their discipline or field (Adamson & Holloway, 2012 ; Bronken et al., 2012 ; Colón-Emeric et al., 2010 ; Jackson, Botelho, Welch, Joseph, & Tennstedt, 2012 ; Mawn et al., 2010 ; Snyder-Young, 2011 ). There were exceptions in the methods category ( Table III ), where cases were selected by researchers to report on a new or innovative method. The cases emerged through heuristic study, and were reported to be particular, relative to the existing methods literature (Ajodhia-Andrews & Berman, 2009 ; Buckley & Waring, 2013 ; Cunsolo Willox et al., 2013 ; De Haene, Grietens, & Verschueren, 2010 ; Gratton & O'Donnell, 2011 ; Sumsion, 2013 ; Wimpenny & Savin-Baden, 2012 ).

Case selection processes were sometimes insufficient to understand why the case was selected from the global population of cases, or what study of this case would contribute to knowledge as compared with other possible cases (Adamson & Holloway, 2012 ; Bronken et al., 2012 ; Colón-Emeric et al., 2010 ; Jackson et al., 2012 ; Mawn et al., 2010 ). In two studies, local cases were selected (Barone, 2010 ; Fourie & Theron, 2012 ) because the researcher was familiar with and had access to the case. Possible limitations of a convenience sample were not acknowledged. Purposeful sampling was used to recruit participants within the case of one study, but not of the case itself (Gallagher et al., 2013 ). Random sampling was completed for case selection in two studies (Colón-Emeric et al., 2010 ; Jackson et al., 2012 ), which has limited meaning in interpretive qualitative research.

To demonstrate how researchers provided a good justification for the selection of case study approaches, four examples are provided. The first, cases of residential care homes, were selected because of reported occurrences of mistreatment, which included residents being locked in rooms at night (Rytterström, Unosson, & Arman, 2013 ). Roscigno et al. ( 2012 ) selected cases of parents who were admitted for early hospitalization in neonatal intensive care with a threatened preterm delivery before 26 weeks. Hooghe et al. ( 2012 ) used random sampling to select 20 couples that had experienced the death of a child; however, the case study was of one couple and a particular metaphor described only by them. The final example, Coltart and Henwood ( 2012 ), provided a detailed account of how they selected two cases from a sample of 46 fathers based on personal characteristics and beliefs. They described how the analysis of the two cases would contribute to their larger study on first time fathers and parenting.

Contextually bound case study

The limits or boundaries of the case are a defining factor of case study methodology (Merriam, 2009 ; Ragin & Becker, 1992 ; Stake, 1995 ; Yin, 2009 ). Adequate contextual description is required to understand the setting or context in which the case is revealed. In the health category, case studies were used to illustrate a clinical phenomenon or issue such as compliance and health behaviour (Colón-Emeric et al., 2010 ; D'Enbeau, Buzzanell, & Duckworth, 2010 ; Gallagher et al., 2013 ; Hooghe et al., 2012 ; Jackson et al., 2012 ; Roscigno et al., 2012 ). In these case studies, contextual boundaries, such as physical and institutional descriptions, were not sufficient to understand the case as a holistic system, for example, the general practitioner (GP) clinic in Gallagher et al. ( 2013 ), or the nursing home in Colón-Emeric et al. ( 2010 ). Similarly, in the social science and methods categories, attention was paid to some components of the case context, but not others, missing important information required to understand the case as a holistic system (Alexander, Moreira, & Kumar, 2012 ; Buzzanell & D'Enbeau, 2009 ; Nairn & Panelli, 2009 ; Wimpenny & Savin-Baden, 2012 ).

In two studies, vicarious experience or vignettes (Nairn & Panelli, 2009 ) and images (Jorrín-Abellán et al., 2008 ) were effective to support description of context, and might have been a useful addition for other case studies. Missing contextual boundaries suggests that the case might not be adequately defined. Additional information, such as the physical, institutional, political, and community context, would improve understanding of the case (Stake, 1998 ). In Boxes 1 and 2 , we present brief synopses of two studies that were reviewed, which demonstrated a well bounded case. In Box 1 , Ledderer ( 2011 ) used a qualitative case study design informed by Stake's tradition. In Box 2 , Gillard, Witt, and Watts ( 2011 ) were informed by Yin's tradition. By providing a brief outline of the case studies in Boxes 1 and 2 , we demonstrate how effective case boundaries can be constructed and reported, which may be of particular interest to prospective case study researchers.

Article synopsis of case study research using Stake's tradition

Ledderer ( 2011 ) used a qualitative case study research design, informed by modern ethnography. The study is bounded to 10 general practice clinics in Denmark, who had received federal funding to implement preventative care services based on a Motivational Interviewing intervention. The researcher question focussed on “why is it so difficult to create change in medical practice?” (Ledderer, 2011 , p. 27). The study context was adequately described, providing detail on the general practitioner (GP) clinics and relevant political and economic influences. Methodological decisions are described in first person narrative, providing insight on researcher perspectives and interaction with the case. Forty-four interviews were conducted, which focussed on how GPs conducted consultations, and the form, nature and content, rather than asking their opinion or experience (Ledderer, 2011 , p. 30). The duration and intensity of researcher immersion in the case enhanced depth of description and trustworthiness of study findings. Analysis was consistent with Stake's tradition, and the researcher provided examples of inquiry techniques used to challenge assumptions about emerging themes. Several other seminal qualitative works were cited. The themes and typology constructed are rich in narrative data and storytelling by clinic staff, demonstrating individual clinic experiences as well as shared meanings and understandings about changing from a biomedical to psychological approach to preventative health intervention. Conclusions make note of social and cultural meanings and lessons learned, which might not have been uncovered using a different methodology.

Article synopsis of case study research using Yin's tradition

Gillard et al. ( 2011 ) study of camps for adolescents living with HIV/AIDs provided a good example of Yin's interpretive case study approach. The context of the case is bounded by the three summer camps of which the researchers had prior professional involvement. A case study protocol was developed that used multiple methods to gather information at three data collection points coinciding with three youth camps (Teen Forum, Discover Camp, and Camp Strong). Gillard and colleagues followed Yin's ( 2009 ) principles, using a consistent data protocol that enhanced cross-case analysis. Data described the young people, the camp physical environment, camp schedule, objectives and outcomes, and the staff of three youth camps. The findings provided a detailed description of the context, with less detail of individual participants, including insight into researcher's interpretations and methodological decisions throughout the data collection and analysis process. Findings provided the reader with a sense of “being there,” and are discovered through constant comparison of the case with the research issues; the case is the unit of analysis. There is evidence of researcher immersion in the case, and Gillard reports spending significant time in the field in a naturalistic and integrated youth mentor role.

This case study is not intended to have a significant impact on broader health policy, although does have implications for health professionals working with adolescents. Study conclusions will inform future camps for young people with chronic disease, and practitioners are able to compare similarities between this case and their own practice (for knowledge translation). No limitations of this article were reported. Limitations related to publication of this case study were that it was 20 pages long and used three tables to provide sufficient description of the camp and program components, and relationships with the research issue.

Researcher and case interactions and triangulation

Researcher and case interactions and transactions are a defining feature of case study methodology (Stake, 1995 ). Narrative stories, vignettes, and thick description are used to provoke vicarious experience and a sense of being there with the researcher in their interaction with the case. Few of the case studies reviewed provided details of the researcher's relationship with the case, researcher–case interactions, and how these influenced the development of the case study (Buzzanell & D'Enbeau, 2009 ; D'Enbeau et al., 2010 ; Gallagher et al., 2013 ; Gillard et al., 2011 ; Ledderer, 2011 ; Nagar-Ron & Motzafi-Haller, 2011 ). The role and position of the researcher needed to be self-examined and understood by readers, to understand how this influenced interactions with participants, and to determine what triangulation is needed (Merriam, 2009 ; Stake, 1995 ).

Gillard et al. ( 2011 ) provided a good example of triangulation, comparing data sources in a table (p. 1513). Triangulation of sources was used to reveal as much depth as possible in the study by Nagar-Ron and Motzafi-Haller ( 2011 ), while also enhancing confirmation validity. There were several case studies that would have benefited from improved range and use of data sources, and descriptions of researcher–case interactions (Ajodhia-Andrews & Berman, 2009 ; Bronken et al., 2012 ; Fincham, Scourfield, & Langer, 2008 ; Fourie & Theron, 2012 ; Hooghe et al., 2012 ; Snyder-Young, 2011 ; Yeh, 2013 ).

Study design inconsistent with methodology

Good, rigorous case studies require a strong methodological justification (Meyer, 2001 ) and a logical and coherent argument that defines paradigm, methodological position, and selection of study methods (Denzin & Lincoln, 2011b ). Methodological justification was insufficient in several of the studies reviewed (Barone, 2010 ; Bronken et al., 2012 ; Hooghe et al., 2012 ; Mawn et al., 2010 ; Roscigno et al., 2012 ; Yeh, 2013 ). This was judged by the absence, or inadequate or inconsistent reference to case study methodology in-text.

In six studies, the methodological justification provided did not relate to case study. There were common issues identified. Secondary sources were used as primary methodological references indicating that study design might not have been theoretically sound (Colón-Emeric et al., 2010 ; Coltart & Henwood, 2012 ; Roscigno et al., 2012 ; Snyder-Young, 2011 ). Authors and sources cited in methodological descriptions were inconsistent with the actual study design and practices used (Fourie & Theron, 2012 ; Hooghe et al., 2012 ; Jorrín-Abellán et al., 2008 ; Mawn et al., 2010 ; Rytterström et al., 2013 ; Wimpenny & Savin-Baden, 2012 ). This occurred when researchers cited Stake or Yin, or both (Mawn et al., 2010 ; Rytterström et al., 2013 ), although did not follow their paradigmatic or methodological approach. In 26 studies there were no citations for a case study methodological approach.

The findings of this study have highlighted a number of issues for researchers. A considerable number of case studies reviewed were missing key elements that define qualitative case study methodology and the tradition cited. A significant number of studies did not provide a clear methodological description or justification relevant to case study. Case studies in health and social sciences did not provide sufficient information for the reader to understand case selection, and why this case was chosen above others. The context of the cases were not described in adequate detail to understand all relevant elements of the case context, which indicated that cases may have not been contextually bounded. There were inconsistencies between reported methodology, study design, and paradigmatic approach in case studies reviewed, which made it difficult to understand the study methodology and theoretical foundations. These issues have implications for methodological integrity and honesty when reporting study design, which are values of the qualitative research tradition and are ethical requirements (Wager & Kleinert, 2010a ). Poorly described methodological descriptions may lead the reader to misinterpret or discredit study findings, which limits the impact of the study, and, as a collective, hinders advancements in the broader qualitative research field.

The issues highlighted in our review build on current debates in the case study literature, and queries about the value of this methodology. Case study research can be situated within different paradigms or designed with an array of methods. In order to maintain the creativity and flexibility that is valued in this methodology, clearer descriptions of paradigm and theoretical position and methods should be provided so that study findings are not undervalued or discredited. Case study research is an interdisciplinary practice, which means that clear methodological descriptions might be more important for this approach than other methodologies that are predominantly driven by fewer disciplines (Creswell, 2013b ).

Authors frequently omit elements of methodologies and include others to strengthen study design, and we do not propose a rigid or purist ideology in this paper. On the contrary, we encourage new ideas about using case study, together with adequate reporting, which will advance the value and practice of case study. The implications of unclear methodological descriptions in the studies reviewed were that study design appeared to be inconsistent with reported methodology, and key elements required for making judgements of rigour were missing. It was not clear whether the deviations from methodological tradition were made by researchers to strengthen the study design, or because of misinterpretations. Morse ( 2011 ) recommended that innovations and deviations from practice are best made by experienced researchers, and that a novice might be unaware of the issues involved with making these changes. To perpetuate the tradition of case study research, applications in the published literature should have consistencies with traditional methodological constructions, and deviations should be described with a rationale that is inherent in study conduct and findings. Providing methodological descriptions that demonstrate a strong theoretical foundation and coherent study design will add credibility to the study, while ensuring the intrinsic meaning of case study is maintained.

The value of this review is that it contributes to discussion of whether case study is a methodology or method. We propose possible reasons why researchers might make this misinterpretation. Researchers may interchange the terms methods and methodology, and conduct research without adequate attention to epistemology and historical tradition (Carter & Little, 2007 ; Sandelowski, 2010 ). If the rich meaning that naming a qualitative methodology brings to the study is not recognized, a case study might appear to be inconsistent with the traditional approaches described by principal authors (Creswell, 2013a ; Merriam, 2009 ; Stake, 1995 ; Yin, 2009 ). If case studies are not methodologically and theoretically situated, then they might appear to be a case report.

Case reports are promoted by university and medical journals as a method of reporting on medical or scientific cases; guidelines for case reports are publicly available on websites ( http://www.hopkinsmedicine.org/institutional_review_board/guidelines_policies/guidelines/case_report.html ). The various case report guidelines provide a general criteria for case reports, which describes that this form of report does not meet the criteria of research, is used for retrospective analysis of up to three clinical cases, and is primarily illustrative and for educational purposes. Case reports can be published in academic journals, but do not require approval from a human research ethics committee. Traditionally, case reports describe a single case, to explain how and what occurred in a selected setting, for example, to illustrate a new phenomenon that has emerged from a larger study. A case report is not necessarily particular or the study of a case in its entirety, and the larger study would usually be guided by a different research methodology.

This description of a case report is similar to what was provided in some studies reviewed. This form of report lacks methodological grounding and qualities of research rigour. The case report has publication value in demonstrating an example and for dissemination of knowledge (Flanagan, 1999 ). However, case reports have different meaning and purpose to case study, which needs to be distinguished. Findings of our review suggest that the medical understanding of a case report has been confused with qualitative case study approaches.

In this review, a number of case studies did not have methodological descriptions that included key characteristics of case study listed in the adapted criteria, and several issues have been discussed. There have been calls for improvements in publication quality of qualitative research (Morse, 2011 ), and for improvements in peer review of submitted manuscripts (Carter & Little, 2007 ; Jasper, Vaismoradi, Bondas, & Turunen, 2013 ). The challenging nature of editor and reviewers responsibilities are acknowledged in the literature (Hames, 2013 ; Wager & Kleinert, 2010b ); however, review of case study methodology should be prioritized because of disputes on methodological value.

Authors using case study approaches are recommended to describe their theoretical framework and methods clearly, and to seek and follow specialist methodological advice when needed (Wager & Kleinert, 2010a ). Adequate page space for case study description would contribute to better publications (Gillard et al., 2011 ). Capitalizing on the ability to publish complementary resources should be considered.

Limitations of the review

There is a level of subjectivity involved in this type of review and this should be considered when interpreting study findings. Qualitative methods journals were selected because the aims and scope of these journals are to publish studies that contribute to methodological discussion and development of qualitative research. Generalist health and social science journals were excluded that might have contained good quality case studies. Journals in business or education were also excluded, although a review of case studies in international business journals has been published elsewhere (Piekkari et al., 2009 ).

The criteria used to assess the quality of the case studies were a set of qualitative indicators. A numerical or ranking system might have resulted in different results. Stake's ( 1995 ) criteria have been referenced elsewhere, and was deemed the best available (Creswell, 2013b ; Crowe et al., 2011 ). Not all qualitative studies are reported in a consistent way and some authors choose to report findings in a narrative form in comparison to a typical biomedical report style (Sandelowski & Barroso, 2002 ), if misinterpretations were made this may have affected the review.

Case study research is an increasingly popular approach among qualitative researchers, which provides methodological flexibility through the incorporation of different paradigmatic positions, study designs, and methods. However, whereas flexibility can be an advantage, a myriad of different interpretations has resulted in critics questioning the use of case study as a methodology. Using an adaptation of established criteria, we aimed to identify and assess the methodological descriptions of case studies in high impact, qualitative methods journals. Few articles were identified that applied qualitative case study approaches as described by experts in case study design. There were inconsistencies in methodology and study design, which indicated that researchers were confused whether case study was a methodology or a method. Commonly, there appeared to be confusion between case studies and case reports. Without clear understanding and application of the principles and key elements of case study methodology, there is a risk that the flexibility of the approach will result in haphazard reporting, and will limit its global application as a valuable, theoretically supported methodology that can be rigorously applied across disciplines and fields.

Conflict of interest and funding

The authors have not received any funding or benefits from industry or elsewhere to conduct this study.

• Adamson S, Holloway M. Negotiating sensitivities and grappling with intangibles: Experiences from a study of spirituality and funerals. Qualitative Research. 2012; 12 (6):735–752. doi: 10.1177/1468794112439008. [ CrossRef ] [ Google Scholar ]
• Ajodhia-Andrews A, Berman R. Exploring school life from the lens of a child who does not use speech to communicate. Qualitative Inquiry. 2009; 15 (5):931–951. doi: 10.1177/1077800408322789. [ CrossRef ] [ Google Scholar ]
• Alexander B. K, Moreira C, Kumar H. S. Resisting (resistance) stories: A tri-autoethnographic exploration of father narratives across shades of difference. Qualitative Inquiry. 2012; 18 (2):121–133. doi: 10.1177/1077800411429087. [ CrossRef ] [ Google Scholar ]
• Austin W, Park C, Goble E. From interdisciplinary to transdisciplinary research: A case study. Qualitative Health Research. 2008; 18 (4):557–564. doi: 10.1177/1049732307308514. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ayres L, Kavanaugh K, Knafl K. A. Within-case and across-case approaches to qualitative data analysis. Qualitative Health Research. 2003; 13 (6):871–883. doi: 10.1177/1049732303013006008. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Barone T. L. Culturally sensitive care 1969–2000: The Indian Chicano Health Center. Qualitative Health Research. 2010; 20 (4):453–464. doi: 10.1177/1049732310361893. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bassey M. A solution to the problem of generalisation in educational research: Fuzzy prediction. Oxford Review of Education. 2001; 27 (1):5–22. doi: 10.1080/03054980123773. [ CrossRef ] [ Google Scholar ]
• Bronken B. A, Kirkevold M, Martinsen R, Kvigne K. The aphasic storyteller: Coconstructing stories to promote psychosocial well-being after stroke. Qualitative Health Research. 2012; 22 (10):1303–1316. doi: 10.1177/1049732312450366. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Broyles L. M, Rodriguez K. L, Price P. A, Bayliss N. K, Sevick M. A. Overcoming barriers to the recruitment of nurses as participants in health care research. Qualitative Health Research. 2011; 21 (12):1705–1718. doi: 10.1177/1049732311417727. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Buckley C. A, Waring M. J. Using diagrams to support the research process: Examples from grounded theory. Qualitative Research. 2013; 13 (2):148–172. doi: 10.1177/1468794112472280. [ CrossRef ] [ Google Scholar ]
• Buzzanell P. M, D'Enbeau S. Stories of caregiving: Intersections of academic research and women's everyday experiences. Qualitative Inquiry. 2009; 15 (7):1199–1224. doi: 10.1177/1077800409338025. [ CrossRef ] [ Google Scholar ]
• Carter S. M, Little M. Justifying knowledge, justifying method, taking action: Epistemologies, methodologies, and methods in qualitative research. Qualitative Health Research. 2007; 17 (10):1316–1328. doi: 10.1177/1049732307306927. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cheek J, Garnham B, Quan J. What's in a number? Issues in providing evidence of impact and quality of research(ers) Qualitative Health Research. 2006; 16 (3):423–435. doi: 10.1177/1049732305285701. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Colón-Emeric C. S, Plowman D, Bailey D, Corazzini K, Utley-Smith Q, Ammarell N, et al. Regulation and mindful resident care in nursing homes. Qualitative Health Research. 2010; 20 (9):1283–1294. doi: 10.1177/1049732310369337. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Coltart C, Henwood K. On paternal subjectivity: A qualitative longitudinal and psychosocial case analysis of men's classed positions and transitions to first-time fatherhood. Qualitative Research. 2012; 12 (1):35–52. doi: 10.1177/1468794111426224. [ CrossRef ] [ Google Scholar ]
• Creswell J. W. Five qualitative approaches to inquiry. In: Creswell J. W, editor. Qualitative inquiry and research design: Choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage; 2013a. pp. 53–84. [ Google Scholar ]
• Creswell J. W. Qualitative inquiry and research design: Choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage; 2013b. [ Google Scholar ]
• Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Medical Research Methodology. 2011; 11 (1):1–9. doi: 10.1186/1471-2288-11-100. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cunsolo Willox A, Harper S. L, Edge V. L, ‘My Word’: Storytelling and Digital Media Lab, & Rigolet Inuit Community Government Storytelling in a digital age: Digital storytelling as an emerging narrative method for preserving and promoting indigenous oral wisdom. Qualitative Research. 2013; 13 (2):127–147. doi: 10.1177/1468794112446105. [ CrossRef ] [ Google Scholar ]
• De Haene L, Grietens H, Verschueren K. Holding harm: Narrative methods in mental health research on refugee trauma. Qualitative Health Research. 2010; 20 (12):1664–1676. doi: 10.1177/1049732310376521. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• D'Enbeau S, Buzzanell P. M, Duckworth J. Problematizing classed identities in fatherhood: Development of integrative case studies for analysis and praxis. Qualitative Inquiry. 2010; 16 (9):709–720. doi: 10.1177/1077800410374183. [ CrossRef ] [ Google Scholar ]
• Denzin N. K, Lincoln Y. S. Introduction: Disciplining the practice of qualitative research. In: Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011a. pp. 1–6. [ Google Scholar ]
• Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011b. [ Google Scholar ]
• Edwards R, Weller S. Shifting analytic ontology: Using I-poems in qualitative longitudinal research. Qualitative Research. 2012; 12 (2):202–217. doi: 10.1177/1468794111422040. [ CrossRef ] [ Google Scholar ]
• Eisenhardt K. M. Building theories from case study research. The Academy of Management Review. 1989; 14 (4):532–550. doi: 10.2307/258557. [ CrossRef ] [ Google Scholar ]
• Fincham B, Scourfield J, Langer S. The impact of working with disturbing secondary data: Reading suicide files in a coroner's office. Qualitative Health Research. 2008; 18 (6):853–862. doi: 10.1177/1049732307308945. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Flanagan J. Public participation in the design of educational programmes for cancer nurses: A case report. European Journal of Cancer Care. 1999; 8 (2):107–112. doi: 10.1046/j.1365-2354.1999.00141.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Flyvbjerg B. Five misunderstandings about case-study research. Qualitative Inquiry. 2006; 12 (2):219–245. doi: 10.1177/1077800405284.363. [ CrossRef ] [ Google Scholar ]
• Flyvbjerg B. Case study. In: Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011. pp. 301–316. [ Google Scholar ]
• Fourie C. L, Theron L. C. Resilience in the face of fragile X syndrome. Qualitative Health Research. 2012; 22 (10):1355–1368. doi: 10.1177/1049732312451871. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gallagher N, MacFarlane A, Murphy A. W, Freeman G. K, Glynn L. G, Bradley C. P. Service users’ and caregivers’ perspectives on continuity of care in out-of-hours primary care. Qualitative Health Research. 2013; 23 (3):407–421. doi: 10.1177/1049732312470521. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gerring J. What is a case study and what is it good for? American Political Science Review. 2004; 98 (2):341–354. doi: 10.1017/S0003055404001182. [ CrossRef ] [ Google Scholar ]
• Gillard A, Witt P. A, Watts C. E. Outcomes and processes at a camp for youth with HIV/AIDS. Qualitative Health Research. 2011; 21 (11):1508–1526. doi: 10.1177/1049732311413907. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Grant M, Booth A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal. 2009; 26 :91–108. doi: 10.1111/j.1471-1842.2009.00848.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gratton M.-F, O'Donnell S. Communication technologies for focus groups with remote communities: A case study of research with First Nations in Canada. Qualitative Research. 2011; 11 (2):159–175. doi: 10.1177/1468794110394068. [ CrossRef ] [ Google Scholar ]
• Hallberg L. Quality criteria and generalization of results from qualitative studies. International Journal of Qualitative Studies on Health and Wellbeing. 2013; 8 :1. doi: 10.3402/qhw.v8i0.20647. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hames I. Committee on Publication Ethics, 1. 2013, March. COPE Ethical guidelines for peer reviewers. Retrieved April 7, 2013, from http://publicationethics.org/resources/guidelines . [ Google Scholar ]
• Hooghe A, Neimeyer R. A, Rober P. “Cycling around an emotional core of sadness”: Emotion regulation in a couple after the loss of a child. Qualitative Health Research. 2012; 22 (9):1220–1231. doi: 10.1177/1049732312449209. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jackson C. B, Botelho E. M, Welch L. C, Joseph J, Tennstedt S. L. Talking with others about stigmatized health conditions: Implications for managing symptoms. Qualitative Health Research. 2012; 22 (11):1468–1475. doi: 10.1177/1049732312450323. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jasper M, Vaismoradi M, Bondas T, Turunen H. Validity and reliability of the scientific review process in nursing journals—time for a rethink? Nursing Inquiry. 2013 doi: 10.1111/nin.12030. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jensen J. L, Rodgers R. Cumulating the intellectual gold of case study research. Public Administration Review. 2001; 61 (2):235–246. doi: 10.1111/0033-3352.00025. [ CrossRef ] [ Google Scholar ]
• Jorrín-Abellán I. M, Rubia-Avi B, Anguita-Martínez R, Gómez-Sánchez E, Martínez-Mones A. Bouncing between the dark and bright sides: Can technology help qualitative research? Qualitative Inquiry. 2008; 14 (7):1187–1204. doi: 10.1177/1077800408318435. [ CrossRef ] [ Google Scholar ]
• Ledderer L. Understanding change in medical practice: The role of shared meaning in preventive treatment. Qualitative Health Research. 2011; 21 (1):27–40. doi: 10.1177/1049732310377451. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lincoln Y. S. Emerging criteria for quality in qualitative and interpretive research. Qualitative Inquiry. 1995; 1 (3):275–289. doi: 10.1177/107780049500100301. [ CrossRef ] [ Google Scholar ]
• Luck L, Jackson D, Usher K. Case study: A bridge across the paradigms. Nursing Inquiry. 2006; 13 (2):103–109. doi: 10.1111/j.1440-1800.2006.00309.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mawn B, Siqueira E, Koren A, Slatin C, Devereaux Melillo K, Pearce C, et al. Health disparities among health care workers. Qualitative Health Research. 2010; 20 (1):68–80. doi: 10.1177/1049732309355590. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Merriam S. B. Qualitative research: A guide to design and implementation. 3rd ed. San Francisco, CA: Jossey-Bass; 2009. [ Google Scholar ]
• Meyer C. B. A case in case study methodology. Field Methods. 2001; 13 (4):329–352. doi: 10.1177/1525822x0101300402. [ CrossRef ] [ Google Scholar ]
• Morse J. M. Mixing qualitative methods. Qualitative Health Research. 2009; 19 (11):1523–1524. doi: 10.1177/1049732309349360. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Morse J. M. Molding qualitative health research. Qualitative Health Research. 2011; 21 (8):1019–1021. doi: 10.1177/1049732311404706. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Morse J. M, Dimitroff L. J, Harper R, Koontz A, Kumra S, Matthew-Maich N, et al. Considering the qualitative–quantitative language divide. Qualitative Health Research. 2011; 21 (9):1302–1303. doi: 10.1177/1049732310392386. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nagar-Ron S, Motzafi-Haller P. “My life? There is not much to tell”: On voice, silence and agency in interviews with first-generation Mizrahi Jewish women immigrants to Israel. Qualitative Inquiry. 2011; 17 (7):653–663. doi: 10.1177/1077800411414007. [ CrossRef ] [ Google Scholar ]
• Nairn K, Panelli R. Using fiction to make meaning in research with young people in rural New Zealand. Qualitative Inquiry. 2009; 15 (1):96–112. doi: 10.1177/1077800408318314. [ CrossRef ] [ Google Scholar ]
• Nespor J. The afterlife of “teachers’ beliefs”: Qualitative methodology and the textline. Qualitative Inquiry. 2012; 18 (5):449–460. doi: 10.1177/1077800412439530. [ CrossRef ] [ Google Scholar ]
• Piekkari R, Welch C, Paavilainen E. The case study as disciplinary convention: Evidence from international business journals. Organizational Research Methods. 2009; 12 (3):567–589. doi: 10.1177/1094428108319905. [ CrossRef ] [ Google Scholar ]
• Ragin C. C, Becker H. S. What is a case?: Exploring the foundations of social inquiry. Cambridge: Cambridge University Press; 1992. [ Google Scholar ]
• Roscigno C. I, Savage T. A, Kavanaugh K, Moro T. T, Kilpatrick S. J, Strassner H. T, et al. Divergent views of hope influencing communications between parents and hospital providers. Qualitative Health Research. 2012; 22 (9):1232–1246. doi: 10.1177/1049732312449210. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rosenberg J. P, Yates P. M. Schematic representation of case study research designs. Journal of Advanced Nursing. 2007; 60 (4):447–452. doi: 10.1111/j.1365-2648.2007.04385.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rytterström P, Unosson M, Arman M. Care culture as a meaning- making process: A study of a mistreatment investigation. Qualitative Health Research. 2013; 23 :1179–1187. doi: 10.1177/1049732312470760. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M. Whatever happened to qualitative description? Research in Nursing & Health. 2000; 23 (4):334–340. doi: 10.1002/1098-240X. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M. What's in a name? Qualitative description revisited. Research in Nursing & Health. 2010; 33 (1):77–84. doi: 10.1002/nur.20362. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M, Barroso J. Reading qualitative studies. International Journal of Qualitative Methods. 2002; 1 (1):74–108. [ Google Scholar ]
• Snyder-Young D. “Here to tell her story”: Analyzing the autoethnographic performances of others. Qualitative Inquiry. 2011; 17 (10):943–951. doi: 10.1177/1077800411425149. [ CrossRef ] [ Google Scholar ]
• Stake R. E. The case study method in social inquiry. Educational Researcher. 1978; 7 (2):5–8. [ Google Scholar ]
• Stake R. E. The art of case study research. Thousand Oaks, CA: Sage; 1995. [ Google Scholar ]
• Stake R. E. Case studies. In: Denzin N. K, Lincoln Y. S, editors. Strategies of qualitative inquiry. Thousand Oaks, CA: Sage; 1998. pp. 86–109. [ Google Scholar ]
• Sumsion J. Opening up possibilities through team research: Investigating infants’ experiences of early childhood education and care. Qualitative Research. 2013; 14 (2):149–165. doi: 10.1177/1468794112468471.. [ CrossRef ] [ Google Scholar ]
• Thomas G. Doing case study: Abduction not induction, phronesis not theory. Qualitative Inquiry. 2010; 16 (7):575–582. doi: 10.1177/1077800410372601. [ CrossRef ] [ Google Scholar ]
• Thomas G. A typology for the case study in social science following a review of definition, discourse, and structure. Qualitative Inquiry. 2011; 17 (6):511–521. doi: 10.1177/1077800411409884. [ CrossRef ] [ Google Scholar ]
• Tight M. The curious case of case study: A viewpoint. International Journal of Social Research Methodology. 2010; 13 (4):329–339. doi: 10.1080/13645570903187181. [ CrossRef ] [ Google Scholar ]
• Wager E, Kleinert S. Responsible research publication: International standards for authors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. In: Mayer T, Steneck N, editors. Promoting research integrity in a global environment. Singapore: Imperial College Press/World Scientific; 2010a. pp. 309–316. [ Google Scholar ]
• Wager E, Kleinert S. Responsible research publication: International standards for editors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. In: Mayer T, Steneck N, editors. Promoting research integrity in a global environment. Singapore: Imperial College Press/World Scientific; 2010b. pp. 317–328. [ Google Scholar ]
• Webb C, Kevern J. Focus groups as a research method: A critique of some aspects of their use in nursing research. Journal of Advanced Nursing. 2000; 33 (6):798–805. doi: 10.1046/j.1365-2648.2001.01720.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wimpenny K, Savin-Baden M. Exploring and implementing participatory action synthesis. Qualitative Inquiry. 2012; 18 (8):689–698. doi: 10.1177/1077800412452854. [ CrossRef ] [ Google Scholar ]
• Yeh H.-Y. Boundaries, entities, and modern vegetarianism: Examining the emergence of the first vegetarian organization. Qualitative Inquiry. 2013; 19 (4):298–309. doi: 10.1177/1077800412471516. [ CrossRef ] [ Google Scholar ]
• Yin R. K. Enhancing the quality of case studies in health services research. Health Services Research. 1999; 34 (5 Pt 2):1209–1224. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Yin R. K. Case study research: Design and methods. 4th ed. Thousand Oaks, CA: Sage; 2009. [ Google Scholar ]
• Yin R. K. Applications of case study research. 3rd ed. Thousand Oaks, CA: Sage; 2012. [ Google Scholar ]