Single-Case Design, Analysis, and Quality Assessment for Intervention Research

Affiliation.

  • 1 Biomechanics & Movement Science Program, Department of Physical Therapy, University of Delaware, Newark, Delaware (M.A.L., A.B.C., I.B.); and Division of Educational Psychology & Methodology, State University of New York at Albany, Albany, New York (M.M.).
  • PMID: 28628553
  • PMCID: PMC5492992
  • DOI: 10.1097/NPT.0000000000000187

Background and purpose: The purpose of this article is to describe single-case studies and contrast them with case studies and randomized clinical trials. We highlight current research designs, analysis techniques, and quality appraisal tools relevant for single-case rehabilitation research.

Summary of key points: Single-case studies can provide a viable alternative to large group studies such as randomized clinical trials. Single-case studies involve repeated measures and manipulation of an independent variable. They can be designed to have strong internal validity for assessing causal relationships between interventions and outcomes, as well as external validity for generalizability of results, particularly when the study designs incorporate replication, randomization, and multiple participants. Single-case studies should not be confused with case studies/series (ie, case reports), which are reports of clinical management of a patient or a small series of patients.

Recommendations for clinical practice: When rigorously designed, single-case studies can be particularly useful experimental designs in a variety of situations, such as when research resources are limited, studied conditions have low incidences, or when examining effects of novel or expensive interventions. Readers will be directed to examples from the published literature in which these techniques have been discussed, evaluated for quality, and implemented.

  • Cohort Studies
  • Medical Records*
  • Quality Assurance, Health Care*
  • Randomized Controlled Trials as Topic
  • Research Design*

Grants and funding

  • R21 HD076092/HD/NICHD NIH HHS/United States

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • View all journals
  • Explore content
  • About the journal
  • Publish with us
  • Sign up for alerts
  • Perspective
  • Published: 22 November 2022

Single case studies are a powerful tool for developing, testing and extending theories

  • Lyndsey Nickels   ORCID: orcid.org/0000-0002-0311-3524 1 , 2 ,
  • Simon Fischer-Baum   ORCID: orcid.org/0000-0002-6067-0538 3 &
  • Wendy Best   ORCID: orcid.org/0000-0001-8375-5916 4  

Nature Reviews Psychology volume  1 ,  pages 733–747 ( 2022 ) Cite this article

577 Accesses

5 Citations

26 Altmetric

Metrics details

  • Neurological disorders

Psychology embraces a diverse range of methodologies. However, most rely on averaging group data to draw conclusions. In this Perspective, we argue that single case methodology is a valuable tool for developing and extending psychological theories. We stress the importance of single case and case series research, drawing on classic and contemporary cases in which cognitive and perceptual deficits provide insights into typical cognitive processes in domains such as memory, delusions, reading and face perception. We unpack the key features of single case methodology, describe its strengths, its value in adjudicating between theories, and outline its benefits for a better understanding of deficits and hence more appropriate interventions. The unique insights that single case studies have provided illustrate the value of in-depth investigation within an individual. Single case methodology has an important place in the psychologist’s toolkit and it should be valued as a primary research tool.

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 12 digital issues and online access to articles

55,14 € per year

only 4,60 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

single case study of intervention

Corkin, S. Permanent Present Tense: The Unforgettable Life Of The Amnesic Patient, H. M . Vol. XIX, 364 (Basic Books, 2013).

Lilienfeld, S. O. Psychology: From Inquiry To Understanding (Pearson, 2019).

Schacter, D. L., Gilbert, D. T., Nock, M. K. & Wegner, D. M. Psychology (Worth Publishers, 2019).

Eysenck, M. W. & Brysbaert, M. Fundamentals Of Cognition (Routledge, 2018).

Squire, L. R. Memory and brain systems: 1969–2009. J. Neurosci. 29 , 12711–12716 (2009).

Article   PubMed   PubMed Central   Google Scholar  

Corkin, S. What’s new with the amnesic patient H.M.? Nat. Rev. Neurosci. 3 , 153–160 (2002).

Article   PubMed   Google Scholar  

Schubert, T. M. et al. Lack of awareness despite complex visual processing: evidence from event-related potentials in a case of selective metamorphopsia. Proc. Natl Acad. Sci. USA 117 , 16055–16064 (2020).

Behrmann, M. & Plaut, D. C. Bilateral hemispheric processing of words and faces: evidence from word impairments in prosopagnosia and face impairments in pure alexia. Cereb. Cortex 24 , 1102–1118 (2014).

Plaut, D. C. & Behrmann, M. Complementary neural representations for faces and words: a computational exploration. Cogn. Neuropsychol. 28 , 251–275 (2011).

Haxby, J. V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293 , 2425–2430 (2001).

Hirshorn, E. A. et al. Decoding and disrupting left midfusiform gyrus activity during word reading. Proc. Natl Acad. Sci. USA 113 , 8162–8167 (2016).

Kosakowski, H. L. et al. Selective responses to faces, scenes, and bodies in the ventral visual pathway of infants. Curr. Biol. 32 , 265–274.e5 (2022).

Harlow, J. Passage of an iron rod through the head. Boston Med. Surgical J . https://doi.org/10.1176/jnp.11.2.281 (1848).

Broca, P. Remarks on the seat of the faculty of articulated language, following an observation of aphemia (loss of speech). Bull. Soc. Anat. 6 , 330–357 (1861).

Google Scholar  

Dejerine, J. Contribution A L’étude Anatomo-pathologique Et Clinique Des Différentes Variétés De Cécité Verbale: I. Cécité Verbale Avec Agraphie Ou Troubles Très Marqués De L’écriture; II. Cécité Verbale Pure Avec Intégrité De L’écriture Spontanée Et Sous Dictée (Société de Biologie, 1892).

Liepmann, H. Das Krankheitsbild der Apraxie (“motorischen Asymbolie”) auf Grund eines Falles von einseitiger Apraxie (Fortsetzung). Eur. Neurol. 8 , 102–116 (1900).

Article   Google Scholar  

Basso, A., Spinnler, H., Vallar, G. & Zanobio, M. E. Left hemisphere damage and selective impairment of auditory verbal short-term memory. A case study. Neuropsychologia 20 , 263–274 (1982).

Humphreys, G. W. & Riddoch, M. J. The fractionation of visual agnosia. In Visual Object Processing: A Cognitive Neuropsychological Approach 281–306 (Lawrence Erlbaum, 1987).

Whitworth, A., Webster, J. & Howard, D. A Cognitive Neuropsychological Approach To Assessment And Intervention In Aphasia (Psychology Press, 2014).

Caramazza, A. On drawing inferences about the structure of normal cognitive systems from the analysis of patterns of impaired performance: the case for single-patient studies. Brain Cogn. 5 , 41–66 (1986).

Caramazza, A. & McCloskey, M. The case for single-patient studies. Cogn. Neuropsychol. 5 , 517–527 (1988).

Shallice, T. Cognitive neuropsychology and its vicissitudes: the fate of Caramazza’s axioms. Cogn. Neuropsychol. 32 , 385–411 (2015).

Shallice, T. From Neuropsychology To Mental Structure (Cambridge Univ. Press, 1988).

Coltheart, M. Assumptions and methods in cognitive neuropscyhology. In The Handbook Of Cognitive Neuropsychology: What Deficits Reveal About The Human Mind (ed. Rapp, B.) 3–22 (Psychology Press, 2001).

McCloskey, M. & Chaisilprungraung, T. The value of cognitive neuropsychology: the case of vision research. Cogn. Neuropsychol. 34 , 412–419 (2017).

McCloskey, M. The future of cognitive neuropsychology. In The Handbook Of Cognitive Neuropsychology: What Deficits Reveal About The Human Mind (ed. Rapp, B.) 593–610 (Psychology Press, 2001).

Lashley, K. S. In search of the engram. In Physiological Mechanisms in Animal Behavior 454–482 (Academic Press, 1950).

Squire, L. R. & Wixted, J. T. The cognitive neuroscience of human memory since H.M. Annu. Rev. Neurosci. 34 , 259–288 (2011).

Stone, G. O., Vanhoy, M. & Orden, G. C. V. Perception is a two-way street: feedforward and feedback phonology in visual word recognition. J. Mem. Lang. 36 , 337–359 (1997).

Perfetti, C. A. The psycholinguistics of spelling and reading. In Learning To Spell: Research, Theory, And Practice Across Languages 21–38 (Lawrence Erlbaum, 1997).

Nickels, L. The autocue? self-generated phonemic cues in the treatment of a disorder of reading and naming. Cogn. Neuropsychol. 9 , 155–182 (1992).

Rapp, B., Benzing, L. & Caramazza, A. The autonomy of lexical orthography. Cogn. Neuropsychol. 14 , 71–104 (1997).

Bonin, P., Roux, S. & Barry, C. Translating nonverbal pictures into verbal word names. Understanding lexical access and retrieval. In Past, Present, And Future Contributions Of Cognitive Writing Research To Cognitive Psychology 315–522 (Psychology Press, 2011).

Bonin, P., Fayol, M. & Gombert, J.-E. Role of phonological and orthographic codes in picture naming and writing: an interference paradigm study. Cah. Psychol. Cogn./Current Psychol. Cogn. 16 , 299–324 (1997).

Bonin, P., Fayol, M. & Peereman, R. Masked form priming in writing words from pictures: evidence for direct retrieval of orthographic codes. Acta Psychol. 99 , 311–328 (1998).

Bentin, S., Allison, T., Puce, A., Perez, E. & McCarthy, G. Electrophysiological studies of face perception in humans. J. Cogn. Neurosci. 8 , 551–565 (1996).

Jeffreys, D. A. Evoked potential studies of face and object processing. Vis. Cogn. 3 , 1–38 (1996).

Laganaro, M., Morand, S., Michel, C. M., Spinelli, L. & Schnider, A. ERP correlates of word production before and after stroke in an aphasic patient. J. Cogn. Neurosci. 23 , 374–381 (2011).

Indefrey, P. & Levelt, W. J. M. The spatial and temporal signatures of word production components. Cognition 92 , 101–144 (2004).

Valente, A., Burki, A. & Laganaro, M. ERP correlates of word production predictors in picture naming: a trial by trial multiple regression analysis from stimulus onset to response. Front. Neurosci. 8 , 390 (2014).

Kittredge, A. K., Dell, G. S., Verkuilen, J. & Schwartz, M. F. Where is the effect of frequency in word production? Insights from aphasic picture-naming errors. Cogn. Neuropsychol. 25 , 463–492 (2008).

Domdei, N. et al. Ultra-high contrast retinal display system for single photoreceptor psychophysics. Biomed. Opt. Express 9 , 157 (2018).

Poldrack, R. A. et al. Long-term neural and physiological phenotyping of a single human. Nat. Commun. 6 , 8885 (2015).

Coltheart, M. The assumptions of cognitive neuropsychology: reflections on Caramazza (1984, 1986). Cogn. Neuropsychol. 34 , 397–402 (2017).

Badecker, W. & Caramazza, A. A final brief in the case against agrammatism: the role of theory in the selection of data. Cognition 24 , 277–282 (1986).

Fischer-Baum, S. Making sense of deviance: Identifying dissociating cases within the case series approach. Cogn. Neuropsychol. 30 , 597–617 (2013).

Nickels, L., Howard, D. & Best, W. On the use of different methodologies in cognitive neuropsychology: drink deep and from several sources. Cogn. Neuropsychol. 28 , 475–485 (2011).

Dell, G. S. & Schwartz, M. F. Who’s in and who’s out? Inclusion criteria, model evaluation, and the treatment of exceptions in case series. Cogn. Neuropsychol. 28 , 515–520 (2011).

Schwartz, M. F. & Dell, G. S. Case series investigations in cognitive neuropsychology. Cogn. Neuropsychol. 27 , 477–494 (2010).

Cohen, J. A power primer. Psychol. Bull. 112 , 155–159 (1992).

Martin, R. C. & Allen, C. Case studies in neuropsychology. In APA Handbook Of Research Methods In Psychology Vol. 2 Research Designs: Quantitative, Qualitative, Neuropsychological, And Biological (eds Cooper, H. et al.) 633–646 (American Psychological Association, 2012).

Leivada, E., Westergaard, M., Duñabeitia, J. A. & Rothman, J. On the phantom-like appearance of bilingualism effects on neurocognition: (how) should we proceed? Bilingualism 24 , 197–210 (2021).

Arnett, J. J. The neglected 95%: why American psychology needs to become less American. Am. Psychol. 63 , 602–614 (2008).

Stolz, J. A., Besner, D. & Carr, T. H. Implications of measures of reliability for theories of priming: activity in semantic memory is inherently noisy and uncoordinated. Vis. Cogn. 12 , 284–336 (2005).

Cipora, K. et al. A minority pulls the sample mean: on the individual prevalence of robust group-level cognitive phenomena — the instance of the SNARC effect. Preprint at psyArXiv https://doi.org/10.31234/osf.io/bwyr3 (2019).

Andrews, S., Lo, S. & Xia, V. Individual differences in automatic semantic priming. J. Exp. Psychol. Hum. Percept. Perform. 43 , 1025–1039 (2017).

Tan, L. C. & Yap, M. J. Are individual differences in masked repetition and semantic priming reliable? Vis. Cogn. 24 , 182–200 (2016).

Olsson-Collentine, A., Wicherts, J. M. & van Assen, M. A. L. M. Heterogeneity in direct replications in psychology and its association with effect size. Psychol. Bull. 146 , 922–940 (2020).

Gratton, C. & Braga, R. M. Editorial overview: deep imaging of the individual brain: past, practice, and promise. Curr. Opin. Behav. Sci. 40 , iii–vi (2021).

Fedorenko, E. The early origins and the growing popularity of the individual-subject analytic approach in human neuroscience. Curr. Opin. Behav. Sci. 40 , 105–112 (2021).

Xue, A. et al. The detailed organization of the human cerebellum estimated by intrinsic functional connectivity within the individual. J. Neurophysiol. 125 , 358–384 (2021).

Petit, S. et al. Toward an individualized neural assessment of receptive language in children. J. Speech Lang. Hear. Res. 63 , 2361–2385 (2020).

Jung, K.-H. et al. Heterogeneity of cerebral white matter lesions and clinical correlates in older adults. Stroke 52 , 620–630 (2021).

Falcon, M. I., Jirsa, V. & Solodkin, A. A new neuroinformatics approach to personalized medicine in neurology: the virtual brain. Curr. Opin. Neurol. 29 , 429–436 (2016).

Duncan, G. J., Engel, M., Claessens, A. & Dowsett, C. J. Replication and robustness in developmental research. Dev. Psychol. 50 , 2417–2425 (2014).

Open Science Collaboration. Estimating the reproducibility of psychological science. Science 349 , aac4716 (2015).

Tackett, J. L., Brandes, C. M., King, K. M. & Markon, K. E. Psychology’s replication crisis and clinical psychological science. Annu. Rev. Clin. Psychol. 15 , 579–604 (2019).

Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav. 1 , 0021 (2017).

Oldfield, R. C. & Wingfield, A. The time it takes to name an object. Nature 202 , 1031–1032 (1964).

Oldfield, R. C. & Wingfield, A. Response latencies in naming objects. Q. J. Exp. Psychol. 17 , 273–281 (1965).

Brysbaert, M. How many participants do we have to include in properly powered experiments? A tutorial of power analysis with reference tables. J. Cogn. 2 , 16 (2019).

Brysbaert, M. Power considerations in bilingualism research: time to step up our game. Bilingualism https://doi.org/10.1017/S1366728920000437 (2020).

Machery, E. What is a replication? Phil. Sci. 87 , 545–567 (2020).

Nosek, B. A. & Errington, T. M. What is replication? PLoS Biol. 18 , e3000691 (2020).

Li, X., Huang, L., Yao, P. & Hyönä, J. Universal and specific reading mechanisms across different writing systems. Nat. Rev. Psychol. 1 , 133–144 (2022).

Rapp, B. (Ed.) The Handbook Of Cognitive Neuropsychology: What Deficits Reveal About The Human Mind (Psychology Press, 2001).

Code, C. et al. Classic Cases In Neuropsychology (Psychology Press, 1996).

Patterson, K., Marshall, J. C. & Coltheart, M. Surface Dyslexia: Neuropsychological And Cognitive Studies Of Phonological Reading (Routledge, 2017).

Marshall, J. C. & Newcombe, F. Patterns of paralexia: a psycholinguistic approach. J. Psycholinguist. Res. 2 , 175–199 (1973).

Castles, A. & Coltheart, M. Varieties of developmental dyslexia. Cognition 47 , 149–180 (1993).

Khentov-Kraus, L. & Friedmann, N. Vowel letter dyslexia. Cogn. Neuropsychol. 35 , 223–270 (2018).

Winskel, H. Orthographic and phonological parafoveal processing of consonants, vowels, and tones when reading Thai. Appl. Psycholinguist. 32 , 739–759 (2011).

Hepner, C., McCloskey, M. & Rapp, B. Do reading and spelling share orthographic representations? Evidence from developmental dysgraphia. Cogn. Neuropsychol. 34 , 119–143 (2017).

Hanley, J. R. & Sotiropoulos, A. Developmental surface dysgraphia without surface dyslexia. Cogn. Neuropsychol. 35 , 333–341 (2018).

Zihl, J. & Heywood, C. A. The contribution of single case studies to the neuroscience of vision: single case studies in vision neuroscience. Psych. J. 5 , 5–17 (2016).

Bouvier, S. E. & Engel, S. A. Behavioral deficits and cortical damage loci in cerebral achromatopsia. Cereb. Cortex 16 , 183–191 (2006).

Zihl, J. & Heywood, C. A. The contribution of LM to the neuroscience of movement vision. Front. Integr. Neurosci. 9 , 6 (2015).

Dotan, D. & Friedmann, N. Separate mechanisms for number reading and word reading: evidence from selective impairments. Cortex 114 , 176–192 (2019).

McCloskey, M. & Schubert, T. Shared versus separate processes for letter and digit identification. Cogn. Neuropsychol. 31 , 437–460 (2014).

Fayol, M. & Seron, X. On numerical representations. Insights from experimental, neuropsychological, and developmental research. In Handbook of Mathematical Cognition (ed. Campbell, J.) 3–23 (Psychological Press, 2005).

Bornstein, B. & Kidron, D. P. Prosopagnosia. J. Neurol. Neurosurg. Psychiat. 22 , 124–131 (1959).

Kühn, C. D., Gerlach, C., Andersen, K. B., Poulsen, M. & Starrfelt, R. Face recognition in developmental dyslexia: evidence for dissociation between faces and words. Cogn. Neuropsychol. 38 , 107–115 (2021).

Barton, J. J. S., Albonico, A., Susilo, T., Duchaine, B. & Corrow, S. L. Object recognition in acquired and developmental prosopagnosia. Cogn. Neuropsychol. 36 , 54–84 (2019).

Renault, B., Signoret, J.-L., Debruille, B., Breton, F. & Bolgert, F. Brain potentials reveal covert facial recognition in prosopagnosia. Neuropsychologia 27 , 905–912 (1989).

Bauer, R. M. Autonomic recognition of names and faces in prosopagnosia: a neuropsychological application of the guilty knowledge test. Neuropsychologia 22 , 457–469 (1984).

Haan, E. H. F., de, Young, A. & Newcombe, F. Face recognition without awareness. Cogn. Neuropsychol. 4 , 385–415 (1987).

Ellis, H. D. & Lewis, M. B. Capgras delusion: a window on face recognition. Trends Cogn. Sci. 5 , 149–156 (2001).

Ellis, H. D., Young, A. W., Quayle, A. H. & De Pauw, K. W. Reduced autonomic responses to faces in Capgras delusion. Proc. R. Soc. Lond. B 264 , 1085–1092 (1997).

Collins, M. N., Hawthorne, M. E., Gribbin, N. & Jacobson, R. Capgras’ syndrome with organic disorders. Postgrad. Med. J. 66 , 1064–1067 (1990).

Enoch, D., Puri, B. K. & Ball, H. Uncommon Psychiatric Syndromes 5th edn (Routledge, 2020).

Tranel, D., Damasio, H. & Damasio, A. R. Double dissociation between overt and covert face recognition. J. Cogn. Neurosci. 7 , 425–432 (1995).

Brighetti, G., Bonifacci, P., Borlimi, R. & Ottaviani, C. “Far from the heart far from the eye”: evidence from the Capgras delusion. Cogn. Neuropsychiat. 12 , 189–197 (2007).

Coltheart, M., Langdon, R. & McKay, R. Delusional belief. Annu. Rev. Psychol. 62 , 271–298 (2011).

Coltheart, M. Cognitive neuropsychiatry and delusional belief. Q. J. Exp. Psychol. 60 , 1041–1062 (2007).

Coltheart, M. & Davies, M. How unexpected observations lead to new beliefs: a Peircean pathway. Conscious. Cogn. 87 , 103037 (2021).

Coltheart, M. & Davies, M. Failure of hypothesis evaluation as a factor in delusional belief. Cogn. Neuropsychiat. 26 , 213–230 (2021).

McCloskey, M. et al. A developmental deficit in localizing objects from vision. Psychol. Sci. 6 , 112–117 (1995).

McCloskey, M., Valtonen, J. & Cohen Sherman, J. Representing orientation: a coordinate-system hypothesis and evidence from developmental deficits. Cogn. Neuropsychol. 23 , 680–713 (2006).

McCloskey, M. Spatial representations and multiple-visual-systems hypotheses: evidence from a developmental deficit in visual location and orientation processing. Cortex 40 , 677–694 (2004).

Gregory, E. & McCloskey, M. Mirror-image confusions: implications for representation and processing of object orientation. Cognition 116 , 110–129 (2010).

Gregory, E., Landau, B. & McCloskey, M. Representation of object orientation in children: evidence from mirror-image confusions. Vis. Cogn. 19 , 1035–1062 (2011).

Laine, M. & Martin, N. Cognitive neuropsychology has been, is, and will be significant to aphasiology. Aphasiology 26 , 1362–1376 (2012).

Howard, D. & Patterson, K. The Pyramids And Palm Trees Test: A Test Of Semantic Access From Words And Pictures (Thames Valley Test Co., 1992).

Kay, J., Lesser, R. & Coltheart, M. PALPA: Psycholinguistic Assessments Of Language Processing In Aphasia. 2: Picture & Word Semantics, Sentence Comprehension (Erlbaum, 2001).

Franklin, S. Dissociations in auditory word comprehension; evidence from nine fluent aphasic patients. Aphasiology 3 , 189–207 (1989).

Howard, D., Swinburn, K. & Porter, G. Putting the CAT out: what the comprehensive aphasia test has to offer. Aphasiology 24 , 56–74 (2010).

Conti-Ramsden, G., Crutchley, A. & Botting, N. The extent to which psychometric tests differentiate subgroups of children with SLI. J. Speech Lang. Hear. Res. 40 , 765–777 (1997).

Bishop, D. V. M. & McArthur, G. M. Individual differences in auditory processing in specific language impairment: a follow-up study using event-related potentials and behavioural thresholds. Cortex 41 , 327–341 (2005).

Bishop, D. V. M., Snowling, M. J., Thompson, P. A. & Greenhalgh, T., and the CATALISE-2 consortium. Phase 2 of CATALISE: a multinational and multidisciplinary Delphi consensus study of problems with language development: terminology. J. Child. Psychol. Psychiat. 58 , 1068–1080 (2017).

Wilson, A. J. et al. Principles underlying the design of ‘the number race’, an adaptive computer game for remediation of dyscalculia. Behav. Brain Funct. 2 , 19 (2006).

Basso, A. & Marangolo, P. Cognitive neuropsychological rehabilitation: the emperor’s new clothes? Neuropsychol. Rehabil. 10 , 219–229 (2000).

Murad, M. H., Asi, N., Alsawas, M. & Alahdab, F. New evidence pyramid. Evidence-based Med. 21 , 125–127 (2016).

Greenhalgh, T., Howick, J. & Maskrey, N., for the Evidence Based Medicine Renaissance Group. Evidence based medicine: a movement in crisis? Br. Med. J. 348 , g3725–g3725 (2014).

Best, W., Ping Sze, W., Edmundson, A. & Nickels, L. What counts as evidence? Swimming against the tide: valuing both clinically informed experimentally controlled case series and randomized controlled trials in intervention research. Evidence-based Commun. Assess. Interv. 13 , 107–135 (2019).

Best, W. et al. Understanding differing outcomes from semantic and phonological interventions with children with word-finding difficulties: a group and case series study. Cortex 134 , 145–161 (2021).

OCEBM Levels of Evidence Working Group. The Oxford Levels of Evidence 2. CEBM https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence (2011).

Holler, D. E., Behrmann, M. & Snow, J. C. Real-world size coding of solid objects, but not 2-D or 3-D images, in visual agnosia patients with bilateral ventral lesions. Cortex 119 , 555–568 (2019).

Duchaine, B. C., Yovel, G., Butterworth, E. J. & Nakayama, K. Prosopagnosia as an impairment to face-specific mechanisms: elimination of the alternative hypotheses in a developmental case. Cogn. Neuropsychol. 23 , 714–747 (2006).

Hartley, T. et al. The hippocampus is required for short-term topographical memory in humans. Hippocampus 17 , 34–48 (2007).

Pishnamazi, M. et al. Attentional bias towards and away from fearful faces is modulated by developmental amygdala damage. Cortex 81 , 24–34 (2016).

Rapp, B., Fischer-Baum, S. & Miozzo, M. Modality and morphology: what we write may not be what we say. Psychol. Sci. 26 , 892–902 (2015).

Yong, K. X. X., Warren, J. D., Warrington, E. K. & Crutch, S. J. Intact reading in patients with profound early visual dysfunction. Cortex 49 , 2294–2306 (2013).

Rockland, K. S. & Van Hoesen, G. W. Direct temporal–occipital feedback connections to striate cortex (V1) in the macaque monkey. Cereb. Cortex 4 , 300–313 (1994).

Haynes, J.-D., Driver, J. & Rees, G. Visibility reflects dynamic changes of effective connectivity between V1 and fusiform cortex. Neuron 46 , 811–821 (2005).

Tanaka, K. Mechanisms of visual object recognition: monkey and human studies. Curr. Opin. Neurobiol. 7 , 523–529 (1997).

Fischer-Baum, S., McCloskey, M. & Rapp, B. Representation of letter position in spelling: evidence from acquired dysgraphia. Cognition 115 , 466–490 (2010).

Houghton, G. The problem of serial order: a neural network model of sequence learning and recall. In Current Research In Natural Language Generation (eds Dale, R., Mellish, C. & Zock, M.) 287–319 (Academic Press, 1990).

Fieder, N., Nickels, L., Biedermann, B. & Best, W. From “some butter” to “a butter”: an investigation of mass and count representation and processing. Cogn. Neuropsychol. 31 , 313–349 (2014).

Fieder, N., Nickels, L., Biedermann, B. & Best, W. How ‘some garlic’ becomes ‘a garlic’ or ‘some onion’: mass and count processing in aphasia. Neuropsychologia 75 , 626–645 (2015).

Schröder, A., Burchert, F. & Stadie, N. Training-induced improvement of noncanonical sentence production does not generalize to comprehension: evidence for modality-specific processes. Cogn. Neuropsychol. 32 , 195–220 (2015).

Stadie, N. et al. Unambiguous generalization effects after treatment of non-canonical sentence production in German agrammatism. Brain Lang. 104 , 211–229 (2008).

Schapiro, A. C., Gregory, E., Landau, B., McCloskey, M. & Turk-Browne, N. B. The necessity of the medial temporal lobe for statistical learning. J. Cogn. Neurosci. 26 , 1736–1747 (2014).

Schapiro, A. C., Kustner, L. V. & Turk-Browne, N. B. Shaping of object representations in the human medial temporal lobe based on temporal regularities. Curr. Biol. 22 , 1622–1627 (2012).

Baddeley, A., Vargha-Khadem, F. & Mishkin, M. Preserved recognition in a case of developmental amnesia: implications for the acaquisition of semantic memory? J. Cogn. Neurosci. 13 , 357–369 (2001).

Snyder, J. J. & Chatterjee, A. Spatial-temporal anisometries following right parietal damage. Neuropsychologia 42 , 1703–1708 (2004).

Ashkenazi, S., Henik, A., Ifergane, G. & Shelef, I. Basic numerical processing in left intraparietal sulcus (IPS) acalculia. Cortex 44 , 439–448 (2008).

Lebrun, M.-A., Moreau, P., McNally-Gagnon, A., Mignault Goulet, G. & Peretz, I. Congenital amusia in childhood: a case study. Cortex 48 , 683–688 (2012).

Vannuscorps, G., Andres, M. & Pillon, A. When does action comprehension need motor involvement? Evidence from upper limb aplasia. Cogn. Neuropsychol. 30 , 253–283 (2013).

Jeannerod, M. Neural simulation of action: a unifying mechanism for motor cognition. NeuroImage 14 , S103–S109 (2001).

Blakemore, S.-J. & Decety, J. From the perception of action to the understanding of intention. Nat. Rev. Neurosci. 2 , 561–567 (2001).

Rizzolatti, G. & Craighero, L. The mirror-neuron system. Annu. Rev. Neurosci. 27 , 169–192 (2004).

Forde, E. M. E., Humphreys, G. W. & Remoundou, M. Disordered knowledge of action order in action disorganisation syndrome. Neurocase 10 , 19–28 (2004).

Mazzi, C. & Savazzi, S. The glamor of old-style single-case studies in the neuroimaging era: insights from a patient with hemianopia. Front. Psychol. 10 , 965 (2019).

Coltheart, M. What has functional neuroimaging told us about the mind (so far)? (Position Paper Presented to the European Cognitive Neuropsychology Workshop, Bressanone, 2005). Cortex 42 , 323–331 (2006).

Page, M. P. A. What can’t functional neuroimaging tell the cognitive psychologist? Cortex 42 , 428–443 (2006).

Blank, I. A., Kiran, S. & Fedorenko, E. Can neuroimaging help aphasia researchers? Addressing generalizability, variability, and interpretability. Cogn. Neuropsychol. 34 , 377–393 (2017).

Niv, Y. The primacy of behavioral research for understanding the brain. Behav. Neurosci. 135 , 601–609 (2021).

Crawford, J. R. & Howell, D. C. Comparing an individual’s test score against norms derived from small samples. Clin. Neuropsychol. 12 , 482–486 (1998).

Crawford, J. R., Garthwaite, P. H. & Ryan, K. Comparing a single case to a control sample: testing for neuropsychological deficits and dissociations in the presence of covariates. Cortex 47 , 1166–1178 (2011).

McIntosh, R. D. & Rittmo, J. Ö. Power calculations in single-case neuropsychology: a practical primer. Cortex 135 , 146–158 (2021).

Patterson, K. & Plaut, D. C. “Shallow draughts intoxicate the brain”: lessons from cognitive science for cognitive neuropsychology. Top. Cogn. Sci. 1 , 39–58 (2009).

Lambon Ralph, M. A., Patterson, K. & Plaut, D. C. Finite case series or infinite single-case studies? Comments on “Case series investigations in cognitive neuropsychology” by Schwartz and Dell (2010). Cogn. Neuropsychol. 28 , 466–474 (2011).

Horien, C., Shen, X., Scheinost, D. & Constable, R. T. The individual functional connectome is unique and stable over months to years. NeuroImage 189 , 676–687 (2019).

Epelbaum, S. et al. Pure alexia as a disconnection syndrome: new diffusion imaging evidence for an old concept. Cortex 44 , 962–974 (2008).

Fischer-Baum, S. & Campana, G. Neuroplasticity and the logic of cognitive neuropsychology. Cogn. Neuropsychol. 34 , 403–411 (2017).

Paul, S., Baca, E. & Fischer-Baum, S. Cerebellar contributions to orthographic working memory: a single case cognitive neuropsychological investigation. Neuropsychologia 171 , 108242 (2022).

Feinstein, J. S., Adolphs, R., Damasio, A. & Tranel, D. The human amygdala and the induction and experience of fear. Curr. Biol. 21 , 34–38 (2011).

Crawford, J., Garthwaite, P. & Gray, C. Wanted: fully operational definitions of dissociations in single-case studies. Cortex 39 , 357–370 (2003).

McIntosh, R. D. Simple dissociations for a higher-powered neuropsychology. Cortex 103 , 256–265 (2018).

McIntosh, R. D. & Brooks, J. L. Current tests and trends in single-case neuropsychology. Cortex 47 , 1151–1159 (2011).

Best, W., Schröder, A. & Herbert, R. An investigation of a relative impairment in naming non-living items: theoretical and methodological implications. J. Neurolinguistics 19 , 96–123 (2006).

Franklin, S., Howard, D. & Patterson, K. Abstract word anomia. Cogn. Neuropsychol. 12 , 549–566 (1995).

Coltheart, M., Patterson, K. E. & Marshall, J. C. Deep Dyslexia (Routledge, 1980).

Nickels, L., Kohnen, S. & Biedermann, B. An untapped resource: treatment as a tool for revealing the nature of cognitive processes. Cogn. Neuropsychol. 27 , 539–562 (2010).

Download references

Acknowledgements

The authors thank all of those pioneers of and advocates for single case study research who have mentored, inspired and encouraged us over the years, and the many other colleagues with whom we have discussed these issues.

Author information

Authors and affiliations.

School of Psychological Sciences & Macquarie University Centre for Reading, Macquarie University, Sydney, New South Wales, Australia

Lyndsey Nickels

NHMRC Centre of Research Excellence in Aphasia Recovery and Rehabilitation, Australia

Psychological Sciences, Rice University, Houston, TX, USA

Simon Fischer-Baum

Psychology and Language Sciences, University College London, London, UK

You can also search for this author in PubMed   Google Scholar

Contributions

L.N. led and was primarily responsible for the structuring and writing of the manuscript. All authors contributed to all aspects of the article.

Corresponding author

Correspondence to Lyndsey Nickels .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Reviews Psychology thanks Yanchao Bi, Rob McIntosh, and the other, anonymous, reviewer for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Nickels, L., Fischer-Baum, S. & Best, W. Single case studies are a powerful tool for developing, testing and extending theories. Nat Rev Psychol 1 , 733–747 (2022). https://doi.org/10.1038/s44159-022-00127-y

Download citation

Accepted : 13 October 2022

Published : 22 November 2022

Issue Date : December 2022

DOI : https://doi.org/10.1038/s44159-022-00127-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

  • Explore articles by subject
  • Guide to authors
  • Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

single case study of intervention

  • Subject List
  • Take a Tour
  • For Authors
  • Subscriber Services
  • Publications
  • African American Studies
  • African Studies
  • American Literature
  • Anthropology
  • Architecture Planning and Preservation
  • Art History
  • Atlantic History
  • Biblical Studies
  • British and Irish Literature
  • Childhood Studies
  • Chinese Studies
  • Cinema and Media Studies
  • Communication
  • Criminology
  • Environmental Science
  • Evolutionary Biology
  • International Law
  • International Relations
  • Islamic Studies
  • Jewish Studies
  • Latin American Studies
  • Latino Studies
  • Linguistics
  • Literary and Critical Theory
  • Medieval Studies
  • Military History
  • Political Science
  • Public Health
  • Renaissance and Reformation
  • Social Work
  • Urban Studies
  • Victorian Literature
  • Browse All Subjects

How to Subscribe

  • Free Trials

In This Article Expand or collapse the "in this article" section Single-Case Experimental Designs

Introduction, general overviews and primary textbooks.

  • Textbooks in Applied Behavior Analysis
  • Types of Single-Case Experimental Designs
  • Model Building and Randomization in Single-Case Experimental Designs
  • Visual Analysis of Single-Case Experimental Designs
  • Effect Size Estimates in Single-Case Experimental Designs
  • Reporting Single-Case Design Intervention Research

Related Articles Expand or collapse the "related articles" section about

About related articles close popup.

Lorem Ipsum Sit Dolor Amet

Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Aliquam ligula odio, euismod ut aliquam et, vestibulum nec risus. Nulla viverra, arcu et iaculis consequat, justo diam ornare tellus, semper ultrices tellus nunc eu tellus.

  • Action Research
  • Ambulatory Assessment in Behavioral Science
  • Effect Size
  • Mediation Analysis
  • Path Models
  • Research Methods for Studying Daily Life

Other Subject Areas

Forthcoming articles expand or collapse the "forthcoming articles" section.

  • Data Visualization
  • Melanie Klein
  • Remote Work
  • Find more forthcoming articles...
  • Export Citations
  • Share This Facebook LinkedIn Twitter

Single-Case Experimental Designs by S. Andrew Garbacz , Thomas R. Kratochwill LAST MODIFIED: 29 July 2020 DOI: 10.1093/obo/9780199828340-0265

Single-case experimental designs are a family of experimental designs that are characterized by researcher manipulation of an independent variable and repeated measurement of a dependent variable before (i.e., baseline) and after (i.e., intervention phase) introducing the independent variable. In single-case experimental designs a case is the unit of intervention and analysis (e.g., a child, a school). Because measurement within each case is conducted before and after manipulation of the independent variable, the case typically serves as its own control. Experimental variants of single-case designs provide a basis for determining a causal relation by replication of the intervention through (a) introducing and withdrawing the independent variable, (b) manipulating the independent variable across different phases, and (c) introducing the independent variable in a staggered fashion across different points in time. Due to their economy of resources, single-case designs may be useful during development activities and allow for rapid replication across studies.

Several sources provide overviews of single-case experimental designs. Barlow, et al. 2009 includes an overview for the development of single-case experimental designs, describes key considerations for designing and conducting single-case experimental design research, and reviews procedural elements, assessment strategies, and replication considerations. Kazdin 2011 provides detailed coverage of single-case experimental design variants as well as approaches for evaluating data in single-case experimental designs. Kratochwill and Levin 2014 describes key methodological features that underlie single-case experimental designs, including philosophical and statistical foundations and data evaluation. Ledford and Gast 2018 covers research conceptualization and writing, design variants within single-case experimental design, definitions of variables and associated measurement, and approaches to organize and evaluate data. Riley-Tillman and Burns 2009 provides a practical orientation to single-case experimental designs to facilitate uptake and use in applied settings.

Barlow, D. H., M. K. Nock, and M. Hersen, eds. 2009. Single case experimental designs: Strategies for studying behavior change . 3d ed. New York: Pearson.

A comprehensive reference about the process of designing and conducting single-case experimental design studies. Chapters are integrative but can stand alone.

Kazdin, A. E. 2011. Single-case research designs: Methods for clinical and applied settings . 2d ed. New York: Oxford Univ. Press.

A complete overview and description of single-case experimental design variants as well as information about data evaluation.

Kratochwill, T. R., and J. R. Levin, eds. 2014. Single-case intervention research: Methodological and statistical advances . New York: Routledge.

The authors describe in depth the methodological and analytic considerations necessary for designing and conducting research that uses a single-case experimental design. In addition, the text includes chapters from leaders in psychology and education who provide critical perspectives about the use of single-case experimental designs.

Ledford, J. R., and D. L. Gast, eds. 2018. Single case research methodology: Applications in special education and behavioral sciences . New York: Routledge.

Covers the research process from writing literature reviews, to designing, conducting, and evaluating single-case experimental design studies.

Riley-Tillman, T. C., and M. K. Burns. 2009. Evaluating education interventions: Single-case design for measuring response to intervention . New York: Guilford Press.

Focuses on accelerating uptake and use of single-case experimental designs in applied settings. This book provides a practical, “nuts and bolts” orientation to conducting single-case experimental design research.

back to top

Users without a subscription are not able to see the full content on this page. Please subscribe or login .

Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here .

  • About Psychology »
  • Meet the Editorial Board »
  • Abnormal Psychology
  • Academic Assessment
  • Acculturation and Health
  • Action Regulation Theory
  • Addictive Behavior
  • Adolescence
  • Adoption, Social, Psychological, and Evolutionary Perspect...
  • Advanced Theory of Mind
  • Affective Forecasting
  • Affirmative Action
  • Ageism at Work
  • Allport, Gordon
  • Alzheimer’s Disease
  • Analysis of Covariance (ANCOVA)
  • Animal Behavior
  • Animal Learning
  • Anxiety Disorders
  • Art and Aesthetics, Psychology of
  • Artificial Intelligence, Machine Learning, and Psychology
  • Assessment and Clinical Applications of Individual Differe...
  • Attachment in Social and Emotional Development across the ...
  • Attention-Deficit/Hyperactivity Disorder (ADHD) in Adults
  • Attention-Deficit/Hyperactivity Disorder (ADHD) in Childre...
  • Attitudinal Ambivalence
  • Attraction in Close Relationships
  • Attribution Theory
  • Authoritarian Personality
  • Bayesian Statistical Methods in Psychology
  • Behavior Therapy, Rational Emotive
  • Behavioral Economics
  • Behavioral Genetics
  • Belief Perseverance
  • Bereavement and Grief
  • Biological Psychology
  • Birth Order
  • Body Image in Men and Women
  • Bystander Effect
  • Categorical Data Analysis in Psychology
  • Childhood and Adolescence, Peer Victimization and Bullying...
  • Clark, Mamie Phipps
  • Clinical Neuropsychology
  • Clinical Psychology
  • Cognitive Consistency Theories
  • Cognitive Dissonance Theory
  • Cognitive Neuroscience
  • Communication, Nonverbal Cues and
  • Comparative Psychology
  • Competence to Stand Trial: Restoration Services
  • Competency to Stand Trial
  • Computational Psychology
  • Conflict Management in the Workplace
  • Conformity, Compliance, and Obedience
  • Consciousness
  • Coping Processes
  • Correspondence Analysis in Psychology
  • Counseling Psychology
  • Creativity at Work
  • Critical Thinking
  • Cross-Cultural Psychology
  • Cultural Psychology
  • Daily Life, Research Methods for Studying
  • Data Science Methods for Psychology
  • Data Sharing in Psychology
  • Death and Dying
  • Deceiving and Detecting Deceit
  • Defensive Processes
  • Depressive Disorders
  • Development, Prenatal
  • Developmental Psychology (Cognitive)
  • Developmental Psychology (Social)
  • Diagnostic and Statistical Manual of Mental Disorders (DSM...
  • Discrimination
  • Dissociative Disorders
  • Drugs and Behavior
  • Eating Disorders
  • Ecological Psychology
  • Educational Settings, Assessment of Thinking in
  • Embodiment and Embodied Cognition
  • Emerging Adulthood
  • Emotional Intelligence
  • Empathy and Altruism
  • Employee Stress and Well-Being
  • Environmental Neuroscience and Environmental Psychology
  • Ethics in Psychological Practice
  • Event Perception
  • Evolutionary Psychology
  • Expansive Posture
  • Experimental Existential Psychology
  • Exploratory Data Analysis
  • Eyewitness Testimony
  • Eysenck, Hans
  • Factor Analysis
  • Festinger, Leon
  • Five-Factor Model of Personality
  • Flynn Effect, The
  • Forensic Psychology
  • Forgiveness
  • Friendships, Children's
  • Fundamental Attribution Error/Correspondence Bias
  • Gambler's Fallacy
  • Game Theory and Psychology
  • Geropsychology, Clinical
  • Global Mental Health
  • Habit Formation and Behavior Change
  • Health Psychology
  • Health Psychology Research and Practice, Measurement in
  • Heider, Fritz
  • Heuristics and Biases
  • History of Psychology
  • Human Factors
  • Humanistic Psychology
  • Implicit Association Test (IAT)
  • Industrial and Organizational Psychology
  • Inferential Statistics in Psychology
  • Insanity Defense, The
  • Intelligence
  • Intelligence, Crystallized and Fluid
  • Intercultural Psychology
  • Intergroup Conflict
  • International Classification of Diseases and Related Healt...
  • International Psychology
  • Interviewing in Forensic Settings
  • Intimate Partner Violence, Psychological Perspectives on
  • Introversion–Extraversion
  • Item Response Theory
  • Law, Psychology and
  • Lazarus, Richard
  • Learned Helplessness
  • Learning Theory
  • Learning versus Performance
  • LGBTQ+ Romantic Relationships
  • Lie Detection in a Forensic Context
  • Life-Span Development
  • Locus of Control
  • Loneliness and Health
  • Mathematical Psychology
  • Meaning in Life
  • Mechanisms and Processes of Peer Contagion
  • Media Violence, Psychological Perspectives on
  • Memories, Autobiographical
  • Memories, Flashbulb
  • Memories, Repressed and Recovered
  • Memory, False
  • Memory, Human
  • Memory, Implicit versus Explicit
  • Memory in Educational Settings
  • Memory, Semantic
  • Meta-Analysis
  • Metacognition
  • Metaphor, Psychological Perspectives on
  • Microaggressions
  • Military Psychology
  • Mindfulness
  • Mindfulness and Education
  • Minnesota Multiphasic Personality Inventory (MMPI)
  • Money, Psychology of
  • Moral Conviction
  • Moral Development
  • Moral Psychology
  • Moral Reasoning
  • Nature versus Nurture Debate in Psychology
  • Neuroscience of Associative Learning
  • Nonergodicity in Psychology and Neuroscience
  • Nonparametric Statistical Analysis in Psychology
  • Observational (Non-Randomized) Studies
  • Obsessive-Complusive Disorder (OCD)
  • Occupational Health Psychology
  • Olfaction, Human
  • Operant Conditioning
  • Optimism and Pessimism
  • Organizational Justice
  • Parenting Stress
  • Parenting Styles
  • Parents' Beliefs about Children
  • Peace Psychology
  • Perception, Person
  • Performance Appraisal
  • Personality and Health
  • Personality Disorders
  • Personality Psychology
  • Phenomenological Psychology
  • Placebo Effects in Psychology
  • Play Behavior
  • Positive Psychological Capital (PsyCap)
  • Positive Psychology
  • Posttraumatic Stress Disorder (PTSD)
  • Prejudice and Stereotyping
  • Pretrial Publicity
  • Prisoner's Dilemma
  • Problem Solving and Decision Making
  • Procrastination
  • Prosocial Behavior
  • Prosocial Spending and Well-Being
  • Protocol Analysis
  • Psycholinguistics
  • Psychological Literacy
  • Psychological Perspectives on Food and Eating
  • Psychology, Political
  • Psychoneuroimmunology
  • Psychophysics, Visual
  • Psychotherapy
  • Psychotic Disorders
  • Publication Bias in Psychology
  • Reasoning, Counterfactual
  • Rehabilitation Psychology
  • Relationships
  • Reliability–Contemporary Psychometric Conceptions
  • Religion, Psychology and
  • Replication Initiatives in Psychology
  • Research Methods
  • Risk Taking
  • Role of the Expert Witness in Forensic Psychology, The
  • Sample Size Planning for Statistical Power and Accurate Es...
  • Schizophrenic Disorders
  • School Psychology
  • School Psychology, Counseling Services in
  • Self, Gender and
  • Self, Psychology of the
  • Self-Construal
  • Self-Control
  • Self-Deception
  • Self-Determination Theory
  • Self-Efficacy
  • Self-Esteem
  • Self-Monitoring
  • Self-Regulation in Educational Settings
  • Self-Report Tests, Measures, and Inventories in Clinical P...
  • Sensation Seeking
  • Sex and Gender
  • Sexual Minority Parenting
  • Sexual Orientation
  • Signal Detection Theory and its Applications
  • Simpson's Paradox in Psychology
  • Single People
  • Single-Case Experimental Designs
  • Skinner, B.F.
  • Sleep and Dreaming
  • Small Groups
  • Social Class and Social Status
  • Social Cognition
  • Social Neuroscience
  • Social Support
  • Social Touch and Massage Therapy Research
  • Somatoform Disorders
  • Spatial Attention
  • Sports Psychology
  • Stanford Prison Experiment (SPE): Icon and Controversy
  • Stereotype Threat
  • Stereotypes
  • Stress and Coping, Psychology of
  • Student Success in College
  • Subjective Wellbeing Homeostasis
  • Taste, Psychological Perspectives on
  • Teaching of Psychology
  • Terror Management Theory
  • Testing and Assessment
  • The Concept of Validity in Psychological Assessment
  • The Neuroscience of Emotion Regulation
  • The Reasoned Action Approach and the Theories of Reasoned ...
  • The Weapon Focus Effect in Eyewitness Memory
  • Theory of Mind
  • Therapies, Person-Centered
  • Therapy, Cognitive-Behavioral
  • Thinking Skills in Educational Settings
  • Time Perception
  • Trait Perspective
  • Trauma Psychology
  • Twin Studies
  • Type A Behavior Pattern (Coronary Prone Personality)
  • Unconscious Processes
  • Video Games and Violent Content
  • Virtues and Character Strengths
  • Women and Science, Technology, Engineering, and Math (STEM...
  • Women, Psychology of
  • Work Well-Being
  • Wundt, Wilhelm
  • Privacy Policy
  • Cookie Policy
  • Legal Notice
  • Accessibility

Powered by:

  • [66.249.64.20|185.80.149.115]
  • 185.80.149.115
  • Search Menu
  • Animal Research
  • Cardiovascular/Pulmonary
  • Health Services
  • Health Policy
  • Health Promotion
  • History of Physical Therapy
  • Implementation Science
  • Integumentary
  • Musculoskeletal
  • Orthopedics
  • Pain Management
  • Pelvic Health
  • Pharmacology
  • Population Health
  • Professional Issues
  • Psychosocial
  • Advance Articles
  • COVID-19 Collection
  • Featured Collections
  • Special Issues
  • Author Guidelines
  • Submission Site
  • Why Publish With PTJ?
  • Open Access
  • Call for Papers
  • Self-Archiving Policy
  • Promote your Article
  • About Physical Therapy
  • Editorial Board
  • Advertising & Corporate Services
  • Permissions
  • Journals on Oxford Academic
  • Books on Oxford Academic

Issue Cover

Article Contents

Initial steps, premeeting activities, consensus meeting, postmeeting activities, postpublication activities, conclusions, the single-case reporting guideline in behavioural interventions (scribe) 2016 statement.

  • Article contents
  • Figures & tables
  • Supplementary Data

Robyn L. Tate, Michael Perdices, Ulrike Rosenkoetter, William Shadish, Sunita Vohra, David H. Barlow, Robert Horner, Alan Kazdin, Thomas Kratochwill, Skye McDonald, Margaret Sampson, Larissa Shamseer, Leanne Togher, Richard Albin, Catherine Backman, Jacinta Douglas, Jonathan J. Evans, David Gast, Rumen Manolov, Geoffrey Mitchell, Lyndsey Nickels, Jane Nikles, Tamara Ownsworth, Miranda Rose, Christopher H. Schmid, Barbara Wilson, The Single-Case Reporting Guideline In BEhavioural Interventions (SCRIBE) 2016 Statement, Physical Therapy , Volume 96, Issue 7, 1 July 2016, Pages e1–e10, https://doi.org/10.2522/ptj.2016.96.7.e1

  • Permissions Icon Permissions

We developed a reporting guideline to provide authors with guidance about what should be reported when writing a paper for publication in a scientific journal using a particular type of research design: the single-case experimental design. This report describes the methods used to develop the Single-Case Reporting guideline In BEhavioural interventions (SCRIBE) 2016. As a result of 2 online surveys and a 2-day meeting of experts, the SCRIBE 2016 checklist was developed, which is a set of 26 items that authors need to address when writing about single-case research. This article complements the more detailed SCRIBE 2016 Explanation and Elaboration article ( Tate et al., 2016 ) that provides a rationale for each of the items and examples of adequate reporting from the literature. Both these resources will assist authors to prepare reports of single-case research with clarity, completeness, accuracy, and transparency. They will also provide journal reviewers and editors with a practical checklist against which such reports may be critically evaluated. We recommend that the SCRIBE 2016 is used by authors preparing manuscripts describing single-case research for publication, as well as journal reviewers and editors who are evaluating such manuscripts.

Reporting guidelines, such as the Consolidated Standards of Reporting Trials (CONSORT) Statement, improve the reporting of research in the medical literature ( Turner et al., 2012 ). Many such guidelines exist and the CONSORT Extension to Nonpharmacological Trials ( Boutron et al., 2008 ) provides suitable guidance for reporting between-groups intervention studies in the behavioral sciences. The CONSORT Extension for N -of-1 Trials (CENT 2015) was developed for multiple crossover trials with single individuals in the medical sciences ( Shamseer et al., 2015 ; Vohra et al., 2015 ), but there is no reporting guideline in the CONSORT tradition for single-case research used in the behavioral sciences. We developed the Single-Case Reporting guideline In BEhavioural interventions (SCRIBE) 2016 to meet this need. This Statement article describes the methodology of the development of the SCRIBE 2016, along with the outcome of 2 Delphi surveys and a consensus meeting of experts. We present the resulting 26-item SCRIBE 2016 checklist. The article complements the more detailed SCRIBE 2016 Explanation and Elaboration article ( Tate et al., 2016 ) that provides a rationale for each of the items and examples of adequate reporting from the literature. Both these resources will assist authors to prepare reports of single-case research with clarity, completeness, accuracy, and transparency. They will also provide journal reviewers and editors with a practical checklist against which such reports may be critically evaluated.

Keywords: single-case design, methodology, reporting guidelines, publication standards

Supplemental materials: http://dx.doi.org/10.1037/arc0000026.supp

University courses generally prepare students of the behavioral sciences very well for research using parallel, between-groups designs. By contrast, single-case methodology is “rarely taught in undergraduate, graduate and postdoctoral training” ( Kazdin, 2011 , p. vii). Consequently, there is a risk that researchers conducting and publishing studies using single-case experimental designs (and journal reviewers of such studies) are not necessarily knowledgeable about single-case methodology nor well trained in using such designs in applied settings. This circumstance, in turn, impacts the conduct and report of single-case research. Even though single-case experimental intervention research has comparable frequency to between-groups research in the aphasiology, education, psychology, and neurorehabilitation literature ( Beeson & Robey, 2006 ; Perdices & Tate, 2009 ; Shadish & Sullivan, 2011 ), evidence of inadequate and incomplete reporting is documented in multiple surveys of this literature in different populations ( Barker et al., 2013 ; Didden et al., 2006 ; Maggin et al., 2011 ; Smith, 2012 ; Tate et al., 2014 ).

To address these issues we developed a reporting guideline, entitled the Single-Case Reporting guideline In BEhavioural interventions (SCRIBE) 2016, to assist authors, journal reviewers and editors to improve the reporting of single-case research. This Statement provides the methodology and development of the SCRIBE 2016. The companion SCRIBE 2016 Explanation and Elaboration (E&E) article ( Tate et al., 2016 ) provides detailed background to and rationale for each of the 26 items in the SCRIBE checklist, along with examples of adequate reporting in the published literature.

The SCRIBE 2016 Statement is intended for use with the family of single-case experimental designs 1 used in the behavioral sciences. It applies to four prototypical designs (withdrawal/reversal, multiple-baseline, alternating-treatments, and changing-criterion designs), including combinations and variants of these designs, as well as adaptive designs. Figure 1 presents the common designs using a single case based on surveys in the literature (see, e.g., Perdices & Tate, 2009 ; Shadish & Sullivan, 2011 ).

Common designs in the literature using a single participant. Reproduced from the expanded manual for the Risk of Bias in N-of-1 Trials (RoBiNT) Scale (Tate et al., 2015) with permission of the authors; an earlier version of the figure, taken from the original RoBiNT Scale manual (Tate et al., 2013a) was also published in 2013 (Tate et al., 2013b).

Common designs in the literature using a single participant. Reproduced from the expanded manual for the Risk of Bias in N -of-1 Trials (RoBiNT) Scale ( Tate et al., 2015 ) with permission of the authors; an earlier version of the figure, taken from the original RoBiNT Scale manual ( Tate et al., 2013a ) was also published in 2013 ( Tate et al., 2013b ).

The figure mainly draws on the behavioral sciences literature, which includes a broad range of designs using a single participant. Only those designs above the solid horizontal line use single-case methodology (i.e., an intervention is systematically manipulated across multiple phases during each of which the dependent variable is measured repeatedly and, ideally, frequently). None of the designs below the solid horizontal line meets these criteria and they are not considered single-case experiments: The B-phase training study comprises only a single (intervention) phase; the so-called “pre–post” study does not take repeated measurements during the intervention phase; and the case description is a report, usually compiled retrospectively, that is purely descriptive without systematic manipulation of an intervention.

The A-B design, also labeled “phase change without reversal” ( Shadish & Sullivan, 2011 ), is widely regarded as the basic single-case design. It differs from the “pre–post” study in that measurement of the dependent variable occurs during the intervention (B) phase. In the Figure , we place the A-B design in an intermediate position between the nonexperimental single-case designs (below the solid horizontal line) and the four experimental designs above the dotted horizontal line because it has weak internal validity, there being no control for history or maturation, among other variables. As a result, it is regarded as a quasiexperimental design ( Barlow et al., 2009 ).

Designs above the dotted horizontal line are experimental in that the control of threats to internal validity is stronger than in the A-B design. Nonetheless, within each class of design the adequacy of such controls and whether or not the degree of experimental control meets design standards (see Horner et al., 2005 ; Kratochwill et al., 2013 ) vary considerably (cf. A-B-A vs. A-B-A-B; multiple-baseline designs with two vs. three baselines/tiers). Consequently, reports of these designs in the literature have variable scientific quality and features of internal and external validity can be evaluated with scales measuring scientific robustness in single-case designs, such as described in Maggin et al. (2014) and Tate et al. (2013b) .

The structure of the four prototypical experimental designs in Figure 1 differ significantly: The withdrawal/reversal design systematically applies and withdraws an intervention in a sequential manner, the multiple-baseline design systematically applies an intervention in a sequential manner that also has a staggered introduction across a particular parameter (e.g., participants, behaviors), the alternating/simultaneous-treatments design compares multiple interventions in a concurrent manner by rapidly alternating the application of the interventions, and the changing-criterion design establishes a number of hierarchically based criterion levels that are implemented in a sequential manner. Each of the single-case experimental designs has the capacity to introduce randomization into the design (cf. the small gray rectangle within each of the designs in Figure 1 ), although in practice randomization in single-case research is not common.

The medical N -of-1 trial is depicted within the withdrawal/reversal paradigm of Figure 1 . The analogous reporting guide for the medical sciences, CONSORT Extension for N -of-1 Trials (CENT 2015; Shamseer et al., 2015 ; Vohra et al., 2015 ), is available for the reporting of medical N -of-1 trials. These trials consist of multiple cross-overs (described as challenge-withdrawal-challenge-withdrawal in Vohra et al.) in a single participant who serves as his or her own control, often incorporating randomization and blinding.

As with other reporting guidelines in the CONSORT tradition, the SCRIBE 2016 does not make recommendations about how to design, conduct or analyze data from single-case experiments. Rather, its primary purpose is to provide authors with a checklist of items that a consensus from experts identified as the minimum standard for facilitating comprehensive and transparent reporting. This checklist includes the specific aspects of the methodology to be reported and suggestions about how to report. Consequently, readers are provided with a clear, complete, accurate, and transparent account of the context, plan, implementation and outcomes of a study. Readers will then be in a position to critically evaluate the adequacy of the study, as well as to replicate and validate the research. Clinicians and researchers who want guidance on how to design, conduct and analyze data for single-case experiments should consult any of the many current textbooks and reports (e.g., Barker et al., 2011 ; Barlow, Nock, & Hersen, 2009 ; Gast & Ledford, 2014 ; Horner et al., 2005 ; Kazdin, 2011 ; Kennedy, 2005 ; Kratochwill et al., 2013 ; Kratochwill & Levin, 2014 ; Morgan & Morgan, 2009 ; Riley-Tilman & Burns, 2009 ; Vannest, Davis, & Parker, 2013 ), as well as recent special issues of journals (e.g., Journal of Behavioral Education in 2012, Remedial and Special Education in 2013, the Journal of School Psychology and Neuropsychological Rehabilitation in 2014, Aphasiology in 2015) and methodological quality recommendations ( Horner et al., 2005 ; Kratochwill et al., 2013 ; Maggin et al., 2014 ; Smith, 2012 ; Tate et al., 2013b ).

The impetus to develop the SCRIBE 2016 arose during the course of discussion at the CENT consensus meeting in May 2009 in Alberta, Canada (see Shamseer et al., 2015 ; Vohra et al., 2015 ). The CENT initiative was devoted to developing a reporting guideline for a specific design and a specific discipline: N -of-1 trials in the medical sciences. At that meeting the need was identified for development of a separate reporting guideline for the broader family of single-case experimental designs as used in the behavioral sciences (see Figure 1 ).

A 13-member steering committee for the SCRIBE project was formed comprising a Sydney, Australia, executive (authors RLT, convenor, and SM, MP, LT, with UR appointed as project manager). An additional three members who had spearheaded the CENT initiative (CENT convenor, SV, along with MS and LS) were invited because of their experience and expertise in developing a CONSORT-type reporting guideline in a closely related field ( N -of-1 trials). In order to ensure representation from experts in areas of single-case investigations in clinical psychology, special education and single-case methodology and data analysis, another five experts were invited to the steering committee (authors DHB, RH, AK, TK, and WS). Of course, other content experts exist who would have been eligible for the steering committee, but a guiding consideration was to keep the number of members to a reasonable size so that the project was manageable. In the early stages of the project, steering committee members were instrumental in item development and refinement for the Delphi survey.

The methodology used to develop the SCRIBE 2016 followed the procedures outlined by Moher et al. (2010) . At the time of project commencement, the literature on evidence of bias in reporting single-case research was very limited and it has only recently started to emerge. Members of the steering committee, however, were already knowledgeable about the quality of the existing single-case literature, which had prompted independent work in the United States (specifically in compiling competency standards of design and evidence; Hitchcock et al., 2014 ; Horner et al., 2005 ; Kratochwill et al., 2010 , 2013 ) and Australia (in developing an instrument to evaluate the scientific quality of single-case experiments; Tate et al., 2008 , 2013b ). No reporting guideline, in the CONSORT tradition, emerged from literature review.

Since commencement of the SCRIBE project, a reporting guide for single-case experimental designs was published by Wolery, Dunlap, and Ledford (2011) . That guide was not developed following the same series of steps as in previously developed reporting guidelines such as those of the CONSORT family (see Moher et al., 2011 ) and is not as comprehensive as the CONSORT-type guidelines on which the current project is based, covering about half of the items in the SCRIBE 2016. Nevertheless, the convergence between the recommendations of Wolery and colleagues regarding the need to report on features such as inclusion and exclusion criteria for participants, design rationale, operational definitions of the target behavior versus the corresponding items presented in the SCRIBE 2016 is noteworthy and adds validity to the SCRIBE 2016. Funding for the SCRIBE project was obtained from the Lifetime Care and Support Authority of New South Wales, Australia. The funds were used to employ the project manager, set up and develop a web-based survey, hold a consensus meeting, and sponsor participants to attend the consensus meeting.

Methodology of the Delphi Process

The Delphi technique is a group decision-making tool and consensus procedure that is well suited to establishing expert consensus on a given set of items ( Brewer, 2007 ). The nature of the process allows for it to be conducted online, and responses can be given anonymously. The Delphi procedure consists of several steps, beginning with the identification, selection, and invitation of a panel of experts in the pertinent field to participate in the consensus process. Subsequently, the items are distributed to experts who rate the importance of each topic contained in the items. As we did for the present project, a Likert scale is often used, ranging from 1 to 10, whereby 1 indicates very low importance and 10 very high importance . All expert feedback is then collated and reported back to the panel, including the mean, standard deviation, and median for each item, a graph indicating the distribution of responses, as well as any comments made by other experts to inform further decision-making. When high consensus is achieved, which may take several rounds, the Delphi exercise is completed. Von der Gracht (2012) reviews a number of methods to determine consensus for the Delphi procedure. Methods include using the interquartile range (IQR), with consensus operationalized as no more than 2 units on a 10-unit scale.

The SCRIBE Delphi Procedure

A set of potential items was drawn up by the SCRIBE steering committee for the Delphi survey. The items initially came from two sources available at the time: (a) those identified in a systematic review previously conducted by the CENT group ( Punja et al., in press ), and subsequently refined during the CENT consensus meeting process, and (b) items used to develop the Single-Case Experimental Design Scale published by the Sydney-based members as part of an independent project ( Tate et al., 2008 ). Steering committee members suggested additional items, as well as rephrasing of existing items. We formatted the resulting 44 initial items for distribution in the Delphi exercise, using an online survey tool, SurveyMonkey.

Two rounds of a Delphi survey were conducted in April and September 2011. Figure 2 provides a flow diagram of the Delphi survey participants. In total, we identified 131 experts worldwide as potential Delphi panel members (128 for the initial round and an additional three participants were added at Round 2) based on their track record of published work in the field of single-case research (either methodologically or empirically based) and/or reporting guideline development. We used several strategies to identify suitable respondents. The Sydney executive drew up lists of authors who published single-case experimental designs in the behavioral sciences, by consulting reference lists of books and journal articles and our PsycBITE database ( www.psycbite.com ). We examined the quality of authors' work, as described in their reports, using our methodological quality scale ( Tate et al., 2008 ), and invited authors of scientifically sound reports. In addition, we conducted Google searches of editorial board members of journals that were known to publish single-case reports, as well as the authors publishing in such journals and evaluated the quality of their work. Finally, steering committee members made recommendations of suitable authors. This group of 131 invitees represents a sample of all world experts. We distributed invitations by e-mail for ease of communication and speed of contact. An “opt-in” consent arrangement was used and thus consent to participate required the invitee's active response. Of the pool of 128 invitations for Round 1, 54 did not respond to the invitation (we sent one reminder e-mail), eight did respond but declined (mainly on the grounds of not having sufficient time), and four e-mail addresses were undeliverable. The remaining 62 responders who consented to participate in Round 1 were sent the survey link.

Flow diagram of the Delphi surveys.

Flow diagram of the Delphi surveys.

In Round 1, 53 of 62 consenting experts responded within the 2-week time frame of the survey, with 50 providing a complete data set of responses to the original set of 44 items. Results were entered into a database. Importance ratings of the items were uniformly high, with no item receiving a group median rating <7/10. The items thus remained unrevised for Round 2, which was conducted to elicit additional comment on the items. These decision-making criteria are compatible with that used in the development of the CENT 2015, which excluded items with mean importance ratings <5/10 ( Vohra et al., 2015 ).

For Round 2, the survey link was sent to 59 of the original 62 consenting participants to Round 1 (the three participants who consented but did not complete Round 1 did not provide reasons for their early discontinuance and were not recontacted), and an additional three experts recommended by steering committee members. Graphed results were provided to respondents, along with anonymous comments on the items from the other panel members. A complete data set of responses for Round 2 was collected from 45 participants. Again, the ratings of importance for each item were mostly very high, all items having median importance ratings of at least 8/10, but the range of responses decreased. According to the criteria of von der Gracht (2012) consensus was achieved for 82% of items (36/44) which had IQRs of 2 or less on the 10-point scale. The remaining eight items had IQRs from 2.25 to 4 and were discussed in detail at the consensus meeting.

As depicted in Figure 2 , across the two rounds of the Delphi exercise 65/131 invited experts consented to participate (62 participants in Round 1 and an additional three participants in Round 2). Forty participants provided a complete data set of responses to both Round 1 and Round 2, representing a 62% response rate (40/65). The 40 responders represented 31% of the total of 131 experts invited to participate in the survey.

Sixteen world experts in single-case methodology and reporting guideline development attended a 2-day consensus meeting, along with the Sydney executive and two research staff. Representation included clinical-research content experts in clinical and neuropsychology, educational psychology and special education, medicine, occupational therapy, and speech pathology; as well as single-case methodologists and statisticians; journal editors and a medical librarian; and guideline developers. Delegates met in Sydney on December 8 and 9, 2011. Each participant received a folder which contained reference material pertinent to the SCRIBE project, and results from both rounds of the Delphi survey. Each of the Delphi items contained a graph of the distribution of scores, the mean and median scores of each round of the survey, along with the delegate's own scores when s/he completed the Delphi surveys.

The meeting commenced with a series of brief presentations from steering committee members on the topics of reporting guideline development, single-case methods and terminology, evolution of the SCRIBE project, and description of the CENT. Results of the Delphi survey were then presented. Delegates had their folder of materials to consult and a PowerPoint presentation that projected onto a screen to facilitate discussion. A primary aim of the consensus meeting was to develop the final set of items for the SCRIBE checklist. The final stages of the meeting discussed the documents to be published, authorship, and knowledge dissemination strategy.

During the meeting the 44 Delphi items were discussed, item by item, over the course of four sessions, each led by two facilitators. The guiding principles for discussion were twofold. First, item content was scrutinized to ensure that (a) it captured the essence of the intended issue under consideration and (b) the scope of the item covered the necessary and sufficient information to be reported. Second, the relevance of the item was examined in terms of its capacity to ensure clarity and accuracy of reporting.

Three delegates at the consensus meeting (authors RLT and SM, and a research staff member, DW) took notes about the amalgamation and merging of items where applicable and refinements to wording of items. Final wording of items was typed, live-time, into a computer that projected onto a screen so that delegates could see the changes, engage in further discussion, give approval, and commit to the group decision. In addition, the meeting was audiotaped for the purpose of later transcription to have a record of the discussion of the items and inform the direction and points to describe in the E&E document.

Figure 3 illustrates the discussion process that occurred during the consensus meeting. The figure presents a screen-shot of the PowerPoint presentation of one of the items (Item 31 of the Delphi survey, Treatment Fidelity, which was broadened to encompass procedural fidelity as a result of discussion at the consensus meeting, and became item 17 of the SCRIBE). The figure shows the results of each round of the Delphi survey (the results for Round 1 and Round 2 appear in the Figure as the left- and right-sided graphs respectively), along with discussion points. These points comprised comments made by the Delphi survey participants when completing the online surveys, as well as suggestions prepared by the Sydney executive that emerged from the consolidated comments. The points were used to stimulate discussion among the conference delegates, but discussion was not restricted to the prepared points.

Screen-shot of a discussion item at the consensus meeting.

Screen-shot of a discussion item at the consensus meeting.

By the end of the meeting, delegates reached consensus on endorsing 26 items that thus constitute the minimum set of reporting items comprising the SCRIBE 2016 checklist. The SCRIBE 2016 checklist consists of six sections in which the 26 aspects of report writing pertinent to single-case methodology are addressed. The first two sections focus on the title/abstract and introduction, each section containing two items. Section 3, method, consists of 14 items addressing various aspects of study methodology and procedure. Items include description of the design (e.g., randomization, blinding, planned replication), participants, setting, ethics approval, measures and materials (including the types of measures, their frequency of measurement, and demonstration of their reliability), interventions, and proposed analyses. The results (Section 4) and discussion (Section 5), each contains three items. Section 6 (documentation) contains two items pertaining to protocol availability and funding for the investigation.

In total, 24 Delphi were merged into seven SCRIBE items because they referred to the same topics: (a) SCRIBE Item 5 (design) contained three Delphi items (design structure, number of sequences, and decision rules for phase change); (b) Item 8 (randomization), two Delphi items (sequence and onset of randomization); (c) Item 11 (participant characteristics), two Delphi items (demographics and etiology); (d) Item 13 (approvals), two Delphi items (ethics approval and participant consent); (e) Item 14 (measures), nine Delphi items (operational definitions of the target behavior, who selected it, how it was measured, independent assessor blind to phase, interrater agreement, follow-up measures, measures of generalization and social validity, and methods to enhance quality of measurement); (f) Item 19 (results), two Delphi items (sequence completed and early stopping); and (g) Item 20 (raw data), four Delphi items (results, raw data record, access to raw data, and stability of baseline). One of the Delphi items relating to meta-analysis, was considered not to represent a minimum standard of reporting for single-case experimental designs and accordingly was deleted.

The audio recording of the 2-day consensus meeting was transcribed. The final guideline items were confirmed after close examination of the conference transcript and the SCRIBE 2016 checklist was developed (see Table 1 ). The meeting report was prepared and distributed to the steering committee members in June 2012. The Sydney executive then began the process of drafting background information sections for each item and integrating these with the broader literature for the E&E article. Multiple versions of the E&E article were distributed over the next 2 years to the steering committee members for their comment and subsequent versions incorporated the feedback.

The Single-Case Reporting Guideline In BEhavioural Interventions (SCRIBE) 2016 Checklist

Authors can use the checklist to help with writing a research report and readers (including journal editors/reviewers) can use the checklist to evaluate whether the report meets the points outlined in the guideline. Users will find the detailed SCRIBE 2016 E&E document ( Tate et al., 2016 ) helpful for providing rationale for the items, with examples of adequate reporting from the literature.

Following publication of this SCRIBE 2016 Statement and the E&E article ( Tate et al., 2016 ), the next stage of activity focuses on further dissemination. Obtaining journal endorsement for the SCRIBE 2016 is a vital task because it has been demonstrated that journals that endorse specific reporting guidelines are associated with better reporting than journals where such endorsement does not exist ( Turner et al., 2012 ). The SCRIBE project is indexed on the EQUATOR network ( http://www.equator-network.org/ ) and a SCRIBE website ( www.sydney.edu.au/medicine/research/scribe ) provides information and links to the SCRIBE 2016 publications. SCRIBE users are encouraged to access the website and provide feedback on their experiences using the SCRIBE and suggestions for future revisions of the guideline. Future research will evaluate the uptake and impact of the SCRIBE 2016.

We expect that the publication rate of single-case experiments and the research into single-case methodology will expand over the years, given the evidence of such a trend (e.g., Hammond & Gast, 2010 ) and also considering the recent interest shown in journal publication of special issues dedicated to single-case design research referred to earlier in this article. As is common for guidelines, the SCRIBE 2016 will likely require updates and revisions to remain current and aligned with the best evidence available on methodological standards.

We developed the SCRIBE 2016 to provide authors, journal reviewers, and editors with a recommended minimum set of items that should be addressed in reports describing single-case research. Adherence to the SCRIBE 2016 should improve the clarity, completeness, transparency, and accuracy of reporting single-case research in the behavioral sciences. In turn, this will facilitate (a) replication, which is of critical importance for establishing generality, (b) the coding of different aspects of the studies as potential moderators in meta-analysis, and (c) evaluation of the scientific quality of the research. All of these factors are relevant to the development of evidence-based practices.

Single-case methodology is defined as the intensive and prospective study of the individual in which (a) the intervention/s is manipulated in an experimentally controlled manner across a series of discrete phases, and (b) measurement of the behavior targeted by the intervention is made repeatedly (and, ideally, frequently) throughout all phases. Professional guidelines call for the experimental effect to be demonstrated on at least three occasions by systematically manipulating the independent variable ( Horner et al., 2005 ; Kratochwill et al., 2010 , 2013 ). This criterion helps control for the confounding effect of extraneous variables that may adversely affect internal validity (e.g., history, maturation) and allows a functional cause and effect relationship to be established between the independent and dependent variables.

The SCRIBE Group wishes to pay special tribute to our esteemed colleague Professor William Shadish (1949–2016) who passed away on the eve of publication of this article. His contribution at all stages of the SCRIBE project was seminal.

Funding for the SCRIBE project was provided by the Lifetime Care and Support Authority of New South Wales, Australia. The funding body was not involved in the conduct, interpretation or writing of this work. We acknowledge the contribution of the responders to the Delphi surveys, as well as administrative assistance provided by Kali Godbee and Donna Wakim at the SCRIBE consensus meeting. Lyndsey Nickels was funded by an Australian Research Council Future Fellowship (FT120100102) and Australian Research Council Centre of Excellence in Cognition and Its Disorders (CE110001021). For further discussion on this topic, please visit the Archives of Scientific Psychology online public forum at http://arcblog.apa.org .

In order to encourage dissemination of the SCRIBE Statement, this article is freely accessible through Archives of Scientific Psychology and will also be published in the American Journal of Occupational Therapy , Aphasiology , Canadian Journal of Occupational Therapy , Evidence-Based Communication Assessment and Intervention , Journal of Clinical Epidemiology , Journal of School Psychology , Neuropsychological Rehabilitation , Physical Therapy , and Remedial and Special Education . The authors jointly hold the copyright for this article.

Barker , J. , McCarthy , P. , Jones , M. , Moran , A. ( 2011 ). Single case research methods in sport and exercise psychology . London, United Kingdom : Rout-ledge .

Google Scholar

Google Preview

Barker , J. B. , Mellalieu , S. D. , McCarthy , P. J. , Jones , M. V. , Moran , A. ( 2013 ). A review of single-case research in sport psychology 1997 2012: Research trends and future directions . Journal of Applied Sport Psychology ., 25 , 4 – 32 . http://dx.doi.org/10.1080/10413200.2012.709579

Barlow , D. H. , Nock , M. K. , Hersen , M. ( 2009 ). Single case experimental designs: Strategies for studying behavior change . (3rd ed.). Boston, MA : Pearson .

Beeson , P. M. , Robey , R. R. ( 2006 ). Evaluating single-subject treatment research: Lessons learned from the aphasia literature . Neuropsychology Review ., 16 , 161 – 169 . http://dx.doi.org/10.1007/s11065-006-9013-7

Boutron , I. , Moher , D. , Altman , D. G. , Schulz , K. F. , Ravaud , P. the CONSORT Group . ( 2008 ). Extending the CONSORT Statement to randomized trials of nonpharmacologic treatment: Explanation and elaboration . Annals of Internal Medicine ., 148 , 295 – 309 . http://dx.doi.org/10.7326/0003-4819-148-4-200802190-00008

Brewer , E. W. ( 2007 ). Delphi technique . In Salkind , N. J. (Ed.), Encyclopaedia of measurement and statistics . ( Vol. 1 , pp. 240 – 246 ). Thousand Oaks, CA : Sage . http://dx.doi.org/10.4135/9781412952644.n128

Didden , R. , Korzilius , H. , van Oorsouw , W. , Sturmey , P. ( 2006 ). Behavioral treatment of challenging behaviors in individuals with mild mental retardation: Meta-analysis of single-subject research . American Journal on Mental Retardation ., 111 , 290 – 298 . http://dx.doi.org/10.1352/0895-8017(2006)111[290:btocbi]2.0.co;2

Gast , D. L. , Ledford , J. R. ( 2014 ). Single case research methodology: Applications in special education and behavioral sciences . (2nd ed.). New York, NY : Routledge .

Hammond , D. , Gast , D. L. ( 2010 ). Descriptive analysis of single subject research designs: 1983–2007 . Education and Training in Autism and Developmental Disabilities ., 45 , 187 – 202 . http://www.jstor.org/stable/23879806

Hitchcock , J. H. , Horner , R. H. , Kratochwill , T. R. , Levin , J. R. , Odom , S. L. , Rindskopf , D. M. , Shadish , W. R. ( 2014 ). The What Works Clearinghouse single-case design pilot standards: Who will guard the guards? Remedial and Special Education ., 35 , 145 – 152 . http://dx.doi.org/10.1177/0741932513518979

Horner , R. H. , Carr , E. G. , Halle , J. , McGee , G. , Odom , S. , Wolery , M. ( 2005 ). The use of single-subject research to identify evidence-based practice in special education . Exceptional Children ., 71 , 165 – 179 . http://dx.doi.org/10.1177/001440290507100203

Kazdin , A. E. ( 2011 ). Single-case research designs: Methods for clinical and applied settings . New York, NY : Oxford University Press .

Kennedy , C. H. ( 2005 ). Single-case designs for educational research . Boston, MA : Pearson .

Kratochwill , T. R. , Hitchcock , J. , Horner , R. H. , Levin , J. R. , Odom , S. L. , Rindskopf , D. M. , Shadish , W. R. ( 2010 ). Single-case designs technical documentation . Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf

Kratochwill , T. R. , Hitchcock , J. H. , Horner , R. H. , Levin , J. R. , Odom , S. L. , Rindskopf , D. M. , Shadish , W. R. ( 2013 ). Single-case intervention research design standards . Remedial and Special Education ., 34 , 26 – 38 . http://dx.doi.org/10.1177/0741932512452794

Kratochwill , T. R. , Levin , J. R. ( 2014 ). Single-case intervention research: Methodological and statistical advances . Washington, DC : American Psychological Association . http://dx.doi.org/10.1037/14376-000

Maggin , D. M. , Briesch , A. M. , Chafouleas , S. M. , Ferguson , T. D. , Clark , C. ( 2014 ). A comparison of rubrics for identifying empirically supported practices with single-case research . Journal of Behavioral Education ., 23 , 287 – 311 . http://dx.doi.org/10.1007/s10864-013-9187-z

Maggin , D. M. , Chafouleas , S. M. , Goddard , K. M. , Johnson , A. H. ( 2011 ). A systematic evaluation of token economies as a classroom management tool for students with challenging behavior . Journal of School Psychology ., 49 , 529 – 554 . http://dx.doi.org/10.1016/j.jsp.2011.05.001

Moher , D. , Schulz , K. F. , Simera , I. , Altman , D. G. ( 2010 ). Guidance for developers of health research reporting guidelines . PLoS Medicine ., 7 , e1000217 . http://dx.doi.org/10.1371/journal.pmed.1000217

Moher , D. , Weeks , L. , Ocampo , M. , Seely , D. , Sampson , M. , Altman , D.G. , Hoey , J. ( 2011 ). Describing reporting guidelines for health research: A systematic review . Journal of Clinical Epidemiology ., 64 , 718 – 742 . http://dx.doi.org/10.1016/j.jclinepi.2010.09.013

Morgan , D. L. , Morgan , R. K. ( 2009 ). Single-case research methods for the behavioral and health sciences . Los Angeles, CA : Sage . http://dx.doi.org/10.4135/9781483329697

Perdices , M. , Tate , R. L. ( 2009 ). Single-subject designs as a tool for evidence-based clinical practice: Are they unrecognised and undervalued? Neuropsychological Rehabilitation ., 19 , 904 – 927 . http://dx.doi.org/10.1080/09602010903040691

Punja , S. , Bukutu , C. , Shamseer , L. , Sampson , M. , Hartling , L. , Urichuk , L. , Vohra , S. (in press) . Systematic review of the methods, statistical analysis, and meta-analysis of N-of-1 trials . Journal of Clinical Epidemiology .

Riley-Tillman , T. C. , Burns , M. K. ( 2009 ). Evaluating educational interventions: Single-case design for measuring response to intervention . New York, NY : Guilford Press .

Shadish , W. R. , Sullivan , K. J. ( 2011 ). Characteristics of single-case designs used to assess intervention effects in 2008 . Behavior Research Methods ., 43 , 971 – 980 . http://dx.doi.org/10.3758/s13428-011-0111-y

Shamseer , L. , Sampson , M. , Bukutu , C. , Schmid , C. H. , Nikles , J. , Tate , R. , the CENT group . ( 2015 ). CONSORT extension for reporting N -of-1 trials (CENT) 2015: Explanation and elaboration . British Medical Journal ., 350 , h1793 . http://dx.doi.org/10.1136/bmj/h1793

Smith , J. D. ( 2012 ). Single-case experimental designs: A systematic review of published research and current standards . Psychological Methods ., 17 , 510 – 550 . http://dx.doi.org/10.1037/a0029312

Tate , R. L. , McDonald , S. , Perdices , M. , Togher , L. , Schultz , R. , Savage , S. ( 2008 ). Rating the methodological quality of single-subject designs and N -of-1 trials: Introducing the Single-Case Experimental Design (SCED) Scale . Neuropsychological Rehabilitation ., 18 , 385 – 401 . http://dx.doi.org/10.1080/09602010802009201

Tate , R. L. , Perdices , M. , McDonald , S. , Togher , L. , Rosenkoetter , U. ( 2014 ). The design, conduct and report of single-case research: Resources to improve the quality of the neurorehabilitation literature . Neuropsychological Rehabilitation ., 24 , 315 – 331 . http://dx.doi.org/10.1080/09602011.2013.875043

Tate , R. , Perdices , M. , Rosenkoetter , U. , McDonald , S. , Togher , L. , Shadish , W. , for the SCRIBE Group . ( 2016 ). The Single-Case Reporting guideline In BEhavioural interventions (SCRIBE) 2016: Explanation and elaboration . Archives of Scientific Psychology ., 4 , 10 – 31 .

Tate , R. L. , Perdices , M. , Rosenkoetter , U. , Wakim , D. , Godbee , K. , Togher , L. , McDonald , S. ( 2013a ). Manual for the critical appraisal of single-case reports using the Risk of Bias in N-of-1 Trials (RoBiNT) Scale . Unpublished manuscript , University of Sydney , Australia .

Tate , R. L. , Perdices , M. , Rosenkoetter , U. , Wakim , D. , Godbee , K. , Togher , L. , McDonald , S. ( 2013b ). Revision of a method quality rating scale for single-case experimental designs and N -of-1 trials: The 15-item Risk of Bias in N -of-1 Trials (RoBiNT) Scale . Neuropsychological Rehabilitation ., 23 , 619 – 638 . http://dx.doi.org/10.1080/09602011.2013.824383

Tate , R. L. , Rosenkoetter , U. , Wakim , D. , Sigmundsdottir , L. , Doubleday , J. , Togher , L. , Perdices , M. ( 2015 ). The Risk of Bias in N-of-1 Trials (RoBiNT) Scale: An expanded manual for the critical appraisal of single-case reports . Sydney, Australia : Author .

Turner , L. , Shamseer , L. , Altman , D. G. , Weeks , L. , Peters , J. , Kober , T. , Moher , D. ( 2012 ). Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals . Cochrane Database of Systematic Reviews ., 11 , MR000030 . http://dx.doi.org/10.1002/14651858.mr000030.pub2

Vannest , K. J. , Davis , J. L. , Parker , R. I. ( 2013 ). Single case research in schools: Practical guidelines for school-based professionals . New York, NY : Routledge .

Vohra , S. , Shamseer , L. , Sampson , M. , Bukutu , C. , Schmid , C. H. , Tate , R. , the CENT group . ( 2015 ). CONSORT extension for reporting N -of-1 trials (CENT) 2015 Statement . British Medical Journal ., 350 , h1738 . http://dx.doi.org/10.1136/bmj/h1738

Von der Gracht , H. A. ( 2012 ). Consensus measurement in Delphi studies. Review and implications . Technological Forecasting and Social Change ., 79 , 1525 – 1536 . http://dx.doi.org/10.1016/j.techfore.2012.04.013

Wolery , M. , Dunlap , G. , Ledford , J. R. ( 2011 ). Single-case experimental methods: Suggestions for reporting . Journal of Early Intervention ., 33 , 103 – 109 . http://dx.doi.org/10.1177/1053815111418235

Email alerts

Citing articles via.

  • Recommend to Your Librarian
  • Advertising and Corporate Services
  • Journals Career Network

Affiliations

  • Online ISSN 1538-6724
  • Copyright © 2024 American Physical Therapy Association
  • About Oxford Academic
  • Publish journals with us
  • University press partners
  • What we publish
  • New features  
  • Open access
  • Institutional account management
  • Rights and permissions
  • Get help with access
  • Accessibility
  • Advertising
  • Media enquiries
  • Oxford University Press
  • Oxford Languages
  • University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

  • Copyright © 2024 Oxford University Press
  • Cookie settings
  • Cookie policy
  • Privacy policy
  • Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Evaluating What Works

Chapter 18 single case designs.

single case study of intervention

The single case design, also known as N-of-1 trial, or small N design, is a commonly used intervention design in speech and language therapy, clinical psychology, education, and neuropsychology, including aphasia therapy ( Perdices & Tate, 2009 ) . The single case design may be regarded as an extreme version of a within-subjects design, where two more more conditions are compared within a single person. This type of trial is sometimes dismissed as providing poor quality evidence, but a well-designed single case trial can be an efficient way to obtain an estimate of treatment efficacy in an individual. Very often, a set of single case trials is combined into a case series (see below). It is important to note that a single case trial is not a simple case report, but rather a study that is designed and analysed in a way that controls as far as possible for the kind of unwanted influences on results described in chapters 2-5.

18.1 Logic of single case designs

Table 18.1 compares the logic of the standard RCT and single case designs.

The first row of Table 18.1 shows the design for a simple 2-arm RCT, where intervention is varied between participants who are assessed on the same occasion and on the same outcome. The second row shows a version of the single case design where the invention is varied in a single subject at different time points. The third row shows the case where intervention is assessed by comparing treated vs untreated outcomes in the same subject on the same occasion - this is referred to by Krasny-Pacini & Evans ( 2018 ) as a multiple baseline design across behaviours and by Ledford et al. ( 2019 ) as an Adapted Alternating Treatment Design.

Whatever design is used, the key requirements are analogous to those of the RCT:

  • To minimize unwanted variance (noise) that may mask effects of interest.
  • To ensure that the effect we observe is as unbiased as possible.
  • To have sufficient data to reliably detect effects of interest

18.1.1 Minimising unwanted variance

In the RCT, this is achieved by having a large enough sample of participants to distinguish variation associated with intervention from idiosyncratic differences between individuals, and by keeping other aspects of the trial, such as timing and outcomes, as constant as possible.

With single case trials, we do not control for variation associated with individual participant characteristics - indeed we are interested in how different people respond to intervention - but we do need to control as far as possible for other sources of variation. The ABA design is a popular single-case design that involves contrasting an outcome during periods of intervention (B) versus periods of no intervention (A). For example, Armson & Stuart ( 1998 ) studied the impact of frequency-altered auditory feedback on 12 people who stuttered. They contrasted a baseline period (A), a period with auditory feedback (B), and a post-intervention period (A), taking several measures of stuttering during each period. Figure 18.1 shows data from two participants during a reading condition. Although the initial amount of stuttering differs for the two individuals, in both cases there is a precipitate drop in stuttering at the 5 minute point corresponding to the onset of the masking, which is sustained for some minutes before gradually rising back towards baseline levels. The baseline period is useful for providing a set of estimates of stuttering prior to intervention, so we can see that the drop in stuttering, at least initially, is outside the range of variation that occurs spontaneously.

Outcome over time in a single case ABA design. Redrawn from digitized data from two participants from Figure 2 of Armson et al (1998)

Figure 18.1: Outcome over time in a single case ABA design. Redrawn from digitized data from two participants from Figure 2 of Armson et al (1998)

In the context of neurorehabilitation and speech-and-language therapy, there would appear to be a major drawback of the ABA design. In the course of a historical review of this approach, Mirza et al. ( 2017 ) described the “N-of-1 niche” as follows:

“The design is most suited to assessing interventions that act and cease to act quickly. It is particularly useful in clinical contexts in which variability in patient responses is large, when the evidence is limited, and/or when the patient differs in important ways from the people who have participated in conventional randomized controlled trials.”

While the characteristics in the second sentence fit well with speech-and-language therapy interventions, the first requirement - that the intervention should “act and cease to act quickly” is clearly inapplicable. As described in the previous chapter, with few exceptions, interventions offered by those working in education as well as speech and language therapists and those working in other allied health professions are intended to produce long-term change that persists long after the therapy has ended. Indeed, a therapy that worked only during the period of administration would not be regarded as a success. This means that ABA designs, which compare an outcomes for periods with (B) and without (A) intervention, anticipating that scores will go up transiently during the intervention block, will be unsuitable. In this regard, behavioural interventions are quite different from many pharmaceutical interventions, where ABA designs are increasingly being used to compare a series of active and washout periods for a drug.

Despite this limitation, it is feasible to use an approach where we compare different time periods with and without intervention in some situations, most notably when there is good evidence that the targeted behaviour is unlikely to improve spontaneously. Inclusion of a baseline period, where behaviour is repeatedly sampled before intervention has begun, may give confidence that this is the case. An example of this multiple baseline approach from a study by Swain et al. ( 2020 ) is discussed below. Where the same intervention can be applied to a group of participants, then a hybrid method known as the multiple baseline across participants design can be used, which combines both between and within-subjects comparisons. A study of this kind by Koutsoftas et al. ( 2009 ) is discussed in the Class Exercise for this chapter.

In another kind of single case approach, the multiple baseline across behaviours design, it is the outcome measure that is varied. This approach is applicable where a single intervention has potential to target several specific behaviours or skills. This gives fields such as speech and language therapy an edge that drug trials often lack: we can change the specific outcome that is targeted by the intervention and compare it with another outcome that acts as a within-person control measure. To demonstrate effectiveness, we need to show that it is the targeted behaviour that improves, while the comparison behaviour remains unaffected.

For instance, Best et al. ( 2013 ) evaluated a cueing therapy for anomia in acquired aphasia in a case series of 16 patients, with the aim of comparing naming ability for 100 words that had been trained versus 100 untrained words. By using a large number of words, carefully selected to be of similar initial difficulty, they had sufficient data to show whether or not there was selective improvement for the trained words in individual participants.

Figure 18.2 is redrawn from data of Best et al. ( 2013 ) . The black points show N items correct on the two sets of items prior to intervention. They were selected to be of similar difficulty and hence they cluster around the dotted line, which shows the point where scores on both item sets are equivalent. The red points show scores after intervention. Points that fall above the dotted line correspond to cases who did better with trained than untrained words; those below the line did better with untrained than trained words. The red points tend to be placed vertically above the pre-test scores for each individual, indicating that there is improvement after intervention in the trained items (y-axis), but not on control items (x-axis).

Outcome over time in multiple outcomes design. Reconstructed data from 16 participants, Best et al (2013)

Figure 18.2: Outcome over time in multiple outcomes design. Reconstructed data from 16 participants, Best et al (2013)

Given the large number of items in each set, it is possible to do a simple comparison of proportions to see whether each person’s post-intervention score is reliably higher than their pre-intervention score for each item set. For 14 of the 16 cases, there is a statistically significant increase in scores from pre-intervention to post-intervention for target items (corresponding to those with lines that extend vertically above the dotted line), whereas this is the case for only two of the cases when control items are considered (corresponding to cases which show change in the horizontal direction from pre-intervention to post-intervention).

18.1.2 Minimising systematic bias

We have seen in previous chapters how the RCT has evolved to minimize numerous sources of unwanted systematic bias. We need to be alert to similar biases affecting results of single case trials. This is a particular concern for trial designs where we compare different time periods that do or do not include intervention. On the one hand, we may have the kinds of time-linked effects of maturation, practice or spontaneous recovery that lead to a general improvement over time, regardless of the intervention (see Chapter 4 ), and on the other hand there may be specific events that affect a person’s performance, such as life events or illness, which may have a prolonged beneficial or detrimental effect on performance.

The general assumption of this method is that if we use a sufficient number of time intervals, time-linked biases will average out, but while this may be true for transient environmental effects, such as noise or other distractions, it is not the case for systematic influences that continue over time. It is important to be aware of such limitations, and it may be worth considering combining this kind of design with other elements that control for time-related biases more effectively (see below).

18.1.3 The need for sufficient data

Some early single case studies in neuropsychology may have drawn over-optimistic conclusions because they had insufficient replications of outcome measures, assuming that the observed result was a valid indication of outcome without taking into account error of measurement. For instance, if someone’s score improved from 2/10 items correct prior to intervention to 5/10 correct after intervention, it can be hard to draw firm conclusions on the basis of this data alone: the change could just be part of random variability in the measure. The more measurements we have in this type of study, the more confidence we can place in results: whereas in RCTs we need sufficient participants to get a sense of how much variation there is in outcomes, in single case studies we need sufficient observations, and should never rely just a few instances.

In effect, we need to use the same kind of logic that we saw in Chapter 10 , where we estimated statistical power of a study by checking how likely we would be to get a statistically significant result from a given sample size. Table 18.2 shows power to detect a true effect of a given size in a multiple baseline across behaviours design of the kind used by Best et al. ( 2013 ) , where we have a set of trained vs untrained items, each of which is scored either right or wrong. The entries in this table show power, which is the probability that a study would detect a true effect of a given size on a one-tailed test. These entries were obtained by simulating 1000 datasets with each of the different combinations of sample size and effect size.

The columns show the effect size as the raw difference in proportion items correct for trained vs untrained words. It is assumed that these two sets were equated for difficulty prior to intervention, and the table shows the difference in proportion correct between the two sets after intervention. So if the initial proportion correct was .3 for both trained and untrained items, but after intervention, we expect accuracy on trained items to increase to .6 and the untrained to stay at .3, then the difference between the two sets after treatment is .3, shown in the 4th column of the table. We can then read down this column to see the point at which power reaches 80% or more. This occurs at the 4th row of the table, when there are 40 items in each set. If we anticipated a smaller increase in proportion correct for trained items of .2, then we would need 80 items per set to achieve 80% power.

18.2 Examples of studies using different types of single case design

As noted above, single case designs cover a wide range of options, and can vary the periods of observation or the classes of observations made for each individual.

18.2.1 A multiple baseline design: Speech and language therapy for adolescents in youth justice.

The key feature of a multiple baseline design is onset of intervention is staggered across at least three different points in time. Potentially, this could be done by having three or more participants, each of whom was measured in a baseline and an intervention phase, but with the timing of the intervention phase varied across participants. Alternatively, one can have different outcomes assessed in a single participant. Figure 18.3 from Swain et al. ( 2020 ) provides an illustration of the latter approach with a single participant, where different outcomes are targeted at different points in a series of intervention sessions. Typically, the timing of the interventions is not preplanned, but rather, they are introduced in sequence, with the second intervention only started after there is a documented effect from the first intervention, and so on ( Horner & Odom, 2014 ) .

Data from one case in the @swain2020 study. The shaded region shows sessions with intervention for each of the three outcomes.

Figure 18.3: Data from one case in the Swain et al. ( 2020 ) study. The shaded region shows sessions with intervention for each of the three outcomes.

The three panels show percentages correct on outcome probes for three skills: spelling-phonics, spelling-morphology and vocabulary. These were targeted sequentially in different sessions, and evidence for intervention effectiveness is obtained when a selective increase in performance is shown for the period during and after intervention. Note that for all three tasks, there is little or no overlap for scores during baseline and those obtained during and after intervention. The baseline data establish that although targeted behaviours vary from day to day, there is no systematic upward trend in performance until the intervention is administered. Furthermore, the fact that improvement is specific to the behaviour that is targeted in that session gives confidence that this is not just down to some general placebo effect.

In the other case studies reported by Swain et al. ( 2020 ) , different behaviours were targeted, according to the specific needs of the adolescents who were studied.

18.2.2 A study using multiple baseline across behaviours: Effectiveness of electropalatography

We noted in the previous chapter how, electropalatography, a biofeedback intervention that provides information about the position of articulators to help clients improve production of speech sounds, is ill-suited to evaluation in a RCT. It is potentially applicable to people with a wide variety of aetiologies, so the treated population is likely to be highly heterogenous, it requires expensive equipment including an individualized artificial palate, and the intervention is delivered over many one-to-one sessions. The goal of the intervention is to develop and consolidate new patterns of articulation that will persist after the intervention ends. It would not, therefore, make much sense to do a single case trial of electropalatography using an ABA design that involved comparing blocks of intervention vs no intervention. One can, however, run a trial that tests whether there is more improvement on targeted speech sounds than on other speech sounds that are not explicitly treated.

Leniston & Ebbels ( 2021 ) applied this approach to seven adolescents with severe speech disorders, all of whom were already familiar with electropalatography. Diagnoses included verbal dyspraxia, structural abnormalities of articulators (velopharyngeal insufficiency), mosaic Turner syndrome, and right-sided hemiplegia. At the start of each school term, two sounds were identified for each case: a target sound, which would be trained, and a control sound, which was also produced incorrectly, but which was not trained. Electropalatography training was administered twice a week in 30 minute sessions. The number of terms where intervention was given ranged from 1 to 3.

Individual results for targets and controls at each term (Redrawn Fig 3 from Leniston & Ebbels, data kindly provided by Susan Ebbels)

Figure 18.4: Individual results for targets and controls at each term (Redrawn Fig 3 from Leniston & Ebbels, data kindly provided by Susan Ebbels)

An analysis of group data found no main effect of target or time, but a large interaction between these, indicating greater improvement on trained speech sounds. The design of the study made it possible to look at individual cases, which gave greater insights into variation of the impact of intervention. As shown in Figure 18.4 , in the first term of intervention, there was a main effect of time for three of the participants (IDs 1, 3, and 4), but no interaction with sound type. In other words, these children improved over the course of the term, but this was seen for the untrained as well as the trained sound. By term 2, one of four children showed an interaction between time and sound type (ID4), and both children who continued training into term 3 (ID 1 and 2) showed such an interaction. Three children did not show any convincing evidence of benefit - all of these stopped intervention after one term.

As the authors noted, there is a key limitation of the study: when a significant interaction is found between time and sound type, this provides evidence that the intervention was effective. But when both trained and untrained sounds improve, this is ambiguous. It could mean that the intervention was effective, and its impact generalized beyond the trained sounds. But it could also mean that the intervention was ineffective, with improvement being due to other factors, such as maturation or practice on the outcome measure. Inclusion of a series of baseline measures might have helped establish how plausible these two possibilities were.

In sum, this method can handle the (typical) situation where intervention effects are sustained, but it is most effective if we do not expect any generalization of learning beyond the targeted behaviour or skill. Unfortunately, this is often at odds with speech and language therapy methods. For instance, in phonological therapy, the therapist may focus on helping a child distinguish and/or produce a specific sound pair, such as [d] vs [g], but there are good theoretical reasons to expect that if therapy is successful, it might generalize to other sound pairs, such as [t] vs [k], which depend on the same articulatory contrast between alveolar vs velar place. Indeed, if we think of the child’s phonology as part of a general system of contrasts, it might be expected that training on one sound pair could lead the whole system to reorganize. This is exactly what we would like to see in intervention, but it can make single case studies extremely difficult to interpret. Before designing such a study, it is worthwhile anticipating different outcomes and considering how they might be interpreted.

18.2.3 Example of an analysis of case series data

The terms ‘single case’ and ‘N-of-1’ are misleading in implying that only one participant is trained. More commonly, studies assemble a series of N-of-1 cases. Where the same intervention is used for all cases, regular group statistics may be applied. But unlike in RCTs, heterogeneity of response is expected and needs to be documented. In fact, in a single case case series, the interest is less in whether an overall intervention effect is statistically significant, as in whether the data provide evidence of individual variation in response to intervention, as this is what would justify analysis of individual cases. Formally, it is possible to test whether treatment effects vary significantly across participants by comparing a model that does or does not contain a term representing this effect, using linear mixed models , but we would recommend that researchers consult a statistician, as those methods are complex and require specific types of data. In practice, it is usually possible to judge how heterogeneous responses to intervention are by inspecting plots for individual participants.

Typically the small sample sizes in N-of-1 case series preclude any strong conclusions about the characteristics of those who do and do not show intervention effects, but results may subsequently be combined across groups, and specific hypotheses formulated about the characteristics of those who show a positive response.

An example comes from the study by Best et al. ( 2013 ) evaluating rehabilitation for anomia in acquired aphasia. As described above, researchers contrasted naming ability for words that had been trained, using a cueing approach, versus a set of untrained control words, a multiple baseline across behaviours design. In general, results were consistent with prior work in showing that improvement was largely confined to trained words. As noted above, this result allows us to draw a clear conclusion that the intervention was responsible for the improvement, but from a therapeutic perspective it was disappointing, as one might hope to see generalization to novel words.

The authors subdivided the participants according to their language profiles, and suggested that improvement on untrained words was seen in a subset of cases with a specific profile of semantic and phonological strengths. This result, however, was not striking and would need to be replicated.

18.2.4 Combining approaches to strengthen study design

In practice, aspects of different single-case designs can be combined - e.g. the cross-over design by Varley et al. ( 2016 ) that we described in Chapter 17 compared an intervention across two time points and two groups of participants, and also compared naming performance on three sets of items: trained words, untrained words that were phonemically similar to the trained words, and untrained words that were dissimilar to the trained words. Furthermore, baseline measures were taken in both groups to check the stability of naming responses. That study was not, however, analysed as a single case design: rather the focus was on average outcomes without analysing individual differences. However, the inclusion of multiple outcomes and multiple time points meant that responses of individuals could also have been investigated.

18.3 Statistical approaches to single case designs

Early reports of single case studies often focused on simple visualization of results to determine intervention effects, and this is still a common practice ( Perdices & Tate, 2009 ) . This is perfectly acceptable provided that differences are very obvious, as in Figure 18.1 above. We can think back to our discussion of analysis methods for RCTs: the aim is always to ask whether the variation associated with differences in intervention is greater than the variation within the intervention condition. In Figure 18.1 there is very little overlap in the values for the intervention vs non-intervention periods, and statistics are unnecessary. However, results can be less clearcut than this. Figure 18.5 shows data from two other participants in the study by Armson & Stuart ( 1998 ) , where people may disagree about whether or not there was an intervention effect. Indeed, one criticism of the use of visual analysis in single case designs is that it is too subjective, with poor inter-rater agreement about whether effects are seen. In addition, time series data will show dependencies: autocorrelation. This can create a spurious impression of visual separation in data for different time periods ( Kratochwill et al., 2014 ) . A more quantitative approach that adopts similar logic is to measure the degree of non-overlap between distributions for datapoints associated with intervention and those from baseline or control conditions ( Parker et al., 2014 ) . This has the advantage of simplicity, and relative ease of interpretation, but may be bedevilled by temporal trends in the data, and have relatively low statistical power unless there are large numbers of observations.

Outcome over time in a single case ABA design. Digitized data from two participants from Figure 2 of Armson et al (1998)

Figure 18.5: Outcome over time in a single case ABA design. Digitized data from two participants from Figure 2 of Armson et al (1998)

Unfortunately, rather than a well-worked-out set of recommendations for statistical analysis of single case trials, there is a plethora of methods in use, which can be challenging, or even overwhelming, for anyone starting out in this field to navigate ( Kratochwill & Levin, 2014 ) . Furthermore, most of the focus has been on ABA and related designs, with limited advice on how to deal with designs that use comparisons between treated and untreated outcomes.

Our view is that single-case designs have considerable potential. There has been much argument about how one should analyse single case study data; multilevel models have been proposed as a useful way of answering a number of questions with a single analysis - how large the treatment effect is for individual cases, how far the effect varies across cases, and how large the average effect is. However, caution has been urged, because, as Rindskopf & Ferron ( 2014 ) noted, these more complex models make far more assumptions about the data than simpler models, and results may be misleading if they are not met. We suggest that the best way to find the optimal analysis method may be to simulate data from a single-case study design, so that one can then compare the power and efficiency of different analytic approaches, and also their robustness to aspects of the data such as departures from normality. Simulation of such data is complicated by the fact that repeated observations from a single person will show autocorrelation, but this property can be incorporated in a simulation. A start has been made on this approach: see this website by James Pustejovsky. The fact that single-case studies typically make raw data available means there is a wealth of examples that could be tested in simulations.

18.4 Overview of considerations for single case designs

In most of the examples used here, the single case design could be embedded in natural therapy sessions, include heterogeneous participants, and be adapted to fit into regular clinical practice. This makes the method attractive to clinicians, but it should be noted that while incorporating evaluation into clinical activities is highly desirable, it often creates difficulties for controlling aspects of internal validity. For instance, in the study by Swain et al. ( 2020 ) , the researchers noted an element of unpredictability about data collection, because the young offenders that they worked with might either be unavailable, or unwilling to take part in intervention sessions on a given day. In the Leniston & Ebbels ( 2021 ) study the target and control probes were not always well-matched at baseline, and for some children, the amount of available data was too small to give a powerful test of the intervention effect. Our view is that it is far better to aim to evaluate interventions than not to do so, provided limitations of particular designs are understood and discussed. Table 18.3 can be used as a checklist against which to assess characteristics of a given study, to evaluate how far internal validity has been controlled.

We are not aware of specific evidence on this point, but it seems likely that the field of single case studies, just like other fields, is likely to suffer from problems of publication bias (Chapter 19 ), whereby results are reported when an intervention is successful, but not when it fails. If studies are adequately powered - and they should be designed so that they are - then all results should be reported, including those which may be ambiguous or unwanted, so that we can learn from what doesn’t work, as well as from what does.

A final point, which cannot be stressed enough, is that when evaluating a given intervention, a single study is never enough. Practical constraints usually make it impossible to devise the perfect study that gives entirely unambiguous results: rather we should aim for our studies to reduce the uncertainty in our understanding of the effectiveness of intervention, with each study building on those that have gone before. With single case studies, it is common to report the raw data in the paper, in either numeric or graphical form, and this is particularly useful in allowing other researchers to combine results across studies to form stronger conclusions (see Chapter 21 ).

18.5 Class exercise

  • Koutsoftas et al. ( 2009 ) conducted a study of effectiveness of phonemic awareness intervention with a group of children who showed poor sound awareness after receiving high quality whole-classroom teaching focused on this skill. Intervention sessions were administered by speech-language pathologists or experienced teachers to 13 groups of 2-4 children twice per week for a baseline and post-intervention period, and once per week during the 6 week intervention. Active intervention was preceded by a baseline period - one week (with two outcome measurement points) for seven groups of children, and two weeks (4 measurement points) for the other six groups. Outcome probes involved identifying the initial sound from a set of three words in each session. The researchers reported effect sizes for individual children that were calculated by comparing score on the probes in the post-intervention period with those in the baseline period, showing that most children showed significant gains on the outcome measure. Group results on the outcome measure (redrawn from Table 2 of the paper) are shown in Figure 18.6 .

Group means from Koutsoftas et al, 2009. Filled points show intervention phase, unfilled show baseline or post-intervention

Figure 18.6: Group means from Koutsoftas et al, 2009. Filled points show intervention phase, unfilled show baseline or post-intervention

Consider the following questions about this study. a. What kind of design is this? b. How well does this design guard against the biases shown in Table 18.3 ? c. Could the fact that intervention was delivered in small groups affect study validity? (Clue: see Chapter 16 ). d. If you were designing a study to follow up on this result, what changes might you make to the study design? e. What would be the logistic challenges in implementing these changes?

The SCRIBE guidelines have been developed to improve reporting of single case studies in the literature. An article by Tate et al. ( 2016 ) describing the guidelines with explanation and elaboration is available here , with a shorter article summarising the guidelines here . Identify a single case study in the published literature in your area and check it against the guidelines to see how much of the necessary information is provided. This kind of exercise can be more useful than just reading the guidelines, as it forces the reader to read an article carefully and consider what the guidelines mean.

In the previous chapter, we described a study by Calder et al. ( 2021 ) , which used a cross-over design to evaluate the effect of an intervention designed to improve grammatical morphology. This study also included probes to test mastery of untrained morphological endings. The trained structure was past tense -ed; a ‘generalization’ probe was another verb ending, 3rd person singular -s, and a control probe was possessive -s. Before studying Figure 18.7 make a note of your predictions about what you might expect to see with these additional probes.

Mean % correct for all 3 probes in delayed cross-over study by Calder et al, 2021 (data plotted from Calder et al's Table 2).

Figure 18.7: Mean % correct for all 3 probes in delayed cross-over study by Calder et al, 2021 (data plotted from Calder et al’s Table 2).

Once you have studied the Figure, consider whether you think the inclusion of the probes has strengthened your confidence in the conclusion that the intervention is effective.

  • Systematic review
  • Open access
  • Published: 19 February 2024

‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice

  • Annette Boaz   ORCID: orcid.org/0000-0003-0557-1294 1 ,
  • Juan Baeza 2 ,
  • Alec Fraser   ORCID: orcid.org/0000-0003-1121-1551 2 &
  • Erik Persson 3  

Implementation Science volume  19 , Article number:  15 ( 2024 ) Cite this article

990 Accesses

55 Altmetric

Metrics details

The gap between research findings and clinical practice is well documented and a range of strategies have been developed to support the implementation of research into clinical practice. The objective of this study was to update and extend two previous reviews of systematic reviews of strategies designed to implement research evidence into clinical practice.

We developed a comprehensive systematic literature search strategy based on the terms used in the previous reviews to identify studies that looked explicitly at interventions designed to turn research evidence into practice. The search was performed in June 2022 in four electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched from January 2010 up to June 2022 and applied no language restrictions. Two independent reviewers appraised the quality of included studies using a quality assessment checklist. To reduce the risk of bias, papers were excluded following discussion between all members of the team. Data were synthesised using descriptive and narrative techniques to identify themes and patterns linked to intervention strategies, targeted behaviours, study settings and study outcomes.

We identified 32 reviews conducted between 2010 and 2022. The reviews are mainly of multi-faceted interventions ( n  = 20) although there are reviews focusing on single strategies (ICT, educational, reminders, local opinion leaders, audit and feedback, social media and toolkits). The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Furthermore, a lot of nuance lies behind these headline findings, and this is increasingly commented upon in the reviews themselves.

Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been identified. We need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of research perspectives (including social science) in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed.

Peer Review reports

Contribution to the literature

Considerable time and money is invested in implementing and evaluating strategies to increase the implementation of research into clinical practice.

The growing body of evidence is not providing the anticipated clear lessons to support improved implementation.

Instead what is needed is better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice.

This would involve a more central role in implementation science for a wider range of perspectives, especially from the social, economic, political and behavioural sciences and for greater use of different types of synthesis, such as realist synthesis.

Introduction

The gap between research findings and clinical practice is well documented and a range of interventions has been developed to increase the implementation of research into clinical practice [ 1 , 2 ]. In recent years researchers have worked to improve the consistency in the ways in which these interventions (often called strategies) are described to support their evaluation. One notable development has been the emergence of Implementation Science as a field focusing explicitly on “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice” ([ 3 ] p. 1). The work of implementation science focuses on closing, or at least narrowing, the gap between research and practice. One contribution has been to map existing interventions, identifying 73 discreet strategies to support research implementation [ 4 ] which have been grouped into 9 clusters [ 5 ]. The authors note that they have not considered the evidence of effectiveness of the individual strategies and that a next step is to understand better which strategies perform best in which combinations and for what purposes [ 4 ]. Other authors have noted that there is also scope to learn more from other related fields of study such as policy implementation [ 6 ] and to draw on methods designed to support the evaluation of complex interventions [ 7 ].

The increase in activity designed to support the implementation of research into practice and improvements in reporting provided the impetus for an update of a review of systematic reviews of the effectiveness of interventions designed to support the use of research in clinical practice [ 8 ] which was itself an update of the review conducted by Grimshaw and colleagues in 2001. The 2001 review [ 9 ] identified 41 reviews considering a range of strategies including educational interventions, audit and feedback, computerised decision support to financial incentives and combined interventions. The authors concluded that all the interventions had the potential to promote the uptake of evidence in practice, although no one intervention seemed to be more effective than the others in all settings. They concluded that combined interventions were more likely to be effective than single interventions. The 2011 review identified a further 13 systematic reviews containing 313 discrete primary studies. Consistent with the previous review, four main strategy types were identified: audit and feedback; computerised decision support; opinion leaders; and multi-faceted interventions (MFIs). Nine of the reviews reported on MFIs. The review highlighted the small effects of single interventions such as audit and feedback, computerised decision support and opinion leaders. MFIs claimed an improvement in effectiveness over single interventions, although effect sizes remained small to moderate and this improvement in effectiveness relating to MFIs has been questioned in a subsequent review [ 10 ]. In updating the review, we anticipated a larger pool of reviews and an opportunity to consolidate learning from more recent systematic reviews of interventions.

This review updates and extends our previous review of systematic reviews of interventions designed to implement research evidence into clinical practice. To identify potentially relevant peer-reviewed research papers, we developed a comprehensive systematic literature search strategy based on the terms used in the Grimshaw et al. [ 9 ] and Boaz, Baeza and Fraser [ 8 ] overview articles. To ensure optimal retrieval, our search strategy was refined with support from an expert university librarian, considering the ongoing improvements in the development of search filters for systematic reviews since our first review [ 11 ]. We also wanted to include technology-related terms (e.g. apps, algorithms, machine learning, artificial intelligence) to find studies that explored interventions based on the use of technological innovations as mechanistic tools for increasing the use of evidence into practice (see Additional file 1 : Appendix A for full search strategy).

The search was performed in June 2022 in the following electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched for articles published since the 2011 review. We searched from January 2010 up to June 2022 and applied no language restrictions. Reference lists of relevant papers were also examined.

We uploaded the results using EPPI-Reviewer, a web-based tool that facilitated semi-automation of the screening process and removal of duplicate studies. We made particular use of a priority screening function to reduce screening workload and avoid ‘data deluge’ [ 12 ]. Through machine learning, one reviewer screened a smaller number of records ( n  = 1200) to train the software to predict whether a given record was more likely to be relevant or irrelevant, thus pulling the relevant studies towards the beginning of the screening process. This automation did not replace manual work but helped the reviewer to identify eligible studies more quickly. During the selection process, we included studies that looked explicitly at interventions designed to turn research evidence into practice. Studies were included if they met the following pre-determined inclusion criteria:

The study was a systematic review

Search terms were included

Focused on the implementation of research evidence into practice

The methodological quality of the included studies was assessed as part of the review

Study populations included healthcare providers and patients. The EPOC taxonomy [ 13 ] was used to categorise the strategies. The EPOC taxonomy has four domains: delivery arrangements, financial arrangements, governance arrangements and implementation strategies. The implementation strategies domain includes 20 strategies targeted at healthcare workers. Numerous EPOC strategies were assessed in the review including educational strategies, local opinion leaders, reminders, ICT-focused approaches and audit and feedback. Some strategies that did not fit easily within the EPOC categories were also included. These were social media strategies and toolkits, and multi-faceted interventions (MFIs) (see Table  2 ). Some systematic reviews included comparisons of different interventions while other reviews compared one type of intervention against a control group. Outcomes related to improvements in health care processes or patient well-being. Numerous individual study types (RCT, CCT, BA, ITS) were included within the systematic reviews.

We excluded papers that:

Focused on changing patient rather than provider behaviour

Had no demonstrable outcomes

Made unclear or no reference to research evidence

The last of these criteria was sometimes difficult to judge, and there was considerable discussion amongst the research team as to whether the link between research evidence and practice was sufficiently explicit in the interventions analysed. As we discussed in the previous review [ 8 ] in the field of healthcare, the principle of evidence-based practice is widely acknowledged and tools to change behaviour such as guidelines are often seen to be an implicit codification of evidence, despite the fact that this is not always the case.

Reviewers employed a two-stage process to select papers for inclusion. First, all titles and abstracts were screened by one reviewer to determine whether the study met the inclusion criteria. Two papers [ 14 , 15 ] were identified that fell just before the 2010 cut-off. As they were not identified in the searches for the first review [ 8 ] they were included and progressed to assessment. Each paper was rated as include, exclude or maybe. The full texts of 111 relevant papers were assessed independently by at least two authors. To reduce the risk of bias, papers were excluded following discussion between all members of the team. 32 papers met the inclusion criteria and proceeded to data extraction. The study selection procedure is documented in a PRISMA literature flow diagram (see Fig.  1 ). We were able to include French, Spanish and Portuguese papers in the selection reflecting the language skills in the study team, but none of the papers identified met the inclusion criteria. Other non- English language papers were excluded.

figure 1

PRISMA flow diagram. Source: authors

One reviewer extracted data on strategy type, number of included studies, local, target population, effectiveness and scope of impact from the included studies. Two reviewers then independently read each paper and noted key findings and broad themes of interest which were then discussed amongst the wider authorial team. Two independent reviewers appraised the quality of included studies using a Quality Assessment Checklist based on Oxman and Guyatt [ 16 ] and Francke et al. [ 17 ]. Each study was rated a quality score ranging from 1 (extensive flaws) to 7 (minimal flaws) (see Additional file 2 : Appendix B). All disagreements were resolved through discussion. Studies were not excluded in this updated overview based on methodological quality as we aimed to reflect the full extent of current research into this topic.

The extracted data were synthesised using descriptive and narrative techniques to identify themes and patterns in the data linked to intervention strategies, targeted behaviours, study settings and study outcomes.

Thirty-two studies were included in the systematic review. Table 1. provides a detailed overview of the included systematic reviews comprising reference, strategy type, quality score, number of included studies, local, target population, effectiveness and scope of impact (see Table  1. at the end of the manuscript). Overall, the quality of the studies was high. Twenty-three studies scored 7, six studies scored 6, one study scored 5, one study scored 4 and one study scored 3. The primary focus of the review was on reviews of effectiveness studies, but a small number of reviews did include data from a wider range of methods including qualitative studies which added to the analysis in the papers [ 18 , 19 , 20 , 21 ]. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. In this section, we discuss the different EPOC-defined implementation strategies in turn. Interestingly, we found only two ‘new’ approaches in this review that did not fit into the existing EPOC approaches. These are a review focused on the use of social media and a review considering toolkits. In addition to single interventions, we also discuss multi-faceted interventions. These were the most common intervention approach overall. A summary is provided in Table  2 .

Educational strategies

The overview identified three systematic reviews focusing on educational strategies. Grudniewicz et al. [ 22 ] explored the effectiveness of printed educational materials on primary care physician knowledge, behaviour and patient outcomes and concluded they were not effective in any of these aspects. Koota, Kääriäinen and Melender [ 23 ] focused on educational interventions promoting evidence-based practice among emergency room/accident and emergency nurses and found that interventions involving face-to-face contact led to significant or highly significant effects on patient benefits and emergency nurses’ knowledge, skills and behaviour. Interventions using written self-directed learning materials also led to significant improvements in nurses’ knowledge of evidence-based practice. Although the quality of the studies was high, the review primarily included small studies with low response rates, and many of them relied on self-assessed outcomes; consequently, the strength of the evidence for these outcomes is modest. Wu et al. [ 20 ] questioned if educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes. Although based on evaluation projects and qualitative data, their results also suggest that positive changes on patient outcomes can be made following the implementation of specific evidence-based approaches (or projects). The differing positive outcomes for educational strategies aimed at nurses might indicate that the target audience is important.

Local opinion leaders

Flodgren et al. [ 24 ] was the only systemic review focusing solely on opinion leaders. The review found that local opinion leaders alone, or in combination with other interventions, can be effective in promoting evidence‐based practice, but this varies both within and between studies and the effect on patient outcomes is uncertain. The review found that, overall, any intervention involving opinion leaders probably improves healthcare professionals’ compliance with evidence-based practice but varies within and across studies. However, how opinion leaders had an impact could not be determined because of insufficient details were provided, illustrating that reporting specific details in published studies is important if diffusion of effective methods of increasing evidence-based practice is to be spread across a system. The usefulness of this review is questionable because it cannot provide evidence of what is an effective opinion leader, whether teams of opinion leaders or a single opinion leader are most effective, or the most effective methods used by opinion leaders.

Pantoja et al. [ 26 ] was the only systemic review focusing solely on manually generated reminders delivered on paper included in the overview. The review explored how these affected professional practice and patient outcomes. The review concluded that manually generated reminders delivered on paper as a single intervention probably led to small to moderate increases in adherence to clinical recommendations, and they could be used as a single quality improvement intervention. However, the authors indicated that this intervention would make little or no difference to patient outcomes. The authors state that such a low-tech intervention may be useful in low- and middle-income countries where paper records are more likely to be the norm.

ICT-focused approaches

The three ICT-focused reviews [ 14 , 27 , 28 ] showed mixed results. Jamal, McKenzie and Clark [ 14 ] explored the impact of health information technology on the quality of medical and health care. They examined the impact of electronic health record, computerised provider order-entry, or decision support system. This showed a positive improvement in adherence to evidence-based guidelines but not to patient outcomes. The number of studies included in the review was low and so a conclusive recommendation could not be reached based on this review. Similarly, Brown et al. [ 28 ] found that technology-enabled knowledge translation interventions may improve knowledge of health professionals, but all eight studies raised concerns of bias. The De Angelis et al. [ 27 ] review was more promising, reporting that ICT can be a good way of disseminating clinical practice guidelines but conclude that it is unclear which type of ICT method is the most effective.

Audit and feedback

Sykes, McAnuff and Kolehmainen [ 29 ] examined whether audit and feedback were effective in dementia care and concluded that it remains unclear which ingredients of audit and feedback are successful as the reviewed papers illustrated large variations in the effectiveness of interventions using audit and feedback.

Non-EPOC listed strategies: social media, toolkits

There were two new (non-EPOC listed) intervention types identified in this review compared to the 2011 review — fewer than anticipated. We categorised a third — ‘care bundles’ [ 36 ] as a multi-faceted intervention due to its description in practice and a fourth — ‘Technology Enhanced Knowledge Transfer’ [ 28 ] was classified as an ICT-focused approach. The first new strategy was identified in Bhatt et al.’s [ 30 ] systematic review of the use of social media for the dissemination of clinical practice guidelines. They reported that the use of social media resulted in a significant improvement in knowledge and compliance with evidence-based guidelines compared with more traditional methods. They noted that a wide selection of different healthcare professionals and patients engaged with this type of social media and its global reach may be significant for low- and middle-income countries. This review was also noteworthy for developing a simple stepwise method for using social media for the dissemination of clinical practice guidelines. However, it is debatable whether social media can be classified as an intervention or just a different way of delivering an intervention. For example, the review discussed involving opinion leaders and patient advocates through social media. However, this was a small review that included only five studies, so further research in this new area is needed. Yamada et al. [ 31 ] draw on 39 studies to explore the application of toolkits, 18 of which had toolkits embedded within larger KT interventions, and 21 of which evaluated toolkits as standalone interventions. The individual component strategies of the toolkits were highly variable though the authors suggest that they align most closely with educational strategies. The authors conclude that toolkits as either standalone strategies or as part of MFIs hold some promise for facilitating evidence use in practice but caution that the quality of many of the primary studies included is considered weak limiting these findings.

Multi-faceted interventions

The majority of the systematic reviews ( n  = 20) reported on more than one intervention type. Some of these systematic reviews focus exclusively on multi-faceted interventions, whilst others compare different single or combined interventions aimed at achieving similar outcomes in particular settings. While these two approaches are often described in a similar way, they are actually quite distinct from each other as the former report how multiple strategies may be strategically combined in pursuance of an agreed goal, whilst the latter report how different strategies may be incidentally used in sometimes contrasting settings in the pursuance of similar goals. Ariyo et al. [ 35 ] helpfully summarise five key elements often found in effective MFI strategies in LMICs — but which may also be transferrable to HICs. First, effective MFIs encourage a multi-disciplinary approach acknowledging the roles played by different professional groups to collectively incorporate evidence-informed practice. Second, they utilise leadership drawing on a wide set of clinical and non-clinical actors including managers and even government officials. Third, multiple types of educational practices are utilised — including input from patients as stakeholders in some cases. Fourth, protocols, checklists and bundles are used — most effectively when local ownership is encouraged. Finally, most MFIs included an emphasis on monitoring and evaluation [ 35 ]. In contrast, other studies offer little information about the nature of the different MFI components of included studies which makes it difficult to extrapolate much learning from them in relation to why or how MFIs might affect practice (e.g. [ 28 , 38 ]). Ultimately, context matters, which some review authors argue makes it difficult to say with real certainty whether single or MFI strategies are superior (e.g. [ 21 , 27 ]). Taking all the systematic reviews together we may conclude that MFIs appear to be more likely to generate positive results than single interventions (e.g. [ 34 , 45 ]) though other reviews should make us cautious (e.g. [ 32 , 43 ]).

While multi-faceted interventions still seem to be more effective than single-strategy interventions, there were important distinctions between how the results of reviews of MFIs are interpreted in this review as compared to the previous reviews [ 8 , 9 ], reflecting greater nuance and debate in the literature. This was particularly noticeable where the effectiveness of MFIs was compared to single strategies, reflecting developments widely discussed in previous studies [ 10 ]. We found that most systematic reviews are bounded by their clinical, professional, spatial, system, or setting criteria and often seek to draw out implications for the implementation of evidence in their areas of specific interest (such as nursing or acute care). Frequently this means combining all relevant studies to explore the respective foci of each systematic review. Therefore, most reviews we categorised as MFIs actually include highly variable numbers and combinations of intervention strategies and highly heterogeneous original study designs. This makes statistical analyses of the type used by Squires et al. [ 10 ] on the three reviews in their paper not possible. Further, it also makes extrapolating findings and commenting on broad themes complex and difficult. This may suggest that future research should shift its focus from merely examining ‘what works’ to ‘what works where and what works for whom’ — perhaps pointing to the value of realist approaches to these complex review topics [ 48 , 49 ] and other more theory-informed approaches [ 50 ].

Some reviews have a relatively small number of studies (i.e. fewer than 10) and the authors are often understandably reluctant to engage with wider debates about the implications of their findings. Other larger studies do engage in deeper discussions about internal comparisons of findings across included studies and also contextualise these in wider debates. Some of the most informative studies (e.g. [ 35 , 40 ]) move beyond EPOC categories and contextualise MFIs within wider systems thinking and implementation theory. This distinction between MFIs and single interventions can actually be very useful as it offers lessons about the contexts in which individual interventions might have bounded effectiveness (i.e. educational interventions for individual change). Taken as a whole, this may also then help in terms of how and when to conjoin single interventions into effective MFIs.

In the two previous reviews, a consistent finding was that MFIs were more effective than single interventions [ 8 , 9 ]. However, like Squires et al. [ 10 ] this overview is more equivocal on this important issue. There are four points which may help account for the differences in findings in this regard. Firstly, the diversity of the systematic reviews in terms of clinical topic or setting is an important factor. Secondly, there is heterogeneity of the studies within the included systematic reviews themselves. Thirdly, there is a lack of consistency with regards to the definition and strategies included within of MFIs. Finally, there are epistemological differences across the papers and the reviews. This means that the results that are presented depend on the methods used to measure, report, and synthesise them. For instance, some reviews highlight that education strategies can be useful to improve provider understanding — but without wider organisational or system-level change, they may struggle to deliver sustained transformation [ 19 , 44 ].

It is also worth highlighting the importance of the theory of change underlying the different interventions. Where authors of the systematic reviews draw on theory, there is space to discuss/explain findings. We note a distinction between theoretical and atheoretical systematic review discussion sections. Atheoretical reviews tend to present acontextual findings (for instance, one study found very positive results for one intervention, and this gets highlighted in the abstract) whilst theoretically informed reviews attempt to contextualise and explain patterns within the included studies. Theory-informed systematic reviews seem more likely to offer more profound and useful insights (see [ 19 , 35 , 40 , 43 , 45 ]). We find that the most insightful systematic reviews of MFIs engage in theoretical generalisation — they attempt to go beyond the data of individual studies and discuss the wider implications of the findings of the studies within their reviews drawing on implementation theory. At the same time, they highlight the active role of context and the wider relational and system-wide issues linked to implementation. It is these types of investigations that can help providers further develop evidence-based practice.

This overview has identified a small, but insightful set of papers that interrogate and help theorise why, how, for whom, and in which circumstances it might be the case that MFIs are superior (see [ 19 , 35 , 40 ] once more). At the level of this overview — and in most of the systematic reviews included — it appears to be the case that MFIs struggle with the question of attribution. In addition, there are other important elements that are often unmeasured, or unreported (e.g. costs of the intervention — see [ 40 ]). Finally, the stronger systematic reviews [ 19 , 35 , 40 , 43 , 45 ] engage with systems issues, human agency and context [ 18 ] in a way that was not evident in the systematic reviews identified in the previous reviews [ 8 , 9 ]. The earlier reviews lacked any theory of change that might explain why MFIs might be more effective than single ones — whereas now some systematic reviews do this, which enables them to conclude that sometimes single interventions can still be more effective.

As Nilsen et al. ([ 6 ] p. 7) note ‘Study findings concerning the effectiveness of various approaches are continuously synthesized and assembled in systematic reviews’. We may have gone as far as we can in understanding the implementation of evidence through systematic reviews of single and multi-faceted interventions and the next step would be to conduct more research exploring the complex and situated nature of evidence used in clinical practice and by particular professional groups. This would further build on the nuanced discussion and conclusion sections in a subset of the papers we reviewed. This might also support the field to move away from isolating individual implementation strategies [ 6 ] to explore the complex processes involving a range of actors with differing capacities [ 51 ] working in diverse organisational cultures. Taxonomies of implementation strategies do not fully account for the complex process of implementation, which involves a range of different actors with different capacities and skills across multiple system levels. There is plenty of work to build on, particularly in the social sciences, which currently sits at the margins of debates about evidence implementation (see for example, Normalisation Process Theory [ 52 ]).

There are several changes that we have identified in this overview of systematic reviews in comparison to the review we published in 2011 [ 8 ]. A consistent and welcome finding is that the overall quality of the systematic reviews themselves appears to have improved between the two reviews, although this is not reflected upon in the papers. This is exhibited through better, clearer reporting mechanisms in relation to the mechanics of the reviews, alongside a greater attention to, and deeper description of, how potential biases in included papers are discussed. Additionally, there is an increased, but still limited, inclusion of original studies conducted in low- and middle-income countries as opposed to just high-income countries. Importantly, we found that many of these systematic reviews are attuned to, and comment upon the contextual distinctions of pursuing evidence-informed interventions in health care settings in different economic settings. Furthermore, systematic reviews included in this updated article cover a wider set of clinical specialities (both within and beyond hospital settings) and have a focus on a wider set of healthcare professions — discussing both similarities, differences and inter-professional challenges faced therein, compared to the earlier reviews. These wider ranges of studies highlight that a particular intervention or group of interventions may work well for one professional group but be ineffective for another. This diversity of study settings allows us to consider the important role context (in its many forms) plays on implementing evidence into practice. Examining the complex and varied context of health care will help us address what Nilsen et al. ([ 6 ] p. 1) described as, ‘society’s health problems [that] require research-based knowledge acted on by healthcare practitioners together with implementation of political measures from governmental agencies’. This will help us shift implementation science to move, ‘beyond a success or failure perspective towards improved analysis of variables that could explain the impact of the implementation process’ ([ 6 ] p. 2).

This review brings together 32 papers considering individual and multi-faceted interventions designed to support the use of evidence in clinical practice. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been conducted. As a whole, this substantial body of knowledge struggles to tell us more about the use of individual and MFIs than: ‘it depends’. To really move forwards in addressing the gap between research evidence and practice, we may need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of perspectives, especially from the social, economic, political and behavioural sciences in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed. Harvey et al. [ 53 ] suggest that when context is likely to be critical to implementation success there are a range of primary research approaches (participatory research, realist evaluation, developmental evaluation, ethnography, quality/ rapid cycle improvement) that are likely to be appropriate and insightful. While these approaches often form part of implementation studies in the form of process evaluations, they are usually relatively small scale in relation to implementation research as a whole. As a result, the findings often do not make it into the subsequent systematic reviews. This review provides further evidence that we need to bring qualitative approaches in from the periphery to play a central role in many implementation studies and subsequent evidence syntheses. It would be helpful for systematic reviews, at the very least, to include more detail about the interventions and their implementation in terms of how and why they worked.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Before and after study

Controlled clinical trial

Effective Practice and Organisation of Care

High-income countries

Information and Communications Technology

Interrupted time series

Knowledge translation

Low- and middle-income countries

Randomised controlled trial

Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–30. https://doi.org/10.1016/S0140-6736(03)14546-1 .

Article   PubMed   Google Scholar  

Green LA, Seifert CM. Translation of research into practice: why we can’t “just do it.” J Am Board Fam Pract. 2005;18:541–5. https://doi.org/10.3122/jabfm.18.6.541 .

Eccles MP, Mittman BS. Welcome to Implementation Science. Implement Sci. 2006;1:1–3. https://doi.org/10.1186/1748-5908-1-1 .

Article   PubMed Central   Google Scholar  

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:2–14. https://doi.org/10.1186/s13012-015-0209-1 .

Article   Google Scholar  

Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:1–8. https://doi.org/10.1186/s13012-015-0295-0 .

Nilsen P, Ståhl C, Roback K, et al. Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Sci. 2013;8:2–12. https://doi.org/10.1186/1748-5908-8-63 .

Rycroft-Malone J, Seers K, Eldh AC, et al. A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: an exemplar. Implementation Sci. 2018;13:1–15. https://doi.org/10.1186/s13012-018-0811-0 .

Boaz A, Baeza J, Fraser A, European Implementation Score Collaborative Group (EIS). Effective implementation of research into practice: an overview of systematic reviews of the health literature. BMC Res Notes. 2011;4:212. https://doi.org/10.1186/1756-0500-4-212 .

Article   PubMed   PubMed Central   Google Scholar  

Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior – an overview of systematic reviews of interventions. Med Care. 2001;39 8Suppl 2:II2–45.

Google Scholar  

Squires JE, Sullivan K, Eccles MP, et al. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:1–22. https://doi.org/10.1186/s13012-014-0152-6 .

Salvador-Oliván JA, Marco-Cuenca G, Arquero-Avilés R. Development of an efficient search filter to retrieve systematic reviews from PubMed. J Med Libr Assoc. 2021;109:561–74. https://doi.org/10.5195/jmla.2021.1223 .

Thomas JM. Diffusion of innovation in systematic review methodology: why is study selection not yet assisted by automation? OA Evid Based Med. 2013;1:1–6.

Effective Practice and Organisation of Care (EPOC). The EPOC taxonomy of health systems interventions. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2016. epoc.cochrane.org/epoc-taxonomy . Accessed 9 Oct 2023.

Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: a systematic review. Health Inf Manag. 2009;38:26–37. https://doi.org/10.1177/183335830903800305 .

Menon A, Korner-Bitensky N, Kastner M, et al. Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med. 2009;41:1024–32. https://doi.org/10.2340/16501977-0451 .

Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44:1271–8. https://doi.org/10.1016/0895-4356(91)90160-b .

Article   CAS   PubMed   Google Scholar  

Francke AL, Smit MC, de Veer AJ, et al. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:1–11. https://doi.org/10.1186/1472-6947-8-38 .

Jones CA, Roop SC, Pohar SL, et al. Translating knowledge in rehabilitation: systematic review. Phys Ther. 2015;95:663–77. https://doi.org/10.2522/ptj.20130512 .

Scott D, Albrecht L, O’Leary K, Ball GDC, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:1–17. https://doi.org/10.1186/1748-5908-7-70 .

Wu Y, Brettle A, Zhou C, Ou J, et al. Do educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes? A systematic review. Nurse Educ Today. 2018;70:109–14. https://doi.org/10.1016/j.nedt.2018.08.026 .

Yost J, Ganann R, Thompson D, Aloweni F, et al. The effectiveness of knowledge translation interventions for promoting evidence-informed decision-making among nurses in tertiary care: a systematic review and meta-analysis. Implement Sci. 2015;10:1–15. https://doi.org/10.1186/s13012-015-0286-1 .

Grudniewicz A, Kealy R, Rodseth RN, Hamid J, et al. What is the effectiveness of printed educational materials on primary care physician knowledge, behaviour, and patient outcomes: a systematic review and meta-analyses. Implement Sci. 2015;10:2–12. https://doi.org/10.1186/s13012-015-0347-5 .

Koota E, Kääriäinen M, Melender HL. Educational interventions promoting evidence-based practice among emergency nurses: a systematic review. Int Emerg Nurs. 2018;41:51–8. https://doi.org/10.1016/j.ienj.2018.06.004 .

Flodgren G, O’Brien MA, Parmelli E, et al. Local opinion leaders: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD000125.pub5 .

Arditi C, Rège-Walther M, Durieux P, et al. Computer-generated reminders delivered on paper to healthcare professionals: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2017. https://doi.org/10.1002/14651858.CD001175.pub4 .

Pantoja T, Grimshaw JM, Colomer N, et al. Manually-generated reminders delivered on paper: effects on professional practice and patient outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD001174.pub4 .

De Angelis G, Davies B, King J, McEwan J, et al. Information and communication technologies for the dissemination of clinical practice guidelines to health professionals: a systematic review. JMIR Med Educ. 2016;2:e16. https://doi.org/10.2196/mededu.6288 .

Brown A, Barnes C, Byaruhanga J, McLaughlin M, et al. Effectiveness of technology-enabled knowledge translation strategies in improving the use of research in public health: systematic review. J Med Internet Res. 2020;22:e17274. https://doi.org/10.2196/17274 .

Sykes MJ, McAnuff J, Kolehmainen N. When is audit and feedback effective in dementia care? A systematic review. Int J Nurs Stud. 2018;79:27–35. https://doi.org/10.1016/j.ijnurstu.2017.10.013 .

Bhatt NR, Czarniecki SW, Borgmann H, et al. A systematic review of the use of social media for dissemination of clinical practice guidelines. Eur Urol Focus. 2021;7:1195–204. https://doi.org/10.1016/j.euf.2020.10.008 .

Yamada J, Shorkey A, Barwick M, Widger K, et al. The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ Open. 2015;5:e006808. https://doi.org/10.1136/bmjopen-2014-006808 .

Afari-Asiedu S, Abdulai MA, Tostmann A, et al. Interventions to improve dispensing of antibiotics at the community level in low and middle income countries: a systematic review. J Glob Antimicrob Resist. 2022;29:259–74. https://doi.org/10.1016/j.jgar.2022.03.009 .

Boonacker CW, Hoes AW, Dikhoff MJ, Schilder AG, et al. Interventions in health care professionals to improve treatment in children with upper respiratory tract infections. Int J Pediatr Otorhinolaryngol. 2010;74:1113–21. https://doi.org/10.1016/j.ijporl.2010.07.008 .

Al Zoubi FM, Menon A, Mayo NE, et al. The effectiveness of interventions designed to increase the uptake of clinical practice guidelines and best practices among musculoskeletal professionals: a systematic review. BMC Health Serv Res. 2018;18:2–11. https://doi.org/10.1186/s12913-018-3253-0 .

Ariyo P, Zayed B, Riese V, Anton B, et al. Implementation strategies to reduce surgical site infections: a systematic review. Infect Control Hosp Epidemiol. 2019;3:287–300. https://doi.org/10.1017/ice.2018.355 .

Borgert MJ, Goossens A, Dongelmans DA. What are effective strategies for the implementation of care bundles on ICUs: a systematic review. Implement Sci. 2015;10:1–11. https://doi.org/10.1186/s13012-015-0306-1 .

Cahill LS, Carey LM, Lannin NA, et al. Implementation interventions to promote the uptake of evidence-based practices in stroke rehabilitation. Cochrane Database Syst Rev. 2020. https://doi.org/10.1002/14651858.CD012575.pub2 .

Pedersen ER, Rubenstein L, Kandrack R, Danz M, et al. Elusive search for effective provider interventions: a systematic review of provider interventions to increase adherence to evidence-based treatment for depression. Implement Sci. 2018;13:1–30. https://doi.org/10.1186/s13012-018-0788-8 .

Jenkins HJ, Hancock MJ, French SD, Maher CG, et al. Effectiveness of interventions designed to reduce the use of imaging for low-back pain: a systematic review. CMAJ. 2015;187:401–8. https://doi.org/10.1503/cmaj.141183 .

Bennett S, Laver K, MacAndrew M, Beattie E, et al. Implementation of evidence-based, non-pharmacological interventions addressing behavior and psychological symptoms of dementia: a systematic review focused on implementation strategies. Int Psychogeriatr. 2021;33:947–75. https://doi.org/10.1017/S1041610220001702 .

Noonan VK, Wolfe DL, Thorogood NP, et al. Knowledge translation and implementation in spinal cord injury: a systematic review. Spinal Cord. 2014;52:578–87. https://doi.org/10.1038/sc.2014.62 .

Albrecht L, Archibald M, Snelgrove-Clarke E, et al. Systematic review of knowledge translation strategies to promote research uptake in child health settings. J Pediatr Nurs. 2016;31:235–54. https://doi.org/10.1016/j.pedn.2015.12.002 .

Campbell A, Louie-Poon S, Slater L, et al. Knowledge translation strategies used by healthcare professionals in child health settings: an updated systematic review. J Pediatr Nurs. 2019;47:114–20. https://doi.org/10.1016/j.pedn.2019.04.026 .

Bird ML, Miller T, Connell LA, et al. Moving stroke rehabilitation evidence into practice: a systematic review of randomized controlled trials. Clin Rehabil. 2019;33:1586–95. https://doi.org/10.1177/0269215519847253 .

Goorts K, Dizon J, Milanese S. The effectiveness of implementation strategies for promoting evidence informed interventions in allied healthcare: a systematic review. BMC Health Serv Res. 2021;21:1–11. https://doi.org/10.1186/s12913-021-06190-0 .

Zadro JR, O’Keeffe M, Allison JL, Lembke KA, et al. Effectiveness of implementation strategies to improve adherence of physical therapist treatment choices to clinical practice guidelines for musculoskeletal conditions: systematic review. Phys Ther. 2020;100:1516–41. https://doi.org/10.1093/ptj/pzaa101 .

Van der Veer SN, Jager KJ, Nache AM, et al. Translating knowledge on best practice into improving quality of RRT care: a systematic review of implementation strategies. Kidney Int. 2011;80:1021–34. https://doi.org/10.1038/ki.2011.222 .

Pawson R, Greenhalgh T, Harvey G, et al. Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10Suppl 1:21–34. https://doi.org/10.1258/1355819054308530 .

Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: illustrating the method for implementation research. Implementation Sci. 2012;7:1–10. https://doi.org/10.1186/1748-5908-7-33 .

Johnson MJ, May CR. Promoting professional behaviour change in healthcare: what interventions work, and why? A theory-led overview of systematic reviews. BMJ Open. 2015;5:e008592. https://doi.org/10.1136/bmjopen-2015-008592 .

Metz A, Jensen T, Farley A, Boaz A, et al. Is implementation research out of step with implementation practice? Pathways to effective implementation support over the last decade. Implement Res Pract. 2022;3:1–11. https://doi.org/10.1177/26334895221105585 .

May CR, Finch TL, Cornford J, Exley C, et al. Integrating telecare for chronic disease management in the community: What needs to be done? BMC Health Serv Res. 2011;11:1–11. https://doi.org/10.1186/1472-6963-11-131 .

Harvey G, Rycroft-Malone J, Seers K, Wilson P, et al. Connecting the science and practice of implementation – applying the lens of context to inform study design in implementation research. Front Health Serv. 2023;3:1–15. https://doi.org/10.3389/frhs.2023.1162762 .

Download references

Acknowledgements

The authors would like to thank Professor Kathryn Oliver for her support in the planning the review, Professor Steve Hanney for reading and commenting on the final manuscript and the staff at LSHTM library for their support in planning and conducting the literature search.

This study was supported by LSHTM’s Research England QR strategic priorities funding allocation and the National Institute for Health and Care Research (NIHR) Applied Research Collaboration South London (NIHR ARC South London) at King’s College Hospital NHS Foundation Trust. Grant number NIHR200152. The views expressed are those of the author(s) and not necessarily those of the NIHR, the Department of Health and Social Care or Research England.

Author information

Authors and affiliations.

Health and Social Care Workforce Research Unit, The Policy Institute, King’s College London, Virginia Woolf Building, 22 Kingsway, London, WC2B 6LE, UK

Annette Boaz

King’s Business School, King’s College London, 30 Aldwych, London, WC2B 4BG, UK

Juan Baeza & Alec Fraser

Federal University of Santa Catarina (UFSC), Campus Universitário Reitor João Davi Ferreira Lima, Florianópolis, SC, 88.040-900, Brazil

Erik Persson

You can also search for this author in PubMed   Google Scholar

Contributions

AB led the conceptual development and structure of the manuscript. EP conducted the searches and data extraction. All authors contributed to screening and quality appraisal. EP and AF wrote the first draft of the methods section. AB, JB and AF performed result synthesis and contributed to the analyses. AB wrote the first draft of the manuscript and incorporated feedback and revisions from all other authors. All authors revised and approved the final manuscript.

Corresponding author

Correspondence to Annette Boaz .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix a., additional file 2: appendix b., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Boaz, A., Baeza, J., Fraser, A. et al. ‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice. Implementation Sci 19 , 15 (2024). https://doi.org/10.1186/s13012-024-01337-z

Download citation

Received : 01 November 2023

Accepted : 05 January 2024

Published : 19 February 2024

DOI : https://doi.org/10.1186/s13012-024-01337-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Implementation
  • Interventions
  • Clinical practice
  • Research evidence
  • Multi-faceted

Implementation Science

ISSN: 1748-5908

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

single case study of intervention

A Systematic Review of Single-Case Research Examining Culturally Responsive Behavior Interventions in Schools

  • Original Paper
  • Published: 09 February 2023

Cite this article

  • Sean C. Austin   ORCID: orcid.org/0000-0001-9696-7889 1 ,
  • Kent McIntosh 1 ,
  • Sara Izzard 1 &
  • Brooke Daugherty 1  

572 Accesses

6 Altmetric

Explore all metrics

Cultural responsiveness, or building on the strengths of the learning histories and social contingencies of students, is an important feature of instruction to engage students and ensure they feel represented in their classrooms. This systematic literature review examined which single-case designs have been used to study culturally responsive behavior interventions, the settings in which they have been tested, and the elements of cultural responsiveness present among them. This review included 12 studies that used experimental single-case design, demonstrated consideration of student culture in their design, and examined the effect of the intervention on student behavior. Studies were coded for their inclusion of culturally responsive elements in design or implementation, the size of student intervention groups, and the conditions used as a comparison to determine effectiveness of culturally responsive intervention. The most common culturally responsive elements were those that used knowledge of student identities and student input in intervention design; however, input from families or the community were infrequently used. Interventions were delivered across a spectrum of group sizes, including in whole classrooms, small groups, and with individual students. Only two studies directly compared non-adapted intervention with culturally responsive intervention within a multi-treatment design. This review has implications for how practitioners may evaluate behavior interventions for use in their classrooms and for the design of future studies to evaluate potential additive and equity-enhancing effects of culturally responsive behavior interventions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

single case study of intervention

Barnes, J. C., & Motz, R. T. (2018). Reducing racial inequalities in adulthood arrest by reducing inequalities in school discipline: Evidence from the school-to-prison pipeline. Developmental Psychology, 54 (12), 2328–2340. https://doi.org/10.1037/dev0000613

Article   PubMed   Google Scholar  

Barrett, N., McEachin, A., Mills, J. N., & Valant, J. (2021). Disparities and discrimination in student discipline by race and family income. Journal of Human Resources, 56 (3), 711–748.

Article   Google Scholar  

Bottiani, J. H., Larson, K. E., Debnam, K. J., Bischoff, C. M., & Bradshaw, C. P. (2018). Promoting educators’ use of culturally responsive practices: A systematic review of inservice interventions. Journal of Teacher Education, 69 (4), 367–385.

Bradshaw, C. P., Pas, E. T., Bottiani, J. H., Debnam, K. J., Reinke, W. M., Herman, K. C., & Rosenberg, M. S. (2018). Promoting cultural responsivity and student engagement through double check coaching of classroom teachers: An efficacy study. School Psychology Review, 47 (2), 118–134. https://doi.org/10.17105/SPR-2017-0119.V47-2

Brown, C., Maggin, D. M., & Buren, M. (2018). Systematic review of cultural adaptations of school-based social, emotional, and behavioral interventions for students of color. Education and Treatment of Children, 41 (4), 431–456.

Castro-Olivo, S. M. (2014). Promoting social-emotional learning in adolescent Latino ELLs: A study of the culturally adapted Strong Teens program. School Psychology Quarterly, 29 (4), 567.

Castro-Olivo, S., & Merrell, K. (2012). Validating cultural adaptations of a school-based social-emotional learning programme for use with Latino immigrant adolescents. Advances in School Mental Health Promotion, 5 , 78–92. https://doi.org/10.1080/1754730X.2012.689193

Castro-Olivo, S., Preciado, J., Le, L., Marciante, M., & Garcia, M. (2018). The effects of culturally adapted version of “First Steps to Success” for Latino English Language Learners: Preliminary pilot study. Psychology in the Schools, 55 (1), 36–49. https://doi.org/10.1002/pits.22092

Collins, T. A., Hawkins, R. O., Flowers, E. M., Kalra, H. D., Richard, J., & Haas, L. E. (2018). Behavior Bingo: The effects of a culturally relevant group contingency intervention for students with EBD. Psychology in the Schools, 55 (1), 63–75. https://doi.org/10.1002/pits.22091

Covidence systematic review software. (2020). Veritas Health Innovation.

Fallon, L. M., O’Keeffe, B. V., & Sugai, G. (2012). Consideration of culture and context in School-wide Positive Behavior Support: A review of current literature. Journal of Positive Behavior Interventions, 14 , 209–219.

Fallon, L. M., Cathcart, S. C., DeFouw, E. R., O’Keeffe, B. V., & Sugai, G. (2018). Promoting teachers’ implementation of culturally and contextually relevant class-wide behavior plans. Psychology in the Schools, 55 (3), 278–294.

Fallon, L. M., DeFouw, E. R., Cathcart, S. C., Berkman, T. S., Robinson-Link, P., O’Keeffe, B. V., & Sugai, G. (2021). School-based supports and interventions to improve social and behavioral outcomes with racially and ethnically minoritized youth: A review of recent quantitative research. Journal of Behavioral Education, 31 (1), 123–156.

Farmer, T. W. (2020). Reforming research to support culturally and ecologically responsive and developmentally meaningful practice in schools. Educational Psychologist, 55 (1), 32–39. https://doi.org/10.1080/00461520.2019.1698298

Gay, G. (2018). Culturally responsive teaching: Theory, research, and practice . Teachers College Press.

Google Scholar  

Gion, C., McIntosh, K., & Falcon, S. (2022). Effects of a multifaceted classroom intervention on racial disproportionality [Article]. School Psychology Review, 51 (1), 67–83. https://doi.org/10.1080/2372966X.2020.1788906

Girvan, E. J., McIntosh, K., & Santiago-Rosario, M. R. (2021). Community-level implicit and explicit racial biases predict racial disparities in school discipline. School Psychology Review . https://doi.org/10.1080/2372966X.2020.1838232

Gladney, D., Lo, Y.-Y., Kourea, L., & Johnson, H. N. (2021). Using multilevel coaching to improve general education teachers’ implementation fidelity of culturally responsive social skill instruction. Preventing School Failure: Alternative Education for Children and Youth, 65 (2), 175–184.

Guskey, T. R. (1986). Staff development and the process of teacher change. Educational Researcher, 15 , 5–12.

Guskey, T. R. (2002). Professional development and teacher change. Teachers and Teaching, 8 , 381–391.

Hershfeldt, P. A., Sechrest, R., Pell, K. L., Rosenberg, M. S., Bradshaw, C. P., & Leaf, P. J. (2009). Double-Check: A framework of cultural responsiveness applied to classroom behavior. TEACHING Exceptional Children plus, 6 (2), 2–18.

Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for School-wide Positive Behavior Support. Focus on Exceptional Children, 42 (8), 1–14.

Ishimaru, A. M., & Galloway, M. K. (2019). Hearts and minds first: Institutional logics in pursuit of educational equity. Educational Administration Quarterly, 57 (3), 470–502.

Kaufman, J. S., Jaser, S. S., Vaughan, E. L., Reynolds, J. S., DiDonato, J., Bernard, S. N., et al. (2010). Patterns in office referral data by grade, race/ethnicity, and gender. Journal of Positive Behavior Interventions, 12 , 44–54.

Klingner, J. K., Artiles, A. J., Kozleski, E., Harry, B., Zion, S., Tate, W., Durán, G. Z., & Riley, D. (2005). Addressing the disproportionate representation of culturally and linguistically diverse students in special education through culturally responsive educational systems. Education Policy Analysis Archives, 13 (38), 1–40.

Knochel, A. E., Blair, K.-S.C., & Sofarelli, R. (2021). Culturally focused classroom staff training to increase praise for students with autism spectrum disorder in Ghana. Journal of Positive Behavior Interventions, 23 (2), 106–117. https://doi.org/10.1177/1098300720929351

Leverson, M., Smith, K., McIntosh, K., Rose, J., & Pinkelman, S. (2021). PBIS cultural responsiveness field guide: Resources for trainers and coaches. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports .

Lo, Y.-Y., Mustian, A. L., Brophy, A., & White, R. B. (2011). Peer-mediated social skill instruction for African American males with or at risk for mild disabilities. Exceptionality, 19 (3), 191–209. https://doi.org/10.1080/09362835.2011.579851

Lo, Y.-Y., Correa, V. I., & Anderson, A. L. (2015). Culturally responsive social skill instruction for Latino male students. Journal of Positive Behavior Interventions, 17 (1), 15–27. https://doi.org/10.1177/1098300714532133

McIntosh, K., Girvan, E. J., Falcon, S., McDaniel, S., Smolkowski, K., Bastable, E., Santiago-Rosario, M. R., Izzard, S., Austin, S. C., Nese, R. N. T., & Baldy, T. (2021). Equity-focused PBIS approach reduces racial inequities in school discipline: A randomized controlled trial. School Psychology., 36 (6), 433.

McIntosh, K., Gion, C., & Bastable, E. (2018). Do schools implementing SWPBIS have decreased racial and ethnic disproportionality in school discipline? PBIS Evaluation Brief. OSEP National Technical Assistance Center on Positive Behavioral Interventions and Supports .

McKenney, E. L. W., Mann, K. A., Brown, D. L., & Jewell, J. D. (2017). Addressing cultural responsiveness in consultation: An empirical demonstration. Journal of Educational & Psychological Consultation, 27 (3), 289–316. https://doi.org/10.1080/10474412.2017.1287575

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Group P. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. Open Medicine : a Peer-Reviewed, Independent, Open-Access Journal, 3 (3), e123–e130.

PubMed   Google Scholar  

Muldrew, A. C., & Miller, F. G. (2021). Examining the effects of the personal matrix activity with diverse students. Psychology in the Schools, 58 (3), 515–533.

Peguero, A. A., & Bracy, N. L. (2015). School order, justice, and education: Climate, discipline practices, and dropping out. Journal of Research on Adolescence, 25 (3), 412–426.

Preciado, J. A., Horner, R. H., & Baker, S. K. (2009). Using a function-based approach to decrease problem behavior and increase reading academic engagement for Latino English language learners. Journal of Special Education, 42 , 227–240.

Robinson-Ervin, P., Cartledge, G., Musti-Rao, S., Gibson, L., Jr., & Keyes, S. E. (2016). Social skills instruction for urban learners with emotional and behavioral disorders: A culturally responsive and computer-based intervention. Behavioral Disorders, 41 (4), 209–225. https://doi.org/10.17988/bedi-41-04-209-225.1

Rocque, M., & Paternoster, R. (2011). Understanding the antecedents of the" school-to-jail" link: The relationship between race and school discipline. The Journal of Criminal Law and Criminology, 101 (2), 633–665.

Saeedi, S., & Richardson, E. (2019). A black lives matter and critical race theory–informed critique of code-switching pedagogy. Race, Justice, and Activism in Literacy Instruction , 147.

Skiba, R. J., Michael, R. S., Nardo, A. C., & Peterson, R. L. (2002). The color of discipline: Sources of racial and gender disproportionality in school punishment. The Urban Review, 34 , 317–342. https://doi.org/10.1023/A:1021320817372

Skiba, R. J., Arredondo, M. I., & Williams, N. T. (2014). More than a metaphor: The contribution of exclusionary discipline to a school-to-prison pipeline. Equity Excellence in Education, 47 (4), 546–564. https://doi.org/10.1080/10665684.2014.958965

Smith, J. D. (2012). Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods, 17 (4), 510.

Smith, J. D., Knoble, N. B., Zerr, A. A., Dishion, T. J., & Stormshak, E. A. (2014). Family check-up effects across diverse ethnic groups: Reducing early-adolescence antisocial behavior by reducing family conflict. Journal of Clinical Child & Adolescent Psychology, 43 (3), 400–414.

Smolkowski, K., Girvan, E. J., McIntosh, K., Nese, R. N. T., & Horner, R. H. (2016). Vulnerable decision points in school discipline: Comparison of discipline for African American compared to White students in elementary schools. Behavioral Disorders, 41 , 178–195.

Sugai, G., Horner, R. H., & McIntosh, K. (2008). Best practices in developing a broad-scale system of support for school-wide positive behavior support. In A. Thomas & J. P. Grimes (Eds.), Best practices in school psychology V (pp. 765–780). National Association of School Psychologists.

Sugai, G., O’Keeffe, B. V., & Fallon, L. M. (2012). A contextual consideration of culture and school-wide positive behavior support. Journal of Positive Behavior Interventions, 14 , 197–208.

Tobin, T. J., & Vincent, C. G. (2011). Strategies for preventing disproportionate exclusions of African American students. Preventing School Failure, 55 , 192–201. https://doi.org/10.1080/1045988X.2010.532520

U.S. Department of Education Office for Civil Rights. (2018). 2015–16 Civil Rights Data Collection: School Climate and Safety . https://www2.ed.gov/about/offices/list/ocr/docs/school-climate-and-safety.pdf

U.S. Department of Education Office for Civil Rights. (2021). An Overview of Exclusionary Discipline Practices in Public Schools for the 2017–18 School Year . https://www2.ed.gov/about/offices/list/ocr/docs/crdc-exclusionary-school-discipline.pdf

Vincent, C. G., Swain-Bradway, J., Tobin, T. J., & May, S. (2011). Disciplinary referrals for culturally and linguistically diverse students with and without disabilities: Patterns resulting from school-wide positive behavior support. Exceptionality, 19 , 175–190.

Wallace, J. M. J., Goodkind, S., Wallace, C. M., & Bachman, J. G. (2008). Racial, ethnic, and gender differences in school discipline among U.S. high school students: 1991–2005. Negro Educational Review, 59 , 47–62.

Weinstein, C. S., Tomlinson-Clarke, S., & Curran, M. (2004). Toward a conception of culturally responsive classroom management. Journal of Teacher Education, 55 , 25–38.

Welsh, R. O., & Little, S. (2018). The school discipline dilemma: A comprehensive review of disparities and alternative approaches. Review of Educational Research, 88 (5), 752–794. https://doi.org/10.3102/0034654318791582

What Works Clearinghouse (2020). What Works Clearinghouse Standards Handbook, Version 4.1. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/wwc/handbooks

Download references

The research reported here was supported by the Institute of Education Sciences, US Department of Education, through Grant R324A170034 to University of Oregon. The opinions expressed are those of the authors and do not represent views of the Institute or the US Department of Education.

Author information

Authors and affiliations.

Department of Special Education and Clinical Sciences, University of Oregon, 1571 Alder St, Eugene, OR, 97401, USA

Sean C. Austin, Kent McIntosh, Sara Izzard & Brooke Daugherty

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sean C. Austin .

Ethics declarations

Conflicts of interest.

There are no potential conflicts of interest to disclose for the authors of this manuscript.

Ethical Approval

Because study did not require human or animal participants (and therefore no informed consent), approval from the Institutional Review Board was not sought.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Austin, S.C., McIntosh, K., Izzard, S. et al. A Systematic Review of Single-Case Research Examining Culturally Responsive Behavior Interventions in Schools. J Behav Educ (2023). https://doi.org/10.1007/s10864-023-09506-8

Download citation

Accepted : 30 January 2023

Published : 09 February 2023

DOI : https://doi.org/10.1007/s10864-023-09506-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Cultural responsiveness
  • Intervention
  • Single case
  • Literature review
  • School-based
  • Find a journal
  • Publish with us
  • Track your research

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Advanced Search
  • Journal List
  • Biochem Med (Zagreb)
  • v.24(2); 2014 Jun

Observational and interventional study design types; an overview

The appropriate choice in study design is essential for the successful execution of biomedical and public health research. There are many study designs to choose from within two broad categories of observational and interventional studies. Each design has its own strengths and weaknesses, and the need to understand these limitations is necessary to arrive at correct study conclusions.

Observational study designs, also called epidemiologic study designs, are often retrospective and are used to assess potential causation in exposure-outcome relationships and therefore influence preventive methods. Observational study designs include ecological designs, cross sectional, case-control, case-crossover, retrospective and prospective cohorts. An important subset of observational studies is diagnostic study designs, which evaluate the accuracy of diagnostic procedures and tests as compared to other diagnostic measures. These include diagnostic accuracy designs, diagnostic cohort designs, and diagnostic randomized controlled trials.

Interventional studies are often prospective and are specifically tailored to evaluate direct impacts of treatment or preventive measures on disease. Each study design has specific outcome measures that rely on the type and quality of data utilized. Additionally, each study design has potential limitations that are more severe and need to be addressed in the design phase of the study. This manuscript is meant to provide an overview of study design types, strengths and weaknesses of common observational and interventional study designs.

Introduction

Study design plays an important role in the quality, execution, and interpretation of biomedical and public health research ( 1 – 12 ). Each study design has their own inherent strengths and weaknesses, and there can be a general hierarchy in study designs, however, any hierarchy cannot be applied uniformly across study design types ( 3 , 5 , 6 , 9 ). Epidemiological and interventional research studies include three elements; 1) definition and measure of exposure in two or more groups, 2) measure of health outcome(s) in these same groups, and 3) statistical comparison made between groups to assess potential relationships between the exposure and outcome, all of which are defined by the researcher ( 1 – 4 , 8 , 13 ). The measure of exposure in epidemiologic studies may be tobacco use (“Yes” vs . “No”) to define the two groups and may be the treatment (Active drug vs . placebo) in interventional studies. Health outcome(s) can be the development of a disease or symptom (e.g. lung cancer) or curing a disease or symptom (e.g. reduction of pain). Descriptive studies, which are not epidemiological or interventional, lack one or more of these elements and have limited application. High quality epidemiological and interventional studies contain detailed information on the design, execution and interpretation of results, with methodology clearly written and able to be reproduced by other researchers.

Research is generally considered as primary or secondary research. Primary research relies upon data gathered from original research expressly for that purpose ( 1 , 3 , 5 ). Secondary research focuses on single or multiple data sources that are not collected for a single research purpose ( 14 , 15 ). Secondary research includes meta-analyses and best practice guidelines for treatments. This paper will focus on the study designs and their strengths, weaknesses, and common statistical outcomes of primary research.

The choice of a study design hinges on many factors, including prior research, availability of study participants, funding, and time constraints. One common decision point is the desire to suggest causation. The most common causation criteria are proposed by Hill ( 16 ). Of these, demonstrating temporality is the only mandatory criterion for suggesting temporality. Therefore, prospective studies that follow study participants forward through time, including prospective cohort studies and interventional studies, are best suited for suggesting causation. Causal conclusions cannot be proven from an observational study. Additionally, causation between an exposure and an outcome cannot be proven by one study alone; multiple studies across different populations should be considered when making causation assessments ( 17 ).

Primary research has been categorized in different ways. Common categorization schema include temporal nature of the study design (retrospective or prospective), usability of the study results (basic or applied), investigative purpose (descriptive or analytical), purpose (prevention, diagnosis or treatment), or role of the investigator (observational or interventional). This manuscript categorizes study designs by observational and interventional criteria, however, other categorization methods are described as well.

Observational and interventional studies

Within primary research there are observational studies and interventional studies. Observational studies, also called epidemiological studies, are those where the investigator is not acting upon study participants, but instead observing natural relationships between factors and outcomes. Diagnostic studies are classified as observational studies, but are a unique category and will be discussed independently. Interventional studies, also called experimental studies, are those where the researcher intercedes as part of the study design. Additionally, study designs may be classified by the role that time plays in the data collection, either retrospective or prospective. Retrospective studies are those where data are collected from the past, either through records created at that time or by asking participants to remember their exposures or outcomes. Retrospective studies cannot demonstrate temporality as easily and are more prone to different biases, particularly recall bias. Prospective studies follow participants forward through time, collecting data in the process. Prospective studies are less prone to some types of bias and can more easily demonstrate that the exposure preceded the disease, thereby more strongly suggesting causation. Table 1 describes the broad categories of observational studies: the disease measures applicable to each, the appropriate measures of risk, and temporality of each study design. Epidemiologic measures include point prevalence, the proportion of participants with disease at a given point in time, period prevalence, the proportion of participants with disease within a specified time frame, and incidence, the accumulation of new cases over time. Measures of risk are generally categorized into two categories: those that only demonstrate an association, such as an odds ratio (and some other measures), and those that demonstrate temporality and therefore suggest causation, such as hazard ratio. Table 2 outlines the strengths and weaknesses of each observational study design.

Observational study design measures of disease, measures of risk, and temporality.

Observational study design strengths and weaknesses.

Observational studies

Ecological study design.

The most basic observational study is an ecological study. This study design compares clusters of people, usually grouped based on their geographical location or temporal associations ( 1 , 2 , 6 , 9 ). Ecological studies assign one exposure level for each distinct group and can provide a rough estimation of prevalence of disease within a population. Ecological studies are generally retrospective. An example of an ecological study is the comparison of the prevalence of obesity in the United States and France. The geographic area is considered the exposure and the outcome is obesity. There are inherent potential weaknesses with this approach, including loss of data resolution and potential misclassification ( 10 , 11 , 13 , 18 , 19 ). This type of study design also has additional weaknesses. Typically these studies derive their data from large databases that are created for purposes other than research, which may introduce error or misclassification ( 10 , 11 ). Quantification of both the number of cases and the total population can be difficult, leading to error or bias. Lastly, due to the limited amount of data available, it is difficult to control for other factors that may mask or falsely suggest a relationship between the exposure and the outcome. However, ecological studies are generally very cost effective and are a starting point for hypothesis generation.

Proportional mortality ratio study design

Proportional mortality ratio studies (PMR) utilize the defined well recorded outcome of death and subsequent records that are maintained regarding the decedent ( 1 , 6 , 8 , 20 ). By using records, this study design is able to identify potential relationships between exposures, such as geographic location, occupation, or age and cause of death. The epidemiological outcomes of this study design are proportional mortality ratio and standardized mortality ratio. In general these are the ratio of the proportion of cause-specific deaths out of all deaths between exposure categories ( 20 ). As an example, these studies can address questions about higher proportion of cardiovascular deaths among different ethnic and racial groups ( 21 ). A significant drawback to the PMR study design is that these studies are limited to death as an outcome ( 3 , 5 , 22 ). Additionally, the reliance on death records makes it difficult to control for individual confounding factors, variables that either conceal or falsely demonstrate associations between the exposure and outcome. An example of a confounder is tobacco use confounding the relationship between coffee intake and cardiovascular disease. Historically people often smoked and drank coffee while on coffee breaks. If researchers ignore smoking they would inaccurately find a strong relationship between coffee use and cardiovascular disease, where some of the risk is actually due to smoking. There are also concerns regarding the accuracy of death certificate data. Strengths of the study design include the well-defined outcome of death, the relative ease and low cost of obtaining data, and the uniformity of collection of these data across different geographical areas.

Cross-sectional study design

Cross-sectional studies are also called prevalence studies because one of the main measures available is study population prevalence ( 1 – 12 ). These studies consist of assessing a population, as represented by the study sample, at a single point in time. A common cross-sectional study type is the diagnostic accuracy study, which is discussed later. Cross-sectional study samples are selected based on their exposure status, without regard for their outcome status. Outcome status is obtained after participants are enrolled. Ideally, a wider distribution of exposure will allow for a higher likelihood of finding an association between the exposure and outcome if one exists ( 1 – 3 , 5 , 8 ). Cross-sectional studies are retrospective in nature. An example of a cross-sectional study would be enrolling participants who are either current smokers or never smokers, and assessing whether or not they have respiratory deficiencies. Random sampling of the population being assessed is more important in cross-sectional studies as compared to other observational study designs. Selection bias from non-random sampling may result in flawed measure of prevalence and calculation of risk. The study sample is assessed for both exposure and outcome at a single point in time. Because both exposure and outcome are assessed at the same time, temporality cannot be demonstrated, i.e. it cannot be demonstrated that the exposure preceded the disease ( 1 – 3 , 5 , 8 ). Point prevalence and period prevalence can be calculated in cross-sectional studies. Measures of risk for the exposure-outcome relationship that can be calculated in cross-sectional study design are odds ratio, prevalence odds ratio, prevalence ratio, and prevalence difference. Cross-sectional studies are relatively inexpensive and have data collected on an individual which allows for more complete control for confounding. Additionally, cross-sectional studies allow for multiple outcomes to be assessed simultaneously.

Case-control study design

Case-control studies were traditionally referred to as retrospective studies, due to the nature of the study design and execution ( 1 – 12 , 23 , 24 ). In this study design, researchers identify study participants based on their case status, i.e. diseased or not diseased. Quantification of the number of individuals among the cases and the controls who are exposed allow for statistical associations between exposure and outcomes to be established ( 1 – 3 , 5 , 8 ). An example of a case control study is analysing the relationship between obesity and knee replacement surgery. Cases are participants who have had knee surgery, and controls are a random sampling of those who have not, and the comparison is the relative odds of being obese if you have knee surgery as compared to those that do not. Matching on one or more potential confounders allows for minimization of those factors as potential confounders in the exposure-outcome relationship ( 1 – 3 , 5 , 8 ). Additionally, case-control studies are at increased risk for bias, particularly recall bias, due to the known case status of study participants ( 1 – 3 , 5 , 8 ). Other points of consideration that have specific weight in case-control studies include the appropriate selection of controls that balance generalizability and minimize bias, the minimization of survivor bias, and the potential for length time bias ( 25 ). The largest strength of case-control studies is that this study design is the most efficient study design for rare diseases. Additional strengths include low cost, relatively fast execution compared to cohort studies, the ability to collect individual participant specific data, the ability to control for multiple confounders, and the ability to assess multiple exposures of interest. The measure of risk that is calculated in case-control studies is the odds ratio, which are the odds of having the exposure if you have the disease. Other measures of risk are not applicable to case-control studies. Any measure of prevalence and associated measures, such as prevalence odds ratio, in a case-control study is artificial because the researcher arbitrarily sets the proportion of cases to non-cases in this study design. Temporality can be suggested, however, it is rarely definitively demonstrated because it is unknown if the development of the disease truly preceded the exposure. It should be noted that for certain outcomes, particularly death, the criteria for demonstrating temporality in that specific exposure-outcome relationship are met and the use of relative risk as a measure of risk may be justified.

Case-crossover study design

A case-crossover study relies upon an individual to act as their own control for comparison issues, thereby minimizing some potential confounders ( 1 , 5 , 12 ). This study design should not be confused with a crossover study design which is an interventional study type and is described below. For case-crossover studies, cases are assessed for their exposure status immediately prior to the time they became a case, and then compared to their own exposure at a prior point where they didn’t become a case. The selection of the prior point for comparison issues is often chosen at random or relies upon a mean measure of exposure over time. Case-crossover studies are always retrospective. An example of a case-crossover study would be evaluating the exposure of talking on a cell phone and being involved in an automobile crash. Cases are drivers involved in a crash and the comparison is that same driver at a random timeframe where they were not involved in a crash. These types of studies are particularly good for exposure-outcome relationships where the outcome is acute and well defined, e.g. electrocutions, lacerations, automobile crashes, etc. ( 1 , 5 ). Exposure-outcome relationships that are assessed using case-crossover designs should have health outcomes that do not have a subclinical or undiagnosed period prior to becoming a “case” in the study ( 12 ). The exposure is cell phone use during the exposure periods, both before the crash and during the control period. Additionally, the reliance upon prior exposure time requires that the exposure not have an additive or cumulative effect over time ( 1 , 5 ). Case-crossover study designs are at higher risk for having recall bias as compared with other study designs ( 12 ). Study participants are more likely to remember an exposure prior to becoming a case, as compared to not becoming a case.

Retrospective and prospective cohort study design

Cohort studies involve identifying study participants based on their exposure status and either following them through time to identify which participants develop the outcome(s) of interest, or look back at data that were created in the past, prior to the development of the outcome. Prospective cohort studies are considered the gold standard of observational research ( 1 – 3 , 5 , 8 , 10 , 11 ). These studies begin with a cross-sectional study to categorize exposure and identify cases at baseline. Disease-free participants are then followed and cases are measured as they develop. Retrospective cohort studies also begin with a cross-sectional study to categorize exposure and identify cases. Exposures are then measured based on records created at that time. Additionally, in an ideal retrospective cohort, case status is also tracked using historical data that were created at that point in time. Occupational groups, particularly those that have regular surveillance or certifications such as Commercial Truck Drivers, are particularly well positioned for retrospective cohort studies because records of both exposure and outcome are created as part of commercial and regulatory purposes ( 8 ). These types of studies have the ability to demonstrate temporality and therefore identify true risk factors, not associated factors, as can be done in other types of studies.

Cohort studies are the only observational study that can calculate incidence, both cumulative incidence and an incidence rate ( 1 , 3 , 5 , 6 , 10 , 11 ). Also, because the inception of a cohort study is identical to a cross-sectional study, both point prevalence and period prevalence can be calculated. There are many measures of risk that can be calculated from cohort study data. Again, the measures of risk for the exposure-outcome relationship that can be calculated in cross-sectional study design of odds ratio, prevalence odds ratio, prevalence ratio, and prevalence difference can be calculated in cohort studies as well. Measures of risk that leverage a cohort study’s ability to calculate incidence include incidence rate ratio, relative risk, risk ratio, and hazard ratio. These measures that demonstrate temporality are considered stronger measures for demonstrating causation and identification of risk factors.

Diagnostic testing and evaluation study designs

A specific study design is the diagnostic accuracy study, which is often used as part of the clinical decision making process. Diagnostic accuracy study designs are those that compare a new diagnostic method with the current “gold standard” diagnostic procedure in a cross-section of both diseased and healthy study participants. Gold standard diagnostic procedures are the current best-practice for diagnosing a disease. An example is comparing a new rapid test for a cancer with the gold standard method of biopsy. There are many intricacies to diagnostic testing study designs that should be considered. The proper selection of the gold standard evaluation is important for defining the true measures of accuracy for the new diagnostic procedure. Evaluations of diagnostic test results should be blinded to the case status of the participant. Similar to the intention-to-treat concept discussed later in interventional studies, diagnostic tests have a procedure of analyses called intention to diagnose (ITD), where participants are analysed in the diagnostic category they were assigned, regardless of the process in which a diagnosis was obtained. Performing analyses according to an a priori defined protocol, called per protocol analyses (PP or PPA), is another potential strength to diagnostic study testing. Many measures of the new diagnostic procedure, including accuracy, sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio can be calculated. These measures of the diagnostic test allow for comparison with other diagnostic tests and aid the clinician in determining which test to utilize.

Interventional study designs

Interventional study designs, also called experimental study designs, are those where the researcher intervenes at some point throughout the study. The most common and strongest interventional study design is a randomized controlled trial, however, there are other interventional study designs, including pre-post study design, non-randomized controlled trials, and quasi-experiments ( 1 , 5 , 13 ). Experimental studies are used to evaluate study questions related to either therapeutic agents or prevention. Therapeutic agents can include prophylactic agents, treatments, surgical approaches, or diagnostic tests. Prevention can include changes to protective equipment, engineering controls, management, policy or any element that should be evaluated as to a potential cause of disease or injury.

Pre-post study design

A pre-post study measures the occurrence of an outcome before and again after a particular intervention is implemented. A good example is comparing deaths from motor vehicle crashes before and after the enforcement of a seat-belt law. Pre-post studies may be single arm, one group measured before the intervention and again after the intervention, or multiple arms, where there is a comparison between groups. Often there is an arm where there is no intervention. The no-intervention arm acts as the control group in a multi-arm pre-post study. These studies have the strength of temporality to be able to suggest that the outcome is impacted by the intervention, however, pre-post studies do not have control over other elements that are also changing at the same time as the intervention is implemented. Therefore, changes in disease occurrence during the study period cannot be fully attributed to the specific intervention. Outcomes measured for pre-post intervention studies may be binary health outcomes such as incidence or prevalence, or mean values of a continuous outcome such as systolic blood pressure may also be used. The analytic methods of pre-post studies depend on the outcome being measured. If there are multiple treatment arms, it is also likely that the difference from beginning to end within each treatment arm are analysed.

Non-randomized trial study design

Non-randomized trials are interventional study designs that compare a group where an intervention was performed with a group where there was no intervention. These are convenient study designs that are most often performed prospectively and can suggest possible relationships between the intervention and the outcome. However, these study designs are often subject to many types of bias and error and are not considered a strong study design.

Randomized controlled trial study design

Randomized controlled trials (RCTs) are the most common type of interventional study, and can have many modifications ( 26 – 28 ). These trials take a homogenous group of study participants and randomly divide them into two separate groups. If the randomization is successful then these two groups should be the same in all respects, both measured confounders and unmeasured factors. The intervention is then implemented in one group and not the other and comparisons of intervention efficacy between the two groups are analysed. Theoretically, the only difference between the two groups through the entire study is the intervention. An excellent example is the intervention of a new medication to treat a specific disease among a group of patients. This randomization process is arguably the largest strength of an RCT ( 26 – 28 ). Additional methodological elements are utilized among RCTs to further strengthen the causal implication of the intervention’s impact. These include allocation concealment, blinding, measuring compliance, controlling for co-interventions, measuring dropout, analysing results by intention to treat, and assessing each treatment arm at the same time point in the same manner.

Crossover randomized controlled trial study design

A crossover RCT is a type of interventional study design where study participants intentionally “crossover” to the other treatment arm. This should not be confused with the observational case-crossover design. A crossover RCT begins the same as a traditional RCT, however, after the end of the first treatment phase, each participant is re-allocated to the other treatment arm. There is often a wash-out period in between treatment periods. This design has many strengths, including demonstrating reversibility, compensating for unsuccessful randomization, and improving study efficiency by not using time to recruit subjects.

Allocation concealment theoretically guarantees that the implementation of the randomization is free from bias. This is done by ensuring that the randomization scheme is concealed from all individuals involved ( 26 – 30 ). A third party who is not involved in the treatment or assessment of the trial creates the randomization schema and study participants are randomized according to that schema. By concealing the schema, there is a minimization of potential deviation from that randomization, either consciously or otherwise by the participant, researcher, provider, or assessor. The traditional method of allocation concealment relies upon sequentially numbered opaque envelopes with the treatment allocation inside. These envelopes are generated before the study begins using the selected randomization scheme. Participants are then allocated to the specific intervention arm in the pre-determined order dictated by the schema. If allocation concealment is not utilized, there is the possibility of selective enrolment into an intervention arm, potentially with the outcome of biased results.

Blinding in an RCT is withholding the treatment arm from individuals involved in the study. This can be done through use of placebo pills, deactivated treatment modalities, or sham therapy. Sham therapy is a comparison procedure or treatment which is identical to the investigational intervention except it omits a key therapeutic element, thus rendering the treatment ineffective. An example is a sham cortisone injection, where saline solution of the same volume is injected instead of cortisone. This helps ensure that patients do not know if they are receiving the active or control treatment. The process of blinding is utilized to help ensure equal treatment of the different groups, therefore continuing to isolate the difference in outcome between groups to only the intervention being administered ( 28 – 31 ). Blinding within an RCT includes patient blinding, provider blinding, or assessor blinding. In some situations it is difficult or impossible to blind one or more of the parties involved, but an ideal study would have all parties blinded until the end of the study ( 26 – 28 , 31 , 32 ).

Compliance is the degree of how well study participants adhere to the prescribed intervention. Compliance or non-compliance to the intervention can have a significant impact on the results of the study ( 26 – 29 ). If there is a differentiation in the compliance between intervention arms, that differential can mask true differences, or erroneously conclude that there are differences between the groups when one does not exist. The measurement of compliance in studies addresses the potential for differences observed in intervention arms due to intervention adherence, and can allow for partial control of differences either through post hoc stratification or statistical adjustment.

Co-interventions, interventions that impact the outcome other than the primary intervention of the study, can also allow for erroneous conclusions in clinical trials ( 26 – 28 ). If there are differences between treatment arms in the amount or type of additional therapeutic elements then the study conclusions may be incorrect ( 29 ). For example, if a placebo treatment arm utilizes more over-the-counter medication than the experimental treatment arm, both treatment arms may have the same therapeutic improvement and show no effect of the experimental treatment. However, the placebo arm improvement is due to the over-the-counter medication and if that was prohibited, there may be a therapeutic difference between the two treatment arms. The exclusion or tracking and statistical adjustment of co-interventions serves to strengthen an RCT by minimizing this potential effect.

Participants drop out of a study for multiple reasons, but if there are differential dropout rates between intervention arms or high overall dropout rates, there may be biased data or erroneous study conclusions ( 26 – 28 ). A commonly accepted dropout rate is 20% however, studies with dropout rates below 20% may have erroneous conclusions ( 29 ). Common methods for minimizing dropout include incentivizing study participation or short study duration, however, these may also lead to lack of generalizability or validity.

Intention-to-treat (ITT) analysis is a method of analysis that quantitatively addresses deviations from random allocation ( 26 – 28 ). This method analyses individuals based on their allocated intervention, regardless of whether or not that intervention was actually received due to protocol deviations, compliance concerns or subsequent withdrawal. By maintaining individuals in their allocated intervention for analyses, the benefits of randomization will be captured ( 18 , 26 – 29 ). If analysis of actual treatment is solely relied upon, then some of the theoretical benefits of randomization may be lost. This analysis method relies on complete data. There are different approaches regarding the handling of missing data and no consensus has been put forth in the literature. Common approaches are imputation or carrying forward the last observed data from individuals to address issues of missing data ( 18 , 19 ).

Assessment timing can play an important role in the impact of interventions, particularly if intervention effects are acute and short lived ( 26 – 29 , 33 ). The specific timing of assessments are unique to each intervention, however, studies that allow for meaningfully different timing of assessments are subject to erroneous results. For example, if assessments occur differentially after an injection of a particularly fast acting, short-lived medication the difference observed between intervention arms may be due to a higher proportion of participants in one intervention arm being assessed hours after the intervention instead of minutes. By tracking differences in assessment times, researchers can address the potential scope of this problem, and try to address it using statistical or other methods ( 26 – 28 , 33 ).

Randomized controlled trials are the principle method for improving treatment of disease, and there are some standardized methods for grading RCTs, and subsequently creating best practice guidelines ( 29 , 34 – 36 ). Much of the current practice of medicine lacks moderate or high quality RCTs to address what treatment methods have demonstrated efficacy and much of the best practice guidelines remains based on consensus from experts ( 28 , 37 ). The reliance on high quality methodology in all types of studies will allow for continued improvement in the assessment of causal factors for health outcomes and the treatment of diseases.

Standards of research and reporting

There are many published standards for the design, execution and reporting of biomedical research, which can be found in Table 3 . The purpose and content of these standards and guidelines are to improve the quality of biomedical research which will result in providing sound conclusions to base medical decision making upon. There are published standards for categories of study designs such as observational studies (e.g. STROBE), interventional studies (e.g. CONSORT), diagnostic studies (e.g. STARD, QUADAS), systematic reviews and meta-analyses (e.g. PRISMA ), as well as others. The aim of these standards and guideline are to systematize and elevate the quality of biomedical research design, execution, and reporting.

Published standard for study design and reporting.

  • Consolidated Standards Of Reporting Trials (CONSORT, www.consort-statement.org ) are interventional study standards, a 25 item checklist and flowchart specifically designed for RCTs to standardize reporting of key elements including design, analysis and interpretation of the RCT.
  • Strengthening the Reporting of Observational studies in Epidemiology (STROBE, www.strobe-statement.org ) is a collection of guidelines specifically for standardization and improvement of the reporting of observational epidemiological research. There are specific subsets of the STROBE statement including molecular epidemiology (STROBE-ME), infectious diseases (STROBE-ID) and genetic association studies (STREGA).
  • Standards for Reporting Studies of Diagnos tic Accuracy (STARD, www.stard-statement.org ) is a 25 element checklist and flow diagram specifically designed for the reporting of diagnostic accuracy studies.
  • Quality assessment of diagnostic accuracy studies (QUADAS, www.bris.ac.uk/quadas ) is a quality assessment of diagnostic accuracy studies.
  • Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA, www.prisma-statement.org ) is a 27 element checklist and multiphase flow diagram to improve quality of reporting systematic reviews and meta-analyses. It replaces the QUOROM statement.
  • Consolidated criteria for reporting qualitative research (COREQ) is a 32 element checklist designed for reporting of qualitative data from interviews and focus groups.
  • Statistical Analyses and Methods in the Published Literature (SAMPL) is a guideline for statistical methods and analyses of all types of biomedical research.
  • Consensus-based Clinical Case Reporting Guideline Development (CARE, www.carestatement.org ) is a checklist comprised of 13 elements and is designed only for case reports.
  • Standards for Quality Improvement Reporting Excellence (SQUIRE, www.squire-statement.org ) are publication guidelines comprised of 19 elements, for authors aimed at quality improvement in health care reporting.
  • Consolidated Health Economic Evaluation Reporting Standards (CHEERS) is a 24 element checklist of reporting practices for economic evaluations of interventional studies.
  • Enhancing transparency in reporting the synthesis of qualitative research (ENTREQ) is a guideline specifically for standardizing and improving the reporting of qualitative biomedical research.

When designing or evaluating a study it may be helpful to review the applicable standards prior to executing and publishing the study. All published standards and guidelines are available on the web, and are updated based on current best practices as biomedical research evolves. Additionally, there is a network called “Enhancing the quality and transparency of health research” (EQUATOR, www.equator-network.org ) , which has guidelines and checklists for all standards reported in Table 3 and is continually updated with new study design or specialty specific standards.

The appropriate selection of a study design is only one element in successful research. The selection of a study design should incorporate consideration of costs, access to cases, identification of the exposure, the epidemiologic measures that are required, and the level of evidence that is currently published regarding the specific exposure-outcome relationship that is being assessed. Reviewing appropriate published standards when designing a study can substantially strengthen the execution and interpretation of study results.

Potential conflict of interest

None declared.

Webinar: Educational Intervention in Palestine, Afghanistan, South Africa, and Myanmar, 28 February 2pm

Joseph North

  • Share this with Facebook
  • Share this with Twitter
  • Share this with Facebook Messenger
  • Share this with LinkedIn
  • Share this with Email

Educational Intervention in Vulnerable Contexts: Case Studies in Palestine, Afghanistan, South Africa, and Myanmar

This webinar focuses on the well-being of women, children, and refugees. This webinar  explores the impact of educational intervention programs in mitigating the adverse effects of challenging living conditions on these people.  These include: how exposure to conflict-related events affects child behavior in Palestine; the impact of non-physical disciplinary methods on primary school dropout rates in Afghanistan; a community-based model of collective filmmaking pedagogy that familiarizes local youth with accessible animation techniques in South Africa; and bridging gaps in fragile contexts through online courses to further education opportunities in Myanmar.

The webinar is on 28 February, 2pm – 3:30pm UK time. Sign up here .

This webinar is part of the Amplifying the Voices of Engaged Researchers initiative, organised by the Talloires Network and Open Society University Network. The University is a member of the Talloires Network.

Share this story

Cookie preferences

IMAGES

  1. single case study of intervention

    single case study of intervention

  2. single case study of intervention

    single case study of intervention

  3. single case intervention research design standards

    single case study of intervention

  4. single case study of intervention

    single case study of intervention

  5. single case study of intervention

    single case study of intervention

  6. single case intervention research design standards

    single case study of intervention

VIDEO

  1. Clinical Case Scenarios 5

  2. Clinical Case Scenarios 6

  3. OT Case Study Intervention

COMMENTS

  1. Single-Case Design, Analysis, and Quality Assessment for Intervention Research

    The What Works Clearinghouse (WWC) single-case design technical documentation provides an excellent overview of appropriate SC study analysis techniques to evaluate the effectiveness of intervention effects. 1,18 First, visual analyses are recommended to determine whether there is a functional relation between the intervention and the outcome ...

  2. Single-Case Intervention Research

    A well-written and meaningfully structured compendium that includes the foundational and advanced guidelines for conducting accurate single-case intervention designs. Whether you are an undergraduate or a graduate student, or an applied researcher anywhere along the novice-to-expert column, this book promises to be an invaluable addition to ...

  3. Single-Case Design, Analysis, and Quality Assessment for Intervention

    When rigorously designed, single-case studies can be particularly useful experimental designs in a variety of situations, such as when research resources are limited, studied conditions have low incidences, or when examining effects of novel or expensive interventions. ... Single-Case Design, Analysis, and Quality Assessment for Intervention ...

  4. Single-case experimental designs to assess intervention effectiveness

    Single-case experimental designs (SCED) are experimental designs aiming at testing the effect of an intervention using a small number of patients (typically one to three), using repeated measurements, sequential (± randomized) introduction of an intervention and method-specific data analysis, including visual analysis and specific statistics.The aim of this paper is to familiarise ...

  5. Optimizing behavioral health interventions with single-case designs

    Given that the unit of analysis is each case (i.e., participant), a single study could be conceptualized as a series of single-case experiments. ... between baseline and intervention conditions) in a single-case experiment. The mere presence of variability does not mean that a single-case approach should be abandoned, however. Indeed ...

  6. Randomized Single-Case Intervention Designs and Analyses for Health

    Within-Case Intervention-Order Randomization . This form of randomization is implemented when each case is to receive both A (Baseline or Placebo) and B (Intervention) phases or B (Intervention 1) and C (Intervention 2) phases, in two- or multiple-phase crossover designs and in single-case "alternating treatment" designs.

  7. The Single-Case Reporting Guideline In BEhavioural Interventions

    Single-case experimental design (SCED) studies in the behavioral sciences literature are not only common, but their proportion has also increased over past decades. ... (2011) surveyed the contents of 21 journals in psychology and education for the calendar year 2008 and found that 44% of intervention studies used single-case methods. Similarly ...

  8. Single-Case Intervention Research Design Standards

    Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods, 15, 122-144. Crossref. ISI. Google Scholar. ... Outcomes of a novel single case study incorporating Rapid Syllable Tra... Go to citation Crossref Google Scholar.

  9. Single‐case experimental designs: Characteristics, changes, and

    Tactics of Scientific Research (Sidman, 1960) provides a visionary treatise on single-case designs, their scientific underpinnings, and their critical role in understanding behavior. Since the foundational base was provided, single-case designs have proliferated especially in areas of application where they have been used to evaluate interventions with an extraordinary range of clients ...

  10. Characteristics of single-case designs used to assess intervention

    This article reports the results of a study that located, digitized, and coded all 809 single-case designs appearing in 113 studies in the year 2008 in 21 journals in a variety of fields in psychology and education. Coded variables included the specific kind of design, number of cases per study, number of outcomes, data points and phases per case, and autocorrelations for each case. Although ...

  11. PDF Single-Case Design Research Methods

    Studies that use a single-case design (SCD) measure outcomes for cases (such as a child or family) repeatedly during multiple phases of a study to determine the success of an intervention. The number of phases in the study will depend on the research questions, intervention, and outcome(s) of interest (see Types of SCDs on page 4 for examples).

  12. Single-case intervention research design standards: Additional proposed

    Single-case intervention research design standards have evolved considerably over the past decade. These standards serve the dual role of assisting in single-case design (SCD) intervention research methodology and as guidelines for literature syntheses within a particular research domain. ... Several examples of SCD intervention studies that ...

  13. Single case studies are a powerful tool for developing ...

    The unique insights that single case studies have provided illustrate the value of in-depth investigation within an individual. Single case methodology has an important place in the psychologist ...

  14. Single-Case Experimental Designs

    Single-case experimental designs are a family of experimental designs that are characterized by researcher manipulation of an independent variable and repeated measurement of a dependent variable before (i.e., baseline) and after (i.e., intervention phase) introducing the independent variable. In single-case experimental designs a case is the ...

  15. Single-Case Intervention Research: Methodological and ...

    Single-case intervention research has a rich tradition of providing evidence about the efficacy of interventions applied both to solving a diverse range of human problems and to enriching the knowledge base established in many fields of science (Kratochwill, 1978; Kratochwill & Levin, 1992, 2010). In the social sciences the randomized ...

  16. Single-Case Design Interventions

    Single-case designs are a class of experimental research methodology that involves conducting investigations with one subject or one group (Kazdin 2011).According to Kratochwill et al. (), studies that utilize a single-case design include "repeated, systematic measurement of a dependent variable before, during, and after active manipulation of an independent variable" (Kratochwill et al ...

  17. Single-Case Reporting Guideline In BEhavioural Interventions (SCRIBE

    Single-case methodology is defined as the intensive and prospective study of the individual in which (a) the intervention/s is manipulated in an experimentally controlled manner across a series of discrete phases, and (b) measurement of the behavior targeted by the intervention is made repeatedly (and, ideally, frequently) throughout all phases.

  18. Chapter 18 Single case designs

    Chapter 18 Single case designs. The single case design, also known as N-of-1 trial, or small N design, is a commonly used intervention design in speech and language therapy, clinical psychology, education, and neuropsychology, including aphasia therapy (Perdices & Tate, 2009).The single case design may be regarded as an extreme version of a within-subjects design, where two more more ...

  19. Randomized Single-Case Intervention Designs and Analyses for Health

    Aim An expository note introduces health sciences researchers to randomized single-case intervention designs, an adaptation of interrupted time-series methodology, and the staple of a scientifically credible small-sample research paradigm. Methods Detailed examples illustrating two different randomized single-case procedures are presented to highlight the techniques' advantages relative to ...

  20. A Description of Missing Data in Single-Case Experimental Designs

    Performing analyses without considering missing data in an intervention study using SCEDs or a meta-analysis study including SCEDs studies in a topic can lead to biased results and affect the validity of individual or overall results. ... Dealing with missing data by EM in single-case studies. Behavior Research Methods, 52, 131-150. https ...

  21. Single-Case Design, Analysis, and Quality Assessment for Intervention

    The What Works Clearinghouse (WWC) single-case design technical documentation provides an excellent overview of appropriate SC study analysis techniques to evaluate the effectiveness of intervention effects. 1, 18 First, visual analyses are recommended to determine whether there is a functional relationship between the intervention and the ...

  22. PDF Case-Control Study

    LEGEND: Evidence Appraisal of a Single Study Intervention Case-Control Study Development for this appraisal form is based on: 1. Guyatt, G.; Rennie, D.; Evidence-Based Medicine Working Group.; and American Medical Association.: Users' guides to the medical literature : a manual for evidence-based clinical practice.

  23. 'It depends': what 86 systematic reviews tell us about what strategies

    To reduce the risk of bias, papers were excluded following discussion between all members of the team. Data were synthesised using descriptive and narrative techniques to identify themes and patterns linked to intervention strategies, targeted behaviours, study settings and study outcomes. We identified 32 reviews conducted between 2010 and 2022.

  24. A Systematic Review of Single-Case Research Examining ...

    The purpose of this review was to examine single-case behavior intervention studies that used the cultural background of students to inform intervention design, which may hold promise for preventing disproportionate use of exclusionary school discipline. Specifically, this literature review sought to identify (a) what elements of cultural ...

  25. PDF LEGEND: Evidence Appraisal of a Single Study Intervention Randomized

    LEGEND: Evidence Appraisal of a Single Study Intervention Randomized Controlled Trial (RCT) or Controlled Clinical Trial (CCT) Table of Evidence Levels DOMAIN OF CLINICAL QUESTION - TYPE OF STUDY / STUDY DESIGN w s a - s T + udy onal ohort - e ohort e - ol nal) - udy ogy s on) udy s nes s N-of-udy dy on nsus s Intervention 1a 1b 2a 2b ...

  26. Observational and interventional study design types; an overview

    Case-control studies were traditionally referred to as retrospective studies, ... Pre-post studies may be single arm, one group measured before the intervention and again after the intervention, or multiple arms, where there is a comparison between groups. ... Outcomes measured for pre-post intervention studies may be binary health outcomes ...

  27. Integrating large language models in systematic reviews: a framework

    The case study demonstrated that raw per cent agreement was the highest for the ROBINS-I domain of 'Classification of Intervention'. Kendall agreement coefficient was highest for the domains of 'Participant Selection', 'Missing Data' and 'Measurement of Outcomes', suggesting moderate agreement in these domains.

  28. Webinar: Educational Intervention in Palestine, Afghanistan, South

    Educational Intervention in Vulnerable Contexts: Case Studies in Palestine, Afghanistan, South Africa, and Myanmar This webinar focuses on the well-being of women, children, and refugees. This webinar explores the impact of educational intervention programs in mitigating the adverse effects of challenging living conditions on these people.