presentation of clinical outcomes

Jump to navigation

Cochrane Training

Chapter 18: patient-reported outcomes.

Bradley C Johnston, Donald L Patrick, Tahira Devji, Lara J Maxwell, Clifton O Bingham III, Dorcas E Beaton, Maarten Boers, Matthias Briel, Jason W Busse, Alonso Carrasco-Labra, Robin Christensen, Bruno R da Costa, Regina El Dib, Anne Lyddiatt, Raymond W Ostelo, Beverley Shea, Jasvinder Singh, Caroline B Terwee, Paula R Williamson, Joel J Gagnier, Peter Tugwell, Gordon H Guyatt

Key Points:

• Summary data on patient-reported outcomes (PROs) are important to ensure healthcare decision makers are informed about the outcomes most meaningful to patients.
• Authors of systematic reviews that include PROs should have a good understanding of how patient-reported outcome measures (PROMs) are developed, including the constructs they are intended to measure, their reliability, validity and responsiveness.
• Authors should pre-specify at the protocol stage a hierarchy of preferred PROMs to measure the outcomes of interest.

Cite this chapter as: Johnston BC, Patrick DL, Devji T, Maxwell LJ, Bingham III CO, Beaton D, Boers M, Briel M, Busse JW, Carrasco-Labra A, Christensen R, da Costa BR, El Dib R, Lyddiatt A, Ostelo RW, Shea B, Singh J, Terwee CB, Williamson PR, Gagnier JJ, Tugwell P, Guyatt GH. Chapter 18: Patient-reported outcomes. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook .

18.1 Introduction to patient-reported outcomes

18.1.1 what are patient-reported outcomes.

A patient-reported outcome (PRO) is “any report of the status of a patient’s health condition that comes directly from the patient without interpretation of the patient’s response by a clinician or anyone else” (FDA 2009). PROs are one of several clinical outcome assessment methods that complement biomarkers, measures of morbidity (e.g. stroke, myocardial infarction), burden (e.g. hospitalization), and survival used and reported in clinical trials and non-randomized studies (FDA 2018).

Patient-reported outcome measures (PROMs) are instruments that are used to measure the PROs, most often self-report questionnaires. Although investigators may address patient-relevant outcomes via proxy reports or observations from caregivers, health professionals, or parents and guardians, these are not PROMs but rather clinician-reported or observer-reported outcomes (Powers et al 2017).

PROs provide crucial information for patients and clinicians facing choices in health care. Conducting systematic reviews and meta-analyses including PROMs and interpreting their results is not straightforward, and guidance can help review authors address the challenges.

The objectives of this chapter are to: (i) describe the category of outcomes known as PROs and their importance for healthcare decision making; (ii) illustrate the key issues related to reliability, validity and responsiveness that systematic review authors should consider when including PROs; and (iii) address the structure and content (domains, items) of PROs and provide guidance for combining information from different PROs. This chapter outlines a step-by-step approach to addressing each of these elements in the systematic review process. The focus is on the use of PROs in randomized trials, and what is crucial in this context when selecting PROs to include in a meta-analysis. The principles also apply to systematic reviews of non-randomized studies addressing PROs (e.g. dealing with adverse drug reactions).

18.1.2 Why patient-reported outcomes?

PROs provide patients’ perspectives regarding treatment benefit and harm, directly measure treatment benefit and harm beyond survival, major morbid events and biomarkers, and are often the outcomes of most importance to patients and families.

Self-reported outcomes often correlate poorly with physiological and other outcomes such as performance-related outcomes, clinician-reported outcomes, or biomarkers. In asthma, Yohannes and colleagues (Yohannes et al (1998) found that variability in exercise capacity contributed to only 3% of the variability in breathing problems on a patient self-report questionnaire. In chronic obstructive pulmonary disease (COPD), the reported correlations between forced expiratory volume (FEV1) and quality of life (QoL) are weak (r=0.14 to 0.41) (Jones 2001). In peripheral arterial occlusive disease, correlations between haemodynamic variables and QoL are low (e.g. r=–0.17 for QoL pain subscale and Doppler sonographic ankle/brachial pressure index) (Müller-Bühl et al 2003). In osteoarthritis, there is discordance between radiographic arthritis and patient-reported pain (Hannan et al 2000). These findings emphasize the often important limitations of biomarkers for informing the impact of interventions on the patient experience or the patient’s perspective of disease (Bucher et al 2014).

PROs are essential when externally observable patient-important outcomes are rare or unavailable. They provide the only reasonable strategy for evaluating treatment impact of many conditions including pain syndromes, fatigue, disorders such as irritable bowel syndrome, sexual dysfunction, and emotional function and adverse effects such as nausea and anxiety for which physiological measurements are limited or unavailable.

18.2 Formulation of the review

In this section we describe PROMs in more detail and discuss some issues to consider when deciding which PROMs to address in a review.

A common term used in the health status measurement literature is construct . Construct refers to what PROMs are trying to measure, the concept that defines the PROM such as pain, physical function or depressive mood. Constructs are the postulated attributes of the person that investigators hope to capture with the PROM (Cronbach and Meehl 1955).

Many different ways exist to label and classify PROMs and the constructs they measure. For instance, reports from patients include signs (observable manifestations of a condition), sensations (most commonly classified as symptoms that may be attributable to disease and/or treatment), behaviours and abilities (commonly classified as functional status), general perceptions or feelings of well-being, general health, satisfaction with treatment, reports of adverse effects, adherence to treatment, and participation in social or community events and health-related quality of life (HRQoL).

Investigators can use different approaches to capture patient perspectives, including interviews, self-completed questionnaires, diaries, and via different interfaces such as hand-held devices or computers. Review authors must identify the postulated constructs that are important to patients, and then determine the extent to which the PROMs used and reported in the trials address those constructs, the characteristics (measurement properties) of the PROMs used, and communicate this information to the reader (Calvert et al 2013).

Focusing now on HRQoL, an important PRO, some approaches attempt to cover the full range of health-related patient experience – including, for instance, self-care, and physical, emotional and social function – and thus enable comparisons between the impact of treatments on HRQoL across diseases or conditions. Authors often call these approaches generic instruments (Guyatt et al 1989, Patrick and Deyo 1989). These include utility measures such as the EuroQol five dimensions questionnaire (EQ-5D) or the Health Utilities Index (HUI). They also include health profiles such as the Short Form 36-item (SF-36) or the SF-12; these have come to dominate the field of health profiles (Tarlov et al 1989, Ware et al 1995, Ware et al 1996). An alternative approach to measuring PROs is to focus on much more specific constructs: PROMs may be specific to function (e.g. sleep, sexual function), to a disease (e.g. asthma, heart failure), to a population (e.g. the frail elderly) or to a symptom (pain, fatigue) (Guyatt et al 1989, Patrick and Deyo 1989). Another domain-specific measurement system now receiving attention is Patient-Reported Outcomes Measurement Instruments System (PROMIS). PROMIS is a National Institutes of Health funded PROM programme using computerized adaptive testing from large item banks for over 70 domains (e.g. anxiety, depression, pain, social function) relevant to wide variety of chronic diseases (Cella et al 2007, Witter 2016, PROMIS 2018).

Authors often use the terms ‘quality of life’, ‘health status’, ‘functional status’, ‘HRQoL’ and ‘well-being’ loosely and interchangeably. Systematic review authors must therefore consider carefully the constructs that the PROMs have actually measured. To do so, they may need to examine the items or questions included in a PROM.

Another issue to consider is whether and how the individual items of instruments are weighted. A number of approaches can be used to arrive at weights (Wainer 1976). Utility instruments designed for economic analysis put greater emphasis on item weighting, attempting ultimately to present HRQoL as a continuum anchored between death and full health. Many PROMs weight items equally in the calculation of the overall score, a reasonable approach. Readers can refer to a helpful overview of classical test theory and item response theory to understand better the merits and limitations of weighting (Cappelleri et al 2014).

Table 18.2.a presents a framework for considering and reporting PROMs in clinical trials, including their constructs and how they were measured. A good understanding of the PROMs identified in the included studies for a review is essential to appropriate analysis of outcomes across studies, and appraisal of the certainty of the evidence.

Table 18.2.a Checklist for describing and assessing PROMs in clinical trials. Adapted from Guyatt et al (1997)

18.3 Appraisal of evidence

18.3.1 measurement of pros: single versus multiple time-points.

To be useful, instruments must be able to distinguish between situations of interest (Boers et al 1998). When results are available for only one time-point (e.g. for classification), the key issue for PROMs is to be able to distinguish individuals with more desirable scores from those whose scores are less desirable. The key measurement issues in such contexts are reliability and cross-sectional construct validity (Kirshner and Guyatt 1985, Beaton et al 2016).

In longitudinal studies such as randomized trials, investigators usually obtain measurements at multiple time-points, for example at the beginning of the trial and again following administration of the interventions. In this context, PROMs must be able to distinguish those who have experienced positive changes over time from those who have experienced negative changes, those who experienced less positive change, or those who experienced no change at all, and to estimate accurately the magnitude of those changes. The key measurement issues in these contexts – sometimes referred to as evaluative – are responsiveness and longitudinal construct validity (Kirshner and Guyatt 1985, Beaton et al 2016).

18.3.2 Reliability

Intuitively, many think of reliability as obtaining the same scores on repeated administration of an instrument in stable respondents. That stability (or lack of measurement error) is important, but not sufficient. Satisfactory instruments must be able to distinguish between individuals despite measurement error.

Reliability statistics therefore look at the ratio of the variability between respondents (typically the numerator of a reliability statistic) and the total variability (the variability between respondents and the variability within respondents). The most commonly used statistics to measure reliability is a kappa coefficient for categorical data, a weighted kappa coefficient for ordered categorical data, and an intraclass correlation coefficient for continuous data (de Vet et al 2011).

Limitations in reliability will be of most concern for the review author when randomized trials have failed to establish the superiority of an experimental intervention over a comparator intervention. The reason is that lack of reliability cannot create intervention effects that are not present, but can obscure true intervention effects as a result of random error. When a systematic review does not find evidence that an intervention affects a PROM, review authors should consider whether this may be due to poor reliability (e.g. if reliability coefficients are less than 0.7) rather than lack of an effect.

18.3.3 Validity

Validity has to do with whether the instrument is measuring what it is intended to measure. Content validity assessment involves patient and clinician evaluation of the relevance and comprehensiveness of the content contained in the measures, usually obtained through qualitative research with patients and families (Johnston et al 2012). Guidance is available on the assessment of content validity for PROMs used in clinical trials (Patrick et al 2011a, Patrick et al 2011b).

Construct validity involves examining the logical relationships that should exist between assessment measures. For example, in patients with COPD, we would expect that patients with lower treadmill exercise capacity generally will have more dyspnoea (shortness of breath) in daily life than those with higher exercise capacity, and we would expect to see substantial correlations between a new measure of emotional function and existing emotional function questionnaires.

When we are interested in evaluating change over time – that is, in the context of evaluation when measures are available both before and after an intervention – we examine correlations of change scores. For example, patients with COPD who deteriorate in their treadmill exercise capacity should, in general, show increases in dyspnea, while those whose exercise capacity improves should experience less dyspnea. Similarly, a new emotional function instrument should show concurrent improvement in patients who improve on existing measures of emotional function. The technical term for this process is testing an instrument’s longitudinal construct validity. Review authors should look for evidence of the validity of PROMs used in clinical studies. Unfortunately, reports of randomized trials using PROMs seldom review or report evidence of the validity of the instruments they use, but when these are available review authors can gain some reassurance from statements (backed by citations) that the questionnaires have been previously validated, or could seek additional published information on named PROMs. Ideally, review authors should look for systematic reviews of the measurement properties of the instruments in question. The Co nsensus-based s tandards for the selection of health m easurement in struments (COSMIN) website offers a database of such reviews ( COSMIN Database of Systematic Reviews ). In addition, the Patient-Reported Outcomes and Quality of Life Instruments Database ( PROQOLID ) provides documentation of the measurement properties for over 1000 PROs.

If the validity of the PROMs used in a systematic review remains unclear, review authors should consider whether the PROM is an appropriate measure of the review’s planned outcomes, or whether it should be excluded (ideally, this would be considered at the protocol stage), and any included results should be interpreted with appropriate caution. For instance, in a review of flavonoids for haemorrhoids, authors of primary trials used PROMs to ascertain patients’ experience with pain and bleeding (Alonso-Coello et al 2006). Although the wording of these PROMs was simple and made intuitive sense, the absence of formal validation raises concerns over whether these measures can give meaningful data to distinguish between the intervention and its comparators.

A final concern about validity arises if the measurement instrument is used with a different population, or in a culturally and linguistically different environment from the one in which it was developed. Ideally, PROMs should be re-validated in each study, but systematic review authors should be careful not to be too critical on this basis alone .

18.3.4 Responsiveness

In the evaluative context, randomized trial participant measurements are typically available before and after the intervention. PROMs must therefore be able to distinguish among patients who remain the same, improve or deteriorate over the course of the trial (Guyatt et al 1987, Revicki et al 2008). Authors often refer to this measurement property as responsiveness; alternatives are sensitivity to change or ability to detect change.

As with reliability, responsiveness becomes an issue when a meta-analysis suggests no evidence of a difference between an intervention and control. An instrument with a poor ability to measure change can result in false-negative results, in which the intervention improves how patients feel, yet the instrument fails to detect the improvement. This problem may be particularly salient for generic questionnaires that have the advantage of covering all relevant areas of HRQoL, but the disadvantage of covering each area superficially or without the detail required for the particular context of use (Wiebe et al 2003, Johnston et al 2016a). Thus, in studies that show no difference in PROMs between intervention and control, lack of instrument responsiveness is one possible reason. Review authors should look for published evidence of responsiveness. If there is an absence of prior evidence of responsiveness, this represents a potential reason for being less certain about evidence from a series of randomized trials. For instance, a systematic review of respiratory muscle training in COPD found no effect on patients’ function. However, two of the four studies that assessed a PROM used instruments without established responsiveness (Smith et al 1992).

18.3.5 Reporting bias

Studies focusing on PROs often use a number of PROMs to measure the same or similar constructs. This situation creates a risk of selective outcome reporting bias, in which trial authors select for publication a subset of the PROMs on the basis of the results; that is, those that indicate larger intervention effects or statistically significant P values (Kirkham et al 2010). Further detailed discussion of selective outcome reporting is presented in Chapter 7 (Section 7.2.3.3) ; see also Chapter 8 (Section 8.7) .

Systematic reviews focusing on PROs should be alert to this problem. When only a small number of eligible studies have reported results for a particular PROM, particularly if the PROM is mentioned in a study protocol or methods section, or if it is a salient outcome that one would expect conscientious investigators to measure, review authors should note the possibility of reporting bias and consider rating down certainty in evidence as part of their GRADE assessment (see Chapter 14 ) (Guyatt et al 2011). For instance, authors of a systematic review evaluating the responsiveness of PROs among patients with rare lysosomal storage diseases encountered eligible studies in which the use of a PRO was described in the methods, but there were either no data or limited PRO data in the results. When authors did present some information about results, the reports sometimes included only interim or end-of-study results. Such instances are likely to be an indication of selective outcome reporting bias: it seems implausible that, if results showed apparent benefit on PROs, investigators would mention a PRO in the methods and subsequently fail to report results (Johnston et al 2016b).

18.4 Synthesis and interpretation of evidence

18.4.1 selecting from multiple proms.

The definition of a particular PRO may vary between studies, and this may justify use of different instruments (i.e. different PROMs). Even if the definitions are similar (or if, as happens more commonly, the investigators do not define the PRO), the investigators may choose different instruments to measure the PROs, especially if there is a lack of consensus on which instrument to use (Prinsen et al 2016).

When trials report results for more than one instrument, authors should – independent of knowledge of the results and ideally at the protocol stage – create a hierarchy based on reported measurement properties of PROMs (Tendal et al 2011, Christensen et al 2015), considering a detailed understanding of what each PROM measures (see Table 18.2.a ), and its demonstrated reliability, validity, responsiveness and interpretability (see Section 18.3 ). This will allow authors to decide which instruments will be used for data extraction and synthesis. For example, the following instruments are all validated, patient-reported pain instruments that an investigator may use in a primary study to assess an intervention’s usefulness for treating pain:

• 7-item Integrated Pain Score;
• 10-point Visual Analogue Scale for Pain;
• 20-item McGill Pain Questionnaire; and
• 56-item Brief Pain Inventory (PROQOLID 2018).

In some clinical fields core outcome sets are available to guide the use of appropriate PROs (COMET 2018). Only rarely do these include specific guidance on which PROMs are preferable, although methods have been proposed for this (Prinsen et al 2016). Within the field of rheumatology, the Outcome Measures in Rheumatology (OMERACT) initiative has developed a conceptual framework known as OMERACT Filter 2.0 to identify both core domain sets (what outcome should be measured) and core outcome measurement sets (how the outcome should be measured, i.e. which PROM to use) (Boers et al 2014). This is a generic framework and applicable to those developing core outcome sets outside the field of rheumatology.

As an example of a pre-defined hierarchy, for knee osteoarthritis, OMERACT has used a published hierarchy based on responsiveness for extraction of PROMs evaluating pain and physical function for performing systematic reviews (Juhl et al 2012).

Authors should decide in advance whether to exclude PROMs not included in the hierarchy, or to include additional measures where none of the preferred measures are available.

18.4.2 Synthesizing data from multiple PROMs

While a hierarchy can be helpful in identifying the review authors’ preferred measures, and excluding some measures considered inappropriate, it remains likely that authors will encounter studies using several different PROMs to measure a given construct, either within one study or across multiple studies. Authors must then decide how to approach synthesis of multiple measures, and among them, consider which measures to include in a single meta-analysis on a particular construct (Tendal et al 2011, Christensen et al 2015).

When deciding if statistical synthesis is appropriate, review authors will often find themselves reading between the lines to try and get a precise notion of the underlying construct for the PROMs used. They may have to consult the articles that describe the development and prior use of PROMs included in the primary studies, or look at the instruments to understand the concepts being measured.

For example, authors of a Cochrane Review of cognitive behavioural therapy (CBT) for tinnitus included HRQoL as a PRO (Martinez-Devesa et al 2007), assessed with different PROMs: four trials using the Tinnitus Handicap Questionnaire; one trial the Tinnitus Questionnaire; and one trial the Tinnitus Reaction Questionnaire. Review authors compared the content of the PROMs and concluded that statistical pooling was appropriate.

The most compelling evidence regarding the appropriateness of including different PROMs in the same meta-analysis would come from a finding of substantial correlations between the instruments. For example, the two major instruments used to measure HRQoL in patients with COPD are the Chronic Respiratory Questionnaire (CRQ) and the St. George’s Respiratory Questionnaire (SGRQ). Correlations between the two questionnaires in individual studies have varied from 0.3 to 0.6 in both cross-sectional (correlations at a point in time) and longitudinal (correlations of change) comparisons (Rutten-van Mölken et al 1999, Singh et al 2001, Schünemann et al 2003, Schünemann et al 2005). In one study, investigators examined the correlations between group mean changes in the CRQ and SGRQ in 15 studies including 23 patient groups and found a correlation of 0.88 (Puhan et al 2006).

Ideally, the decision to combine scores from different PROMs would be based not only on their measuring similar constructs but also on their satisfactory validity, and, depending on whether before and after intervention or only after intervention measurements were available, and on their responsiveness or reliability. For example, extensive evidence of validity is available for both CRQ and the SGRQ. The CRQ has, however, proved more responsive than the SGRQ: in an investigation that included 15 studies using both instruments, standardized response means of the CRQ (median 0.51, interquartile range (IQR) 0.19 to 0.98) were significantly higher (P <0.001) than those associated with the SGRQ (median 0.26, IQR −0.03 to 0.40) (Puhan et al 2006). As a result, pooling results from trials using these two instruments could lead to underestimates of intervention effect in studies using the SGRQ (Puhan et al 2006, Johnston et al 2010). This can be tested using a sensitivity analysis of studies using the more responsive versus less responsive instrument.

Usually, detailed data such as those described above will be unavailable. Investigators must then fall back on intuitive decisions about the extent to which different instruments are measuring the same underlying concept. For example, the authors of a meta-analysis of psychosocial interventions in the treatment of pre-menstrual syndrome faced a profusion of outcome measures, with 25 PROMs used in their nine eligible studies (Busse et al 2009). They dealt with this problem by having two experienced clinical researchers, knowledgeable to the study area and not otherwise involved in the review, independently examine each instrument – including all domains – and group 16 PROMs into six discrete conceptual categories. Any discrepancies were resolved by discussion to achieve consensus. Table 18.4.a details the categories and the included instruments within each category.

Authors should follow the guidance elsewhere in this Handbook on appropriate methods of synthesizing different outcome measures in a single analysis ( Chapter 10 ) and interpreting these results in a way that is most meaningful for decision makers ( Chapter 15 ).

Table 18.4.a Examples of potentially combinable PROMs measuring similar constructs from a review of psychosocial interventions in the treatment of pre-menstrual syndrome (Busse et al 2009). Reproduced with permission of Karger

Having decided which PROs and subsequently PROMs to include in a meta-analysis, review authors face the challenge of ensuring the results they present are interpretable to their target audiences. For instance, if told that the mean difference between rehabilitation and standard care in a series of randomized trials using the CRQ was 1.0 (95% CI 0.6 to 1.5), many readers would be uncertain whether this represents a trivial, small but important, moderate, or large effect (Guyatt et al 1998, Brozek et al 2006, Schünemann et al 2006). Similarly, the interpretation of a standardized mean difference is challenging for most (Johnston et al 2016b). Chapter 15 summarizes the various statistical presentation approaches that can be used to improve the interpretability of summary estimates. Further, for those interested in additional guidance, the GRADE working group summarizes five presentation approaches to enhancing the interpretability of pooled estimates of PROs when preparing ‘Summary of findings’ tables (Thorlund et al 2011, Guyatt et al 2013, Johnston et al 2013).

18.5 Chapter information

Authors: Bradley C Johnston, Donald L Patrick, Tahira Devji, Lara J Maxwell, Clifton O Bingham III, Dorcas Beaton, Maarten Boers, Matthias Briel, Jason W Busse, Alonso Carrasco-Labra, Robin Christensen, Bruno R da Costa, Regina El Dib, Anne Lyddiatt, Raymond W Ostelo, Beverley Shea, Jasvinder Singh, Caroline B Terwee, Paula R Williamson, Joel J Gagnier, Peter Tugwell, Gordon H Guyatt

Funding: DB is on the executive of OMERACT (Outcome Measurement in Rheumatology) (unpaid position). OMERACT is supported through partnership with multiple industries and OMERACT funds support staff to assist in the development of methods and materials around core outcome set development that influenced this chapter. The Parker Institute, Bispebjerg and Frederiksberg Hospital (RC) is supported by a core grant from the Oak Foundation (OCAY-13-309). TD has received funding from the Canadian Institutes of Health Research for research related to patient-reported outcomes and minimal important differences. RWO received research grants (paid to the Institute) from Netherlands Organisation Scientific Research (NWO); Netherlands Organisation for Health Research and Development (ZonMw); Wetenschappelijk College Fysiotherapie/KNGF Ned Ver Manuele Therapie; European Chiropractors’ Union; Amsterdam Movement Sciences; National Health Care Institute (ZiN); De Friesland Zorgverzekeraar. PRW’s work within the COMET Initiative is funded through grant NIHR Senior Investigator Award (NF-SI_0513-10025).

18.6 References

Alonso-Coello P, Zhou Q, Martinez-Zapata MJ, Mills E, Heels-Ansdell D, Johanson JF, Guyatt G. Meta-analysis of flavonoids for the treatment of haemorrhoids. British Journal of Surgery 2006; 93 : 909-920.

Beaton D, Boers M, Tugwell P. Assessment of Health Outcomes. In: Firestein G, Budd R, Gabriel SE, McInnes IB, O'Dell J, editors. Kelley and Firestein's Textbook of Rheumatology . 10th ed. Philadelphia (PA): Elsevier; 2016. p. 496-508.

Boers M, Brooks P, Strand CV, Tugwell P. The OMERACT filter for Outcome Measures in Rheumatology. Journal of Rheumatology 1998; 25 : 198-199.

Boers M, Kirwan JR, Wells G, Beaton D, Gossec L, d'Agostino MA, Conaghan PG, Bingham CO, 3rd, Brooks P, Landewe R, March L, Simon LS, Singh JA, Strand V, Tugwell P. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. Journal of Clinical Epidemiology 2014; 67 : 745-753.

Brozek JL, Guyatt GH, Schünemann HJ. How a well-grounded minimal important difference can enhance transparency of labelling claims and improve interpretation of a patient reported outcome measure. Health and Quality of Life Outcomes 2006; 4 : 69.

Bucher HC, Cook DJ, Holbrook AM, Guyatt G. Chapter 13.4: Surrogate Outcomes. In: Guyatt G, Rennie D, Meade MO, Cook DJ, editors. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice . 3rd ed. New York: McGraw-Hill Education; 2014.

Busse JW, Montori VM, Krasnik C, Patelis-Siotis I, Guyatt GH. Psychological intervention for premenstrual syndrome: a meta-analysis of randomized controlled trials. Psychotherapy and Psychosomatics 2009; 78 : 6-15.

Calvert M, Blazeby J, Altman DG, Revicki DA, Moher D, Brundage MD. Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA 2013; 309 : 814-822.

Cappelleri JC, Jason Lundy J, Hays RD. Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures. Clinical Therapeutics 2014; 36 : 648-662.

Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, Ader D, Fries JF, Bruce B, Rose M. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Medical Care 2007; 45 : S3-S11.

Christensen R, Maxwell LJ, Jüni P, Tovey D, Williamson PR, Boers M, Goel N, Buchbinder R, March L, Terwee CB, Singh JA, Tugwell P. Consensus on the Need for a Hierarchical List of Patient-reported Pain Outcomes for Metaanalyses of Knee Osteoarthritis Trials: An OMERACT Objective. Journal of Rheumatology 2015; 42 : 1971-1975.

COMET. Core Outcome Measures in Effectiveness Trials 2018. http://www.comet-initiative.org .

Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychological Bulletin 1955; 52 : 281-302.

de Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in Medicine: A Practical Guide . Cambridge: Cambridge University Press; 2011.

FDA. Guidance for Industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Rockville, MD; 2009. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf .

FDA. Clinical Outcome Assessment Program Silver Spring, MD: US Food and Drug Administration; 2018. https://www.fda.gov/Drugs/DevelopmentApprovalProcess/DrugDevelopmentToolsQualificationProgram/ucm284077.htm .

Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. Journal of Chronic Diseases 1987; 40 : 171-178.

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schünemann HJ. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. Journal of Clinical Epidemiology 2011; 64 : 383-394.

Guyatt GH, Veldhuyzen Van Zanten SJ, Feeny DH, Patrick DL. Measuring quality of life in clinical trials: a taxonomy and review. CMAJ: Canadian Medical Association Journal 1989; 140 : 1441-1448.

Guyatt GH, Naylor CD, Juniper E, Heyland DK, Jaeschke R, Cook DJ. Users' guides to the medical literature. XII. How to use articles about health-related quality of life. Evidence-Based Medicine Working Group. JAMA 1997; 277 : 1232-1237.

Guyatt GH, Juniper EF, Walter SD, Griffith LE, Goldstein RS. Interpreting treatment effects in randomised trials. BMJ 1998; 316 : 690-693.

Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, Johnston BC, Karanicolas P, Akl EA, Vist G, Kunz R, Brozek J, Kupper LL, Martin SL, Meerpohl JJ, Alonso-Coello P, Christensen R, Schünemann HJ. GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes. Journal of Clinical Epidemiology 2013; 66 : 173-183.

Hannan MT, Felson DT, Pincus T. Analysis of the discordance between radiographic changes and knee pain in osteoarthritis of the knee. Journal of Rheumatology 2000; 27 : 1513-1517.

Johnston BC, Thorlund K, Schünemann HJ, Xie F, Murad MH, Montori VM, Guyatt GH. Improving the interpretation of quality of life evidence in meta-analyses: the application of minimal important difference units. Health and Quality of Life Outcomes 2010; 8 : 116-116.

Johnston BC, Thorlund K, da Costa BR, Furukawa TA, Guyatt GH. New methods can extend the use of minimal important difference units in meta-analyses of continuous outcome measures. Journal of Clinical Epidemiology 2012; 65 : 817-826.

Johnston BC, Patrick DL, Thorlund K, Busse JW, da Costa BR, Schünemann HJ, Guyatt GH. Patient-reported outcomes in meta-analyses-part 2: methods for improving interpretability for decision-makers. Health and Quality of Life Outcomes 2013; 11 : 211-211.

Johnston BC, Miller PA, Agarwal A, Mulla S, Khokhar R, De Oliveira K, Hitchcock CL, Sadeghirad B, Mohiuddin M, Sekercioglu N, Seweryn M, Koperny M, Bala MM, Adams-Webber T, Granados A, Hamed A, Crawford MW, van der Ploeg AT, Guyatt GH. Limited responsiveness related to the minimal important difference of patient-reported outcomes in rare diseases. Journal of Clinical Epidemiology 2016a; 79 : 10-21.

Johnston BC, Alonso-Coello P, Friedrich JO, Mustafa RA, Tikkinen KA, Neumann I, Vandvik PO, Akl EA, da Costa BR, Adhikari NK, Dalmau GM, Kosunen E, Mustonen J, Crawford MW, Thabane L, Guyatt GH. Do clinicians understand the size of treatment effects? A randomized survey across 8 countries. CMAJ: Canadian Medical Association Journal 2016b; 188 : 25-32.

Jones PW. Health status measurement in chronic obstructive pulmonary disease. Thorax 2001; 56 : 880-887.

Juhl C, Lund H, Roos EM, Zhang W, Christensen R. A hierarchy of patient-reported outcomes for meta-analysis of knee osteoarthritis trials: empirical evidence from a survey of high impact journals. Arthritis 2012; 2012 : 136245.

Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ 2010; 340 : c365.

Kirshner B, Guyatt G. A methodological framework for assessing health indices. Journal of Chronic Diseases 1985; 38 : 27-36.

Martinez-Devesa P, Waddell A, Perera R, Theodoulou M. Cognitive behavioural therapy for tinnitus. Cochrane Database of Systematic Reviews 2007; 9 : CD005233.

Müller-Bühl U, Engeser P, Klimm H-D, Wiesemann A. Quality of life and objective disease criteria in patients with intermittent claudication in general practice. Family Practice 2003; 20 : 36-40.

Patrick DL, Deyo RA. Generic and disease-specific measures in assessing health status and quality of life. Medical Care 1989; 27 : S217-232.

Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, Ring L. Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1--eliciting concepts for a new PRO instrument. Value in Health 2011a; 14 : 967-977.

Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, Ring L. Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2--assessing respondent understanding. Value in Health 2011b; 14 : 978-988.

Powers JH, 3rd, Patrick DL, Walton MK, Marquis P, Cano S, Hobart J, Isaac M, Vamvakas S, Slagle A, Molsen E, Burke LB. Clinician-Reported Outcome Assessments of Treatment Benefit: Report of the ISPOR Clinical Outcome Assessment Emerging Good Practices Task Force. Value in Health 2017; 20 : 2-14.

Prinsen CA, Vohra S, Rose MR, Boers M, Tugwell P, Clarke M, Williamson PR, Terwee CB. How to select outcome measurement instruments for outcomes included in a "Core Outcome Set" - a practical guideline. Trials 2016; 17 : 449.

PROMIS. Patient Reported Outcomes Measurement Information System 2018. http://www.healthmeasures.net/explore-measurement-systems/promis .

PROQOLID. Patient Reported Outcomes and Quality of Life Instruments Database 2018. https://eprovide.mapi-trust.org/about/about-proqolid .

Puhan MA, Soesilo I, Guyatt GH, Schünemann HJ. Combining scores from different patient reported outcome measures in meta-analyses: when is it justified? Health and Quality of Life Outcomes 2006; 4 : 94-94.

Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. Journal of Clinical Epidemiology 2008; 61 : 102-109.

Rutten-van Mölken M, Roos B, Van Noord JA. An empirical comparison of the St George's Respiratory Questionnaire (SGRQ) and the Chronic Respiratory Disease Questionnaire (CRQ) in a clinical trial setting. Thorax 1999; 54 : 995-1003.

Schünemann HJ, Best D, Vist G, Oxman AD, Group GW. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. Canadian Medical Association Journal 2003; 169 : 677-680.

Schünemann HJ, Goldstein R, Mador MJ, McKim D, Stahl E, Puhan M, Griffith LE, Grant B, Austin P, Collins R, Guyatt GH. A randomised trial to evaluate the self-administered standardised chronic respiratory questionnaire. European Respiratory Journal 2005; 25 : 31-40.

Schünemann HJ, Akl EA, Guyatt GH. Interpreting the results of patient reported outcome measures in clinical trials: the clinician's perspective. Health Qual Life Outcomes 2006; 4 : 62.

Singh SJ, Sodergren SC, Hyland ME, Williams J, Morgan MD. A comparison of three disease-specific and two generic health-status measures to evaluate the outcome of pulmonary rehabilitation in COPD. Respiratory Medicine 2001; 95 : 71-77.

Smith K, Cook D, Guyatt GH, Madhavan J, Oxman AD. Respiratory muscle training in chronic airflow limitation: a meta-analysis. American Review of Respiratory Disease 1992; 145 : 533-539.

Tarlov AR, Ware JE, Jr., Greenfield S, Nelson EC, Perrin E, Zubkoff M. The Medical Outcomes Study. An application of methods for monitoring the results of medical care. JAMA 1989; 262 : 925-930.

Tendal B, Nuesch E, Higgins JP, Jüni P, Gøtzsche PC. Multiplicity of data in trial reports and the reliability of meta-analyses: empirical study. BMJ 2011; 343 : d4829.

Thorlund K, Walter SD, Johnston BC, Furukawa TA, Guyatt GH. Pooling health-related quality of life outcomes in meta-analysis-a tutorial and review of methods for enhancing interpretability. Research Synthesis Methods 2011; 2 : 188-203.

Wainer H. Estimating coefficients in linear models: It don't make no nevermind. Psychological Bulletin 1976; 83 : 213-217.

Ware J, Jr., Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Medical Care 1996; 34 : 220-233.

Ware JE, Jr., Kosinski M, Bayliss MS, McHorney CA, Rogers WH, Raczek A. Comparison of methods for the scoring and statistical analysis of SF-36 health profile and summary measures: summary of results from the Medical Outcomes Study. Medical Care 1995; 33 : As264-279.

Wiebe S, Guyatt G, Weaver B, Matijevic S, Sidwell C. Comparative responsiveness of generic and specific quality-of-life instruments. Journal of Clinical Epidemiology 2003; 56 : 52-60.

Witter JP. Introduction: PROMIS a first look across diseases. Journal of Clinical Epidemiology 2016; 73 : 87-88.

Yohannes AM, Roomi J, Waters K, Connolly MJ. Quality of life in elderly patients with COPD: measurement and predictive factors. Respiratory Medicine 1998; 92 : 1231-1236.

For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.

Center for Health Services and Outcomes Research

• CHSOR Seminar
• Collaborators
• Data Resources
• Faculty & Staff
• Get Involved
• Information & Calendar
• Caregiving For Alzheimer's Disease And Related Dementias: Enhancing The National Study Of Caregiving (Nsoc)
• Family Caregivers And Shared Access To The Patient Portal
• Family Involvement In Older Adults’ Physician Visits
• Involving Family To Improve Communication About Safe And Effective Medication Use For Older Primary Care Patients With Dementia
• National Health And Aging Trends Study
• Overuse Of Healthcare Resources In Older Adults
• Using Consumer Credit Data To Identify Precursors And Consequences Of Cognitive Impairment
• Comparative Effectiveness Of Health System Vs. Multi-level Interventions To Reduce Hypertension Disparities
• Comparing Patient Centered Outcomes In The Management Of Pain Between Emergency Departments And Dedicated Acute Care Facilities For Adults With Sickle Cell Disease
• Structural Nested Models To Assess The Safety And Effectiveness Of Generic Drugs
• Economic Evaluation
• Assessment Tools For Palliative Care
• Measuring What Matters In Hospice And Palliative Care
• Achieving Excellence In Biopsychosocial Cancer Pain Management Through A Comprehensive Quality Education Program
• Evaluation Of Implementation Of Point-of-care Palliative Care Quality Measures
• (Dis)respect Reflected And Perpetuated In Medical Records
• Health Information Technology
• Collaborating Centers Of Excellence In Regulatory Science And Innovation
• Effect Of Therapeutic Class On Generic Drug Substitutions
• Impact Of "Pill Mill" Laws On Opioid Prescribing, Dispensing And Utilization
• Medicare At 50
• Medicare Benefit Design And Long-term Services And Supports: Gaps, Opportunities, And Implications For Beneficiaries
• The Development And Testing Of The Frailty Component Of A Novel Ehr-based "Geriatric E-risk" Measure For Predictive Modeling
• A Community Hie-based Hospital Readmission Risk Prediction And Notification System
• Capturing Implementation Context: The Team Check Up Tool (Tct) Project
• Clinical Knowledge Hub - Conceptual Integration Of Rules, Data Sets, And Queries
• Collaborative Research: Modeling Disease Trajectories In Patients With Complex, Multiphenotypic Conditions
• Creating And Validating A Claim-based Indicator Of Frailty
• Developing An Indicator Of Overuse Of Medical Testing To Reduce The Cost Of Health Care
• Developing Next Generation EHR-supported Predictive Modeling: Developing The Johns Hopkins "E-ACG" System
• Evidence-based Practice Center
• Framework For A Vha Population Health Program: Bmi Trajectory Project
• Geographic Variation In Health And Economic Determinants And Outcomes Of Elective Surgery
• Harmonizing Cognitive Assessment In International Surveys On Aging
• Identifying High Risk Pregnancies Through Natural Language Processing
• Long-term Health Impacts Of Physical And Cognitive Occupational Exposures
• Partnership For Quality Improvement Evaluation With The American College Of Physicians
• Pcori Methodology Standards Academic Curriculum
• Stage 3 Meaningful Use Objectives: An Evaluation Of Care Coordination Measures Among Eligible Hospitals
• "Proteus" Patient-reported Outcomes Tools: Engaging Users & Stakeholders
• A Healthy Patient Or A Satisfied Customer (Or Both!)? Patient Satisfaction In Otolaryngology, Surgery, And Pediatrics
• Accelerating Collaborations For Evaluation Matching Service
• Analyzing The Relationship Between Televised Direct-to-consumer Pharmaceutical Advertising And Patient Outcomes
• Comprehensive Home-based Dementia Care Coordination For Medicare-medicaid Dual Eligibles In Maryland
• Developing Patient Reported Outcome-based Performance Measures In Primary Care Practices
• Impact Of Employment Interventions For Persons With Serious Mental Illnesses On Non-employment Patient-centered Outcomes
• Improving Health Care And Palliative Care For Advanced And Serious Illness
• Maryland Multi-payor Patient Centered Medical Home Program Evaluation
• Path Clinical Data Research Network

Presenting Patient-reported Outcomes To Promote Patient And Clinician Understanding And Use

• Preventing Fall Related Injuries In Older Adults (Stride Study)
• Smi Dual Eligibles And Medicare Part D: Impact On Medications Continuity And Outcomes
• Linking Blood Stream Infections To Intensive Care Nursing Context Of Care And Process
• Prevention Of Clot In Orthopaedic Trauma
• RISE – Resilience In Stressful Events
• Safety Program In Perinatal Care-ii Phase 2
• What Is Patient Safety In The Medical Home?
• Reverse Innovation & Community Engagement To Improve Quality Of Care & Patient Outcomes (Project Connect)
• The Effects Of Neighborhood Poverty On Health Care Utilization And Spending
• Communication, Respect And Trust In Sickle Cell Disease
• Comorbid Conditions In Cancer Survivors: Costs And Quality Of Care
• Maximizing Respect And Improving Patient Outcomes In Hiv And Substance Abuse
• Improving Doctor-patient Communication In Cancer Care
• Shared Decision-making And Parent-reported Outcomes In Pediatric Sleep-disordered Breathing
• PI & co-PIs :      Claire Snyder , PhD & Michael Brundage, MD, MSc
• Funder :           Patient-Centered Outcomes Research Institute
• Status :             Results Published

Patient-reported outcomes (PROs) (e.g., symptoms, quality of life) assess the impact of a health condition and its treatment from the patient perspective and can be used to promote patient-centered care. However, there are many different PRO questionnaires, and there is no standard way to score and present PRO results, making it difficult for patients and clinicians to understand and use PROs in practice. Given PROs’ potential to help clinicians and patients tailor care to a particular patient’s needs, research on how to present PRO data so that the results are accurately interpreted and are meaningful and useful for patients and clinicians is critical.

The study aims to (1) learn how current ways of presenting PRO results limit patient and clinician understanding; (2) develop new approaches for presenting PRO results to improve patients’ and clinicians’ ability to use the findings; and (3) evaluate how well the new approaches work in improving patient and clinician understanding and use of PRO data. The long-term objective is to develop best practices for presenting PRO data to patients and clinicians, thereby improving the ability of patients and clinicians to make treatment decisions to meet a particular patient’s needs.

Related Project

Stakeholder-Driven, Evidence-Based Standards for Presenting Patient-Reported Outcomes in Practice. This follow-up PCORI  project  developed evidence-based, stakeholder-driven recommendations for graphically presenting PRO data to patients and clinicians to promote clarity and interpretation accuracy, thereby facilitating patients’ and clinicians’ interpretation of PRO results and serving the long-term goal of promoting patient-centered care.The project addressesed three different applications of PRO data in clinical practice: (1) using individual patients’ PRO data to monitor their functioning and well-being and inform their management; (2) presenting the results of comparative patient-centered research studies to patients in educational materials and decision aids to improve understanding of treatment options and inform decision making; and (3) presenting the results of comparative patient-centered research studies to clinicians to inform their understanding of the PRO impacts of different treatment options and inform their clinical care.

Snyder C, Smith K, Holzner B, Rivera Y, Bantug E, Brundage M. PRO Data Presentation Delphi Panel.  Making a picture worth a thousand numbers:  Recommendations for graphically displaying patient-reported outcomes data . Qual Life Res. 2018 Oct 10 [Epub ahead of print].

Publications:

Brundage MD, Smith KC, Little EA, Bantug ET, Snyder CF, PRO Data Presentation Stakeholder Advisory Board.  Communicating patient-reported outcome scores using graphic formats: results from a mixed methods evaluation . Quality of Life Research. 2015;24:2457-2472.

Smith KC, Brundage MD, Tolbert E, Little EA, Bantug ET, Snyder C, PRO Data Presentation Stakeholder Advisory Board.  Engaging stakeholders to improve presentation of patient-reported outcomes data in clinical practice . Supportive Care in Cancer. 2016;24:4149-4157.

Bantug ET, Coles T, Smith KC, Snyder CF, Rouette J, Brundage MD, PRO Data Presentation Stakeholder Advisory Board.  Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: What makes a PRO picture worth a thousand words?  Patient Education and Counseling. 2016; 99:483-490

Snyder CF, Smith KC, Bantug ET, Tolbert EE, Blackford AL, Brundage MD, PRO Data Presentation Stakeholder Advisory Board.  What do these scores mean? Presenting patient-reported outcomes data to patients and clinicians to improve interpretability . Cancer. 2017;123:1848-1859.

Brundage MD, Blackford A, Tolbert E, Smith K, Bantug E, Snyder C; PRO Data Presentation Stakeholder Advisory Board.  Presenting comparative study PRO results to clinicians and researchers: Beyond the eye of the beholder . Quality of Life Research. 2018 Jan;27(1):75-90.

Tolbert E, Brundage M, Bantug E, Blackford AL, Smith K, Snyder C; PRO Data Presentation Stakeholder Advisory Board.  Picture this: Presenting longitudinal patient-reported outcome research study results to patients . Medical Decision-Making. 2018 Aug 22:272989X18791177.

Snyder C, Smith K, Tolbert E, Bantug E, Brundage M, PRO Data Presentation Stakeholder Advisory Board.  Partnering with stakeholders using an example patient-reported outcomes project . Journal of Community and Supportive Oncology. 2017 Nov 9 [Epub ahead of print].

Tolbert E, Brundage M, Bantug E, Blackford AL, Smith K, Snyder C; PRO Data Presentation Stakeholder Advisory Board.  In Proportion: Approaches for Displaying Patient-reported Outcome Research Study Results as Percentages Responding to Treatment . Qual Life Res. 2018 Nov 29  [Epub ahead of print].

Clinical outcome

A clinical outcome is a measurable change in symptoms, overall health, ability to function, quality of life, or survival outcomes that result from giving care to patients. Clinical outcomes may be used in clinical settings, such as a hospital or doctor’s office, to measure the success of care or to assess a person’s response to an already approved treatment. One or more clinical outcomes may be used in clinical trials as an endpoint to determine how well a new therapy works and/or the safety of a new therapy.

Sourced From U.S. Food and Drug Administration ( FDA ) Patient -Focused Drug Development Glossary Glossary from “BEST (Biomarkers, EndpointS, and other Tools) Resource” [Internet]. Learn More U.S. Food and Drug Administration (FDA): Multiple Endpoints in Clinical Trials Guidance for Industry U.S. Food and Drug Administration (FDA): Surrogate Endpoint Resources for Drug and Biologic Development

• Glossary: Clinician-reported outcomes (ClinRO)
• Glossary: Patient-reported outcome
• Glossary: Surrogate endpoint

• - Google Chrome

Intended for healthcare professionals

• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• How to present patient...

How to present patient cases

• Related content
• Peer review
• Mary Ni Lochlainn , foundation year 2 doctor 1 ,
• Ibrahim Balogun , healthcare of older people/stroke medicine consultant 1
• 1 East Kent Foundation Trust, UK

A guide on how to structure a case presentation

This article contains...

-History of presenting problem

-Medical and surgical history

-Drugs, including allergies to drugs

-Family history

-Social history

-Review of systems

-Findings on examination, including vital signs and observations

-Differential diagnosis/impression

-Investigations

-Management

Presenting patient cases is a key part of everyday clinical practice. A well delivered presentation has the potential to facilitate patient care and improve efficiency on ward rounds, as well as a means of teaching and assessing clinical competence. 1

The purpose of a case presentation is to communicate your diagnostic reasoning to the listener, so that he or she has a clear picture of the patient’s condition and further management can be planned accordingly. 2 To give a high quality presentation you need to take a thorough history. Consultants make decisions about patient care based on information presented to them by junior members of the team, so the importance of accurately presenting your patient cannot be overemphasised.

As a medical student, you are likely to be asked to present in numerous settings. A formal case presentation may take place at a teaching session or even at a conference or scientific meeting. These presentations are usually thorough and have an accompanying PowerPoint presentation or poster. More often, case presentations take place on the wards or over the phone and tend to be brief, using only memory or short, handwritten notes as an aid.

Everyone has their own presenting style, and the context of the presentation will determine how much detail you need to put in. You should anticipate what information your senior colleagues will need to know about the patient’s history and the care he or she has received since admission, to enable them to make further management decisions. In this article, I use a fictitious case to …

Log in using your username and password

BMA Member Log In

If you have a subscription to The BMJ, log in:

• Need to activate
• Log in via institution
• Log in via OpenAthens

Log in through your institution

Subscribe from £184 *.

Subscribe and get access to all BMJ articles, and much more.

* For online subscription

Access this article for 1 day for: £50 / $60/ €56 ( excludes VAT )

You can download a PDF version for your personal record.

Buy this article

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

Patient-Specific Explanations for Predictions of Clinical Outcomes

Mohammadamin tajgardoon.

1 Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, United States

Malarkodi J. Samayamuthu

2 Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States

Luca Calzoni

Shyam visweswaran.

Machine learning models that are used for predicting clinical outcomes can be made more useful by augmenting predictions with simple and reliable patient-specific explanations for each prediction.

This article evaluates the quality of explanations of predictions using physician reviewers. The predictions are obtained from a machine learning model that is developed to predict dire outcomes (severe complications including death) in patients with community acquired pneumonia (CAP).

Using a dataset of patients diagnosed with CAP, we developed a predictive model to predict dire outcomes. On a set of 40 patients, who were predicted to be either at very high risk or at very low risk of developing a dire outcome, we applied an explanation method to generate patient-specific explanations. Three physician reviewers independently evaluated each explanatory feature in the context of the patient’s data and were instructed to disagree with a feature if they did not agree with the magnitude of support, the direction of support (supportive versus contradictory), or both.

The model used for generating predictions achieved a F1 score of 0.43 and area under the receiver operating characteristic curve (AUROC) of 0.84 (95% confidence interval [CI]: 0.81–0.87). Interreviewer agreement between two reviewers was strong (Cohen’s kappa coefficient = 0.87) and fair to moderate between the third reviewer and others (Cohen’s kappa coefficient = 0.49 and 0.33). Agreement rates between reviewers and generated explanations—defined as the proportion of explanatory features with which majority of reviewers agreed—were 0.78 for actual explanations and 0.52 for fabricated explanations, and the difference between the two agreement rates was statistically significant (Chi-square = 19.76, p -value < 0.01).

There was good agreement among physician reviewers on patient-specific explanations that were generated to augment predictions of clinical outcomes. Such explanations can be useful in interpreting predictions of clinical outcomes.

Background and Significance

Sophisticated predictive models are increasingly being developed using machine learning methods to predict clinical outcomes, such as mortality, morbidity, and adverse events. 1 – 9 These models, in most cases, are viewed as black boxes that produce a prediction for an outcome from the features of a patient case. † However, for such models to be practically useful in clinical care, it is critical to provide clear and reliable individual-specific explanations for each prediction. 10 While a prediction provides an estimate of the likely outcome in the future, an explanation provides insight into features that may be useful in clinical decision-making. Moreover, explanations will enable physicians to engender trust in the predictions, interpret them in the clinical context, and help make optimal clinical decisions. 11 In the clinical context, features that are supportive of a prediction provide potentially actionable aspects that may change the predicted outcome. 12 , 13

In the context of predictive models, a subtle but important distinction exists between model explanation and prediction explanation. Model explanation provides an interpretation of the model to the user in terms of structure and parameters, and is useful in the context of making discoveries. 12 , 14 Some predictive models, such as decision trees, linear regression, and rule-based models, are more easily interpretable, though often such models have poorer predictive performance than more abstract models, such as random forests, support vector machines, and neural networks. 12 , 14 In contrast to model explanation, prediction explanation provides an interpretation of the prediction for an individual to whom a model is applied, and will potentially be different from individual to individual. 15 , 16 Useful prediction explanations possess two properties. First, an explanation uses concepts that are understandable to the user, such as clinical variables that are not modified or transformed. Second, the explanation is parsimonious, so that it is readily and rapidly grasped by the user. Prediction explanations may be based on the structure and parameters of the model that yielded the prediction (hence, model dependent) or may be based on an independent method that is applied after the predictive model has been developed (hence, model independent). 14 , 17

Novel methods have been developed for prediction explanations and such methods have been applied in biomedicine and other domains. Table 1 provides a summary of studies that have developed methods for prediction explanations, with a brief description of each explanation method.

Studies that describe methods for prediction explanation

Only a small number of the methods that are listed in Table 1 have been applied to predicting clinical outcomes. For example, Luo applied their method to type-2 diabetes risk prediction 18 , Štrumbelj et al developed and applied their method to breast cancer recurrence predictions, 19 and Reggia and Perricone developed explanations for predictions of the type of stroke. 11 More widespread application of these methods to clinical predictions can provide evidence of applicability and utility of these methods to clinical users.

In this study, we apply and evaluate a recently developed prediction explanation method called Local Interpretable Model-Agnostic Explanations (LIME) 15 for clinical predictions. The developers of LIME demonstrated that human evaluators found explanations generated by LIME to be more reasonable when compared with the explanations generated by alternative methods. To our knowledge, LIME has not been extensively evaluated in the context of clinical predictive models.

Our goal was to evaluate patient-specific explanations for clinical predictions. The aims of our study were to (1) Develop machine learning models to predict dire outcomes (severe complications including death) from readily available clinical data in patients who present with community acquired pneumonia (CAP), followed by application of a model-independent prediction explanation method to generate patient-specific explanations; and (2) Evaluate the agreement among physicians for explanations generated for CAP patients who were predicted to be either at very high risk or at very low risk of developing a dire outcome.

In this section, we briefly describe the pneumonia dataset that we used in the experiments, the methods for development and evaluation of predictive models, the generation of patient-specific explanations, and the measures we used to evaluate agreement among physician reviewers for the explanations. The implementation of the methods is publicly available at: https://github.com/Amin-Tajgardoon/explanation-project .

Description of Dataset

The pneumonia data were collected by the Pneumonia Patient Outcomes Research Team (PORT) 20 during October 1991 to March 1994 at five hospitals in three geographical locations including Pittsburgh, Boston, and Halifax, Nova Scotia. The PORT data from Pittsburgh that we used in the experiments had 2,287 patients diagnosed with CAP who were either hospitalized or seen in ambulatory care. A variety of clinical data were collected at the time of presentation and several outcomes at 30 days were assessed. A key goal of the PORT project was to develop accurate criteria for prognosis of patients with pneumonia that could provide guidance on which patients should be hospitalized and which patients might be safely treated at home.

The PORT dataset contains more than 150 variables including demographic information history and physical examination information, laboratory results, and chest X-ray findings. From the 150 variables, we selected 41 clinical variables that are typically available in the emergency department at the time the decision whether to admit or not is made. Of the 41 variables, 17 are discrete and the remaining 24 are continuous. The 24 continuous variables were discretized based on thresholds provided by clinical experts on the PORT project. 20 A list of the 41 variables with descriptions is provided in Table 2 .

list of variables in the pneumonia PORT study that were used in the present study

Note: Continuous variables were discretized based on clinical judgment of pneumonia experts in the pneumonia PORT project. 20 The label “(Discrete)” in the description indicates that a variable is a discretized version of a continuous variable.

The outcome variable we used as the target variable is called dire outcome and is binary. A patient was considered to have had a dire outcome if any of the following events occurred: (1) death within 30 days of presentation; (2) an intensive care unit admission for respiratory failure, respiratory or cardiac arrest, or shock; or (3) one or more specific, severe complications, such as myocardial infarction, pulmonary embolism, stroke, etc. 21 About 11.4% (261) patients had a dire outcome in the PORT dataset.

Training and test sets:

the data consisting of 2,287 cases was divided into a training dataset of 1,601 cases (70%) and a test dataset of 686 cases (30%) by using stratified random-sampling such that both sets had approximately the same proportion of cases with dire outcomes as the full dataset(11.4% [182/1,601] and 11.5% [79/686] of patients had a dire outcome in the training and test sets, respectively). Missing data were imputed using an iterated k -nearest neighbor method, 22 and continuous variables were discretized based on clinical judgment of pneumonia experts in the pneumonia PORT project.

Development of Predictive Models

We applied several machine learning methods to the training set to develop predictive models, and we applied the best-performing model to the test set to generate predictions.

Machine learning methods:

the machine learning methods that we used for developing predictive models are logistic regression with regularization (LR), random forest (RF), support vector machine (SVM), and naïve Bayes (NB). We selected these methods as representative of the machine-leaning methods that are typically used for developing predictive models in biomedicine. We used the implementations of these methods that are available in the scikit-learn package. 23

We tuned the hyper-parameters using 10-fold cross validation on the training set. The hyper-parameters that we configured included the regularization coefficient ([0.1, 1, 10]) for the LR and SVM models, number of trees ([100, 500, 1,000, 3,000]) for the RF model, and the Laplace smoothing parameter ([0, 0.1, 1, 10, 100]) for the NB model.

Evaluation of model performance:

we evaluated the predictive models on the training set using 10-fold cross validation. The metrics we used included F1 score, area under the receiver operating characteristic curve (AUROC), positive predictive value (PPV), sensitivity, and specificity. The F1 score is the harmonic mean of PPV and sensitivity and ranges between 0 and 1. 24 A high F1 score indicates that both PPV and sensitivity are high. We selected the machine learning method with the highest F1 score and reapplied it to the full-training set to derive the final model. We applied the final model to predict the outcomes for cases in the test set.

Generation of Patient-Specific Explanations

We used LIME to generate explanations for a selected set of 40 cases in the test set. A description on the selection of the 40 cases is provided in the next section. LIME is a model-independent explanation method that provides an explanation for a predicted case by learning an interpretable model from data in the neighborhood of the case (such as a local linear model with a small number of nonzero coefficients). More specifically, LIME provides for each patient feature the magnitude and the direction of support for the predicted outcome (see Fig. 1 ). The magnitude of support is the weight of an explanatory feature, and the direction of support is the sign of the weight, as estimated in LIME’s local regression model. We limited the explanations to the top six features with the highest magnitudes, as we found that, on average, the magnitude of five to seven features accounted for most of the total magnitude. We call the patient features that were included in the explanation as explanatory features.

Example explanation obtained from LIME for a patient who was predicted to have a very high probability of dire outcome by a logistic regression model. The bar plot at the top left shows the predicted probability distribution for dire outcome. The bar plot on the right shows the explanation for the prediction. The explanation is limited to six top-ranked features by magnitude. The magnitude on the horizontal axis represents the weight of a feature in the LIME’s local regression model. Green bars represent the magnitude of predictors that support the predicted outcome, while red bars represent the magnitude of contradictory features. LIME, local interpretable model-agnostic explanations.

Evaluation of Explanations

Three physicians independently evaluated explanations for 40 patient cases that were selected from the test set. We selected cases for which the model correctly predicted the outcome with high confidence (i.e., a patient was predicted to have developed a dire outcome with probability > 0.8 or with probability < 0.1). Of the 40 cases, 20 patients developed a dire outcome and 20 patients did not. Note that patients with and without a dire outcome are expected to have mostly the same predictors; however, the values of those predictors are likely to be different. For example, abnormal values in respiratory rate, arterial blood gases, and lung status are likely to be predictor features in a patient with a dire outcome, whereas normal values in respiratory rate, arterial blood gases, and lung status are likely to be predictor features in a patient without a dire outcome.

For each patient case, we provided the reviewers with a description that included clinical findings and if a dire outcome occurred or not, and the predicted probability of the dire outcome occurring along with the explanation for the prediction (see Fig. 2 ). Each reviewer assessed all 40 cases and the corresponding explanations, and specified if she agreed or disagreed with each explanatory feature. The reviewer was instructed to disagree with an explanatory feature if she did not agree with the magnitude, the direction (supportive vs. contradictory), or both.

An example patient case that gives a description of the patient, followed by an explanation and the questions that were asked of reviewers.

To preclude reviewers from agreeing readily with explanations without careful assessment, we fabricated explanations in 10 of the 40 cases. To generate a fabricated explanation, we replaced the labels (feature name and its value) of six top-ranked features with the labels of six bottom-ranked features, without modifying the magnitude or the direction of support. The reviewers were informed that some of the cases contained fabricated explanations but not which ones. Table 3 shows the stratification of cases according to the type of explanation (actual vs. fabricated) and by outcome (had a dire outcome vs. did not have a dire outcome) that we used for evaluation.

Cases used for evaluation, stratified by type of explanations and outcomes

We analyzed the assessments of the reviewers with several measures as follows: (1) We measured agreement between pairs of reviewers with Cohen’s kappa coefficient 25 and across all reviewers concurrently with Fleiss’ kappa statistic. 26 Cohen’s kappa coefficient measures the degree of agreement between two reviewers on a set of samples, whereas Fleiss’ kappa statistic can assess more than two reviewers simultaneously. (2) For a given set of cases, we calculated an agreement rate as the proportion of explanatory features with which majority of reviewers agreed. For example, for a set of 10 cases where each case had an explanation with six features, the denominator of the agreement rate is 10× 6 = 60 features and the numerator is the number of features with which majority of reviewers agreed. Agreement rates were calculated separately for cases with actual and fabricated explanations, and for cases where the patients had a dire outcome and did not have a dire outcome. (3) To statistically test for difference between two agreement rates that are derived from two sets of cases (e.g., actual vs. fabricated explanations, dire outcome vs. no dire outcome), we used the Chi-square test of independence. 27

We report the performance of the machine learning methods, briefly describe the prediction explanations, and provide the reviewers’ agreement scores.

Performance of Predictive Models

Table 4 shows the performance of five machine learning methods on the training set, as measured by F1 score, AUROC, PPV, sensitivity, and specificity. The two logistic regression models, LR-L1 and LR-L2, were trained with L1 and L2 regularization penalties, respectively. The LR-L1, LR-L2, NB, and SVM models have similar F1 scores, whereas RF has a lower F1 score despite having a similar AUROC to other models. The LR-L1 and LR-L2 models had similar performance; however, we chose the LR-L1 model as the best-performing model because it shrinks some of the regression coefficients to zero and provides a sparse solution.

Performance of five machine learning methods on the training set using 10-fold cross validation

Abbreviations: AUROC, area under the receiver operating characteristic curve; CI, confidence interval; LR-L1, LASSO logistic regression; LR-L2, ridge logistic regression; NB, naïve Bayes; PPV, positive predictive value; RF, random forest; SVM, support vector machine.

Note: The models are sorted in descending order of their F1 scores. The 95% CI for AUROCs were calculated using the Delong’s method, 38 , 39 and the 95% CI for the other measures were calculated using the Wilson’s score interval. 40

Description of Explanations

We applied the LR-L1 model to all cases in the test set and selected 40 cases based on criteria described in Section Methods , “Evaluation of Explanations.” We used LIME to generate explanations for the selected cases. Tables 5 and ​ and6 6 show the explanatory variables and their count of appearance in the actual and fabricated explanations respectively.

Variables and their count of appearance in the 30 actual explanations

Variables and their count of appearance in the 10 fabricated explanations

Agreement among reviewers:

Table 7 shows the agreement scores between pairs of reviewers and across all three reviewers. For both actual and fabricated explanations, Cohen’s kappa coefficients indicate strong agreement between reviewers 1 and 2, and fair to moderate agreement between reviewer 3 and the other two reviewers (according to the agreement levels proposed by McHugh 28 ). The Fleiss’ kappa statistic shows moderate agreement across all reviewers when considering all explanatory features. Much of the disagreement between reviewer 3 and the others was due to differing opinions on headache as an explanatory feature. After excluding headache from the analysis, Cohen’s kappa coefficient for all explanatory features for reviewers 1 and 3 increased from 0.49 to 0.76, and the corresponding Cohen’s kappa coefficient for reviewers 2 and 3 increased from 0.33 to 0.58.

Interreviewer agreement scores

Note: agreements between pairs of reviewers show the Cohen’s kappa coefficient and agreement across all reviewers show the Fleiss’ kappa statistic.

Agreement with LIME-generated explanations:

Table 8 shows agreement rates for explanations as the proportion of explanatory features with which majority of reviewers agreed. The agreement rate was 0.78 (141/180) for actual explanations and 0.52 (31/60) for fabricated explanations; the difference between the two agreement rates was statistically significant (Chi-square = 19.76, p -value < 0.01). For actual explanations, agreement rates were 0.81 (73/90) for cases where the patients had a dire outcomes and 0.76 (68/90) for cases where the patients did not have a dire outcome; the difference between the two agreement rates was not statistically significant (Chi-square = 0.55, p -value = 0.53).

Agreement rates for LIME-generated explanations, stratified by type of explanations and outcomes

Abbreviation: LIME, local interpretable model-agnostic explanations.

When headache was excluded from the analysis, the agreement rate increased from 0.78 to 0.93 for actual explanations. The agreement rate for fabricated explanations did not change from 0.52 because headache did not occur in fabricated explanations.

Computerized clinical decision-supporting systems that utilize predictive models for predicting clinical outcomes can be enhanced with explanations for predictions. Such explanations provide context for the predictions and guide physicians in better understanding supportive and contradictory evidence for the predictions. In this paper, we presented a method to augment clinical outcome predictions—obtained from a predictive model—with simple patient-specific explanations for each prediction. The method uses LIME that generates a patient-specific linear model which provides a weight for each feature. The weight provides insight about the relevance of each feature in terms of magnitude and direction of its contribution to a prediction. LIME has been shown to produce explanations that users find to be useful and trustworthy in general prediction problems. 15

In this study, we developed and evaluated several machine learning methods and chose a logistic regression model since it had the best performance. In this scenario, the model could be used directly to provide explanations—the weight of a feature for an explanation can be computed by multiplying the feature level by the corresponding odds ratio. However, in general, as the size and dimensionality of the data increase, more complex, and less interpretable models, like deep neural networks, are likely to perform better and the use of a model-independent explanation method like LIME becomes necessary.

Using LIME, we generated explanations for 40 cases and evaluated the explanations with three physician reviewers. The reviewers agreed with 78% of LIME-generated explanatory features for actual explanations and agreed with only 52% of explanatory features for fabricated explanations. This result provides evidence that the reviewers are able to distinguish between valid and invalid explanations. The results also indicate that agreement on cases where the patients had a dire outcome is not statistically significantly different from agreement on cases where the patients did not have a dire outcome.

Headache was a feature that was provided as an explanatory feature in most of the cases where the patients experienced a dire outcome. Two of the reviewers deemed headache to be mildly supportive, whereas the third reviewer did not consider headache to be a supportive feature. In support of the third reviewer’s judgment, commonly used scoring systems for assessment of severity of CAP, such as the pneumonia severity index 13 and CURB-65 29 do not include headache as a predictive feature. In the dataset, we used, almost all models included headache as a predictive feature; this may be because the Pittsburgh portion of the PORT data that we used in our experiments may have predictive features, such as headache, that are specific to the region. This indicates that predictive features in a model depend on the dataset that is used and explanations may uncover and inform physicians of features that are locally valid. More generally, this may suggest that predictive models should be derived from data that is from the location where the models will be deployed.

It is plausible that explanations of predictions are likely to be useful in clinical decision making, 10 , 11 and model-independent methods like LIME provide a method to generate explanations from any type of model. 15 However, it needs to be established that such explanations are valid, accurate, and easily grasped by physicians in the context of clinical predictive models. This study provides a first step toward that goal.

Limitations and Future Directions

This study has several limitations. Though LIME has the advantage that it can be used in conjunction with any predictive model, it has the limitation that internally it constructs a simple model. LIME constructs a local linear model from data in the neighborhood of the case of interest, and it seems reasonable to assume linearity in a small region even when the primary model is not linear. However, we and other investigators have noticed that the prediction of LIME’s local model is not always concordant with the prediction of the primary predictive model. 30 Methods like LIME will need to be modified such that the predictions agree with those of the primary predictive model and work is ongoing in the research community to improve LIME.

This study used a single dataset that is relatively old (data collection occurred in the early 1990s), measures only one medical condition, and is limited to patient visits at a single geographical location. Additionally, the number of physician evaluators was relatively small. To explore the generalizability of using LIME with predictive models, newer datasets are needed in which different outcomes are measured and samples are collected from diverse geographical locations. Higher numbers of physician evaluators can also yield more reliable evaluations.

This study demonstrated that it is possible to generate patient-specific explanations to augment predictions of clinical outcomes by using available machine learning methods for both model development and generation of explanations. Moreover, explanations that were generated for predicting dire outcomes in CAP were assessed to be valid by physician evaluators. Such explanations can engender trust in the predictions and enable physicians to interpret the predictions in the clinical context.

Clinical Relevance Statement

This study demonstrated that there was good agreement among physicians on patient-specific explanations that are generated to augment predictions from machine learning models of clinical outcomes. Such explanations will enable physicians to better understand the predictions and interpret them in the clinical context, and might even influence the clinical decisions they make. Computerized clinical decision-supporting systems that deliver predictions can be enhanced to provide explanations to increase their utility.

The research reported in this publication was supported by the National Library of Medicine of the National Institutes of Health under award number R01LM012095. The content of the paper is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the University of Pittsburgh.

Protection of Human and Animal Subjects

All research activities reported in this publication were reviewed and approved by the University of Pittsburgh’s Institutional Review Board.

Conflict of Interest

None declared.

† We distinguish between a variable and a feature. A variable describes an aspect of an individual. A feature is the specification of a variable and its value. For example, “fever” is a variable and “fever = yes” is a feature.

This paper is in the following e-collection/theme issue:

Published on 12.8.2024 in Vol 26 (2024)

Enhancing Patient Understanding of Laboratory Test Results: Systematic Review of Presentation Formats and Their Impact on Perception, Decision, Action, and Memory

Authors of this article:

• Frederieke A M van der Mee 1 * , MD   ; 
• Fleur Schaper 2, 3 * , PhD   ; 
• Jesse Jansen 1 , PhD   ; 
• Judith A P Bons 3 , PhD   ; 
• Steven J R Meex 3 , PhD   ; 
• Jochen W L Cals 1 , MD, Prof Dr  

1 Department of Family Medicine, Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands

2 Department of Clinical Chemistry, Reinier Medical Diagnostic Center, Delft, Netherlands

3 Central Diagnostic Laboratory, Maastricht University Medical Center+, Maastricht, Netherlands

*these authors contributed equally

Corresponding Author:

Frederieke A M van der Mee, MD

Department of Family Medicine

Care and Public Health Research Institute

Maastricht University

P. Debyeplein 1

Maastricht, 6229 HA

Netherlands

Phone: 31 883887059

Email: [email protected]

Background: Direct access of patients to their web-based patient portal, including laboratory test results, has become increasingly common. Numeric laboratory results can be challenging to interpret for patients, which may lead to anxiety, confusion, and unnecessary doctor consultations. Laboratory results can be presented in different formats, but there is limited evidence regarding how these presentation formats impact patients’ processing of the information.

Objective: This study aims to synthesize the evidence on effective formats for presenting numeric laboratory test results with a focus on outcomes related to patients’ information processing, including affective perception, perceived magnitude, cognitive perception, perception of communication, decision, action, and memory.

Methods: The search was conducted in 3 databases (PubMed, Web of Science, and Embase) from inception until May 31, 2023. We included quantitative, qualitative, and mixed methods articles describing or comparing formats for presenting diagnostic laboratory test results to patients. Two reviewers independently extracted and synthesized the characteristics of the articles and presentation formats used. The quality of the included articles was assessed by 2 independent reviewers using the Mixed Methods Appraisal Tool.

Results: A total of 18 studies were included, which were heterogeneous in terms of study design and primary outcomes used. The quality of the articles ranged from poor to excellent. Most studies (n=16, 89%) used mock test results. The most frequently used presentation formats were numerical values with reference ranges (n=12), horizontal line bars with colored blocks (n=12), or a combination of horizontal line bars with numerical values (n=8). All studies examined perception as an outcome, while action and memory were studied in 1 and 3 articles, respectively. In general, participants’ satisfaction and usability were the highest when test results were presented using horizontal line bars with colored blocks. Adding reference ranges or personalized information (eg, goal ranges) further increased participants’ perception. Additionally, horizontal line bars significantly decreased participants’ tendency to search for information or to contact their physician, compared with numerical values with reference ranges.

Conclusions: In this review, we synthesized available evidence on effective presentation formats for laboratory test results. The use of horizontal line bars with reference ranges or personalized goal ranges increased participants’ cognitive perception and perception of communication while decreasing participants’ tendency to contact their physicians. Action and memory were less frequently studied, so no conclusion could be drawn about a single preferred format regarding these outcomes. Therefore, the use of horizontal line bars with reference ranges or personalized goal ranges is recommended to enhance patients’ information processing of laboratory test results. Further research should focus on real-life settings and diverse presentation formats in combination with outcomes related to patients’ information processing.

Introduction

An increasing number of patients have direct access to their own web-based patient portal. This includes diagnostic test results ordered by their health care professional, such as laboratory test results [ 1 , 2 ]. Providing patients with web-based access to patient portals aims to enhance patient involvement in their health management. Improving patients’ knowledge and self-efficacy may enhance disease self-management and interactions with health care providers, and ultimately lead to better health outcomes and increased satisfaction with care [ 3 - 6 ].

However, patient access to web-based patient portals also has potentially negative consequences. For example, misinterpretation or inaccurate knowledge could lead to underestimation of test results and promote a false sense of security [ 7 ]. Similarly, gaining insight into medical test results might trigger feelings of insecurity, anxiety, and confusion [ 8 - 12 ]. Previous studies have shown that poor understanding of test results can lead to an increase in telephone calls or doctor consultations, emergency department visits, and even hospitalizations [ 13 - 15 ]. As a result, the overall utility or benefit of providing lab results directly to patients may depend on how these data are presented to and interpreted by the patient [ 16 , 17 ].

Limited health literacy and numeracy skills are significant barriers to the effective use of web-based patient portals and understanding of laboratory test results [ 18 , 19 ]. Although patient understanding can be improved to some extent by avoiding medical jargon and using plain language, overcoming the problem of incomprehension in its entirety remains an ongoing challenge [ 19 - 21 ]. One of the key issues is the numerical presentation of test results, especially for patients with low numeracy skills (ie, those with limited ability to derive meaning from numbers), who have been shown to have difficulties in interpreting basic laboratory test results and identifying results that fall outside the reference range [ 18 ]. The lack of supporting information and guidance on interpretation of results contributes to the problem of misinterpretation. This challenge becomes even more pronounced when a larger number of test results are presented [ 18 ].

Basic patient portals typically present laboratory test results in a numerical format, often accompanied by a reference range (ie, the range that represents normal values for a particular test) [ 10 , 22 , 23 ]. An alternative approach to communicating test results is the use of visual displays, such as colors or graphics. These formats require less health literacy and numeracy skills for interpretation and may improve patients’ understanding of the results [ 24 - 28 ]. Previous studies have examined a variety of presentation formats for communicating laboratory test results. However, direct comparisons between these studies can be challenging due to the variety of presentation options and clinical contexts. In addition, not all formats may be appropriate for every clinical situation [ 29 ].

There is only limited evidence on the effect of specific presentation formats on patients’ information processing. As highlighted by Witteman and Zikmund-Fisher [ 17 ], laboratory test results often lack meaning for the patients receiving them. Test results represent data, which differs from information and actual knowledge patients commonly encounter in daily life [ 30 , 31 ]. Patients have to complete several steps to go from data perception to usable knowledge. Ancker et al [ 32 ] described these steps as well, based on the Wickens model of human information processing [ 33 ]. In a sequential order, patients need perception and behavioral intention to achieve actual health behavior. Therefore, it is important that these separate steps of information processing are taken into account when presentation formats are evaluated.

Our systematic review aims to synthesize the existing evidence on effective components of presentation formats for laboratory test results focusing on patients’ perception, decision, action, and memory. In this review, we will specifically focus on numeric laboratory test results, and not on results containing only textual or nonnumeric findings.

This review was reported in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses; Multimedia Appendix 1 ) [ 34 ]. A protocol for this review was not previously registered.

Search Strategy

The search was conducted in 3 databases (PubMed, Web of Science, and Embase) from inception up to May 31, 2023. In each database, a search was performed, which was developed by the first author (FM) together with an experienced librarian and contained both thesaurus and free text terms. For the search in Embase, a filter was applied to remove preprint records and to exclude MEDLINE citations, since the latter were already covered by the PubMed search. Additionally, 2 authors (FM and FS) performed backward snowballing by screening reference sections of all selected articles to identify relevant publications missed with the search strategy. A fully reproducible search can be found in Multimedia Appendix 2 .

Study Selection and Eligibility Criteria

All identified titles and abstracts were downloaded to reference management software (Endnote) and duplicates were removed. Two authors (FM and FS) independently screened for potential eligible articles using Covidence, a Cochrane’s technology platform [ 35 ]. First, titles and abstracts were screened against the eligibility criteria. Second, full texts of potentially suitable articles were rescreened using the same criteria. In case of disagreement, consensus was reached by discussion or screening by a third reviewer (JC).

We considered articles fitting for inclusion if they were original research. Studies describing or comparing different ways of presenting diagnostic laboratory test results to patients were included. Only studies examining numeric laboratory test results were included. Furthermore, studies are needed to evaluate the effect of communicating test results on patients’ comprehensibility, attitudes, or experiences. Studies conducted in primary care and secondary/tertiary care settings were eligible, as well as studies including healthy volunteers. Studies had to be written in English or Dutch.

Studies were excluded if they (1) were protocols, reviews, systematic reviews, meta-analyses, book chapters, editorials, letters, practice pointers, oral presentations, or poster presentations; (2) were about development, implementation, or adoption of web-based patient portals in general, or about the type of notification of laboratory test results, if they did not consider patients’ interpretation of the lab results; (3) focused on web-based access to notes, and not to laboratory test results; (4) did not mention type of presentation format of lab results; (5) focused on the development of web-based lifestyle interventions or web-based applications to collect patient-reported outcomes; (6) focused on the safety or privacy issues of web-based patient portals; (7) were about the effect of communicating test results in web-based patient portals on patients’ medication management; (8) tested the effect of test result communication on health care providers; (9) examined communication of other types of diagnostic test results (eg, pharmacogenomics or genomics, radiology, pathology, or microbiology); and (10) examined communication of test results in the context of screening programs.

Data Extraction

Two authors (FM and FS) independently extracted data from the eligible studies into a prepared spreadsheet. The spreadsheet was developed by the multidisciplinary team and piloted by both authors. For each study, the year of publication, country in which the study was performed, study design, number of participants, description of the study population, and the inclusion and exclusion criteria were assessed. Furthermore, information about the presentation of test results in the portal, the type of laboratory tests studied, and whether real or mock data were used, was extracted.

Outcome Measures

Previous research regarding this subject focused on a variety of outcomes related to patients’ information processing. As stated above, Ancker et al [ 32 ] introduced a taxonomy to categorize different outcome measures when communicating numbers in health care. These categories include sequentially; perception, decision/behavioral intention, action/actual health behavior, and memory. Perception is further divided into 4 subcategories: affective perception, perceived magnitude, cognitive perception, and perception of communication [ 32 , 36 , 37 ]. An explanation of the categorized outcome measures can be found in Textbox 1 [ 32 ]. For this review, the outcome measures of each study were extracted and classified into the categories described.

Affective perception

• Feelings about the laboratory result communicated.

Perceived magnitude

• Perceived size of risk associated with a test result, captured with measures as “how large or small does this value seem to you?”

Cognitive perception

• Understanding whether a laboratory result is elevated, normal, or below normal. Being able to identify direction of a trend over time.

Perception of communication

• Preference for presentation format of test result.
• Intention to seek more information or to change behavior after viewing results.
• Change in actual health behavior (eg, search for more information).
• Recall of a specific test result after viewing (ie, verbatim recall).

Quality Assessment

To assess the quality and risk of bias of all included studies, the Mixed Methods Appraisal Tool (MMAT) was used [ 38 ]. The MMAT is designed to concomitantly appraise studies with different designs, such as qualitative, quantitative, and mixed methods studies [ 39 ]. Question sets are specific to the study design, notably qualitative studies, quantitative randomized controlled trials, quantitative nonrandomized studies, quantitative descriptive studies, and mixed methods studies. For each suitable study, the appropriate category was chosen and the criteria stated for this specific category were rated as “yes,” “no,” or “can’t tell.”

Two authors (FM and FS) discussed both data and quality extraction until a consensus was reached.

Data Synthesis

Due to the heterogeneity of study designs and primary outcomes, meta-analysis was considered inappropriate. Instead, narrative synthesis was used to integrate the findings into descriptive summaries regarding ways of presenting laboratory test results and outcomes of interest.

The initial search identified 10,537 references. A total of 3490 duplicate records were removed. After applying the exclusion criteria in the primary title and abstract screening, another 6900 records were removed. During full-text screening of the remaining articles (n=146), it appeared that 1 full text was not available. Furthermore, 127 articles were excluded because they did not meet the eligibility criteria. Describing the implementation of web-based patient portals, unrelated to laboratory test results, was the most common exclusion criterion (55/127, 43.3%; Figure 1 ). A total of 18 studies were found eligible for this systematic review. Cohen κ for interrater reliability was 0.62 for title and abstract screening and 0.80 for full-text screening, indicating respectively a moderate and strong agreement between the 2 reviewers [ 40 ].

Study Characteristics

A total of 2 qualitative studies, 11 quantitative studies, and 5 mixed methods studies were included (n=18). The included studies were published between 2012 and 2021, and the majority were conducted in the United States (n=13, 72%). The total sample size of the included studies was 12,225 participants, ranging from 8 to 6766 participants. Among the articles reporting the following characteristics, sex was almost equally distributed (6219/13,155, 47.3% female), and participants were predominantly middle-aged (mean 51.1 years) and White (8429/10,865, 77.6% on average). Fourteen (78%) of the 18 studies reported educational level, with 48% (5676/11,813) of the participants reporting a higher education (defined as college-degree or higher). Overall characteristics of the included studies and populations are summarized in Table 1 .

a Low education: primary school. Middle education: secondary, high, or trade school or some college. High education: 4-year, college, associate, university, undergraduate, bachelor’s, master’s, advanced, professional, or doctorate degree.

b Not available.

c The following articles are pilot and main studies: Morrow et al (2017) [ 46 ] and (2019) [ 47 ], as well as Zhang et al (2020) [ 15 ] and (2021) [ 53 ].

d The following articles originate from the same parent study: Zikmund-Fisher et al (2017) [ 22 ] and (2018) [ 54 ].

The most frequently used laboratory tests were lipid profile (n=10) and hemoglobin A 1c (HbA 1c ) or glucose (n=5). In total, 4 studies used real patients as study population, other studies used healthy volunteers, a convenience sample, or a mixed sample (n=12) or did not define their study population (n=3). Studies used mock test results (ie, hypothetical results; n=16), real results (n=1, with real patients), or both (n=1). The majority of studies used numerical values with reference ranges (n=12) or horizontal line bars with colored blocks (n=12; Table 2 ). A more detailed overview of the different ways of presenting test results is provided in Multimedia Appendix 3 [ 7 , 15 , 22 , 23 , 41 - 54 ]. An explanation of the different presentation formats can be found in Figure 2 .

a HDL: high-density lipoprotein.

b LDL: low-density lipoprotein.

c eGFR: estimated glomerular filtration rate.

d TG: triglycerides.

e HbA 1c : hemoglobin A1c.

f TSH: thyroid stimulating hormone.

g ALT: alanine aminotransferase.

The quality assessment tool (MMAT) includes 5 assessment criteria per study design, each of which is given a score of 20% if present ( Multimedia Appendix 4 [ 7 , 15 , 22 , 23 , 41 - 54 ]). Both qualitative articles (n=2) scored 100%, indicating excellent quality. Quantitative articles (n=11) scored between 0% and 100%, indicating a broad range of quality. These articles lost points mainly for sampling issues (biased sampling strategies and unrepresentative samples), randomization issues (unclear randomization process and incomparable groups at baseline), unclear blinding process, and lack of clarity about the completeness of outcome data and nonresponse bias. Mixed methods articles (n=5) scored between 60% and 100% (low-to-high quality), for the same reasons as described above. In addition, weaknesses in these articles included having an unclear rationale for using a mixed methods design, unclear presentation format, and failure to adequately interpret the results of the integration of qualitative and quantitative findings.

In all 18 studies, perception was an outcome measure, further categorized into affective perception (n=7), perceived magnitude (n=6), cognitive perception (n=10), and perception of communication (n=14; Table 3 and Textbox 1 ). Additionally, 10 studies assessed behavioral intention, while memory was considered as an outcome measure in 3 of the included studies.

Affective Perception

Several studies explored participants’ confidence and concerns while viewing and interpreting laboratory results [ 15 , 44 , 47 , 49 , 51 ]. Talboom-Kamp et al [ 51 ] demonstrated that presenting laboratory test results in horizontal line bar format with colored blocks and evaluative labels (ie, textual explanation) enhanced participants confidence in managing their own health, although this effect was not significant. No comparison between different presentation formats and the influence on confidence was described. These comparisons were also lacking in the other studies.

When results were presented in a horizontal line bar format with colored blocks and a personalized goal range, the negative affect was significantly higher than when results were presented without colored blocks [ 49 ].

Scherer et al [ 49 ] studied the use of personalized reference values or goal ranges. A type 2 diabetes mellitus scenario was studied, in which standard reference ranges are often not applicable. Replacing standard ranges with goal ranges significantly reduced perceived discouragement compared with situations without goal display, highlighting a positive effect of goal ranges on affective perception [ 49 ]. Furthermore, 2 other studies recommended the use of personalized reference ranges in their discussion [ 44 , 51 ].

In 3 studies, whether laboratory test results were within reference ranges seemed to be more important than the presentation format. As results moved further from the reference range, positive emotions decreased and negative emotions increased [ 15 , 46 , 47 ]. This change in affective perception was not influenced by message format.

Perceived Magnitude

The perceived magnitude of risk of extremely out-of-range results remained unaffected by the presentation formats in all studies. However, for near-normal or slightly out-of-range results participants encountered difficulties in estimating test result severity. Accurate risk perception was lacking, since the severity of these results was inconsistently overestimated or underestimated [ 7 , 22 , 41 , 47 , 54 ]. Zikmund-Fisher et al [ 54 ] demonstrated that the incorporation of harm anchors (ie, a threshold line outside the reference range labeled “many doctors are not concerned until here”) significantly enhanced adequate estimations of test result severity for slightly out-of-range results.

Three studies investigated the effect of presentation format on the perceived size of risk [ 22 , 23 , 47 ]. Morrow et al [ 47 ] compared horizontal line bars with both numerical and video-enhanced formats. For both low- and borderline-risk scenarios, the perceived magnitude of risk was significantly higher when horizontal line bars were used, indicating that participants tend to overestimate risk for normal results [ 47 ]. Tao et al [ 23 ] did not specify whether result normality affected risk perception using different types of horizontal line bars. However, when personalized information was added to the line bar, the risk was perceived as significantly higher. Interestingly, despite this, participants expressed a preference for personalized line bars [ 23 ]. Zikmund-Fisher et al [ 22 ] compared different types of horizontal line bars with a numerical format. Participants expressed the highest risk perception when near-normal results were presented in a numerical format with a reference range, whereas the perceived risk was lowest when horizontal line bars with gradient colors were used [ 22 ].

Cognitive Perception

In all 10 studies assessing this outcome, participants consistently demonstrated the ability to understand or identify out-of-range results. There was consensus among these studies that presenting numbers with a reference range only was insufficient and that tailored information was needed [ 45 , 52 , 53 ]. A qualitative study revealed that participants preferred the inclusion of evaluative labels [ 43 ]. In 2 studies using horizontal line bars as the presentation format, the understanding was significantly increased when color, text, or personalized information (eg, goal range) was added [ 23 , 49 ].

Perception of Communication

The majority of included studies observed a significant association between presentation format, participant satisfaction, and ease of use. In general, satisfaction and ease of use were rated higher when test results were presented using horizontal line bars with colored blocks, as compared with other presentation formats [ 22 , 23 , 42 , 43 , 47 , 51 , 53 ]. In one qualitative study, numerical presentation with reference ranges was deemed insufficient, while graphs were considered too complex for easy comprehension [ 43 ]. Both quantitative and qualitative studies demonstrated that adding evaluative labels, such as explanations about the meaning and normality of test results, and background information about testing, enhanced understanding and effective use of results. Particularly, the use of lay terms played an important role [ 15 , 23 , 44 , 45 , 48 , 51 - 53 ]. Furthermore, 2 studies found a significant positive effect on participant satisfaction when personalized information or goal ranges were incorporated [ 23 , 51 ]. This addition was also recommended by 2 qualitative studies [ 43 , 53 ]. Zikmund-Fisher et al [ 54 ] specifically studied different types of horizontal line bars and showed no significant differences in participants’ preferences among the studied formats.

The behavioral intention was assessed in 10 studies, with varying focuses among them. Some authors examined whether participants would contact their physician [ 7 , 22 , 48 , 49 , 54 ], while others inquired about participants seeking additional web-based information [ 41 , 45 , 48 ], or making lifestyle changes after reviewing lab results [ 47 , 48 , 51 ].

Two studies demonstrated that the presentation format did not significantly influence participants’ need to contact their health care provider [ 7 , 49 ]. Conversely, Zikmund-Fisher et al [ 22 , 54 ] demonstrated in 2 studies that participants who viewed near-normal results in a numerical format were significantly more likely to contact their doctor compared with those viewing the same results in one of the horizontal line formats. The use of harm anchors in horizontal line bars substantially reduced the number of participants who would want to contact their physician [ 22 , 54 ].

Participants’ tendency to seek web-based information was significantly influenced by the presentation format, with a significantly higher inclination observed for the numerical format compared with the textual format [ 41 ]. Kelman et al [ 45 ] and Nystrom et al [ 48 ] similarly found that approximately half of the participants would look for additional information after receiving test results in numerical format with reference ranges and evaluative labels, or horizontal line bars with colored blocks, respectively. However, no comparison was made between presentation formats in these studies [ 45 , 48 ].

Intention to make lifestyle changes after viewing laboratory results was mentioned as an outcome in 3 studies [ 47 , 48 , 51 ]. Only one of these studies compared several presentation formats but found no significant differences between using a numerical format, horizontal line bars with colored blocks, or video-enhanced format in terms of health-beneficial intentions [ 47 ].

There was limited data concerning the actions patients take to comprehend their test results. One mixed methods study used a numerical format with reference ranges as a presentation format [ 15 ]. Participants with abnormal test results were significantly more likely to take action compared with those with normal test results. As no comparison between presentation formats was investigated, the effect of format on action remains unstudied.

Variation in the presentation format of test results, using either a numerical format with reference ranges and evaluative labels, horizontal line bars with colored blocks, video presentation, or grouped presentation, did not significantly impact participant recall [ 7 , 42 , 47 ]. However, one study found a small but statistically significant effect of test result normality on memory [ 47 ].

Struikman et al [ 50 ] looked at patient health engagement (PHE), a composite measure comprising affective perception, cognitive perception, and behavioral intention. When test results were presented with explanatory text and visualization, PHE was significantly higher compared with when no explanatory information was provided [ 50 ].

Principal Findings

Based on reviewing 18 articles assessing various presentation formats of laboratory test results, we can conclude there is not only one optimal presentation format in terms of patients’ perception, decision, action, and memory. Nevertheless, the results do indicate that presentation format is important for patients’ information processing.

Presentation formats differed between articles, but numerical values with reference ranges or horizontal line bars with colored blocks were most commonly used. All included studies investigated perception as an outcome measure, most frequently perception of communication (n=14). Patients’ cognitive perception and perception of communication improved when results were presented using horizontal line bars accompanied with colored blocks and evaluative labels or textual information. Incorporation of reference ranges or personalized goal ranges further enhanced patients’ perception levels. Using horizontal line bars with harm anchors significantly reduced the number of participants who would want to contact their physician compared with using a numerical format. Furthermore, using the numerical format significantly increased participants’ tendency to search for web-based information, compared with a textual format. Therefore, although no specific format is dissuaded in the included studies, the results suggest that presenting only numbers with reference ranges is suboptimal. Furthermore, adding too many colors and other information to test results could lead to an overload of visual information for some patients, and therefore ultimately decrease the amount of usable knowledge [ 49 ]. Action and memory were less frequently studied, respectively in 1 and 3 studies. Action was studied in a descriptive study not comparing different presentation formats, while memory was not significantly impacted by presentation format.

Several studies highlighted that patients’ affective perception, action, and memory were not only influenced by presentation format, but also by whether test results were within or outside the reference range. Presentation format appeared to be secondary to test result normality if results were extremely out-of-range. Nevertheless, when results were near-normal, presentation format was more important than result normality regarding effects on patients’ information processing.

Overall, the results of this review indicate that presentation format affects patients’ information processing, especially in the case of normal or near-normal test results.

Strengths and Limitations

A multidisciplinary team of general practitioners, behavioral scientists, and clinical chemists was involved in this review, which is one of its strengths. Both presentation formats and outcomes used in the included studies were standardized by the authors using a published taxonomy to enable comparison of different studies. As the results of our review are narrative, there is a potential risk of bias when describing them, introduced by the authors. Furthermore, selection bias arising from the heterogeneity of studies represents a notable limitation of this review.

A limitation of the included studies is the use of volunteers or participants recruited via convenience sampling. Only 3 out of 18 studies used real patients, of which one study used real test results. Sixteen studies used mock test results. Displaying mock data is common practice in system evaluation. This method involves less burden and privacy risks for participants, as no personal medical data are collected. Nonetheless, participants lack personal relevance of test results when hypothetical scenarios are used. Therefore, it is possible that most of the included studies did not reflect how participants would respond in real life to their own personal health information. This may limit the generalizability of the findings. However, using personal test results could have negatively affected the comparability between studies, as each participant would have encountered different data.

Among the articles reporting educational level, 48% (5676/11,813) of the participants reported a higher education level, which is higher than in the general population. This may limit the generalizability of the findings to the overall population. Another limitation is the study heterogeneity. Included articles varied widely in methods, presentation formats, and outcome measures used. Comparison of presentation formats is challenging, especially since laboratory test result communication can have a wide range of possible purposes, from interpreting one single value to identifying important trends on time [ 24 ]. Therefore, useful presentation formats may vary per clinical scenario, which presents new challenges for designing a preferred format.

As stated above, patients have to complete several steps to go from data perception to usable knowledge [ 17 , 32 ]. The majority of the included studies studied the first 2 steps of this taxonomy, perception and decision. Only one study examined action as outcome measure, and 3 studies obtained information about memory. Therefore, little is known about the impact of presentation formats on actual health behavior and usable knowledge.

Comparison With Prior Work

An increasing number of patients can directly access their laboratory test results via web; thus, it is becoming more important to make the available data meaningful to laypeople [ 55 ]. As highlighted in this review, presentation format affects patients’ information processing as described above. In cognitive science, this principle is generally known as information evaluability, in other words using relevant contextual reference information to make it easier to evaluate the meaning of in this case numerical laboratory test results (eg, is this test result good or bad, is it normal or abnormal) [ 56 , 57 ]. The presentation formats for laboratory test results as studied in this review could be considered as different forms of contextual information, or evaluative categories [ 58 ]. Prior research has shown that these evaluative categories add both affective and cognitive meaning to numeric test results. This enhances patients’ information processing by adding meaning and evaluability to numeric information [ 58 - 60 ]. Furthermore, our findings are in line with recommendations made by Witteman and Zikmund-Fisher [ 17 ]. The authors formulated 10 recommendations to communicate laboratory test results via web-based portals in ways that support understanding and actionable knowledge for patients. Our findings align with several of their recommendations, such as the importance of providing a clear takeaway message for each result, establishing thresholds for concern and action whenever feasible, and personalizing the frame of reference by permitting custom reference ranges.

This review explored different strategies to improve patients’ interpretation and comprehension of their laboratory test results. The included studies predominantly focused on the effect of graphical presentation only including a subset of the available visualization options. Other formats such as clocks or pie charts been shown in the broader numeracy literature to improve cognitive outcomes and could be the focus of further research in the context of communicating laboratory test results [ 61 ]. Graphical presentation formats might mitigate the effects of low numeracy. However, it is important to acknowledge that graphical information may not be automatically useful for individuals with limited graph literacy [ 62 ]. Besides numeracy and graph literacy, other factors such as age, educational level, health literacy, and statistical literacy (eg, understanding of concepts of uncertainty and chance) also influence patients’ information processing of such graphical results [ 61 - 63 ]. If one of these factors causes patients to not completely understand a specific presentation format, they may consider this format as not suitable. Therefore, some patients may require extra instructions to be able to adequately process and interpret graphical presentation formats [ 61 ]. For that reason, the interaction between patients’ literacy, numeracy, age, and educational level should be taken into account when performing future work around test result interpretation.

Several initiatives aim to inform and educate patients about laboratory test results while incorporating the insights described above. One example is Lab Tests Online, a website that provides patients with general information about laboratory tests and their meaning [ 64 ]. Recently, the usability of ChatGPT (ie, an upcoming tool based on natural language processing) to interpret laboratory test results were examined [ 65 ]. ChatGPT appeared to provide somewhat superficial interpretations, which were not always correct, and is therefore not yet usable as a primary information source for patients. However, this may change in the future with the further development of these types of tools. While our review focused on different presentation formats of laboratory test results, interpretative comments provided by laboratory specialists were not studied. Laboratory specialists often add comments to test results to assist general practitioners [ 66 , 67 ]. A pilot study by Verboeket-van de Venne et al [ 68 ] demonstrated a positive impact on patient empowerment when patients had access to these patient-specific comments. Therefore, further research should explore the impact of adding interpretative comments to laboratory test results on patients’ information processing.

Patients now have web-based access to not only their laboratory test results but also to medical imaging and microbiology results. Given the variations in these types of diagnostic test results, further research is warranted to explore effective components for communicating these other types of test results to patients in their web-based patient portal.

Conclusions

As patients increasingly receive their diagnostic laboratory test results via web-based patient portals, it is becoming more and more important to make test results meaningful to them. Unnecessary confusion or anxiety should be avoided, especially when test results are outside of the reference range. The data from our systematic review suggest that horizontal line bars with colored blocks and reference ranges or personalized goal range increase patients’ cognitive perception and perception of communication. Furthermore, this format might reduce patients’ concerns and their tendency to contact their physicians. Therefore, to improve patients’ understanding of near-normal laboratory test results and prevent anxiety and concerns after viewing these results, implementing horizontal line bars with colored blocks and reference ranges or personalized goal ranges in web-based patient portals would be a prudent choice. Our review highlights the importance of taking end users (ie, patients) into consideration when designing new presentation formats. These results can guide the development and improvement of (new) web-based patient portals. Nevertheless, there is a need for further research that involves more comprehensive data collection and reporting, as well as more systematic evaluation methods. By using these findings, further research could inform the development of an interpretation support tool for laboratory test results.

Acknowledgments

This study received funding from a partnership between Care Research Netherlands and the Medical Sciences domain of the Dutch Research Council (Dutch abbreviation ZonMW; project 08391052120002). ZonMW had no direct involvement in any facet of the design of the study, analysis or interpretation of the data, or writing of the manuscript.

Authors' Contributions

All authors contributed to the conception of the review. FM and FS conducted the screening and data extraction. FM and FS drafted the manuscript. JJ, JB, SM, and JC gave guidance throughout the whole research process. All authors critically revised and approved the manuscript.

Conflicts of Interest

None declared.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 checklist.

Search strategy for PubMed, Web of Science, and Embase up to May 31, 2023.

Detailed overview of laboratory test results presentation formats in all included studies (n=18).

Detailed overview of quality assessment of all included studies (n=18).

• Kaelber DC, Jha AK, Johnston D, Middleton B, Bates DW. A research agenda for personal health records (PHRs). J Am Med Inform Assoc. 2008;15(6):729-736. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc. 2006;13(2):121-126. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ancker JS, Osorio SN, Cheriff A, Cole CL, Silver M, Kaushal R. Patient activation and use of an electronic patient portal. Inform Health Soc Care. 2015;40(3):254-266. [ CrossRef ] [ Medline ]
• Lee CI, Langlotz CP, Elmore JG. Implications of direct patient online access to radiology reports through patient web portals. J Am Coll Radiol. 2016;13(12 Pt B):1608-1614. [ CrossRef ] [ Medline ]
• Fisher B, Bhavnani V, Winfield M. How patients use access to their full health records: a qualitative study of patients in general practice. J R Soc Med. 2009;102(12):539-544. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bhavnani V, Fisher B, Winfield M, Seed P. How patients use access to their electronic GP record--a quantitative study. Fam Pract. 2011;28(2):188-194. [ CrossRef ] [ Medline ]
• Fraccaro P, Vigo M, Balatsoukas P, van der Veer SN, Hassan L, Williams R, et al. Presentation of laboratory test results in patient portals: influence of interface design on risk interpretation and visual search behaviour. BMC Med Inform Decis Mak. 2018;18(1):11. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Parker RM, Ratzan SC, Lurie N. Health literacy: a policy challenge for advancing high-quality health care. Health Aff (Millwood). 2003;22(4):147-153. [ CrossRef ] [ Medline ]
• Garrido T, Jamieson L, Zhou Y, Wiesenthal A, Liang L. Effect of electronic health records in ambulatory care: retrospective, serial, cross sectional study. BMJ. 2005;330(7491):581. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Krist AH, Woolf SH. A vision for patient-centered health information systems. JAMA. 2011;305(3):300-301. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pillemer F, Price RA, Paone S, Martich GD, Albert S, Haidari L, et al. Direct release of test results to patients increases patient engagement and utilization of care. PLoS One. 2016;11(6):e0154743. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sung S, Forman-Hoffman V, Wilson MC, Cram P. Direct reporting of laboratory test results to patients by mail to enhance patient safety. J Gen Intern Med. 2006;21(10):1075-1078. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Giardina TD, Baldwin J, Nystrom DT, Sittig DF, Singh H. Patient perceptions of receiving test results via online portals: a mixed-methods study. J Am Med Inform Assoc. 2018;25(4):440-446. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Palen TE, Ross C, Powers JD, Xu S. Association of online patient access to clinicians and medical records with use of clinical services. JAMA. 2012;308(19):2012-2019. [ CrossRef ] [ Medline ]
• Zhang Z, Citardi D, Xing A, Luo X, Lu Y, He Z. Patient challenges and needs in comprehending laboratory test results: mixed methods study. J Med Internet Res. 2020;22(12):e18725. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Irizarry T, DeVito Dabbs A, Curran CR. Patient portals and patient engagement: a state of the science review. J Med Internet Res. 2015;17(6):e148. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Witteman HO, Zikmund-Fisher BJ. Communicating laboratory results to patients and families. Clin Chem Lab Med. 2019;57(3):359-364. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zikmund-Fisher BJ, Exe NL, Witteman HO. Numeracy and literacy independently predict patients' ability to identify out-of-range test results. J Med Internet Res. 2014;16(8):e187. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Keselman A, Smith CA. A classification of errors in lay comprehension of medical documents. J Biomed Inform. 2012;45(6):1151-1163. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McDonald KM, Bryce CL, Graber ML. The patient is in: patient involvement strategies for diagnostic error mitigation. BMJ Qual Saf. 2013;22 Suppl 2(Suppl 2):ii33-ii39. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Britto MT, Jimison HB, Munafo JK, Wissman J, Rogers ML, Hersh W. Usability testing finds problems for novice users of pediatric portals. J Am Med Inform Assoc. 2009;16(5):660-669. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zikmund-Fisher BJ, Scherer AM, Witteman HO, Solomon JB, Exe NL, Tarini BA, et al. Graphics help patients distinguish between urgent and non-urgent deviations in laboratory test results. J Am Med Inform Assoc. 2017;24(3):520-528. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tao D, Yuan J, Qu X. Presenting self-monitoring test results for consumers: the effects of graphical formats and age. J Am Med Inform Assoc. 2018;25(8):1036-1046. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Turchioe MR, Myers A, Isaac S, Baik D, Grossman LV, Ancker JS, et al. A systematic review of patient-facing visualizations of personal health data. Appl Clin Inform. 2019;10(4):751-770. [ FREE Full text ] [ CrossRef ] [ Medline ]
• O'Kane M, Freedman D, Zikmund-Fisher BJ. Can patients use test results effectively if they have direct access? BMJ. 2015;350:h673. [ CrossRef ] [ Medline ]
• Zikmund-Fisher BJ, Witteman HO, Dickson M, Fuhrel-Forbis A, Kahn VC, Exe NL, et al. Blocks, ovals, or people? icon type affects risk perceptions and recall of pictographs. Med Decis Making. 2014;34(4):443-453. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Watson ID. Making test results more easily understood by patients. BMJ. 2015;350:h1942. [ CrossRef ] [ Medline ]
• Garcia-Retamero R, Cokely ET. Designing visual aids that promote risk literacy: a systematic review of health research and evidence-based design heuristics. Hum Factors. 2017;59(4):582-627. [ CrossRef ] [ Medline ]
• Torsvik T, Lillebo B, Mikkelsen G. Presentation of clinical laboratory results: an experimental comparison of four visualization techniques. J Am Med Inform Assoc. 2013;20(2):325-331. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Boisot M, Canals A. Data, information and knowledge: have we got it right? J. Evol. Econ. 2004;14:43-67. [ CrossRef ]
• Chen M, Ebert D, Hagen H, Laramee RS, van Liere R, Ma KL, et al. Data, information, and knowledge in visualization. IEEE Comput Graph Appl. 2009;29(1):12-19. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ancker JS, Benda NC, Sharma MM, Johnson SB, Weiner S, Zikmund‐Fisher BJ. Taxonomies for synthesizing the evidence on communicating numbers in health: goals, format, and structure. Risk Analysis. 2022;42(12):2656-2670. [ CrossRef ]
• Wickens CD, Helton W, Hollands JG, Parasuraman R, Banbury S. Engineering Psychology & Human Performance. United Kingdom. Routledge; 2021.
• Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Covidence Systematic Review Software, Veritas Health Innovation, Melbourne, Australia. URL: https://www.covidence.org/ [accessed 2023-08-25]
• Becker MH. The health belief model and sick role behavior. Health Education Monographs. 1974;2(4):324-508. [ CrossRef ]
• Witte K. Putting the fear back into fear appeals: the extended parallel process model. Commun Monogr. 1992;59(4):329-349. [ CrossRef ]
• Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. EFI. 2018;34(4):285-291. [ CrossRef ]
• Pace R, Pluye P, Bartlett G, Macaulay AC, Salsberg J, Jagosh J, et al. Testing the reliability and efficiency of the pilot mixed methods appraisal tool (MMAT) for systematic mixed studies review. Int J Nurs Stud. 2012;49(1):47-53. [ CrossRef ] [ Medline ]
• McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276-282. [ FREE Full text ] [ Medline ]
• Bar-Lev S, Beimel D. Numbers, graphs and words - do we really understand the lab test results accessible via the patient portals? Isr J Health Policy Res. 2020;9(1):58. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brewer NT, Gilkey MB, Lillie SE, Hesse BW, Sheridan SL. Tables or bar graphs? presenting test results in electronic medical records. Med Decis Making. 2012;32(4):545-553. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Elder NC, Barney K. "But what does it mean for me?" primary care patients' communication preferences for test results notification. Jt Comm J Qual Patient Saf. 2012;38(4):168-176. [ CrossRef ] [ Medline ]
• Hohenstein JC, Baumer EP, Reynolds L, Murnane EL, O'Dell D, Lee S, et al. Supporting accurate interpretation of self-administered medical test results for mobile health: assessment of design, demographics, and health condition. JMIR Hum Factors. 2018;5(1):e9. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kelman A, Robinson CO, Cochin E, Ahluwalia NJ, Braverman J, Chiauzzi E, et al. Communicating laboratory test results for rheumatoid factor: what do patients and physicians want? Patient Prefer Adherence. 2016;10:2501-2517. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Morrow D, Hasegawa-Johnson M, Huang T, Schuh W, Azevedo RFL, Gu K, et al. A multidisciplinary approach to designing and evaluating electronic medical record portal messages that support patient self-care. J Biomed Inform. 2017;69:63-74. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Morrow D, Azevedo RFL, Garcia-Retamero R, Hasegawa-Johnson M, Huang T, Schuh W, et al. Contextualizing numeric clinical test results for gist comprehension: implications for EHR patient portals. J Exp Psychol Appl. 2019;25(1):41-61. [ CrossRef ] [ Medline ]
• Nystrom DT, Singh H, Baldwin J, Sittig DF, Giardina TD. Methods for patient-centered interface design of test result display in online portals. EGEMS (Wash DC). 2018;6(1):15. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Scherer AM, Witteman HO, Solomon J, Exe NL, Fagerlin A, Zikmund-Fisher BJ. Improving the understanding of test results by substituting (not adding) goal ranges: web-based between-subjects experiment. J Med Internet Res. 2018;20(10):e11027. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Struikman B, Bol N, Goedhart A, van Weert JCM, Talboom-Kamp E, van Delft S, et al. Features of a patient portal for blood test results and patient health engagement: web-based pre-post experiment. J Med Internet Res. 2020;22(7):e15798. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Talboom-Kamp E, Tossaint-Schoenmakers R, Goedhart A, Versluis A, Kasteleyn M. Patients' attitudes toward an online patient portal for communicating laboratory test results: real-world study using the eHealth impact questionnaire. JMIR Form Res. 2020;4(3):e17060. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zarcadoolas C, Vaughon WL, Czaja SJ, Levy J, Rockoff ML. Consumers' perceptions of patient-accessible electronic medical records. J Med Internet Res. 2013;15(8):e168. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhang Z, Kmoth L, Luo X, He Z. User-centered system design for communicating clinical laboratory test results: design and evaluation study. JMIR Hum Factors. 2021;8(4):e26017. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zikmund-Fisher BJ, Scherer AM, Witteman HO, Solomon JB, Exe NL, Fagerlin A. Effect of harm anchors in visual displays of test results on patient perceptions of urgency about near-normal values: experimental study. J Med Internet Res. 2018;20(3):e98. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zikmund-Fisher BJ. Helping people know whether measurements have good or bad implications: increasing the evaluability of health and science data communications. PIBBS. 2019;6(1):29-37. [ CrossRef ]
• Hsee CK. The evaluability hypothesis: an explanation for preference reversals between joint and separate evaluations of alternatives. Organ Behav Hum Decis Process. 1996;67(3):247-257. [ CrossRef ]
• Hsee CK, Zhang J. General evaluability theory. Perspect Psychol Sci. 2010;5(4):343-355. [ CrossRef ] [ Medline ]
• Peters E, Dieckmann NF, Västfjäll D, Mertz CK, Slovic P, Hibbard JH. Bringing meaning to numbers: the impact of evaluative categories on decisions. J Exp Psychol Appl. 2009;15(3):213-227. [ CrossRef ] [ Medline ]
• Zikmund-Fisher BJ, Fagerlin A, Keeton K, Ubel PA. Does labeling prenatal screening test results as negative or positive affect a woman's responses? Am J Obstet Gynecol. 2007;197(5):528. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peters E. The functions of affect in the construction of preferences. In: The Construction of Preference. New York. Cambridge University Press; 2006:454-463.
• Ancker JS, Senathirajah Y, Kukafka R, Starren JB. Design features of graphs in health risk communication: a systematic review. J Am Med Inform Assoc. 2006;13(6):608-618. [ FREE Full text ] [ CrossRef ] [ Medline ]
• van Weert JCM, Alblas MC, van Dijk L, Jansen J. Preference for and understanding of graphs presenting health risk information. the role of age, health literacy, numeracy and graph literacy. Patient Educ Couns. 2021;104(1):109-117. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ancker JS, Kaufman D. Rethinking health numeracy: a multidisciplinary literature review. J Am Med Inform Assoc. 2007;14(6):713-721. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Campbell B, Linzer G, Dufour DR. Lab tests online and consumer understanding of laboratory testing. Clin Chim Acta. 2014;432:162-165. [ CrossRef ] [ Medline ]
• Cadamuro J, Cabitza F, Debeljak Z, de Bruyne S, Frans G, Perez SM, et al. Potentials and pitfalls of ChatGPT and natural-language artificial intelligence models for the understanding of laboratory medicine test results. an assessment by the european federation of clinical chemistry and laboratory medicine (EFLM) working group on artificial intelligence (WG-AI). Clin Chem Lab Med. 2023;61(7):1158-1166. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Barlow IM. Are biochemistry interpretative comments helpful? results of a general practitioner and nurse practitioner survey. Ann Clin Biochem. 2008;45(Pt 1):88-90. [ CrossRef ] [ Medline ]
• Verboeket-van de Venne WPHG, Oosterhuis WP, Keuren JFW, Kleinveld HA. Reflective testing in the Netherlands: usefulness to improve the diagnostic and therapeutic process in general practice. Ann Clin Biochem. 2009;46(Pt 4):346-347. [ CrossRef ] [ Medline ]
• Verboeket-van de Venne WPHG, Hendriks-Dybicz AM, Oosterhuis WP. Patiënten informeren over laboratoriumuitslagen in het kader van patient empowerment [Inform patients about laboratory test results in the context of patient empowerment]. Ned Tijdschr Klin Chem Labgeneesk [Dutch Journal of Clinical Chemistry and Laboratory Medicine]. 2015. URL: https://www.nvkc.nl/files/ntkc/N60_049408_BW_NVKC_Juli2015_WQ_012.pdf [accessed 2023-09-07]

Abbreviations

Edited by A Mavragani; submitted 26.10.23; peer-reviewed by B Zikmund-Fisher, B Steitz; comments to author 26.02.24; revised version received 07.04.24; accepted 27.05.24; published 12.08.24.

©Frederieke A M van der Mee, Fleur Schaper, Jesse Jansen, Judith A P Bons, Steven J R Meex, Jochen W L Cals. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 12.08.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• Introduction
• Conclusions
• Article Information

This histogram plots the overall distribution of surgeons’ preference (probability intraoperative TEE use by surgeon). The blue lines demarcate surgeons with equivocal preference for intraoperative TEE (eg, surgeons with probability of TEE use between 0.30 and 0.70).

Overview of the study design. The left panels illustrate the all-patient, across-hospital, across-surgeon matched comparison. The right panels illustrate the within-hospital, within-surgeon (with equivocal TEE preference: TEE 0.30–0.70) matched comparison.

eAppendix 1. Descriptive Statistics of the Study Cohort

eAppendix 2. Definition of Primary, Secondary, and Negative Control Outcomes

eAppendix 3. Variability in TEE Preference

eAppendix 4. Details on Statistical Matching Methodology

eAppendix 5. Covariate Balance After Matching

eAppendix 6. Additional Details on Outcome Analysis

eAppendix 7. Subgroup Analysis

eAppendix 8. Sensitivity Analysis I: Assessing Robustness of Primary Outcome Findings to Unmeasured Confounding

eAppendix 9. Sensitivity Analysis II: Assessing Robustness of Results to the Missingness of the TEE Status

eAppendix 10. Details On the Negative Control Outcome

eAppendix 11. R and Stata Code

eReferences

See More About

Sign up for emails based on your interests, select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing

Get the latest research based on your areas of interest.

Others also liked.

• Download PDF
• X Facebook More LinkedIn

MacKay EJ , Zhang B , Augoustides JG , Groeneveld PW , Desai ND. Association of Intraoperative Transesophageal Echocardiography and Clinical Outcomes After Open Cardiac Valve or Proximal Aortic Surgery. JAMA Netw Open. 2022;5(2):e2147820. doi:10.1001/jamanetworkopen.2021.47820

Manage citations:

© 2024

• Permissions

Association of Intraoperative Transesophageal Echocardiography and Clinical Outcomes After Open Cardiac Valve or Proximal Aortic Surgery

• 1 Department of Anesthesiology and Critical Care, Perelman School of Medicine at the University of Pennsylvania, Philadelphia
• 2 Penn Center for Perioperative Outcomes Research and Transformation (CPORT), University of Pennsylvania, Philadelphia
• 3 Penn’s Cardiovascular Outcomes, Quality and Evaluative Research Center (CAVOQER), University of Pennsylvania, Philadelphia
• 4 Leonard Davis Institute of Health Economics (LDI), University of Pennsylvania, Philadelphia
• 5 Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia
• 6 Department of Internal Medicine, Perelman School of Medicine at the University of Pennsylvania, Philadelphia
• 7 Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia
• 8 Division of Cardiovascular Surgery, Perelman School of Medicine at the University of Pennsylvania, Philadelphia

Question   Is intraoperative transesophageal echocardiography (TEE) use associated with improved clinical outcomes among patients undergoing cardiac valve or proximal aortic surgery?

Findings   This matched cohort study of 872 936 patients undergoing cardiac valve or aortic surgery between 2011 and 2019 found that intraoperative TEE use was associated with lower 30-day mortality, a lower incidence of stroke or 30-day mortality, and a lower incidence of cardiac reoperation or 30-day mortality.

Meaning   These findings suggest that intraoperative TEE may improve clinical outcomes after open cardiac valve (repair or replacement) and/or aortic surgery.

Importance   Intraoperative transesophageal echocardiography (TEE) is used frequently in cardiac valve and proximal aortic surgical procedures, but there is a lack of evidence associating TEE use with improved clinical outcomes.

Objective   To test the association between intraoperative TEE use and clinical outcomes following cardiac valve or proximal aortic surgery.

Design, Setting, and Participants   This matched, retrospective cohort study used national registry data from the Society of Thoracic Surgeon (STS) Adult Cardiac Surgery Database (ACSD) to compare clinical outcomes among patients undergoing cardiac valve or proximal aortic surgery with vs without intraoperative TEE. Statistical analyses used optimal matching within propensity score calipers to conduct multiple matched comparisons including within-hospital and within-surgeon matches, a negative control outcome analysis, and sensitivity analyses. STS ACSD data encompasses more than 90% of all hospitals that perform cardiac surgery in the US. The study cohort consisted of all patients aged at least 18 years undergoing open cardiac valve repair or replacement surgery and/or proximal aortic surgery between 2011 and 2019. Statistical analysis was performed from October 2020 to April 2021.

Exposures   The exposure was receipt of intraoperative TEE during the cardiac valve or proximal aortic surgery.

Main Outcomes and Measures   The primary outcome was death within 30 days of surgery. The secondary outcomes were (1) a composite outcome of stroke or 30-day mortality and (2) a composite outcome of reoperation or 30-day mortality.

Results   Of the 872 936 patients undergoing valve or aortic surgery, 540 229 (61.89%) were male; 63 565 (7.28%) were Black and 742 384 (85.04%) were White; 711 326 (81.5%) received TEE and 161 610 (18.5%) did not receive TEE; the mean (SD) age was 65.61 years (13.17) years. After matching, intraoperative TEE was significantly associated with a lower 30-day mortality rate compared with no TEE: 3.81% vs 5.27% (odds ratio [OR], 0.69 [95% CI, 0.67-0.72]; P  < .001), a lower incidence of stroke or 30-day mortality: 5.56% vs 7.01% (OR, 0.77 [95% CI, 0.74-0.79]; P  < .001), and a lower incidence of reoperation or 30-day mortality: 7.18% vs 8.87% (OR, 0.78 [95% CI, 0.76-0.80]; P  < .001). Results were similar across all matched comparisons (including within-hospital, within-surgeon matched analyses) and were robust to a negative control and sensitivity analyses.

Conclusions and Relevance   Among adults undergoing cardiac valve or proximal aortic surgery, intraoperative TEE use was associated with improved clinical outcomes in this cohort study. These findings support routine use of TEE in these procedures.

Each year, 150 000 patients undergo high-risk, 1 - 3 open cardiac valve or proximal aortic surgery in the US. 4 Transesophageal echocardiography (TEE) is an ultrasonography-based, cardiac imaging tool used in cardiac surgical procedures to facilitate informed surgical decision making 5 - 7 and manage intraoperative complications. 5 - 9 However, the current American Heart Association (AHA) and American College of Cardiology (ACC) guidelines do not specifically recommend for or against the use of intraoperative TEE in the majority of cardiac surgical procedures 10 - 12 because prior to 2020, observational studies have not directly associated intraoperative TEE with improved clinical outcomes. 5 - 9 Recently, evidence has begun to accumulate on improved outcomes with TEE use after cardiac valve and coronary artery bypass graft (CABG) surgical procedures. 13 - 15 But there is no research directly comparing outcomes after proximal aortic surgery with TEE vs without TEE, and the existing study on improved outcomes with TEE after cardiac valve repair or replacement surgery used administrative claims data. 15

To go beyond prior observational work using claims data, 15 , 16 this study aimed to test the association between intraoperative TEE and clinical outcomes using data from the Society of Thoracic Surgeon (STS), Adult Cardiac Surgery Data (ACSD) registry database. These STS data allowed the application of rigorous statistical matching techniques to directly compare similar patients who underwent cardiac valve or proximal aortic surgery with vs without intraoperative TEE. We hypothesized that intraoperative TEE would be associated with a decreased incidence of 30-day mortality, stroke or 30-day mortality, and reoperation or 30-day mortality.

The STS ACSD contains 6.9 million surgical records, has 3800 participating surgeons, and encompasses more than 90% of the hospitals that perform cardiac surgery in the US. 17 For this analysis, data across STS ACSD versions 2.73, 2.81, and 2.90 were queried. 18 All data management and statistical analyses were performed in accordance with the STS Participant User Files Data Use Agreement. Our study adheres to the Strengthening the Reporting of Observational Studies in Epidemiology ( STROBE ) reporting guideline for observational studies. 19 All aspects of this study were reviewed and approved by the University of Pennsylvania institutional review board and informed consent was waived given the deidentified nature of the data.

This study’s data (including race and ethnicity) were collected as variables retrospectively by each institution participating in the STS ACSD. Because this study analyzed national data across multiple institutions, it is unknown how each institution recorded race and ethnicity (eg, whether by electronic medical record or by patient report). However, the STS Research Center quality-controls all variables by audit regularly.

The study cohort consisted of all patients aged at least 18 years undergoing at least one of the following surgical procedures between July 1, 2011, and June 30, 2019: (1) open valve (aortic, mitral, pulmonic, or tricuspid) repair or replacement; (2) open, ascending aortic, and/or proximal aortic arch surgery (eg, aortic root replacement with a valved conduit, aortic valve sparing root, aortic homograft, or nonvalved conduit replacement with or without aortic hemiarch replacement). Patients were excluded if undergoing any of the following surgical procedures: (1) isolated CABG surgery; (2) isolated other cardiac surgery; (3) unspecified valve repair or replacement surgery; or (4) unspecified aortic surgery.

Our primary outcome was death within 30 days of surgery. 20 Our secondary outcomes were (1) composite of in-hospital stroke or 30-day mortality; or (2) composite of in-hospital reoperation (for bleeding, valve or CABG reintervention) or 30-day mortality. Information on outcome variable labels across STS ACSD versions (eg, 2.73, 2.81, and 2.90) may be found in eAppendix 2 in the Supplement .

The exposure variable was receipt of an intraoperative TEE. This was defined using the STS ACSD variable called intraoperative TEE post procedure (consistent across versions 2.73, 2.81, and 2.90).

Independent covariates were used for matching. The categories included: demographics, admission status, preexisting comorbidities, hemodynamic data, laboratory values, intraoperative surgical variables, surgery type, surgical volume by hospital and surgeon and STS projected risk scores.

Because of baseline covariate differences (eAppendix 1 in the Supplement ) between patients undergoing cardiac valve or proximal aortic surgery with vs without intraoperative TEE, we performed 2 matched comparisons. 21 , 22 First, an all-patient, across-hospital, across-surgeon matched comparison and a second, within-hospital, within-surgeon matched comparison. All matched analyses involved exact matching on key covariates, finely balancing the joint distribution of key nominal variables, 23 and balanced on all remaining variables. The all-patient matched comparison was based on optimal matching within propensity score caliper. The 2 within-surgeon matched comparisons were based on optimal subset matching. A detailed discussion on statistical matching methodology is presented in eAppendix 4 in the Supplement .

In the first all-patient, across-hospital, across-surgeon, matched comparison, each patient who did not receive a TEE was matched to a comparable patient who did receive TEE during surgery. To ensure each matched pair of patients were as similar as possible, we applied strict matching criteria. First, we matched exactly on New York Heart Association (NYHA) Classification (1 to 4 or absent) and projected 30-day mortality by quartile. Next, because TEE differed across surgery types (eAppendix 1 in the Supplement ), we finely balanced 23 on the 9 major surgery types, secondary procedures, an indicator of previous cardiac surgery, (ie, redo sternotomy) normal ejection fraction (eg, EF greater than or equal to 55%), preexisting hypertension, and admission status. Finally, we balanced all other variables under the categories of demographics, preexisting conditions, hemodynamic data, laboratory values, intraoperative surgical variables, cardiac surgical volume (by hospital and surgeon), and surgery type. Optimal matching within propensity score calipers was implemented using the R package bigmatch. See eAppendix 4 in the Supplement for details on the overall statistical matching methodology and for the all-patient matched comparison.

Because there were differences in TEE by hospital and by surgeon (eAppendix 3 in the Supplement ), we elected to undertake a second, within-hospital, within-surgeon, matched comparison. For this analysis, each patient undergoing a valve or aortic surgery at a given hospital, by a given surgeon with TEE, was matched to a similar patient undergoing valve or aortic surgery at that same hospital by the same surgeon without TEE. Because intraoperative TEE varied predominantly by surgery type (eAppendix 1 in the Supplement ), we applied exact matching to all 9 surgical categories. Covariates used for exact matching included (1) hospital; (2) surgeon; (3) all 9 surgery types; (4) normal EF ( at least 55%); (5) NYHA Classification; and (6) projected 30-day mortality (by quartile). As was done in the all-patient match, we balanced all other variables as specified above. To further reduce selection bias that could occur with surgeons who always (or never) used TEE during cardiac surgery, we considered only surgeons whose preference for TEE was equivocal (TEE probability range of 0.30 to 0.70) ( Figure 1 ). Finally, we elected to conduct an additional, supplementary, within-hospital, within-surgeon matched comparison across all surgeons, regardless of intraoperative TEE frequency (TEE probability range 0.00 to 1.00). To characterize differences in outcomes by surgery type (with TEE vs without), we performed subgroup analyses among patients undergoing similar surgical procedures. These subgroups were categorized based on anatomical location of surgery, surgery type, and those with similar risk profiles. The surgical procedures classified into each subgroup may be found in eAppendix 7 in the Supplement . Statistical matching was implemented using the R package rcbsubset with default settings.

The quality of statistical matching was assessed using standardized differences (SD). A match was considered acceptable if all covariates had a SD less than 0.10 between the TEE and the no-TEE groups. 21 , 24 , 25

We first conducted an unmatched, unadjusted, analysis of outcomes, where patients undergoing cardiac valve or aortic surgery with vs without TEE were compared. We analyzed the binary clinical outcomes using the Fisher exact test. We next conducted an analysis of outcomes among the matched cohorts. The binary clinical outcomes were analyzed using the McNemar test. 26 , 27

Statistical sensitivity analyses were conducted to assess the robustness of our findings to unmeasured confounding using Rosenbaum bounds and amplification techniques. 27 , 28 The sensitivity analysis excluding those missing the TEE exposure was conducted using the same statistical tests as previously described. The negative control outcome analysis of elevation in postoperative creatinine was analyzed using a t test (unadjusted) and a difference-in-means estimator for the matched pair study design. 29 All hypothesis testing was 2-sided and significance was set at P  < .05. Data management, including data cleaning, data categorizing, and merging across ACSD versions was done using Stata version 15.0 (StataCorp). Additional data management required for matching and statistical analyses were conducted by R version 4.0.3 (R Project for Statistical Computing) using the R package dplyr. 30 , 31 Statistical analysis was performed from October 2020 to April 2021. A link to the GitHub code repository is provided in eAppendix 11 in the Supplement .

Following exclusions ( Figure 2 ), our study cohort included 872 936 patients undergoing valve or aortic surgery. Of the 872 936 patients, 540 229 (61.89%) were male, 63 565 (7.28%) were Black, 742 384 (85.04%) were White, 711 326 (81.5%) received TEE, and 161 610 (18.5%) did not receive TEE; the mean (SD) age was 65.61 years (13.17) years. Compared with patients who did not receive TEE, those who did receive TEE were similar demographically and hemodynamically, but had higher rates of preexisting comorbidities, ( Table 1 ) and varied by surgery type. The complete baseline characteristics between the TEE vs no TEE groups and the TEE distribution by surgery type are presented in eAppendix 1 in the Supplement .

Overall, 39 078 patients (4.32%) died within 30 days. Patients who received an intraoperative TEE had a lower 30-day mortality: 3.92% vs 5.27% (odds ratio [OR], 0.73 [95% CI, 0.72-0.75]; P  < .001), a lower incidence of stroke or 30-day mortality: 5.63% vs 7.01% (OR, 0.79 [95% CI, 0.77-0.81]; P  < .001), and a lower incidence of reoperation or 30-day mortality: 7.31% vs 8.87% (OR, 0.81 [95% CI, 0.79-0.83]; P  < .001). Unadjusted outcomes reported in McNemar format may be found in eAppendix 6 in the Supplement .

Our first, across-hospital, across-surgeon match consisted of 161 610 matched pairs that were similar in observable covariates ( Table 2 ). After matching, standardized differences across all variables were less than 0.10. The full covariate balance after matching is presented in eAppendix 5 in the Supplement . Our second, within-hospital, within equivocal-TEE-surgeon match consisted of 22 739 matched pairs that were similar in observable covariates; all with standardized differences less than 0.10. The full covariate balance is presented in eAppendix 5 in the Supplement .

The all patient across-hospital, across-surgeon matched analysis found that that among 161 610 matched pairs, intraoperative TEE was significantly associated with a lower 30-day mortality rate: 3.81% vs 5.27% (OR, 0.69 [95% CI, 0.67-0.72]; P  < .001), a lower incidence of stroke or 30-day mortality: 5.56% vs 7.01% (OR, 0.77 [95% CI, 0.74-0.79]; P  < .001), and a lower incidence of reoperation or 30-day mortality: 7.18% vs 8.87% (OR, 0.78 [95% CI, 0.76-0.80]; P  < .001) ( Table 3 ). Outcomes reported in McNemar format may be found in eAppendix 6 in the Supplement .

The within-hospital, within-surgeon with equivocal TEE preference (TEE probability: 0.30-0.70), matched analysis found that among 22 739 matched pairs, intraoperative TEE was significantly associated with a lower 30-day mortality rate: 2.79% vs 3.22% (OR, 0.86 [95% CI, 0.77-0.96]; P  = .008) and a lower incidence of stroke or 30-day mortality: 4.38% vs 4.76% (OR, 0.91 [95% CI, 0.83-1.00]; P  = .048). Intraoperative TEE was not statistically significantly associated with a lower incidence of reoperation or 30-day mortality: 5.58% vs 5.77% (OR, 0.94 [95% CI, 0.85-1.04]; P  = .24) ( Table 3 ). The 30-day mortality on the within-hospital, within-surgeon matched comparison was approximately 1% to 2% lower than the all-patient, across-hospital, across-surgeon matched comparison (30-day mortality with TEE: 3.81% on all-patient, across-hospital, across-surgeon matched comparison vs 2.79% on the within-hospital, within-surgeon, matched comparison; 30-day mortality without TEE: 5.27% on all-patient, across-hospital, across-surgeon matched comparison vs 3.22% on the within-hospital, within-surgeon, matched comparison) ( Table 3 ). Outcomes reported in McNemar format may be found in eAppendix 6 in the Supplement . An additional supplementary within-hospital, within-surgeon matched comparison across all surgeons, regardless of the probability of intraoperative TEE use (65 340 matched pairs), found comparable results (eAppendix 6 in the Supplement ). Additional subgroup analyses investigating outcomes with TEE vs without among patients indicated that patients undergoing mitral valve replacement or proximal aortic surgical procedures seem to benefit more from TEE compared with the overall cohort. These results are presented in eAppendix 7 in the Supplement .

To test the robustness of our findings, we completed sensitivity analyses and a negative control outcome analysis. The first sensitivity analysis indicated that according to the Rosenbaum bounds and associated amplification analysis, 27 , 28 to nullify the primary outcome finding from the all-patient, across-hospital, across-surgeon matched comparison, it would take an unmeasured confounder that doubled the odds of 30-day mortality and tripled the odds of TEE use (eAppendix 8 in the Supplement ). To nullify the primary outcome finding from the within-hospital, within-surgeon matched comparison, it would take an unmeasured confounder that increased the odds of 30-day mortality by 40% and the odds of TEE use by more than 40% (eAppendix 8 in the Supplement ). The second sensitivity analysis tested the robustness of our results by excluding the 2% of the cohort missing the TEE exposure and revealed findings that agreed with our presented results (eAppendix 9 in the Supplement ). Finally, our negative control outcome analysis compared elevation in postoperative creatinine between the TEE and no-TEE groups. Three of the 4 negative control outcome analyses were either statistically insignificant or incongruent with the primary results—an additional indication that residual confounding was controlled (eAppendix 6 in the Supplement ). A detailed explanation of the negative control outcome including rationale for selection and interpretation of the results is presented in eAppendix in the Supplement .

Among 872 936 patients undergoing valve or aortic surgery, across all analyses, intraoperative TEE was statistically significantly associated with a lower 30-day mortality, a lower incidence of stroke or 30-day mortality, and in the all-patient match, TEE was statistically significantly associated with a lower incidence of reoperation or 30-day mortality. These results were supported by multiple sensitivity analyses 27 , 28 that established the presented results would remain statistically significant at a .05 level in the presence of an unmeasured confounder that doubled the odds of 30-day mortality and tripled the odds of intraoperative TEE use, suggesting the presented findings would be robust to residual, unmeasured confounding. 22 , 28

Current AHA/ACC guidelines 10 , 11 do not specifically recommend for or against the use of intraoperative TEE for all cardiac valve replacement surgical procedures, 10 most cardiac valve repair surgical procedures, 10 and all proximal aortic aneurysm surgical procedures. 11 Presumably, this equivocal, class IIa, AHA/ACC stance on intraoperative TEE use is due to the absence of research comparing clinical outcomes among patients undergoing cardiac surgery with vs without TEE. 5 , 6 , 8 , 9 Only very recently has the impact of intraoperative TEE on clinical outcomes among patients undergoing cardiac surgery with vs without TEE been directly compared. 14 - 16

The current study’s finding that intraoperative TEE is associated with improved clinical outcomes is consistent with recent previous comparative effectiveness research by both our group 15 , 16 and others. 14 In 2020, we used propensity score matching to compare 219 238 Medicare beneficiaries undergoing cardiac valve surgery and found TEE was associated with a lower 30-day mortality. 15 In 2021, we used instrumental variable methods to compare 114 871 Medicare beneficiaries undergoing isolated CABG surgery and found that TEE was associated with lower in-hospital stroke and lower 30-day mortality. 13 Subsequently, an independent study by Metkus and colleagues used STS data and propensity score matching to compare 1.3 million patients undergoing isolated CABG surgery with vs without TEE and found a mortality benefit to the use of TEE. 14

The current study improves upon our previous work 15 , 16 in several noteworthy respects. First, the detailed, patient-level data found in the STS ADCS data registry allowed us to apply very strict matching criteria in order to minimize patient-level differences, controlling for far more observed patient-level covariate differences between those undergoing cardiac surgery with TEE vs without TEE. Second, the size of this STS cohort afforded us the opportunity to undertake within-hospital, within-surgeon matches. By creating matched pairs of 2 patients (1 with TEE vs 1 without TEE) admitted to the same hospital, and operated on by the same surgeon, we reduced hospital-level, and surgeon-level, unobserved confounding that could have biased our results. Third, in this study the exposure variable of TEE was found to be a true, intraoperative TEE; an improvement on our previous that could only identify a TEE within a hospitalization. 15 , 16 , 32 Fourth, by performing comprehensive sensitivity analyses, we were able to quantify how much residual, unobserved confounding would be required to alter the conclusions of our analyses. Across all analyses, our findings indicate an association between TEE and improved perioperative outcomes after open cardiac valve or proximal aortic surgery.

Although this matched retrospective observational study cannot elucidate the exact reasons for the clinical outcomes benefit observed with intraoperative TEE, it is likely that intraoperative TEE is conferring some degree of benefit because the association persisted on the strict, within-hospital, within-surgeon matched comparisons. Diagnostic information provided by TEE, interpreted by an experienced echocardiographer—cardiologist or anesthesiologist—could identify surgical complications 5 - 9 and improve outcomes by facilitating informed intraoperative decision making by the cardiac surgeon. 5 - 7 For instance, in valve surgery, paravalvular regurgitation identified by TEE after valve repair or replacement could prompt an immediate valve revision 5 , 6 and reduce the risk of reoperation (along with the complications associated with a second surgery). Additionally, TEE imaging can reduce the risk of stroke from air embolism by ensuring the dissipation intracardiac air prior to separation from cardiopulmonary bypass 33 or decrease the incidence of embolic stroke by ensuring an aortic cannulation or aortic cross clamp site does not embolize atheromatous plaque. 33 - 36 But equal to diagnostic information provided by the TEE imaging itself, it is possible that the association between intraoperative TEE and improved outcomes in this study could be related to the availability of an experienced cardiologist or anesthesiologist certified to perform and interpret a TEE in the operating room.

Our study must be interpreted with awareness of its limitations. First, the observational, nonrandomized design of this study cannot confirm a causal link between TEE and improved clinical outcomes because of the inability to completely eliminate residual confounding; particularly related to inherent differences among those who did not receive TEE. For instance, residual unobserved confounding could be introduced by anatomical considerations at the patient-level or differences in intraoperative management and TEE performance at the clinician-level that might indicate systematic differences among those who did not receive TEE compared with those who did receive TEE. An example of patient-level confounding could be introduced by our inability to exclude patients with anatomical contraindications to TEE such as esophageal (eg, esophagectomy, varices, or strictures) 37 or gastric (eg, previous gastric bypass surgery, gastric ulcer, or hiatal hernia) 37 diagnoses. But given the consistent results across all analyses, and the rare prevalence of these diagnoses (<0.5%), 37 we are reassured that residual patient-level confounding would not change the stated results. An example of clinician-level confounding could be related to the availability of a clinician with the specialization to perform an intraoperative TEE (eg, cardiologist or anesthesiologist). This example of clinician-level confounding could have persisted even after the within-hospital, within-surgeon match because STS data does not identify the clinician performing the intraoperative TEE. Second, while the within-hospital, within-surgeon matched completely controlled for TEE preference by surgical type because we exactly matched on all 9 surgical procedures, there is the possibility that we could not fully adjust for a surgeon who might have variability in TEE preference within the same surgery. For instance, a surgeon who would request TEE for a complex mitral repair, but not request TEE for a more straightforward mitral repair. Third, the 30-day mortality on the within-hospital, within-surgeon matched comparison was 1% to 2% lower than the all-patient, across-hospital, across-surgeon matched comparison. This difference in mortality could be an indication that comparing only surgeons with a probability for intraoperative TEE between 0.30 and 0.70 may represent a different, less sick, patient population compared with the all-patient, across-hospital, across-surgeon match. Fourth, fewer than 23% of patients receiving TEE are included in the matched analyses which potentially limits the generalizability of the stated results. Nevertheless, because results were similar across all analyses—including comprehensive sensitivity analyses—we are reassured of the robustness of the stated results indicating a clinical outcomes benefit to the use of TEE in cardiac valve or aortic surgery.

The current study, particularly in combination with recent observational research demonstrating a consistent outcomes benefit to the use of TEE during cardiac valve 15 and CABG surgery 14 , 16 may have important health policy implications. Because lack of equipoise, it is unlikely that a randomized controlled trial comparing TEE vs no TEE among cardiac surgical patients undergoing cardiac valve or aortic surgery would ever be conducted. Thus, rigorous observational studies such as the current work and previous work 15 , 16 are required to inform future AHA/ACC guideline recommendations for the routine use of TEE in cardiac surgery.

This cohort study found that the use of intraoperative TEE was associated with a lower 30-day mortality and a lower incidence of stroke or 30-day mortality among patients undergoing open cardiac valve or aortic surgery. These findings provide evidence to support the routine use of intraoperative TEE in all open cardiac valve and proximal aortic surgical procedures.

Accepted for Publication: December 16, 2021.

Published: February 9, 2022. doi:10.1001/jamanetworkopen.2021.47820

Open Access: This is an open access article distributed under the terms of the CC-BY License . © 2022 MacKay EJ et al. JAMA Network Open .

Corresponding Author: Emily J. MacKay, DO, MSHP, University of Pennsylvania, 423 Guardian Dr, 310 Blockley Hall, Philadelphia, PA 19104 ( [email protected] ; [email protected] ).

Author Contributions: Dr MacKay had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: MacKay, Zhang, Desai.

Drafting of the manuscript: MacKay, Augoustides, Desai.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: MacKay, Zhang, Groeneveld, Desai.

Obtained funding: MacKay.

Administrative, technical, or material support: Groeneveld.

Supervision: Augoustides, Desai.

Conflict of Interest Disclosures: None reported.

Funding/Support: This work was funded by (1) the Foundation for Anesthesia Education and Research (FAER) Mentored Research Training Grant (MRTG) (MRTG-08-15-2020; 581700) to Dr MacKay; (2) Department of Anesthesiology and Critical Care, University of Pennsylvania to Dr MacKay. The Department of Anesthesiology and Critical Care at the University of Pennsylvania funding (to Dr MacKay) provided the resources to purchase the Adult Cardiac Surgery Data (ACSD) from the Society of Thoracic Surgeons (STS) national registry.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, and approval of the manuscript; and decision to submit the manuscript for publication.

• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Published: 02 July 2013

Presentation, management, and outcomes of ischaemic heart disease in women

• Viola Vaccarino 1 ,
• Lina Badimon 2 ,
• Roberto Corti 3 ,
• Cor de Wit 4 ,
• Maria Dorobantu 5 ,
• Olivia Manfrini 8 ,
• Akos Koller 6 ,
• Axel Pries 7 ,
• Edina Cenko 8 &
• Raffaele Bugiardini 8  

Nature Reviews Cardiology volume  10 ,  pages 508–518 ( 2013 ) Cite this article

2293 Accesses

90 Citations

1 Altmetric

Metrics details

• Coronary artery disease and stable angina
• Therapeutics

Scientific interest in ischaemic heart disease (IHD) in women has grown considerably over the past 2 decades. A substantial amount of the literature on this subject is centred on sex differences in clinical aspects of IHD. Many reports have documented sex-related differences in presentation, risk profiles, and outcomes among patients with IHD, particularly acute myocardial infarction. Such differences have often been attributed to inequalities between men and women in the referral and treatment of IHD, but data are insufficient to support this assessment. The determinants of sex differences in presentation are unclear, and few clues are available as to why young, premenopausal women paradoxically have a greater incidence of adverse outcomes after acute myocardial infarction than men, despite having less-severe coronary artery disease. Although differential treatment on the basis of patient sex continues to be described, the extent to which such inequalities persist and whether they reflect true disparity is unclear. Additionally, much uncertainty surrounds possible sex-related differences in response to cardiovascular therapies, partly because of a persistent lack of female-specific data from cardiovascular clinical trials. In this Review, we assess the evidence for sex-related differences in the clinical presentation, treatment, and outcome of IHD, and identify gaps in the literature that need to be addressed in future research efforts.

Important differences exist between women and men in clinical presentation, recognition of symptoms by patients and physicians, outcome, and response to treatment for ischaemic heart disease (IHD)

Among patients with IHD, environmental or behavioural causes of sex-related differences in outcomes might be more important than biological factors

Onset of IHD in women, manifesting as an acute myocardial infarction before the age of 65 years, is associated with adverse outcomes compared with men of a similar age

A traditional diagnostic strategy, focusing on detection of severe coronary stenoses, is likely to be inadequate in women

Additional invasive testing aimed at determining endothelial coronary dysfunction might be useful to risk-stratify women with chest pain and minimal or no obstructive coronary artery disease

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 12 print issues and online access

195,33 € per year

only 16,28 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Sex-related impacts on clinical outcomes after percutaneous coronary intervention

Sex-related similarities and differences in responses to heart failure therapies

Sex-specific and ethnicity-specific differences in MINOCA

Go, A. S. et al . Heart disease and stroke statistics—2013 update: a report from the American Heart Association. Circulation 127 , e6–e245 (2013).

PubMed   Google Scholar  

Poon, S. et al . Bridging the gender gap: insights from a contemporary analysis of sex-related differences in the treatment and outcomes of patients with acute coronary syndromes. Am. Heart J. 163 , 66–73 (2012).

Article   PubMed   Google Scholar  

Rajadurai, J. et al . Women's cardiovascular health: perspectives from South-East Asia. Nat. Rev. Cardiol. 9 , 464–477 (2012).

Vaccarino, V. Ischemic heart disease in women: many questions, few facts. Circ. Cardiovasc. Qual. Outcomes 3 , 111–115 (2010).

Article   PubMed   PubMed Central   Google Scholar  

Murabito, J. M., Evans, J. C., Larson, M. G. & Levy, D. Prognosis after the onset of coronary heart disease. An investigation of differences in outcome between the sexes according to initial coronary disease presentation. Circulation 88 , 2548–2555 (1993).

Article   CAS   PubMed   Google Scholar  

Lerner, D. J. & Kannel, W. B. Patterns of coronary heart disease morbidity and mortality in the sexes: a 26-year follow-up of the Framingham population. Am. Heart J. 111 , 383–390 (1986).

Hemingway, H. et al . Prevalence of angina in women versus men: a systematic review and meta-analysis of international variations across 31 countries. Circulation 117 , 1526–1536 (2008).

Hochman, J. S. et al . Sex, clinical presentation, and outcome in patients with acute coronary syndromes. N. Engl. J. Med. 341 , 226–232 (1999).

Berger, J. S. et al . Sex differences in mortality following acute coronary syndromes. JAMA 302 , 874–882 (2009).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Genders, T. S. et al . A clinical prediction rule for the diagnosis of coronary artery disease: validation, updating, and extension. Eur. Heart J. 32 , 1316–1330 (2011).

Milner, K. A., Funk, M., Arnold, A. & Vaccarino, V. Typical symptoms are predictive of acute coronary syndromes in women. Am. Heart J. 143 , 283–288 (2002).

Canto, J. G. et al . Association of age and sex with myocardial infarction symptom presentation and in-hospital mortality. JAMA 307 , 813–822 (2012).

CAS   PubMed   PubMed Central   Google Scholar  

Milner, K. A. et al . Gender differences in symptom presentation associated with coronary heart disease. Am. J. Cardiol. 84 , 396–399 (1999).

Bugiardini, R. Women, 'nonspecific' chest pain, and normal or near-normal coronary angiograms are not synonymous with favourable outcome. Eur. Heart J. 27 , 1387–1389 (2006).

Kreatsoulas, C., Shannon, H. S., Giacomini, M., Velianou, J. L. & Anand, S. S. Reconstructing angina: cardiac symptoms are the same in women and men. JAMA Intern. Med. 173 , 829–833 (2013).

Mackay, M. H., Ratner, P. A., Johnson, J. L., Humphries, K. H. & Buller, C. E. Gender differences in symptoms of myocardial ischaemia. Eur. Heart J. 32 , 3107–3114 (2011).

Bugiardini, R. & Bairey Merz, C. N. Angina with “normal” coronary arteries: a changing philosophy. JAMA 293 , 477–484 (2005).

Bugiardini, R. et al . Angina, “normal” coronary angiography, and vascular dysfunction: risk assessment strategies. PLoS Med. 4 , e12 (2007).

Shaw, L. J., Bugiardini, R. & Merz, C. N. Women and ischemic heart disease: evolving knowledge. J. Am. Coll. Cardiol. 54 , 1561–1575 (2009).

Mosca, L., Mochari-Greenberger, H., Dolor, R. J., Newby, L. K. & Robb, K. J. Twelve-year follow-up of American women's awareness of cardiovascular disease risk and barriers to heart health. Circ. Cardiovasc. Qual. Outcomes 3 , 120–127 (2010).

Mosca, L. et al . Fifteen-year trends in awareness of heart disease in women: results of a 2012 American Heart Association national survey. Circulation 127 , 1254–1263 (2013).

Fukuoka, Y. et al . Is severity of chest pain a cue for women and men to recognize acute myocardial infarction symptoms as cardiac in origin? Prog. Cardiovasc. Nurs. 22 , 132–137 (2007).

Rosengren, A. et al . Sex, age, and clinical presentation of acute coronary syndromes. Eur. Heart J. 25 , 663–670 (2004).

Mulvagh, S. L. et al . Contrast echocardiography: current and future applications. J. Am. Soc. Echocardiogr. 13 , 331–342 (2000).

Nandalur, K. R., Dwamena, B. A., Choudhri, A. F., Nandalur, M. R. & Carlos, R. C. Diagnostic performance of stress cardiac magnetic resonance imaging in the detection of coronary artery disease: a meta-analysis. J. Am. Coll. Cardiol. 50 , 1343–1353 (2007).

Centers for Disease Control and Prevention (CDC). Prevalence of heart disease—United States, 2006–2010. MMWR Morb. Mortal. Wkly Rep. 60 , 1377–1381 (2011).

Gibbons, R. J. et al . ACC/AHA 2002 guideline update for exercise testing: summary article: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to Update the 1997 Exercise Testing Guidelines). Circulation 106 , 1883–1892 (2002).

Shaw, L. J. et al . The value of estimated functional capacity in estimating outcome: results from the NHBLI-Sponsored Women's Ischemia Syndrome Evaluation (WISE) Study. J. Am. Coll. Cardiol. 47 , S36–S43 (2006).

Mieres, J. H. et al . Role of noninvasive testing in the clinical evaluation of women with suspected coronary artery disease: consensus statement from the Cardiac Imaging Committee, Council on Clinical Cardiology, and the Cardiovascular Imaging and Intervention Committee, Council on Cardiovascular Radiology and Intervention, American Heart Association. Circulation 111 , 682–696 (2005).

Grzybowski, A. et al . How to improve noninvasive coronary artery disease diagnostics in premenopausal women? The influence of menstrual cycle on ST depression, left ventricle contractility, and chest pain observed during exercise echocardiography in women with angina and normal coronary angiogram. Am. Heart J. 156 , 964.e1–964.e5 (2008).

Article   Google Scholar  

Shaw, L. J. et al . Insights from the NHLBI-Sponsored Women's Ischemia Syndrome Evaluation (WISE) study: part I: gender differences in traditional and novel risk factors, symptom evaluation, and gender-optimized diagnostic strategies. J. Am. Coll. Cardiol. 47 (Suppl.), S4–S20 (2006).

Lanza, G. A. & Crea, F. Primary coronary microvascular dysfunction: clinical presentation, pathophysiology, and management. Circulation 121 , 2317–2325 (2010).

Maseri, A., Crea, F., Kaski, J. C. & Crake, T. Mechanisms of angina pectoris in syndrome X. J. Am. Coll. Cardiol. 17 , 499–506 (1991).

Bugiardini, R., Pozzati, A., Ottani, F., Morgagni, G. L. & Puddu, P. Vasotonic angina: a spectrum of ischemic syndromes involving functional abnormalities of the epicardial and microvascular coronary circulation. J. Am. Coll. Cardiol. 22 , 417–425 (1993).

Gan, S. C. et al . Treatment of acute myocardial infarction and 30-day mortality among women and men. N. Engl. J. Med. 343 , 8–15 (2000).

Bugiardini, R., Manfrini, O. & De Ferrari, G. M. Unanswered questions for management of acute coronary syndrome: risk stratification of patients with minimal disease or normal findings on coronary angiography. Arch. Intern. Med. 166 , 1391–1395 (2006).

Scanlon, P. J. et al . ACC/AHA guidelines for coronary angiography: executive summary and recommendations. A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee on Coronary Angiography) developed in collaboration with the Society for Cardiac Angiography and Interventions. Circulation 99 , 2345–2357 (1999).

Ong, P. et al . High prevalence of a pathological response to acetylcholine testing in patients with stable angina pectoris and unobstructed coronary arteries. The ACOVA Study (Abnormal COronary VAsomotion in patients with stable angina and unobstructed coronary arteries). J. Am. Coll. Cardiol. 59 , 655–662 (2012).

Budoff, M. J. et al . Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evaluation of coronary artery stenosis in individuals without known coronary artery disease: results from the prospective multicentre ACCURACY (Assessment by Coronary Computed Tomographic Angiography of Individuals Undergoing Invasive Coronary Angiography) trial. J. Am. Coll. Cardiol. 52 , 1724–1732 (2008).

Beigel, R. et al . Prognostic implications of nonobstructive coronary artery disease in patients undergoing coronary computed tomographic angiography for acute chest pain. Am. J. Cardiol. 111 , 941–945 (2013).

Lin, F. Y. et al . Mortality risk in symptomatic patients with nonobstructive coronary artery disease: a prospective 2-centre study of 2,583 patients undergoing 64-detector row coronary computed tomographic angiography. J. Am. Coll. Cardiol. 58 , 510–519 (2011).

Min, J. K. et al . Age- and sex-related differences in all-cause mortality risk based on coronary computed tomography angiography findings results from the International Multicentre CONFIRM (Coronary CT Angiography Evaluation for Clinical Outcomes: An International Multicentre Registry) of 23,854 patients without known coronary artery disease. J. Am. Coll. Cardiol. 58 , 849–860 (2011).

Martinez, M. W. et al . Myocardial infarction with normal coronary arteries: a role for MRI? Clin. Chem. 53 , 995–996 (2007).

Steg, P. G. et al . Impact of collateral flow to the occluded infarct-related artery on clinical outcomes in patients with recent myocardial infarction: a report from the randomized occluded artery trial. Circulation 121 , 2724–2730 (2010).

Wolff, S. D. et al . Myocardial first-pass perfusion magnetic resonance imaging: a multicentre dose-ranging study. Circulation 110 , 732–737 (2004).

Schwitter, J. et al . MR-IMPACT: comparison of perfusion-cardiac magnetic resonance with single-photon emission computed tomography for the detection of coronary artery disease in a multicentre, multivendor, randomized trial. Eur. Heart J. 29 , 480–489 (2008).

Panting, J. R. et al . Abnormal subendocardial perfusion in cardiac syndrome X detected by cardiovascular magnetic resonance imaging. N. Engl. J. Med. 346 , 1948–1953 (2002).

Schwitter, J. Extending the frontiers of cardiac magnetic resonance. Circulation 118 , 109–112 (2008).

Reynolds, H. R. et al . Mechanisms of myocardial infarction in women without angiographically obstructive coronary artery disease. Circulation 124 , 1414–1425 (2011).

Mieres, J. H. et al . American Society of Nuclear Cardiology consensus statement: Task Force on Women and Coronary Artery Disease—the role of myocardial perfusion imaging in the clinical evaluation of coronary artery disease in women [correction]. J. Nucl. Cardiol. 10 , 95–101 (2003).

Klocke, F. J. et al . ACC/AHA/ASNC guidelines for the clinical use of cardiac radionuclide imaging--executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (ACC/AHA/ASNC Committee to revise the 1995 guidelines for the clinical use of cardiac radionuclide imaging). Circulation 108 , 1404–1418 (2003).

Matsunari, I. et al . Attenuation-corrected rest thallium-201/stress technetium 99m sestamibi myocardial SPECT in normals. J. Nucl. Cardiol. 5 , 48–55 (1998).

Hemingway, H. et al . Incidence and prognostic implications of stable angina pectoris among women and men. JAMA 295 , 1404–1411 (2006).

Daly, C. et al . Gender differences in the management and clinical outcome of stable angina. Circulation 113 , 490–498 (2006).

Fox, K. et al . Guidelines on the management of stable angina pectoris: executive summary: the Task Force on the Management of Stable Angina Pectoris of the European Society of Cardiology. Eur. Heart J. 27 , 1341–1381 (2006).

Crilly, M., Bundred, P., Hu, X., Leckey, L. & Johnstone, F. Gender differences in the clinical management of patients with angina pectoris: a cross-sectional survey in primary care. BMC Health Serv. Res. 7 , 142 (2007).

Johnston, N., Schenck-Gustafsson, K. & Lagerqvist, B. Are we using cardiovascular medications and coronary angiography appropriately in men and women with chest pain? Eur. Heart J. 32 , 1331–1336 (2011).

Vaccarino, V., Krumholz, H. M., Berkman, L. F. & Horwitz, R. I. Sex differences in mortality after myocardial infarction. Is there evidence for an increased risk for women? Circulation 91 , 1861–1871 (1995).

Wenger, N. K., Shaw, L. J. & Vaccarino, V. Coronary heart disease in women: update 2008. Clin. Pharmacol. Ther. 83 , 37–51 (2008).

Champney, K. P. et al . The joint contribution of sex, age and type of myocardial infarction on hospital mortality following acute myocardial infarction. Heart 95 , 895–899 (2009).

Jneid, H. et al . Sex differences in medical care and early death after acute myocardial infarction. Circulation 118 , 2803–2810 (2008).

Capewell, S. et al . Short-term and long-term outcomes in 133,429 emergency patients admitted with angina or myocardial infarction in Scotland, 1990–2000: population-based cohort study. Heart 92 , 1563–1570 (2006).

Fuster, V. Elucidation of the role of plaque instability and rupture in acute coronary events. Am. J. Cardiol. 76 , 24C–33C (1995).

Fibrinolytic Therapy Trialists' (FTT) Collaborative Group. Indications for fibrinolytic therapy in suspected acute myocardial infarction: collaborative overview of early mortality and major morbidity results from all randomised trials of more than 1,000 patients. Lancet 343 , 311–322 (1994).

Tamis-Holland, J. E. et al . Benefits of direct angioplasty for women and men with acute myocardial infarction: results of the Global Use of Strategies to Open Occluded Arteries in Acute Coronary Syndromes Angioplasty (GUSTO II-B) angioplasty substudy. Am. Heart J. 147 , 133–139 (2004).

Kim, E. S. & Menon, V. Status of women in cardiovascular clinical trials. Arterioscler. Thromb. Vac. Biol. 29 , 279–283 (2009).

Article   CAS   Google Scholar  

Bavry, A. A. et al . Invasive therapy along with glycoprotein IIb/IIIa inhibitors and intracoronary stents improves survival in non-ST-segment elevation acute coronary syndromes: a meta-analysis and review of the literature. Am. J. Cardiol. 93 , 830–835 (2004).

Glaser, R. et al . Benefit of an early invasive management strategy in women with acute coronary syndromes. JAMA 288 , 3124–3129 (2002).

Lansky, A. J. et al . Percutaneous coronary intervention and adjunctive pharmacotherapy in women: a statement for healthcare professionals from the American Heart Association. Circulation 111 , 940–953 (2005).

Cho, L. et al . Clinical benefit of glycoprotein IIb/IIIa blockade with abciximab is independent of gender: pooled analysis from EPIC, EPILOG and EPISTENT trials. Evaluation of 7E3 for the prevention of ischemic complications. Evaluation in percutaneous transluminal coronary angioplasty to improve long-term outcome with abciximab GP IIb/IIIa blockade. Evaluation of platelet IIb/IIIa inhibitor for stent. J. Am. Coll. Cardiol. 36 , 381–386 (2000).

Boersma, E. et al . Platelet glycoprotein IIb/IIIa inhibitors in acute coronary syndromes: a meta-analysis of all major randomised clinical trials. Lancet 359 , 189–198 (2002).

Healy, B. The Yentl syndrome. N. Engl. J. Med. 325 , 274–276 (1991).

Bugiardini, R., Oestrada, J. L., Nikus, K., Hall, A. S. & Manfrini, O. Gender bias in acute coronary syndromes. Curr. Vasc. Pharmacol. 8 , 276–284 (2010).

Blomkalns, A. L. et al . Gender disparities in the diagnosis and treatment of non-ST-segment elevation acute coronary syndromes: large-scale observations from the CRUSADE (Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes With Early Implementation of the American College of Cardiology/American Heart Association Guidelines) national quality improvement initiative. J. Am. Coll. Cardiol. 45 , 832–837 (2005).

Anand, S. S. et al . Differences in the management and prognosis of women and men who suffer from acute coronary syndromes. J. Am. Coll. Cardiol. 46 , 1845–1851 (2005).

Dey, S. et al . Sex-related differences in the presentation, treatment and outcomes among patients with acute coronary syndromes: the Global Registry of Acute Coronary Events. Heart 95 , 20–26 (2009).

Vaccarino, V. et al . Sex and racial differences in the management of acute myocardial infarction, 1994 through 2002. N. Engl. J. Med. 353 , 671–682 (2005).

Karp, I., Chen, S. F. & Pilote, L. Sex differences in the effectiveness of statins after myocardial infarction. CMAJ 176 , 333–338 (2007).

Walsh, J. M. & Pignone, M. Drug treatment of hyperlipidemia in women. JAMA 291 , 2243–2252 (2004).

Shekelle, P. G. et al . Efficacy of angiotensin-converting enzyme inhibitors and beta-blockers in the management of left ventricular systolic dysfunction according to race, gender, and diabetic status: a meta-analysis of major clinical trials. J. Am. Coll. Cardiol. 41 , 1529–1538 (2003).

First International Study of Infarct Survival Collaborative Group. Randomised trial of intravenous atenolol among 16,027 cases of suspected acute myocardial infarction: ISIS-1. Lancet 2 , 57–66 (1986).

Institute of Medicine (eds Wizemann, T. & Pardue, M.) Exploring the Biological Contributions to Human Health: Does Sex Matter? (The National Academy Press, 2001).

Aldea, G. S. et al . Effect of gender on postoperative outcomes and hospital stays after coronary artery bypass grafting. Ann. Thorac. Surg. 67 , 1097–1103 (1999).

Stern, S. & Bayes de Luna, A. Coronary artery spasm: a 2009 update. Circulation 119 , 2531–2534 (2009).

Selzer, A., Langston, M., Ruggeroli, C. & Cohn, K. Clinical syndrome of variant angina with normal coronary arteriogram. N. Engl. J. Med. 295 , 1343–1347 (1976).

Seung-Woon, R. et al . The impact of gender difference on angiographic characteristics during intracoronary acetylcholine provocation test in Korean patients [abstract TCT-437]. J. Am. Coll. Cardiol. 60 (Suppl. B), B124 (2012).

Google Scholar  

Bory, M. et al . Coronary artery spasm in patients with normal or near normal coronary arteries. Long-term follow-up of 277 patients. Eur. Heart J. 17 , 1015–1021 (1996).

Pozzati, A., Pancaldi, L. G., Di Pasquale, G., Pinelli, G. & Bugiardini, R. Transient sympathovagal imbalance triggers “ischemic” sudden death in patients undergoing electrocardiographic Holter monitoring. J. Am. Coll. Cardiol. 27 , 847–852 (1996).

Yoo, S. Y. & Kim, J. Y. Recent insights into the mechanisms of vasospastic angina. Korean Circ. J. 39 , 505–511 (2009).

Egashira, K. et al . Basal release of endothelium-derived nitric oxide at site of spasm in patients with variant angina. J. Am. Coll. Cardiol. 27 , 1444–1449 (1996).

Walling, A. et al . Long-term prognosis of patients with variant angina. Circulation 76 , 990–997 (1987).

Mishra, P. K. Variations in presentation and various options in management of variant angina. Eur. J. Cardiothorac. Surg. 29 , 748–759 (2006).

Waters, D. D. et al . Factors influencing the long-term prognosis of treated patients with variant angina. Circulation 68 , 258–265 (1983).

Ong, P., Athanasiadis, A., Borgulya, G., Voehringer, M. & Sechtem, U. 3-year follow-up of patients with coronary artery spasm as cause of acute coronary syndrome: the CASPAR (coronary artery spasm in patients with acute coronary syndrome) study follow-up. J. Am. Coll. Cardiol. 57 , 147–152 (2011).

Asbury, E. A., Creed, F. & Collins, P. Distinct psychosocial differences between women with coronary heart disease and cardiac syndrome X. Eur. Heart J. 25 , 1695–1701 (2004).

Camici, P. G. & Crea, F. Coronary microvascular dysfunction. N. Engl. J. Med. 356 , 830–840 (2007).

Khuddus, M. A. et al . An intravascular ultrasound analysis in women experiencing chest pain in the absence of obstructive coronary artery disease: a substudy from the National Heart, Lung and Blood Institute-sponsored Women's Ischemia Syndrome Evaluation (WISE). J. Interv. Cardiol. 23 , 511–519 (2010).

Wenger, N. K. Women and coronary heart disease: a century after Herrick: understudied, underdiagnosed, and undertreated. Circulation 126 , 604–611 (2012).

Bairey Merz, C. N. et al . Insights from the NHLBI-Sponsored Women's Ischemia Syndrome Evaluation (WISE) study: part II: gender differences in presentation, diagnosis, and outcome with regard to gender-based pathophysiology of atherosclerosis and macrovascular and microvascular coronary disease. J. Am. Coll. Cardiol. 47 (3 Suppl.), S21–S29 (2006).

Jespersen, L. et al . Stable angina pectoris with no obstructive coronary artery disease is associated with increased risks of major adverse cardiovascular events. Eur. Heart J. 33 , 734–744 (2012).

Kaski, J. C. et al . Cardiac syndrome X: clinical characteristics and left ventricular function. Long-term follow-up study. J. Am. Coll. Cardiol. 25 , 807–814 (1995).

Gulati, M. et al . Adverse cardiovascular outcomes in women with nonobstructive coronary artery disease: a report from the Women's Ischemia Syndrome Evaluation Study and the St James Women Take Heart Project. Arch. Intern. Med. 169 , 843–850 (2009).

Johnson, B. D. et al . Prognosis in women with myocardial ischemia in the absence of obstructive coronary disease: results from the National Institutes of Health-National Heart, Lung, and Blood Institute-Sponsored Women's Ischemia Syndrome Evaluation (WISE). Circulation 109 , 2993–2999 (2004).

Oerlemans, J. G., Lagro-Janssen, A. L. & Bakx, C. Angina pectoris and normal coronary arteries: prevalence and prognosis in men and women [Dutch]. Ned. Tijdschr. Geneeskd. 144 , 522–527 (2000).

CAS   PubMed   Google Scholar  

von Mering, G. O. et al . Abnormal coronary vasomotion as a prognostic indicator of cardiovascular events in women: results from the National Heart, Lung, and Blood Institute-Sponsored Women's Ischemia Syndrome Evaluation (WISE). Circulation 109 , 722–725 (2004).

Bugiardini, R., Manfrini, O., Pizzi, C., Fontana, F. & Morgagni, G. Endothelial function predicts future development of coronary artery disease: a study of women with chest pain and normal coronary angiograms. Circulation 109 , 2518–2523 (2004).

Pepine, C. J. et al . Coronary microvascular reactivity to adenosine predicts adverse outcome in women evaluated for suspected ischemia: results from the National Heart, Lung and Blood Institute WISE (Women's Ischemia Syndrome Evaluation) Study. J. Am. Coll. Cardiol. 55 , 2825–2832 (2010).

Britten, M. B., Zeiher, A. M. & Schächinger, V. Microvascular dysfunction in angiographically normal or mildly diseased coronary arteries predicts adverse cardiovascular long-term outcome. Coron. Artery Dis. 15 , 259–264 (2004).

Fragasso, G. et al . Coronary slow-flow causing transient myocardial hypoperfusion in patients with cardiac syndrome X: long-term clinical and functional prognosis. Int. J. Cardiol. 137 , 137–144 (2009).

Bugiardini, R., Borghi, A., Biagetti, L. & Puddu, P. Comparison of verapamil versus propranolol therapy in syndrome X. Am. J. Cardiol. 63 , 286–290 (1989).

Xhyheri, B. & Bugiardini, R. Diagnosis and treatment of heart disease: are women different from men? Prog. Cardiovasc. Dis. 53 , 227–236 (2010).

Kaski, J. C., Rosano, G., Gavrielides, S. & Chen, L. Effects of angiotensin-converting enzyme inhibition on exercise-induced angina and ST segment depression in patients with microvascular angina. J. Am. Coll. Cardiol. 23 , 652–657 (1994).

Nalbantgil, I. et al . Therapeutic benefits of cilazapril in patients with syndrome X. Cardiology 89 , 130–133 (1998).

Pauly, D. F. et al . In women with symptoms of cardiac ischemia, nonobstructive coronary arteries, and microvascular dysfunction, angiotensin-converting enzyme inhibition is associated with improved microvascular function: A double-blind randomized study from the National Heart, Lung and Blood Institute Women's Ischemia Syndrome Evaluation (WISE). Am. Heart J. 162 , 678–684 (2011).

Akashi, Y. J., Nef, H. M., Mollmann, H. & Ueyama, T. Stress cardiomyopathy. Annu. Rev. Med. 61 , 271–286 (2010).

Yoshioka, T. et al . Clinical implications of midventricular obstruction and intravenous propranolol use in transient left ventricular apical ballooning (Tako-tsubo cardiomyopathy). Am. Heart J. 155 , 526.e1–526.e7 (2008).

Wittstein, I. S. et al . Neurohumoral features of myocardial stunning due to sudden emotional stress. N. Engl. J. Med. 352 , 539–548 (2005).

Bielecka-Dabrowa, A. et al . Takotsubo cardiomyopathy—the current state of knowledge. Int. J. Cardiol. 142 , 120–125 (2010).

Akashi, Y. J., Goldstein, D. S., Barbaro, G. & Ueyama, T. Takotsubo cardiomyopathy: a new form of acute, reversible heart failure. Circulation 118 , 2754–2762 (2008).

Edwards, F. H., Carey, J. S., Grover, F. L., Bero, J. W. & Hartz, R. S. Impact of gender on coronary bypass operative mortality. Ann. Thorac. Surg. 66 , 125–131 (1998).

Weintraub, W. S., Wenger, N. K., Jones, E. L., Craver, J. M. & Guyton, R. A. Changing clinical characteristics of coronary surgery patients. Differences between men and women. Circulation 88 , II79–II86 (1993).

Woods, S. E., Noble, G., Smith, J. M. & Hasselfeld, K. The influence of gender in patients undergoing coronary artery bypass graft surgery: an eight-year prospective hospitalized cohort study. J. Am. Coll. Surg. 196 , 428–434 (2003).

Vaccarino, V., Abramson, J. L., Veledar, E. & Weintraub, W. S. Sex differences in hospital mortality after coronary artery bypass surgery: evidence for a higher mortality in younger women. Circulation 105 , 1176–1181 (2002).

Kelsey, S. F. et al . Results of percutaneous transluminal coronary angioplasty in women. 1985–1986 National Heart, Lung, and Blood Institute's Coronary Angioplasty Registry. Circulation 87 , 720–727 (1993).

Holubkov, R. et al . Angina 1 year after percutaneous coronary intervention: a report from the NHLBI Dynamic Registry. Am. Heart J. 144 , 826–833 (2002).

Jacobs, A. K. et al . Better outcome for women compared with men undergoing coronary revascularization: a report from the Bypass Angioplasty Revascularization Investigation (BARI). Circulation 98 , 1279–1285 (1998).

Thompson, C. A. et al . Gender-based differences of percutaneous coronary intervention in the drug-eluting stent era. Catheter. Cardiovasc. Interv. 67 , 25–31 (2006).

Abbott, J. D. et al . Gender-based outcomes in percutaneous coronary intervention with drug-eluting stents (from the National Heart, Lung, and Blood Institute Dynamic Registry). Am. J. Cardiol. 99 , 626–631 (2007).

Argulian, E. et al . Gender differences in short-term cardiovascular outcomes after percutaneous coronary interventions. Am. J. Cardiol. 98 , 48–53 (2006).

Yang, F., Minutello, R. M., Bhagan, S., Sharma, A. & Wong, S. C. The impact of gender on vessel size in patients with angiographically normal coronary arteries. J. Interv. Cardiol. 19 , 340–344 (2006).

Mehilli, J. et al . Gender and restenosis after coronary artery stenting. Eur. Heart J. 24 , 1523–1530 (2003).

Dickerson, J. A., Nagaraja, H. N. & Raman, S. V. Gender-related differences in coronary artery dimensions: a volumetric analysis. Clin. Cardiol. 33 , E44–E49 (2010).

Vaccarino, V. et al . Sex differences in mortality after acute myocardial infarction: changes from 1994 to 2006. Arch. Intern. Med. 169 , 1767–1774 (2009).

PubMed   PubMed Central   Google Scholar  

Vaccarino, V., Parsons, L., Every, N. R., Barron, H. V. & Krumholz, H. M. Sex-based differences in early mortality after myocardial infarction. National Registry of Myocardial Infarction 2 Participants. N. Engl. J. Med. 341 , 217–225 (1999).

Andrikopoulos, G. K. et al . Younger age potentiates post myocardial infarction survival disadvantage of women. Int. J. Cardiol. 108 , 320–325 (2006).

Koek, H. L. et al . Short- and long-term prognosis after acute myocardial infarction in men versus women. Am. J. Cardiol. 98 , 993–999 (2006).

Radovanovic, D. et al . Gender differences in management and outcomes in patients with Acute Coronary Syndromes: results on 20,290 patients from the AMIS Plus Registry. Heart 93 , 1369–1375 (2007).

Vaccarino, V. et al . Sex differences in health status after coronary artery bypass surgery. Circulation 108 , 2642–2647 (2003).

Vaccarino, V. et al . Gender differences in recovery after coronary artery bypass surgery. J. Am. Coll. Cardiol. 41 , 307–314 (2003).

Abramson, J. L., Veledar, E., Weintraub, W. S. & Vaccarino, V. Association between gender and in-hospital mortality after percutaneous coronary intervention according to age. Am. J. Cardiol. 91 , 968–971 (2003).

Rosengren, A. et al . Sex differences in survival after myocardial infarction in Sweden; data from the Swedish National Acute Myocardial Infarction Register. Eur. Heart J. 22 , 314–322 (2001).

Russo, A. M. et al . Influence of gender on arrhythmia characteristics and outcome in the Multicentre UnSustained Tachycardia Trial. J. Cardiovasc. Electrophysiol. 15 , 993–998 (2004).

Salomaa, V. et al . Decline in out-of-hospital coronary heart disease deaths has contributed the main part to the overall decline in coronary heart disease mortality rates among persons 35 to 64 years of age in Finland: the FINAMI study. Circulation 108 , 691–696 (2003).

MacIntyre, K. et al . Gender and survival: a population-based study of 201,114 men and women following a first acute myocardial infarction. J. Am. Coll. Cardiol. 38 , 729–735 (2001).

Centers for Disease Control and Prevention (CDC). State-specific mortality from sudden cardiac death—United States, 1999. MMWR Morb. Mortal. Wkly Rep. 51 , 123–126 (2002).

Njølstad, I., Arnesen, E. & Lund-Larsen, P. G. Smoking, serum lipids, blood pressure, and sex differences in myocardial infarction. A 12-year follow-up of the Finnmark Study. Circulation 93 , 450–456 (1996).

Kemp, H. G. Jr, Vokonas, P. S., Cohn, P. F. & Gorlin, R. The anginal syndrome associated with normal coronary arteriograms. Report of a six year experience. Am. J. Med. 54 , 735–742 (1973).

Cannon, R. O. 3rd et al . Abnormal cardiac sensitivity in patients with chest pain and normal coronary arteries. J. Am. Coll. Cardiol. 16 , 1359–1366 (1990).

Cannon R. O. 3rd & Epstein, S. E. “Microvascular angina” as a cause of chest pain with angiographically normal coronary arteries. Am. J. Cardiol. 61 , 1338–1343 (1988).

Reis, S. E. et al . Coronary flow velocity response to adenosine characterizes coronary microvascular function in women with chest pain and no obstructive coronary disease. Results from the pilot phase of the Women's Ischemia Syndrome Evaluation (WISE) study. J. Am. Coll. Cardiol. 33 , 1469–1475 (1999).

Download references

Acknowledgements

The authors of this Review are members of the European Society of Cardiology Working Group on Coronary Pathophysiology and Microcirculation, and acknowledge the European Society of Cardiology for financial support. Dr Vaccarino is supported by the National Institutes of Health, grant K24HL077506.

Author information

Authors and affiliations.

Emory University Rollins School of Public Health and School of Medicine, USA

Viola Vaccarino

Cardiovascular Research Centre, CSIC-ICCC, Spain

Lina Badimon

University Hospital, Zurich, Switzerland

Roberto Corti

Institut für Physiologie, Universität zu Lübeck, Germany

Floreasca Emergency Hospital, Romania

Maria Dorobantu

University of Pecs, Pécs, Hungary

Akos Koller

Charité-Universitätsmedizin Berlin, Germany

Department of Experimental, Diagnostics and Specialized Medicine, Section of Cardiology, Policlinico Sant'Orsola-Malpighi, Bologna, 40138, Italy

Olivia Manfrini, Edina Cenko & Raffaele Bugiardini

You can also search for this author in PubMed   Google Scholar

Contributions

V. Vaccarino, R. Corti, O. Manfrini, E. Cenko, and R. Bugiardini researched data for the article. V. Vaccarino, L. Badimon, M. Dorobantu, O. Manfrini, A. Pries, E. Cenko, and R. Bugiardini contributed substantially to the discussion of content. The article was written by V. Vaccarino, R. Corti, and R. Bugiardini. V. Vaccarino, L. Badimon, R. Corti, C. de Wit, M. Dorobantu, O. Manfrini, A. Koller, E. Cenko, and R. Bugiardini reviewed/edited the manuscript before submission.

Corresponding author

Correspondence to Raffaele Bugiardini .

Ethics declarations

Competing interests.

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Vaccarino, V., Badimon, L., Corti, R. et al. Presentation, management, and outcomes of ischaemic heart disease in women. Nat Rev Cardiol 10 , 508–518 (2013). https://doi.org/10.1038/nrcardio.2013.93

Download citation

Published : 02 July 2013

Issue Date : September 2013

DOI : https://doi.org/10.1038/nrcardio.2013.93

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Cardiac medication use in patients with suspected ischaemia without obstructive coronary arteries: sex differences and psychological distress.

• P. M. C. Mommersteeg
• J. Roeters van Lennep
• J. Widdershoven

Netherlands Heart Journal (2021)

Facial Expressions of Emotions During Pharmacological and Exercise Stress Testing: the Role of Myocardial Ischemia and Cardiac Symptoms

• Maria T. Bekendam
• Willem J. Kop
• Paula M. C. Mommersteeg

International Journal of Behavioral Medicine (2021)

Sex and age differences in the association of heart rate responses to adenosine and myocardial ischemia in patients undergoing myocardial perfusion imaging

• Catherine Gebhard
• Michael Messerli
• Philipp A. Kaufmann

Journal of Nuclear Cardiology (2020)

Sex-Related Differences in Pharmacokinetics and Pharmacodynamics of Frequently Prescribed Drugs: A Review of the Literature

• André Farkouh
• Thomas Riedl
• Alexandra Kautzky-Willer

Advances in Therapy (2020)

Gender Differences in Residual Risk Factors for Major Adverse Cardiovascular Events Following ACS and How to Bridge the Gap

• Olivia Manfrini
• Edina Cenko
• Raffaele Bugiardini

Current Atherosclerosis Reports (2020)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Study protocol
• Open access
• Published: 05 August 2024

A pragmatic, stepped-wedge, hybrid type II trial of interoperable clinical decision support to improve venous thromboembolism prophylaxis for patients with traumatic brain injury

• Christopher J. Tignanelli   ORCID: orcid.org/0000-0002-8079-5565 1 , 2 , 3 , 4 ,
• Surbhi Shah 5 ,
• David Vock 6 ,
• Lianne Siegel 6 ,
• Carlos Serrano 6 ,
• Elliott Haut 7 ,
• Sean Switzer 8 ,
• Christie L. Martin 9 ,
• Rubina Rizvi 2 , 3 ,
• Vincent Peta 1 ,
• Peter C. Jenkins 10 ,
• Nicholas Lemke 1 ,
• Thankam Thyvalikakath 11 , 12 ,
• Jerome A. Osheroff 13 ,
• Denise Torres 14 ,
• David Vawdrey 15 ,
• Rachael A. Callcut 16 ,
• Mary Butler 3 , 17 &
• Genevieve B. Melton 1 , 2 , 3  

Implementation Science volume  19 , Article number:  57 ( 2024 ) Cite this article

274 Accesses

2 Altmetric

Metrics details

Venous thromboembolism (VTE) is a preventable medical condition which has substantial impact on patient morbidity, mortality, and disability. Unfortunately, adherence to the published best practices for VTE prevention, based on patient centered outcomes research (PCOR), is highly variable across U.S. hospitals, which represents a gap between current evidence and clinical practice leading to adverse patient outcomes.

This gap is especially large in the case of traumatic brain injury (TBI), where reluctance to initiate VTE prevention due to concerns for potentially increasing the rates of intracranial bleeding drives poor rates of VTE prophylaxis. This is despite research which has shown early initiation of VTE prophylaxis to be safe in TBI without increased risk of delayed neurosurgical intervention or death. Clinical decision support (CDS) is an indispensable solution to close this practice gap; however, design and implementation barriers hinder CDS adoption and successful scaling across health systems. Clinical practice guidelines (CPGs) informed by PCOR evidence can be deployed using CDS systems to improve the evidence to practice gap. In the Scaling AcceptabLE cDs (SCALED) study, we will implement a VTE prevention CPG within an interoperable CDS system and evaluate both CPG effectiveness (improved clinical outcomes) and CDS implementation.

The SCALED trial is a hybrid type 2 randomized stepped wedge effectiveness-implementation trial to scale the CDS across 4 heterogeneous healthcare systems. Trial outcomes will be assessed using the RE 2 -AIM planning and evaluation framework. Efforts will be made to ensure implementation consistency. Nonetheless, it is expected that CDS adoption will vary across each site. To assess these differences, we will evaluate implementation processes across trial sites using the Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework (a determinant framework) using mixed-methods. Finally, it is critical that PCOR CPGs are maintained as evidence evolves. To date, an accepted process for evidence maintenance does not exist. We will pilot a “Living Guideline” process model for the VTE prevention CDS system.

The stepped wedge hybrid type 2 trial will provide evidence regarding the effectiveness of CDS based on the Berne-Norwood criteria for VTE prevention in patients with TBI. Additionally, it will provide evidence regarding a successful strategy to scale interoperable CDS systems across U.S. healthcare systems, advancing both the fields of implementation science and health informatics.

Trial registration

Clinicaltrials.gov – NCT05628207. Prospectively registered 11/28/2022, https://classic.clinicaltrials.gov/ct2/show/NCT05628207 .

Contributions to the Literature

This paper provides a study protocol for a new and novel stepped wedge study variation which includes external control sites to take into account external influences on the uptake of traumatic brain injury guidelines nationally

This paper provides a study design for one of the largest trauma pragmatic trials in the U.S. of 9 heterogenous hospitals

This study is also unique and first-in-kind feature as the guideline may change over time during the study due to the “living” nature of the guideline being implemented.

Introduction

Venous thromboembolism (VTE) is a preventable complication of traumatic brain injury (TBI), which has a substantial impact on patient morbidity, mortality, disability. It is also associated with significant economic burden > $1.5 billion per year [ 1 , 2 ]. VTE is considered a preventable medical condition in the majority of cases [ 2 , 3 ]. Unfortunately, adherence with patient centered outcomes research (PCOR)-informed VTE prevention best practices is highly variable and often poor across U.S. hospitals. Compliance with best practice is especially relevant in the case of TBI as 54% of TBI patients will develop a VTE if they do not receive appropriate anticoagulation [ 4 ]. The delivery of appropriate VTE prophylaxis to TBI patients is such an important quality measure that adherence is tracked nationally and benchmarked by the American College of Surgeons Trauma Quality Improvement Program (ACS-TQIP) [ 5 ]. We have previously shown that instituting a hospital-wide VTE prevention initiative modeled after the Berne-Norwood criteria for VTE prophylaxis in TBI was associated with significantly increased compliance with VTE-related process and improved outcome metrics [ 6 ]. Specifically, we observed improved adherence with the Berne-Norwood criteria [ 7 , 8 ], reduced time to initiation of VTE prophylaxis, and reduced VTE events [ 9 ]. Multiple studies have shown that VTE prophylaxis in trauma patients not only reduces VTE events, but also significantly reduces mortality [ 10 ]. We noted the same reduction in mortality for TBI patients following the initiation of a VTE prophylaxis guideline for patients with TBI [ 11 ]. Unfortunately, despite widely published PCOR-informed best practice, nationally there is reluctance to initiate VTE prevention due to concerns for progression of intracranial hemorrhage. This is despite research which has shown early initiation of VTE prophylaxis to be safe in TBI without increased risk of delayed neurosurgical intervention or death [ 12 , 13 , 14 , 15 , 16 ].

Since approximately 40% of TBI patients do not receive DVT prophylaxis in a timely manner, there is a critical and timely need to close the gap between current PCOR evidence and clinical practice. [ 17 , 18 , 19 , 20 , 21 , 22 , 23 ]. Clinical decision support (CDS) systems are an indispensable solution to close this practice gap; however, design and implementation barriers hinder CDS adoption [ 24 , 25 ]. Another significant challenge to the implementation of CDS is that health information technology (IT) needs a common language for PCOR evidence to translate it into practice across multiple organizations [ 26 ]. Because of these challenges, we will deploy CDS using fast healthcare interoperability resources (FHIR) standards to rapidly implement PCOR evidence into practice [ 27 , 28 ]. We hypothesize that, FHIR standards will reduce CDS development and maintenance costs, increase PCOR uptake in rural and other underserved sites, and speed the development timeline to build a comprehensive suite of CDS for PCOR evidence [ 29 ].

Few studies have investigated specific barriers to and facilitating factors for adoption of interoperable FHIR-based CDS [ 30 ]. For example, many current studies investigating barriers and facilitators for interoperable CDS are limited to expert opinion [ 30 , 31 ] or lack a formal implementation science framework-guided investigation [ 32 , 33 ]. Barriers to and facilitating factors for adoption of interoperable CDS following real-life implementation and multicenter scaling guided by validated implementation science frameworks should be rigorously investigated. This study will facilitate comprehensive exploration of clinician and environmental (internal and external) contextual elements that influence interoperable CDS implementation success. In this study, we will scale and assess the effectiveness of a CDS system for a VTE prophylaxis guideline in patients with TBI and evaluate implementation across 9 sites within 4 U.S. trauma systems.

Study aims and implementation framework

This trial consists of a stepped wedge hybrid effectiveness-implementation trial to scale the CDS system across 4 trauma systems and in parallel evaluate implementation strategy guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework (Fig.  1 a) [ 34 ]. We anticipate variability in CDS adoption across sites during the implementation trial. This variation represents a unique opportunity to study implementation at each site and understand what strategies, system factors, and engagement of specific stakeholders are associated with improved CDS adoption. We will rigorously evaluate each implementation phase, guided by The EPIS Implementation Framework [ 34 ], our determinant framework (Fig.  1 b). We will apply the EPIS framework to guide assessment of implementation phases, barriers, and facilitators (Fig.  2 ) [ 34 ]. EPIS comprises 16 constructs over 4 domains (outer context, inner context, bridging factors, and innovation factors). We selected EPIS as our determinant framework as it includes clearly delineated implementation stages and allows for examination of change at multiple levels, across time, and through phases that build toward implementation. While EPIS was initially developed for implementation in public service, it has since been translated to healthcare, especially for complex multi-institutional healthcare interventions [ 34 , 35 , 36 ].

a Randomized Stepped Wedge design of the SCALED clinical trial. b Parallel, implementation evaluation guided by Explore, Preparation, Implementation and Sustain (EPIS) framework

Implementation evaluation across study sites

Trial overview, setting, and inclusion/exclusion criteria

This trial will be conducted at 4 healthcare systems with 1–3 hospitals per system and is projected to occur over a 3 to 4-year period. The trial uses a randomized stepped-wedge design to scale an interoperable CDS system for the Berne-Norwood TBI CPG. Figure  1 a provides a schematic for the trial design. The order of health systems and sites will be randomly determined. This study will include a heterogeneous number of hospitals by trauma verification status, electronic health record (EHR) platform, bed size, and setting (Table  1 ). Our target population is adult patients admitted with an acute TBI defined as International Classification of Disease 10 Clinical Modification (ICD-10-CM): S06.1 – S06.9 or S06.A. Patients who die within 24 h of hospital admission and patients documented as “comfort cares” during the first 72 h of hospitalization will be excluded, as they would have a limited opportunity to receive adherence with the Berne-Norwood criteria. Additionally, patients with a pre-existing VTE or inferior vena cava (IVC) filter at the time of admission, and patients with a mechanical heart valve or ventricular assist device will be excluded from final analysis.

This study will also include up to 3 control sites (Fig.  1 a), a feature not typically included with historic stepped-wedge trial designs, which will strengthen our ability to understand external influences on the study findings. These control sites, which do not receive the CDS intervention and do not have any planned initiatives around guideline implementation, will allow the study to assess baseline adherence and variation in clinical practice over the study period.

CDS Intervention

TBI diagnosis upon admission will activate an interoperable CDS system leveraging the Stanson Health (Charlotte, NC) CDS platform [ 37 ], which is being expanded to include interoperable offerings for TBI VTE prophylaxis. This system provides a knowledge representation framework to faithfully express the intent of the Berne-Norwood prevention criteria computationally (Table  2 ). The interoperable FHIR data standard will be used for bi-directional data transfer between each site’s EHR and the CDS platform. Workflow integration includes a combination of both passive and interruptive provider and trauma system leader information and “nudges”. Table 2 represents the Standards-based, Machine-readable, Adaptive, Requirements-based, and Testable (SMART) L2 layer [ 38 ] of the Berne-Norwood criteria.

CDS user-centered design

We will complete a rapid cycle CDS evaluation to optimize CDS workflow integration by conducting a user-driven simulation and expert-driven heuristic usability optimization as we have previously done [ 39 ]. For rapid cycle CDS evaluation, multidisciplinary trauma end-user “teams” will complete up to 3 scenarios designed to represent various extremes in TBI VTE prevention decision making. Simulation usability testing will be overseen by usability experts, who will catalogue usability issues that arise during simulation. Via consensus ranking, the development and planning teams will rank usability issues from 0 (cosmetic) to 5 (usability catastrophe). Using 10 predefined heuristics for usability design [ 40 ], we will conduct a heuristic evaluation of the CDS, then catalogue and rank usability issues. These results will inform CDS application design, optimized for TBI workflow integration.

Implementation strategy

Following CDS development, our healthcare system relies on a time-tested approach for the implementation and scaling of user-centered CDS: this approach is called the Scaling AcceptabLE cDs (SCALED) Strategy [ 41 ]. This framework integrates multiple evidence-based implementation strategies (Table  3 ).

Study outcomes

The primary implementation outcome is patient-level adherence with the CPG: Specifically, did the patient received guideline-concordant care? Adherence will be measured as an all-or-none measure (binary endpoint at the encounter/patient-level). Thus, if a patient is low-risk for TBI progression, by 24 h they should have risk-specific VTE prevention ordered; if they receive this after 24 h, or if they receive the intermediate risk VTE prevention regimen, this would be deemed non-adherent. The primary effectiveness outcome is VTE (binary endpoint at the patient-encounter level). Safety outcomes evaluated include: TBI progression, in-hospital mortality, and bleeding events. A secondary hypothesis is that as the trial scales to additional sites, iterative implementations will be more efficient (reduced implementation time) and more effective (improved adoption). Secondary hypotheses will be evaluated using the RE 2 -AIM framework [ 42 , 43 ] and are displayed in Table  4 .

Clinical trial data collection methods

Data sources used in this trial include the Stanson Health CDS eCaseReport and site trauma registry. The eCaseReport is a living registry of all patients, and their associated clinical trial data elements, that were eligible for the CDS. All sites also maintain a trauma registry adhering to the National Trauma Data Standards [ 44 ], a requirement for ACS trauma center verification. This dataset is manually annotated by trained clinical abstractors. Data will be sent to the biostatistical team at 6-month intervals. Control and pre-implementation sites will provide their trauma registry in addition to supplemental standards-based EHR extraction of clinical trial data elements or manual abstraction. A data dictionary has been created for the study and will be made available on the trial webpage.

Multiple methods evaluation of implementation success at each EPIS phase

Survey instruments will be prepared using Likert-type scales. Outcomes will be calculated based on scoring guides for the following validated scales: Program Sustainability Assessment Tool (PSAT) [ 45 ], Clinical Sustainability Assessment Tool (CSAT) [ 46 ], Implementation Leadership Scale (ILS) [ 47 ], and Evidenced-based Practice Attitude Scale-36 (EBPAS-36) [ 48 ]. Two scales do not have scoring rubrics: the Organizational Readiness for Change Questionnaire [ 49 , 50 ] and the Normalization Measure Development (NoMAD) Questionnaire [ 51 , 52 , 53 ]. Since both of these scales group questions into constructs, they will be analyzed by generating mean Likert scores and standard deviations per construct, and a mean across constructs, at each of the four implementation phases [ 54 ].

To deeply investigate barriers and facilitators of successful implementation, semi-structured qualitative interviews of key personnel (clinical leadership and end-users, IT leadership and staff) will be conducted at each of the 4 implementation phases. Studies suggest saturation of new ideas occurs after approximately 12 interviews [ 55 ]. Additional samples will be added as needed if thematic saturation is not achieved. Following informed consent, interviews will be performed by a trained qualitative research assistant, audio recorded, and transcribed verbatim. An interview guide, informed by the EPIS framework, was developed to collect key informant experiences with CDS implementation with a focus on inner and outer context factors [ 56 ]. A hybrid approach, primarily deductive and secondarily inductive, approach will be applied. All interviews will be independently double-coded and coding discrepancies will be resolved through discussion. A descriptive thematic analysis approach [ 57 ] will be used to characterize the codes into themes and sub-themes representing the barriers and facilitators to implementation success.

Results for all instruments will be primarily stratified according to site implementation success at each study phase. Additional stratifications may include respondent role, discipline, and hospital system. Bar charts displaying mean survey domains with integrative quotations from the qualitative analysis will be used to facilitate data visualization and understanding of key themes representing barriers and facilitators to successful CDSS implementation.

Statistical analysis

Mixed-effects logistic regression models will be fit to test whether or not CDS implementation changes the likelihood of a VTE event during TBI admission (effectiveness outcome) and the likelihood that the clinical guideline was followed (implementation outcome). The models for these outcomes include fixed-effects for month (when available, to account for secular trends) and an indicator variable for whether the center had the CDS integrated in the EHR. The primary test statistic will be a Wald test of the coefficient for this treatment indicator. We will include random center-specific intercepts to account for correlation within center. Assuming there are 9 sites enrolled with an average of 400 TBI admissions per year and the typical site has between 20%-40% adherence to the clinical guidelines, we will have > 80.0% and > 99.9% power to detect a 5 and 10 percentage point increase in the adherence. Similarly, assuming the typical site has between a VTE event rate of 5–6%, we will have > 80.0% power to detect a 40%-50% reduction in VTE consistent with our published data [ 11 ].

Study oversight

This study is overseen by the University of Minnesota Surgical Clinical Trials Office and by an independent Data Safety Monitoring Board (DSMB). Even though this intervention is deploying a TBI clinical guideline that is currently considered best practice, we believe the addition of a DSMB will improve trial safety, data quality, and trial integrity [ 58 ]. DSMB membership will be independent from the study investigators and will consist of 3 members including: 1 trauma surgeon, 1 informaticist, and 1 statistician. Annual reports including data from all sites, including control sites, will be shared with the DSMB to assure timely monitoring of safety and data quality. The trial will not be stopped early in the event of CDS efficacy because a critical secondary outcome focuses on studying implementation and effectiveness over time.

VTE guideline monitoring and maintenance

Given the potential for a changing evidence-base, it is possible that best practice VTE prevention guidance may change during the study period or afterwards. A critical element in improving adherence with PCOR evidence is updating guidance based on this evidence – in this study, this requires ensuring that the CDS system remains current.

We will pilot a model for producing and maintaining TBI VTE prophylaxis 'Living Guidance and CDS' to ensure that the CDS remains current (Fig.  3 ). The University of Minnesota Evidence-based Practice Center (EPC) Evidence Generation team will conduct and maintain a “living” systematic review. Systematic review data will be uploaded to the AHRQ’s Systematic Review Data Repository (SRDR). “Living” implies that every 6 months the EPC team will evaluate and synthesize new evidence related to TBI VTE prophylaxis, update the existing systematic review and deliver it to a multi-stakeholder Guideline Committee. The Guideline Committee will then use the GRADE (Grading of Recommendations, Assessment, Development and Evaluations) evidence-to-decision (EtD) framework to develop VTE prophylaxis guidelines for patients with TBI [ 59 , 60 , 61 ]. A computational representation of these guidelines will be updated and maintained within the CDS platform by Stanson Health, the CDS Vendor.

Pilot process for “Living Guideline”

Spreading successful results beyond study sites

The ultimate goal of this study is to spread successful CDS tools and strategies to broadly improve TBI VTE-related care processes and outcomes. The research outlined above will surface sharable insights about what information needs to be presented to which people in what formats through what channels at what times to reliably deliver guideline-based care – i.e., specific instantiations of the “CDS 5 Rights Framework” applied to this target [ 62 ]. We will use Health Service Blueprint tools to describe our recommended implementation approaches; these tools are being applied in an increasing number of public and private care delivery organizations as a structured approach to ‘get the CDS 5 Right right’ for various improvement targets. We will further adapt and apply Health Service Blueprint foundations supported by VA and AHRQ [ 63 ] to capture VTE care transformation guidance in Health Service Blueprint tooling [ 64 ]. Presenting recommended CDS-enabled workflow, information flow – as well as and related implementation considerations and broader healthcare ecosystem implications – in this structured format will help organizations beyond the initial study participants put study results into action efficiently and effectively.

In this paper, we present the protocol for the SCALED trial, a stepped-wedge cluster randomized trial of a CDS intervention to improve adherence with VTE prevention best practices for patients with TBI. As a hybrid type 2 trial, this study will evaluate both implementation and effectiveness outcomes. In addition to investigating effectiveness, we will also be able to provide insight into the implementation challenges for deploying interoperable CDS across heterogenous health systems. In our pilot study [ 9 ], while patients who received guideline-concordant care had significantly improved outcomes, we noted that not all patients receive guideline concordant care following implementation. Additionally, best strategies for scaling interoperable CDS systems are poorly studied. Thus, this study represents one of the earliest implementation evaluations of scaling interoperable CDS systems across heterogeneous health systems.

This study has several strengths. First, it will rigorously test implementation of a CPG for VTE prevention across 9 U.S. trauma centers using a multi-faceted CDS platform supporting both passive and interruptive decision support. Second, it will rigorously investigate scalable and interoperable CDS strategies to deploy CPGs. Third, this study leverages a centralized eCaseReport generated by the CDS system, a solution which can drive data collection for future pragmatic trials. Importantly, this study takes place at trauma centers which are geographically distinct, utilize different EHR vendors, include both ACS-verified level 1 through level 3 trauma centers, and include rural, community, and university-based trauma centers. In addition to helping spread recommended care transformation strategies beyond additional study sites, documenting these approaches in Health Service Blueprint tools will also support creation of learning communities for sharing, implementing, and enhancing these strategies.

This study also has limitations. First, we are only investigating 4 trauma systems which already have fairly advanced informatics divisions and experience implementing interoperable CDS systems. Thus, these findings may not be broadly applicable to health systems with less informatics experience and expertise. Second, we are only investigating implementation across two EHR vendors: Epic and Cerner, thus these findings may not be applicable to health systems with different EHR vendors such as Meditech or Allscripts. However, the Health Service Blueprint implementation strategy representations should still enable users of other systems to glean valuable insights about components of the transformation approach less dependent on specific EHRs used.

In summary, this study will implement and scale a CDS-enabled care transformation approach across a diverse collaborative CDS community, serving as an important demonstration of this critical healthcare challenge. We will integrate lessons learned for a planned national scaling in collaboration with U.S. trauma societies. Finally, we will pilot an approach for the “Living Guideline” and use that to maintain evidenced-based decision logic within CDS platforms.

Availability of data and materials

Following trial completion data will be made available upon request through the University of Minnesota Data Repository.

Heit JA. Venous thromboembolism: disease burden, outcomes and risk factors. J Thromb Haemost. 2005;3(8):1611–7.

Article   CAS   PubMed   Google Scholar  

Yorkgitis BK, Berndtson AE, Cross A, Kennedy R, Kochuba MP, Tignanelli C, Tominaga GT, Jacobs DG, Marx WH, Ashley DW, Ley EJ, Napolitano L, Costantini TW. American Association for the Surgery of Trauma/American College of Surgeons-Committee on Trauma Clinical Protocol for inpatient venous thromboembolism prophylaxis after trauma. J Trauma Acute Care Surg. 2022;92(3):597–604.

Article   PubMed   Google Scholar  

Nicholson M, Chan N, Bhagirath V, Ginsberg J. Prevention of Venous Thromboembolism in 2020 and Beyond. J Clin Med. 2020;9(8):2467.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Geerts WH, Code KI, Jay RM, Chen E, Szalai JP. A prospective study of venous thromboembolism after major trauma. N Engl J Med. 1994;331(24):1601–6.

Nathens AB, Cryer HG, Fildes J. The American College of Surgeons Trauma Quality Improvement Program. Surg Clin North Am. 2012;92(2):441–54, x−xi.

Ingraham NE, Lotfi-Emran S, Thielen BK, Techar K, Morris RS, Holtan SG, Dudley RA, Tignanelli CJ. Immunomodulation in COVID-19. Lancet Respir Med. 2020;8(6):544–6.

Phelan HA, Eastman AL, Madden CJ, Aldy K, Berne JD, Norwood SH, Scott WW, Bernstein IH, Pruitt J, Butler G, Rogers L, Minei JP. TBI risk stratification at presentation: a prospective study of the incidence and timing of radiographic worsening in the Parkland Protocol. J Trauma Acute Care Surg. 2012;73(2 Suppl 1):S122–7.

Pastorek RA, Cripps MW, Bernstein IH, Scott WW, Madden CJ, Rickert KL, Wolf SE, Phelan HA. The Parkland Protocol’s modified Berne-Norwood criteria predict two tiers of risk for traumatic brain injury progression. J Neurotrauma. 2014;31(20):1737–43.

Article   PubMed   PubMed Central   Google Scholar  

Tignanelli CJ, Gipson J, Nguyen A, Martinez R, Yang S, Reicks PL, Sybrant C, Roach R, Thorson M, West MA. Implementation of a Prophylactic Anticoagulation Guideline for Patients with Traumatic Brain Injury. Jt Comm J Qual Patient Saf. 2020;46(4):185–91.

PubMed   Google Scholar  

Jacobs BN, Cain-Nielsen AH, Jakubus JL, Mikhail JN, Fath JJ, Regenbogen SE, Hemmila MR. Unfractionated heparin versus low-molecular-weight heparin for venous thromboembolism prophylaxis in trauma. J Trauma Acute Care Surg. 2017;83(1):151–8.

Tignanelli CJ, Silverman GM, Lindemann EA, Trembley AL, Gipson JC, Beilman G, Lyng JW, Finzel R, McEwan R, Knoll BC, Pakhomov S, Melton GB. Natural language processing of prehospital emergency medical services trauma records allows for automated characterization of treatment appropriateness. J Trauma Acute Care Surg. 2020;88(5):607–14.

Kim J, Gearhart MM, Zurick A, Zuccarello M, James L, Luchette FA. Preliminary report on the safety of heparin for deep venous thrombosis prophylaxis after severe head injury. J Trauma. 2002;53(1):38–42; discussion 3.

Cothren CC, Smith WR, Moore EE, Morgan SJ. Utility of once-daily dose of low-molecular-weight heparin to prevent venous thromboembolism in multisystem trauma patients. World J Surg. 2007;31(1):98–104.

Norwood SH, Berne JD, Rowe SA, Villarreal DH, Ledlie JT. Early venous thromboembolism prophylaxis with enoxaparin in patients with blunt traumatic brain injury. J Trauma. 2008;65(5):1021–6; discussion 6-7.

CAS   PubMed   Google Scholar  

Scudday T, Brasel K, Webb T, Codner P, Somberg L, Weigelt J, Herrmann D, Peppard W. Safety and efficacy of prophylactic anticoagulation in patients with traumatic brain injury. J Am Coll Surg. 2011;213(1):148–53; discussion 53-4.

Byrne JP, Mason SA, Gomez D, Hoeft C, Subacius H, Xiong W, Neal M, Pirouzmand F, Nathens AB. Timing of Pharmacologic Venous Thromboembolism Prophylaxis in Severe Traumatic Brain Injury: A Propensity-Matched Cohort Study. J Am Coll Surg. 2016;223(4):621-31e5.

Lau R, Stevenson F, Ong BN, Dziedzic K, Eldridge S, Everitt H, Kennedy A, Kontopantelis E, Little P, Qureshi N, Rogers A, Treweek S, Peacock R, Murray E. Addressing the evidence to practice gap for complex interventions in primary care: a systematic review of reviews protocol. BMJ Open. 2014;4(6): e005548.

Tignanelli CJ, Vander Kolk WE, Mikhail JN, Delano MJ, Hemmila MR. Noncompliance with American College of Surgeons Committee on Trauma recommended criteria for full trauma team activation is associated with undertriage deaths. J Trauma Acute Care Surg. 2018;84(2):287–94.

Robbins AJ, Ingraham NE, Sheka AC, Pendleton KM, Morris R, Rix A, Vakayil V, Chipman JG, Charles A, Tignanelli CJ. Discordant Cardiopulmonary Resuscitation and Code Status at Death. J Pain Symptom Manage. 2021;61(4):770–780.e1.

Tignanelli CJ, Watarai B, Fan Y, Petersen A, Hemmila M, Napolitano L, Jarosek S, Charles A. Racial Disparities at Mixed-Race and Minority Hospitals: Treatment of African American Males With High-Grade Splenic Injuries. Am Surg. 2020;86(5):441–9.

Tignanelli CJ, Rix A, Napolitano LM, Hemmila MR, Ma S, Kummerfeld E. Association Between Adherence to Evidence-Based Practices for Treatment of Patients With Traumatic Rib Fractures and Mortality Rates Among US Trauma Centers. JAMA Netw Open. 2020;3(3): e201316.

Oliphant BW, Tignanelli CJ, Napolitano LM, Goulet JA, Hemmila MR. American College of Surgeons Committee on Trauma verification level affects trauma center management of pelvic ring injuries and patient mortality. J Trauma Acute Care Surg. 2019;86(1):1–10.

Tignanelli CJ, Wiktor AJ, Vatsaas CJ, Sachdev G, Heung M, Park PK, Raghavendran K, Napolitano LM. Outcomes of Acute Kidney Injury in Patients With Severe ARDS Due to Influenza A(H1N1) pdm09 Virus. Am J Crit Care. 2018;27(1):67–73.

Khairat S, Marc D, Crosby W, Al SA. Reasons For Physicians Not Adopting Clinical Decision Support Systems: Critical Analysis. JMIR Med Inform. 2018;6(2): e24.

Jones EK, Ninkovic I, Bahr M, Dodge S, Doering M, Martin D, Ottosen J, Allen T, Melton GB, Tignanelli CJ. A novel, evidence-based, comprehensive clinical decision support system improves outcomes for patients with traumatic rib fractures. J Trauma Acute Care Surg. 2023;95(2):161–71.

Marcos M, Maldonado JA, Martinez-Salvador B, Bosca D, Robles M. Interoperability of clinical decision-support systems and electronic health records using archetypes: a case study in clinical trial eligibility. J Biomed Inform. 2013;46(4):676–89.

FHIR Clinical Guidelines. http://build.fhir.org/ig/HL7/cqf-recommendations/ . Accessed 14 Sep 2021. 

Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc. 2016;23(5):899–908.

Goldberg HS, Paterno MD, Rocha BH, Schaeffer M, Wright A, Erickson JL, Middleton B. A highly scalable, interoperable clinical decision support service. J Am Med Inform Assoc. 2014;21(e1):e55-62.

Marcial LH, Blumenfeld B, Harle C, Jing X, Keller MS, Lee V, Lin Z, Dover A, Midboe AM, Al-Showk S, Bradley V, Breen J, Fadden M, Lomotan E, Marco-Ruiz L, Mohamed R, O’Connor P, Rosendale D, Solomon H, Kawamoto K. Barriers, Facilitators, and Potential Solutions to Advancing Interoperable Clinical Decision Support: Multi-Stakeholder Consensus Recommendations for the Opioid Use Case. AMIA Annu Symp Proc. 2019;2019:637–46.

Lomotan EA, Meadows G, Michaels M, Michel JJ, Miller K. To Share is Human! Advancing Evidence into Practice through a National Repository of Interoperable Clinical Decision Support. Appl Clin Inform. 2020;11(1):112–21.

Dolin RH, Boxwala A, Shalaby J. A Pharmacogenomics Clinical Decision Support Service Based on FHIR and CDS Hooks. Methods Inf Med. 2018;57(S 02):e115–23.

Dorr DA, D’Autremont C, Pizzimenti C, Weiskopf N, Rope R, Kassakian S, Richardson JE, McClure R, Eisenberg F. Assessing Data Adequacy for High Blood Pressure Clinical Decision Support: A Quantitative Analysis. Appl Clin Inform. 2021;12(4):710–20.

Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implement Sci. 2019;14(1):1.

Becan JE, Bartkowski JP, Knight DK, Wiley TRA, DiClemente R, Ducharme L, Welsh WN, Bowser D, McCollister K, Hiller M, Spaulding AC, Flynn PM, Swartzendruber A, Dickson MF, Fisher JH, Aarons GA. A model for rigorously applying the Exploration, Preparation, Implementation, Sustainment (EPIS) framework in the design and measurement of a large scale collaborative multi-site study. Health Justice. 2018;6(1):9.

Idalski Carcone A, Coyle K, Gurung S, Cain D, Dilones RE, Jadwin-Cakmak L, Parsons JT, Naar S. Implementation Science Research Examining the Integration of Evidence-Based Practices Into HIV Prevention and Clinical Care: Protocol for a Mixed-Methods Study Using the Exploration, Preparation, Implementation, and Sustainment (EPIS) Model. JMIR Res Protoc. 2019;8(5): e11202.

Jackson JM, Witek MA, Hupert ML, Brady C, Pullagurla S, Kamande J, Aufforth RD, Tignanelli CJ, Torphy RJ, Yeh JJ, Soper SA. UV activation of polymeric high aspect ratio microstructures: ramifications in antibody surface loading for circulating tumor cell selection. Lab Chip. 2014;14(1):106–17.

Mazzag B, Tignanelli CJ, Smith GD. The effect of residual Ca2+ on the stochastic gating of Ca2+-regulated Ca2+ channel models. J Theor Biol. 2005;235(1):121–50.

Jones EK, Hultman G, Schmoke K, Ninkovic I, Dodge S, Bahr M, Melton GB, Marquard J, Tignanelli CJ. Combined Expert and User-Driven Usability Assessment of Trauma Decision Support Systems Improves User-Centered Design. Surgery. 2022;172(5):1537–48.

Jakob N. Enhancing the explanatory power of usability heuristics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '94). New York: Association for Computing Machinery; 1994. p. 152–8. https://doi.org/10.1145/191666.191729 .

Shah S, Switzer S, Shippee ND, Wogensen P, Kosednar K, Jones E, Pestka DL, Badlani S, Butler M, Wagner B, White K, Rhein J, Benson B, Reding M, Usher M, Melton GB, Tignanelli CJ. Implementation of an Anticoagulation Practice Guideline for COVID-19 via a Clinical Decision Support System in a Large Academic Health System and Its Evaluation: Observational Study. JMIR Med Inform. 2021;9(11): e30743.

Ingraham NE, Jones EK, King S, Dries J, Phillips M, Loftus T, Evans HL, Melton GB, Tignanelli CJ. Re-Aiming Equity Evaluation in Clinical Decision Support: A Scoping Review of Equity Assessments in Surgical Decision Support Systems. Ann Surg. 2023;277(3):359–64.

Holtrop JS, Estabrooks PA, Gaglio B, Harden SM, Kessler RS, King DK, Kwan BM, Ory MG, Rabin BA, Shelton RC, Glasgow RE. Understanding and applying the RE-AIM framework: Clarifications and resources. J Clin Transl Sci. 2021;5(1): e126.

https://www.facs.org/-/media/files/quality-programs/trauma/ntdb/ntds/data-dictionaries/ntds_data_dictionary_2022.ashx . Accessed 14 Sep 2021. ACoSNTDSDDA.

https://www.cdc.gov/pcd/issues/2014/13_0184.htm . Accessed 1/3/2021.

Malone S, Prewitt K, Hackett R, Lin JC, McKay V, Walsh-Bailey C, Luke DA. The Clinical Sustainability Assessment Tool: measuring organizational capacity to promote sustainability in healthcare. Implement Sci Commun. 2021;2(1):77.

Aarons GA, Ehrhart MG, Farahnak LR. The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45.

Rye M, Torres EM, Friborg O, Skre I, Aarons GA. The Evidence-based Practice Attitude Scale-36 (EBPAS-36): a brief and pragmatic measure of attitudes to evidence-based practice validated in US and Norwegian samples. Implement Sci. 2017;12(1):44.

Holt DT, Armenakis AA, Feild HS, Harris SG. Readiness for Organizational Change. J Appl Behav Sci. 2007;43(2):232–55.

Article   Google Scholar  

Weiner BJ. A theory of organizational readiness for change. Implement Sci. 2009;4:67.

Goodridge D, Rana M, Harrison EL, Rotter T, Dobson R, Groot G, Udod S, Lloyd J. Assessing the implementation processes of a large-scale, multi-year quality improvement initiative: survey of health care providers. BMC Health Serv Res. 2018;18(1):237.

Vis C, Ruwaard J, Finch T, Rapley T, de Beurs D, van Stel H, van Lettow B, Mol M, Kleiboer A, Riper H, Smit J. Toward an Objective Assessment of Implementation Processes for Innovations in Health Care: Psychometric Evaluation of the Normalization Measure Development (NoMAD) Questionnaire Among Mental Health Care Professionals. J Med Internet Res. 2019;21(2): e12376.

NoMAD. https://www.implementall.eu/17-nomad.html . Accessed 1/2/2021.

Ng F, McGrath BA, Seth R, et al. Measuring multidisciplinary staff engagement in a national tracheostomy quality improvement project using the NoMAD instrument. Br J Anesth. 2019;123(4):e506.

Guest G, Bunce A, Johnson L. How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability. Field Methods. 2006;18:59–82.

Beidas RS, Stewart RE, Adams DR, Fernandez T, Lustbader S, Powell BJ, Aarons GA, Hoagwood KE, Evans AC, Hurford MO, Rubin R, Hadley T, Mandell DS, Barg FK. A Multi-Level Examination of Stakeholder Perspectives of Implementation of Evidence-Based Practices in a Large Urban Publicly-Funded Mental Health System. Adm Policy Ment Health. 2016;43(6):893–908.

Braun V, Clarke V. Thematic analysis. In Cooper H, Camic PM, Long DL, Panter AT, Rindskopf D, Sher KJ, editors. APA handbooks in psychology®. APA handbook of research methods in psychology, vol. 2. Research designs: Quantitative, qualitative, neuropsychological, and biological. American Psychological Association; 2012. p. 57–71.

Fiscella K, Sanders M, Holder T, Carroll JK, Luque A, Cassells A, Johnson BA, Williams SK, Tobin JN. The role of data and safety monitoring boards in implementation trials: When are they justified? J Clin Transl Sci. 2020;4(3):229–32.

Alonso-Coello P, Schunemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Rada G, Rosenbaum S, Morelli A, Guyatt GH, Oxman AF, Group GW. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016.

Rosenbaum SE, Moberg J, Glenton C, Schunemann HJ, Lewin S, Akl E, Mustafa RA, Morelli A, Vogel JP, Alonso-Coello P, Rada G, Vasquez J, Parmelli E, Gulmezoglu AM, Flottorp SA, Oxman AD. Developing Evidence to Decision Frameworks and an Interactive Evidence to Decision Tool for Making and Using Decisions and Recommendations in Health Care. Glob Chall. 2018;2(9):1700081.

Alonso-Coello P, Oxman AD, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Vandvik PO, Meerpohl J, Guyatt GH, Schunemann HJ, Group GW. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ. 2016;353:i2089.

Osheroff JA. CDS and and the CDS & LHS 5 Rights. CDS/PI Collaborative: Getting Better Faster Together.

ACTS Project Team. Patient Journey and Service Blueprint How Tos. AHRQ evidence-based Care Transformation Support (ACTS) Home. [Online] October 2021. https://cmext.ahrq.gov/confluence/display/PUB/Patient+Journey+and+Service+Blueprint+How+Tos .

CDS Approach for Optimizing VTE Prophylaxis (VTEP) Society of Hospital Medicine (SHM) Recommendations1 Version 2; March, 2013. [online] https://www.healthit.gov/sites/default/files/cds/Detailed%20Inpatient%20CDS-QI%20Worksheet%20-%20VTE%20Example%20-%20Recommendations.xlsx .

Download references

This research was supported by the Agency for Healthcare Research and Quality (AHRQ), grant R18HS028583, the University of Minnesota Center for Learning Health System Sciences – a partnership between the University of Minnesota Medical School and the School of Public Health. The authors have no other conflicts of interest.

Author information

Authors and affiliations.

Department of Surgery, University of Minnesota, 420 Delaware St SE, MMC 195, Minneapolis, MN, 55455, USA

Christopher J. Tignanelli, Vincent Peta, Nicholas Lemke & Genevieve B. Melton

Institute for Health Informatics, University of Minnesota, Minneapolis, MN, USA

Christopher J. Tignanelli, Rubina Rizvi & Genevieve B. Melton

Center for Learning Health Systems Science, University of Minnesota, Minneapolis, MN, USA

Christopher J. Tignanelli, Rubina Rizvi, Mary Butler & Genevieve B. Melton

Center for Quality Outcomes, Discovery and Evaluation, University of Minnesota, Minneapolis, MN, USA

Christopher J. Tignanelli

Department of Medicine, Mayo Clinic, Scottsdale, AZ, USA

Surbhi Shah

Division of Biostatistics and Health Data Science, University of Minnesota, Minneapolis, MN, USA

David Vock, Lianne Siegel & Carlos Serrano

Department of Surgery, Johns Hopkins University, Baltimore, MD, USA

Elliott Haut

M Health Fairview, Minneapolis, MN, USA

Sean Switzer

School of Nursing, University of Minnesota, Minneapolis, MN, USA

Christie L. Martin

Department of Surgery, Indiana University School of Medicine, Indianapolis, IN, USA

Peter C. Jenkins

Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, IN, USA

Thankam Thyvalikakath

Indiana University School of Dentistry, Indianapolis, IN, USA

TMIT Consulting, LLC, Naples, FL, USA

Jerome A. Osheroff

Department of Surgery, Geisinger Health, Danville, PA, USA

Denise Torres

Department of Biomedical Informatics, Geisinger Health, Danville, PA, USA

David Vawdrey

Department of Surgery, UC Davis School of Medicine, Sacramento, CA, USA

Rachael A. Callcut

School of Publish Health, University of Minnesota, Minneapolis, MN, USA

Mary Butler

You can also search for this author in PubMed   Google Scholar

Contributions

CT conceived and jointly designed the study protocol and helped write and critically revise this protocol paper, SS conceived and jointly designed the study protocol and helped write and critically revise this protocol paper, DV jointly designed the study protocol and helped write and critically revise this protocol paper, LS jointly designed the study protocol and helped write and critically revise this protocol paper, CS jointly designed the study protocol and helped write and critically revise this protocol paper, EH jointly designed the study protocol and helped write and critically revise this protocol paper, SS jointly designed the study protocol and helped write and critically revise this protocol paper, CM jointly designed the study protocol and helped write and critically revise this protocol paper, RR jointly designed the study protocol and helped write and critically revise this protocol paper, VP jointly designed the study protocol and helped write and critically revise this protocol paper, PJ jointly designed the study protocol and helped write and critically revise this protocol paper, NL jointly designed the study protocol and helped write and critically revise this protocol paper, TT jointly designed the study protocol and helped write and critically revise this protocol paper, JO jointly designed the study protocol and helped write and critically revise this protocol paper, DT jointly designed the study protocol and helped write and critically revise this protocol paper, DV jointly designed the study protocol and helped write and critically revise this protocol paper, RC jointly designed the study protocol and helped write and critically revise this protocol paper, MB jointly designed the study protocol and helped write and critically revise this protocol paper, GM conceived and jointly designed the study protocol and helped write and critically revise this protocol paper.

Corresponding author

Correspondence to Christopher J. Tignanelli .

Ethics declarations

Ethics approval and consent to participate.

This study protocol was given the determination of “Exempt” as secondary research for which consent is not required. The Mixed Methods investigation was given the determination of “Not Human Research” as a quality improvement activity.

Consent for publication

Not applicable.

Competing interests

The authors have no conflicts of interest to report.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Tignanelli, C.J., Shah, S., Vock, D. et al. A pragmatic, stepped-wedge, hybrid type II trial of interoperable clinical decision support to improve venous thromboembolism prophylaxis for patients with traumatic brain injury. Implementation Sci 19 , 57 (2024). https://doi.org/10.1186/s13012-024-01386-4

Download citation

Received : 09 June 2024

Accepted : 14 July 2024

Published : 05 August 2024

DOI : https://doi.org/10.1186/s13012-024-01386-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Traumatic brain injury
• Prophylaxis
• Venous thromboembolism
• Stepped wedge
• Implementation science
• Mixed methods
• Clinical decision support
• Randomized controlled trial
• Learning health system
• Health informatics

Implementation Science

ISSN: 1748-5908

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

ORIGINAL RESEARCH article

Clinical presentation, magnetic resonance imaging findings and outcome of 80 dachshunds with cervical intervertebral disc extrusion.

• 1 Willows Veterinary Centre & Referral Service, part of Linnaeus Veterinary Limited, Solihull, United Kingdom
• 2 AniCura Istituto Veterinario Novara, Novara, Italy
• 3 Studio Veterinario Associato Vet2Vet Ferri e Porporato, Torino, Italy
• 4 Clinica Neurologica Veterinaria - NVA, Milano, Italy
• 5 Dipartimento di Medicina Animale, Produzioni e Salute - Università degli Studi di Padova, Padua, Italy
• 6 Clinic for Small Animal Internal Medicine, Vetsuisse Faculty, University of Zurich, Zurich, Zurich, Switzerland

The final, formatted version of the article will be published soon.

Select one of your emails

You have multiple emails registered with Frontiers:

Notify me on publication

Please enter your email address:

If you already have an account, please login

You don't have a Frontiers account ? You can register here

Large clinical studies regarding cervical intervertebral disc extrusion (IVDE) in Dachshunds are lacking. This retrospective multicentric study therefore aims to describe the clinical features, magnetic resonance imaging (MRI) findings and outcomes of Dachshunds diagnosed with cervical IVDE. Medical records of Dachshunds with cervical IVDE were reviewed for signalment, onset of clinical signs, neurological examination, MRI features, treatment and outcome. Eighty Dachshunds were included in the study, mostly ambulatory (55% grade 1 and 33% grade 2) and without nerve root signature (85% of cases) on presentation. Information on coat type was available for 56% of dogs; specifically, 41% were smooth-haired, 9% were long-haired and 6% were wire-haired Dachshunds. There were 29 (36%) neutered female, 27 (34%) male entire, 15 (19%) male neutered and 9 (11%) entire female dogs. The onset of clinical signs was most often > 48 hours (84%). The most common intervertebral disc space affected was C2-C3 (38%) and foraminal IVDEs were reported in 14 % of dogs. A foraminal IVDE was diagnosed in only 25% of dogs presented with nerve root signatures. Most dogs (77.5%) were treated surgically. In this group, a higher body condition score on presentation and a higher mean spinal cord compression ratio calculated on MRI were directly and moderately associated with a longer hospitalization time (r=0.490 p=0.005 and r=0.310 p=0.012, respectively). The recovery time was longer in dogs with an onset of clinical signs 48 hours (3.1±6.5 days versus 1.6±6.2, p<0.001) in both medically and surgically treated groups. Data about the outcome was available for 83% of dogs. Eighty percent of the entire population of dogs was considered to have completely returned to normal. There was no association between the therapeutic choice (surgical versus medical management) and the outcome of the dogs included in this study.

Keywords: intervertebral disc extrusion, Dachshund, cervical disc extrusions, spinal surgery, MRI

Received: 25 May 2024; Accepted: 14 Aug 2024.

Copyright: © 2024 Violini, Tirrito, Cozzi, Contiero, Anesi, Zini and Toni. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Francesca Violini, Willows Veterinary Centre & Referral Service, part of Linnaeus Veterinary Limited, Solihull, United Kingdom

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

• Open access
• Published: 12 August 2024

Micropapillary breast carcinoma in comparison with invasive duct carcinoma. Does it have an aggressive clinical presentation and an unfavorable prognosis?

• Yasmine Hany Abdel Moamen Elzohery 1 , 5 ,
• Amira H. Radwan 2 , 5 ,
• Sherihan W. Y. Gareer 2 , 5 ,
• Mona M. Mamdouh 3 , 5 ,
• Inas Moaz 4 , 5 ,
• Abdelrahman Mohammad Khalifa 5 ,
• Osama Abdel Mohen 5 ,
• Mohamed Fathy Abdelfattah Abdelrahman Elithy 5   nAff6 &
• Mahmoud Hassaan 5   nAff7  

BMC Cancer volume  24 , Article number:  992 ( 2024 ) Cite this article

133 Accesses

Metrics details

Invasive micropapillary carcinoma (IMPC) was first proposed as an entity by Fisher et al. In the 2003 World Health Organization (WHO) guidelines for histologic classification of the breast tumors. IMPC was recognized as a distinct, rare histological subtype of breast cancer.

IMPC is emerging as a surgical and oncological challenge due to its tendency to manifest as a palpable mass, larger in size and higher in grade than IDC with more rate of lymphovascular invasion (LVI) and lymph node (LN) involvement, which changes the surgical and adjuvant management plans to more aggressive, with comparative prognosis still being a point of ongoing debate.

Aim of the study

In this study, we compared the clinicopathological characteristics, survival and surgical management of breast cancer patients having invasive micropapillary carcinoma pathological subtype in comparison to those having invasive duct carcinoma.

This is a comparative study on female patients presented to Baheya center for early detection and treatment of breast cancer, in the period from 2015 to 2022 diagnosed with breast cancer of IMPC subtype in one group compared with another group of invasive duct carcinoma. we analyzed 138 cases of IMPC and 500 cases of IDC.

The incidence of LVI in the IMPC group was 88.3% in comparison to 47.0% in the IDC group (p < 0.001). IMPC had a higher incidence of lymph node involvement than the IDC group (68.8% and 56% respectively). IMPC had a lower rate of breast conserving surgery (26% vs.37.8%) compared with IDC.

The survival analysis indicated that IMPC patients had no significant difference in overall survival compared with IDC patients and no differences were noted in locoregional recurrence rate and distant metastasis rate comparing IMPCs with IDCs.

The results from our PSM analysis suggested that there was no statistically significant difference in prognosis between IMPC and IDC patients after matching them with similar clinical characteristics. However, IMPC was found to be more aggressive, had larger tumor size, greater lymph node metastasis rate and an advanced tumor stage.

Peer Review reports

Introduction

Breast cancer is the most common cancer in women. In the 2012 World Health Organization (WHO) classification of breast cancer. Breast Cancer is classified into up to 21 different histological types depending on cell growth, morphology and architecture patterns [ 1 ]. The invasive carcinoma of no special type (IBC-NST), which is known as invasive ductal carcinoma (IDC), is the most frequently occurring histological type, which constitutes around 75% of invasive breast carcinoma [ 2 ].

Invasive micropapillary carcinoma (IMPC) was first proposed as an entity by Fisher et al. in 1980 [ 3 ] and first described as the term “invasive micropapillary carcinoma” by Siriaunkgul et al. [ 4 ] in 1993.

In the 2003 World Health Organization (WHO) guidelines for histologic classification of the breast tumors [ 5 ]. IMPC was recognized as a distinct, rare histological subtype of breast cancer. While micropapillary histological architecture is present in 2–8% of breast carcinomas, pure micropapillary carcinoma is uncommon and accounts for 0.9–2% of all breast cancers [ 6 ].

IMPC exhibits more distinct morphologic architecture than the IDC, characterized by pseudopapillary and tubuloalveolar arrangements of tumor cell clusters in clear empty sponge-like spaces that resemble extensive lymphatic invasion [ 7 ]. The neoplastic cell exhibits an “inside-out” pattern, known as the reverse polarity pattern [ 2 ].

Most studies demonstrate that the radiological findings of IMPC are irregular-shaped masses with an angular or spiculated margin on ultrasound, mammography and MRI with heterogeneous enhancement and washout kinetics on MRI [ 8 ].

IMPC had tendency to manifest as a palpable mass, larger in size and higher in grade than IDC with more rate of lymphovascular invasion (LVI) and lymph node (LN) involvement, which changes the surgical and adjuvant management plans to more aggressive, with comparative prognosis still being a point of ongoing debate [ 9 ].

In this study, we compared the clinicopathological characteristics, survival and surgical management of breast cancer patients having invasive micropapillary carcinoma pathological subtype in comparison to those having invasive ductal carcinoma.

Patient and method

This is a comparative study on female patients presented to Baheya center for early detection and treatment of breast cancer, in the period from 2015 to 2022 diagnosed with breast cancer of IMPC subtype in one group compared with another group of invasive duct carcinoma.

This retrospective study analyzed 138 cases of IMPC and 500 cases of IDC. Informed consent was obtained from all patients. Ethical approval is obtained from Baheya center for early detection and treatment of breast cancer and National research center ethics committee. Baheya IRB protocol number:202305150022.

The following clinical-pathological features were analyzed for each case: patient age at diagnosis, clinical presentation, laterality, imaging findings, histopathological examination, treatment plan with either primary surgical intervention or other treatment protocol according to tumor stage and biological subtypes.

A breast pathologist evaluated the tumor size, type, grade, lymphovascular invasion, estrogen receptor (ER), progesterone receptor (PR), human epidermal growth factor receptor 2 (HER2) receptor and the axillary lymph node involvement.

According to the ASCO/CAP guideline update, 2019: Samples with 1% to 100% of tumor nuclei positive for ER or progesterone receptor (PgR) are interpreted as positive. If ER (not PgR), 1% to 10% of tumor cell nuclei are immunoreactive, the sample are reported as ER Low Positive. There are limited data on the overall benefit of endocrine therapies for patients with low level (1%-10%) ER expression, but they currently suggest possible benefit, so patients are considered eligible for endocrine treatment. A sample is considered negative for ER or PgR if < 1% or 0% of tumor cell nuclei are immunoreactive [ 10 ]. An Allred score between 0 and 8. This scoring system looks at what percentage of cells test positive for hormone receptors, along with how well the receptors show up after staining, called intensity: proportion of cells staining (0, no staining; 1, < 1%; 2, between 1 and 10%; 3, between 11 and 33%; 4, between 34 and 66% and 5, between 67%–100% of the cells staining). Intensity of positive tumor cells (0, none; 1, weak, 2, intermediate; and 3, strong) [ 11 ].

HER2 Test Guideline IHC Recommendations, 2018. IHC 0: as defined by no staining observed or membrane staining that is incomplete and is faint/barely perceptible and within <  = 10% of the invasive tumor cells. IHC 1 + : as defined by incomplete membrane staining that is faint/barely perceptible and within > 10% of the invasive tumor cells. IHC 2 + : The revised definition of IHC 2 + (equivocal) is weak to moderate complete membrane staining observed in > 10% of tumor cells. IHC 3 + : based on circumferential membrane staining that is complete, intense in > 10% of tumor cells. [ 12 ].

ASCO–CAP HER2 SISH Test Guideline Recommendations,2018 Twenty nuclei (each containing red (Chr17) and black (HER2) signals) should be enumerated. The final results for the HER2 status are reported based on the ratio formed by dividing the sum of HER2 signals for all 20 nuclei divided by the sum of Chromosome 17 signals for all 20 nuclei. The amplification status is defined as Amplified if the HER2/Chromosome 17 ratio > / = 2.0 and the average Her2 gene copy number is > / = 4.0. It is non-Amplified if the HER2/Chromosome 17 ratio < 2.0 with the Her2 gene copy number is < 4.0. If the HER2/Chr17 ratio is < 2 and the Her2 gene copy number is between 4.0 and 6.0, or, HER2/Chr17 ratio is > / = 2 and the Her2 gene copy number is < 4, or HER2/Chr17 ratio is < 2 and the Her2 gene copy number is > / = 6.0, an additional work should be done. [ 12 ].

Follow-up duration was calculated from the date of diagnosis to the date of the last follow-up. Patients still alive at the last follow-up censored or to the date of occurrence of any event or death.

Disease-free survival was defined as the duration (months) from the initial diagnosis of breast cancer to first any type of recurrence (invasive ipsilateral breast tumor recurrence, local invasive recurrence, regional invasive recurrence, invasive contra lateral breast cancer, distant metastasis.

Overall survival (OS) is defined as the time from diagnosis of breast cancer to death from any cause.

Data were statistically analyzed using an IBM-compatible personal computer with Statistical Package for the Social Sciences (SPSS) version 23. Quantitative data were expressed as mean, standard deviation (SD) and range (minimum–maximum). Qualitative data were expressed as Number (N) and percentage (%), while A P value of < 0.05 was statistically significant. For comparison of unmatched data, chi-square tests were used for categorical variables and t-tests or Mann–Whitney tests for continuous variables.

In this study, we analyzed 138 cases of IMPC which presented to our center in the period from 2015 to 2022.We included a total number of 500 cases of IDC as controls with a ratio of controls to cases 4:1.

Propensity score matching (PSM) is a method for filtrating experimental and control cases of similar characteristics, which are called the matching variables, from existing data to make them comparable in a retrospective analysis. PSM reduce the effect of selection bias. So, the comparison of outcomes between two groups can be fair.

The variables for propensity score matching were selected as follows: age (years), tumour size (cm), nodal status, HR status and HER2 status.

To diminish the effects of baseline differences and potential confounds in clinical characteristics and patients across histology subtypes for outcome differences (disease-free survival and overall survival), PSM method was applied with each micropapillary patient matched to one IDC patient who showed similar baseline characteristics in terms of: menopausal status, comorbidities, multiplicity, histologic grade, tumor size, stage, nodal status, ER /PR status. Differences in prognosis were assessed by Kaplan–Meier analysis.

Most of the patients were postmenopausal, the mean age of patients in IMPC group was 57.36 ± 11.321 years while the mean age of the IDC group was 56.63 ± 9.719 years ( p  = 0.45) (Table 1 ).

The most common presentation of IMPC on breast mammography was an irregular shaped mass with a non-circumscribed spiculated margin. while, the most common sonographic finding of IMPC was hypoechoic mass with irregular shapes and spiculated margins. Associated microcalcifications were found in 49 patients (35.5%) of IMPC group. Figs. ( 1 , 2 ): Radiological characteristics of IMPC.

A , B 37-years-old female patient presented with Left breast UOQ extensive fine pleomorphic and amorphous calcifications of segmental distribution, with UOQ multiple indistinct irregular masses. C ultrasound showed left breast UOQ multiple irregular hypoechoic masses with calcific echogenic foci, the largest is seen at 1 o’clock measuring 13 × 15mm. Intraductal echogenic lesions are noted

A , B , C 40-years-old female patient presented with left UOQ extensive pleomorphic microcalcifications of segmental distribution reaching the areola, with multiple well-circumscribed small obscured masses. D , E complementary Ultrasound showed left 2 o’clock multiple ill-defined and well-defined hypoechoic masses (BIRADS 5)

All patients underwent axillary sonography where 77 patients (55.8%) of the IMPC group exhibited pathological lymph nodes and 18 patients (13%) had indeterminate lymph nodes demonstrating preserved hila and associated with either a symmetrical increase of their cortical thickness reaching 3mm or with a focal increase in the cortical thickness.

Multiple lesions were detected in 30% of IMPC patients in comparison to 7% of IDC patients. Intra-ductal extension with nipple involvement was found in 44 patients (31.9%) of the IMPC group (Table 2 ).

MRI was done for 5 cases (3.6%), while CESM was performed for 18 cases (13%) of the IMPC group, the commonest presentation of IMPC in contrast study was irregular shaped enhanced mass in 21 patients and non-mass enhancement was found in 5 patients. Figs. ( 3 , 4 ).

Further imaging modalities. A , B , C 60-years-old female patient had right breast irregular hypoechoic solid mass by ultrasound (BIRADS 5). D , E CESM showed a right breast irregular heterogeneously enhancing solid mass

Role of CESM in diagnosis of IMPC patients. A , B 42-years-old patient presented with a left LIQ irregular spiculated mass with suspicious microcalcifications, other similar lesions were seen anterior and posterior at the same line. C Ultrasound showed a heterogeneously hypoechoic irregular mass with a spiculated outline with multiple similar satellite lesions were seen anterior and posterior to the main lesions

The average tumor size in the IMPC and IDC groups was 3.37 ± 2.04 cm and 2.72 ± 1.39 cm, respectively ( P  < 0.001).

The percentage of tumors larger than 5cm, was reported 9.5% in IMPC and 7.4% in IDC.

The pure form of IMPC was the most common type and found in 90 cases (65%) and 47 cases (34%) were mixed type where IDC was the commonest associated type.

There are 6 cases in the IMPC group diagnosed as invasive mucinous carcinoma on biopsy, then in the specimen was mixed invasive micropapillary, IBC-NST and invasive mucinous carcinoma.

On core biopsy, 28 cases were diagnosed as IMPC with focal IDC component, but in corresponding specimens 10 cases were only approved to be mixed invasive micropapillary and invasive duct carcinoma, while others diagnosed as pure invasive micropapillary carcinoma without IDC component.

On the other hand, 48 of our cases were diagnosed as IDC on core biopsy, but in the final specimen examination, 17 of these cases were diagnosed as pure invasive micropapillary carcinoma without invasive ductal component.

The explanation of controversy in proper histologic subtyping of carcinoma on core biopsy and the definite subtype on the corresponding specimen was that the ductal component which only represented in the biopsy is a very minor component of the tumor or the limited sampling, tissue fragmentation and architecture distortion in core biopsy may cause diagnostic pitfalls as regard precise subtyping of the tumor.

The incidence of LVI in the IMPC group was 88.3% in comparison to 47.0% in the IDC group ( p  < 0.001).

IMPC had a higher incidence of lymph node involvement than the IDC group (68.8% and 56% respectively) with N3 stage reported in 12.4% of IMPC patients.

IMPC had a higher nuclear grade than the IDC group (25.1% and 15.2% respectively).

The percentage of ER-positive patients was 97.8% in the IMPC group and 87.6% in the IDC group ( p  < 0.001), while PR-positive cases were 98.6% in the IMPC group and 88.8% in the IDC group ( p  < 0.001). HER2 status was positive in 4.3% of IMPCs and 8% of IDCs ( p  = 0.23) (Table 3 ) (Figs. 5 ,  6 ).

A case of invasive micropapillary carcinoma. A case of invasive micropapillary carcinoma, grade II. A Tissue core biopsy, × 100, B MRM specimen × 100 with Positive metastatic L. nodes 2/15, C ER is positive in > 90% of tumor cells, × 100, D PR is positive in > 90% of tumor cells, × 400, E HER2/neu is negative, × 400 and F) Ki-67 labelling index is high, × 200. This case was considered as luminal type pure invasive micropapillary carcinoma. (100 micron 20__ 50 micron 40)

A case of invasive duct carcinoma. A case of invasive duct carcinoma, grade II. A Tissue core biopsy, × 100, B MRM specimen, × 200 with negative L. nodes 0/16, C ER is positive in > 90% of tumor cells, × 200, D PR is positive in > 90% of tumor cells, × 100, E HER2/neu is negative, × 400. This case was considered as luminal type pure invasive duct carcinoma

Regarding definitive surgical management, IMPC had a lower rate of breast conserving surgery (26% vs.37.8%) compared with IDC. While, 49.3% of IMPC patients underwent modified radical mastectomy in comparison to 46% of the IDC patients. Such high incidence of mastectomy was due to the advanced stage at presentation, presence of multiple lesions and presence of intra-ductal extension with nipple involvement.

The incidence of re-surgery in the IMPC group was only in 3 cases, two of them underwent completion mastectomy after the initial conservative breast surgery and axillary clearance. While one patient underwent wider margin excision as positive margin for an invasive residual disease was found.

Two patients in the IMPC group had distant metastasis at the initial diagnosis, they had multiple metastatic lesions and received systemic treatment but one of them underwent palliative mastectomy.

Systemic chemotherapy was administered to 107 patients (77.5%) in the IMPC group and to 207 patients (41%) in the IDC group. Hormonal therapy was administered to all IMPC patients and 76% patients in the IDC group (Table 4 ).

The overall median follow-up duration was 21 months (range 6 – 88 months) with mean follow up duration = 29.8months.

Among the 138 IMPC patients, local recurrence developed in 3 cases, they developed a recurrence at 6,18 and 48 months postoperative. Distant metastasis developed in 5 patients in the form of bone, lung, hepatic and mediastinal lymph node metastasis.

The survival analysis indicated that IMPC patients had no significant difference in overall survival compared with IDC patients and no differences were noted in locoregional recurrence rate comparing IMPCs with IDCs (2.2% and 0.4% respectively). P value for local recurrence = 0.12 (yates corrected chi square).

Distant metastasis rate comparing IMPCs with IDCs was (3.7% and 5.4% respectively). P value for distant metastasis = 0.53 (Table 5 ).

Comparison of OS between IDC and micropapillary cases (Matched by propensity score matching -PSM).

Case Processing Summary

Overall Comparisons

• Test of equality of survival distributions for the different levels type

Disease free survival

• Test of equality of survival distributions for the different levels of type

IMPC is a highly invasive type of breast cancer. Hashmi A.A. et al. [ 13 ] found that the incidence of IMPC is very low accounting for 0.76–3.8% of breast carcinomas.

Shi WB et al.; [ 7 ] in a study comparing 188 IMPC cases and 1,289 invasive ductal carcinoma (IDC) cases from China showed that IMPC can occur either alone or mixed with other histological types, such as ductal carcinoma in situ, mucinous carcinoma and IDC. Furthermore, the majority of patients had mixed IMPC.

Fakhry et al. [ 14 ] reported that 64.7% of IMPC patients were pure type. In our study, we found that the pure form of IMPC was the commonest type and presented in 90 patients (65%) and 47 cases (34%) were mixed type which was similar to that reported by Nassar et al. [ 15 ], and Guo et al. [ 16 ] in their studies.

In our study, the commonest finding of IMPC on breast mammography was an irregular shaped mass with a non-circumscribed spiculated margin. While, the commonest sonographic finding of IMPC was hypoechoic mass with irregular shapes and spiculated margins.

These findings were similar to the results demonstrated by Jones et al., [ 17 ] which found that the commonest morphologic finding of IMPC was an irregular high-density lesion (50% of patients) with spiculated margin (42% of patients). However, Günhan-Bilgen et al. [ 18 ] reported that an ovoid or round lesion was found in 53.8% of patients.

Alsharif et al., [ 19 ] reported that the commonest sonographic finding of IMPC was hypoechoic masse (39/41, 95%) with irregular shape (30/41, 73.2%) and angular or spiculated margin (26/41, 63.4%).

In our study, MRI was done for 5 cases (3.6%), while CESM was performed for 18 cases (13%) of the IMPC group, the commonest presentation of IMPC in contrast study was irregular shaped enhanced lesion in 21 cases and non-mass enhancement was presented in 5 cases.

Nangogn et al. [ 20 ] and yoon et al. [ 8 ] recorded that the commonest finding of IMPCs in MRI was spiculated irregular mass with early rapid initial heterogenous enhancement, indicating that the MRI findings correlated with the invasiveness of IMPC.

Fakhry et al. [ 14 ] conducted a study on 68 cases, out of which 17 cases underwent CEM. In all of these cases, the masses showed pathological enhancement, which was either in the form of mass enhancement (12/17 patients, 70.6%) or non-mass enhancement (4/17 patients, 23.5%). The majority of the enhanced masses were irregular in shape (11/12 patients, 91.7%).

All patients underwent axillary sonography and 77 patients (55.8%) of the IMPC group exhibited pathological lymph nodes; this percentage was similar to that recorded by Nangong et al. [ 20 ] which was 54.8% and lower than that recorded by Jones et al. [ 17 ] but higher than that of Günhan et al. [ 18 ] which were 67% and 38% respectively.

Günhan et al. [ 18 ] reported microcalcification in about 66.7% of the cases. In our study, associated microcalcifications were found in 49 patients (35.5%) of the IMPC group. Yun et al. [ 21 ] and Adrada et al. [ 22 ] showed a fine pleomorphic appearance (66.7% and 68%).

Hao et al. [ 23 ] compared the rate of tumors larger than 5cm, reporting 3% in IDC and 4.3% in IMPC. In our study, the rate of tumors larger than 5cm, was reported 7.4% in the IDC patients and 9.5% in the IMPC patients.

Yu et al., et al. [ 24 ] documented in a study comparing 72 cases of IMPC and 144 cases of IDC of the breast that IMPC had a higher nuclear grade than IDC (52.8% vs. 37.5% respectively). In our study, IMPC had a higher nuclear grade than the IDC group (25.1% and 15.2% respectively).

Verras GI et al.; [ 9 ] demonstrated that IMPC was an aggressive breast cancer subtype with a great tendency to lymphovascular invasion and lymph node metastasis. In our study, the incidence of LVI in the IMPC patients was 88.3% in comparison to 47.0% in the IDC patients ( p  < 0.001). Tang et al., [ 25 ] also reported that lymphovascular involvement was more common among the IIMPC group than IDC group, with a percentage of 14.7% compared to only 0.1% in the IDC group.

Also, Shi et al. [ 7 ] reported that LVI was detected in 74.5% of cases. Furthermore, the frequency of LVI was found to be greater in IMPC cases when compared to IDC cases. Jones et al., [ 17 ] recorded angiolymphatic invasion in 69% of cases.

Hashmi et al. [ 13 ] reported in his comparative study that nodal involvement was present in 49.5% of IDC patients and N3 stage was only 15.6% in IDC patients compared to 33% in IMPC patients. In our study, the percentage of lymph node involvement of IMPC and IDC patients were 68.8% and 56% respectively with N3 stage reported in 12.4% of IMPC patients.

Guan et al. [ 26 ], Lewis et al., [ 27 ], Pettinato et al., [ 28 ] and De La Cruz et al., [ 29 ] recorded a higher percentage of lymph node metastasis in IMPC patients, reaching 90%, 92.9%,55.2% and 60.9% respectively.

The management of IMPC remains controversial, particularly among breast surgeons. Modified radical mastectomy was the preferred surgical procedure for the majority of IMPC case reports, as found in a study conducted by Yu et al., [ 24 ] where 99% of IMPC cases underwent modified radical mastectomy. Fakhry et al. [ 14 ] reported that 76.5% of the patients underwent modified radical mastectomy. In our study, 49.3% of IMPC patients received modified radical mastectomy.

IMPC patients were also prone to accept BCS rather than mastectomy in the previous series conducted by Lewis GD,et al. [ 27 ] and Vingiani, A. et al. [ 30 ]. However, the precise prognosis value of BCS for patients with IMPC remained unknowable. In our study, IMPC had a lower rate of breast conserving surgery (26% vs.37.8%) compared with IDC.

IMPC was characterized by a high incidence of ER and PR positivity. Our study recorded a high percentage of ER (97.8%) and PR (98.6%) expression. Our findings are similar to those found by Walsh et al., [ 31 ] who reported ER and PR expression of 90% and 70%, respectively. Zekioglu et al. [ 32 ] demonstrated a rate of ER and PR expression of 68% and 61%respectively.

In this study, we reported a relatively lower percentage of HER-2 positivity (4.3%). Also, Nangong et al. [ 20 ] showed HER 2 overexpression in 26.4% of cases.

However, Cui et al. [ 33 ] reported a much higher incidence of HER 2 positivity and Perron et al., [ 34 ] reported that 65% of IMPCs were HER-2 positive.

Chen, A et al. [ 35 ] reported that that the percentage of radiation therapy for IMPC patients was similar to those seen in IDC patients and demonstrates a similar benefit of radiation treatment in both groups. In our study,77.5% patients received radiotherapy in IMPC group in compared to 59.4% patients in IDC group.

Shi et al. [ 7 ] found that patients with IMPC had worse recurrence-free survival (RFS) and overall survival (OS) rates as compared to those with IDC. However, because IMPC is relatively rare, most studies had reported on small sample sizes with limited follow-ups.

Yu et al., [ 24 ] conducted a comparison between IMPC and IDC patients, and the results showed that the IMPC group had a greater tendency for LRR compared to the IDC group ( P  = 0.03), but the distant metastasis rate ( P  = 0.52) and OS rate ( P  = 0.67) of the IMPC showed no statistical differences from the IDC group.

Nevertheless, several recent studies documented that IMPC had better or similar prognosis in comparison to IDC.

Hao et al. [ 23 ] and Vingiani et al. [ 30 ] documented that there was no statistically significant difference in OS and disease-free survival between IMPC patients and IDC patients which was similar to our results. locoregional recurrence rate comparing IMPCs with IDCs was (2.2% and 0.4% respectively). P value for local recurrence = 0.12 (yates corrected chi square). Distant metastasis rate comparing IMPCs with IDCs was (3.7% and 5.4% respectively). P value for distant metastasis = 0.53.

Chen H et al. [ 36 ], compared the overall survival in patient groups with similar nodal involvement and found that IMPC group had better breast cancer–specific survival and overall survival than IDC group.

Availability of data and materials

No datasets were generated or analysed during the current study.

Abbreviations

Invasive micropapillary carcinoma

Invasive duct carcinoma

Modified radical mastectomy

Conserving breast surgery

Estrogen receptor

Progesterone receptor

Lymphovascular invasion

Contrast enhanced spectral mammography

Overall survival

Lakhani SR. International Agency for Research on Cancer Press and World Health Organization. WHO Classification of Tumours of the Breast. Lyon: International Agency for Research on Cancer Press; 2012.

Wu Y, Zhang N, Yang Q. The prognosis of invasive micropapillary carcinoma compared with invasive ductal carcinoma in the breast: a meta-analysis. BMC Cancer. 2017;17:839.

Article   PubMed   PubMed Central   Google Scholar  

Fisher ER, Palekar AS, et al. Pathologic findings from the national surgical adjuvant breast project (protocol no. 4). Vi. Invasive papillary cancer. Am J Clin Pathol. 1980;73:313–22.

Article   PubMed   CAS   Google Scholar  

Siriaunkgul S, Tavassoli FA. Invasive micropapillary carcinoma of the breast. Mod Pathol. 1993;6:660–2.

PubMed   CAS   Google Scholar  

Hanby AM, walker C, Tavassoli FA, Devilee P. Pathology and Genetics: Tumours of the Breast and Female Genital Organs. WHO Classification of Tumours series. Breast Cancer Res. Lyon: IARC Press; 2004;4(6):133. https://doi.org/10.1186/bcr788 .

Yang YL, Liu BB, Zhang X, Fu L. Invasive micropapillary carcinoma of the breast: an update. Arch Pathol Lab Med. 2016;140(8):799–805. https://doi.org/10.5858/arpa.2016-0040-RA .

Shi WB, Yang LJ, et al. Clinico-pathological features and prognosis of invasive micropapillary carcinoma compared to invasive ductal carcinoma: a population-based study from china. PLoS ONE. 2014;9:e101390.

Yoon GY, Cha JH, Kim HH, Shin HJ, Chae EY, Choi WJ. Comparison of invasive micropapillary and invasive ductal carcinoma of the breast: a matched cohort study. Acta Radiol. 2019;60(11):1405–13.

Article   PubMed   Google Scholar  

Verras GI, et al. Micropapillary breast carcinoma: from molecular pathogenesis to prognosis. Breast Cancer (Dove Med Press). 2022;12(14):41–61.

Google Scholar  

Allison KH, Hammond MEH, Dowsett M, McKernin SE, Carey LA, Fitzgibbons PL, et al. Estrogen and Progesterone Receptor Testing in Breast Cancer: ASCO/CAP Guideline Update. J Clin Oncol. 2020;38(12):1346–66. https://doi.org/10.1200/JCO.19.02309 . 

Fitzgibbons PL, Dillon DA, Alsabeh R, Berman MA, Hayes DF, Hicks DG, Hughes KS, Nofech-Mozes S. Template for reporting results of biomarker testing of specimens from patients with carcinoma of the breast. Arch Pathol Lab Med. 2014;138(5):595–601.

Ahn S, Woo JW, Lee K, Park SY. HER2 status in breast cancer: changes in guidelines and complicating factors for interpretation. J PatholTransl Med. 2020;54(1):34.

Hashmi AA, et al. Clinicopathologic features of invasive metaplastic and micropapillary breast carcinoma: comparison with invasive ductal carcinoma of breast. BMC Res Notes. 2018;11:1–7.

Article   Google Scholar  

Fakhry S, et al. Radiological characteristics of invasive micropapillary carcinoma of the breast. Clin Radiol. 2024;79(1):e34–40.

Nassar H, Wallis T, Andea A, et al. Clinicopathologic analysis of invasive micropapillary differentiation in breast carcinoma. Mod Pathol. 2001;14:836e41.

Guo X, Chen L, Lang R, et al. Invasive micropapillary carcinoma of the breast: association of pathologic features with lymph node metastasis. Am J Clin Pathol. 2006;126:740e6.

Jones KN, Guimaraes LS, Reynolds CA, Ghosh K, Degnim AC, Glazebrook KN. Invasive micropapillary carcinoma of the breast: imaging features with clinical and pathologic correlation. AJR Am J Roentgenol. 2013;200:689–95.

Günhan-Bilgen I, et al. Invasive micropapillary carcinoma of the breast: clinical, mammographic, and sonographic findings with histopathologic correlation. AJR Am J Roentgenol. 2002;179:927–31.

Alsharif S, et al. Mammographic, sonographic and MR imaging features of invasive micropapillary breast cancer. Eur J Radiol. 2014;83(8):1375–80.

Nangong J, Cheng Z, Yu L, Zheng X, Ding G. Invasive micropapillary breast carcinoma: a retrospective study on the clinical imaging features and pathologic findings. Front Surg. 2022;23(9):1011773.

Yun SU, Choi BB, Shu KS, et al. Imaging findings of invasive micropapillary carcinoma of the breast. J Breast Cancer. 2012;15:57e64.

Adrada B, Arribas E, Gilcrease M, et al. Invasive micropapillary carcinoma of the breast: mammographic, sonographic, and MRI features. AJR Am J Roentgenol. 2009;193:58e63.

Hao S, Zhao Y, Peng J, et al. Invasive micropapillary carcinoma of the breast had no difference in prognosis compared with invasive ductal carcinoma: a propensity-matched analysis. Sci Rep. 2019;9:1–8.

Yu JI, Choi DH, Huh SJ, et al. Differences in prognostic factors and failure patterns between invasive micropapillary carcinoma and carcinoma with micropapillary component versus invasive ductal carcinoma of the breast: retrospective multicenter case-control study (KROG 13–06). Clin Breast Cancer. 2015;15:353–361.e2.

Tang S-L, Yang J-Q, Du Z-G, et al. Clinicopathologic study of invasive micropapillary carcinoma of the breast. Oncotarget. 2017;8:42455–65.

Guan X, Xu G, Shi A, et al. Comparison of clinicopathological characteristics and prognosis among patients with pure invasive ductal carcinoma, invasive ductal carcinoma coexisted with invasive micropapillary carcinoma, and invasive ductal carcinoma coexisted with ductal carcinoma. Medicine (Baltimore). 2020;99:e23487.

Lewis GD, Xing Y, Haque W, et al. The impact of molecular status on survival outcomes for invasive micropapillary carcinoma of the breast. Breast J. 2019;25:1171e6.

Pettinato G, Pambuccian SE, Di Prisco B, et al. Fine needle aspiration cytology of invasive micropapillary (pseudopapillary) carcinoma of the breast: report of 11 cases with clinicopathologic findings. Acta Cytol. 2002;46:1088e94.

De La Cruz C, et al. Invasive micropapillary carcinoma of the breast: clinicopathological and immunohistochemical study. Pathol Int. 2004;54:90–6.

Vingiani A, et al. The clinical relevance of micropapillary carcinoma of the breast: a case–control study. Histopathology. 2013;63:217–24.

Walsh MM, Bleiweiss IJ. Invasive micropapillary carcinoma of the breast: eighty cases of an underrecognized entity. Hum Pathol. 2001;32:583–9.

Zekioglu O, et al. Invasive micropapillary carcinoma of the breast: high incidence of lymph node metastasis with extranodal extension and its immunohistochemical profile compared with invasive ductal carcinoma. Histopathology. 2004;44:18–23.

Cui ZQ, et al. Clinicopathological features of invasive micropapillary carcinoma of the breast. Oncol Lett. 2015;9:1163–6.

Perron M, Wen HY, Hanna MG, Brogi E, Ross DS. HER2 Immunohistochemistry in invasive micropapillary breast carcinoma: complete assessment of an incomplete pattern. Arch Pathol Lab Med. 2021;145:979–87.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Chen A, Paulino A, Schwartz M, et al. Population-based comparison of prognostic factors in invasive micropapillary and invasive ductal carcinoma of the breast. Br J Cancer. 2014;111:619–22.

Chen H, Wu K, Wang M, Wang F, Zhang M, Zhang P. Invasive micropapillary carcinoma of the breast has a better long-term survival than invasive ductal carcinoma of the breast in spite of its aggressive clinical presentations: a comparison based on large population database and case–control analysis. Cancer Med. 2017;6:2775–86.

Download references

Acknowledgements

Not applicable.

Open access funding provided by The Science, Technology & Innovation Funding Authority (STDF) in cooperation with The Egyptian Knowledge Bank (EKB).

Author information

Mohamed Fathy Abdelfattah Abdelrahman Elithy

Present address: Department of Surgical Oncology, Faculty of Medicine, Al Azhar University, Cairo, Egypt

Mahmoud Hassaan

Present address: Departement of Surgical Oncology, National Cancer Institute, Cairo University, Giza, Egypt

Authors and Affiliations

Department of General Surgery, Faculty of Medicine, Ain Shams University, Cairo, Egypt

Yasmine Hany Abdel Moamen Elzohery

Department of Radiodiagnosis, NCI, Cairo University, Giza, Egypt

Amira H. Radwan & Sherihan W. Y. Gareer

Department of Pathology, National Cancer Institute, Cairo University, Giza, Egypt

Mona M. Mamdouh

Department of Epidemiology and Preventive Medicine, National Liver Institute, Menoufia, Egypt

Baheya Center for Early Detection and Treatment of Breast Cancer, Giza, Egypt

Yasmine Hany Abdel Moamen Elzohery, Amira H. Radwan, Sherihan W. Y. Gareer, Mona M. Mamdouh, Inas Moaz, Abdelrahman Mohammad Khalifa, Osama Abdel Mohen, Mohamed Fathy Abdelfattah Abdelrahman Elithy & Mahmoud Hassaan

You can also search for this author in PubMed   Google Scholar

Contributions

Mohamed fathy participated in the sequence alignment and Yasmine hany drafted the manuscript. Mahmoud Hassan participated in the design of the study. Inas Moaz and Abdelrahman Mohammad performed the statistical analysis. Amira H. Radwan and Sherihan WY Gareer conceived the study. Mona M Mamdouh and Osama abdel Mohen participated in its design and coordination and helped to draft the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Yasmine Hany Abdel Moamen Elzohery .

Ethics declarations

Ethics approval and consent to participate.

Available all patients provided informed consent for publication. All patients provided signed written informed consent.

Consent for publication

Ethical approval is obtained from Baheya center for early detection and treatment of breast cancer and National research center ethics committee. Baheya IRB protocol number: 202305150022.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Elzohery, Y.H.A.M., Radwan, A.H., Gareer, S.W.Y. et al. Micropapillary breast carcinoma in comparison with invasive duct carcinoma. Does it have an aggressive clinical presentation and an unfavorable prognosis?. BMC Cancer 24 , 992 (2024). https://doi.org/10.1186/s12885-024-12673-0

Download citation

Received : 05 April 2024

Accepted : 23 July 2024

Published : 12 August 2024

DOI : https://doi.org/10.1186/s12885-024-12673-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

ISSN: 1471-2407

• Submission enquiries: [email protected]
• General enquiries: [email protected]