Which terminology system would the nurse use to enter nursing diagnoses intervention and outcomes in electronic health records?

J Am Med Inform Assoc. 2019 Nov; 26(11): 1401–1411.

Abstract

Objective

The study sought to present the findings of a systematic review of studies involving secondary analyses of data coded with standardized nursing terminologies (SNTs) retrieved from electronic health records (EHRs).

Materials and Methods

We identified studies that performed secondary analysis of SNT-coded nursing EHR data from PubMed, CINAHL, and Google Scholar. We screened 2570 unique records and identified 44 articles of interest. We extracted research questions, nursing terminologies, sample characteristics, variables, and statistical techniques used from these articles. An adapted STROBE (Strengthening The Reporting of OBservational Studies in Epidemiology) Statement checklist for observational studies was used for reproducibility assessment.

Results

Forty-four articles were identified. Their study foci were grouped into 3 categories: (1) potential uses of SNT-coded nursing data or challenges associated with this type of data (feasibility of standardizing nursing data), (2) analysis of SNT-coded nursing data to describe the characteristics of nursing care (characterization of nursing care), and (3) analysis of SNT-coded nursing data to understand the impact or effectiveness of nursing care (impact of nursing care). The analytical techniques varied including bivariate analysis, data mining, and predictive modeling.

Discussion

SNT-coded nursing data extracted from EHRs is useful in characterizing nursing practice and offers the potential for demonstrating its impact on patient outcomes.

Conclusions

Our study provides evidence of the value of SNT-coded nursing data in EHRs. Future studies are needed to identify additional useful methods of analyzing SNT-coded nursing data and to combine nursing data with other data elements in EHRs to fully characterize the patient’s health care experience.

Keywords: nursing informatics, electronic health records, standardized nursing terminology

INTRODUCTION

In the hospital setting, nurses are typically the main frontline providers of care. In their role, nurses continuously identify patients’ issues and subsequently plan and implement care to achieve desired patient outcomes. In 2016, there were 3 million registered nurses working in the United States, of whom 61% worked in hospitals,1 compared with only 25% of 312 500 pharmacists and 12% of 854 698 physicians who practiced full-time in hospitals.2–4 Despite these statistics, it has been difficult to effectively evaluate the impact of nursing care on patient outcomes. The rapid adoption of electronic health record (EHR) systems provides a growing opportunity to expand our knowledge about nursing practices using nursing data in EHRs. The documentation of nursing care in EHRs with standardized terms provides a means for producing consistent data about nursing needed to share, compare, and merge with other data across systems.5–7 In this article, we present a systematic review of studies characterizing nursing care through the analysis of standardized nursing data retrieved from EHRs. The review also provides a foundation for future paths of inquiries using nursing and other data retrievable from EHRs.

Nursing data include elements of nursing services (eg, personnel, equipment), patient demographics, progress notes, assessment data, and care plans.8,9 In particular, care plan data are a rich source of information that can improve our understanding of the nursing care provided to patients. Care plan information represents nurses’ clinical reasoning and includes patient problems, target outcomes, and the planned and implemented nursing interventions.10

In the mid-1970s, nursing researchers began developing standardized nursing terminologies (SNTs) to help bedside nurses document diagnoses, as well as the care they provided to patients and families, which is different from medical diagnoses.11 SNTs are controlled vocabularies that contain standardized terms to represent nursing diagnoses, interventions, and outcomes.7 Over time, SNTs evolved from alphabetical lists to conceptual systems that guide the decision-making process of nursing care at the individual, caregiver, group, family, and community levels.12 The American Nurses Association recognizes the following 7 SNTs12,13:

• Clinical Care Classification (CCC)14

• International Classification for Nursing Practice (ICNP)15

• Omaha System16

• Perioperative Nursing Data Set (PNDS)17

• NANDA-International (NANDA-I) (diagnoses)18

• Nursing Outcomes Classification (NOC) (outcomes)19

• Nursing Interventions Classification (NIC) (interventions)20

The first 4 SNTs (ie, CCC, ICNP, Omaha System, PNDS) contain terms for nursing diagnoses, interventions, and outcomes. The NANDA-I, NIC, and NOC, each representing 1 element (diagnoses, interventions, and outcomes, respectively), are often used together and referred to as a set (NNN).

To date, there are few reviews of secondary analyses of nursing data coded in SNTs. Tastan et al21 focused on determining how SNT usage has evolved over the years, mainly describing the study focus and frequency of SNTs in publications. More recently, Kim et al22 evaluated the coverage of the bioCADDIE (biomedical and healthCAre Data Discovery Index Ecosystem) metadata specification in representing nursing data from published studies. Our systematic review fills an important knowledge gap: we explored the analysis of SNT-coded nursing data in research studies and the potential of using these data to show the impact of nursing care.

The objective of this review was to understand how SNT-coded nursing data, retrieved from EHRs, have been utilized in research studies to answer important questions about nursing practice.

MATERIALS AND METHODS

We performed a systematic review of studies that conducted secondary data analysis on SNT-coded nursing data extracted from EHRs. Our protocol23 was registered on PROSPERO (available at https://www.crd.york.ac.uk/prospero/display_record.php? RecordID=99830), and developed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Statement.24 The different phases of our systematic review are illustrated in Figure 1.

Flow diagram illustrating the different phases of the study selection process. EHR: electronic health record; SNT: standardized nursing terminology.

Data sources and search strategy

We built on previous systematic review on SNT usage in studies from the 1960s to 2011,21 which reported 19 studies focusing on secondary uses of SNT-coded nursing documentation. We then proceeded to search on PubMed and CINAHL for studies published from 2011 to May 27, 2018, in the English language, and with abstracts available. The search of the databases included the following themes: nursing, electronic health records, and the American Nurses Association–recognized SNTs. The database EMBASE was not included because we did not have access to it through our institution’s library at the time of our search.

With the help of the sponsoring institution’s nursing librarian, along with the authors of this review, a search strategy was developed including a comprehensive set of text words and MeSH (Medical Subject Headings) terms for electronic health records and SNTs. Details on the search strategies are presented in Table 1.

Table 1.

Search strategies for PubMed and CINAHL databases

We downloaded potential publications into a reference management program (EndNote X7 V.17.8.0.13453, Thompson Reuters, Philadelphia, PA/USA), in which we identified and removed duplicates.

Abstract and full-text screening

We retained 2551 articles after duplicates were removed. Three reviewers (T.M., N.D., T.C.)—specifically, 2 doctoral students in nursing informatics (T.M. and N.D.) and 1 faculty member (T.C.) with expertise in nursing terminologies—participated in screening abstracts from all articles. At least 2 of these reviewers independently screened each abstract. To ensure consistency, a guide was created to assist in a systematic process for abstracts screening. The guide directed reviewers to include a record if its abstract described (1) any of the SNTs noted previously or words such as nursing diagnoses or nursing interventions, (2) if the analyzed SNT-coded nursing data were retrieved from EHRs or electronic medical records, and (3) if the nursing data were documented at the point of care, as part of their care routine. A record was excluded if it was clear in the abstract that nursing data were collected in a research setting, not part of patient’s standard of care. All discrepancies were discussed until consensus was reached.

Four reviewers (T.M., T.C., M.S., and K.D.L.) then participated in full-text reviews of 87 articles using the same guide described previously. We further excluded articles that are letters, commentaries, news reports, and literature reviews. At least 2 of the reviewers independently reviewed each full-text article. Discrepancies were resolved by discussion and consultation with 2 other faculty members, a specialist in nursing informatics (G.K.) and a statistician (Y.Y.). Figure 1 lists reasons for exclusions after full-text review.

Google Scholar search for additional articles

Secondary analysis of nursing data is often an interdisciplinary effort. Nursing researchers often collaborate with experts outside of biomedical domain. Thus, relevant studies may be reported in journals of other fields (eg, computer science journals) and may not be indexed in PubMed or CINAHL. We searched on Google Scholar for authors (n = 134) whose articles were included in our study after full-text review. The same screening process described above was used to locate and include additional relevant articles.

Data extraction and synthesis

Data were extracted into individual tables for each study included in the qualitative synthesis process. The following items were extracted:

• Research questions answered by the analysis of SNT-coded nursing data

• Specific SNTs used to code the data

• Sample characteristics (setting, number of records, type of patient)

• Variables (descriptive, predictor, outcome, covariate)

• Statistical techniques applied

Three reviewers (T.M., T.C., and M.S.) extracted the data for included articles. For each article, 2 reviewers independently extracted the data. T.M. and M.S. reached an agreement of 93% for the data extracted. The interrater reliability for the second pair of reviewers (T.M. and T.C.) was 84%.

Reproducibility assessment

We used an adapted version of the STROBE (Strengthening The Reporting of OBservational Studies in Epidemiology) Statement checklist for observational studies25 to assess the reproducibility of studies included in our review. Pairs of reviewers (T.M. and M.S.; T.M. and T.C.) independently evaluated the reproducibility of these studies, given their descriptions of (1) setting, (2) participants, (3) variables, (4) data sources or measurement, and (5) statistical methods. The reviewers also evaluated if the variables were described with sufficient clarity to be collected in future studies, scoring them as 1 (not defined), 2 (partially defined), and 3 (well defined). Interrater reliability scores were an agreement of 83% for T.M. and M.S. and agreement of 89% for T.M. and T.C..

RESULTS

The initial and updated searches on the 2 databases resulted in 3240 records. A total of 2551 records remained after duplicates were deleted and were submitted to abstract screening. 2483 records were further removed because they were not related to nursing or to SNTs. Reasons for the initial high volume of publications matching our search keywords but outside the scope of nursing or SNTs could be attributed to our use of the acronyms for each SNT as a text word (eg, CCC). Many of these acronyms have meanings other than those for nursing terminologies. We also used the Boolean term OR to combine either nursing or SNTs to EHRs (and its variations), which resulted in a more thorough search but also more false hits.

Sixty-eight records from PubMed and CINAHL and 19 records from Tastan et al21 were included in full-text review, of which 40 met our inclusion criteria. Four additional records were identified through search on Google Scholar. After qualitative synthesis, 44 articles were included. In 1 article, 2 studies were reported. Thus, the findings below refer to a sample of 45 studies.

SNTs used

Amongst the 45 studies, we found that only 5 of the 7 SNTs were used. There was no study that analyzed nursing data coded with CCC or PNDS. Twenty-six (58%) studies analyzed nursing data coded with NANDA-I,26–28 NOC,29,30 NIC,31–33 a combination of these terminologies (eg, NIC and NOC),34–36 or the NNN set.37–51 Fourteen (31%) studies used Omaha System,52–65 while 4 (9%) used ICNP.66–68 One (2%) study compared nursing data coded with NANDA-I and ICNP from different EHRs.69

Study foci and variables

The specific study foci varied widely across studies. They can be grouped into the following areas: (1) feasibility of standardizing nursing data, (2) characterization of nursing care, and (3) impact of nursing care. Table 2 includes a description of the different research questions answered by SNT-coded nursing data in our sample of studies.

Table 2.

Summary of study focus and statistical methods of included studies grouped by publications on 1 dataset

Descriptive studies analyzed nursing diagnoses, interventions, and outcomes relevant to SNTs. For studies that explored associations, the majority of independent variables were these same 3 nursing care plan elements, along with other factors related to delivery of care. Dependent variables in those studies varied significantly. The following sections summarize the variables analyzed across studies to help contextualize their main focus.

Feasibility of standardizing nursing data

Nine of 45 (20%) studies focused on potential uses of SNT-coded nursing data or challenges associated with this type of data. Keenan et al42 reported benefits of using SNTs in EHRs to increase availability, validity, and reliability of data for research purposes and statistical data analyses. Johnson et al53 demonstrated differences between t test and Cohen’s d in describing changes in nursing outcomes ratings. Park et al67 analyzed nursing interventions provided to hospitalized patients compared with standard nursing care from guidelines. Farri et al52 and Melton et al58 analyzed nursing free-text entries to standardized terms to describe potential flaws in a terminology’s content coverage. Rivas et al45 compared standardized and nonstandardized nursing care plans for delivery of health promotion and prevention services. Park and Lee50 and Westra et al61 compared SNT-coded nursing data from different EHR vendors. Kim et al68 tested different nursing intervention terms as search strategy to identify and analyze the incidence of contrast-media hypersensitivity.

Characterization of nursing care

Sixteen (36%) studies of 45 described characteristics of nursing care. Ten of 16 (63%) studies described the most common nursing diagnoses, interventions, and outcomes for patients with heart failure48,67; older patients with pneumonia49; younger patients with dehydration35; obstetric patients51; low- and high-risk groups55; infants with low birth weight60; patients with cancer69; patients with pressure ulcers66; and patients with pneumonia, total hip or knee replacement, or heart failure.29 In 6 of 16 (37%) studies, researchers explored interactions between nursing diagnoses, interventions, or outcomes with patient’s age and sex27,28,36; patient’s age and length of stay34; medical diagnoses and hospital characteristics26; and number of home-visits performed by nurses, patient’s age, and race.63

Impact of nursing care

Twenty of 45 (44%) studies measured the impact or effectiveness of nursing care. Table 3 lists patient outcomes and key predictors for the 20 studies.

Table 3.

Summary of predictors and outcomes for included studies that measured impact of nursing care

Garcia et al59 compared nursing outcome ratings given at admission and discharge to measure effectiveness of nursing care. Yao et al39 compared pain-related nursing diagnoses and outcome ratings across different hospitals to investigate the quality of current pain management practices. Shever et al32 associated number of times a day the intervention “surveillance” was documented with total hospital costs for older patients at risk of falling to measure the cost of delivering this nursing intervention. On the other hand, Jenkins and Welton30 multiplied the sum of nursing outcomes ratings per shift by actual nursing wages and patient’s length of stay to estimate nursing care costs.

Olsen et al65 explored association of nursing outcome ratings with patient’s age, body mass index, nursing diagnoses, and nursing interventions. Monsen et al54 investigated associations among nurses, nursing interventions, patient characteristics, and nursing outcomes ratings used to determine patients’ health literacy. Meanwhile, Gao et al62 translated specific nursing diagnoses into social and behavioral determinants of health and frailty concepts and determined association between those and nursing outcomes ratings.

Monsen et al64 used similar approach to assess the frequency of nursing diagnoses and used them as a measure of maternal risk index. Monsen et al56 tested different data management approaches to identify nursing interventions groups associated with patient’s hospitalization in homecare.

Of 20 studies, 7 (35%) focused on establishing associations among a broader number of nursing-related variables such as patient characteristics, support system factors, and nursing characteristics. For instance, Lodhi et al37,41,43 explored associations between patient’s age, nurse’s years of experience, length of stay, nursing diagnoses, and interventions, with outcomes for end-of-life patients. Almasalha et al40 focused on analyzing length of stay and nursing interventions to predict whether or not a patient would meet pain-related outcomes. Yao et al44 identified nursing diagnoses and interventions associated with changes in outcomes that could be used as an indicator of a shift from standard nursing care to palliative care. Lodhi et al38 created predictive models for hospital readmission using patient’s age, length of stay, nurse years of experience, time of admission and discharge, and pain outcomes ratings. Last, Stifter et al47 used nursing diagnoses, interventions, and outcomes to determine presence of pressure ulcers in hospitalized patients, and then examined their association with nurse staffing (eg, nurse years of experience, education, shift length, work pattern) and nurse continuity.

Other 5 (25%) studies of 20 merged nursing data to other parts of EHRs, namely unit characteristics, medical and pharmacy prescriptions,31,33 national outcome scales and psychiatric medical diagnoses,46 and medical diagnoses or treatment conditions, nutritional therapies, and patient’s service utilization.57

Statistical analysis

Fifteen of 45 (33%) studies used descriptive statistics only. Eleven of 15 (73%) studies exclusively analyzed nursing care plans.48–50,52,55,58,60,61,66–68 Three of 15 (20%) studies provided descriptive analyses of care plans along with other nursing data (eg, patient’s age and sex, nurse’s years of experience)27,34,42 and 1 (7%)51 analyzed nursing care plans merged with other parts of EHR (eg, medical diagnoses).

The remaining studies (30 of 45, 67%) applied descriptive statistics and other statistical methods specified in Table 2. At least 9 (30%) of 30 studies used only chi-square or t tests26,28,29,34–36,39,45,69; meanwhile, other studies from the 30 used a combination of these basic tests and more complex methods such as data mining,37,38,40,41,43,56,57 logistic regression,33,41,43,47,54,56,57 analysis of covariance,64 multiple regression,46,63,65 and generalized estimating equations.31,32

Among 30 studies that conducted association analyses, 6 (20%) exclusively analyzed nursing care plans.29,35,53,56,62,69 Eight (27%) of 30 merged these data with other parts of EHRs.26,31–33,45,46,57,67 The majority (16 of 30, 53%) of studies analyzed care plans and other nursing data.28,30,36–41,43,44,47,54,59,63–65

Table 2 shows 22 SNT-coded nursing datasets. Ten (45%) of 22 were retrieved from at least 2 different institutions/clinics. Some of the datasets were analyzed in more than 1 study (see Table 2). For example, the datasets referred as “B” (data from 1 family home visiting services) and “G” (data from 1 university hospital and 3 community hospitals) were analyzed in at least 5 different publications each since 2011. Earlier publications typically provided descriptive statistics on aspects of the broader dataset. Through the years, researchers began using more specific and complex statistical analysis (eg, generalized estimating equations, different types of regressions, and data mining procedures) to analyze smaller parts of the dataset.

It can also be noted in Table 2 that descriptive or basic statistics were used by the researchers that focused on exploring the feasibility of standardizing nursing data (study focus area 1) or characterizing nursing care (study focus area 2). For studies on the feasibility of standardizing nursing data (9 of 45, 20%), earlier publications (eg, the 2010s) employed descriptive analysis only and evolved to the use of chi-square and t tests across the years. Among studies that focused on the characterization of nursing care (16 of 45, 36%), publications from the early 2000s applied t tests and other basic statistics to smaller samples. From 2011 to 2017, the studies used descriptive statistics only, but to analyze larger amounts of readily available data.

More complex statistical analyses were employed by researchers interested in measuring the impact of nursing care (study focus area 3). Within this study focus area, publications from 2008 reported using generalized estimating equations and logistic regression and have evolved to the use of other advanced statistical methods, such as data mining (first study in 2011) and generalized linear mixed-effects models (since 2013), among others.

Reproducibility of publications included

All studies described sample sizes and listed statistical methods applied. Thirty-three (73%) of 45 studies defined their variables well and scored a 3, while 11 (25%) studies partially defined variables and received a 2.30,34,42,45,51,52,54,57,67,68 Only 1 (2%) study59 of 45 did not define variables clearly. Overall, our sample’s completeness in reporting information was rated “of good reproducibility.”

DISCUSSION

In this systematic review, we comprehensively identified and evaluated 45 studies to determine how SNT-coded nursing data retrieved from EHRs are being analyzed to answer important questions about nursing practice. We identified studies that analyzed nursing data documented with the terminologies NANDA-I, NOC, NIC, Omaha System, and ICNP. We observed that, although distinct, these terminologies structured the nursing care plans in the same manner across EHRs, which enabled easy retrieval and analyses of nursing diagnoses, interventions, and outcomes for different groups of patients. Additionally, the use of SNTs to code nursing care plans supported the analysis of data from multiple institutions (eg, 105 primary healthcare centers)28,36 and from distinct healthcare practices, such as 2 primary healthcare centers and a hospital.27

We synthesized foci of included studies into the following categories: (1) feasibility of standardizing nursing data, (2) characterization of nursing care, and (3) impact of nursing care. Within each focus, studies addressed different questions and analyzed distinct variables. For example, some studies that measured the impact of nursing care examined associations between nursing interventions and patient outcomes, such as patient’s health literacy, readmission, and presence of pressure ulcers, while others focused on developing predictive models. In 2014, a systematic review by Topaz et al70 on the use of the Omaha System described a shift from studies focusing on terminology development to the analysis of standardized nursing data to understand patient outcomes, through the years. Our findings support those of Topaz et al.70

The potential and versatility of SNT terms to characterize patients’ social, financial, and behavioral status were evidenced by the use of nursing diagnoses and interventions to operationalize patients’ income, social contact, and physical activity level, among others.62,63 The range of research questions studied using SNT-coded data demonstrates the potential that SNTs have in helping generate new knowledge about care offered by nurses. These studies show that incorporation of SNTs in electronic nursing documentation allows nursing care data to be available in a consistent format amenable to feasible analyses.71,72 The use of SNTs in EHR offers an advantage over allowing nurses to document with free-text. Two studies in our review52,58 compared free-text documentation with standard terms and reported that the use of SNTs reduces errors, such as typos and duplicated information. Nonetheless, the use of methods such as natural language processing offers the potential to augment standardized data through analyses of free-text notes entered by nurses into EHRs, further enhancing our understanding of the impact of nursing care.

Most studies used descriptive statistical analysis or basic tests, such as t tests, correlations, and chi-square tests. Our results indicate that these statistical methods have been more often used by researchers studying the feasibility of standardizing nursing data and characterization of nursing care. These statistical methods provide crucial results to understand a dataset and the population under study.73 We also noted that these basic statistical tests have been applied increasingly to larger datasets across the years. Our results suggest SNTs are more widely implemented to code point-of-care nursing documentation in EHRs within clinical settings than previously thought.74,75

The vast majority of studies conducted association analyses, employing methods such as multiple regression analysis, data clustering, and association mining techniques to select clinically important features and reduce dimensions of the dataset. Some also used decision trees, k-nearest neighbors, and support vector machines to identify hidden patterns in nursing data or to construct predictive models for patient outcomes.

We observed a research trend toward using more sophisticated statistical methods when analyzing SNT-coded nursing data, mainly to measure the impact of nursing care. Monsen et al64 showed how specific interventions delivered by public health nurses during home visits were associated with decreased risks for pregnant women and their families suffering from social disadvantages and poverty. Lodhi et al38 showed that SNT-coded nursing care plans are valuable in predicting hospital readmissions, which may help practitioners develop strategies to identify at-risk patients and potentially reduce healthcare costs in the future.

We also observed efforts from research teams to ensure full and valid representation of SNTs in EHRs through the analysis of the same datasets across the years. Specifically, 2 research teams stood out with nine37–44,47 and five53–55,63,64 publications each in the past 8 years. The dedication of these teams generated evidence of the feasibility of SNT-coded nursing data to characterize nursing care and enable analysis on the impact of nursing care on patient outcomes. Studies, however, are needed to examine the impact of nursing as component of total health care and these can be done through merging standardized nursing data with other types of data captured within EHRs.

Finally, our review showed that SNT-coded nursing data enables aggregation and comparison of nursing care plans providing the context of the care delivered.71 In addition, our systematic review provides evidence that SNT-coded nursing data are a viable foundation for systematically building knowledge to demonstrate nursing’s contribution to health care. Only 9 studies in our sample, however, included variables such as medical diagnoses, medical and pharmaceutical treatments. In future studies that utilize SNT-coded data, we recommend that other EHR data (eg, free text and non-nursing elements) be added to the analyses for the purpose of deepening our understanding of the impact of nursing on patient outcomes.

Limitations of this study

Our systematic review has a few limitations. Although some articles from other sources were identified through Google Scholar, our review ultimately included only 2 databases. Furthermore, the fact that our review focused on study methods rather than study results limits the overall conclusions on the impact of nursing care that we offer.

CONCLUSION

The need for controlled vocabularies in EHRs is well known. Our systematic review showed that SNTs are a foundation for creating sharable and comparable nursing data in larger datasets. This review also revealed the value of initially examining nursing variables separate from other sources. Research is needed, however, to expand and refine methods that will unleash the deep knowledge captured in standardized nursing data when it is merged with other data elements stored in EHRs. The adoption of SNTs in EHRs was shown to be stable, where future refinement may not be needed for a longer period of time to allow comparable findings. We, therefore, recommend wider adoption of SNTs in EHRs to document nursing care plans.

FUNDING

This work was supported by National Institutes of Health, National Institute for Nursing Research grant number R01 NR012949-01 (GMK) and National Institutes of Health, National Center for Advancing Translational Sciences grant number UL1TR001427 (JB).

AUTHOR CONTRIBUTIONS

TM and GK conceived of the main conceptual idea. TM, TC, MS, and KDL collected and performed the analysis of the data. YY and GK acted as experts throughout. TM wrote the article, with all authors contributing significant edits to all versions of the work. All authors approved the final version to be published.

ACKNOWLEDGEMENTS

The authors would like to thank Maggie Ansell for her expert literature search assistance and Nicolle Davis for her contribution during abstract screenings in the earlier stages of this systematic review. The authors also thank the National Council for Scientific and Technological Development (CNPq, Brazil) for their doctorate fellowship to the author Tamara G R Macieira.

Conflict of interest statement

None declared.

REFERENCES

3. U.S. Department of Health and Human Services, Health Resources and Services Administration, National Center for Health Workforce Analysis. The U.S. Health Workforce Chartbook Rockville, MD; 2013. https://bhw.hrsa.gov/sites/default/files/bhw/nchwa/chartbookpart1.pdf. Accessed August 9, 2018.

5. Baro E, Degoul S, Beuscart R, et al. Toward a literature-driven definition of big data in healthcare. Biomed Res Int 2015; 2015: 639021.. [PMC free article] [PubMed] [Google Scholar]

6. Van Poucke S, Thomeer M, Hadzic A.. 2015, big data in healthcare: for whom the bell tolls? Crit Care 2015; 191: 171–2. [PMC free article] [PubMed] [Google Scholar]

7. Bernhart-Just A, Lassen B, Schwendimann R.. Representing the nursing process with nursing terminologies in electronic medical record systems: a swiss approach. Comput Inform Nurs 2010; 286: 345–52. [PubMed] [Google Scholar]

8. Pruinelli L, Delaney CW, Garcia A, et al. Nursing management minimum data set: cost-effective tool to demonstrate the value of nurse staffing in the big data science era. Nurs Econ 2016; 342: 66–71. [PubMed] [Google Scholar]

9. Werley HH, Devine CE, Zorn CR, et al. The nursing minimum data set: abstraction tool for standardized, comparable, essential data. Am J Public Health 1991; 814: 421–6. [PMC free article] [PubMed] [Google Scholar]

10. Gallagher-Lepak S. Nursing diagnosis basics In: Herdman TH, Kamitsuru S, eds. NANDA International Nursing Diagnoses: Definitions & Classification, 2015–2017. Oxford, UK: Wiley Blackwell; 2014: 21–30. [Google Scholar]

11. Boyd AD, Dunn Lopez K, Lugaresi C, et al. Physician nurse care: a new use of UMLS to measure professional contribution: are we talking about the same patient a new graph matching algorithm? Int J Med Inform 2018; 113: 63–71. [PMC free article] [PubMed] [Google Scholar]

12. Lundberg C, Warren JJ, Brokel J, et al. Selecting a standardized terminology for the electronic health record that reveals the impact of nursing on patient care. Online J Nurs Inform 2008; 122: 1–20. [Google Scholar]

14. Saba V, ed. Clinical Care Classification (CCC) System Version 2.5. 2nd ed New York, NY: Springer; 2012. [Google Scholar]

16. Martin KS. The Omaha System: A Key to Practice, Documentation, and Information Management. 2nd ed Omaha, NE: Health Connections Press; 2005. [Google Scholar]

17. Beyea S, ed. Perioperative Nursing Data Set. 2nd ed Denver, CO: Association of periOperative Registered Nurses; 2002. [Google Scholar]

18. Herdman TH, Kamitsuru S, eds. NANDA International Nursing Diagnoses: Definitions & Classification, 2018–2020. 11th ed New York, NY: Thieme; 2018. [Google Scholar]

19. Moorhead S, Johnson M, Maas M, Swanson E, eds. Nursing Outcomes Classification (NOC). 6th ed St. Louis, MO: Elsevier; 2018. [Google Scholar]

20. Butcher H, Bulechek GM, Dochterman JMM, Wagner C, eds. Nursing Interventions Classification (NIC). 7th ed St. Louis, MO: Elsevier; 2018. [Google Scholar]

21. Tastan S, Linch GCF, Keenan GM, et al. Evidence for the existing American Nurses Association-recognized standardized nursing terminologies: a systematic review. Int J Nurs Stud 2014; 518: 1160–70. [PMC free article] [PubMed] [Google Scholar]

22. Kim H, Jang I, Quach J, et al. Explorative analyses of nursing research data. West J Nurs Res 2017; 391: 5–19. [PubMed] [Google Scholar]

24. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009; 1514: 264–70. [PubMed] [Google Scholar]

25. von Elm E, Altman DG, Egger M, et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Int J Surg 2014; 1212: 1495–9. [PubMed] [Google Scholar]

26. Frauenfelder F, van Achterberg T, Müller Staub M.. Nursing diagnoses related to psychiatric adult inpatient care. J Clin Nurs 2018; 27 (3–4): e463–e475. [PubMed] [Google Scholar]

27. González-Rodríguez R, Martelo-Baro MA, Bas-Sarmiento P.. Diagnostic labels of NANDA-I in Southern region of Spain. Rev Lat Am Enfermagem 2017; 8: e2911. [PMC free article] [PubMed] [Google Scholar]

28. Rodríguez-Álvaro M, Brito-Brito PR, García-Hernández AM, et al. The grieving nursing diagnoses in the primary healthcare setting. Int J Nurs Knowl 2019; 301: 34–42. [PubMed] [Google Scholar]

29. Scherb CA. Outcome research: making a difference in practice. Outcomes Manag 2002; 61: 22–6. [PubMed] [Google Scholar]

30. Jenkins P, Welton J.. Measuring direct nursing cost per patient in the acute care setting. J Nurs Adm 2014; 445: 257–62. [PubMed] [Google Scholar]

31. Titler MG, Shever LL, Kanak MF, et al. Factors associated with falls during hospitalization in an older population. Res Theory Nurs Pract 2011; 252: 127–52. [PubMed] [Google Scholar]

32. Shever LL, Titler MG, Kerr P, et al. The effect of high nursing surveillance on hospital cost. J Nurs Scholarsh 2008; 402: 161–9. [PubMed] [Google Scholar]

33. Shever LL. The impact of nursing surveillance on failure to rescue. Res Theory Nurs Pract 2011; 252: 107–26. [PubMed] [Google Scholar]

34. Tseng H, Moorhead S.. The use of standardized terminology to represent nursing knowledge: nursing interventions relevant to safety for patients with cancer. Stud Health Technol Inform 2014; 201: 298–303. [PubMed] [Google Scholar]

35. Scherb CA, Stevens MS, Busman C.. Outcomes related to dehydration in the pediatric population. J Pediatr Nurs 2007; 225: 376–82. [PubMed] [Google Scholar]

36. Rodríguez-Álvaro M, García-Hernández AM, Brito-Brito PR, et al. Bereavement care interventions and outcome criteria planned by community nurses in the Canary Islands. Enferm Clin 2018; 284: 217–78. [PubMed] [Google Scholar]

37. Lodhi MK, Cheema U, Stifter J, et al. Death anxiety in hospitalized end-of-life patients’ as captured from a structured electronic record: difference by patient and nurse characteristics. Res Gerontol Nurs 2014; 75: 224–34. [PMC free article] [PubMed] [Google Scholar]

38. Lodhi MK, Ansari R, Yao Y, et al. Predicting hospital re-admissions from nursing care data of hospitalized patients. Adv Data Min 2017; 2017: 181–93. [PMC free article] [PubMed] [Google Scholar]

39. Yao Y, Keenan G, Al-Masalha F, et al. Current state of pain care for hospitalized patients at end of life. Am J Hosp Palliat Care 2013; 302: 128–36. [PMC free article] [PubMed] [Google Scholar]

40. Almasalha F, Xu D, Keenan G, et al. Data mining nursing care plans of end-of-life patients: a study to improve healthcare decision making. Int J Nurs Knowl 2013; 241: 15–24. [PMC free article] [PubMed] [Google Scholar]

41. Lodhi MK, Ansari R, Yao Y, et al. Predictive modeling for comfortable death outcome using electronic health records. Proc IEEE Int Congr Big Data 2015; 2015: 409–15. [PMC free article] [PubMed] [Google Scholar]

42. Keenan G, Yakel E, Yao Y, et al. Maintaining a consistent big picture: meaningful use of a web-based POC EHR system. Int J Nurs Knowl 2012; 233: 119–33. [PMC free article] [PubMed] [Google Scholar]

43. Lodhi MK, Stifter J, Yao Y, et al. Predictive modeling for end-of-life pain outcome using electronic health records. Adv Data Min 2015; 9165: 56–68. [PMC free article] [PubMed] [Google Scholar]

44. Yao Y, Stifter J, Ezenwa MO, et al. Informarkers for transition to goals consistent with palliative care in dying patients. Palliat Support Care 2015; 135: 1427–34. [PMC free article] [PubMed] [Google Scholar]

45. Rivas PFJ, Martín-Iglesias S, del Cerro JLP, et al. Effectiveness of nursing process use in primary care. Int J Nurs Knowl 2016; 271: 43–8. [PubMed] [Google Scholar]

46. Escalada-Hernandez P, Munoz-Hermoso P, Gonzalez-Fraile E, et al. A retrospective study of nursing diagnoses, outcomes, and interventions for patients with mental disorders. Appl Nurs Res 2015; 282: 92–8. [PubMed] [Google Scholar]

47. Stifter J, Yao Y, Lodhi MK, et al. Nurse continuity and hospital-acquired pressure ulcers: a comparative analysis using an electronic health record “big data” set. Nurs Res 2015; 645: 361–71. [PMC free article] [PubMed] [Google Scholar]

48. Scherb CA, Head BJ, Maas ML, et al. Most frequent nursing diagnosis, nursing interventions and nursing sensitive patient outcomes of hospitalized older adults with heart failure: part 1. Int J Nurs Terminol Classif 2011; 221: 13–22. [PubMed] [Google Scholar]

49. Head BJ, Scherb CA, Reed D, et al. Nursing diagnoses, interventions, and patient outcomes for hospitalized older adults with pneumonia. Res Gerontol Nurs 2011; 42: 95–105. [PubMed] [Google Scholar]

50. Park H, Lee E.. Incorporating standardized nursing languages into an electronic nursing documentation system in Korea: a pilot study. Int J Nurs Terminol Knowledge 2015; 261: 35–42. [PubMed] [Google Scholar]

51. Yang MJ, Kim HY, Ko E, et al. Identification of nursing-outcome-intervention linkages for inpatients in the obstetrics department nursing unit in south Korea. Int J Nurs Terminol Knowl 2019; 301: 12–20. [PubMed] [Google Scholar]

52. Farri O, Monsen K, Westra B, et al. Analysis of free text with Omaha system targets in community-based care to inform practice and terminology development. Appl Clin Inform 2011; 23: 304–16. [PMC free article] [PubMed] [Google Scholar]

53. Johnson K, McMorris B, Raynor L, et al. What big size you have! using effect sizes to determine the impact of public health nursing interventions. Appl Clin Inform 2013; 43: 434–44. [PMC free article] [PubMed] [Google Scholar]

54. Monsen KA, Chatterjee SB, Timm JE, et al. Factors explaining variability in health literacy outcomes of public health nursing clients. Public Health Nurs 2015; 322: 94–100. [PubMed] [Google Scholar]

55. Monsen KA, Radosevich DM, Kerr MJ, et al. Public health nurses tailor interventions for families at risk. Public Health Nurs 2011; 282: 119–28. [PubMed] [Google Scholar]

56. Monsen KA, Westra BL, Oancea SC, et al. Linking home care interventions and hospitalization outcomes for frail and non-frail elderly patients. Res Nurs Health 2011; 342: 160–8. [PMC free article] [PubMed] [Google Scholar]

57. Westra BL, Savik K, Oancea C, et al. Predicting improvement in urinary and bowel incontinence for home health patients using electronic health record data. J Wound Ostomy Continence Nurs 2011; 381: 77–87. [PMC free article] [PubMed] [Google Scholar]

58. Melton GB, Westra BL, Raman N, et al. Informing standard development and understanding user needs with Omaha system signs and symptoms text entries in community-based care settings. AMIA Annu Symp Proc 2010; 2010: 512–6 [PMC free article] [PubMed] [Google Scholar]

59. Garcia C, McNaughton D, Radosevich DM, et al. Family home visiting outcomes for latina mothers with and without mental health problems. Public Health Nurs 2013; 305: 429–38. [PubMed] [Google Scholar]

60. Orlygsdottir B. Use of NIDSEC-Compliant CIS in community-based nursing-directed prenatal care to determine support of nursing minimum data set of objectives. Comput Inform Nurs 2007; 255: 283–93. [PubMed] [Google Scholar]

61. Westra BL, Oancea C, Savik K, et al. The feasibility of integrating the Omaha system data across home care agencies and vendors. Comput Inform Nurs 2010; 283: 162–71. [PMC free article] [PubMed] [Google Scholar]

62. Gao G, Maganti S, Ka M.. Older adults, frailty, and the social behavioral determinants of health. Big Data & Information Analytics 2017; 2 (3-4): 191–202. [Google Scholar]

63. Horning ML, Olsen JM, Lell S, et al. Description of public health nursing nutrition assessment and interventions for home-visited women. Public Health Nurs 2018; 344: 317–26. [PubMed] [Google Scholar]

64. Monsen KA, Peterson JJ, Mathiason MA, et al. Discovering public health nurse-specific family home visiting intervention patterns using visualization techniques. West J Nurs Res 2017; 391: 127–46. [PubMed] [Google Scholar]

65. Olsen JM, Horning ML, Thorson D, Monsen KA.. Relationships between public health nurse-delivered physical activity interventions and client physical activity behavior. Appl Nurs Res 2018; 40: 13–9. [PubMed] [Google Scholar]

66. Park H, Cho I, Chung E.. Exploring use of a clinical data repository containing international classification for nursing practice-based nursing practice data. Comput Inform Nurs 2011; 297: 419–26. [PubMed] [Google Scholar]

67. Park HA, Cho I, Ahn H.. Use of narrative nursing records for nursing research. NI 2012 (2012) 2012; 2012: 316–21. [PMC free article] [PubMed] [Google Scholar]

68. Kim M-H, Park C-H, Kim D-I, et al. Surveillance of contrast-media-induced hypersensitivity reactions using signals from an electronic medical recording system. Ann Allergy Asthma Immunol 2012; 1083: 167–71. [PubMed] [Google Scholar]

69. Rabelo-Silva ER, Cavalcanti ACD, Caldas M, et al. Advanced nursing process quality: comparing the international classification for nursing practice (ICNP) with the NANDA-international (NANDA-I) and nursing interventions classification (NIC). J Clin Nurs 2017; 26 (3–4): 379–87. [PubMed] [Google Scholar]

70. Topaz M, Golfenshtein N, Bowles KH.. The Omaha System: a systematic review of the recent literature. J Am Med Inform Assoc 2014; 211: 163–70. [PMC free article] [PubMed] [Google Scholar]

71. Cimino JJ. Desiderata for controlled medical vocabularies in the twenty-first century. Methods Inf Med 1998; 37 (4–5): 394–403. [PMC free article] [PubMed] [Google Scholar]

72. Westra BL, Latimer GE, Matney SA, et al. A national action plan for sharable and comparable nursing data to support practice and translational research for transforming health care. J Am Med Inform Assoc 2015; 223: 600–7. [PubMed] [Google Scholar]

73. Khokhar A, Lodhi MK, Yao Y, et al. Framework for mining and analysis of standardized nursing care plan data. West J Nurs Res 2017; 391: 20–41. [PMC free article] [PubMed] [Google Scholar]

74. Strudwick G, Hardiker NR.. Understanding the use of standardized nursing terminology and classification systems in published research: a case study using the international classification of nursing practice. Int J Med Inform 2016; 94: 214–21. [PubMed] [Google Scholar]

75. Törnvall E, Jansson I.. Preliminary evidence for the usefulness of standardized nursing terminologies in different fields of application: a literature review. Int J Nurs Knowl 2017; 282: 119. [PubMed] [Google Scholar]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of Oxford University Press