Electronic health record–based disease surveillance systems: A systematic literature review on challenges and solutions

o2 Health Management and Economics Research Center, Department of Health Information Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran

Hossein Ansari

o3 Department of Epidemiology and Biostatistics, Zahedan University of Medical Sciences, Zahedan, Iran

o1 Department of Health Information Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran

o2 Health Management and Economics Research Center, Department of Health Information Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran

o3 Department of Epidemiology and Biostatistics, Zahedan University of Medical Sciences, Zahedan, Iran

Corresponding Author: Abbas Sheikhtaheri, PhD, Health Management and Economics Research Center, Department of Health Information Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran; ri.ca.smui@a.irehathkiehs

Copyright © The Author(s) 2020. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For permissions, please email: journals.permissions@oup.com

This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Associated Data

Abstract

Objective

Disease surveillance systems are expanding using electronic health records (EHRs). However, there are many challenges in this regard. In the present study, the solutions and challenges of implementing EHR-based disease surveillance systems (EHR-DS) have been reviewed.

Materials and Methods

We searched the related keywords in ProQuest, PubMed, Web of Science, Cochrane Library, Embase, and Scopus. Then, we assessed and selected articles using the inclusion and exclusion criteria and, finally, classified the identified solutions and challenges.

Results

Finally, 50 studies were included, and 52 unique solutions and 47 challenges were organized into 6 main themes (policy and regulatory, technical, management, standardization, financial, and data quality). The results indicate that due to the multifaceted nature of the challenges, the implementation of EHR-DS is not low cost and easy to implement and requires a variety of interventions. On the one hand, the most common challenges include the need to invest significant time and resources; the poor data quality in EHRs; difficulty in analyzing, cleaning, and accessing unstructured data; data privacy and security; and the lack of interoperability standards. On the other hand, the most common solutions are the use of natural language processing and machine learning algorithms for unstructured data; the use of appropriate technical solutions for data retrieval, extraction, identification, and visualization; the collaboration of health and clinical departments to access data; standardizing EHR content for public health; and using a unique health identifier for individuals.

Conclusions

EHR systems have an important role in modernizing disease surveillance systems. However, there are many problems and challenges facing the development and implementation of EHR-DS that need to be appropriately addressed.

INTRODUCTION

Background and Significance

The World Health Organization defines disease surveillance as systematic and continuous processes of collecting and analyzing data for public health purposes and timely dissemination of public health information to assess and respond to public health problems. 1 , 2 Important components of disease surveillance systems include the collection, analysis, and use of health data continuously. 3 Health authorities regularly use surveillance systems as an essential part of public and population health programs, at local, regional, and national levels. 4

With the development of electronic health records (EHRs), current disease surveillance systems have shifted toward the use of EHRs instead of traditional data collection methods. 5 , 6 EHRs can provide up-to-date, standard, and low-cost data for disease surveillance systems without producing duplicate or manipulated data. 7 In fact, data can be collected once and reused, instead of duplicate data being collected by different systems. 8 EHRs have the capacity for collecting data from the early stages of a disease, which increases the likelihood of timely data use. EHRs can also provide data on different subpopulations in different geographical areas and health conditions. 9 , 10 Accurate and longitudinal EHR data, with a large population coverage, have the potential to be used to identify and analyze new risk factors, targeted interventions, and outcomes at the individual and population levels. 11 This data makes EHRs a possible source of valuable information for traditional disease surveillance systems. 12–15

EHR-based disease surveillance systems (EHR-DS) are being developed in different countries. For example, the Canadian population health record, the U.S. EHR-based population health surveillance, and EHR-public health have been suggested to provide a better understanding of the health status of the population. 9 , 16 However, the scope of these systems varies considerably. 6 For example, in 2013, the Department of Health and Mental Hygiene (New York, NY) used EHR data to produce annual estimates of diabetes prevalence, body mass index, smoking, and some other population health indicators. 17

EHR-based disease surveillance systems are still in their infancy, and there has been little focus on the use of these systems. 6 , 9 These systems encounter relatively similar challenges when collecting data from EHRs, and their successful implementation requires overcoming a variety of challenges, such as technical, managerial, financial, political, and standardization challenges.

Objective

Our purpose in the present study was to review recent studies in the literature to identify and classify challenges and solutions for applying EHRs to disease surveillance systems.

MATERIALS AND METHODS

Before conducting this review, it was registered in our organization (2017.9321563005). We conducted this review in terms of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement as follows. 18

Definitions

We defined disease surveillance as a systematic and continuous process of collecting and analyzing data and timely dissemination of health data to assess and respond to public health problems. 1 , 2 These systems are usually developed by public health agencies to monitor public health programs. 3 We also defined EHR-DS as any type of disease surveillance systems that collect all or some parts of their data from EHR systems.

Search strategy and information sources

This review was conducted on published articles in the literature, up to October 31, 2019. Six electronic databases, including ProQuest, PubMed, Web of Science, Cochrane Library, Embase, and Scopus, were searched. Also, unpublished studies including theses and the gray literature were searched in Google Scholar and Google, and the first 100 results were included in the review. 19 References of the selected articles were also reviewed, and the authors of the selected articles were contacted for the full text of the articles, if deemed necessary. Searches were carried out in the literature using different keywords such as “electronic health record,” “EHR,” “surveillance systems,” “registry system,” “population health,” and “public health,” and their synonyms from MeSH (Medical Subject Headings) and Emtree terms and the related keywords were extracted from the initial review of some articles. The full search strategy in the 6 databases is presented in Supplementary Table 1.

Inclusion and exclusion criteria

Table 1 shows the inclusion and exclusion criteria for this review.

Table 1.

Inclusion and exclusion criteria

EHR-DS: electronic health record–based disease surveillance systems.

Study selection

Records were downloaded into an EndNote library. After removing duplicate records, we screened titles and abstracts based on the inclusion and exclusion criteria. Then, 2 authors (A.A. and A.S.) independently reviewed the articles by their titles and abstracts, and subsequently unrelated studies were excluded. The full text of the articles was then independently reviewed by the same authors. Any disagreement was resolved through discussions among all authors. In cases that we did not have access to the full text of articles, we sent emails to the corresponding authors. Also, the reference section of the selected articles was manually reviewed, and the related studies were retrieved. Finally, 49 peer-reviewed articles and 1 government report were included ( Figure 1 ).

PRISMA flow diagram for the review process. EHR-DS: electronic health record–based disease surveillance systems.

Data extraction, charting, and synthesis

We developed a data extraction form. The extracted data items for each article included the author(s), year of publication, country, research method, study purpose, challenges, solutions, and other related results. Two authors independently extracted data for each article. Any disagreement between the 2 authors was resolved through discussion among all authors. In research studies and studies reporting an implementation of a system, we identified challenges that the authors faced or looked for solutions to implement in order to solve the challenges. In nonempirical studies, we considered only those solutions or challenges that the authors had explicitly introduced and had conceptual or theoretical arguments.

Given the heterogeneity of studies and the qualitative nature of the extracted data, we analyzed data through a narrative synthesis. This synthesis consisted of grouping the collected data by considering the patterns within and across groups. 20 In this regard, we classified empirically or conceptually similar challenges and solutions under a specific category. To ensure the accuracy of the data charting and classification process, the themes and subthemes were cross-examined and discussed by all authors. Finally, we reported the frequencies of challenges and solutions.

Ethical consideration

Ethical approval was granted by Iran University of Medical Sciences (IR.IUMS.REC 1396.9321563005).

RESULTS

Selection of sources of evidence

A total of 16 602 articles were identified, which eventually were reduced to 49 articles 6 , 8 , 10 , 13–16 , 21–62 and 1 gray study ( Figure 1 ). 17 The details of the selected studies are presented in Supplementary Table 2. Based on our analysis, the findings of the articles were categorized into 6 thematic categories in terms of challenges and solutions of EHR-DS including policy and regulatory, technical, managerial, standardization, financial, and data quality.

Distribution of studies by publication year and country

As shown in Figure 2 , the first article was published in 2005 and the peak of 7 published articles was reached in 2017. The majority of the included studies were conducted in the United States (n = 39; 78%), Canada (n = 4; 8%), and European countries (n = 4; 8%), including Norway (2 studies), France (1 study), and Belgium (1 study). Australia, Thailand, and China published 1 study each in this regard.

Distribution of studies by publication year.

Challenges and solutions for implementing the EHR-DS

Tables 2 and ​ and3 3 present the challenges and solutions of the EHR-DS. We identified 145 challenges and 130 solutions (275 in total) and then organized them into 6 main themes (policy and regulatory, technical, management, standardization, financial, and data quality) and 99 unique subthemes (47 challenges and 52 solutions). Challenges and solutions related to policy and regulations issues (14 challenges and 12 solutions), technical (8 challenges and 12 solutions), and management (9 challenges and 11 solutions) had the highest frequency of all 99 subthemes.

Table 2.

Challenges for the development and implantation of EHR-DS

In each category, challenges are ordered by the number of references.

EHR: electronic health record; EHR-DS: electronic health record–based disease surveillance systems.

Table 3.

Solutions for the development and implantation of EHR-DS

In each category, solutions are ordered by the number of references.

EHR: electronic health record; EHR-DS: electronic health record–based disease surveillance systems; HIPAA: Health Insurance Portability and Accountability Act.

Regarding policy and regulation factors, privacy, confidentiality, and data security were the most frequently cited challenges (9 studies) ( Table 2 ). 6 , 8 , 16 , 22 , 23 , 29 , 31 , 33 , 34 Increasing and supporting public health reports with meaningful use of EHRs 14 , 15 , 44 , 57 , 61 and data security, privacy, and patient consent, 26 , 34 , 60 as well as the formulation of governmental policies and the creation of the political will to implement EHR-DS, were the most common solutions. 31 , 48 For example, In the United States, the Health Insurance Portability and Accountability Act permits “covered entities” to transmit individually identifiable health information from EHRs to public health authorities. 16 ( Table 3 ).

Regarding the technical challenges and barriers, the most frequently cited technical challenges were difficulty in accessing, cleaning, and analyzing unstructured EHR data (10 studies) 14 , 16 , 25 , 32 , 38 , 40 , 42 , 45 , 50 , 57 and sharing patients’ data from EHRs for use in disease surveillance systems (7 studies) ( Table 2 ). 17 , 21 , 31 , 40 , 48 , 59 , 61 In this regard, the most frequent solutions were using natural language processing, ontology, machine learning, and data mining in processing unstructured data for use in disease surveillance systems (9 studies) 8 , 16 , 33 , 34 , 38 , 42 , 44 , 47 , 56 and the use of software techniques and tools for data retrieval, extraction, de-identification, and visualization (8 studies) ( Table 3 ). 8 , 14 , 24 , 28 , 29 , 34 , 42 , 51

Managerial challenges and solutions subthemes had 20 of the 99 subthemes. Problems related to inadequate population coverage in the EHRs, 8 , 14 , 15 , 17 , 23 , 43 and the collaboration of health and clinical departments to access and use EHRs were frequently cited challenge and solution, respectively. 8 , 15 , 30 , 37 , 39 , 44 , 57 , 62

Fourteen of 99 of the identified challenges and solutions were related to standardization. The lack of interoperability standards or not widespread use of these standards was the most frequently cited challenge. 21 , 25 , 27 , 40 , 46 , 59 , 62 Lack of data on social variables and risk factors in the EHRs 10 , 14 , 43 , 53 , 57 and the existence of different data collection standards 10 , 31 , 37 , 61 were other important challenges ( Table 2 ). On the other hand, the EHR standardization approach based on the data needed for disease surveillance system, 13 , 25 , 30 , 36 , 38 , 49 , 54 the use of a unique individual health identifier, 16 , 29 , 31 , 32 , 36 , 59 , 60 and the use of interoperability standards 6 , 8 , 21 , 25 , 29 , 54 were the most important solutions ( Table 3 ).

With respect to resources, the need for investment, time, and resources were found to be the most frequent factor. 6 , 10 , 13 , 14 , 21 , 23 , 24 , 27 , 30 , 31 , 37 , 40 , 44 , 45 , 48 , 55 , 57 Furthermore, data quality factors accounted for 9 of 99 challenges and solutions. Inappropriate data quality including lost data, inconsistent and missing data, duplicate data, variability of the collected data, poor temporal and geographic representation of data, and low data representativeness, 6 , 14 , 17 , 26 , 27 , 30 , 33 , 45–47 , 52 , 56 , 57 , 59 , 61 , 62 and problems related to multiple records for each patient and possible double-counting as well as reliability and validity of unstructured EHR data, are common ( Table 2 ). 10 , 14 , 29 , 61 , 62

DISCUSSION

There are many challenges in the implementation of EHR-DS including policy, regulatory, technical, financial issues, management, standardizations, and also data quality. Although these challenges may differ from country to country and from project to project, some common factors play important roles in the successful implementation of these systems that will be discussed in the following sections.

Policies and regulations

EHR-DS are not just about technology; other factors, such as policies and regulations, are also very important, the most important of which are security, privacy, confidentiality, and informed consent. 6 , 8 , 16 , 22 , 23 , 29 , 31 , 33 , 34 These challenges stem from the availability of electronic data and the exchange of information between healthcare organizations. There is concern that patient information may be used commercially. In this regard, Friedman et al 16 consider ensuring the security and privacy of patients' data as the reason for the public acceptance of these systems, which will lead to a decrease in community resistance. As a rule, confidentiality and privacy regulations protect patients' data and prevent access to medical records unless the patient has given consent. 26 , 34 , 60 This particular situation may seriously undermine the EHR-DS because the refusal of some patients to use their EHR data for nonclinical purposes eliminates a representative sample from EHR-DS. This problem can be remedied by allowing access only to the de-identified data. Furthermore, the establishment of data transfer requirements and appropriate regulations for public health purposes may reduce objections to using EHR data. For example, the Health Insurance Portability and Accountability Act privacy rule generally prohibits the use or disclosure of protected health information. However, there are exceptions in public health. Without individual authorization, a covered entity may disclose protected health information to a public health authority that is legally authorized to collect information for the purposes of preventing or controlling diseases and conducting public health surveillance. 16

There are also some uncertainties about access to EHR data for use in disease surveillance systems. For example, the question as to who or what organizations should have access to EHR data for these systems or which authorities should organize and govern disease surveillance systems. 15 , 16 , 61 Before using EHR data in disease surveillance systems, issues such as access to information, national regulations, and authorization for access to information by the government should be considered. Moreover, healthcare organizations need to accept data sharing agreements to use EHR de-identified data in these systems.

Technical factors

One of the most important technical challenges is how to use unstructured data in the EHR-DS. 14 , 16 , 25 , 32 , 38 , 40 , 42 , 45 , 50 , 57 Large volumes of EHR data are unstructured, while the surveillance systems rely heavily on structured data to monitor population health, and for developing and visualizing health indicators. Additionally, unstructured data are not easily transformed into a form that can provide the algorithms needed for population health and be visualized at the same time. Hence, surveillance systems require better methods such as natural language processing to identify the main structures in unstructured data. The use of machine learning and text mining is recommended to gain implicit knowledge in the unstructured context and to convert the highly detailed data into a data format the can be understood and useable in disease surveillance systems. 8 , 16 , 33 , 34 , 38 , 42 , 44 , 47 , 56

Different EHR systems are used, each with its own different technical specifications and functionalities. 6 , 28 , 31 , 35 On the one hand, these differences make it difficult to create a standard format for data sharing. 17 , 21 , 31 , 40 , 48 , 59 , 61 On the other hand, the varying status of public health information technology infrastructure in different public health agencies and the different software used to support public health measures increase the complexities of implementing EHR-DS. 6 , 10 , 13 , 31 Therefore, compatibility of EHRs with other public health information systems needs to be considered.

Managerial factors

The inadequate population coverage at EHR systems and training of users, and the lack of mutual understanding between the medical and technical community, are among the most serious managerial barriers to the implementation of the EHR-DS. 10 , 14 , 25 , 39 , 46 , 55 The implementation of these systems requires a strong strategic, executive, and technical committee. Strong leadership, user training, collaboration between health and clinical departments, and coordination between providers and EHR vendors and encouraging clinical partners to cooperate are essential. Therefore, management support should be considered as one of the most effective ways to coordinate different clinical, technical and public health partners.

Another major challenge is the poor performance of the EHR-DS in the geographical areas with less EHR coverage. 8 , 14 , 15 , 17 , 23 , 43 In these areas, the population estimates and indicators may not be valid and complete regarding the local population covered by the EHRs. In addition to encouraging the adoption and use of EHRs, more EHR systems should be integrated with disease surveillance systems so the population is covered by these systems and is more generalizable to the general population. Another major challenge is the feeling of data ownership among clinicians. 39 Healthcare providers are not usually willing to share EHR data with disease surveillance systems because of their sense of data ownership. Therefore, developing data sharing policies and providing incentives can reduce concern in this regard.

Standardization

Weak interoperability between information systems prevents the development of an effective and integrated information system 21 , 25 , 27 , 40 , 46 , 59 , 62 ; hence, the use of a set of guidelines and standards in developing an EHR-DS is a key step in improving interoperability. The formulation of these standards should also focus on key aspects of population and public health. On the one hand, for example, owing to the lack of data on social variables, risk factors, and geographic data in EHRs, 10 , 14 , 43 , 53 , 57 the content of EHRs should, therefore, be expanded and standardized to include this type of data for use in public health. 13 , 25 , 30 , 36 , 38 , 49 , 54 On the other hand, in the EHR-DS, data related to a particular disease or condition must be identifiable and extractable from EHRs, so the use of standard terminologies enhances the ability of these systems to interoperate with EHRs.

Accurate locating and linking a person's health records over time and from different EHR systems is one of the other major challenges in this regard; thus, creating a unique health identifier for individuals is suggested as an essential component for EHR-DS. 16 , 29 , 31 , 32 , 36 , 59 , 60 These unique identifiers are important for reducing costs associated with slow or inaccurate patient identification and record location algorithms. Moreover, creating, agreeing, and applying inclusion and exclusion criteria to accurately identify patients eligible for disease surveillance systems from EHRs also raises concerns 49 because of the unstructured data and diverse semantic structures in EHRs. Hence, more work needs to be done on standardizing EHR content in this regard.

Financial factors

Financing and providing resources and the costs and time have always been a major concern for EHR-DS. 6 , 10 , 13 , 14 , 21 , 23 , 24 , 27 , 30 , 31 , 37 , 40 , 44 , 45 , 48 , 55 , 57 Paul et al 6 mentioned several failed national and regional EHR-DS owing to insufficient resources. EHRs are generally provided for patient clinical practice and lack the psychological, behavioral, social, and environmental data and other data required for public health practices, and efforts to incorporate these data into EHRs require substantial investment and relationships with EHR vendors. It may also create an additional burden for clinical staff. 10 The findings also highlight that the most important strategy in this regard is to invest significantly in public health infrastructure to support EHR data, with support from financial institutions and sustainable financial and budget commitments. 10 , 14 , 31 , 39 , 57 Therefore, the role of governments in supporting these systems is undoubtedly significant and can facilitate the overcoming of many financial challenges.

Data quality

Quality of EHR-DS depends on the quality of data and its processes. Data quality is more of a human challenge than a technical one. People who document data in EHRs should strive to continuously improve data quality. On the one hand, with high-quality data, disease surveillances can better monitor and manage a disease and its outcomes. 39 , 46 , 53 , 62 For example, disease surveillance usually uses International Classification of Diseases codes to identify eligible patients. Therefore, coding errors may result in misidentification of cases. 63 On the other hand, data should be constantly monitored and feedback should be given to the providers. 39 Furthermore, organizations should also regularly evaluate data using appropriate criteria to ensure that all data are gathered in a proper format and according to the data definitions. Definition of data may be different in EHRs and surveillance systems; hence, formulating standard definitions may be needed to improve the quality of data. 15 , 23

Limitations and future studies

This review has several limitations that can be considered as essential suggestions for future studies. First, the articles in the 6 main databases, published in English, were only reviewed. Therefore, it did not include other potential articles. Second, we did not consider an EHR-DS for a particular disease. Developing such a system for a particular disease may incur specific problems and challenges that may require separate studies. Third, because the purpose of the present study was to identify and classify all the relevant challenges and solutions, we did not assess the quality of included studies, and all studies irrespective of quality or design were included. Fourth, we did not have the same quantitative measures in the primary studies. Therefore, we did not assess risk of bias through forest or funnel plots. Fifth, some types of documents were excluded from this review. For future study, researchers can also provide an architecture or model for the successful implementation of the EHR-DS using our findings.

CONCLUSION

EHR-DS are powerful technology that can potentially improve the quality of surveillance systems. This review has shown that due to the multifaceted nature of the challenges, the use and implementation of these systems is not cost-effective and easy to implement, and requires a wide range of interventions. Implementation of EHR-DS is an important but time-consuming process that requires a long-term strategic plan. Furthermore, the successful implementation of EHR-DS requires the enacting of appropriate laws, regulations, and standards. According to our findings, the 5 most common challenges are the need to allocate significant time, budget, and resources; poor data quality in EHRs; difficulty in accessing, cleaning, and analyzing unstructured EHR data; data privacy and security; and the lack of or not widespread use of interoperability standards. The 5 most common solutions are the use of natural language processing, machine learning, and data mining algorithms in unstructured data processing; the use of appropriate technical solutions and software for data retrieval, extraction, identification, and visualization; the collaboration of health and clinical departments to access data; standardizing EHR content for public health; and using a unique health identifier for individuals.

FUNDING

This work is a part of a PhD dissertation supported by Iran University of Medical Sciences (Grant number: IUMS/SHMIS_2017.9321563005). AS received the grant funding from Iran University of Medical Sciences.

AUTHOR CONTRIBUTIONS

AA was involved in conceptualization, data gathering, data analysis, and drafting the manuscript. AS was involved in conceptualization, data gathering and analysis, funding acquisition, project administration, supervision of the project, and drafting and revising the manuscript. HA was involved in conceptualization, data analysis, and revising the manuscript.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.