Poor usability of electronic health records in Norway

    ()

    sporsmal_grey_rgb
    Article

    Electronic health records are not user friendly. This issue must be taken seriously, as it has implications for patient safety as well as economic and workforce sustainability.

    Doctors must interact with digital systems in their daily work. The primary tool is the electronic health record (EHR). The Norwegian Patient Records Act (Pasientjournalloven) and the associated regulations require doctors' work to be documented in an EHR (1, 2). In recent years, there has been a strong focus on the time spent on clinical documentation. This issue has been highlighted in the Norwegian Health Personnel Commission's report Time for Action (NOU 2023:4) and further underscored by negative experiences with the EHR Helseplattformen (based on EPIC®), as summarised in the National Audit Office of Norway's report. (3–5).

    Why is usability important?

    Why is usability important?

    Usability refers to selected characteristics of a computer system as experienced by the user. In his book Usability Engineering, Jakob Nielsen emphasises that usability is not one-dimensional and defines it using five attributes: ease of learning, efficiency of use, memorability, few and non-catastrophic errors, and user satisfaction (6).

    Usability in EHRs is extremely important for ensuring safe and effective health care. Poor usability can affect both individual patients and the overall capacity of the health service, and it represents a significant occupational health risk. Errors in digital health systems can, in the worst-case scenario, result in patient death. System failures, such as incorrect medication dosing, can affect many patients simultaneously (7). The Norwegian Board of Health Supervision warns of the risk of patient harm if the system conceals important information, or if the user fails to locate such information (8).

    Poor usability can affect both individual patients and the overall capacity of the health service

    In a study examining the relationship between usability scores and risk of burnout, higher usability scores correlated with a lower risk of burnout and a reduced workload (9, 10). A recently published Norwegian study showed similar findings (11, 12). Those who experience lower usability report a higher incidence of burnout, insomnia and turnover intention. Poor usability of EHRs can also pose a significant financial risk for healthcare organisations.

    Can usability be measured?

    Can usability be measured?

    There are several validated methods for measuring perceived usability. The System Usability Scale (SUS) is a concise questionnaire consisting of just ten items, designed to assess usability at a general level (13). This scale is used worldwide across all types of user interfaces, ranging from web search engines, microwave ovens and Word, to EHRs (14–16). The maximum score is 100, with scores below 51.6 typically considered a fail (16).

    A more comprehensive methodology is the National Usability-Focused Health Information System Scale (NuHISS), developed and validated by researchers in Finland (17). This method includes several questions grouped into areas such as technical quality, ease of use, information quality, benefits/usefulness, collaboration/communication and feedback. It allows for the study of different characteristics of EHRs (18–20).

    Status in Norway

    Status in Norway

    No studies to date have examined Norwegian doctors' perceived usability of EHRs based on international standardised methods. The Norwegian Medical Association conducted a nationwide survey on usability in February 2022, repeated after 2.5 years in October 2024, called the EHR Thermometer. The survey used both the SUS and NuHISS methodologies and was sent to practising members of the Norwegian Medical Association (about 93 % of all practising doctors in Norway) (21). The initial results were presented in a master's dissertation at the Faculty of Medicine, University of Oslo (22).

    The average reported SUS score at the national level was 53.6 (Table 1), based on 4641 respondents (response rate 14 %). This is somewhat better than comparable measurements among doctors in the United States, where the average SUS score was 45.9 (9). Compared to findings from doctors at St Olav's Hospital, who had a median score of 15, the national scores are much higher (11).

    Table 1

    Usability scores (System Usability Scale (SUS), a scale from 0 to 100, where <51.6 is considered a fail) broken down by doctor role in 2024 (22).

    Role

    Average score

    CI (95 %)

    Median

    Standard deviation

    Number

    All roles

    53.6

    52.8 to 54.4

    57.5

    26.7

    4641

    General practitioner

    70.7

    69.4 to 71.9

    74.7

    20.7

    1043

    Specialist in private practice

    66.8

    64.5 to 69.1

    70.2

    21.1

    323

    Doctor at out-of-hours clinic

    54.3

    48.4 to 60.3

    57.5

    23.1

    61

    Hospital doctor

    47.7

    46.7 to 48.6

    50.0

    26.0

    2783

    Nursing home doctor

    37.0

    34.1 to 39.8

    32.7

    24.9

    294

    When broken down by doctor roles, a different pattern emerges. General practitioners (GPs) report significantly higher levels of perceived usability than hospital doctors, especially compared to nursing home doctors (Table 1). Among hospital doctors, there is also a large difference in satisfaction between EHR systems: Helseplattformen had a reported usability score of just 17.3, compared to 57.7 for the DIPS EHR system.

    GPs report significantly higher usability than hospital doctors, and particularly nursing home doctors

    The SUS scores of hospital doctors grouped by level of experience with EHRs reveal an interesting pattern. For hospital doctors using a combination of DIPS Arena and MetaVision, no statistically significant differences in SUS scores were observed based on user experience with those systems (22). Hospital doctors with more than one year of experience with Helseplattformen reported a statistically significant increase in SUS scores compared to users with less experience. However, this improvement remains far from sufficient to reach the same satisfaction levels as DIPS users.

    Support for collaboration

    Support for collaboration

    The NuHISS method goes into greater detail, and the master's dissertation presents responses to selected questions in the categories of 'ease of use' and 'collaboration and communication' (22). On the six questions about collaboration and information exchange features, GPs in Norway were markedly more satisfied than hospital doctors and nursing home doctors. Hospital doctors rated Helseplattformen considerably lower than other systems. Across Norway's four health regions, nursing home doctors reported similar levels of satisfaction, which were comparable to those reported by users of Helseplattformen. In other words, Helseplattformen does not improve perceived interoperability compared to other nursing home systems, despite offering interoperability features (e.g. e-prescriptions) that the other systems lack.

    It is possible to compare some of the Norwegian findings with international results. A study of Australian hospital doctors revealed slightly lower usability scores than those of Norwegian hospital doctors. However, on questions related to collaboration between doctors, both within the same organisation and across organisational borders, Australian users reported somewhat better experiences than the Norwegian users of Helseplattformen (20). Australian primary care doctors scored higher on the same questions than their hospital-based counterparts. However their scores were markedly lower than those reported by Norwegian GPs (20).

    Methodological weaknesses

    Methodological weaknesses

    The EHR Thermometer is based on survey data sent to all practising members of the Norwegian Medical Association, with the aim of obtaining sufficient responses for each doctor role and each EHR system. We did not assess for selection bias, but given a response rate of 14 %, that risk is undoubtedly present.

    The challenges must be taken seriously

    The challenges must be taken seriously

    One possible explanation for the higher usability scores reported by GPs is the presence of multiple competing systems within the primary care market. Over time, this may have enabled vendors to tailor functionality and usability to the clearly defined needs of the well-established GP role in Norway.

    Hospitals are more complex organisations, with more complicated logistics and a wider range of needs, some of which may be conflicting, across numerous roles. Helseplattformen performs poorly in our survey, and that may be largely attributed to its reliance on rigid, structured data models using SNOMED CT, rather than flexible free-text input (23). This approach was intended to yield benefits through the reuse of structured information. Ensuring that all users uniformly interpret and update the data is difficult and complex. Such precise information management is also time-consuming for the user.

    Helseplattformen performs poorly in our survey, and that may be largely attributed to its reliance on rigid, structured data models

    Integrating primary care services into the system introduces additional, often conflicting requirements, which may create challenges for users of Helseplattformen. Furthermore, specific issues within the user interface will pose a challenge. Items from the NuHISS method, such as 'The arrangement of fields and functions is logical on the computer screen', 'Routine tasks can be performed in a straightforward manner without the need for extra steps using the system', 'It is easy to obtain necessary patient information using the EHR system', and 'Entering and documenting patient data is quick, easy, and smooth' can help shed light on this (22).

    Improved collaboration was supposed to be the main benefit of Helseplattformen, but the system is rated lower than other hospital systems in this regard, and provides few or no benefits for nursing home doctors. This suggests that, in the Norwegian context, an integrated system like Helseplattformen is less effective in supporting collaboration than an ecosystem of EHRs and national interoperability solutions. There is a need to improve digital tools to support collaboration. Despite these challenges, EHRs in Norway perform relatively well by international standards, which may be a reflection of successful national efforts in areas such as electronic messaging, e-prescriptions and the national summary care record.

    System owners and decision-makers in regional health authorities, local authorities, government administration and political bodies should familiarise themselves with usability measurement methods and provide for regular evaluations and improvement initiatives. Usability in EHRs must be taken seriously, as it directly impacts patient safety and the sustainability of financial resources and staffing.

    Comments  ( 0 )
    PDF
    Print
    Reply to article

    Recent Articles