Which of the following is used when an appointment for a patient will take a longer period of time?

AMIA Annu Symp Proc. 2017; 2017: 921–929.

Published online 2018 Apr 16.

Abstract

Improving the efficiency of outpatient clinics is challenging in the face of increased patient loads, decreased reimbursements and potential negative productivity impacts of using electronic health records (EHR). We modeled outpatient ophthalmology clinic workflow using discrete event simulation for testing new scheduling templates that decrease patient wait time and improve clinic efficiency. Despite challenges in implementing the new scheduling templates in one outpatient clinic, the new templates improved patient wait time and clinic session length when they were followed. Analyzing EHR data about these schedules and their adherence to the template provides insight into new policies that can better balance the competing priorities of filling the schedules, meeting patient demand and minimizing wait time.

Introduction

Physicians today are pressured to see more patients in less time for less reimbursement due to persistent concerns about the accessibility and cost of healthcare.1,2 Furthermore, clinicians are concerned that the adoption of electronic health records (EHRs) has negatively impacted their productivity.3–5 For example, at Oregon Health & Science University (OHSU), which completed a successful EHR implementation in 2006 that received national publicity, ophthalmologists currently see 3-5% fewer patients than before EHR implementation and require >40% additional time for each patient encounter.6

Clinic inefficiencies result when patients arrive and clinic resources (staff, exam rooms, and providers) are not available to serve them. This mismatch of arrivals and availability can be caused by ad-hoc scheduling protocols that increase patient wait time.7 Previous work has demonstrated that EHR timing data can be used for building clinic simulation models for studying scheduling templates before implementing them in the clinic.8 These models, along with prior research, confirm that scheduling longer encounters with higher variability at the end of the day helps reduce wait time.9

While simulation models can predict improvements with new scheduling templates, implementing them in clinic faces real-world challenges such as competing clinic priorities, and established scheduling practices. While efficient workflow is a priority in outpatient clinics, so is filling all appointment slots and serving urgent walk-in patients. Clinic staff become adept at navigating clinic schedules, ensuring that all urgent patients are scheduled and appointment slots are filled, including those that are last-minute cancellations. New scheduling templates disrupt these practices and priorities, which can make following new schedules challenging.

We implemented a new scheduling template in one outpatient ophthalmology clinic (LR) at OHSU starting in September 2016 after preliminary studies demonstrated its effectiveness.10 As expected, there were challenges in implementing the new template, but when it was followed, there were reductions in average patient wait times and clinic session lengths. The purpose of this paper is to analyze challenges in a real-world implementation of simulation model based workflow improvements. Analyzing data about adherence to the new template will provide insight into new policies that can better balance the competing priorities of filling the schedules, meeting patient demand, and minimizing wait time, ultimately improving the clinic workflow. We found that 1) complex templates that the simulation predicts as optimal can be challenging to follow, 2) secondary use of EHR data allowed for a thorough analysis of a complex scheduling template and suggested ways to simplify it, 3) when the template was followed, scheduling according to exam length significantly improves patient wait time and session length, especially when short exams are scheduled at the start of the clinic, 4) these insights have impacts for developing new scheduling policies and strategies, and 5) EHR timing data enables detailed evaluation and improvement of clinic workflow modifications.

Methods

This study was approved by the Institutional Review Board at Oregon Health & Science University (OHSU).

Study Environment

OHSU is a large academic medical center in Portland, Oregon. The ophthalmology department includes over 50 faculty providers, who perform over 90,000 annual outpatient examinations. The department provides primary eye care, and serves as a major referral center in Pacific Northwest and nationally. We studied one provider’s outpatient clinic in pediatric ophthalmology (LR).

Over several years, an institution-wide EHR system (EpicCare; Epic Systems, Madison, WI) was implemented throughout OHSU. This vendor develops software for mid-size and large medical practices, is a market share leader among large hospitals, and has implemented its EHR systems at over 200 hospital systems in the United States. In 2006, all ophthalmologists at OHSU began using this EHR. All ambulatory practice management, clinical documentation, order entry, medication prescribing, and billing tasks are performed using components of the EHR.

Clinic Workflow

Interviews and clinic observations determined the basic clinic workflow of the majority of patient appointments as shown in Figure 1. Patients check in and wait to be seen. An ancillary staff member (an ophthalmic technician or orthoptist) performs an initial exam in an exam room. At the end of this exam, the patient’s eyes may be dilated. If this is the case, the patient returns to the waiting room while waiting for the dilation to take effect. After the dilation occurs (approximately 25 minutes), the patient is returned to an exam room and waits for the physician exam. If the patient’s eyes were not dilated, the patient remains in the exam room and waits for the physician. After the physician’s exam, the patients check out and leave. Waiting occurs before each of the two exams if a staff member (2 total) or physician (1 total) is not available.

Basic Clinic Workflow. Flowchart representation of the clinic workflow. Patients see a staff member for an initial exam followed by a physician exam. Patients’ eyes may be dilated before the physician examines them.

Simulation Model Using EHR Timestamp Data

To build models for studying the clinic’s efficiency, we first had to collect large amounts of timing data for each step of the clinic workflow. In prior studies, we identified sources of timestamp data within the EHR and verified them against timing data from in person observations.8 We used the clinical data warehouse and ophthalmology datamart for OHSU’s EHR (EpicCare; Epic Systems, Madison, WI). While these timestamps are specific to OHSU’s implementation in ophthalmology, comparable timestamps are available for other vendors, installations and specialties. We used this timing data in our models, and for measuring the average patient wait time before and after we changed the scheduling templates.

We used Arena simulation software11 to build a discrete event simulation model of a pediatric outpatient ophthalmology clinic’s (LR) workflow using the EHR timing data. Previous studies validated this model and used it to test different scheduling templates. Schedules with the longest appointments near the end of the clinic session minimized average patient wait time, but could also lengthen clinic sessions. As a compromise, scheduling long patients near the end of the clinic, but not at the very end, still reduces wait time without unduly lengthening the clinic.8

New Schedule Template

Based on the results of the simulation studies and input from the clinic provider and staff, we created a new scheduling template to be used within the pediatric provider’s clinic. Patients’ appointments were scheduled according to their anticipated length and dilation status: shortest appointments were earliest, with the longest appointments near the end of the clinic to minimize patient wait time. We classified the short appointments as roughly the shortest quarter ofappointments and the long appointments as the longest quarter. The remaining appointments were designated as medium. Each clinic session was a half-day with patients scheduled in 15 minute blocks. Because of OHSU’s scheduling policies, all appointment blocks must be the same length. The morning clinic schedule is given in Figure 2; the afternoon schedule was similar, but without the empty block and one each fewer of short and medium blocks. Morning clinics ran from 8:00 am until about 12:00 noon with 15 total patients; afternoon clinics began at 12:45 pm or 1:00 pm and ended by 5:00 pm. To ensure that afternoon clinics ended on time, there were usually fewer patients scheduled (13). In both clinic sessions, we scheduled the long patients near the end of the clinic, with 3 non-long patients after them to avoid clinic sessions ending late. In the morning clinic, the staff wished to include an empty slot in the middle of the long patient appointment slots to allow for catchup. The first appointment slot was double booked with one dilated patient and one not. This allowed the provider to get started quickly with the non-dilated patient.

New Scheduling Template. This is a morning schedule with 15 minute scheduled blocks. First block is double booked and block 10 is deliberately left empty for catchup. Shorts are scheduled first followed by medium, then long. The clinic ends with 3 medium appointments. Dilation is mostly alternating. Afternoon schedule is similar, but without the empty block and with one fewer medium and short blocks.

Previous studies identified which exams were expected to be shorts and longs,12 but also found that there was quite a bit of variability among appointment lengths that was not easy to predict (SpatarD, et al. IOVS 2016;57:ARVO E- Abstract 5566). Fortunately, the provider was very good at assessing the length of the exam.12 For this reason, the provider documented her prediction (short, medium, or long) for the next appointment in the followup notes for the encounter, which are viewed by the scheduler when scheduling the next appointment. New appointments are always long.

Finally, the provider and staff wished to try alternating dilated and non-dilated appointments for preventing multiple patients waiting for the provider at the same time. Appointment slots on the schedule were designated as dilated or non-dilated exams, which required schedulers anticipating which type of exam the patient required at time of scheduling. Again, the provider documented her preference in the notes and other information about the appointment helped with this prediction.

The new scheduling templates were put into place starting in April 2016 for appointments scheduled September 1 or after. The provider started documenting her exam length and dilation predictions in her followup notes at this time. Even with a lead time of 5 months, there were still appointments already scheduled after September 1 using the old templates; the clinic decided to leave them as scheduled, regardless of whether they followed the new scheduling template.

New Schedule Template Evaluation Metrics

To assess the performance of the new scheduling template, we used two metrics: patient wait time and clinic length. The patient wait time was calculated using EHR timestamps and data about the appointment from the OHSU datamart. Exam lengths were determined by EHR audit log timestamps recorded between the appointment’s check in and check out times. Wait times are calculated by subtracting the exam length and dilation length (25 minutes for dilated exams, 0 for non-dilated exams) from the entire time the patient was checked in for their appointment.

We found that we needed to adjust this wait time since many patients arrived significantly early for their appointments. Because OHSU provides care for a large geographic region throughout the Pacific Northwest, many patients arrive early because of concerns about traffic and/or with the hope of being examined earlier. Often the patients are seen before their appointment times, so it was not meaningful to count only the wait time that occurred after their scheduled appointment time. Instead we eliminated any wait time that occurred before the exam started that was also before their scheduled appointment time. Once the patient started the exam, all subsequent wait time was counted. Clinic length is calculated as the difference between the first timestamp recorded during an exam in the clinic and the last timestamp recorded for the last exam of the clinic.

New Schedule Template Evaluation

To evaluate the impact of the new schedule in October - December 2016, we first compared the average patient wait times and clinic lengths to baseline sessions for the same clinic from July 2015 - June 2016, when the old scheduling templates were used. We used a t-test to compare our two metrics and investigated differences between the two groups with respect to new patient appointments, adult patients, dilated patients, early and late patients, double booked appointments, unfilled appointment blocks and known long exams at the start of the clinic session.

To further analyze the new schedule, we compared the scheduled appointments with the scheduling template to determine when and how well the templates were followed. Again, we used data from the EHR to assess the adherence to the scheduling template: the followup notes from the previous exam, the date of the previous exam, the type of scheduled encounter, and the age of the patient. The followup notes, when available, provided the provider’s predictions for exam length and dilation. When the notes weren’t available, we used our previously determined rules for predicting exam length based on encounter type and patient age (new vs. return, adult vs. child, pre-op, post-op or followup). The date of the previous appointment determined the dilation status (most patients require dilation once annually). We coded each appointment block according to its predicted exam length and dilation along with the corresponding template block’s exam length and dilation. We used linear regression to determine the wait time and session length impacts of following the schedule according to exam length and dilation.

Results

New vs. Old Scheduling Templates

We compared the performance of the new scheduling templates (Oct - Dec 2016) to our baseline (July 2015 - June 2016), which used the old templates. For our two main metrics, average patient wait time and clinic session length, the new template was not significantly different from the old template as shown in Table 1. The means for wait time (20.6 minutes for new and 22.1 minutes for old) and the means for session length (242.3 minutes for new and 232.1 minutes for old) were not significantly different when compared using a t-test (p = 0.09 and p = 0.08, respectively).

Table 1:

Comparison of New and Old Scheduling Templates. For both patient wait time and session length, the new schedule was not significantly different from baseline (p > 0.05 for both). All other clinic characteristics were similar for both the new schedule and baseline.

We also measured other characteristics relating to the schedules for both the old and new template periods in Table 1. Both groups were similar with respect to the number of dilated, new, adult, early and late patients per session. They also had similar numbers of unfilled blocks and double-booked blocks. Finally, because the new template does not schedule long patient appointments at the start of the clinic session, we compared the number of adult and new patients (always long appointments) at the start of the session. Both were similar, with slightly more new patients at the start of the baseline clinics.

Scheduling Errors

Since the wait times, clinic lengths and clinic characteristics for the new schedule test period were similar to the old templates in the baseline, we analyzed the new schedule test period for adherence to the new schedule template.

Figure 3 shows the impact of schedule adherence on average patient wait time and session length. For each clinic session, the percentage of correct blocks for exam length and dilation was plotted versus the average patent wait time and session length for that clinic session. As the percentage of correctly scheduled exam length blocks increased, the wait time decreased significantly: 3.7 minutes per 10% increase in correct exam length blocks (p < 0.0001). Similarly, the session length decreased by 7.7 minutes per 10% increase in correct exam length blocks (p =0.008), according to the linear regression models shown in the figure. Correctly scheduling the dilation blocks did not significantly decrease the wait time or session length.

Impacts of Following the New Schedule on Wait Time & Session Length. As more blocks are correctly scheduled for exam length, both the patient wait time and session length significantly decreased (p <0. 0001 & p = 0.008, respectively). Following the schedule for dilation did not significantly impact wait time or session length.

To understand how the clinic adhered to the template, we analyzed the scheduling errors. Table 2 outlines the different types of appointment blocks and the errors made in scheduling them; 240/677 (36%) appointment blocks were not scheduled correctly with respect to exam length and 270/677 (40%) were incorrect with respect to dilation. Breaking the appointment blocks down according to exam length shows that similar percentages (31% - 39%) were scheduled incorrectly. Finally, 14/24 (58%) blocks intended to be blank were scheduled with appointments. Since errors with were occurring frequently—30 – 40% of the time—it was important to determine which errors had the biggest impact.

Table 2.

Scheduling Errors. The schedule was not followed for both exam length and dilation: 39.9% of blocks were incorrectly scheduled for dilation and 36.2% of blocks were incorrectly scheduled for exam length.

Figure 4 shows the impact of scheduling errors for short, medium and long exam block errors. The most significant impacts were for short block errors: they increased wait time by 1.5 minutes for each 10% increase in errors (p = 0.008) and they increased clinic length by 4.8 minutes per every 10% increase in errors (p = 0.005) as predicted by the linear regression models shown in the figure. The only other significant impact was on clinic length for medium block errors: those decreased the session length by 5.5 minutes for each 10% increase in errors (p = 0.01).

Impacts of Short, Medium, and Long Block Scheduling Errors. Scheduling short blocks incorrectly had the most significant impact on wait time and session length (p = 0.0009 and p=0.005, respectively). Scheduling medium blocks incorrectly also significantly impacted session length (p = 0.01), but not patient wait time. Long block scheduling errors did not significantly impact patient wait time or session length.

Looking at just the short scheduling errors, we see that scheduling longs in a short exam block had the most significant impact as shown in Figure 5. For every 10% increase in the number of long exams in short blocks the wait time increased by 1.5 minutes (p = 0.01). There wasn’t a significant increase in session length for long exams in short blocks, nor was there a significant impact of scheduling medium exams in short blocks on wait time or session length.

Impact of Short Block Scheduling Errors. The biggest impact of short scheduling errors is scheduling a long exam in a short exam block (p = 0.02); it causes a 1.5 minute increase in wait time per 10% increase in the number of long exams in short exam blocks.

Discussion

The new clinic schedule did not significantly decrease patient wait time or clinic session length over baseline for the entire 3 months we evaluated, presumably because the schedule was not followed for all clinic sessions. This highlights the difficulty in implementing scheduling changes predicted to be optimal based on simulation models. Analyzing the clinic sessions’ schedules for errors gives us insight into improving the scheduling process in the future. In particular, we found:

• 1.

  Complex templates that simulation predicts as optimal can be challenging to follow. At best, 60% of blocks were correct for dilation and 64% were correct for exam length. From our discussions with schedulers, we determined the new template was too difficult to schedule for both exam length and dilation, there weren’t enough dilation blocks (only 277 were designated as dilated blocks, but there were 383 actual dilated patients), not all patients had provider predictions, and there are competing priorities for maintaining clinic volume. When patients cancel, the appointment blocks must be filled, usually with new patients who require long exams. This highlights the substantial challenges for implementing new scheduling templates and the importance of evaluation for determining their benefits.

  Table 3 provides a breakdown of the potential sources for scheduling errors. Some appointments were scheduled prior to the implementation of the scheduling template in April of 2016 and these appointments weren’t changed if they violated the schedule. For example, 14.5% of the errors in short blocks were due to this. Also, when patients canceled close to the appointment time, there is pressure to fill the appointment block; this caused 29.1% of the short block errors. Missing provider predictions for appointments also results in errors; this caused 78.2% of the short block errors. Some of these were new patients who didn’t have predictions (schedulers know they are always longs) or patients whose previous appointment was prior to April of 2016, before the provider started documenting her predictions. The remainder were appointments that should have had predictions, but were missing. Nevertheless, even with predictions, there were still scheduling errors. This highlights the need for more provider predictions and better training for schedulers to follow them. It also motivated us to find ways to simplify the schedule.

  Table 3.

  Sources of Scheduling Errors. A few scheduling errors occurred because of appointments that were scheduled prior to the new template’s implementation: it accounts for 14.5% of the short block errors. Others occurred because of late scheduling, but the majority of errors happened without provider predictions (78.2% of the short block errors), but even with predictions there were still errors (21.8% of the short block errors).

  

• 2.

  Secondary use of EHR data allowed for a thorough analysis of a complex scheduling template and suggested ways to simplify it. We determined dilation block scheduling errors do not appear to worsen wait times or session length and it is difficult to schedule for both dilation and exam length. For both of those reasons, we simplified the template to exclude dilation, which we hope eases the schedulers’ burden.

• 3.

  When the template was followed, scheduling according to exam length significantly improves patient wait time and session length, especially when short exams are scheduled at the start of the clinic. Wait time decreases by 3.7 minutes per 10% increase in correct exam length blocks and session length decreases by 7.7 minutes per 10% increase in correct blocks. Further, scheduling short patients at the start of the clinic helps the clinic stay on time and reduce patient wait times for subsequent patients. It is most significant to avoid scheduling long exams in the short exam blocks, but even medium exams can have an impact. This result is important since it validates our simulation models and previous studies8,9 and confirms that our schedule can improve clinic efficiency if it is followed, potentially allowing for more patients to be seen in a clinic session. It also underscores the importance of schedulers following the new scheduling template.

• 4.

  These insights can lead to better scheduling and potentially innovative dynamic scheduling strategies. There are currently many opportunities and possible innovations for improving scheduling in healthcare13. For example, because scheduling long patients at the start of the clinic has the biggest impact on patient wait time, all attempts should be made to avoid doing this, even when an early block becomes available for a long patient (e.g. a new patient). Schedulers should attempt to shift shorter exams to this block or double book the new patient later, leaving the short block empty. The unfilled blocks (typically 1-2 per clinic session) due to no shows or late cancels can help offset this double booking. Also, since the clinic has a high number of early patients, any that have short or medium exams can be worked in to fill that early blank spot. Other possibilities include innovative dynamic scheduling strategies where patients are scheduled for a given clinic session, but not assigned an appointment time until just prior to the session. That way, patients could be arranged so that the shortest appointments were first, followed by the mediums and longs.

• 5.

  EHR timing data allows for detailed evaluation and improvement of clinic workflow modifications. EHR data can be used to measure metrics such as average patient wait times, session lengths, and adherence to scheduling templates which allows for a detailed evaluation of a new scheduling strategy. While this study focuses on a new scheduling template in an outpatient ophthalmology clinic, EHR timing data can be used to analyze any workflow modification in any setting. This timing data can determine if a new modification is having the desired impact and why, leading to potential improvements. Without using EHR data, metrics would be gathered manually through observations, which is prohibitive for any extended test period and may not provide as rich of insights.

Limitations

This study has the following limitations: 1) It uses EHR timestamps for measuring exam times, calculating patient wait times, and session lengths. If the provider’s or staff’s EHR use doesn’t coincide with their exams, the wait time and session length calculations could be wrong. Previous studies have validated this approach, but errors may still occur. 2) The new schedule was studied for a 3 month time period—its performance may vary if a longer test period including multiple seasons was used. We plan to evaluate the schedule again, particularly after more scheduler training and after relaxing the dilation criteria. 3) We focused our evaluation to quantitative methods, but qualitative evaluation will provide more insight into the benefits and challenges of the new schedule implementation. A followup qualitative study is planned to address this limitation. 4) OHSU’s policy of equal length scheduling blocks eliminates other potential optimal scheduling policies, such as variable length appointments. 5) We assume that long appointments can be scheduled near the end of the clinic; if this is not the case, then other strategies would need to be used to mitigate the potential delays incurred by long patients at the start of the clinic. Future studies are needed to investigate strategies such as variable length appointments or empty “catchup” blocks for early long appointments.

Conclusions

Simulation models predict that scheduling according to length—short appointments first, followed by longer appointments—significantly decrease patient wait times and session length. Real world implementations of these scheduling rules, however, can be challenging, but using EHR timing data can evaluate the effects of compliance. Secondary use of EHR data allows for a detailed analysis of a new scheduling template, with the goal of identifying critical aspects of the modification and those that can be eliminated. This study focuses on a scheduling template in outpatient ophthalmology, but the data analysis techniques apply to the evaluation of any workflow modification in any setting.

Acknowledgements

This study was supported by grants K99 LM12238 and P30 EY10572 from the National Institutes of Health (Bethesda, MD), and by unrestricted departmental funding from Research to Prevent Blindness (New York, NY).

References

1. Blumenthal D, Collins S. Health care coverage under the Affordable Care Act: a progress report. N Engl J Med. 2014;371:275–81. [PubMed] [Google Scholar]

2. Hu P, Reuben DB. Effects of managed care on the length of time that elderly patients spend with physicians during ambulatory visits: National Ambulatory Medical Care Survey. Med Care. 2002 Jul;40(7):606–13. [PubMed] [Google Scholar]

3. Shea S, Hripcsak G. Accelerating the use of electronic health records in physician practices. N Engl J Med. 2010;362(3):192–195. [PubMed] [Google Scholar]

4. Chiang MF, Boland MV, Margolis JW, Lum F, Abramoff MD, Hildebrand PL. Adoption and perceptions of electronic health record systems by ophthalmologists: an American Academy of Ophthalmology survey. Ophthalmology. 2008;115(9):1591–7. [PubMed] [Google Scholar]

5. Boland MV, Chiang MF, Lim MC, Wedemeyer L, Epley KD, McCannel CA. Adoption of electronic health records and preparations for demonstrating meaningful use: an American Academy of Ophthalmology survey. Ophthalmology. 2013;120(8):1702–1710. [PubMed] [Google Scholar]

6. Chiang MF, Read-Brown S, Tu DC, Choi D, Sanders DS, Hwang TS. Evaluation of electronic health record implementation in ophthalmology at an academic medical center (an American Ophthalmology Society thesis) Trans Am Ophthalmol Soc. 2013;(111):34–56. [PMC free article] [PubMed] [Google Scholar]

7. Gupta D, Denton B. Appointment scheduling in health care: Challenges and opportunities. IIE Trans. 2008:800–819. [Google Scholar]

8. Hribar MR, Biermann D, Read-Brown S, Reznick L, Lombardi L, Parikh M, et al. Clinic Workflow Simulations using Secondary EHR Data. In: AMIA Annual Symposium Proceedings. 2016 [PMC free article] [PubMed] [Google Scholar]

9. Rohleder TR, Klassen KJ. Using client-variance information to improve dynamic appointment scheduling performance. Omega. 2000:293–302. [Google Scholar]

10. Reznick L, Hribar MR, Read-Brown S, Yackel TR, Chiang MF. Computer Simulations For Optimizing Clinical Workflow In Pediatric Ophthalmology. In: Proceedings of the American Association for Pediatric Ophthalmology and Strabismus Annual Meeting. 2016 [Google Scholar]

12. Read-Brown S, Hribar MR, Aaker G, Reznick LG, Yackel TR, Chiang MF. In: AMIA Annual Symposium Proceedings. San Francisco: CA; 2015. Identification of Variables that Predict Visit Times for Analyzing Ophthalmology Clinic Workflows. [Google Scholar]

13. Gupta D, Denton B. Appointment Scheduling in Healthcare: Challenges and Opportunities. IIE Trans. 2008:40. [Google Scholar]

Articles from AMIA Annual Symposium Proceedings are provided here courtesy of American Medical Informatics Association