Evidence-based research demonstrates that overcrowding in emergency departments causes ambulance diversion, increased hospital lengths of stay, medical errors, increased patient mortality, financial losses to hospital and physician, and medical negligence claims.
Many hospitals still do not anticipate and prepare for the next day’s volume and admission through the emergency department. And yet, contrary to the conventional wisdom that emergency patient volume is highly unpredictable, the number of admissions per day can be predicted with remarkable accuracy.
Forecasting presentations and admissions is a relatively easy solution. When implemented, it can protect everyone’s access to emergency care.
In April 2008, the American College of Emergency Physicians (ACEP) published a report identifying solutions to the practice of ‘boarding’, or holding, patients admitted to the hospital in the emergency department, which is the primary cause of overcrowding. A boarded patient was defined as a patient who remains in the emergency department after the decision to admit him or her to the hospital has been made. Most emergency departments in the world are critically overcrowded and unable to respond to day-to-day emergencies, and the proposed solutions address the growing global crisis that is harming public access to lifesaving emergency care.
Solutions with the highest impact in reducing boarding and improving the flow of patients through emergency departments are:
Move emergency patients who have been admitted to the hospital out of the emergency department to inpatient areas, such as hallways, conference rooms;
Coordinate the discharge of hospital patients before noon, and
Coordinate the scheduling of elective patients and surgical patients.
Research Study: A Clinically Usable Software Package
The main aim of our study was to develop and validate a clinically usable software package that accurately predicts the number of admissions sourced from emergency department cases on any given day of the year, taking into account peak periods such as public holidays. The primary outcome measure was the accuracy of forecasts when validated against historical data from two differing hospitals. The resultant Patient Admissions Prediction Tool can assist with the allocation of inpatient beds to alleviate overcrowding.
The modelled data consisted of five years of ED presentations and admissions (1/7/02 – 30/6/07) from two hospitals chosen for their different demographic characteristics. Hospital A is a 280-bed regional facility, located 120km away from a major tertiary referral centre and services an area of approximately 410,000 km2 with a resident population of about 280,000. Hospital B is a 750-bed busy urban facility and services a rather itinerant population of around 500,000. It is host to several annual events that attract large amounts of tourists. Despite their differences, presentations numbers for both hospitals across the five years were similar (218,000 – regional Hospital A, 278,000 – urban Hospital B). The urban hospital has a higher rate of admissions (33%) than the regional hospital (20%).
Many useful characteristics that can help shape health management practices have been identified from the data. For example, the date and time when admitted patients leave the ED, indicating the times of highest demand on hospital beds; patient arrival time in the ED, which represents a staffing impact with workload; and the days of the week which represent higher ED workloads and hospital bed demand. The data also enables the analysis of ‘frequent-flyers’ – those patients who presented multiple times during the analysis period.
From the analysis of this data, we have been generating forecast estimates and associated confidence intervals based on several forecasting approaches and validating the forecasts against actual data. The project also included packaging the most accurate technique into a stand-alone software application.
The data includes date and time of admissions which provides useful information on peak admission times experienced within the EDs. Figure 1 indicates the times of highest demand on hospital beds (admitted patients leaving the ED), indicated by a brighter colour. The vertical columns of the plots indicate the hour of day, and admission numbers are indicated by the colour bar. It is apparent that the highest demand for hospital beds occurs in the afternoon and into the evening. Every row on the left-hand plot represents a day from 1/7/02 – 30/6/07, while the rows on the right-hand plot indicate monthly averages throughout the study period. Similar assessment has been done for discharge times of all presentations (not just those admitted) and also for arrival times within the ED. The skew of the data to the end of the day is apparent.
Another point of interest is the time of arrival in the ED, as this represents a staffing impact with workload. Figure 2 shows the ED discharge time for the admissions that are shown in Figure 1. This discharge time refers to the time patients leave the ED and require a bed, as opposed to discharges leaving hospital. It also indicates the arrival time for this group, which peaks around 11:00hrs. However, admitted patients make up only a small subset of all the patients seen in the ED, and the two curves in the upper portion of the plot represent all presentations. We can see that the mean peak discharge time lags behind the peak arrival time by around eight hours and again see the skew of the data to the end of the day.
The hourly fluctuations of the data has also been studied using box-plots as shown in Figure 3, which show, for example, the quietest time (8am) and the busiest time (5pm) for admissions. Median, upper and lower quartiles and outliers are represented in the plots.
It is also of interest to determine the days of the week that represent higher ED workloads and hospital bed demands. For example, Figure 4 shows the mean and 95% Confidence Interval band for the daily and monthly trends in the arrival time of all presentations (Left) and for admitted patients (Right) at the urban hospital. The busiest days for presentations are over the weekend and Mondays.
Considering the arrival time for just those patients that are admitted, it can be seen that Mondays and Tuesdays are the busiest days. There has also been an overall increase (approximately 40%) in the number of patients presenting over the five years. Interestingly the trend over all the months-of-analysis for admitted patients shows a plateau effect, which could be attributed to bed capacity being reached, or the adoption of hospital avoidance strategies.
From the analysis of this data, we have been generating forecast estimates and associated confidence intervals based on several forecasting techniques. This modelling included stepwise multiple regression, exponential smoothing and Box-Jenkins Autoregressive Integrated Moving Average (ARIMA) models.
In our study, accuracy was treated as the main criterion for selecting a forecasting method, and the metric used in our evaluations was the Mean Absolute Percentage Error (MAPE). Data was divided into a training set and evaluated against a separate holdout set. The evaluation dataset spanned one year (364 days), allowing accuracy to be measured across summer and winter months and varying forecast horizons. The effect of varying the size of the training dataset was analysed and training lengths of one, two, three, four and 4.3 years were assessed. Also computed were the width of 95% prediction intervals (± x admissions) and the number of misses outside this prediction interval. This provides the user of the forecasts with worst and best case estimates and a sense of how dependable the forecast is.
Presentations to the ED and subsequent admissions to hospital beds are not random and can be predicted. Forecast accuracy worsened as the forecast time intervals became smaller: when forecasting monthly admissions, the best MAPE was approximately2%, whilst for daily admissions this was 11%, for four-hourly admissions: 41%, and for hourly admissions: 51%. Presentations were more easily forecast than admissions (daily MAPE ~7%). Subgroups within the data with more than 10 admissions or presentations per day had forecast errors statistically similar to the entire dataset.
The best method for forecasting data used in our study was averaging (smoothing) using a fouryear training period, and potential exists for the model to be implemented in other facilities. Sensitivity analysis showed that smoothing techniques worked best with as much historical data as possible, but regression was best with the most recent data.
When compared to existing prediction models at one of the hospitals, the new techniques shave Mean Absolute Percentage Error of daily admission predictions from 20% to 11%. Based on a mean admission rate of 50 admissions per day, this improvement in forecasting performance corresponds to ±5 beds. When a new ED wing opened in the catchment area, the error from existing predictions worsened to 30%, whilst error from the new models was 11.8%. This improvement in forecasting performance corresponds to ±9 beds.
The admissions and presentations predictive modelling has been implemented as a standalone software application. The programme has been designed to run in an unsupervised manner, where forecasts for admissions and presentations are refreshed every hour. It is also possible to run the programme once or repeatedly for a specific date. Initially this choice is determined from the welcome screen, along with the confidence limits to adopt for prediction intervals. The project has also resulted in the development of a User Experience Base via detailed consultation with ED and bed management planning staff to identify user expectations and functional requirements for a prediction tool.
As a result of this study, it can be concluded that accurate forecasting tools are important aids to many areas of hospital management, including elective surgery scheduling, bed management, and staff resourcing. We have produced a tool that can predict ED admissions and thus allow appropriate allocation of in-patient beds and operating theatres. With regular feed of site specific retrospective data, this tool should have considerable utility for acute facility bed management and health service planning.
The project team have identified an extension of this project to formally evaluate the impact of the prediction tool in these areas. Such evaluation is essential to quantify the potential benefits of the model such as reduced ambulance bypass occurrences and elective surgery cancellations. Future research into this aspect has recently commenced.