MIS Dashboards:Buy or Built

Author

Christopher Bain

is Healthcare Management

Informatician, WCMICS,

Melbourne, Australia

The growing trend in Europe (and much of the world) to corporatise hospitals and set performance targets has been accompanied by an escalation in management information requirements. One answer to this challenge lies in Management Information System (MIS) Dashboards. However, as with much else in healthcare IT, there is a major debate about how add-on MIS 'dashboards' compare to customised configurations built in house. Given below is a whistle-stop tour by a specialist in predictive technologies in healthcare management on key issues related to hospital management dashboards – their strengths, weaknesses and what to consider before buying or building a dashboard application.

You’ve seen them, you know what I’m talking about – they are colourful, they have moving dials and flashing lights…… and they are often web based – you can even look at them at home or from a conference – they are ‘dashboards’.

Approving expenditure to purchase or construct a management dashboard application for a healthcare facility may seem a relatively simple task. Particularly if all it needs to do is ‘hook up’ to existing ‘transactional systems’ and start displaying all that wonderful data floating around, to assist in managing a facility. ‘Lets get on with it!!’ you say. But is it really that simple?

To answer this requires a closer look at the situation. Like any major investment, there is a strong case for caveat emptor (let the buyer beware)...and even if you follow that adage, in your reflective moments you may still ask yourself ...’why on earth did we do this?’

So What is a Dashboard?

A dashboard, fundamentally, belongs to the broader category of Executive Information Systems (EIS) which integrate data from a variety of different sources and present it in an optimal format to decision makers.

The essential philosophy behind such systems is to help managers by providing structured information necessary to quickly make decisions which require informed judgement. They are designed to not only be user friendly; the best, indeed, claim to have a near ‘intuitive’ element in -built for users.

Why do Hospital Managers Need Dashboards?

Hospital managers stand to benefit from dashboards for a number of reasons, not the least being the environment in which they operate. Across the world, hospital managers are being asked to track and act on more information, in more dimensions then ever before.

An additional complexity is the lack of consistent evidence, or even guidelines, about the best decision choices in any given situation, which is partly a reflection of the complex permutations and combinations of scenarios driven by different facility types and case mix, different funding mechanisms and different regulatory environments in which hospitals sit.

Given this complex environment, it is worth underlining the following comments in a recent Harvard Business Review article titled ‘The secrets to successful strategy execution’. The article, reflecting the work of a major consulting company, noted that: “The single most common attribute of…… (successful) companies is that their employees are clear about which decisions and actions they are responsible for. As a result, decisions are rarely second-guessed, and accurate competitive information quickly finds its way up the hierarchy and across organisational boundaries. Managers communicate the key drivers of success, so frontline employees have the information they need to understand the impact of their day-to-day actions.”

Dashboard and Information Dimensions to be Considered

A number of dimensions need to be considered in contemplating what a healthcare IT manager need from a dashboard. Often, these have fallen out of organisational performance monitoring frameworks like the balanced scorecard. I would argue that users can get more from a dashboard in the current technological environment by not limiting themselves to overly restrictive frameworks.

For example, the temporal dimension is a newer one to consider; many dashboards historically have used warehoused data which is out of date at the time of usage – but nonetheless potentially helpful in understanding both the recent and distant past. More and more systems are capable of delivering real time data to users. Some systems (and their attached dashboards) are now capable of delivering forecasts into the immediate or near future – for example about issues such as hospital bed usage and occupancy.

Other considerations can also be thought of as breadth, depth and complexity…… and may act as a useful guide when communicating with vendors or in-house developers.

‘Breadth’ means the range of data items and content areas (eg: finance, quality, access, HR) to be included in the dashboard.

Depth entails an ability to drill down on areas of concern. So for example, at COO level, if hospital Emergency Department waits are lengthening to unacceptable levels, what ability does a COO have to drill down by interacting with a GUI to find which care unit or business areas are contributing most to the delays? This takes a dashboard beyond a purely ‘data display’ system to one that can inform decision making and action in more practical ways.

In relation to complexity, there is some broad guidance on what constitutes useful dimensions (or quadrants) of data to incorporate into a dashboard view for healthcare managers. A report by John King and Jeanne Jenkins in the April 2008 issue of Healthcare Financial Management titled ‘Information management: why it’s vital to effective service line operation’ observes: “It is essential for healthcare financial managers to understand and monitor five areas of information critical for effective service line performance tracking: Market share, Operational performance, Physician performance, Clinical documentation and coding, Patient satisfaction.”

These areas, however, are hardly simple or straightforward when extrapolated to a global context. A study by a University of Maryland team on a web based dashboard in managing operating theatre performance notes: “The challenge lies in aggregating and displaying these data in an easily accessible format that provides useful, timely information on current operations.” [Surgical Innovation (Vol. 15, No. 1, 2008)].

It is also known that individual and role-based preferences affect what data is relevant and what are the relevant views to put into a dashboard. In the healthcare area, a variety of studies have pointed out highly variable stakeholder views about ‘report card’ elements to be considered in different con texts and scenarios. As a result, reaching agreement about what information content should be included in any kind of reporting mechanism (card or dashboard) can be problematic, and an area of real concern in the development of dashboards for hospital managers.

In addition, there always are new pieces of information which managers are challenged to keep track of, ranging from the useful to the indispensable – such as best practices. In real life, the trade off lies in the importance of locally keeping track of such information versus the ease and cost of implementing changes in the technical environment and business process to allow that information to be measured, displayed and actioned.

The British National Health Service has seen work on using advanced analytic techniques, specifically an SPC (Statistical Process Control) technique called CUSUM (cumulative sum) charting in a web interface to compare in-hospital mortality, length of stay and emergency readmission rate acrosssites and organisations. This illustrates an example of where such applications may head in the future.

Surely You Just Hook it Up and Go from There…

So if a hospital decides to proceed down the path of investing in this technology, is there anything else managers need to think about…… ‘can’t we just plug it in and get going’? One critical consideration is the business context in which the technology will be hosted.

There have been attempts to provide advice in this area. Dr. John Glaser, CIO of Partners Healthcare in the US, suggests five key steps in setting up an appropriate environment in which to use business intelligence technologies :

1. Establish business needs and value.

2. Obtain buy-in from managers.

3. Create an end-to-end vision.

4. Establish BI governance.

5. Implement specific roles for managing data quality.

Environmental and Contextual Considerations

One key consideration in purchasing or developing a dashboard are the skill sets and knowledge bases of hospital managers  in other words, the technologies they are (and feel) comfortable with. There is evidence that healthcare managers have a poor knowledge of predictive systems and techniques. Exacerbating this is a considerable level of resistance to new technologies, among both hospital employees and administrators.

A second factor concerns the specific healthcare organisational environment (the kind of facility and the services provided) and the implications of these for dashboard purchase or development. The metrics to be include in the dashboard and the relevant time and drill down dimensions may vary enormously depending on such questions.

Cost and obsoloscence should also be taken into account. The main question here is how current and future data requirements relate to the needs of funders, broader corporate requirements and those of regulators. There is little point, for example, in having to run multiple dashboards, or a dashboard to meet a range of separate and non-overlapping sets of needs around management information.

Challenges and Constraints

Several challenges face both in-house developers of dashboards in hospitals and purchasers of off-the-shelf systems. On a technical level, there is the challenge of access to, conversion and integration of, disparate data sources – ranging from historical paper-based collection systems and operational databases, to warehoused data at the other end of the spectrum. Another common challenge is to find a definition for optimal data quality – for instance, real-time occupancy information versus the load this places on the system.

From an organisational perspective, the issue of cost (and hence delivery mechanism) is always a major consideration. Public healthcare in particular is often cash strapped. As a result, ‘expensive’ solutions, especially those impacting on end user devices or client side software, can be problematic; web based applications in a thin client model are increasingly seen as an answer to such challenges.

Resistance to change, and adequate levels of staff engagement are other issues. The role of the clinician ‘leader’ or ‘champion’ is widely recognized as a major factor in clinical systems projects. Management leaders and early adopters can often only assist in the implementation of a developed or purchased solution.

The framework and business processes attached to such initiatives also remain crucial to take into consideration. ‘We have all this information available at our fingertips…… what next‘?

This question is addressed below.

So You Have a Dashboard ….Now What……….?

In the process of deciding to purchase or build a dashboard a quick checklist of questions which healthcare IT managers are advised to consider is provided below:

Ó Does the dashboard relate to an organisation’s performance framework – including financial, quality, access, clinical governance and other dimensions?

Ó How does it relate to business processes – what actions will be taken on the back of the data ?

Ó What ongoing quality processes are in place in relation to the data displayed – and how does that flow on to the feeder systems?

Ó Can the system be easily modified in the event of new

board and executive priorities or changes in funding or regulatory frameworks?

Ó How do prospective users evaluate a dashboard’s utility. It is a tool…… is it working for you or not, and if it isn’t – what are you going to do about it?

Buy Versus Build

Should a hospital IT manager buy or build a dash board ?

As with any MIS or EIS system, it is self-evident that executives be closely involved in the choice/purchase or development of a dashboard. Given the demands on their time and variable level of comfort with computer systems, the attractions of a build scenario with in-house staff are often significant.

However, there is a major trade-off here: pre-existing solutions or those requiring minimal local customisation already carry the benefit of being ‘tried and tested’ with a range of executives and managers from other organisations, thereby minimising the chances of not meeting the users (managers) needs.

There are a number of commercially available applications available in this problem domain, with different emphases, pros and cons.

Other considerations in the buy versus build equation include:

Ó Cost – for instance large licensing fees.

Ó Potential skills transfer or development for internal staff versus outsourcing and being vendor dependent on an ongoing basis.

Ó Dashboards may be better if constructed by embedded staff with the necessary technical, clinical and local organizational knowledge.

Ó Is the system easily maintained or improved moving forward – in terms of functionality or content, depending on user preferences, technological availability (e.g.: new end user devices such as tablets) or organizational needs?

In the final analysis, dashboards can clearly provide an opportunity to assist in hospital management, However, like any significant investment or IT implementation, the key to success lies in making sure users get what they really need, while not getting burnt along the way.