HealthManagement, Volume 1 / Issue 4 Winter 2006
Authors
Dr. Karl A. Stroetmann (top picture) empirica
Communication and Technology Research
Alexander Dobrev
empirica Communication and Technology Research
When the eHealth IMPACT Study began, was a great learning
experience for everyone involved, both the study team and our what was the main
objective – was there anything you set out to prove or disprove?
There is an awful lot of rhetoric about the great benefits from all kinds of eHealth solutions; simplistic, if not sometimes naive calculations of prospective cost savings to be expected in the near future; reports on pilots, but very little, if any, reliable evidence from proven, longer-term, routine use of eHealth. This lack of evidence is retarding the expansion and diffusion of eHealth, and is a barrier to achieving stronger political support. The objective of the study was to provide some of the missing evidence to fill this gap.
In January 2005, the eHealth IMPACT study (www.ehealth-impact.org) was awarded by the European Commission to empirica and its partners. We set out to develop a generic method for evaluating the economic performance of eHealth solutions; identify good practice in routine use of eHealth; and evaluate ten proven sites, to provide an insight into the actual economic impact of effective eHealth solutions. This research was a great learning experience for every-one involved, both the study team and our many partners at each of the sites.
Did the objective change during the course of the study?
Not really. Over the course of the project, the study went along two different but parallel paths. One developed, validated and refined the method, and then applied it. The other identified, selected and collected reliable, verified data on a number of good practice cases for a public database. Initially just part of identifying ten proven solutions, collecting even basic data on a large number of good practices turned out to be much more time consuming and challenging than we had anticipated.
What were the main lessons learned from the study?
The study brought a lot of insights at different levels – about the methodological approach and execution of economic evaluation in the eHealth domain; about the state of play in data availability, internal reporting and controlling at healthcare organisations; and about what makes an eHealth initiative a success. The third level is the most important one, because it can be used to improve existing practices that do not deliver the expected benefits, and for support in business case development, investment decisions, and implementation management.
An important policy result of the ten cases, as a combination of individual solutions, was that eHealth applications, by addressing specific needs, can help increase benefits to health at broadly stable costs. Realising this potential depends on a few key factors:
1) A strong vision, focusing on citizen and healthcare needs should guide a flexible, regularly reviewed and adapting eHealth strategy. It must address concrete needs, and combine this with a pragmatic approach of setting achievable shorter-term goals within an eHealth investment dynamic – a ‘big bang’ with a short lead-in preparation period does not seem to work.
2) A comprehensive assessment is needed that takes into account the benefits and costs of all stakeholders, particularly those of citizens and for society as a whole. Considering a financial return on investment from just one perspective may lead to suboptimal overall decisions.
3) The optimal use of ICT-enabled solutions and comprehensive benefits realisation usually requires substantial changes in clinical and working practices. Simply replacing paper by electronic means will not be enough. New models of providing healthcare, or secondary uses of information recorded once, need to be considered.
4) Strong clinical leadership, good organisational change management, multi-disciplinary teams and well-grounded ICT experience will make the difference between success and failure.
5) Beneficial eHealth investment is like a good wine. It takes a considerable amount of time to mature and fully develop its potential. Policy makers, healthcare providers and other decision makers should take this longer-term perspective.
Were there any surprises in your findings?
One small surprise was the wide range of time for eHealth solutions to deliver a net economic benefit. The first year of net benefits was between two to eight years, depending on the complexity and scale of the specific solution. The average was four years.
Another surprise was the range of benefits distribution. In all ten cases, at least two stakeholder groups shared the benefits – health provider organisations, including the healthcare professionals, and citizens. Citizens reap about 45% of the benefits on average, but sometimes as much as 95%, as in the case of Kind en Gezin’s children vaccination support system in Flanders, Belgium. This has a profound implication for planning eHealth investments – citizens should be taken into account in decision-making. Benefits to them, not usually in the form of cash, can make the difference between an investment being economically worthwhile and one that does not seem to be worth it. An important note: here we are talking about an economic, cost-benefit perspective, not a financial return on investment.
With regards to the economic analysis – how were the benefits of the eHealth solutions assessed?
We identified seven generic types of benefits.
The first five are components of health services quality:
+ better informed citizens and carers, for example preventing citizens from becoming patients in the first place;
+information designed to streamline healthcare processes;
+ improved timeliness of care, not always faster treatment, but at the optimal point in time;
+ contributions to patient safety and risk management; and
+ improved effectiveness of healthcare service, treating those who actually need treatment and not treating those who do not, as well as facilitating multi-disciplinary consultations optimally matching specific diseases.
The other two types of benefits are:
+ improved access to services and – spatial, but also independent of social background, financial situation, etc.; and
+ improved overall economic efficiency, reflecting optimal use of resources.
Benefits were identified according to each case, and monetary values were assigned to them. An example is time savings. For healthcare workers, these were valued at the relevant employment cost. Another example is the avoided cost of achieving the same seven types of benefits, but by conventional, non-ICT methods. This usually involves considerable increases in staff costs.
In some cases, proxies were derived from similar services, and willingness to pay estimates, and expert estimates were also used. A contingency rate increased all costs and decreased all benefit values to reflect potential optimism bias and unaccounted indirect costs. The size of these adjustments depended on the extent to which we had to rely on estimates. Finally, all the results underwent a rigorous sensitivity analysis.
What are the implications of this study concerning future healthcare IT investment decisions?
Probably the most important implication for future eHealth investments is to include cost-benefit analysis (CBA) from a holistic perspective, i.e. accounting for all relevant stakeholders, into the decision making process, be it at the local or the national level. As our colleague, Tom Jones, who collaborated with us on this study, discussed in his article in the last edition of Healthcare IT Management (Volume 1, Issue 3, page 34), CBA is not enough, but it is essential for identifying optimal investment options towards the ultimate goal of health services: healthier people.
The eHealth IMPACT method will be expanded to allow for ex ante assessments and to measure the impact of different risks. This will provide a validated methodology for prospective business cases across the whole eHealth domain.
How transferable are the results of the eHealth IMPACT study to other eHealth activities across the EU?
The method is easily transferable. It was designed to cover the whole eHealth domain in its widest definition. The ten selected sites consider very different solutions – from classical telemedicine applications, through highly complex ICT-enabled Electronic Patient Record (EPR) and meta-search engine solutions in a hospital, to insurance validation and reimbursement procedures, and supply chain management. This breadth of applications is deliberat and has enabled us to test and improve the methodology, and to also be simultaneously rigorous and generic.
Numbers from the ten eHealth IMPACT evaluations cannot be simply transferred to another site. At each new site, all the relevant cost and benefit variables need to be identified, and then used to set up a detailed empirical data model over the full time period, which could be up to 15 years. Economic analysis of eHealth solutions is not an exact science.
The aim is to show whether the estimated benefits exceed costs with an acceptable degree of certainty, or not. The exact individual numbers depend on the assumptions and expert estimates, whereas the overall outcome is usually not very sensitive to them.
Transferability is also limited because the ten evaluated cases were not selected at random. They had a reputation for good performance. This means that the results are showing the potential of eHealth investments, but only if designed, implemented and managed effectively. Different settings will lead to different results, even if the same, proven eHealth systems are transferred.
For more details and the case studies, please see: www.ehealth-impact.org.
