Canada is implementing a network of large data repositories designed to store all diagnostic imaging studies generated in hospitals and clinics across the country. There will be 18 such storage units covering all provinces, called DI-r (Digital Images Repositories). The goal of this project is to make all imaging studies available to healthcare professionals, wherever they are, increase efficiency and decrease redundancies in facilitating comparison to previous.
The DI-r will be integrated with the Electronic Patient Records (EPR) and their architecture is based on accepted standards. They will store images for the life of the patient. This will result in a considerable amount of images and therefore drive the need to use lossy compression to decrease the volume of data, to save money and improve transmission times.
Steps Taken for Successful Implementation
1. Assess the need for lossy compression:
Even if the cost of storage is dropping, the savings are surpassed by the increasing amount of data. If we consider a typical radiologist interpreting 40 CT studies per day, assuming 800 images per study with three windows, in three planes (axial, coronal and sagittal), he may review 300,000 images per day, which means 150 GB of storage. If we add comparison to previous, our radiologist may have to review more than 600,000 images in a single day, which would represent 6.6 images per second for 24 hours, which is impossible. This also means that if the cost of storage may have dropped 200 times in 15 years, the amount of data has increased 200 times, and that at the end of the day, there is no savings.
We also have to take into consideration the high cost of operation and time-consuming data migration, which consumes most of the ongoing maintenance budget. We estimate that an average 45 million diagnostic imaging exams were performed last year in Canada, and that the rate of increase has been a steady 5 percent a year. Canada Health Infoway (CHI) is implementing large data storage units (DIr) across the country to archive all medical images generated in hospitals and clinics and the intent is to retain these images for the life of the patient. Using irreversible compression at 10:1 could save 100 million Canadian dollars per year.
But this is not all; if access to high bandwidth becomes increasingly available in local hospital networks, it is still premature to expect any health professional to use 100 mbps connections on their computers. EHR networks cannot support large medical images and timely access to diagnostic images requires adequate level of compression. This applies also to teleradiology, where turnaround times for report delivery must be as short as possible.
2. Review the types of compression to use:
There are two ways to compress images: lossless or reversible where the decompressed image is numerically identical to the original, such as RLE (Run Length Encoding), or lossless JPEG or JPEG-LS (where JPEG stands for Joint Picture Expert Group), but where only low compression ratios can be obtained, usually no more than 2 or 3:1; lossy or irreversible, where redundant data are discarded during the quantization phase and cannot be recovered, but allowing much higher ratios.
We considered two of the most popular algorithms supported in DICOM, lossy JPEG and lossy JPEG 2000. Technical specificities are not in the scope of this article, but it is worth a reminder that JPEG 2000 is more flexible than lossy JPEG as it supports more image formats, when lossy JPEG is limited to 8 and 12 bit images.
3. Prove the usability of lossy compression:
To prove the usability of lossy compression, Canada Health Infoway looked at official statements, did an extensive review of the literature, asked for legal assessment from two reputed lawyers, and last but not least, in conjunction with Canadian Association of Radiologists (CAR), asked us to conduct a large-scale clinical evaluation.
None of the official positions we looked at prevent the use of lossy compression: the Foods and Drugs Administration(FDA) asks for the compression schemes to be identified by name and the compression ratios to be specifically stated; FDA only excludes lossy compression for mammograms. The American College of Radiology (ACR) endorses the use of compression to increase transmission speed and reduce storage requirements under the direction of a qualified physician, with no reduction in clinically significant diagnostic image quality. DICOM supports lossy compression, but has no position as to particular use of compression.
The literature review covered hundreds of articles published in reputed medical and engineering journals and all concluded that lossy compression could be used with no significant impact on image quality and no loss of diagnostic accuracy within acceptable ratios; some authors even demonstrated a gain in diagnostic accuracy linked to the denoising effect of low level lossy compression. But these articles covered only limited areas of imaging and the evaluation techniques were different from author to author, therefore a need exists for more consistency.
The legal opinions converged to state that lossy compression can be used provided that appropriate ratios are used, there is no clinically significant loss of data, lossy compression is used in primary reading (avoid altering records after primary reading), the technology is not adopted recklessly and due diligence is applied, such as: literature reviews, education, supervision, and finally that technology is used appropriately.
Our large scale evaluation was designed to standardise the disparate evaluation techniques that we encountered in the literature and we opted for a methodology based on two accepted techniques:
- Diagnostic accuracy with ROC (Receiver Operating Characteristic) analysis where the reader is presented with an image, not knowing if it is compressed or not, and asked to identify a pathology, state in which quadrant of the image s/he sees the pathology and give a confidence rating;
- Subjective assessment where the compressed image is compared to the original and ranked on scale of 1 to 6.
We covered five modalities (computed and digital radiography, ultrasound, computed tomography, magnetic resonance and nuclear medicine) and seven anatomical/radiological areas (angiography, body, breast, chest, musculoskeletal, neurology and paediatrics). We looked at the two most commonly used compression algorithms, JPEG and JPEG 2000, at three different ratios of compression.
We gathered more than 2,500 exams and enrolled more than a hundred radiologists from coast to coast, with all Provinces represented by one, and had 23 reading sessions with at least five radiologists per session, in order to have enough statistical power. We had designed a dedicated viewer and the radiologists received the images on a CD, that they had to display on the workstations they used for reporting, but the results were collected on-line on a specially designed server application. After two years, we developed a set of recommendations demonstrated in table 1.
4. Publish Canadian standards:
The Canadian Association of Radiologists (CAR) had endorsed the use of Irreversible Compression at its April 2004 General Assembly, but waited for the results of our evaluation to be presented to the Executive Committee for approval. The Canadian Standard was issued in June 2008.
This standard validates the use of irreversible compression under certain defined circumstances and for specified examination types. It gives indications for operational implementation, and stipulates among others that irreversible compression, when used, must be considered part of image processing and as such the compressed images are those that are used for interpretation and become the legal record. The compressed images used for interpretation are those that are subject to the requirements for archival storage for the legal retention periods. There is no requirement to maintain raw or uncompressed images. Which means that radiologists will have to report on compressed images, which they do anyway when they use teleradiology or report from home.
The standard was modified when validation of the use of irreversible compression for thin slice CT was completed in early 2010. More updates are expected when more validation is completed, mainly for 3D imaging.
5. Implement and identify issues:
We have started to implement lossy compression in different settings, such as several hospital systems and a large teleradiology network, and of course we started to encounter issues, for which some had not been anticipated. Here are just a few, but no doubt more will appear when adoption of lossy compression becomes more universal.
Implementation of lossy compression would be much easier if we had allowed radiologists to report on the original image and then apply compression when sending the images for long-term storage, but the Standard specifies that images stored must be the same as images read. Therefore, is the PACS able to lossy compress right away, those images received from the modalities for display on the workstations, or should we compress at the modality or between the PACS and the modality? We are currently assessing the ability of multiple vendors to perform compression this way.
There are also compatibility issues, as we need to make sure that there is enough standardisation between PACS vendors to visualise the images on any platform, which means supporting the various algorithms used. This will be even more critical when we start importing images from the DI-r into the local PACS.
The adoption of irreversible compression by an organisation or group of radiologists must be subject to the supervision of a qualified radiologist who must ultimately determine whether the image quality after compression has been applied is acceptable. But how will the radiologist perform quality control? He must be aware of the type of compression used, ensure that the vendor has extensively tested compression after implementation and that recompression is not applied to already lossy compressed images. This is easy, but it becomes more complicated when it comes to access the original image and compare with the compressed image to ensure consistency. More testing is required as modalities evolve, with more advanced processing, new sequences, and new technologies.
Canada is in the process of implementing lossy compression of medical images on a large scale, as it is required to optimise the use of the DI-r, for storage and communication. Even if the CAR published its Standard in 2008, there are still a number of issues that we are addressing with the support of Canada Health Infoway. International collaboration will certainly help greatly to foster adoption of lossy compression thanks to initiatives like the one led by the European Society of Radiology; the International Workgroup on Lossy Compression which met under the auspices of ESR and decided to rename Lossy Compression as Diagnostically Acceptable Irreversible Compression (DAIC) and which has since issued a white paper.