Synthetic data alone did not reliably support colorectal cancer survival prediction when models were transferred to a separate hospital dataset, while fine-tuning with local clinical data produced stronger results. A modelling experiment published in the Journal of Medical Internet Research compared synthetic-data pretraining, direct transfer, domain adaptation and ensemble approaches using Korean cancer datasets. Models trained only on synthetic data generally performed worse than hospital-only baselines when applied without adjustment. By contrast, domain adaptation, which fine-tuned pretrained models using hospital data, improved discrimination and calibration in many tested settings. The findings position synthetic data as a supplementary training resource rather than a substitute for real clinical data.
Synthetic Pretraining and Local Adaptation
The modelling framework addressed a common barrier in clinical AI development: access to sufficient patient data for training, testing and external validation. The synthetic dataset came from the National Cancer Data Center colorectal cancer clinical library artificial dataset, generated from real-world information collected at three Korean cancer institutions. Hospital data came from patients who underwent colorectal cancer surgery at Hwasun Jeonnam University Hospital.
The primary outcome was seven-year overall survival. The synthetic dataset contained more variables than the hospital dataset, but model training used only twelve variables common to both sources. These included demographic, tumour, treatment and pathology-related factors. This restriction allowed a controlled comparison between synthetic and hospital data, although it also limited the amount of clinical information available to the models.
Four model variants were assessed. A hospital-only baseline used hospital data for training and evaluation. A zero-shot model used synthetic data alone and was applied directly to hospital data. A domain adaptation model was pretrained on synthetic data and then fine-tuned with hospital data. An ensemble model combined predictions from synthetic, adapted and hospital-trained models. The approach used LightGBM and XGBoost, with different sampling and optimisation settings.
Must Read: Human-AI Co-Design Refines Clinical Prediction Models
Transfer Strategies Show Different Results
Clear distributional differences appeared between the synthetic and hospital datasets across all measured variables. The hospital cohort was older, had a different stage distribution and showed higher levels of adenocarcinoma, perineural invasion and vascular invasion. Treatment patterns also differed, including postoperative chemotherapy categories. These differences created a domain shift between the synthetic source data and the real hospital target data.
The zero-shot approach had limited and inconsistent performance. Applying a model trained only on synthetic data directly to hospital data reduced the area under the precision-recall curve in most settings. Any marginal improvements in the remaining settings were not statistically significant. This finding showed that synthetic data alone did not transfer reliably to real hospital data in this setting.
Domain adaptation produced stronger results. Performance improved in most tested combinations, with statistically significant gains in several settings. The best-performing configuration used XGBoost, random undersampling and F1 optimisation. Ensemble modelling also improved performance in several combinations, although the pattern was not uniformly superior to domain adaptation. The findings indicated that synthetic pretraining became more useful when the model was adapted with local hospital data.
Calibration and Practical Limits
Calibration results supported the same overall pattern. Domain adaptation improved Brier scores in most tested combinations, with a substantial proportion reaching statistical significance. Ensemble modelling also improved calibration in many settings. Zero-shot models were weaker, with several combinations showing worse calibration than the hospital-only baseline.
The absolute predictive performance remained limited. The constrained feature set, single-institution external validation, class imbalance and domain differences may all have contributed to this result. The hospital-only baseline was also restricted to the same twelve shared variables, so the comparison did not test whether adapted models could outperform a fully optimised local clinical prediction model using all available hospital variables.
Several limitations affected transferability. Synthetic data showed discrepancies in clinically relevant variables, including perineural invasion and histological subtype distribution. A substantial number of synthetic records were excluded because of missing essential covariates. The external validation dataset came from one healthcare organisation, so broader validation across different settings remains necessary. Future work is expected to assess missingness, rare variables, multicentre validation and other diseases or prediction tasks.
Synthetic data can support clinical prediction model development when it is used as an auxiliary pretraining resource and combined with real hospital data through domain adaptation. Direct application without adaptation performed poorly in most settings. The results therefore do not support synthetic data as a substitute for real clinical models. Instead, they show that synthetic data may help improve model transfer strategies in privacy-constrained environments when limited local clinical data are available for adaptation and evaluation.
Source: Journal of Medical Internet Research
Image Credit: iStock
References:
Jang Y, Kwon JH, Kim H et al. (2026) Evaluation of the Applicability of Synthetic Data in the Development of Colorectal Cancer Survival Prediction Models: External Validation of Advanced Machine Learning Models Based on National Cancer Data Center Data. J Med Internet Res;28:e86087.
