Generative AI Moves Beyond Language in Medicine

Generative artificial intelligence in medicine can extend beyond large language models and text-based clinical tasks. A 2026 publication in The Lancet Digital Health focuses on direct tokenisation of medical data, including laboratory results, medicines, vital signs and time-linked clinical events. The approach turns these data into discrete units that transformer-based models can process as patient health timelines, without first converting clinical information into ordinary language. Enhanced Transformer for Health Outcome Simulation (ETHOS) uses tokenised medical records to forecast possible future timelines and support clinical decision-making. The framework also links tokenised modelling with multimodal data, privacy-preserving model sharing, fairness and the need for large, diverse datasets across healthcare systems.

Tokenised Data Reframe Clinical Modelling

Tokenisation means breaking complex information into smaller units that AI models can process. In language models, these units can be words, characters or parts of words. In medicine, tokens can represent laboratory findings, medication events, vital signs, imaging components, signal features, abnormal biomarkers or the time between clinical encounters. This allows structured data, continuous measurements and unstructured clinical information to become ordered sequences that transformer-based systems can analyse.

Electronic medical records (EMRs) already contain many elements that can be tokenised. A laboratory test, medication order, administered treatment, diagnostic code or vital sign can become a token or a group of tokens when the information is more complex. Time intervals between clinical events can also become tokens, so the sequence captures both what happened and when it happened. These sequences form patient health timelines, which represent interactions with healthcare systems over time.

ETHOS uses this structure to simulate possible future patient health timelines by generating tokens step by step. Clinical inference comes from the distribution of these simulated timelines. The model can support use cases from causal health prediction to financial modelling in healthcare delivery without retraining or fine-tuning. Token-based models can also help identify patterns in patient timelines, forecast disease progression, guide treatment planning and improve hospital resource allocation.

Must Read: AI Supports Triage of Medical Device Data

Multimodal Models Support Clinical Reasoning

Medical information includes structured records, free-text notes, medical images, waveforms and videos. Data-tokenising models offer a way to combine these formats within one computational structure. Structured clinical data can be aligned with unstructured clinical text, including progress notes, radiology reports and pathology reports, when the text is tokenised and placed within the patient timeline. Medical images can also be incorporated through fixed-length embeddings, allowing visual inputs to appear in the same tokenised space as time-linked clinical events.

Data-tokenising models and LLMs can also work together. ETHOS functions as a patient-specific model that forecasts individualised health timelines. An LLM can then support reasoning, abstraction and communication by turning structured outputs into summaries for clinicians or personalised insights for patients. Simulated future timelines can also become inputs for further reasoning and machine-learning tasks in routine clinical practice.

This combined approach could help answer clinically relevant questions, simulate counterfactuals and support complex decision-making. The system is not designed to work independently of clinicians. Its role is to support clinical expertise rather than replace it. The architecture brings together predictive modelling and adaptive reasoning, while keeping collaboration with physicians at the centre of use.

Privacy and Fairness Shape Deployment

Large-scale tokenised modelling needs broad access to clinical data while keeping patient information protected. ETHOS uses a privacy-preserving model-sharing framework in which models are trained locally. Instead of transferring patient data or model updates between organisations, local models generate synthetic patient health timelines that resemble the original data without containing actual patient records. These synthetic timelines then train a central model.

The privacy approach has two layers. Tokenisation removes sensitive elements such as exact dates, names and precise numerical values, reducing reidentification risk. The central model then learns only from synthetic timelines generated by local models, keeping real patient data away from aggregation and model-sharing. Original data remain inside institutional boundaries, while trained models can support wider collaboration.

Local models capture regional healthcare practices and patient demographics. Synthetic data can still contain noise, incomplete representations or biases from the datasets they mimic. Combining diverse synthetic datasets from multiple institutions could help a central model identify stronger patterns while reducing the limits of individual datasets.

Fairness also depends on data diversity and scale. Models trained on narrow datasets can reflect only part of the population and risk worsening differences in prediction accuracy or treatment recommendations for under-represented groups. Broader tokenised frameworks can bring together data from different institutions, care settings and patient populations. Fairness still requires monitoring across population subgroups, engagement with marginalised communities and bias mitigation throughout development and deployment.

Tokenised medical data extend generative AI beyond language processing and towards a general computing model for multimodal clinical information. Transformer-based systems can use ordered clinical events to model patient timelines, forecast possible outcomes and support personalised decision-making. ETHOS shows how tokenised records can connect prediction, multimodal integration, privacy protection and collaborative model development. Challenges remain in data representation, computing needs and interpretability. Future progress depends on models that integrate diverse medical data while preserving privacy, supporting fairness and explaining predictions in terms that clinicians and patients can understand.

Source: The Lancet Digital Health

Image Credit: iStock