The rapid spread of medical AI solutions, especially large language model applications, creates a governance challenge for healthcare organisations. A recent publication in the International Journal of Medical Informatics links the proper use of generative AI and LLMs in medicine to a renewed form of knowledge engineering, adapted to current technological conditions. Even commercial LLM-based systems depend heavily on instructions created by developers, deployers and users. In medicine, such instructions cannot remain individual, non-standardised, heuristic, anecdotal or opaque. The central concern is not only model capability but also how goals, context, domain knowledge and expected outputs shape system behaviour. Knowledge engineering offers a route to make these controls more explicit, auditable and aligned with clinical standards.
Prompt Engineering Becomes Knowledge Control
Early AI and LLMs appear to rest on very different paradigms. Older AI systems separated knowledge bases from inference engines and used rules, frames, semantic networks or causal probabilistic models to represent knowledge. Their reasoning mechanisms and knowledge representations were explicitly programmed, which made transparency part of the design. LLMs operate differently. They embed knowledge in high-dimensional parameter spaces and produce output through stochastic generation, making the shift from symbolic reasoning to probabilistic text generation seem irreversible. This contrast matters for clinical governance because transparency in earlier systems came from explicit structures, while LLM behaviour depends on internal parameters that are not directly inspectable in the same way.
Practical use of LLM chatbots still depends on prompt engineering, which functions as a form of knowledge-based control. Prompt engineering builds queries that specify goals, context and expected output formats. Good prompting requires domain knowledge and an understanding of how LLMs behave, including typical answer patterns. Output quality depends strongly on the quality and structure of the input query. Relevant and well-organised knowledge improves the answers generated by the model. Formal knowledge representation principles can therefore strengthen prompt engineering, particularly in safety-critical medical settings where unstructured instruction practices create governance concerns.
RAG Systems Reconnect Knowledge and Inference
The need for explicit knowledge modelling becomes clearer as medical LLM-based systems grow more complex. Advanced systems progressively separate knowledge and inference again through prompt construction strategies, retrieval mechanisms, structured external data and curated knowledge sources. These systems no longer depend only on implicit knowledge stored in model parameters. External knowledge can be independently controlled, updated and validated, giving healthcare organisations a clearer basis for oversight. This separation also creates a practical route for maintaining knowledge content as clinical standards and organisational requirements evolve.
Retrieval-augmented generation systems apply this approach by coupling LLMs with retrieval pipelines. These pipelines select relevant information segments from external knowledge sources, including curated document collections, databases and knowledge repositories and incorporate them into the prompt provided to the model. Predefined schemas can also guide output generation, making responses more consistent, interpretable and adapted to the medical domain.
Must Read: LLMs Require Cautious Integration in Healthcare
RAG systems rely on components that belong to knowledge-based systems. Knowledge acquisition occurs through the selection of knowledge sources. Knowledge structuring depends on human expert knowledge. Schema-based modelling can include domain terminology and information about care processes when such information is available. Ontologies, semantic networks and knowledge graphs can further enhance retrieval by extending searchable concepts. In medical applications, this neuro-symbolic approach can improve RAG systems, support reasoning and facilitate explainability.
Agentic AI Raises the Governance Threshold
Agentic systems create a further need for renewed knowledge engineering. These systems place different LLMs inside complex software environments that support planning, iterative retrieval, web interaction and user feedback loops. Their design requires knowledge about tasks and goals, knowledge about the application domain and knowledge about interactions between components. Explicit design constraints and simulation environments can improve safety and control in these systems. Without such structures, system behaviour becomes harder to organise across multiple models, tools and feedback processes.
Transparency should become mandatory across medical applications of agentic systems. Certification bodies and healthcare organisations implementing and deploying such systems need required model cards that reflect this level of transparency. Several concepts from knowledge-based systems can help organise the key components of newer AI tools. LLMs can function as general-purpose inference engines. Document repositories, external databases and case descriptions can serve as knowledge sources with varying degrees of formalisation. Agentic AI orchestrators can act as high-level reasoning mechanisms, similar to blackboard architectures.
This framing keeps a fundamental design distinction visible: what is known should remain separable from how that knowledge is used. For LLM-based medical AI, such separation supports externally managed knowledge sources that can be audited, updated and aligned with clinical standards.
Trustworthy AI-enabled healthcare can benefit from knowledge engineering principles when LLM-based systems rely on explicit, managed and validated knowledge sources. Prompt design, knowledge chunking, retrieval optimisation, reasoning augmentation, process modelling and system integration all need coherent methods, tools and evaluation frameworks. Further work must address interactions between system components and reasoning mechanisms within LLM environments. These principles also connect prompt engineering, RAG design and agentic system oversight through a shared focus on explicit knowledge control. For healthcare organisations, the governance challenge centres on making knowledge sources, instructions and system behaviour more transparent, controllable and clinically aligned.
Source: International Journal of Medical Informatics
Image Credit: iStock
References:
Bellazzi R (2026) Large Language Models and the return of knowledge engineering. International Journal of Medical Informatics: In Press.
