The healthcare sector is increasingly reliant on software engineering for various medical applications, including research. However, there's a lack of guidance for health researchers on effectively developing digital health interventions, leading to challenges such as expertise gaps, high costs, and time constraints. Generative artificial intelligence (GenAI) technologies like ChatGPT can potentially aid researchers by providing frameworks and processes for software development. These systems can assist both non-programmers and experienced programmers in coding, troubleshooting, and communication between technical and non-technical team members. ChatGPT has already shown promise in clinical decision support and coding assistance. A recent case study published in JMIR Human Factors explores the use of ChatGPT to recreate a personalised automatic messaging system (PAMS) for patient engagement in a digital diabetes prevention programme. It examines the capacity, advantages, and limitations of ChatGPT in supporting product ideation, content development, and the software engineering process. The insights provided aim to facilitate the development of usable, reliable, and modern coding practice-compliant computational tools with GenAI assistance.
Evaluating ChatGPT's Role in Digital Product Development: A Comparative Study
The study assessed ChatGPT's ability to aid in the conceptualization, content development, and software engineering of digital products, particularly by recreating a personalised automatic messaging system (PAMS) using GenAI. Eleven evaluators, with diverse expertise in fields including medicine, implementation science, and computer science, reviewed outputs generated by ChatGPT and compared them to human-generated ones. They rated ChatGPT's outputs based on factors such as understandability, novelty, usability, relevance, efficiency, and potential for bias. Evaluators also provided an overall score for ChatGPT's output quality. Following the review, a group discussion was held to explore the perceptions and rationale behind the ratings.
Leveraging ChatGPT-4 for Software Development: Insights from PAMS Recreation
This study utilised ChatGPT-4 to recreate content features and software development processes for a personalised automatic messaging system (PAMS). ChatGPT facilitated various stages of the software development life cycle, including conceptualization, feature identification, user story development, and code generation. While the ChatGPT-generated output was generally comprehensive, occasional ambiguities required clarification or adjustment by the research team. The output demonstrated high accuracy in capturing intended requirements and significantly reduced development time compared to human effort. However, limitations were observed, particularly in providing evidence for design choices and generating code with nonidiomatic constructs and bugs. These limitations underscore the importance of human expertise in interpreting ChatGPT outputs and guiding the development process. ChatGPT should be used as a support tool, with human experts validating outputs and refining prompts to improve results. Well-structured prompts incorporating human expertise are crucial for obtaining accurate results. Overall, while ChatGPT offers promise for efficient software development, human expertise remains essential for guiding and validating its outputs effectively.
Future Directions for ChatGPT in Digital Health Intervention Development
The authors suggest several future directions for leveraging ChatGPT in digital health intervention development. Plans include enhancing prompt engineering expertise within the team to utilise ChatGPT for developing additional features of the personalised automatic messaging system (PAMS), potentially saving considerable time and effort. Additionally, ChatGPT may be used to streamline documentation tasks such as coding documentation and research archival work. The authors believe that ChatGPT can effectively augment existing skills and bridge skill gaps within healthcare technology research teams, benefiting both technical and non-technical team members. For those with programming expertise, ChatGPT can automate low-impact tasks and provide technical support, while for those without programming skills, it can offer educational tools and skill-building opportunities.
Overall, the study highlights ChatGPT's potential as a feasible and resource-saving tool for developing digital health products, though limitations exist, particularly regarding output completeness. Future efforts will focus on developing more targeted artificial intelligence tools to support researchers with varying levels of technical skills and improving tools for objectively evaluating GenAI outputs.
Source: JMIR Human factors
Image Credit: iStock
