Non-Human Partners in Rehabilitation: How Healthcare Can Embrace Human-Machine Systems

In conventional rehabilitation care, doctors and therapists interact with the patients in a human-human interaction to customise rehabilitation based on an individualised assessment. Human-Machine Systems (HMS) offer accuracy in assessment, monitoring and supportive tasks and contribute to heightened productivity across various fields of rehabilitation. The objective of this study is to describe some of the HMS and raise awareness for their potential usage in the management of rehabilitation services and the quality of care offered.

Key Points

• Human-Machine Systems (HMS) integrate multiple sensors, robotic systems, virtual reality and machine learning algorithms.
• HMS promote accuracy in assessment, monitoring and support in rehabilitation.
• Improvement in rehabilitation services productivity, patient safety, and overall quality of care can be obtained by wise use of HMS.
• In more complex clinical contexts, HMS may be the only viable resource in aspects that are fundamental to quality of life, such as communication, wheelchair control or even entertainment.

Introduction

An ageing population sees increased physical disability in the elderly and high longevity for people unfortunate to have such disabilities from a young age due to multiple causes. Such is a medical and social problem in many countries (Chen et al. 2016). Rehabilitation medicine is about restoring or compensating for the individual’s lost or diminished abilities (Akdoğan and Adli 2011). Demand for quality of care in rehabilitation is increasing; health managers and clinicians want to offer more and better but often face challenges to justify treatment effectiveness to payors and other regulatory entities. Usually, rehabilitation practitioners customise rehabilitation plans for their patients based on an individualised assessment of physical, cognitive, emotional, and social systems in order to diagnose their specific needs, supports, and barriers (Wagner 2014). Traditionally, this task is performed manually by doctors and therapists in a human-human interaction involving physical and non-physical interactions (e.g., audiovisual interactions) (Küçüktabak et al. 2021). In more dependent patients, rehabilitation treatments are very work-intensive and often demand several therapists together to support one patient manually to perform the training (Diaz et al. 2011), making it even more difficult to meet the requirements of high-intensity forms of training. Therefore, there is an increasing demand to develop new techniques and assistance methods to recover lost or impaired motion control and to release therapists from the intensive labour of rehabilitation training (Brown-Triolo et al. 2002).

One way to controllably customise the interaction between the patient and the rehabilitation professional is to connect them to a robot (Baur et al. 2019). Besides controlling the desired interaction dynamics between the humans, the robots could display a virtual external environment, more engaging (Ganesh et al. 2014; Takagi et al. 2018).

It is also possible to use passive mechanical devices to transmit physical information between humans or sensors to collect information for transmission between them. These passive-device-mediated systems are not robots. The term human-machine-human (HMH) interaction includes both robot-mediated and passive-device-mediated systems (Küçüktabak et al. 2021).

Human-Machine Interaction (HMI) encompasses the communication and interaction between humans and machines; several interfaces and systems could be used with the primary aim of facilitating effective and accurate information exchange, commands, and feedback, empowering users to control and interact with technology effortlessly. In Human-Computer Interaction (HCI), the former is used to control a computer. HMIs and HCIs both serve as an interface for controlling a device. HCI is general for screen users. HMI is for any tool, object, or robot that can interact with humans; it could involve hand-held end-effector type manipulators, haptic devices, sensorised objects, virtual environment (VR) or robotic tools.

HMI technology can be broadly categorised into the following five categories (Kaur 2021):

1. Optical technology: Utilising cameras as the primary hardware for computer vision, this technology enables users to interact with devices through hand gestures without any physical contact.
2. Acoustic technology: Primarily employing speech recognition, this technology converts spoken words into text, facilitating device control and communication. It is commonly applied in home automation systems and voice-operated wheelchairs.
3. Bionic technology: Combines biology, robotics, and computer science. Generally, there are two forms of bionic systems: invasive and non-invasive. In the non-invasive form, the bioelectric signals are recorded using electrodes connected outside the body. Examples include Electromyographic (EMG), Electro-oculographic (EOG), and Electroencephalographic (EEG) signals generated from different parts of the body, which can be utilised as control signals to interpret the user’s intention. EEG signals, representing brain activity, are obtained by placing electrodes on a person’s scalp and include four types of waves – delta, theta, alpha and beta – along with event-related potentials (ERP) and steady-state visual evoked potentials (SSVEP), commonly used in Brain-Computer Interfaces (BCI). In the EMG technique, signals are monitored from muscles, whereas EOG measures eye movements by placing electrodes around the eye. In the invasive form, electrodes are surgically implanted in the human body, such as intracortical and electrocorticography, to monitor brain activity and implantable myoelectric sensors combined with targeted muscle reinnervation to collect electromyographic signals. Both invasive and non-invasive bioelectric signals can play a crucial role in controlling different devices, including neural prostheses, robotic limbs, exoskeletons, and wheelchairs, within the contexts of rehabilitation and assistive technology.
4. Tactile technology: This method necessitates physical touch, such as button pressing, for interaction with devices. It finds utility in various applications like environmental control systems, touch-based light controls, and pressure-sensitive interfaces. Additionally, tactile technology extends to innovative uses, such as artificial skin with sensory feedback, enabling remote communication between individuals.
5. Motion technology: This category encompasses all HMIs that detect motion, often utilising gyroscopes and accelerometers or their combination to achieve precise motion detection. Applications include motion-sensitive mouse capable of responding to various hand gestures, as well as wheelchair controllers where input is provided through head rotation.

A successful HMI is characterised by intuitive and user-friendly interfaces that enable efficient and pleasant interactions, allowing humans to operate and engage seamlessly with machines, devices, or software.  

In HCI systems, many interface-related factors must be considered, including the type of interaction, screen resolution, display size, and even colour contrast. The ultimate goal is not only to enhance communication between users and computers but also to personalise the context and environment in which the system is accessed.

There are three interaction types in HMHs (Küçüktabak et al. 2021): (1) physical interaction, (2) non-physical interaction, and (3) a combination of both. Physical interaction is usually obtained by rendering a spring/damper system between subjects via robotic devices. Non-physical interaction includes auditory or visual interaction. Visuo-physical interaction generally results in better performance than visual interaction alone.

The choice of interaction mode has a significant impact on task performance and engagement. According to Küçüktabak et al. (2021), it involves four options:

• Collaborative: Partners share a common task goal and collaborate to achieve it, with roles not predetermined.
• Cooperative: Partners have a shared task goal but are assigned distinct roles (e.g., teacher and student).
• Co-active: The task is divisible, and each individual works independently, yet there is still interaction.
• Competitive: Each individual strives to achieve their own goal, which may conflict with the goals of others.

Having a highly skilled partner tends to enhance dyadic task performance to a greater extent than partnering with someone less skilled.

The production of «smart» HMI involves adaptive learning to understand user preferences and behaviour over time, coupled with efforts to reduce the complexity of the design, increasing ease-of-use for operators and their ability to connect to the Internet, as well as share and receive data the HMI itself to digital and online platforms such as Bluetooth and the Cloud. Additionally, applications that could benefit from hardware upgrades can have their HMIs roll out with that hardware already part of the system at a low cost, ready and waiting to be activated with the appropriate update. The Internet of Things (IoT) demands more connectivity between machines and their operators.

The trend is moving towards smaller devices with enhanced functionality in consumer electronics, influencing HMI design.  Technologies such as 3D printing simplify production at low cost. In the realm of production, HMIs are available in three distinct forms: custom-designed HMI platforms, open HMI platforms, and ruggedised HMIs (Kalkal et al. 2022).

• Custom-designed HMI platforms are typically favoured by companies necessitating proprietary hardware and software, such as those in the military and medical sectors. While custom solutions entail greater development expenses, they can yield reduced per-unit costs during production, thereby diminishing overall expenses with larger production quantities.
• Open HMI platforms, on the other hand, offer universality and are capable of running various software packages across popular operating systems. These platforms are well-suited for companies seeking to develop their own custom application software. However, due to elevated costs, they may not be the most economical choice for applications requiring extensive production runs.
• Rugged HMIs are engineered to function in the harshest and most perilous environments. The design of rugged HMIs is heavily influenced by anticipated exposures in such environments.

Technology Used in HMS for Rehabilitation and Assistance

The broad concept of HMS includes the use of different technologies that can be used to promote an increase in the efficiency and quality of rehabilitation in the near future. These are: 1. Sensors; 2. Robotic devices; 3. Brain-computer interfaces (BCI) and Brain-machine interfaces (BMI); and 4. Virtual reality.

1. Sensors

There are two categories of sensors: wearable sensors (WS) and non-wearable sensors (NWS). Wearable sensors (depicted in Figure 1) are typically compact, cost-effective, and inconspicuous devices that offer precise, quantitative, and uninterrupted data concerning motor activity across various settings. In clinical contexts, wearable sensors have been employed for evaluation purposes, including the instrumentation of common mobility assessments, detection of abnormal movement patterns, characterisation of disease progression, management of falls, and recognition of different activities. Moreover, they have been utilised to enhance therapeutic interventions, such as facilitating gait training through biofeedback. Clinical uses of wearable sensors encompass remote monitoring, mobile health initiatives, and broadening the scope of health metrics beyond conventional clinical environments. The portability of wearable sensors enables their deployment in everyday environments, thereby yielding more realistic and comprehensive health-related data. Wearable sensors present an opportunity for the aggregation of extensive data across clinical and real-life scenarios, fostering the advancement of personalised and precise medical practices (Porciuncula et al. 2018; Dhawan 2016).

Gate analysis is very important for the clinical assessment of patient rehabilitation (Prasanth et al. 2021). It usually includes force-based sensors and inertial motion units (IMU). Force-based sensors, commonly integrated with footwear, measure the interaction of the body with the ground during walking. Gyroscopes detect the rate of change of angular motion by sensing Coriolis forces within a rotating reference frame, reflecting the limb's angular rotation speed. Accelerometers monitor body movements based on speed changes. Additionally, magnetometers detect the Earth's gravitational vector, providing compass heading data and a reference for body orientation relative to gravity (Rueterbories et al. 2010).

Individuals experiencing hemiparesis often need to monitor and assess hand movement performance throughout their rehabilitation regimen. Hence, wearable sensors that don't impede limb mobility can be utilised for tracking and monitoring purposes. Insight into joint movement data is pivotal for refining and adapting the rehabilitation protocol (Yao et al. 2018). Machine learning technology can amalgamate and forecast data collected by sensors employed in disease rehabilitation, thereby enhancing the precision of stroke and other disease diagnoses and aiding rehabilitation practitioners in forecasting the patient's recovery path (Mainali et al. 2021; Mennella et al. 2023; Liao et al. 2020).

In addition to wearable sensors, non-wearable sensors (NWS) offer another avenue for movement monitoring, divided into two main categories: those employing image processing (IP) and those utilising floor sensors (FS). IP systems utilise optic sensors, including cameras and laser range scanners, to capture subject movements and analyse various parameters through digital image processing. FS systems, on the other hand, rely on sensors embedded in floor-based force platforms to measure gait information, including pressure and ground reaction forces exerted by the subject's feet during walking (Muro-de-la-Herran et al. 2014).

2. Robotic Devices

Rehabilitation robots play a vital role in therapy by offering high-intensity treatments and objective assessments. They have the capability to evaluate irregular movement patterns and boost motivation through interactive games and tasks displayed graphically (Baur et al. 2019; Veerbeek et al. 2017). Typically, therapists oversee robot-assisted training and assessment, setting parameters and supervising the process. However, advancements include robots connected to both patient and therapist, incorporating three-dimensional haptic systems. The potential of robotic systems lies in their ability to enhance sensitivity during patient assessments and offer valuable biofeedback (Lambercy et al. 2012).

Two main types of robotic rehabilitation devices are available: wearable devices and platform-based devices. Wearable devices, such as robotic orthoses and exoskeletons, cater to upper limb support (see example of hand exoskeleton in Figure 2) and gait correction while enhancing ankle performance during walking. On the other hand, platform-based devices primarily focus on improving ankle performance (Payedimarri et al. 2022). In recent years, exoskeleton robotic devices, also known as wearable robots, have emerged as practical tools for therapists to assist with impaired joints or limbs. These devices have evolved to encompass full-limb exoskeletons, including support for shoulders, elbows, wrists, and ankles (Shi et al. 2019).

Robotic gait devices (depicted in Figure 3) provide electromechanical support to help individuals achieve a natural walking pattern. These devices have the potential to address practical challenges and facilitate intensive gait training by reducing the need for therapist intervention. With robotic assistance, users can undergo high repetitions of the gait cycle while experiencing reduced reliance on therapists to guide limb movements or assist with trunk stabilisation (Mehrholz et al. 2017). Additional benefits encompass decreased spasticity and pain. Nonetheless, their effectiveness is restricted by the substantial expenses associated with walking assistance, challenges in acquiring necessary skills and strength, and the inability to sustain therapy outside clinical settings.

3. Brain-Computer Interfaces (BCI) and Brain Machine Interfaces (BMI)

The enhancement of assistive restoration greatly relies on electrophysiological signals, which are essential for evaluating human movement capacity and behaviour in ongoing research. EMG is commonly utilised in device control techniques due to its ability to directly reflect the user's movement intention or muscular action (Lalitharatne et al. 2014). However, when patients exhibit minimal or no motor activity, EMG may prove ineffective in detecting the user's intention, necessitating alternative solutions. Brain activity measured through EEG stands out as a non-invasive and promising method suitable for motor neurorehabilitation applications, especially for stroke survivors, when utilising BCIs/BMIs as a facilitator for neuroplasticity (Soekadar et al. 2015). In this context, one of the most common BCI relies on the modulation of sensorimotor rhythms (SMR) through motor imagery or intention of movement (with no overt motor output/execution). These BCIs have been tested in combination with different types of feedback, such as proprioceptive (haptic) obtained with robotic devices or exteroceptive (visual) feedback, as depicted in Figure 5a), or based on peripheral stimulation such as functional electrical stimulation (FES). A recent comprehensive analysis of BCI usage in motor rehabilitation following strokes highlighted studies where BCIs were utilised to command robotic or orthotic devices (Mansour et al. 2022). These studies showed considerable to moderate improvements in motor impairment. Additionally, emerging evidence in upper limb rehabilitation indicates that BCI-assisted robotic training after a stroke is superior to robotic training alone in facilitating motor recovery (Mansour et al. 2022). Other studies (Lennon et al. 2020) explored the direct neural interfacing with robotic gait devices in stroke rehabilitation with promising results, although with a wide heterogeneity. Non-invasive brain stimulation, including transcranial direct current stimulation (tDCS) and transcranial magnetic stimulation (TMS), are also currently being researched as promising tools to enhance motor learning (Soekadar et al. 2015). 

BCIs/BMIs can also play a crucial role for stroke survivors as assistive technology. In addition to facilitating neural plasticity, BCIs can be used to substitute lost motor functions, for example, by controlling devices like exoskeletons or powered wheelchairs (Cruz et al. 2021) (Figure 4b) in daily life activities. Furthermore, for stroke survivors in a locked-in state, BCIs can serve as a vital communication channel (Pires et al. 2022). 

4. Virtual Reality

Neuroplasticity, a fundamental concept in neuroscience, underscores the regenerative capacity of the central nervous system (Garraway et al. 2016). Task repetition is crucial for establishing movement patterns, activating neural circuits responsible for motor patterns, enhancing sensory functions, and regulating afferent input, mirroring daily activities (Smith and Knikou 2017; Rosly et al. 2017). Active patient participation in motivating environments is key to enhancing rehabilitation outcomes (Weber and Stein 2018). Emerging technologies like robotic devices, BMI systems, and virtual reality (VR) address these aspects. VR, for instance, activates the mirror neuron system, fostering cortical reorganisation and functional recovery (Puyuelo-Quintana et al. 2017). VR systems vary in immersion level, ranging from semi-immersive or non-immersive setups using screens to immersive setups integrating users fully into virtual environments (Figure 5). Immersive systems, like VR caves or head-mounted displays, can incorporate additional sensory devices for enhanced feedback (Henderson et al. 2007). Combining VR with telemedicine shows promise for rehabilitating motor impairment from neurological disorders (Putrino 2014). VR is often integrated into robotic devices, such as Lokomat®, to provide complementary and motivating therapy modules (Agudo 2019). These systems have seen significant development over the past 15 years, offering high-intensity repetition-based therapies that have shown efficacy and cost-effectiveness in conditions like stroke (Iman and Jarus 2014; Sin and Lee 2013; Kim et al. 2009; Dominguez-Tellez et al. 2019), cerebral palsy (Booth et al. 2018; Johansen et al. 2019), Parkinson’s disease (Feng et al. 2019; Lei et al. 2019), and multiple sclerosis (Moreno-Verdu et al. 2019; Norouzi et al. 2021; Maggio et al. 2019).

Advantages and Challenges of HMS

Several advantages and challenges of using HMS in rehabilitation medicine can be identified. While the advantages can motivate management to discuss with clinicians how and when to invest and investigate further their usage, the challenges are presented to bring awareness that, like in any quality-of-care innovation, it is important to be open and transparent about the limitations and risks to best overcome them. As such, some of the advantages of Human-Machine Systems (HMS) are:

1. Improved Usability: HMS prioritise creating interfaces that are intuitive and user-friendly, facilitating seamless interaction between humans and machines. This leads to quicker learning, effortless navigation, and optimal utilisation of technology.
2. Error Prevention and Recovery: With clear instructions, visual cues, and informative feedback, HMS minimise the chances of user errors or critical mistakes. Furthermore, effective error recovery mechanisms and intuitive interfaces help users promptly resolve issues.
3. Enhanced Safety Alerts: HMS play a pivotal role in averting accidents and safeguarding user well-being. By providing clear warnings, alerts, and feedback, users gain better awareness of potential risks associated with machine operation, enabling them to take necessary precautions.

The challenges associated with Human-Machine Systems in rehabilitation may include:

1. Reduced Human Interaction: The increasing reliance on digital communication and virtual interfaces may diminish face-to-face interactions, potentially affecting social connections.
2. Skill and Training Requirements: Despite efforts to simplify interactions, certain machines or systems may still demand specialised skills and training for effective operation. Users may need to acquire new knowledge or undergo training programmes, adding to initial costs and time investments.
3. Ethical and Privacy Concerns: The integration of advanced technologies in HMS raises ethical and privacy considerations. Issues like data privacy, security breaches, surveillance, and unintended consequences of automation must be addressed to uphold individual rights, ensure confidentiality, and maintain ethical standards.

In the dynamic realm of human-machine interface (HMI), it is essential for designers, developers, and researchers to acknowledge these advantages and disadvantages to optimise benefits while mitigating potential challenges. An ethical and inclusive approach to deploying HMI systems is imperative for fostering a positive and sustainable relationship between humans and machines (Herington et al. 2023).

Conclusion

Human-Machine Systems offer an intuitive and effective means of interacting with complex machinery and processes, streamlining control, monitoring, and configuration tasks. By minimising user effort, HMS contributes to heightened productivity across various fields. Human-Computer Interaction practices identify and address potential sources of errors and user frustration. Despite requiring additional hardware and software components for operation and maintenance, such as computers, monitors, keyboards, and operating systems, the benefits in productivity and efficacy often outweigh these costs. The future of interaction design will prioritise a human-centric approach, emphasising accessibility, inclusivity, security, and privacy. As HMIs continue to advance, the lines between humans and machines will blur further, ushering in a more interconnected and technologically empowered era. Instead of fearing automation and AI, organisations should embrace them as integral tools for enhancing productivity and efficiency.

Conflict of Interest

None.