SWALKIT: A generic augmented walker kit to provide haptic feedback navigation assistance to people with both visual and motor impairments

Feedback solutions are a privileged form of assistance in order to increase mobility and independence of people with both motor and visual impairments. Indeed, it empowers the ability of the person to make decisions and take actions based on the provided information. Moreover, it maintains the use of the walker, and thus the residual locomotor skills. We here propose the SWALKIT, an open-source, cost-efficient, lightweight, easy to install and generic augmented walker kit. The SWALKIT can be equipped on any walker without requiring modifications of the structure or advanced technical knowledge. Vibrotactile feedback is provided through the handles to indicate the proximity of obstacles on the way of the user. The open source project is reproducible thanks to the online repository https://github.com/IH2A/Swalkit. In this paper, we present the design of the SWALKIT based on a user-centered approach following target users and therapists guidelines. Then, we present a technical validation study performed with 14 able-bodied blindfolded participants on a cardboard circuit. They were asked to use a standard walker with and without activation of the SWALKIT system. Results of this pilot study showed the efficiency and reliability of the proposed solution. Finally, we provide feedback after 2 months of daily life use by a target user.


SWALKIT: A generic augmented walker kit to provide haptic feedback navigation assistance to people with both visual and motor impairments
Fabien Grzeskowiak, Louise Devigne, François Pasteau, Gustavo Souza Vieira Dutra, Marie Babel, Sylvain Guégan

To cite this version:
Fabien Grzeskowiak, Louise Devigne, François Pasteau, Gustavo Souza Vieira Dutra, Marie Babel, et al.. SWALKIT: A generic augmented walker kit to provide haptic feedback navigation assistance to people with both visual and motor impairments.IEEE Int.Conf. on Rehabilitation Robotics, ICORR'22, Jul 2022, Rotterdam, Netherlands.pp.1-6.hal-03709659 SWALKIT: A generic augmented walker kit to provide haptic feedback navigation assistance to people with both visual and motor impairments Fabien Grzeskowiak, Member, IEEE, Louise Devigne, Member, IEEE, Franc ¸ois Pasteau, Member, IEEE, Gustavo Souza Vieira Dutra, Marie Babel, Member, IEEE, and Sylvain Guégan, Member, IEEE Abstract-Feedback solutions are a privileged form of assistance in order to increase mobility and independence of people with both motor and visual impairments.Indeed, it empowers the ability of the person to make decisions and take actions based on the provided information.Moreover, it maintains the use of the walker, and thus the residual locomotor skills.We here propose the SWALKIT, an open-source, costefficient, lightweight, easy to install and generic augmented walker kit.The SWALKIT can be equipped on any walker without requiring modifications of the structure or advanced technical knowledge.Vibrotactile feedback is provided through the handles to indicate the proximity of obstacles on the way of the user.The open source project is reproducible thanks to the online repository https://github.com/IH2A/Swalkit.In this paper, we present the design of the SWALKIT based on a usercentered approach following target users and therapists guidelines.Then, we present a technical validation study performed with 14 able-bodied blindfolded participants on a cardboard circuit.They were asked to use a standard walker with and without activation of the SWALKIT system.Results of this pilot study showed the efficiency and reliability of the proposed solution.Finally, we provide feedback after 2 months of daily life use by a target user.
Index Terms-haptic feedback, navigation assistance, assistive technology, mobility, augmented walker, open source.

I. INTRODUCTION
W ALKERS are mobility aid devices consisting of a stable frame which provides more support than canes to people who have difficulties walking and maintaining balance.They can be standard walkers which the user must pick up and place, or rollators, that are equipped with wheels that make them more mobile and easier to maneuver.Walkers are usually used by the elderly but can also be used by people with disabilities who are regaining their ability to walk or who have balance issues.As opposed to alternative mobility devices such as wheelchairs or scooters, walkers allow to keep using their remaining locomotion capacities [1].
Some people who experience difficulties walking can also be affected by neurological impairments leading to cognitive, visual and/or visuo-spatial impairments.For example, some cognitive impairments such as memory disorder or F. Grzeskowiak, F. Pasteau, G. Souza Vieira Dutra and M. Babel are with Univ Rennes, INSA, Inria, CNRS, IRISA -Rennes, France; e-mail: marie.babel@irisa.fr).
L. Devigne is with Inria, Univ Rennes, CNRS, IRISA,-Rennes, France.S. Guégan is with Univ Rennes, INSA Rennes, LGCGM-EA3913, -Rennes, France; e-mail: sylvain.guegan@insa-rennes.fr spatiotemporal awareness impairments can heavily challenge individual's ability to walk around safely.Furthermore, partial or total visual impairments combined with walking difficulties can make the ambulation task very difficult in particular around unknown environments.With impaired vision and without an appropriate technical aid, people need to be accompanied: they indeed need vocal assistance of someone who gives them information about directions and surrounding obstacles along the way.However, it does not ensure their autonomy as they need a third party to guide them.If they are alone, they can rely on touch, such as letting their arm rub on the walls to follow a corridor, or be aware of the presence of a nearby obstacle after colliding with it, but this can be cognitively exhausting, frustrating or even dangerous, in particular if there are steps or stairs on the way.
The field of Assistive Technology (AT) aims to design systems dedicated to compensate impairments.In the case of visual impairments, the compensation usually consists in providing feedback to the user.This allows the user to better understand the surrounding environment while keeping the ability to autonomously decide which actions to take.Therefore, users can benefit from a technical aid in the form of a sensory stimuli to understand the configuration of the surrounding environment [2].In the case of people having both visual and motor impairments requiring the use of a walker, they cannot use a white cane or any other feedback device to take in hand.In this context, a hand-free solution should be provided.This solution could be a wearable device, or directly integrated on the walker.Therefore, the walker can be equipped with additional features to improve the user's understanding of the surrounding environment and secure the ambulation.
In the literature, we can differentiate 3 types of augmented walkers for different purposes: motion/trajectory control, gait analysis, and feedback to the user.These devices can be instrumented with sensors to analyse user gait while walking, detect obstacles, localize the walker and even drive autonomously or provide emergency stops.For example, the AGoRA [3], ISR-AIWALKER [4] and CAIROW [5] walkers decode user's intention and provide autonomous navigation and obstacle detection.The MOBOT walker provides fall detection, obstacle avoidance and physical monitoring [6].Another augmented walker called ASBGo has similar features but also provides biofeedback regarding user gait pos-ture [7] [8].One more walker called UFES detects obstacles and provides feedback to assist the user to follow a safe path [9].The majority of the aforementioned augmented walkers require major structural modifications to a standard walker or, more often, a custom design.There are exceptions for some other walkers such as the one in [10] which provides Walking Pattern Recognition with an embedded computer and RGB-D camera on a standard rollator.Other exceptions can be found such as the system proposed by [11] which provides navigational guidance for users with severe visual impairments.This system uses haptic or audio feedback based on ultrasonic sensors measurements or Time-of-Flight camera.Other solution such as [12] provides vibro-tactile feedback through a belt or the handles of the walker.Both walker are tested with able-bodied participants without visual impairments during blindfolded navigation experiment, except for [12] with one blind participant but without need of a walker nor prior experience with a walker.Although both systems seem promising to provide assistance to people with visual impairments, the designs of the proposed systems are still experimental with several issues regarding the use in real life such as an impact on the form factor, expensive sensors and high power consumption.Moreover, both systems have not been tested yet with target users with combined visual and motor impairments.
As emphasized in [2], assistive devices for people with visual impairments should be low energy consuming, lightweight, easy-to-use, adaptable, and efficient in real-time.In addition to these generic guidelines on AT, the design of one device should take end-user expertise into account to ensure an efficient and relevant solution.Therefore, we propose the SWALKIT: an open-source kit which has been designed following a user-centered approach and which can be adapted on any walker.
This paper is organized as follows.Section II introduces the requirements for the design of the SWALKIT system resulting from an end-user centered approach.Section III presents the design of our proposed SWALKIT.Section IV presents an experiment with 14 able-bodied participants blindfolded on a cardboard circuit with and without SWALKIT assistance.Section V presents the feedback after two months of use by a blind end-user.Results and perspectives are discussed in Section VI.

II. AUGMENTED WALKER REQUIREMENTS
The objective of an augmented walker is to provide navigation assistance through feedback for people who have visual impairments combined with walking difficulties that require the use of a walker.In the first step of this user-centered approach, it is necessary to collect guidelines from therapists and users.This section therefore presents the guidelines on the augmented walker design based on the recommendations of occupational therapists, physiotherapists and target users.
a) Adaptable to any walker: Although target users may require the same feedback assistance solution to help them navigate, each person has a walker model that is adapted to their needs, and customized to match their individual characteristics.Therefore, the provided navigation assistance solution should be adaptable and customizable to any type of walker, in order to be accessible to anybody who may need it.
b) Accessible and cost-effective: The use of adapted and appropriate AT can enhance the quality of life.In particular, people with visual impairments can greatly benefit from AT to assist their mobility.The provided assistance solution should then be affordable and easy to get by people who may need it.
c) Easy to reproduce even with minimal technical knowledge: To benefit as many people as possible, the augmented walker should be easy to reproduce.Therefore, the production process should use accessible materials and production techniques with open-source comprehensive documentation and assembly instructions.The system should be a robust prototyping platform, using well-known technologies in the "Makers" community.
d) Configurable and customizable to user preferences: We assume that target users already own a walker that ensure their safety.The augmented walker should be fully adaptable to user specific needs and preferences i.e. the user can adapt the number of sensors and their positioning on the walker frame.The system should also be configurable in terms of "navigation modes" (indoor mode, outdoor mode, home mode, etc.) i.e. having the possibility to tune the sensitivity of the obstacle detection and feedback thresholds.Moreover, feedback should be fully customizable depending on the modality, for example if the feedback modality is vibrotactile, frequency, amplitude and pattern should be easily tuned.
e) Maintained form factor and maneuverability: Augmented walker height and width must not be larger than the standard walker width so the navigation comfort is not impacted.Indeed, a larger or heavier walker would increase maneuver difficulties (for example door passing) as well as risk of collisions.The increased risk of collisions due to a larger form factor would also increase the need of maintenance, which is detrimental to the user because he/she can no longer use the device until repair.
f) Intuitive and easy to use: The augmented walker should provide an intuitive and easily understandable solution to facilitate acceptance and avoid discouragement or abandonment of the navigation assistance solution.
Based on these recommendations, we have developed the SWALKIT: an open-source and plug-and-play system which can be installed on any standard walker.The resulting augmented walker provides a sensor-based haptic feedback through its handles.

III. DESIGN OF THE SWALKIT
The design of the SWALKIT consists of 4 different designs: electronics, mechanics, feedback and control application.The presented system is the result of a close collaboration with a target user (presented in section V) and therapists.Thanks to their feedback, we were able to iterate different versions.

A. General overview
The proposed solution is affordable (a kit costs between 110$ and 125$) and easy to reproduce with minimal technical knowledge, or with the help of structures such as fab labs.Indeed, all the components can be interconnected with Grove® cables and prototyping cables.The Android SWALKIT app (Fig. 3) can be used to tune the system settings.All the software components are provided on an open-source platform.The system housings are 3D-printed with flexible fit so that they can be adapted to other walker models (Fig. 1).To make a SWALKIT, all necessary files are available online1 .

B. Electronic design
The electronic architecture is presented on Fig. 2. As main component a M5Stack Core Gray powered by an ESP32 microcontroler was used as it already provides useful basic functionalities such as an LCD display and buttons to allow basic user interactions without any soldering.A power bank is used as power source and a switch gives the ability to turn the system on and off.Moreover the system enters deep sleep mode if no activity is detected by the M5Stack Core embedded accelerometer.The Core communicates with the sensors using an I2C connection.Infrared time of flight distance sensors VL53L0X manufactured by STMicroelectronics were chosen for their high measurement accuracy consistent under any indoor working conditions, their small footprint and low power consumption.The vibration motors are driven from general purpose output pins, connected through an audio amplifier.As such, a GF1002 Audio Amplifier Module was chosen since it is small, easy to connect and provides a potentiometer to fine tune the intensity of the vibrations.With a 10Ah battery, the system can be last about 5 days week without charging.

C. Mechanical design
The mechanical designs are based on the measurements of the walker.3D printing models are provided to ensure quick and cost-effective reproduction of the housings by any 3D-printer.The kit is lightweight, with a battery weighting 200g.Therefore, there is no imbalance impact on the walker.

D. Haptic feedback design
We chose haptic feedback over the use of audio feedback as it requires less cognitive load [13] [14].In fact, audio feedback is not a sustainable form of feedback as it is not applicable in a noisy environment and it can overload the cognitive capacities because the user has to distinguish the sound feedback of the device and the other sounds (solicitations from other people, ambient sound, etc).We used Linear Resonant Actuator (LRA) from foster to generate haptic feedback.Frequencies and patterns can be configured thanks to an Android smartphone application.We continued the user-centered approach throughout the design process by regularly testing successive prototypes with end-users.
The M5Stack Core code has two main tasks: reading and interpreting the sensor values to provide the feedback and enabling a Bluetooth connection with the App to modify the system settings.The sensor values are constantly read by the system.Based on these values and given thresholds saved on an SD-card, feedback is provided in the form of different vibration frequencies and pulsing.Four critical distance thresholds are stored in each profile: far, near and danger on each side as well as a critical threshold for frontal danger.Once the sensor measurements fall below one of these critical values the corresponding frequency and pulse length are activated indicating the rough distance and direction on the obstacle.In the case of a frontal danger the frontal settings are prioritised to make both sides vibrate.In any other case the side closer to the obstacle vibrates.To keep a low power consumption, the accelerometer embedded in the M5stack Core is used to detect inactivity.In this case, haptic feedback is switched off and the M5Stack Core enters deep sleep.

E. Application design
An intuitive and easy to use Android application (Fig. 3) allows the user to overview and change the settings of profiles saved on the M5Stack core.In the application several profiles can be stored to make switching to a new profile fast and easy.A table provides an overview over critical distance thresholds as well as their corresponding frequencies and pulses and allows to modify them.Additionally the application displays a diagram illustrating the current sensor values to simplify diagnoses in the case of malfunctioning.Three languages are available : English, French and German.

IV. EXPERIMENT WITH ABLE-BODIED PARTICIPANTS
14 unpaid able-bodied participants, recruited via internal mailing lists amongst students and staff, volunteered for the experiment (5 F, 9 M; age: avg=29.9std=11.8,min=20, max=53).They were all naive to the purpose of the experiment and did not need assistive device.They gave written and informed consent.The study conformed to the declaration of Helsinki, and was approved by the Inria internal ethical committee (COERLE).They were asked to complete a circuit while being blindfolded in 2 two different conditions: • C 1 : Standard walker (no feedback) • C 2 : SWALKIT (feedback activated) To avoid training effect, the conditions as well the direction (clockwise/counterclockwise) in which the participant completed the circuit were randomized.For both conditions, we collected the time of completion and the number of collisions.At the end of each condition, participants were asked to answer 2 questionnaires: the NASA-TLX questionnaire (simplified to 7-point likert scale) to assess the cognitive load [15], and P, L and C items from a walkerrelated questionnaire from [16] composed of several 5-point likert scale questions shown on Fig. 4. Fig. 4: 5-point likert scale questionnaire for the user's evaluation adapted from [16].

A. Training phase
Before the experiment, participants completed a two stage adaptation trial using the walker without being blindfolded.Participants were first using the walker without haptic feedback to get accustomed to its handling.Participants were then using the walker with haptic feedback turn on to understand the different types of information given by the vibrators.The profile (see III-E) was the same for all users.Participants were given as much time as they needed.The adaptation time for just handling the walker was on average 27,4s (Standard Deviation (SD): 11,2s) and the adaptation time for the SWALKIT system was on average 119,8s (SD: 46,4s).

B. Task
After the training phase, participants were blindfolded and safely guided to the walker placed at one of the 2 starting points (randomly selected) of the cardboard circuit (Fig. 5).They had to go through the circuit following the two conditions, with indications to complete the task without collisions and as fast as possible.They were informed that they could ask for assistance if they felt lost, or quit at anytime, and that the experimenter would intervene if they went backwards.

C. Results
Our statistics were computed two set of paired variables, which were A) the number of collisions with the assistance On and with the assistance Off and B) the time of completion (On and Off), shown respectively on Fig. 6a and Fig. 6b.For all dependant variables (A and B), we set the level of significance to α = 0.05.A Shapiro Wilk test was performed on the difference to evaluate whether differential data followed a normal distribution.For each dependant variable, we cannot reject the hypothesis that the differential data are not following a normal distribution (p-values: [A:0.5596],[B:0.4387]).Post hoc, in order to compare each paired variable, we compute a paired Student test on each dependant variable.For each paired value, the difference between the means is significant (p-values:[A:3.475e-07],[B:0.002783]).
As shown in Fig. 7 and Fig. 8, haptic feedback was rated less frustrating and required less physical and mental workload while performances were better perceived in C 2 condition.Temporal demand was consistently rated as average across the two conditions.

V. FEEDBACK AFTER TWO MONTHS OF USE BY A BLIND END-USER
A duplicate of our smart walker has been given to the user who has participated to the co-conception of the SWALKIT.He is 19 years old, he is blind and has a cerebellar syndrome.He cannot walk by himself because he has a major risk of falling.He usually moves around with a walker while being guided by someone else's voice, or else, he must let go of one of the handles to touch the obstacle which leads to an increase risk of falling.He can also move around by holding the arm of another person.Hence, although he has a walker, he cannot get around safely and independently.His activity level is very low, he usually sits all day if he does not have a helper.
After 2 months of daily-use of the augmented walker, carers and helpers made the following observations.First of  all, it only took 3 sessions of 20 minutes during locomotion instruction sessions for the user to learn how to use the augmented walker, despite his complex disability conditions.The user is now able to walk around the IME building by himself and it can happen that he does this route in autonomy up to 8 times in the same day.He can now go to different places of activity (restaurant, communal room, etc) on his own.He can walk fast and has gained a lot of confidence in himself.His level of activities has significantly increased.He is well liked by the other residents, and their perception of him has changed now that he can go to different places on his own and get around independently.Finally, the caregivers who work in the medico-social institute have quickly adapted to the system and do not feel constrained by its use within the structure.

VI. DISCUSSION
During the experiment on the cardboard circuit, participants had significantly less collisions, as well as a significantly longer completion time.This means that the SWALKIT improves the navigation safety, even if it takes more time to perform the task.Participants were asked to complete the task as fast as possible.This could explain the fact that the time of completion is lower without assistance.Indeed, we can hypothesize that it is due to the fact that a collision with a cardbox leads to its displacement without any physical risk.Whereas when haptic feedback was activated, participants gave more attention to the obstacle avoidance, leading to a higher completion time.This hypothesis would be considered in future investigations.
Regarding the subjective results, questionnaires did not provide significant results in terms of use or cognitive load.We cannot directly conclude that the SWALKIT system is reducing the cognitive load of the participants based on this experiment.On the other hand, after using SWALKIT for more than 2 months, the blind participant who is representative of the target population shows high improvements in his daily life.Based on these results, next clinical trials on circuit with target users with combined motor and visual impairments in need of a walker to move will surely allow to have more conclusive results in terms of impact on usage and cognitive load.Still, our target user required 3 sessions of 20 minutes to properly learn how to use the augmented walker on his own.Therefore, clinical trials with other target users might have mitigated results due to a limited training.
All the participants gave quite positive feedback overall.They indicated that they would have prefer more time to train as they thought that it would have a positive impact on their performances.We can consider that a drawback of our experiment on the circuit is a too short training phase.On the other hand we can see that proper training does not require extensive time or effort to be efficient.Indeed, our target user had in total only 1 hour of training with an occupational therapist, and the assistance provided by the SWALKIT ever since is a relevant benefit in his daily life.We therefore can anticipate that even if people are able to equip their own walker with the help of a carer and/or a fablab, they still need proper training either with a therapist or with a family member.
Regarding the experiments on the circuit, we only enrolled able-bodied participants, while our approach is clearly targeted to people with both motor and visual impairments.However, it is an ethical requirement to show the effectiveness and safety of any assistive solution first with people without disability.Only after, we can seek approval for a clinical trial.For this reason, our nex plan is to perform ecological clinical trials enrolling target users.These ecological trials will take place in rehabilitation center as well as public places such as museums.
(a) Overall SWALKIT view.(b) Control and sensor units.(c) Vibration motor unit.

Fig. 2 :
Fig. 2: Structure of the electronic design of the SWALKIT.
(a) Layout and dimensions in cm of the cardboard circuit.(b) Cardboard circuit.

Fig. 5 :
Fig. 5: Characteristics of the cardboard circuit used for the experiment.
(a) Number of collisions.(b) Time of completion.

Fig. 6 :
Fig. 6: User performances: Mean and standard deviation of the number of collision and type of completion, for the two conditions (with and without the SWALKIT).

Fig. 8 :
Fig. 8: Results for the 5-point likert scale questionnaire for the user's evaluation adapted from [16].Blue: condition C 2 , Yellow: condition C 1 .Square dots represent the median values.