Sensing Technologies for In-Home Monitoring of People with Dementia

In this paper we discuss sensing technologies (such as smart carpet, posture sensor, bed sensor, door sensor etc.) developed for assisting family caregivers of people with mild dementia. The technologies provide automatic and unobtrusive monitoring of patient’s activity, assessing possible risks the patient may face and alerting the caregiver in emergency by video, audio and text delivered to his/her mobile phone or PC. We discuss designs of the sensing modules and outline their application in prototype home monitoring system.


Introduction
Dementia is a syndrome which deteriorates memory, thinking, behavior, language and the ability to perform everyday activities.Nowadays, over 4.6 Million people (15% of population over 65) are suffering from dementia in Japan and the number is predicted to reach 7.3M in 2025 (1) .Carrying for a person having dementia is very difficult.The task requires constant (24/7) monitoring as person with cognitive and perceptual deficits can make wrong judgment or be lost if left alone.It takes time, disrupts sleep, and causes enormous stress on caregivers, who frequently are relatives of the carried person, live in same home or flat and do the caregiving voluntarily.Even if family caregivers live close to the carried person, they have their jobs, own families, and obviously cannot supervise around the clock.Due to overwhelming stress, anxiety, irritability and a tall the task puts on caregiver's health and job (2) , almost 70% of families suffering from dementia in Japan (3) are forced to opt for nursing homes though neither people with dementia nor their spouses and families want to do so.Majority of the families however cannot afford this option due to the enormously high cost of nursing facilities and long span of the decease.More than 60% of people with dementia in Japan now stay at home (2) , placing a major burden on their relatives.Hiring a personal caregiver is also expensive and going to be much harder with the fast aging of population.Therefore, many families do not have any way but suffer.Clearly, new technologies, capable of reducing the burden of caregiving burden at home, are increasingly important.
Numerous information and communication technologies have been developed to assist people with cognitive impairment.The technologies can be active, enforcing a PD to wear a sensor, to pull a cord, or to push an alarm, or be passive, i.e. embedded into environment to detect potential problems (4) .Examples range from active technologies that directly interact with the user to support daily living activities such as wheelchairs and large-button cellphones, to systems such as computer controlled "smart homes" that provide support for persons with cognitive impairment.Majority of existing smart systems however, target internet-based reminiscent therapy for seniors and telecommunication between the patient and the physician (5) .The most advanced systems support remote collection of patient data, whether physiological or emotional, send alerts to patients on changes in health status, medication, reminders, upcoming appointments, or motivational statements, sends alerts to caregivers and providers on changes in health status and/or warning signs (6) .Some systems can help PD at work or at home by memorizing and delivering necessary data (7)(8) .The others aim to assist old people at group homes and hospitals (9)(10)(11)(12)(13) .
Several systems (14)(15)(16)(17)(18)(19) have been proposed to assist family caregivers at home.They can detect whether the PD is up and moving around the home; receive phone/text/email alerts if medications are missed; receive alerts if the person fell down or left the premises; chat "face-to-face" with the person via live video calls; sending the person an email without the need of computer; and creating a socialization and networking environment for the patient.However, these systems either cost up to several thousand dollars or require the PD to wear devices or force the caregiver to be in a close proximity of PD thus limiting the caregiver's ability to do other work.
The main goal of our work was to study technologies capable of automatic monitoring of PD at home.In this work, we developed a novel intelligent system for assisting family caregivers by unobtrusive automatic patient monitoring, assessing the risk the patient may have, alerting the caregiver if help necessary, and enabling the caregiver to watch and talk with the PD and access the monitoring data online via smartphone or PC.Unlike existing solutions, the system is realized with conventional off-the-shelf components, inexpensive, yet effective.In our prior work (20) we discussed the system architecture, functionality, user interface and its prototype implementation.This work focuses on design of heterogeneous, cost-efficient sensing devices that support automatic, around-the-clock monitoring of the PD activity (e.g.fall-down, exit from home, patient immobility, etc.)

Application overview
The developed technologies are dedicated for helping the caregiver with unobtrusive patient monitoring, automatic assessing of risks to the patient's health and alerting the caregiver when a risk exists.They include a number of heterogeneous sensors, such as smart carpet, bed monitor, posture sensor, door sensor.Together with video cameras and speakers these sensors are installed in the PD's flat or home (in living and bed rooms, kitchen, corridor, bathroom, toilet, etc.).Fig. 1 illustrates an example of their installation and use in a caregiver assisting system.The sensor's readings are wirelessly sent to the server that fuses the data and applies artificial intelligence to: (a) identify and monitor the person's position and activity (e.g.lying in bed/sofa/floor, sitting on the chair/ sofa/bed/ floor, walking, exiting room, vising toilet, bath, corridor, attempting to open home doors, etc.) (b) assess the risks that a PD may have in the current situation; (c) alert the caregiver in emergency for assistance in form of customizable alarm, text, and voice; (d) record the patient's activities, assessed risks, generated alarms, as well as inputs from the caregiver (e.g.schedule of medical treatment, exercises, etc.), (e) display results of patient's monitoring on caregiver's device (phone or PC), deliver a real-time view of the patient' location as well as audio communication with the patient; (f) provide graphic interface for viewing the history of patient's movement, his activities, frequency of visiting facilities, and frequency of alertness.
The results of monitoring and assessment are stored in the server's data base and can be viewed online from the caregiver's personal device or PC.The caregiver can also watch the monitored person in real time and communicate with him or her if necessary.The system frees the caregiver from the need to be at home and monitor the patient all the time.It provides the caregiver with ambient intelligence, which enhances the caregiver's monitoring ability, memory, problem solving and mobility.
Below we discuss the sensing technologies in details.

Smart Carpet
The smart carpet (SC) is placed in rooms, hallway leading to the exit door, in pathways, etc., to identify position, motion and activity of PD.The SC consists of an array of mats, each having a pressure sensor FSR406 (21) located under an expanse of carpeting.Fig. 2 illustrates the SC organization on example of 4x4 mats (left image), cross-sectional view of a mat (right-top) and the mat electronics (right-bottom).As one can see, the mat electronics is very simple: one small sensor and one diode.
In every machine cycle, the microcontroller sequentially activates the vertical lines of the mat array and checks the sensor readings from the horizontal lines of the array.If no pressure is put over a mat, the sensor idles.Otherwise, the sensor connects vertical and horizontal lines producing a signal at the horizontal line.The coordinates of the active mats are detected by the microcontroller and sent wirelessly (via the XBee transmitter) to the server.Because each mat has a fixed position in the carpet, the location of a person over the SC is easily defined through the reading of mat sensors.Walking, staying, sitting or lying down over the SC is reflected by a proper signal pattern and the time-difference between the patterns.Fig. 3 exemplifies the patterns displayed by the server PC and the smartphone, when a person was walking and lying over the SC (of 4x7 mats, 40cm x40cm in size each).The mats that sensed the pressure are shown in red.The tests (22) revealed that the SC detects person's motion and fall accurately.The sensors in the mats successfully detect gait characteristics and are not perceptible to the people as they walked across the SC.Similarly to the mats, the pressure sensors can be embedded in chairs, sofas, etc. to determine the patient's position and trace his/her activity.The total cost of a single mat is around US8$.

Posture sensor
The posture sensor is dedicated for areas (e.g.toilet or bathroom) where the smart carpet cannot be allocated.The posture sensor detects the person's fall, or a situation when a monitored person cannot stand up or is motionless longer then a pre-defined time.The design consists of two motion sensors (S1, S2) placed one over another on a distance (D) in between and connected via microcontroller to wireless XBee transmitter, as shown in Fig. 4(a).The sensors are implemented using pyro-electric infrared motion sensing module SE-10 PIR which can detect motion up to 5 m (Fig. 4, b).The position of sensors is set to distinguish a standing posture from a posture of a person, who sits or lies on the floor (see Fig. 4, c).For a person with height of 176cm, the values of h1, h2, h3 are 40cm, 60cm, and 87cm, respectively.For the assumptions, we derived empirically that placing the sensor S2 at the height of h1 from the floor, and sensor S1 at the distance D=24cm from S2 provides good differentiation of postures.Although each sensor was able to sense the person over 2m distance (Fig. 4,  d), the best results (100% of correct human pose detection) were observed at up to 140cm.The total cost of the posture sensor consisting of two sensors, the Arduino Uno microcontroller, XBee transmitter, two resistances, and the breadboard does not exceed US50$.

Door Sensor
Door sensors (DS) are employed to alert a caregiver when a monitored person exits his room and/or opens an entrance door.Unlike existing DS that just sense the door opening and closing, our sensor detects who opens/closes the door and alerts the caregiver when it is done by person with cognitive impairment (PCI).The device combines a magnetic sensor with a radio-frequency identifier of the patient, as shown in Fig. 5 (a).To identify the PD, we embed a RFID passive tag in his/her slippers, as shown in Fig. 5 (b) bottom-left, and place antenna of active RF-reader (Fig. 5(b), right image) under the carpet mat in front of a door, similarly as in [22].However, unlike [22], our RFID readers are wireless.Both the magnetic sensor and the RF-reader are connected to the Xbee transmitters, which signal the server whether the door is open or close and whether the person at the door is the PD or not, respectively.Based on this pair of signals, the server (see Fig. 5,a) assesses if the door is opened by the PD.The tests confirmed that both tag and the RFID antenna are not perceptible as they walked.Also, using slippers as RFID-tag is an acceptable solution, since elderly people usually wear slippers at home all the time.The passive RFID tags do not require charging batteries minimizing the maintenance tasks.The overall cost of the sensor is US30$

Bed sensor
The bed sensor (BS) detects the presence of a person in bed.We studied two different designs.The first one consists of an air bag placed between the mattress and the base in the patient's bed, as shown in Fig. 6 (a).The air bag is connected by a hose to an air pressure sensor that converts pressure changes into electrical signal.Fig. 6 (b) depicts the operation of the pressure sensor.The pressure switch is connected to microcontroller and to XBee transmitter to signal the server when the pressure switch terminals open or close.When a person sits or lies on the bed, he or she compresses the airbag and increases the air pressure within it.The high pressure in the airbag pushes the plunger rightward against spring tension, closing the switch and causing the transmitter to send an "ON" signal to the server, reporting the presence of the person on the bed.When the person gets off the bed, the air pressure within the airbag falls, the switch opens and the transmitter sends an "OFF" signal, indicating that the person has left the bed.It matters very little where the person lies or sits on the bed.His weight is distributed over a large area of the air mattress, compressing it almost everywhere, and this raises the air pressure within it enough to close the pressure switch.The second design (Fig. 7) is implemented based on a 10mm×622mm pressure sensing strip FSR 408 shown in Fig. 7(a).The bed sensor contains four strips of pressure sensors each of each connected to a microcontroller and a power source, as shown in Fig. 7 (b).The sensors are positioned at distance D from each other and fixed over a pad of 160×80 cm 2 in size, as shown in Fig. 7(c).The pad (shown by the dark pattern in the figure) is placed over the mattress and under the sheet.When a person sits or lies on the bed, he or she presses the sensor strips and thus forcing them to generate signals to the microcontroller.Because each sensing strip has a fixed position at the pad, the location of a PD over the BS is easily defined through the reading of the pressure sensors.As our tests revealed, to correctly distinguish the in-bed event from the out-of-bed event for patients having different height (see Fig. 7, c) and shape, the strips have to be placed at the distance D=40cm.A patient is considered in bed if more than 2 pressure sensors are activated.When none of the sensing strips is activated, the patient is assumed to be out of bed.The cost of BS is US70$ for the first design, and 100$ for the second design, including the 8bit ATMEGA 328 microcontroller.

Other Sensor
In addition to abovementioned sensing technologies, we use simple motion sensors installed over the bed, rooms, hallways, etc. to detect patient motion.Also in order to provide visual and audio communication with the patient in a case of emergency, video cameras (such as AXIS M1033-W) equipped with speakers and microphones are used.We assume that at least one video camera (with embedded microphone and speakers) is installed in each room of the home

Implementation and Evaluation
For experimental evaluation, we built a prototype caregiver assisting system (Fig. 8).The system's server receives information from the sensor network, assesses position and activity of the PD and alerts the caregiver if help is necessary by sending text messages via internet to a mobile phone or PC of caregiver.In current implementation, the Activity Assessment Module (AAM) recognizes the following PD's activities: in-bed; out-of-bed; room exit; in-bathroom, in bathroom too long; person fall; entrance door; and exit from home.Fig. 9 shows the flowchart of the assessment algorithm.Here, parentheses determine location of the corresponding door sensor: room, bathroom or home entrance; T1~T3 are programmable timers; alert and alarm are two types of messages that are sent to caregiver.An alarm requires immediate action from caregiver; an alert is information only.The initial values of T1~T3 as well as emergency levels, associated with activities, can be changed by caregiver based on the PD's condition.For example, the caregiver may assign alarm to 'exit from bed' for a weak or unstable PD, whereas for a physically stable individual it has alert level.With new message received, the AAM determines its source, assesses the PD's activity, generates a corresponding alert or alarm to caregiver.After logging the activity in data base, the AAM re-assesses its criticality in a timing loop or waits for a new message from sensors.For example, 'in-bed' event usually has low emergency level.However, when the patient has been 'in bed' motionlessly longer than the time limit T1, the emergency level is elevated to alarm.Any reading from the motion sensor or SC resets the timer.Similarly, when the system detects that the PD is inside the bathroom, it starts timer T2.If the timer goes off, a 'Bathroom Occupied Too Long' event is detected.An input from the bathroom door sensor resets the timer T2.
We implemented the server on a Toshiba PC (2GHz Intel Core Duo CPU, 2GB RAM) and used 2012 Asus Nexus 7 as mobile (caregiver's) device.The mobile device has Android™ 4.3 OS, Tegra3 T30L 1.3GHz CPU, 1GB RAM, 7-inch display.The software was created in C++ by using the Microsoft Visual Studio 2010 (Eclipse 3.6) and the Android software development kit.The user interface and the client software were developed in Java using Java-script interpreter.The client-server communication was implemented through Internet Socket API (ws2_32.lib),WiFi Local Area Network (LAN) and TCP/IP transport protocol.To support the OS-based control of the communication, a dedicated programming interface was also created.The sensor-server communication was implemented based on Xbee Wifi communication protocol using the I2C master-slave based bus interface.
The prototype system has three operation modes, Setting, Monitoring, and Review, which are activated through simple selections from the touch panel.In the Setting mode, the caregiver sets parameters for timers, assigns emergency levels for PD activities, selects camera (if several) for visual monitoring, sets reminders for medication, services, consultations, etc.The "Monitoring" mode provides monitoring of the PD in his room or flat, textual information about alerts, reminders and the real-time visual and audio communication with the PD.In the "Review" mode, the user can view the previous PD activities for the specified time interval (hour, day, month, and year) in the forms of graphic or row data (time, place, activity, generated message) or see frequency or duration of selected activities (e.g.leaving the room, approaching the entrance door, visiting toilet, staying in bed, etc.) over the given period of time (day, month, year).To evaluate quality of monitoring, we performed a number of tests in a mock-up bedroom setting, with a bed and other typical bedroom furnishings.Each test involved six participants, which were asked to perform a series of postures, namely sitting/lying down in bed, walking/standing inside room, falling, lying down on the SC in a "stretched" position, and a "tucked" position, sitting/lying on SC without motion for pre-defined duration, exiting the room, etc.These scenarios were repeated three times by each subject in a random order.
The test positions totaled 103 fall-simulated tasks and 142 non-fall-simulated tasks.The true positive rate for fall detection was 98% with a false positive rate of 0.03%.The bed sensor displayed perfect (100%) accuracy in differentiating the patient in-bed and out-of-bed, respectively.Similarly, the door sensor was confirmed to correctly identify the events, related to room exit; entrance door, exit from home.The posture detection rate of the posture sensor was 94%.(The false positive (in-bed) events were observed when bags were put over the bed).The delay of displaying the results on Nexus 7 device was 0.6s.Overall, the conducted tests have suggested high efficacy of the proposed system in person monitoring,

Conclusion
As the population continues to age and the number of people suffering from dementia grows, the need for information and communication technologies and systems capable of remote monitoring of PD will grow.In this paper, we presented several wireless technologies which can assist family caregivers in monitoring PD activities at home.The technologies are inexpensive, easy to install and maintain.As tests revealed, combining them in a system, allows effectively monitoring of persons, assessing risks and report the events online in real time.By using the system, the caregiver does not need to supervise the PD all the time, as the system does it.At the same time, the caregiver can assess the monitoring events from PC or smartphone, anywhere and anytime, communicating with the patient remotely.As the system does not produce alerts or alarms, the caregiver can do any other work, go shopping, or relax.The ambient intelligence of the technologies and the system enhances the caregiver's monitoring ability and mobility.Although much work remains to perfect the system, we plan to install it in real residential settings of dementia people to conduct methodologically rigorous trials.

Fig. 5 :Fig. 6 :
Fig. 5: An illustration of door sensor and its implementation components

Fig. 7 :Fig. 9 :
Fig. 7: Bed sensor design 2: (a) A photo of FSR408, (b) bed sensor circuit; positioning of the pressure sensors over the bed

Fig. 10
Fig.10 shows a screenshot of the Monitoring mode.The window at the left is the real-time visual of the event.The central window displays statistics of the monitored events.The window at the right displays the flat's floor-plan with location of the monitored person.The text over the window (in yellow) describes the detected activity/event: in bed (in Japanese).

Fig 10 :
Fig 10: A screenshot of the system interface at the PC and smartphone