Activity Recognition for Healthcare Based on Slow Intelligence Systems

Activity recognition requires the deployment of sensors either on the human body or in an environment where the subject lives to identify activities. However, due to the fact that sensors are subjected to a variety of errors and the nature of human activities are dynamic and uncertainties, to develop an accurate activity recognition system merely with sensor readings is a challenging task. In this study, we present a location-aware approach for improving activity recognition accuracy by analyzing images captured by a wearable camera. We focus on recognizing activities of daily living for the potential application in senior healthcare. We adopt speed-up robust features (SURF) to detect objects and provide its location information combined with sensor readings to identify the performed activity. With the location information, some unlikely activities can be ruled out so the recognition accuracy can be improved. Our experiments show that, on average, the proposed approach has better recognition performance compared to the same classifier without location information from various test scenarios.


Introduction
The increase of aging population with disabilities has become a major concern in developed and developing countries, causing the rising demand for long-term care.The service for home healthcare is an important issue in aging societies worldwide as many reports show that most seniors prefer to age in place.Aging in place refers to the ability to live in one's own home independently.
The decrease of fertility rate in Taiwan has accelerated the progress of aging population, leading to a potential need of senior care.The goal of this study is to help seniors maintain living independence within the comfort of their own homes with the assistance of modern technology.According to the definition of aging society from World Health Organization (WHO), Taiwan has become an aging society since 1993 (1) .The percentage of the population in Taiwan aged over 65 years is predicted to be doubled by 2017 (2) .Additionally, the homecare services in Taiwan are impaired by the shortage of registered nurses and high nurse-patient rate (3) .Therefore, it is necessary to pay more attention to finding alternatives for senior care to cope with the rapidly aging population.
There are many ways to improve the quality of life and keep people healthy by taking advantage of technology.Human activity recognition has many potential applications in healthcare.Successful examples include helping doctors keep track of patient history, analyze patient behaviors in order to make a better diagnosis.In this study, we endeavor to create an effective activity recognition system that can help people live in their own homes safely and comfortably with the assistance of technology.
The remainder of this paper is organized as follows: Section 2 reviews the related work in activity recognition.Section 3 describes the software framework of the proposed healthcare system, followed by experimental results in Section 4. Finally, the conclusions are drawn in Section 5.

Related Work
Activity recognition has been investigated for more than two decades (4) .Numerous approaches with different applications have been proposed in past research ( 5)- (7) .The type of activity recognition can be broadly divided into sensor-based and vision-based recognition.
By deploying sensors in an environment or attaching on a human body, sensor-based activity recognition attempts to identify different activities from recognizing sensor readings.Although many efforts have been devoted, this type of activity recognition still remains challenging as humans perform activities differently and sensor readings may contain incorrect or noisy data.
As to vision-based activity recognition, the challenging part is to recognize activities through captured images.So, it heavily relies on image and video processing techniques.The privacy issue is another concern in vision-based activity recognition.
Activity recognition can be considered as the problem of activity classification.In the past decade, various classification algorithms have been applied to the design of activity recognition, such as logistic regression, decision trees, naïve Bayes, and support vector machines (8) .Each classification algorithm has its pros and cons, and no single algorithm outperforms others in all cases.
Learning the patterns from sensor data is critical for activity inference.The location where a person is engaged provides some extent of useful information for activity reasoning.For example, if the user is in the kitchen, the activity of eating or meal preparation would be more probable than dressing or bathing.When the user is in a bathroom, it is more probable to take a shower, brush teeth or flush the toilet than take a rest.Scene information provides a useful clue for reasoning activity.
However, most of the sensor-based activity recognition is a lack of location features.Therefore, the development of a robust and complex algorithm is required to obtain an accurate recognition model.In this study, we adopted the sensor-based activity recognition approach combined with location information through analyzing image inputs from a wearable camera to increase the recognition accuracy.

The Software Framework of the Proposed System
Considering the software usability and scalability, we proposed a component-based software architecture for the location-aware activity recognition system.The proposed framework, as shown in Fig. 1

Slow Intelligence System (SIS) Server
The SIS Server component is responsible for routing and processing messages among components (9) .When the server accepts a request message from a component, the message will be processed and routed to the designated components as responses.All request and response messages are wrapped in the XML format for usability across the Internet.Some particular messages were also defined in the server to execute specific operations.

System Interface (SI)
The System Interface component provides a graphic user interface (GUI) for users to specify system properties, including user names, which e-mail address an alert should be sent to.The properties set by this component are stored in a shared database for other components to access.

Universal Interface (UI)
The Universal Interface component is used to simulate messages routing among components.Any message sent from one component to another will be displayed.It is useful when testing the developed system by observing routing messages through the UI.The user can enter a message and observe the corresponding output message.

Activity Recognizer (AR)
The Activity Recognizer component is designed to recognize activities by analyzing sensor data from the Environment Sensor component and image inputs from the Image Sensor component.The Activity Recognizer component consists of four units: a data preparation unit, an image processing unit, a recognition unit, and a decision unit.The data preparation unit is to convert sensing data into feature vectors.The image processing unit is to process images and perform feature extraction for the recognition unit, while the recognition unit is to recognize activities.The decision unit is to make a decision of whether an alert needs to be sent to the caregiver when detecting an anomalous condition.

Alerter
The Alerter component is responsible for sending an alert e-mail or text message to the user to notify an anomaly is detected so that the caregiver can take action accordingly.

Environment Sensor (ES)
The Environment Sensor component is designed to read and store all available sensor data for observing environmental conditions.In addition to analog sensor readings, Environment Sensor also can handle binary sensor data such as pressure switches and proximity sensors.

Image Sensor (IS)
The Image Sensor component is designed to read and store the images captured by the wearable camera from an egocentric point of view without exposing the user's body to lower the privacy concern.When a newly captured image is generated, a message will be generated and sent to notify the SIS server.The stored images are analyzed by Activity Recognizer.
Image Sensor provides the location information through scene analysis.In this study, we used the speeded up robust features (SURF) algorithm to detect whether a significant object is present.SURF is a scale and rotation-invariant interest point detector and descriptor.The training images were collected and manually annotated for SURF feature descriptors.The SURF descriptors are feature vectors describing certain sections of an image.
The image processing unit in Activity Recognizer first computes the SURF feature descriptors over all the training images for an observed object.This is done to avoid re-computation of these values for each image received.The idea is that the image whose SURF descriptors match the given threshold are likely to be showing the same object.
When an image is received, the image processing unit will start to load the image and perform the SURF algorithm to detect whether a certain object such as a TV or bed is present.This information can help the Activity Recognizer identifies activities.
The information obtained from Image Sensor is integrated with the information from Environment Sensor as data fusion so that the recognition performance can be improved.When an abnormal activity is detected, an alert will be sent to the caregiver.The procedures performed by the Activity Recognizer are described as follows.
Step 1: When a captured image is loaded, the SURF descriptors are computed on that image to identify regions of interest in the image.
Step 2: The SURF descriptors are matched using the Fast Approximate Nearest Neighbor algorithm against the pre-computed descriptors of every training image in the training set to find the closest matching descriptor and then compute the distance.The matched vector is the one that has the least Euclidean distance.
Step 3: For each of the matched sets, the overall normalized match strengths are computed.The cosine distance between each of the matched vectors was adopted as similarity measure.
Step 4: Those matches whose distance is greater than the experimentally determined threshold are considered as bad matches and are discarded.Note that this step is critical as it will eliminate many bad matches, but also preserve the good matches as the process of concentration.The image containing the most matches is selected.
Step 5: The count of matches for the final image above the threshold value is to determine if a significant object is found.

Experimental Results
There are several public datasets available online for the assessment of activity recognition.Washington State University's Centre for Advanced Studies in Adaptive Systems (CASAS) is a widely used data repository for activity recognition.To evaluate the performance of the proposed approach using scene information, we used the 'Milan' dataset taken from CASAS (10) .Fig. 2 show the layout of the Milan test bed where 33 sensors were deployed.It includes 15 different activities as listed in Table 1.With the information of the relationship between activities and the place where a person is engaged, the recognition accuracy can be improved.Table 2 lists the relationship between some commonly used objects and their probable locations in a home setting.In most families, the living room, kitchen, bathroom, and bedroom are the areas where activities are performed in a residential home.Therefore, we focus on recognizing objects and activities in those areas.By ruling out activities that are unlikely occurred in a specific area, the recognition accuracy can be improved.Table 3 lists the relationship between areas and their probable activities.For example, the activities 'Bed-to-Toilet' is unlikely to happen in the living room.Table 4 shows the recognition accuracy by a naive Bayesian classifier without using the scene information.When a wooden sofa is detected, as shown in Fig. 3, this scene information provides the inference with a useful clue that the user is in the living room area.Therefore, some unlikely activates can then be ruled out, which will increase the recognition accuracy as listed in Table 5.The total instances, correct classified instances, and incorrect classified instances for this case are 914, 734, 180, respectively.The overall accuracy is 0.80.When a toilet is detected, as shown in Fig. 4, this scene information infers that the user is in the bathroom area.Therefore, some unlikely activates can then be ruled out as listed in Table 6.The total instances, correct classified instances, and incorrect classified instances for this case are 990, 902, 88, respectively.The overall accuracy is 0.91.When a refrigerator is detected, as shown in Fig. 5, this scene information infers that the user is in the kitchen area.Therefore, some unlikely activates can then be ruled out as listed in Table 7.The total instances, correct classified instances, and incorrect classified instances for this case are 659, 570, 89, respectively.The overall accuracy is 0.86.
When a bed was detected, as shown in Fig. 6, this scene information infers that the user is in the bedroom area.Therefore, some unlikely activates can then be ruled out as listed in Table 8.The total instances, correct classified instances, and incorrect classified instances for this case are 904, 684, 222, respectively.The overall accuracy is 0.75.
Due to space limitation, we only provide some examples in this paper.From the experimental results, we found that adding the scene information in the inference process can increase the recognition performance.

Conclusions
Activity recognition plays an important role in ubiquitous healthcare applications.The activity of a senior patient reveals useful information about his or her health status.With activity information, medical professionals can provide senior patients medications efficiently and assess the health problems such as Alzheimer's in an early stage.Activities and locations have some sort of relationships that provides a useful clue for reasoning activities.In this study, we have presented an effective approach and a software framework for activity recognition by using the scene information gathered from a wearable camera.Experimental results have shown that the proposed approach offers a feasible solution to activity recognition for healthcare.

Fig. 1 .
Fig. 1.The software architecture of the proposed activity recognition system.

Fig. 2
Fig.2The layout of the Milan dataset from the CASAS dataset.

Fig. 3
Fig.3The matched process for the wooden sofa detection , consists of eight software components, including Universal Interface (UI), Slow Intelligent System (SIS) Server, System Interface (SI), Alerter, Activity Recognizer (AR), Environment Sensor (ES), Image Sensor (IS), and Activities of Daily Living (ADL) Knowledge Base.

Table 1 .
Activities included in the Milan dataset

Table 2 .
The relationship between objects and their probable locations

Table 3 .
The relationship of areas and their probable activities

Table 5 .
Recognition accuracy with scene information (recognized object: wooden sofa)

Table 6 .
Recognition accuracy with scene information (recognized object: toilet)

Table 7 .
Recognition accuracy with scene information (recognized object: refrigerator)

Table 8 .
Recognition accuracy with scene information (recognized object: bed)