INtelligent senior wellbeing
We are designing and will demonstrate an integrated solution for the remote monitoring, assessment and support of seniors living independently at home. We aim at improving the speed and reliability of health risk detection and support timely, personalized intervention.
We will investigate the use of multiple sensors for the detection and recording of daily activities, lifestyle patterns, emotions, and vital signs, as well as the development of intelligent mechanisms for translating multi-sensor inputs into accurate situational assessment and rapid response. Our goal is to allow seniors to extend their capacity to live at home, improve their quality of life and avoid unnecessary and costly relocations into institutional care. We aim to advance the understanding of how sensor-detected behavioral and cognitive cues correlate with meaningful fluctuations in health status.
We are running 2 pilots: Onlok and Gerijoy.
One takes place at Onlok home-care facilities. We will install non intrusive sensors to detect the target activities automatically of volunteers and design algorithms to automatically analyze long term low level sensorial information. Other pilot is a collaboration with Gerijoy and their platform. The participants interact with the platform and we extract and analyze visual and natural language to asses the participant's mood/emotions automatically.
|
The activities at the top of the video are automatically detected by our algorithm
Activities: we are monitoring 17 activities of clinical relevance:
Eating, Sleeping, Falls, Slowed Movements, Unstable Transfers, Front door Loitering, Day-Night reversals, Fluid Intake, Immobility (bed or chair), Urinary frequency, Restlessness, Fever, Alcoholic Consumption, Pill Consumption, High Salt Diet, Substance Abuse, Food Consumption.
We are members of the Stanford Program in Artificial Intelligence assisted care (PAC), an interdisciplinary team composed of Stanford university academics and private institutions.
We aim at helping people located at the top of the US population pyramid belonging to the following groups:
|
|
demo of long term activities |
The automatically detected activities are displayed at top left of the video
|
The system recognizes automatically the ongoing activity in a video or live stream. By several stages of grouping the human motion (i.e. the lines in the video below) and mapping the motion groups to activity labels (e.g. eating). How it works? see publications 2014.
|
The system configuration takes 4 minutes (for 6 activities x camera). The algorithm uses a single example (one shot) activity, and active learning for a fast configuration. In 2015/2016 we work on technology that require of no configuration.
The recognized activities are used to build temporal graphs of "humanized" statistics that can help doctors to better diagnose diseases and predict potential health risks. Also, abnormal situations can be detected and alarms are triggered when abnormalities occur for example a fall.
The recognized activities are used to build temporal graphs of "humanized" statistics that can help doctors to better diagnose diseases and predict potential health risks. Also, abnormal situations can be detected and alarms are triggered when abnormalities occur for example a fall.
|
thermal SEnsors |
|
We trained a deep neural networks to detect 11 activities in total darkness using thermal data.
The model was trained with few examples of sleeping, sitting, standing and empty. About 800 frames of each. The model is a CNN of 9 layers. 80/20 training/testing. Testing error top1=7.3% almost no overfitting. Only 24 epochs. Quantitatively you can see that in the validation sequence there are some confusions. This is a toy example, more training data is needed. |
|
demo sentiment analysis |
|
The sentiment detections (labels) are detected automatically. The video is unconstrained, downloaded from youtube. The algorithm was developed at PAC, and it is based on state of the art deep machine learning techniques, which requires no configuration. The analysis of sentiments is complementary information to the long term activities detection.
|