Skip to Navigation Skip to Page Content

These images demonstrate fall detection using a “voxel person.” The top row shows one image sequence of a person falling, and the bottom row shows the corresponding voxel person representations computed from the camera images. The color indicates the classification from the automated program: red for an upright position, blue for on the ground, and green for in between.

Help! I’ve fallen and I can’t get up!”

It’s a catchphrase that has been the stuff of jokes and parodies since a 1980s stiffly dramatized commercial for a medical alarm company, but fall risk assessment and prevention for the elderly is serious business.

Researchers at the University of Missouri have been collaborating for the past five years on a project aimed at helping to prevent the nightmarish scenario of elderly parents or grandparents who fall and lie injured for hours before help arrives.

The eldertech team,” as it’s been nicknamed, is an interdisciplinary group of remarkable scale, with researchers from multiple fields working closely together. Marjorie Skubic, professor of electrical and computer engineering, is leading the effort on the engineering side.

It’s a fantastic example of interdisciplinary work. It illustrates engineers helping people in everyday life. It brings interesting dimensions. We work with nurses and physical therapists and social workers; we’re publishing in engineering and nursing journals,” Skubic said.

At the heart of the eldertech research is the development of a full suite of sensors designed to be installed in a home, log its inhabitants’ movements, and raise alarms if they detect potential falls. The team also intends to make it more readily available than existing manual methods for assessing fall risk.

TigerPlace is a retirement community envisioned by the MU Sinclair School of Nursing and built and operated by Americare. It’s a pleasantly attractive series of apartments with individual screened porches. Inside the main building, a life-size stuffed tiger rests next to the grand piano in the foyer and flat-panel televisions hang next to inviting furniture. There is a small movie theater and meals are prepared by chefs. From appearances, you wouldn’t know it’s actually a state-of-the-art eldercare facility where residents volunteer to participate in research projects.

In an empty apartment, Skubic points out where the inconspicuous sensors would be if a full set were installed. A full suite of sensors would include motion sensors on walls, the ceiling over the door, by the stove, in the fridge, in the cabinets,” she said.

Besides monitoring movement, sensors also collect data on when the stove is turned on, when the fridge and cabinets are opened, when the washing machine is put in use, when the shower is on, and so on. Taken as a whole, the data builds a picture of the inhabitant’s overall lifestyle pattern and levels of activity.

That’s the scope of the eldertech work — not just fall risk assessment, but actual prevention by monitoring an elderly person’s activity and capability for independent living.

The general idea is to identify patterns in residents’ activities and recognize changes in those patterns to work toward early illness recognition. We want to identify early signs of change or predict when decline occurs so we can take preventive steps,” Skubic said.

In other words, if grandma suddenly stops cooking every day or hasn’t stepped outside her apartment in days, the data could indicate the onset of a physical or emotional problem — depression has been correlated with low activity levels.

Besides motion sensors, other technology currently under development include 2-D and 3-D video camera systems. This kind of monitoring is passive.” Current active” alarm systems requiring elderly people to wear alarm pendants and press them during emergencies have not always been  effective.

We don’t want the residents to have to ‘do’ anything. If they have to put a pendant on, it may not be on them when they need it. Even if they have it on, they may not be able or willing to activate it,” Skubic said.

Another eldertech product is a footstep-sensing rug. Having one’s home literally carpeted with sensors might seem a little invasive, but maintaining monitored individuals’ privacy has always been a primary concern. To this end, much thought and many graduate students’ work have been invested to develop the technology of silhouette extraction, software that can take raw video data of humans in a home environment and reduce the images down to human-shaped blobs devoid of identifying characteristics, or even indications of what kind of clothing they’re wearing.

This is part of what makes the research cutting-edge.

Our group’s focus is sensory analysis, pattern recognition, and intelligent decision-making. The trick to it isn’t the sensors or transducers; it’s how the sensor data are processed,” Skubic said.

One of Skubic’s graduate students, Fang Wang, has been working on this problem since 2008. The doctoral student  in electrical and computer engineering, demonstrated results from the team’s silhouette extracting software with a batch of data from a recent project in which researchers recruited 10 elderly residents of TigerPlace to act out various scenarios — like housecleaning or receiving visitors — for data collection.

On the computer screen, an animated blue blob, composed of three-dimensional pixels, or voxels,” moves around in a 3-D grid. A head, two legs, and an occasional arm are represented, but it’s otherwise unidentifiable. Wang said they do not save the raw image data and only study the silhouettes.

The silhouette extraction software isn’t perfect; shadows and obscuring environments can be tricky. But using common webcams, both the 2-D and 3-D camera systems provide a way to visually monitor and assess elderly inhabitants’ gait, movement, and potential falls, portably and relatively inexpensively. Expensive systems such as the Vicon motion capture system are confined to labs and studios.

Previously, to assess fall risk, residents had to go to a doctor or physical therapist every six months. Now we can monitor daily before something drastic happens,” Wang said.

The camera systems have been tested during hundreds of data runs, involving both residents of TigerPlace and stunt actors trained to perform 21 different kinds of falls.

In addition to motion sensors for recording activity around the house, camera systems to monitor for falls and collect data for fall risk assessment, and silhouette software to maintain residents’ privacy, the research group recently received grants to investigate acoustic and radar sensing as well.

Jim Keller, professor of electrical and computer engineering, got involved with the eldertech research at Skubic’s request.

Since I’m getting old, of course,” he said wryly.

Keller has a long history with pattern recognition, image processing and computational intelligence — skills which apply to the eldertech research as well as to his other fields of expertise. He says radar could come in useful for detecting breathing patterns and monitoring heart rate, especially at night when the resident is sleeping.

We use radar in landmine detection, and it’s the same principle: send a pulse out and it comes back and tells us what it hit,” Keller said.

Meanwhile, the acoustic sensors, possibly consisting of a circular array of microphones, would detect the sound signatures of potential falls, perhaps steering a camera system to that location and, most importantly, pinpointing the height at which the audio signal is sensed. If the sound is several feet up, the system can conclude that the signal is not due to a fall.

Additional technology includes a pneumatic bed sensor strip, originally designed at the University of Virginia. The strip measures pulse and respiration rates and restlessness in bed at night. The group also is considering adding pyroelectric sensors. The raw signals are promising,” according to Skubic.

We’ll never know which sensor will give the most useful information at any given moment,” Keller said. So we’re aiming for fusion. We want to take lots of information and fuse it together to get a big picture.”

The eldertech team is a good example of fusion. Marilyn Rantz, professor with the School of Nursing, came to the College of Engineering over five years ago looking for collaborators on a project for incorporating technology into TigerPlace. It took a while to brainstorm the idea of a sensory network and to get funding for it, but the group has since grown greatly in size and is now bringing in large grants. The professors,  graduate students and undergraduates all working on eldertech projects will benefit from the $1.3 million budget slated for fiscal year 2010 as a result of three new grants awarded this fall.

Skubic said the eldertech team is unusual in that all the disciplines within it are balanced out.

Often, we see one discipline take over. That’s not the case here. We all work together because we recognize that the work can’t be done by one discipline alone. It requires all to coordinate together,” Skubic said. We approach the problem in a more integrated fashion, which has flavored our work.”

You never know if a good technical idea might come from the nurses,” Keller said. We’ve developed a relationship of accepting others’ input as valuable.”

The eldertech group has made a splash in the larger scientific community. Recently, they submitted nine research papers to the IEEE Engineering in Medicine & Biology Society conference and almost had an entire presentation aisle to themselves. Graduate students have used the project as a springboard for their own careers. One in particular spun it off into his own dissertation and got a funded fellowship for informatics.

Skubic intends for the project to eventually install full systems into private homes for 24/7 monitoring.

If we can show early recognition of illness, detect falls and identify risk factors, we can help keep people in their homes longer, improve their quality of life and keep them functionally capable for longer,” Skubic said. We are especially interested in  keeping them moving safely and keeping them independent.”

Skubic’s team is investigating methods for displaying data in an effort to better interpret it and understand potential correlations to health conditions. Shown here are two examples of two different resident’s month-long activity density map generated from motion sensors positioned in the various rooms of the apartment. Each horizontal line in the map represents activity density for one day of the month. The black regions are the times of the day that the resident was out of the apartment. From the maps, you can see the tendencies of eating breakfast, lunch, and dinner out of the apartment, as well as other away-from-home times. Color is used to indicate the level of activity as computed by the number of motion sensor hits per time unit in the apartment; a more colorful map indicates a more active resident. The two maps indicate two different residents with very different lifestyles. Over time, a reduction in color (indicating less motion in the home), and fewer black regions (indicating less time outside the home), may indicate possible depression, or may indicate a health problem that needs to be addressed.



Categorized as:


This story is tagged as:

Magazine Issue: