Thoughtful, detailed coverage of the Mac, iPhone, and iPad, plus the TidBITS Content Network for Apple consultants.

Researchers Demo Ubiquitous Gestural Control via Wi-Fi Sensing

We’ve become accustomed to tapping and swiping our smartphones, but what if you could control devices in your home or office with gestures made from wherever you’re standing? WiSee, a technology developed by University of Washington computer scientists, can recognize gestures by analyzing how specific body motions disrupt Wi-Fi signals — no special sensors or cameras are necessary. The overall effect is a lot like the Xbox Kinect, which relies on cameras and thus works only in a single room.

“This is repurposing wireless signals that already exist in new ways,” said lead researcher Shyam Gollakota, a UW assistant professor of computer science and engineering. “You can actually use wireless for gesture recognition without needing to deploy more sensors.”

The UW researchers created a special receiver that listens to the Wi-Fi signals from all the devices in the home, and looks at the minute Doppler shifts and multi-path distortions that result from human movement within the environment. Using the MIMO (multiple-input and multiple-output) technology inherent to 802.11n, WiSee can differentiate among up to five people within the space. Since you wouldn’t want random movements to be interpreted as gestures, WiSee requires a trigger gesture, after which it watches only that person, presumably until a timer elapses or someone else takes control. The trigger gesture could also serve as a password of sorts. At the moment, only a single device can be controlled at a time, though the researchers are looking at how to control multiple devices simultaneously.

In testing in a two-bedroom apartment and an office environment, WiSee was able to identify nine whole-body gestures with an average accuracy rate of 94 percent (the accuracy rate for random guesses is only 11 percent). False positives — unintentional movements that were interpreted as gestures — happened roughly 2.5 times per hour when two gestural repetitions were required; increasing the necessary repetitions to four essentially eliminated false positives (0.07 events per hour). See “Whole-Home Gesture Recognition Using Wireless Signals” (PDF) for the full paper.

Although WiSee’s proof-of-concept requires a special receiver, the researchers say that the necessary technology could be embedded in consumer-level wireless access points. Also, the paper and demo video say nothing about how WiSee controls things like music volume, television station, and the like — although such hand-waving (hah!) is understandable in a proof-of-concept, a real-world system would have to come up with a coherent method of controlling a potentially wide variety of devices and applications. That’s not impossible, but it is a non-trivial problem.


Backblaze is unlimited, unthrottled backup for Macs at $5/month.
Web access to files means your data is always available. Restore
by Mail allows you to recover files via a hard drive or USB.
Start your 15-day trial today! <>

Comments about Researchers Demo Ubiquitous Gestural Control via Wi-Fi Sensing
(Comments are closed.)

Laine Lee  2013-06-06 11:10
Hey, that suggests a new concept for exercise machines. Anybody want to patent it? You're welcome.
Tonya Engst  An apple icon for a TidBITS Staffer 2013-06-06 14:34
Before I get too old to enjoy it, it is my hope that it will be mainstream to type by dancing.
Kirke Godfrey  2013-06-17 23:52
Wow NO privacy implications AT ALL!
Adam Engst  An apple icon for a TidBITS Staffer 2013-06-18 05:33
First off, this is a proof of concept, not a product, so there are no privacy implications at all at the moment.

That said, I thought about the privacy implications of WiSee for a while when I was writing, but I couldn't really come up with any that are significantly different from those that exist by merely having Wi-Fi in your house now. After all, WiSee supposedly works on a network you control, and in conjunction with hardware that exists only on your network (the gateways to other devices), so full WPA2 security would have to be breached before that WiSee data could be read, as I understand it. And even it were, is there significant privacy implications in knowing that someone raised or lowered the music volume, or changed the TV channel?

Now, if you wanted to extrapolate to beaming your own Wi-Fi signals around someone else's space (which would require setting multiple devices up in the area), you could in theory "spy" on the location of people within the space, but that's going to one heck of a lot of effort when you could instead just look in the window or watch for lit rooms at night. And yes, I can imagine some highly specific scenario where such technology is used to determine the rounds of a guard when trying to breach a secure environment, but let's not let the movie plot fantasies bleed into real life.
Brian Ogilvie  2013-06-18 07:01
I was thinking that this could provide motion detection for a home alarm system without needing to put IR sensors up, though it might require a battery for backup power.
Adam Engst  An apple icon for a TidBITS Staffer 2013-06-18 07:09
Yes, I could see that working, possibly even with gesture-based disarming. And it would probably be smarter about ignoring pets too. :-)