News
Huckaby: For Creative Developers, Evolving Natural User Interface Design Opens Up Opportunities
Natural UI expert Tim Huckaby live-demonstrated at Visual Studio Live! in Austin this week some well-developed but primitive NUI capabilities with his phone that show what is capable in the very near future as the technology matures.
In the future, there may be ways we interact with our computers that we haven't even though of yet. That was the theme of Tim Huckaby's entertaining and forward-looking keynote session on different aspects of natural user interface (NUI) design. Huckaby, the chairman/founder of InterKnowlogy and Actus Interactive Software, and a leading expert on NUI, led Wednesday's keynote session at the Austin, Texas Visual Studio Live! event on June 3. The event at the Austin Hyatt Regency ran from June 1-4.
Huckaby led a session full of demos that explored theoretical and real-world applications of NUI methods, including facial recognition, gesture, touch and voice recognition.
"We've built a killer demo that is the plumbing for the next generation of our apps," he said.
The demo centered on demographic profiling and emotional tracking. "We had a 3D camera looking at you that can tell if you're smiling or not," he said. The goal was to achieve facial recognition with accuracy at scale. "That means I could do facial recognition with $200 camera and be accurate out of user population of 250 million people."
While facial recognition technology has been around for a while, Huckaby's application of it and the fact he can achieve such results for such low cost is groundbreaking. "I run out into the audience and hand my phone to a random person. They point it at me and it recognizes me. That shows that, 'Hey, there's lot of software in the backend. They've built the calculus and trig to structure the face."
Huckaby continued the demo by pointing his phone at the person he initially contacted. "I point it at that person and it doesn't recognize them, but then I add them live. Then it recognizes them. Then I have person follow me up on stage, and the system does the facial recognition thing and picks us out of the crowd."
Using a different device with a 3D camera, the system then attempted a demographic profile. It looked at Huckaby's audience participant (who happened to be Brian Randell, another Visual Studio Live! presenter who also has fairly long hair) and determined it was 75 percent confident he's male, 80 percent confident he's white and sort of confident he's 45 years old. "That shows the weakness of demographic profiling," Huckaby said.
"The facial recognition is bulletproof. The demographic profiling isn't as accurate. It never will be and that's Okay."
The applications include consumer marketing and of course, enhanced security. "The consumer story is demographic driven content and consumers using loyalty programs," Huckaby says. "I agree to be facially recognized in an airport lounge or in a casino in order to get all my loyalty benefits."
He continued with the casino example to discuss potential security applications. "From a security perspective, it's awesome and compelling because you can do it inexpensively. Casinos always have a handful of people who are not allowed in casino, whether they're banned forever or banned for six months," he says. "Yet these people sneak in all the time. It makes total sense to have this form of security running in digital time." He explained that while a banned patron is looking at a digital sign, an embedded camera can conduct facial recognition and alert security that this person has returned.
Another possible implementation is an additional layer of ATM security. "A gazillion people still write their 4-digit pin on their ATM cards," he says. "So it makes sense to have a $200 camera looking down at the person using ATM. If they expect Grandma Huckaby (based on the PIN) and they see some 20-year-old guy, they'll know there's a problem."
Huckaby also tried to assuage any privacy concerns. "I kept telling the audience, 'We're not big brother and we never will be. We're just doing amazing things with the technology.'"
He also discussed other NUI input methods such as touch, gesture and neural input. "The culture has changed. It used to be the only method of input was the keyboard. Then we got mice, and it was the most unnatural thing in the world," he says. "Now it's part of culture. A two-year-old knows how to mouse. They've seen it done or they see it on TV."
That cultural shift now involves touch input. "Now a two-year-old knows how to use touch." The increasing prevalence of touch will lead to similar acceptance of other input methods.
"In a short timeframe, like three to five years, gesture is going to become the prevalent way to use computer systems," he says. "Users will start demanding NUI, touch and gesture-based interfaces—especially on the Web."
About the Author
Lafe Low has been a technology editor and writer for more than 25 years. Most recently, he was the editor in chief of TechNet magazine. He has also held various editorial positions with Redmond magazine, CIO magazine and InfoWorld. He also launched his own magazine entitled Explore New England, and has published four editions of his guidebook The Best in Tent Camping: New England.