Huckaby: For Creative Developers, Evolving Natural User Interface Design Opens Up Opportunities

Natural UI expert Tim Huckaby live-demonstrated at Visual Studio Live! in Austin this week some well-developed but primitive NUI capabilities with his phone that show what is capable in the very near future as the technology matures.

In the future, there may be ways we interact with our computers that we haven't even though of yet. That was the theme of Tim Huckaby's entertaining and forward-looking keynote session on different aspects of natural user interface (NUI) design. Huckaby, the chairman/founder of InterKnowlogy and Actus Interactive Software, and a leading expert on NUI, led Wednesday's keynote session at the Austin, Texas Visual Studio Live! event on June 3. The event at the Austin Hyatt Regency ran from June 1-4.

Huckaby led a session full of demos that explored theoretical and real-world applications of NUI methods, including facial recognition, gesture, touch and voice recognition.

"We've built a killer demo that is the plumbing for the next generation of our apps," he said.

The demo centered on demographic profiling and emotional tracking. "We had a 3D camera looking at you that can tell if you're smiling or not," he said. The goal was to achieve facial recognition with accuracy at scale. "That means I could do facial recognition with $200 camera and be accurate out of user population of 250 million people."

While facial recognition technology has been around for a while, Huckaby's application of it and the fact he can achieve such results for such low cost is groundbreaking. "I run out into the audience and hand my phone to a random person. They point it at me and it recognizes me. That shows that, 'Hey, there's lot of software in the backend. They've built the calculus and trig to structure the face."

Huckaby continued the demo by pointing his phone at the person he initially contacted. "I point it at that person and it doesn't recognize them, but then I add them live. Then it recognizes them. Then I have person follow me up on stage, and the system does the facial recognition thing and picks us out of the crowd."

Using a different device with a 3D camera, the system then attempted a demographic profile. It looked at Huckaby's audience participant (who happened to be Brian Randell, another Visual Studio Live! presenter who also has fairly long hair) and determined it was 75 percent confident he's male, 80 percent confident he's white and sort of confident he's 45 years old. "That shows the weakness of demographic profiling," Huckaby said.

"The facial recognition is bulletproof. The demographic profiling isn't as accurate. It never will be and that's Okay."

The applications include consumer marketing and of course, enhanced security. "The consumer story is demographic driven content and consumers using loyalty programs," Huckaby says. "I agree to be facially recognized in an airport lounge or in a casino in order to get all my loyalty benefits."

He continued with the casino example to discuss potential security applications. "From a security perspective, it's awesome and compelling because you can do it inexpensively. Casinos always have a handful of people who are not allowed in casino, whether they're banned forever or banned for six months," he says. "Yet these people sneak in all the time. It makes total sense to have this form of security running in digital time." He explained that while a banned patron is looking at a digital sign, an embedded camera can conduct facial recognition and alert security that this person has returned.

Another possible implementation is an additional layer of ATM security. "A gazillion people still write their 4-digit pin on their ATM cards," he says. "So it makes sense to have a $200 camera looking down at the person using ATM. If they expect Grandma Huckaby (based on the PIN) and they see some 20-year-old guy, they'll know there's a problem." Huckaby also tried to assuage any privacy concerns. "I kept telling the audience, 'We're not big brother and we never will be. We're just doing amazing things with the technology.'"

He also discussed other NUI input methods such as touch, gesture and neural input. "The culture has changed. It used to be the only method of input was the keyboard. Then we got mice, and it was the most unnatural thing in the world," he says. "Now it's part of culture. A two-year-old knows how to mouse. They've seen it done or they see it on TV." That cultural shift now involves touch input. "Now a two-year-old knows how to use touch." The increasing prevalence of touch will lead to similar acceptance of other input methods.

"In a short timeframe, like three to five years, gesture is going to become the prevalent way to use computer systems," he says. "Users will start demanding NUI, touch and gesture-based interfaces—especially on the Web."

About the Author

Lafe Low has been a technology editor and writer for more than 25 years. Most recently, he was the editor in chief of TechNet magazine. He has also held various editorial positions with Redmond magazine, CIO magazine and InfoWorld. He also launched his own magazine entitled Explore New England, and has published four editions of his guidebook The Best in Tent Camping: New England.

comments powered by Disqus


  • What's Next for ASP.NET Core and Blazor

    Since its inception as an intriguing experiment in leveraging WebAssembly to enable dynamic web development with C#, Blazor has evolved into a mature, fully featured framework. Integral to the ASP.NET Core ecosystem, Blazor offers developers a unique combination of server-side rendering and rich client-side interactivity.

  • Nearest Centroid Classification for Numeric Data Using C#

    Here's a complete end-to-end demo of what Dr. James McCaffrey of Microsoft Research says is arguably the simplest possible classification technique.

  • .NET MAUI in VS Code Goes GA

    Visual Studio Code's .NET MAUI workload, which evolves the former Xamarin.Forms mobile-centric framework by adding support for creating desktop applications, has reached general availability.

  • Visual Studio Devs Quick to Sound Off on Automatic Updates: 'Please No'

    A five-year-old Visual Studio feature request for automatic IDE updates is finally getting enacted by Microsoft amid a lot of initial developer pushback, seemingly misplaced.

  • First Official OpenAI Library for .NET Goes Beta

    Although it seems Microsoft and OpenAI have been deeply intertwined partners for a long time, they are only now getting around to releasing an official OpenAI library for .NET developers, joining existing community libraries.

Subscribe on YouTube