Log out
My profile and settings
My bookmarks
Comment history
Please complete your account verification. Resend verification email.
This verification token has expired.
Your email address has been verified. Update my profile.
Your account has been deactivated. Sign in to re-activate your account.
View all newsletters in the newsletter archive
You are now unsubscribed from receiving emails.
Sorry, we were unable to unsubscribe you at this time.
Back to profile
Comment Items
You have not left any comments yet.
you replied to a comment:
Saved Posts
You haven’t bookmarked any posts yet.

At a time when some countries seem to be turning inward and retreating from world affairs, China is primed to step up as a global leader.

read more
Become a Gates Notes Insider
Sign up
Log out
Personal Information
This email is already registered
Please verify email address. Click verification link sent to this email address or resend verification email.
Email and Notification Settings
Send me updates from Bill Gates
You must provide an email
Send me Gates Notes survey emails
Send me the weekly Top of Mind newsletter
Email me comment notifications
On-screen comment notifications
Select interests to personalize your profile and experience on Gates Notes.
Saving Lives
Energy Innovation
Improving Education
Book Reviews
About Bill Gates
Account Deactivation
Click the link below to begin the account deactivation process.
If you would like to permanently delete your Gates Notes account and remove it’s content, please send us a request here.

Kinect / Connect

The power of the natural user interface

There’s a reason why the Kinect for Xbox 360 holds the record as the fastest-selling consumer electronics device ever. The innovative software underneath the hood is enabling millions of users worldwide, including me and my family, to interact with and immerse ourselves in games in ways never before possible.


We play some of the Xbox 360 Kinect games in my house and it’s hard to convey just how amazing it is when you find yourself immersed right in the middle of a game, and how much fun it is for the whole family to play games together where we not only control the action in completely natural ways, but we actually are the action.

Kinect is a motion-sensing input device that’s a revolutionary new way to play games using your body and voice instead of a controller. Now, instead of playing a soccer game by mastering a series of button commands that have nothing to do with soccer, you play by using your feet, head, body, and, if you are the goalie, your hands. To play ping pong, you move your hand just the way you would if you were playing in the garage with a ping pong table, a ball, and a paddle.

Kinect is a remarkable technical achievement. The ability to take video cameras, multi-array microphones, and depth sensors and bring them all together in order to recognize people, understand and anticipate how they move, incorporate voice recognition, and insert them into games—all at 30 frames per second—is phenomenal. (It holds the Guinness World Record for the fastest-selling consumer electronics device ever, with 8 million units sold in the first 60 days it was on sale.)

A lot of the advances that make Kinect possible have been under development at Microsoft for years, even decades. For example, computer scientists at Microsoft Research have been working to solve the hard problems in speech recognition and video capture technology for many years.

Kinect is much more than just a cool video game technology—it is the most advanced example of a wave of new advances that are enabling people to interact with technology in entirely new ways.

One of the most important current trends in digital technology is the emergence of natural user interface, or NUI. We’ve had things like touch screen and voice recognition for a while now. But with Kinect, we are seeing the impact when people can interact with technology in the same ways that they interact with each other.

I’m convinced this is a transformational technology. Until now, we have always had to adapt to the limits of technology and conform the way we work with computers to a set of arbitrary conventions and procedures. With NUI, computing devices will adapt to our needs and preferences for the first time and humans will begin to use technology in whatever way is most comfortable and natural for us.

This has profound implications. At the most basic level, it means people will be able to use technology without any special knowledge or training. Advanced computing tools that can only be used by experts with highly specialized skills today will be available to anyone in the not-too-distant future.

Even more important, NUI means people who may not have even the most basic literacy skills will be able to take advantage of some of the benefits of digital technology and the information age for the first time. Kinect’s movement sensor and voice control tech­nologies also have the potential to make computers more accessible to people with physical and cognitive dis­abilities. A surprising number of academic researchers and others are exploring using Kinect in ways we never imagined. In the UK, for example, scientists are developing robots using Kinect’s inexpensive (but so­phisticated) motion-sensing technology to search for survivors in potentially unstable buildings after an earthquake. Researchers in Seattle are exploring how Kinect can give surgeons a “virtual” sense of touch during remote surgi­cal procedures.

Earlier this year, Microsoft Research released a free software development kit for noncommercial applications of Kinect. If you go to the website, you can see dozens of examples of projects—some of them just for fun and others with a more serious purpose.

It’s exciting to think about the many ways a natural user interface will be used, including by people with little knowledge of technology, to tap into the power of computing and the Internet.