Tom Zimmerman, an Inventor’s Look at Video Games and the Human Interface: Past and Future

Human interface - where the bits meet the flesh. It’s these bits that caught Tom Zimmerman’s interest back in the early 80s, when he first dreamed of inventing a way to play the air guitar with motion sensors. Since then he has continued to push the boundaries of how we interact with our technology, inciting the concept of virtual reality with inventions such as the Data Glove. Zimmerman is a human/machine devices expert with IBM Research, where he continues to push the frontiers of technology.

The last 30 years have seen a boom in technological infrastructure and data collecting devices. Very early on, people were interested in developing technologies that allowed them to use their bodies to accomplish digital tasks. As an inventor, Zimmerman always had a playful curiosity and sense of creativity that put him on the very edge of this new movement of motion sensors and simulation. He was able to understand a current technology and realize completely new and revolutionary applications thereof. In fact, he’s still doing the same thing today.

Zimmerman sees a world full of data, most of it now being collected through various means. Sensors on our roads keep track of traffic, satellites monitor the weather, and after a few scans a person’s vital statistics can be put to a spreadsheet. Every facet of a country’s success and failure can be mapped, the tides of the ocean and the temperature of the equator always documented. Even more recently, our social lives have become at least as well documented. Facebook data can be analyzed, text messages and tweets stored away. Specialized scanners and cameras can create impressively detailed 3D models of the scenes they observe. Most of this data is viewed using classical spreadsheets, pie charts, and bar graphs. The more advanced technologies allow for impressive 3D maps of data, showing interaction in vast clouds of content, extremely difficult to read and very cluttered.

How does this relate to video games? Well, Zimmerman has always seen connections between seemingly unrelated things. He converted a device designed to track the bow movements of violin players into a device that broadcast an electric field of data around the user, allowing users to exchange virtual business cards just by shaking hands. Now, Zimmerman loves not just video games, but video games that allow powerful immersion especially. Instead of creating a virtual world from scratch, why not use all of this data from the real world?

Zimmerman is convinced that games will become more and more relevant the more hyper-real they become. He doesn’t want to look at a screen and hold a controller, he wants to hold virtual objects with his hands. He sees a future where students learning about Roman History can walk through a simulated Rome, as it was in its glory days.

And why stop at locations? With all of this personal data floating around, perhaps one day we will be able to map out a personality to the point where we can achieve “synthetic immortality”. Zimmerman’s great grandchildren might be able to talk with a simulated version of himself, ask him what life was like, for advice. A computer could pour over all of the written works of Benjamin Franklin, and create an artificial Ben that students could talk with. The applications of such a technology would be endless.

Tom Zimmerman dreams big, and he believes that video games will be the cornerstone for human interaction in the coming years. We were happy to get just a taste of his vision.