Seeing Sound and Touching Data: How New Senses Will Expand Your World

When Erik Weihenmayer conquered the Seven Summits - the tallest peak on each of the seven continents - his friends were impressed, but not exactly surprised. After all, Erik had scaled Mt. Everest a few years earlier, and he liked to spend his vacations skydiving, skiing and whitewater rafting. Those who didn't know much about him, though, were shocked to learn that Erik has been blind since age 13.

Though not everyone with damaged eyesight aims to become a world-class adventurer, new technologies are transforming the ways that people with visual impairment - or even total blindness - are interacting with the world. Sonic glasses use echolocation to translate sound waves into electrical signals; and vibrotactile displays can map those signals onto a grid of stimulation points across the touch-sensitive surface of the tongue. The newly approved Argus II device, meanwhile, stimulates simple patterns directly onto the optic nerve.

A grid of electrical signals might seem like a poor substitute for the richness of sight - systems like these lack the resolution to render subtleties of color and shading. But as it turns out, advanced technology may not even be necessary for a brain to reconstruct visual imagery from other sensory cues; some blind patients report that, after practice, they're able to literally see the scenes they're "feeling." This isn't pseudoscience - fMRI scans have confirmed that when some blind people touch braille or use echolocation, visual areas of their brain behave as if they were actually seeing an image.

Some scientists and hackers are working to blur sensory boundaries even further. Since the early 2000s, body-modification artists have been implanting electromagnets under their skin - a sense that offers a sort of "buzzing sensation" in the presence of electrically charged objects. And this February, a team of Duke University researchers announced the creation of a brain implant that enables rats to "feel" infrared light through touch-sensitive neural pathways.

Like these scientists, I have a strong suspicion that new technological senses await us in the near future. To explain why, let me try an analogy from a form of technology that's already familiar to us: Phones.

Telephones began as a simple idea: What if we could transmit human voices over wires, just as telegraph lines carried simple electrical pulses? And for the first century or so of the telephone's existence, that sole function was largely its reason for existing - people kept telephones wired up in their homes and offices so they could have real-time audio conversations without the inconvenience of physical travel.

By the early 1990s, though, expectations had shifted. Mobile phones were getting cheaper every year, untethering phone connections from physical wires. Thanks to aggressive marketing by Xerox, fax machines had proliferated throughout the '70s and '80s, cementing the idea that phone lines could send and receive visual images. Although the concepts of fax machines and mobile phones had existed since the turn of the twentieth century, we came to rely on them more and more in our daily lives as they became easier and more reliable to use. Over time, we've begun to regard these tools as extensions of our own senses.

It's not hard to see how this trend has accelerated as phones continue to get lighter and more powerful, scanning and emailing take the place of sending faxes, and audio conversations are ousted by video chats. Touch screens, which seemed thrillingly futuristic just five years ago, now come standard on most smartphones and tablets - a success story that's grown touch screen development into a multi-billion-dollar industry.

Take, for instance, a high-end touch screen manufacturer like Sollensys. Over the past several years, they've helped develop flexible touch-sensitive screens, as well as multi-layer screens that respond to multiple simultaneous tactile inputs. If these developments continue, touch screen companies like Sollensys may soon be printing touch-sensitive menus onto LCD projection screens and walls - and providing flexible electronic "paper" with instant wireless access to the latest news reports.

The next logical step is to mount hands-free wireless devices onto our bodies - which is exactly what Google Glass aims to do in the very near future. And so the phone - which began as a system for hardwired audio communication - has evolved into almost-unrecognizable forms: Wearable, wafer-thin, touch-sensitive pads that link us with a worldwide network of sights and sounds. Useful technology has a way of merging with its users.

Unlikely as it is that most of us will go under the knife for the sake of feeling magnetism or infrared light - or, more to the point, hand over some of our perfectly healthy neurons to questionable new senses - techniques like echolocation, and technologies like vibrotactile tongue maps, seem to hint that such an all-or-nothing approach might not be necessary. If a blind person's brain can be trained to see braille letters - or to create vivid mental pictures from a grid of electrical stimulation - could tomorrow's non-invasive implants help us physically feel the movements of financial markets; of weather systems; of traffic patterns? What new insights might we gain from such intuitive perceptions? What would such expanded senses feel like?

As our relationships with our phones demonstrate, technology tends to expand the range of data we consider integral to our daily lives - and that trend holds a hint about our near future. Though I'm not blind or deaf, I'm still physically unable to perceive most of what's happening around me. I can't see protein molecules or ultraviolet radiation; I can't count a nanosecond or hear a 50,000-Hz sound. Like all human brains, mine evolved, as Richard Dawkins said, "to cope with medium-sized objects moving at medium speeds through medium distances in Africa."

But it also evolved to invent - to build tools that expand its reach light-years beyond that African savannah. Though we might not all want to scale Mt. Everest, we share Erik Weihenmayer's unwillingness to let our limited senses limit our adventures. When I think back to the landlines and card catalogues of the 1950s, I'm amazed that people managed to work and think so efficiently under those constraints. What, then, will our grandchildren think of our touch screens and wireless networks - and with what senses will they look back?