How Virtual Reality Endured Numerous Failures To Go From Sci-Fi Dream To World Changing Tech


In 1968, at MIT’s Lincoln Laboratory, you could walk in and see what many consider the first true virtual reality device. Showing simple wireframe models to the user, it was a bulky, experimental rig, essentially two televisions hung from the ceiling. What many saw was the future of not just computer graphics, but a technology that could potentially change the world, a new way of simulating and solving real world problems. Simulating the real world in exacting detail promised to change engineering, architecture, physics, and a host of other disciplines. The rig at Lincoln was promise, concentrated, hanging from the ceiling in microchips and power tubes.

One observer, though, had more practical concerns. Observing the heavyweight, and the thin wires suspending it, an unknown wit christened it “The Sword of Damocles.” Unknowingly, he predicted the future of VR; just like the Sword of Damocles cuts both ways, VR has found enormous success even as the prize it really wants, a revolution led by a headset on every desk, continues to elude it.

A Dream For The Future

Like so many technological marvels, virtual reality was first theorized in science fiction. In 1935, Stanley Weinbaum’s short story, Pygmalion’s Spectacles, more or less predicted it. The story follows a camera executive and a gnome-like professor who demonstrated a clever device using photography and chemistry to fool the viewer into seeing another world.

Unbelieving, still gripping the arms of that unseen chair, he was staring at a forest. But what a forest! Incredible, unearthly, beautiful! Smooth boles ascended inconceivably toward a brightening sky, trees bizarre as the forests of the Carboniferous age. Infinitely overhead swayed misty fronds, and the verdure showed brown and green in the heights. And there were birds — at least, curiously lovely pipings and twitterings were all about him though he saw no creatures — thin elfin whistlings like fairy bugles sounded softly.

In the ’50s and ’60s, Weinbaum’s unlikely vision became the preoccupation of oddballs, engineers, and most importantly, computer scientists. Recreating reality, and doing so perfectly, was the core goal of computer graphics engineers, and in 1978, the first real breakthrough arrived: The Aspen Movie Map. Using a combination of photography techniques not unlike those of Google Street View and polygon rendering, MIT researchers created a shockingly accurate, for the time, look at the city of Aspen in winter and summer. It was a demonstration that VR could be used to recreate, and simulate the real world.

This inspired Atari, at the time one of the biggest tech companies in America, to start its own virtual reality lab in 1982. Among the employees were Jaron Lanier, who first popularized the term virtual reality. When Atari shut down the lab due to the video game crash of 1984, Lanier took his ideas and technology and founded VPL Research. Meanwhile, across the Atlantic, post-graduate student Jonathan Waldern founded W Industries, a company that was building off his groundbreaking research, in his garage.

Lanier’s interest wasn’t in games, but in creating virtual worlds for practical purposes; VPL was short for “visual programming language.” Lanier had grander ambitions, but to start with, he wanted to make programming a computer as simple as drawing a picture. VR, in Lanier’s view, was a means to a technological revolution. The idea was total immersion, and to that end, Lanier created the components of VR we know so well, the headset, which Lanier called the Eye Phone, and the Data Glove.

But world-changing revolutions aren’t cheap. VPL’s VR rig needed over 6,000 microcontrollers (tiny computers on a chip, a job they ultimately hired a contractor to help with since he was the only one that could program them) and supercomputers that dwarfed the power available from desktop PCs at the time. The cutting edge in consumer computers when VR was being pioneered could perform 40,000,000 operations a second. For comparison, the iPhone 7 is fifty-one times as fast.

As a result, Lanier and Waldern’s greatest successes were largely industrial, with major corporations that could afford supercomputers and saw the appeal of simulating products and environments for testing and training. The military began using it for training pilots as early as 1982, architects would render buildings in polygons and “walk” clients through them, and the automobile industry tested features for safety without putting anyone at risk on the road. VR subtly changed the world around us in the ’80s, with better-trained pilots, better-designed building, and better-tested cars, but it was far from the revolution VR advocates were hoping to launch.

For that, VR had to somehow be in every home. So, hoping to kickstart the revolution they’d dreamed of, VPL hesitantly licensed their Data Glove technology to a toy company, marking the first swipe of the sword of Damocles; VR finally got in front of consumers, but many advocates had to watch it stumble.

The Power Glove… It’s So Bad

The first mass-market VR tool wasn’t a visor or a high powered computer system. It was the Power Glove, Mattel’s attempt to break into the world of Nintendo. Nintendo had confounded American toy companies with its success; after the failure of Atari, many had left the industry for dead, and now found themselves struggling to catch up.

The Power Glove was, at best, a distant cousin to the Data Glove it was based on. Mattel, in order to put the technology in department stores and catalogs at anything resembling an affordable price, had to brutally scale down the technology, and conveniently left out the frame users had to set up for the thing to work from the advertising. Still, it was fascinating, and cool, enough that Nintendo gave it pride of place in its feature film/infomercial starring Fred Savage, The Wizard. It’s easy to forget that movie was a tantalizing hint to kids around the country of the cool stuff Nintendo had in store, and the Power Glove was presented as the most radical, the most awesome, the controller the kid who could beat the title character used. The Power Glove was the future of Nintendo. Every cool kid was going to have one.

The Power Glove still has a strange hold on the popular imagination. Freddy Krueger killed one of his victims with one, it regularly turns up as a piece of ’80s nostalgia, and it’s especially powerful in music, with two dueling metal bands, Powerglove and Power Glove, and musical tributes beyond that. Australian electronica band Knife Party even named a song after it that samples the original commercials.

But at the time, it was a disaster for Mattel. It didn’t work well with most games, and the games designed for it were sold separately, meaning many kids bought a $100 controller and had nothing to play. The Power Glove rapidly went onto the enormous pile of gimmick controllers the NES had all throughout the 1980s. It was, however, a breakthrough in the sense that the public could better understand it. VR was no longer a theoretical idea that preoccupied a handful of Bay Area nerds. It was a concept you could try out at Sears. More importantly, video game companies were engaged in a console war, pouring money into computer graphics research, working with titans of Silicon Valley like Silicon Graphics, and putting millions of low-cost computers into homes. Video games would be, according to VR’s most passionate advocates, the portal to making virtual reality mainstream.

Lanier continued to work on VPL, but Waldern stepped into the gap with his company, now named Virtuality. Lanier found himself increasingly drawn to “telepresence” systems that let you command a robot with VR tools, so Waldern, who had both the enthusiasm and the intellect to sell VR, stepped in. Seeing an opportunity, Waldern began pitching VR as the future of arcade gaming.

As a result, gamers couldn’t open an issue of GamePro, or walk into a high-end arcade, without learning virtual reality was coming. Sega announced the Sega VR, a cutting edge virtual reality headset. The VR revolution in gaming was seemingly all but assured. By 1994, gamers were told, affordable VR would be a reality.

Except, of course, the sword once again cut both ways. The Sega VR never came to market. Waldern, while selling several thousand units of its arcade cabinets, saw a flashy deal with Atari to sell a headset for its Jaguar system fall apart. Consumers started asking what VR did for them that they couldn’t get from a flat screen. The technology was flashy and seemingly futuristic, but the games were uninspired, relying on novelty instead of emphasizing what made VR distinct. VR remained a tempting promise all throughout the ’90s, with headsets like the VFX1 for the PC, and the iGlasses, but none really took off.

Still, the research continued, and VR kept finding new applications in industry as the technology improved. The medical industry began researching VR as a way to analyze patient body scans and train surgeons without putting patients under the knife. Lanier began researching the difficult problem of “telepresence,” using virtual reality tools to interact with distant environments. And the ever-rising demand for processors meant that smaller, more powerful chips were arriving every year, driving the cost of even complex applications like VR down further. In truth, a renaissance was arriving for VR, it was just one made more for professors and researchers than gamers and consumers.

More Than Fun And Games

VR grew steadily behind the scenes, finding success in surprising fields. Archaeologists rebuilt ancient heritage sites in virtual reality, letting students and tourists literally walk through history that either no longer existed or was difficult to reach. Urban planners began building entire cities in VR to test their theories. But the dream of a headset on every desk remained seemingly out of reach.

Oddly, what remains the greatest breakthrough in VR, at least in a commercial sense, came from Nintendo and the Wii. The Virtual Boy, Nintendo’s failed entry into the market, wasn’t a true VR headset, but Nintendo’s interest in the technology lingered. The controller was arguably the most technically complex thing about the Wii, tracking the user’s hand gestures and input in real time. Nintendo’s underpowered console was scoffed at by the gaming industry, only to watch in shock as the Wii easily conquered the PlayStation 3 and the Xbox 360.

And that, in turn, led to the current state of VR. In 2013, Oculus offered the Rift developer’s kit on Kickstarter. It was more a demonstration of a concept than anything else, an attempt to get VR headsets to the point where Lanier’s dream of a headset on every desk was an affordable reality. It was an enormous hit, and the company was quickly bought by Facebook as imitators arrived and established companies like HTC got into the game.

2016 in VR closely resembles 1991, with one important difference: The headsets are either already here, or on the way. The PlayStation VR headset is arriving October 13th. The Oculus Rift has been on sale for months, steadily moving units. Google has even created a cheap VR tool called Cardboard that turns any phone into a VR headset. And Waldern, who has spent his career in VR, thinks this might finally be the moment:

Even back at the start, the biggest kick we got was when we used ISDN lines between Berlin and London, and we linked players up in a game. It was amazing: you could meet, play, talk with people. Now, games have been doing that virtually for a while. But when you can feel you’re really meeting someone, right there, as a jolly alien or Robin Hood. Immersivity is the main thing. This has many years to run, and we have to get to where we are totally convinced – but we’re looking at something which has transformational capability for society.

Still, VR headsets are expensive and need powerful computers, putting them somewhat out of reach for consumers.

Yet the Sword of Damocles always cuts both ways. While, once again, VR’s future as a consumer product is still tenuous, its use elsewhere is expanding and helping people. USC’s Institute for Creative Technologies has developed a project called Bravemind, which uses exposure therapy on veterans suffering from PTSD. By fully immersing themselves in the traumatic memories that haunt them, USC is getting encouraging results helping people process their psychological trauma. UNICEF is using VR to give events a world away an immediacy and shock we wouldn’t otherwise experience.

And then, of course, there’s the future. VR has, to this point, shaped our world in subtle ways, but there are a host of breakthroughs lurking just beyond the horizon. Lanier’s research into telepresence systems means that, eventually, when a natural disaster strikes, instead of flying people across the world to help, they may be able to jump into virtual reality rigs and take command of robots with a suite of powerful tools to lift rubble, detect heartbeats, and administer first aid. VR can take entire colleges virtual, making getting an education as simple as wearing a headset for a few hours. They can also train people in complex tasks and skills at low cost and low risk. Doctors are using VR to expand their skills, as well.

The promise of VR and the revolution Lanier aimed for was to put not just the world at our fingertips, but to allow us passage into our own imaginations and to completely change how we approach being, physically, in a place. That promise remains, and it’s temptingly close.