• A History of Virtual and Augmented Reality
• Thoughts on Virtual Reality (philosophy)
• Timelines of VR and AR (including the arts)
• A Chinese translation

The story of virtual reality dates back to the 1960s. Already in 1961 Charles Comeau and James Bryan at Philco built a head-mounted display, the Headsight. Meanwhile, Bell Helicopter designed a head-mounted display for pilots that communicated with a moving camera. Ivan Sutherland at ARPA speculated about the "Ultimate Display" in 1965. In 1966 he moved to Harvard University where he took Bell Helicopter's head-mounted display and connected it to a computer: the images projected in the display were generated by a computer instead than by a camera. In 1969, after moving to University of Utah, he created a rudimentary virtual-reality system on a PDP-1 attached to a Bell Helicopter's display (with funding from the CIA, ARPA, the Office of Naval Research and Bell Laboratories).

Television was ahead of technology. In 1974 the TV series "Star Trek" imagined a device called the Holodeck that allowed people to enter virtual worlds, but the series was set in the 24th century.

Meanwhile, Myron Krueger at the University of Wisconsin was designing "responsive environments", computer systems that responded in real-time (with primited video and sound) to the user's actions, as described in his 1974 dissertation "Computer-Controlled Responsive Environments" (published in 1983 as the book "Artificial Reality"). They culminated in Videoplace, an interactive system first exhibited at the Milwaukee Art Museum in 1975 whose human-computer interface incorporated the user's full body. Krueger viewed Videoplace as the marriage of two cultures and technologies: television and the computer, television providing "entertainment" and the computer inviting user participation. Videoplace was a work of art, but a work of art that consisted in involving the user in the creation of the artwork.

Thomas Furness at an airforce base in Ohio started work in 1969 on a helmet for pilots that displayed three-dimensional computer graphics (the Visually Coupled Airborne Systems Simulator), first demonstrated in September 1981; and then he used it to design a virtual cockpit (the "Super Cockpit"), first announced in 1986, that allowed a pilot to fly a plane through a computer-simulated landscape by moving his head and his hand. Furness went on to establish in 1989 the University of Washington's Human Interface Technology Lab (HITL) in Seattle.

Meanwhile, in 1979 Eric Howlett in Boston invented an extreme wide-angle stereoscopic photographic technology, LEEP (Large Expanse Extra Perspective). In 1979 Michael Naimark at MIT's Center for Advanced Visual Studies debuted the Aspen Movie Map, a project directed by Andy Lippman that allowed the user to navigate a representation of a city (Aspen) stored on laserdiscs. The "movie map" had been created over two years by wide-angle cameras mounted on top of a car.

In 1984 UC Berkeley's alumnus Michael McGreevy joined NASA Ames Research Center and started the project for the Virtual Planetary Exploration Workstation, an immersive environment, the Virtual Visual Environment Display system (VIVED), first demonstrated in 1986, for which he built a head-mounted display. The system was hosted on a DEC PDP-11 interfacing an Evans & Sutherland Picture System 2. This can be considered the first virtual-reality environment. "Virtual reality" was basically an evolution of the old computer simulation systems, such as the ones pioneered by Evans & Sutherland. The software was interactive, meaning that it recreated the environment based on the user's movements, i.e. the user was able to interact with the computer via body movements.

In 1984 Steve Mann, a student at McMaster University, developed the see-through glasses EyeTap, predating augmented reality by a decade.

The protagonist in James Cameron's film "The Terminator" (1984), who wears a data-displaying headset, also predates the technology. That was the year when William Gibson's dystopian novel "Neuromancer" (1984) launched the cyberpunk genre. A few years later Robert Zemeckis' film "Who Framed Roger Rabbit?" (1988) overlayed digital characters on physical environments, generating even more interest for what will be called "augmented reality".

In 1985 Jaron Lanier, another self-taught videogame expert, opened VPL Research at his house in Palo Alto, the first company to sell products dubbed "virtual reality", originally the "Data Glove" invented by Thomas Zimmerman in 1982. VPL popularized the use of head-mounted displays to render computer-generated worlds: by moving the dataglove the user moved in the virtual world projected into her head-mounted display. One person who got intrigued by VPL's virtual reality was Timothy Leary, the prophet of LSD: he saw virtual reality as a way to experience alternative realities just like hallucinogenic drugs. In 1987 Lanier started using the term "virtual reality".

In 1985 Scott Fisher, an MIT alumnus who had moved to the Bay Area to join Alan Kay's research group at Atari (where he had met Zimmerman and Lanier), joined NASA Ames and by 1989 built the VIrtual Environment Workstation (VIEW), incorporating Zimmerman's dataglove.

The 1991 edition of SIGGRAPH inaugurated the "Tomorrow Realities" section with several virtual-reality exhibits, including pioneers like Myron Krueger and Michael Naimark.

In 1993 Pattie Maes demonstrated at SIGGRAPH the ALIVE (Artificial Life Interactive Video Environment) system, developed at MIT. It was an experiment combining artificial intelligence and artificial life: users were allowed wireless full-body interaction with a virtual world inhabited by autonomous agents. The user interacted via hand and body gestures with the agents in the virtual world. ALIVE offered a choice of three worlds, each inhabited by different agents: a puppet, a hamster and a predator, and a dog called Silas. The ALIVE system incorporated Bruce Blumberg's "Hamsterdam" software for creating intelligent autonomous agents capable of interacting with one another and with the user. Hamsterdam was based on animal behavior models proposed by ethologists.

The history of virtual reality also overlaps with the history of computer games. A MUD (Multi-User Dungeon) is a computer game played by many simultaneous users on different computers all of them connected to the same virtual world. There were predecessors but the game that created the term and started the trend on the Internet was MUD, created in 1978 in Britain by Essex University's student Roy Trubshaw and launched on the Internet in 1980. For a while Britain was the leader in this field with games such as "MIST" (1986) and "AberMUD" (1989), followed by Denmark with "DikuMUD" (1991); but their virtual worlds were text-based, not graphical.

In 1986 Lucasfilm launched "Habitat", a graphical MUD created by Randy Farmer and Chip Morningstar and running on Commodore 64 computers connected via dial-up lines. Habitat was a social virtual world in which each user was represented by an "avatar".

In 1992 the University of Illinois at Chicago demonstrated the CAVE, a virtual-reality environment whose walls and floor were basically just very large screens, and many universities would install a CAVE system in the following decade.

The world-wide web and the Netscape browser (introduced in 1994) allowed creators of virtual world to share them on a more standardized platform, and therefore to reach many more people.

In 1994 Ron Britvich in southern California created WebWorld, later renamed AlphaWorld, in which people could communicate, travel and build; and in Silicon Valley a former hippy named Bruce Damer formed the Contact Consortium that in 1996 launched 3D virtual-reality environments such as a virtual town and a virtual university.

Around that time a new kind of "massively multiplayer game" appeared, the MMORPG. The genre was inaugurated in 1996 by Korean game "Baramue Nara" ("Baram" in the USA), followed by "Meridian 59" (1996), developed by brothers Andrew and Chris Kirmse in Virginia, and "Ultima Online" in 1997, developed by Electronic Arts' game designer Richard Garriott who also coined the term MMORPG. The most famous MMORPG was "World of Warcraft", launched in 2004 by Blizzard Entertainment.

The success of MMORPGs not only triggered the development of software for massive multi-user environments but also increased the demand for powerful hardware. Both software and hardware developed for these videogames ended up enabling a new generation of virtual worlds. The history of virtual reality begins with military applications but then shifts into the subculture of videogames.

The problem for virtual reality was still the physical interface. Nonetheless there was a little VR craze during the 1990s. In 1990 Mark Bolas and others founded Fakespace in Silicon Valley, a spin-off of NASA's Ames Research Center, to build devices for virtual reality. 1990 also saw the first sales of the virtual-reality system developed by W Industries (founded by former IBM scientist Jonathan Walden in 1985 in Britain and later renamed Virtuality), targeting the research labs of big corporations. Virtual Research Systems, founded in 1991 in Sunnyvale by Bruce Bassett, began selling a cheap "Flight Helmet", based on the old design of NASA's VIVED. The world of videogames sensed an opportunity. In 1990 Jonathan Waldern demonstrated a VR arcade machine dubbed Virtuality at an exhibition of computer graphics in London. Sega demonstrated (but never released) the Sega VR in 1993, and Nintendo's Virtual Boy console, the first portable console to display 3D graphics, was introduced in 1995. Future Vision Technologies, a spinoff of the University of Illinois at Urbana-Champaign, developed a head-mounted display for the consumer market, the Stuntmaster, compatible with both the Super Nintendo and the Sega Genesis (it was released in August 1993 by VictorMaxx, followed by an improved model, the CyberMaxx). Virtual I/O in Seattle (i.e. Greg Amadon and his wife Linden Rhoads) released an even cheaper display for home VR, the iGlasses, in 1995. In England, Rupert Neve's Forte (more famous for recording studio consoles) released the VFX1 for the IBM PC in 1994. All head-mounted displays shared the same problems: a true stereo display made of two high-resolution color LCD screens coupled with motion tracking was too expensive and caused serious motion sickness.

Brett Leonard directed the film "The Lawnmower Man" (1992), a sort of Lanier biopic that introduced the virtual reality to the broader movie-going audience, and Robert Longo directed "Johnny Mnemonic" (1995), whose protagonist surfs the web with goggles and glove (incidentally, i am writing this sentence in 2021, the year in which the story of the film takes place).

Virtual reality remained in the realm of "experimental" computer science, generating mostly intellectual interest. Brenda Laurel, co-founder with Scott Fisher of Telepresence Research, published the book "Computers as Theatre" (1991), and in 1993 the IEEE organized the first academic conference on virtual reality: the Virtual Reality Annual International Symposium (VRAIS), held in Seattle.

One wonders how much the art world influenced the evolution of virtual and augmented reality. Artists were among the early and more creative adopters of these new technologies. One can start with Lynn Hershman's navigable dataverse "Deep Contact" (1989) and Jeffrey Shaw's "Virtual Museum" (1991). Nicole Stenger's "Angels" (1992) was the first immersive movie. Then came mesmerizing interactive installations that redefined the relationship between artist and audience: Monika Fleischmann & Wolfgang Strauss's "Home of the Brain" (1992), Brenda Laurel and Rachel Strickland's "Placeholder" (1992), Lawrence Paul Yuxweluptun's "Inherent Rights, Vision Rights" (1992), Kazuhiko Hachiya's "Inter Dis-communication Machine" (1993), Peter d'Agostino's "VR/RV - A Recreational Vehicle in Virtual Reality" (1994), Diane Gromala & Yacov Sharir's "Dancing with the Virtual Dervish - Virtual Bodies" (1994), dancing in VR, Michael Naimark's "Be Here Now" (1995), Masaki Fujihata's "Global Interior Project" (1995), Char Davies's "Ephemere" (1995), Agnes Hegedus's Virtual world "Memory Theater" (1997), Maurice Benayoun's "World Skin" (1997), Rebecca Allen's "Emergence - Bush Soul" (1998), Charlotte Davies's "Osmose" (1998), Simon Penny's "Traces" (1999), John Klima's "Go" (2000), etc.

The blockbuster movie "The Matrix" (1999), which was the Hollywood remake of Fassbinder's film "World on a Wire", popularized the idea of life in a simulated world and inspired a new generation to live inside virtual worlds.

Meanwhile, in 1990 Tom Caudell at Boeing in Seattle had coined the term "augmented reality" to refer to a system capable of overlaying physical and virtual reality. Augmented Reality turns the physical environment (whether a landscape or a building) into a blank canvas: the user armed with an AR device can place digital objects among the physical objects of the physical environment and other users wearing an AR display can find these digital objects. This was achieved via see-through glasses that allowed wearers to move and manipulate 3D content using hand gestures. In 1992 Louis Rosenberg, working at an airforce base in Texas, developed the first functioning augmented reality system, "Virtual Fixtures". In 1993 Paul Milgram's group at the University of Toronto built ARGOS (Augmented Reality through Graphic Overlays on Stereovideo). There were also experiments in augmented-reality conferencing, systems that allowed users to see each other as well as virtual objects, such as Dieter Schmalstieg's Studierstube at Vienna University of Technology (1996) and Jun Rekimoto's Transvision at Sony in Japan (1996). In 1998 Francisco Delgado's team at NASA in Texas, using the "synthetic vision software" LandForm developed in 1995 by Mike Abernathy at Rockwell, tested a "virtual cockpit window" to provide a simulated real-time 3D view to the crew of the X-38, a window-less spacecraft designed as a return vehicle for the International Space Station. In 1999 Hirokazu Kato (from Japan) and Mark Billinghurst (from New Zealand) at the University of Washington in Seattle released an open-source software called ARToolKit that was instrumental in spurring AR development. In 2000 Bruce Thomas at the University of South Australia developed the first augmented-reality videogame, AR Quake, a variation on the popular Quake game.

Augmented reality too was percolating into the cultural world, for example in Julie Martin's augmented reality theatrical play "Dancing In Cyberspace" (1994), in which acrobats danced inside and around virtual objects, in William Gibson's novel "Virtual Light" (1994) and in Steven Spielberg's film "Minority Report" (2002), whose characters use data gloves to display information.

In 2007 the English company Image Metrics created a 3D hologram (a visual clone) of a celebrated but deceased actor (Richard Burton) performing "live" on a stage. In 2008 the US public saw augmented reality in television when a 3D virtual image of reporter Jessica Yellin materialized in the CNN studio. ARToolkit was ported to Adobe Flash in 2009 by Tomohiko Koyama (aka Saqoosha), a fact that turned any web browser into an augmented reality platform. At the end of 2009 Esquire magazine released a special augmented-reality edition, replete with software that allowed its readers to view 3D animated content. In 2011 Ambarish Mitra's London-based Blippar introduced a smartphone app for "visual discovery" that combined image recognition and augmented reality: point your smartphone's camera at an object and the phone displayed something about that object. In 2012 a firm called Paddy Power used Blippar for an advert in which anyone with the app could make Queen Elizabeth come alive on 10-pound notes. In 2015 the Evening Standard came up with a blippable Queen selfie. In 2016 the Guardian newspaper engineered a virtual-reality experience named "6x9" which placed the user inside a solitary-confinement prison cell.

Meanwhile, two Silicon Valley companies had introduced see-through glasses: Google in 2013 (the Google Glass) and Meron Gribetz's Meta in 2014.

In 2014 Google unveiled Project Tango for Android, its effort to bring augmented reality to smartphones, first used commercially in the Lenovo Phab 2 Pro smartphone (2014). It was superseded in 2018 by ARCore, also for Android phones. In 2017 Apple responded with ARKit for the iPhone. Tango and ARKit use the smartphone's camera to understand the phone's position and orientation relative to its surroundings. By the release of ARKit2 in 2018, the software was capable of building a fictional world inside the physical world, and ARKit3 (2019) added features for motion capture, people occlusion and face tracking.

In 2015 Microsoft introduced its head-mounted display Hololens that displayed realistic holograms within the real world. It was untethered (a self-contained computer). In 2016 Microsoft placed a digital sculpture in a Seattle museum that only people wearing Hololens could see. Hololens represented an evolution of the "desktop" metaphor in vogue since the 1980s: the "desktop" computer enabled the user to wander around the "dataverse" (personal and remote data) using keyboard and mouse, while Hololens, replacing the mouse with the user's gaze and the mouse click with the motion of the user's fingers, enabled the user to wander around 3D virtual objects and to manipulate them using gaze and gestures.

AR became truly popular in 2016 when Niantic's videogame PokemonGo placed creatures in the urban landscape that only people with the app could see (and capture).

There were basically three degrees of augmented reality: simple Location tracking in Pokemon Go; two-dimensional AR on smartphones (e.g scanning the back of a cereal box with Blippar), a passive experience; and three-dimensional AR on headsets like Hololens (holograms that can be manipulated and can be explored), a fully immersive/interactive room-scale experience.

Augmented reality has the potential to create a visual encyclopedia for every single place and object in the world, a geographically distributed web of webs. One looks at a place and one can see the history, science, art, etc related to it, and everybody can add more information about it. You look at a gadget and you can browse the operating manual, check the warranty, view a video about the technology behind it, see a list of famous movie scenes where that gadget was used, read the biography of the person who invented that kind of gadget, and so on: an unlimited amount of trivia. Wherever you look, you open a universe of information. Any corner of the world could become a multidimensional universe of universes.

Back to virtual reality, Microsoft introduced its Kinect device in 2010, making 3D-motion capture a lot easier and cheaper. To be fair, Nintendo's game console Wii of 2006 already came with motion tracking and had spawned a new generation of games.

A cultural phenomenon was Ernest Cline's novel "Ready Player One" (2011).

A new generation of head-mounted display was inaugurated by the Oculus Rift in 2012, developed by a Los Angeles teenager, Palmer Luckey, with help from software wizard John Carmack, the brainchild behind id Software's videogames "Doom" (1993) and "Quake" (1996). Oculus' acquisition by Facebook in 2014 for $2 billion inaugurated a new VR craze.

In 2013 Nvidia demonstrated a display based on "near-eye light-field" technology which created a much more realistic impression when exploring a virtual world, and David Holz's Leap Motion in San Francisco introduced a hand-tracking device that (connected to a computer) enabled users to manipulate digital objects with hand motions.

Then came Google's low-cost Cardboard in 2014 (which used an Android smartphone as the virtual-reality device) and in 2015 the HTC headset Vive (tethered to a computer like Oculus but with a larger range of movement thanks to the spatial tracking technology "Lighthouse" developed by Seattle-based Valve, i.e. Gabe Newell and Mike Harrington) and the Samsung Gear VR (based on Oculus technology but also integrated with the smartphone, in this case a Samsung Galaxy).

Intel was briefly in the game thanks to its 3D camera RealSense ZR300 (2016) and the wireless VR headset Project Alloy (2016).

VR-ready game consoles and headsets appeared: in 2016 Sony introduced its PlayStation VR and in 2017 Microsoft launched its Xbox One X. 2018 was the year of the stand-alone headset: Oculus Go (replaced the following year by the more successful Quest), Google's and Lenovo's Mirage Solo, and HTC Vive Focus.

Rony Abovitz's Magic Leap (Silicon Valley) introduced in 2018 the Magic Leap One, a Hololens competitor, capable of displaying holograms in physical spaces (but it became famous mainly as one of the biggest flops in the history of Silicon Valley).

In 2018 it helped that Steven Spielberg came out with a film adaptation of "Ready Player One".

VR games were becoming more sophisticated and interesting, for example: "Fantastic Contraption" (2016), developed in Canada by Andy Moore of Radial Games and the husband-and-wife team of Colin and Sarah Northway; Alex Schwartz's "Job Simulator" (2016); "Accounting+" (2017), developed by television producer Justin Roiland and game designer William Pugh; Dirk Van Welden's "Space Pirate Trainer" (2017); "Virtual Virtual Reality" (2017), a VR game about VR and AI developed by Tender Claws, a Los Angeles-based VR studio founded in 2014 Danny Cannizzaro and Samantha Gorman; "Beat Saber" (2018) by Czech game developer Beat Games (Vladimir Hrincar and Jan Ilavsky); etc.

Non-gaming applications of virtual reality began to expand. Besides education and art exhibition, there was renewed interest in immersive journalism thanks to Chris Milk's "Clouds over Sidra" (2015), a 360-degree video set in a camp for Syrian refugees, and to Nonny de la Pena's VR documentaries "Hunger in Los Angeles" (2012) and "Project Syria" (2014). The "reader" becomes an "observer", and therefore is much more likely to "feel" the story instead of simply "hearing" it. In a 2015 talk Chris Milk touted virtual reality as the "ultimate empathy machine".

Cinema too adopted the new storytelling format of virtual reality. The Oculus Story Studio was founded in 2014 by film director Saschka Unseld (a Pixar veteran) and producer Edward Saatchi. They realized three main VR "films": "Lost" (2015), the 12-minute "Henry" (2015) and especially "Dear Angelica" (2017). Penrose Studios (founded in San Francisco by Oculus alumnus Eugene Chung) produced the VR animated shorts "The Rose And I" (2015), only five-minute long, and "Arden's Wake" (2017), winner of a special prize at the Venice Film Festival, originally 16-minute long but then expanded to 30 minutes. Award-winning VR films included: the nine-minute "Battlescar" (2018), set in 1970s New York punk scene, made by the duo of Los Angeles-based Nico Casavecchia and Barcelona-based Martin Allais, Darren Aronofsky's three-part 45-minute educational documentary "Spheres" (2018), and Brazilian filmmaker Ricardo Laganaro's 12-minute short "The Line" (2019). San Francisco-based Fable Studio (Oculus alumni Edward Saatchi and Pete Billington) made the 20-minute film "Wolves in the Walls" (2019), based on the 2003 graphic novel by Neil Gaiman and Dave McKean, and then (in 2020) even launched the protagonist, Lucy, as a conversational A.I. virtual being on social media.

The Sundance Film Festival already opened a "New Frontier" exhibition in 2007; the Tribeca Film Festival added in 2015 the "Storyscapes" exhibition; and in 2017 "La Biennale di Venezia" launched a section for virtual-reality works.

The Chinese startup Nreal introduced in 2020 the smartphone-powered augemented-reality glasses Light, which, unlike Hololens and Magic Leap One, targeted the consumer market. In 2021 Facebook responded with Stories and Snapchat with Spectacles-4. But several announcements and demonstrations signaled that 2021 was the year of augmented reality: Swiss startup CREAL demonstrated a light-field augmented-reality headset, French startup Lynx demonstrated the standalone headset R1 for both VR and AR, Finnish tartup Varjo set new record of resolution with its XR-3, San Francisco startup Kura Gallium demonstrated AR glasses with a 150-degree diagonal field of view, Google absorbed Canadian startup North, etc.

As the devices got cheaper, VR/AR art installations proliferated. For example: Rachel Rossin: "I Came and Went as a Ghost Hand" (2015), Jennifer Steinkamp: "You" (2016), Tamiko Thiel: Gardens of the Anthropocene (2016), Laurie Anderson and Hsin-Chien Huang: Chalkroom (2017), Alejandro Inarritu: Carne y Arena (2017),, Kathryn Bigelow: "Protectors: Walk in the Ranger's Shoes" (2017), Hayoun Kwon: "The Bird Lady" (2017), Mel Chin: "Wake and Unmoored" (2018), etc.