20

All In Your Mind

Realspace is just so 20^th Century.

Someone, somewhere, sometime in the first decade of the twenty-first century

Some years ago I gave a lecture in Italy over the telephone from my home office in San Jose. From my end it felt too much like talking into a void. A year or two later I repeated the experiment with better technology. This time I was sitting in a videoconferencing room. My audience in the Netherlands could see me and I could see them. Still not quite real, but a good deal closer.

The next time might be closer still. Not only do I save on the airfare, the audience does too. I am at home; so are they. Each of us is wearing earphones and goggles, facing a small video camera. The lenses of the goggles are video screens; what I see is not what is in front of me but what they draw. What they are drawing is a room filled with people. Each is seeing the same room from the other direction – watching me, standing at a virtual podium as I deliver my talk.

Virtual reality not only saves on airfares, it has other advantages as well. The image from my video camera is processed by my computer before being sent on to everyone in my audience. That gives me an opportunity to improve it a little first, to replace my bathrobe with a suit and tie, give me a badly needed shave, remove a decade or so of aging. My audience, too, looks surprisingly attractive, tidy, and well dressed. And while, from my point of view, they are evenly distributed about the hall, each of them is watching me from the best seat in the house.

Long ago I was given the secret of public lectures: Always speak in a room a little too small for the audience. In virtual reality, the scaling is automatic; however many people show up, that is the number of seats in the lecture hall. And for each of them, the lecture hall is custom designed, gold plated if his taste is sufficiently lavish. In virtual reality, gold is as cheap as anything else. If you do not believe me, take a look at one of the gaudier bits of a good video game – the Durance of Hate in Diablo II, say.

Video games are our most familiar form of virtual reality. Staring through the screen you are looking at a world that exists only in the computer’s memory, represented by a pattern of colored dots on the screen. In that world, multiple people can and do interact, each at his or her own computer. In first-person video games, each sees on the screen what he would be seeing if he were the character he is playing in the game. In some, the virtual world comes complete with realistic laws of physics. Myth, so far as I could tell, calculated the flight of every arrow; if a dwarf threw a hand grenade uphill, it rolled back. As the technology gets better, we can expect it to move beyond entertainment. Perhaps I should stay out of airline stocks for a while.

We already know how to do everything I have described. As computers get faster and computer screens – including goggle-sized ones – sharper, we will be able to do it better and cheaper. Within a decade, probably less, we should be able to do sight and sound virtual reality inexpensively at real-world resolution, with video good enough to make the illusion convincing. The audio already is.

However good our screens, this sort of virtual reality suffers from a serious limitation: It only fools two senses. With a little more work we might add a third, but smell does not play a large role in our perceptions. Touch and taste and the kinesthetic senses that tell us what our body is doing are a much harder problem. If my computer screen is good enough the villain may look entirely real, but if I try to punch him I will be in for an unpleasant surprise.¹

Our present technology for creating a virtual reality depends on the brute-force approach, using the sensorium, the collection of tools with which we sense the world around us. Want to hear things? Vibrate air in the ear. Want to see things? Beam photons at the retina. Applying that approach to the remaining senses is harder. And even if we can do it, we are still left with the problem of coordinating what our body is actually doing in realspace with what we are seeing, hearing, and feeling it do in virtual space.

TODAY AND TOMORROW: THE WORLD OF PRIMITIVE VR

If you play video games, virtual reality – the brute-force version – is already part of your life. As it gets better and cheaper, one result will be better games.

Communication is another obvious application. You still will not be able to reach out and touch someone, save metaphorically. But seeing and hearing is better than just hearing. A conference call becomes more like a meeting when you can see who is saying what to whom and read the cues embodied in facial expressions and body movements.

That raises an interesting problem. We all, automatically and routinely, judge the people around us not only by what they say but by how they say it – tone of voice, facial expression, gestures. Most people are poor liars; that is one of the reasons why honesty is the best policy. Having people believe you are honest while taking whatever actions best serve your purposes would be an even better policy – for you, not for those you deal with – but for most of us it is not a practical option.

We call the exceptions con men. They are people who, through talent or training, have mastered the ability to divorce what they are actually thinking and doing from the system of nonverbal signals, the monologue about the inside of our heads, that all of us are continually delivering. Fortunately, not many are really good at it.

My computer can make me look younger. It might also be able to make me look more honest. Once someone has done an adequate job of deciphering the language by which we communicate thoughts and emotions by facial expressions and body postures – for all I know someone already has, possibly someone in the business of training salesmen – we can create computerized con men. I have no talent for lying. My computer, on the other hand ….

The flip side of the problem of virtual con men is that on the internet nobody knows you are a dog. Or a woman. Or a twelve year old. Or crippled. In virtual reality, once we have the real-time editing software worked out properly, you can be anything you can imagine. Homely women can leave their faces behind, precocious children can be judged by the mental age reflected in what they say and do, not the physical age reflected in their faces. When my fourteen-year-old son logs onto World of Warcraft, he ages by five or six years. It’s good practice.

When you interact on Usenet or in an email group you are projecting a persona, giving the other members of the group a mental picture of what sort of person you are. Some years ago, someone suggested a game for the Newsgroup rec.org.sca: Have participants write and post physical descriptions of other participants they had never met. I gained almost nine inches. In virtual reality I never have to be short again.

Unless, of course, I want to be.

My Contribution to Corpore Sano

In the modern world, we no longer have to worry much about escaping predators or running down prey. We no longer have to scratch in the ground with sharp sticks to grow food. For most of us, “work” involves little physical exertion. But there is still play – basketball, soccer, tennis. One objection to video games is that they remove one of the few incentives modern people have to exercise.

Observe someone, perhaps yourself, playing an absorbing video game. Just as with other games, involvement in winning dominates all other concerns. Long ago I discovered the sign of a first-rate computer game – that when I finally left the computer to use the bathroom, it was because I really had to. And lots of players of lots of games have noticed just how tired their thumb is only after the game is over.

If what you want is exercise, the obvious solution is bigger joysticks. Combine a video game with an exercise machine. Working the exercise machine controls what is happening in the game. Just as with real-world athletics, you only notice how tired you are after you have won or lost. Primitive implementations, most notably Dance Dance Revolution, already exist.²

In my improved version, virtual games become better exercise than real games because the environment that the computer creates is being tailored, second by second, to your body’s needs. The setting is the Pacific during World War II. You are controlling an anti-aircraft gun on the Yamato, the world’s biggest battleship, desperately trying to defend it against waves of American bombers attempting by sheer brute force to destroy the glory of the Japanese navy. You traverse the gun left and right with your arms, lower or elevate the barrel with foot controls; when you release the controls it swings back to center. Your strength is physically moving the gun, so it isn’t surprising that it’s a lot of work.

After the third wave, the computer controlling the game notices that you are having trouble swinging the gun rapidly to the left – your left arm is tiring. The next attack comes from the right. As the right arm becomes equally tired, more and more of the attacks require you to adjust the elevation of the gun, shifting the work to your legs. When your heartbeat reaches the upper boundary of your aerobic target zone, there is a break in the attack, during which you hear martial music. As your heartbeat slows, the next wave comes in. Tennis may be both fun and exercise but art, well-done art, improves on nature.

A sophisticated exercise game is one way in which we can use virtual reality. Another is to do dangerous things while only getting virtually killed. Consider the problem of engineering in dangerous environments – the bottom of the Mariana trench, say, five miles beneath the waves, or the surface of the moon. One solution is for the operator of the equipment to be only virtually there. His body is sitting in a safe environment – wearing goggles, manipulating controls. His point of view, just as in a first-person video game, is the point of view of the machine he is operating.

In the lunar case, we have a small technical problem: the speed of light. If the operator is on earth and the machine is on the moon there will be a noticeable lag between when the machine sends him information and when his response, based on that information, gets back to the machine. Some of us have been virtually killed by similar lags in video games. In the case of lunar engineering, while the death would be only virtual for the operator it might be real for the machine, and putting hardware on the moon is not cheap. Perhaps we had better have the operator on the moon too, or in orbit around it – somewhere safer than the tunnel he is digging, closer than the earth he came from.

As these examples suggest, virtual reality, even implemented using the crude technologies we now have, can have important uses in the real world.

DEEP VR – BEYOND THE DREAMING PROBLEM

One elegant solution to the limits of those technologies is the form of virtual reality that most of us experience every night. In a dream, when you tell your arm to move, your virtual arm moves. Your real arm (usually) doesn’t. Dreams are not limited to sight and sound. Suppose we succeed in cracking the dreaming problem, figuring out enough about how the brain works so that we too can create full sense illusions. We then have deep VR. Anyone who wants it has a socket at the back of his neck.³ Signals through the cable plugged into that socket can create a full sense illusion of anything our senses could have experienced.⁴

In thinking about the world that technology makes possible, a useful first step is to distinguish between information transactions and material transactions. Reading this book is an information transaction. The book is a physical object. But reading an illusion of a book, with the same words on the virtual pages, would do just as well. When you hold a conversation, you are using physical vocal cords to vibrate physical air in order to transmit your communications and using a physical eardrum to pick up those vibrations in order to receive the other person’s communications. But that apparatus is merely the machinery for transmitting information. Electronic signals that created the illusion of your voice saying the same words would achieve the same effect.

For a material transaction, consider growing wheat. You could grow virtual wheat, have the sensory experience of planting, weeding, harvesting. But if you tried to live on the virtual wheat you grew you would eventually die of real starvation. People in World of Warcraft cook virtual food, craft virtual weapons and armor. But it isn’t of very much use outside the game.

A sufficiently advanced form of virtual reality can provide for all information transactions. It might assist with some material transactions; the wheat harvester could be run by an operator located somewhere else, giving real instructions to a real machine. The operator’s physical presence would be an illusion, the information he was using real, provided by cameras and microphones on the harvester. But if you want real transactions to produce real results, food, houses, or whatever, someone or something has to really do them.

Beam Me Up, Scotty

In Star Trek, people get beamed from one place to another. I know of no reason to expect it to happen and if it did I would be reluctant to use it, since it is not clear whether using it is transportation or suicide – I die and the machine creates a copy that thinks it is me. But as long as all we are moving is information, virtual reality can do just as well while avoiding the philosophical problems.

Why do I want to visit my friends? To see them, to feel them, to hear them, to do things with them. Unless one of the things is building a house or planting a garden that really has to be built or planted, the whole visit is an information transaction. With good VR, my body stays home and my mind does the walking. If you find this an odd idea, consider a phone call. It too is a substitute for a visit. VR simply increases the bandwidth to cover all our senses, providing a way for any group of two or more people to get together for any information transaction they wish to engage in – a meeting, a peace conference, a trial, a love affair.⁵

And since all that is happening is information moving back and forth over networks, information that can readily be encrypted, we are back in a world of strong privacy. Surveillance technology may make everything in realspace public, but we are no longer doing very much in realspace.

Future Fiction

The potential for the entertainment industry is equally striking. Works of fiction can be experienced in full sight, sound, and touch, no imagination required. Whether that is an improvement is not entirely clear; my daughter has so far refused to see the movie version of The Fellowship of the Ring because she prefers the product of her imagination to the product of the director’s imagination. Role-playing games will become a great deal more vivid when you get to not only see and hear the monsters but feel, smell, and touch them as well. Just how vividly you get to feel the monster’s claws tearing you to bits will be one of the options in the preferences panel; I may go for the low end of that one.

One form of virtual entertainment will be a work of fiction. Another may be a tape recording. You too can climb Everest, plumb the depths of the sea. If there are real wars going on, a few of the soldiers may moonlight in cinema verité, everything that happens to them recorded. Pornography will finally become serious competition for sex.

In each of these cases, we are creating as an illusion the sort of experiences that already exist in reality. Consider in contrast a symphony. It corresponds to nothing in nature. The composer has taken one sense, hearing, and used it to create an aesthetic experience out of his own head. It will be interesting to see what an artist can do with all the senses.

When you are experiencing VR fiction, one question is how real you want it to seem. While the story is happening, do you know it is a story? Is there a little red light glowing at the edge of your peripheral vision to tell you that none of it is real? Perhaps the experience would be more moving, more profound, better art, if you thought it was real. Just like a dream.

Fantasy: Substitute or Complement

Virtual reality, even in its current form, lets you do lots of things that most of us would rather people not do in the real world. Players in World of Warcraft, for instance, spend lots of their time beating people up and taking their stuff. If the game had existed twenty years ago, I expect the people being beat up would have all been male, violence between men being more socially acceptable than violence between a man and a woman. Under current norms, however, gender discrimination is Not Done; a male player will find himself spending a good deal of his time beating up (virtual) females.

That raises an obvious question: Having routinely punched out virtual females online, will he be more willing to punch out real females in the real world? It is an old question, because virtual reality is an old technology, even if one we have recently gotten much better at. George Orwell, writing more than sixty years ago, worried about the corrupting effect on readers of the routine brutality of American crime fiction;⁶ books too are a form of virtual reality, although one that leaves a lot of the work to the reader’s imagination. Later writers worried about the effect of television. The latest concern is the effect of internet porn.

From the standpoint of an economist, the question is a simple empirical one: Is virtual sex and violence a complement to or a substitute for real sex and violence? If you watch a violent movie does that make you more willing to commit violent acts yourself, or does identifying with James Bond onscreen satisfy your desires for violence and excitement without the inconvenient consequences of acting that way yourself? Does online pornography describing rape, bondage, sexual torture, and other ugly things make the reader more likely to engage in such acts, or less?

For quite a long time, evidence on the question – in part motivated by the search for arguments in favor of censoring pornography depicting violence against women – consisted of highly artificial experiments in which people were shown porn films and then questioned to see if their attitudes to sexual violence had been affected. We now, however, have some actual real-world data, thanks to an ingenious piece of work by Tod Kendall.⁷ It occurred to him that the internet greatly increased the availability of pornography and so provided a natural experiment on its effects: Correlate growth of access to the internet, by state, with changes in the frequency of rape. It turned out that the correlation was negative; increases in the availability of the internet, hence of internet porn, were associated with decreases in rape rates. No similar pattern existed for murder. That provides some evidence that virtual violent sex is a substitute for real violent sex.

We don’t know, of course, if the same will be true for the greatly improved virtual sex and violence that would be possible with deep VR, but the study provides at least limited grounds for optimism.

What Matters

I have been sketching a picture of what deep VR could do. If you are not yet getting worried, consider the full-scale version.

Stuff must be produced for real, but human beings do not need much stuff to stay alive. To check that for food, price the cheapest bulk flour, oil, lentils you can find. Calculate how much 2,000 calories a day of each of them would cost. You now have a rough estimate of the lowest cost diet high in carbohydrate, fat, or protein, as you prefer. To be safe, throw in a big jar of vitamins. It may not taste very good, but taste no longer matters. Eating is a material transaction, tasting an informational transaction. Tape-record 10,000 meals from the world’s best restaurants and your lentils are filet mignon, sushi, ice cream sundaes. Much the same holds for other material requirements. My body occupies only five or ten cubic feet of space. With a mind free to rove the virtual universe, who needs a living room – or even a double bed?

Viewed in realspace, it is not much of a world. Everyone is eating the cheapest food that will keep a human body in good condition, living in the human equivalent of coin-operated airport storage, exercising by moving against resistance machines, perhaps as part of virtual reality games, perhaps under automatic control while his mind is elsewhere.

To the people living in it, it is paradise. All women are beautiful, and enough are willing. All men are handsome. Everyone lives in a mansion that he can redecorate at will, gold plated if he so desires.⁸ Anyone, anywhere, any experience in storage, any life that can be created as an illusion, is an instant away. Eat all you want of whatever you like and never put on a pound.

Which is true – slum or paradise? It depends on what matters. If all that matters is sensation, what you perceive, it is a paradise, even if that is not obvious to a superficial inspection.

As evidence against, consider a very old form of virtual sex: masturbation. In your mind you can be making love to the woman of your dreams, at least if you have a good enough imagination. The orgasm, the physical sensations inside your body, the nervous signals reaching your brain, are real. Yet, even with the improved technology of pornographic books and videos, there is still something missing.

If what matters is what we experience, the world I have described is a paradise. If what matters is what really happens, the situation is more complicated. Having someone read a book I wrote, enjoy and be persuaded by my ideas, pleases me just as much if he reads it in virtual reality – as those who read this book on the web currently do. But what about only thinking someone read my book? What if I wake up from a long lifetime as a successful author, basketball star, opera singer, or Casanova to discover it was all a dream? Is that just as good as the real thing? Is it all right as long as I die before I wake up?

Robert Nozick, in Anarchy, State, and Utopia, put the question in terms of an imaginary experience machine, his version of VR.⁹ Plug someone in and he will have experiences just as vivid and just as convincing as in real life – a lifetime of them. Suppose the owner of the experience machine somehow knows the life you are going to live and can offer you a slightly improved version. Plug into his machine for an imaginary life in which your babies cry a little less, your salary is a little higher, your career in a firm with a little more status. If you believe him, do you take the deal? Do you trade a real life for a fictional life? Is what matters rearing children, making a career, planting fruit trees, writing books – or is what matters the feelings you would have as a result of doing all of those things?

You will have to decide for yourself. I wouldn’t touch the thing with a ten-foot pole.

Postscript: And Now We Have a Winner

An early draft of this book, including this chapter, was written and webbed way back in 2002. At that time I had no idea which of the futures I was discussing would arrive first; if I had had to guess it would probably have been the future of strong privacy enabled by public key encryption.

Now we know. The world of virtual reality is here, enabled not by the current primitive technology of motion sensors and video goggles nor the future technology of mind/machine interfaces but by the very old, and very advanced, technology of the human imagination. It turns out that a flat screen and a speaker is sufficient – provided the environment to which they are the gateway is a sufficiently interesting and attractive one.

As of early 2007, World of Warcraft had more than ten million subscribers, including all five members of my immediate family. Counting all massively multiplayer games, the total number of players worldwide is estimated to be at least 100 million.¹⁰ “Chinese gold farmers,” people in poor countries who make their living by playing on such games, earning virtual money and virtual goods and trading them online for real money, are estimated to number in the hundreds of thousands.¹¹

World of Warcraft is a fantasy, a work of fiction, a constructed story in which the player immerses himself as a character interacting with other characters, both other players and nonplayer characters run by the computer. Second Life is a different, and in some ways more interesting, variant on the idea. What it offers is not a story but a world, a framework, within which participants can themselves design and construct things, trade with each other, interact in a wide variety of ways. From one point of view it is an open source version of the more traditional sort of game, constructed by the participants. From another it is a first step toward a plausible substitute, for most people most of the time, for the real world.¹² And, since VR has room for an unlimited number of worlds, it is a substitute that could make real Robert Nozick’s idea of utopia¹³ – a wide variety of different communities run under different rules, with each individual free to choose the one he or she prefers.

While massively multiplayer online games are the most striking example of current virtual reality, there is another that is a little older and much bigger. It used to be that when you saw someone walking down the street talking to an invisible person, you suspected he was nuts. Currently, you assume he is on his cell phone. When you are holding a conversation with someone who is objectively many miles away but subjectively right next to you, you are participating in a (very low bandwidth) version of virtual reality.

“Cyberspace is where a telephone call happens.”