Jaron Lanier, the philosopher, classical music composer, and inventor considered the Father of Virtual Reality, rhapsodizes with physicist and saxophone player Stephon Alexander about the crazy, far-flung theories that could ruin their careers.
JARON LANIER: Over the years, we’ve had a lot of crazy ideas that we kept secret. The reason we didn’t share these crazier ideas, if I remember correctly, is because your mother was concerned it would ruin your career and you’d never get tenure.
STEPHON ALEXANDER: Yes, she was certain it would.
JL: When you ended up at an Ivy League university, I wondered if you were going to publish the weird stuff. I don’t know if it is ever safe enough to pub- lish the strangest stuff.
SA: I don’t either. So over the years, in our friendship and collaborations, we’ve been talking about the basic problem of cosmology. We once toyed with an idea—as crazy as it was—that the entire universe was a virtual reality simulation.
JL: This idea that we’re already in virtual reality has become a very popular question. Virtual reality is the science of experience. All data, all input, and all scientific reasoning must all arrive through experience. We can holistically study human experience through virtual reality. There is no way that virtual reality isn’t going to become fundamental to science. There are a lot of opportunities for people to do stupid things with virtual reality and ruin science by living in fakery instead of in reality. It challenges us as well as giving us some options. It’s not going to be easy.
SA: As a quantum cosmologist, I want to address the hard question in quantum mechanics and in quantum cosmology: How do you reconcile an observer, ideally us, with the happenings of the universe? We know that we have no reason to expect that there were typical observers 14 billion years before structure was even formed, but somehow structure did form. Because quantum mechanics is mathematically incompatible, just as a mathematical structure, you need to reconcile it with the observer.
JL: The observer is essential to the universe, even if the observer comes late in the universe. In my view, computer science also has an observer problem. If you find a really weird alien that’s not human—from one of the many planets we now know are out there—and you give them a present day laptop, what do they see? Since they don’t have any cultural context to interpret the object, it would be equivalent to a lava lamp, something that’s making patterns. It emits heat. The patterns grow deterministically. It won’t mean anything. The point is that without a cultural context, ultimately an idea has to bottom out in somebody’s experiences to mean anything. Information is an alienated experience.
SA: In quantum cosmology, you have this idea that the entire universe is described as a quantum moving function. The entire universe is a very complex waveform that reapplies itself, but there is no intrinsic time for this wave function. It’s timeless. Is it okay that I’m late for my meetings and stuff since time doesn’t really exist?
JL: Well, as a saxophone player you get some dispensation to be late to a certain degree but if you want to be really late you have to take up the drums. You know, time can be treated as the foundation from which everything hap- pens. Make time equal to nature. Make time the starting point for everything. I’ve thought a lot about whether aliens would have the same math we do.
SA: That’s an ongoing discussion.
JL: If any two alien species could figure out a way to have a meeting of minds, they could be similar enough to actually talk to each other and recognize that each other exists. However, you might never have encountered the same ideas. It seems plausible to me that you could have non-overlapping worlds, that each have integrity and are hypothetically compatible with each other except that they have never met. When you start thinking about math on those terms there is an interesting thing that happens. Growing up, I thought that math was beautifully regular and perfect, and that reality was messy. I used math to approximate reality, but reality was always messy compared to the beauty of mathematical equations.
SA: That’s interesting, because it reminds me of something that I’m struggling with as we speak. There are certain things about nature that can never be described with mathematics. Where does that leave us as scientists and as people who use this specific language at the end of the day to make the next, better cell phone?
JL: This brings up something that’s been bugging me lately in the virtual real- ity world. You know I love virtual reality. I think it’s a fascinating window into human experience, but there is something strange about the current revival. The fundamental purpose is to create and push directly against nature as much as possible. Entering into artifices, entering into little mini puzzles that other people make up, should be treated as dangerous, or as a bad way of doing things, in the information age. It distracts us from reality and it turns computation into amusement, making reality smaller for us. I’d much rather see people using virtual reality in a creative, improvisatory way.
SA: It’s very interesting that you mentioned it, yeah. Can virtual reality one day enable me to be a better jazz improviser? Can I create a virtual reality space where I’m engaging in real time? One of the things that I like most about playing and improvising is the collaborative aspect. Obviously there is no such thing as improvising in a closed room by yourself.
JL: True. We’re working within these information artifacts built by other people. There is something about it that’s very limiting. You know, I remember when you were a grad student. I was basically in your fan club and thought that you were so exceptional and creative. It’s been really exciting to watch your career and see your papers published. It has been really great.
SA: It’s weird and strange and eerie to me that some of this crazy stuff that we were talking about ten or fifteen years ago actually seems to be on the horizon of issues that people are talking about now. It’s crazy.