http://www.anthropic-principle.com/?q=book/chapter_8#8c

The World in 2050 [Broadcast by BBC Virtual Reality, August 14th, 2050] (c) 2000, Nick Bostrom SUMMARY This essay explores some of the social, political, economic and technological issues that the world may have to face in the mid-21st century. A central theme is the need to regulate molecular nanotechnology because of its immense abuse potential. Advanced nanotechnology can be used to build small self-replicating machines that can feed on organic matter - a bit like bacteria but much more versatile, and potentially more destructive than the H-bomb. The necessity to prevent irresponsible groups and individuals from getting access to nanotechnological manufacturing capability is a prime concern in 2050. The essay shows how this quest for containment shapes many aspect of society, most notably via the institution of a global surveillance network. A dialogue format is used for two reasons. First, in order to enable several conflicting views to be discussed. And second, to illustrate how public debates may be conducted at that time. [P.S. This essay was written under various external constraints, including that it should deal with the in 2050, so not too much weight should be put on the time line and some other details.] Host: Much has happened over the past fifty years, but what are the changes that really matter? Have they been for better or for worse? And where are we heading in the next fifty years? With us tonight to discuss these questions are three very distinguished polymaths: Emily Brown, CEO of Cisco Systems; Dr. Chun Ten, technology editor of The Economist; and Neil Brigge, professor of philosophy at New York University and a director of the Foresight Institute. Neil, do you think that the human condition has changed over the past half century? Neil: We live longer and healthier lives. We are much richer. We have the ability to choose our sensory experiences to a much greater extent than ever before. Right now, you are watching a smooth-skinned, well-groomed Neil, but the real physical Neil sitting here in my VR-box may be a much less appealing sight - I may not have shaved or washed for a week! Emily: People are still concerned with how they present themselves, both in physical space and in cyberspace. The fraction of our income that we spend on personal presentation has been increasing steadily throughout the century. As it becomes easier to satisfy our material needs, we seem to focus more on what we are in relation to others - there is great demand for what the economists call "positional goods". Host: We are the vainest people ever? Emily: Or the most "people-centered", depending on how you want to see it. Chun: I now see a backlash against conspicuous consumption in some communities. High-status people try to signal their superiority by shunning cosmetic filters and other conveniences that the less well endowed cannot afford to do without. But this "nudism", as they call it, is very partial. It is only a few selected enhancement-options that are being stripped off. Host: At the turn of the century, many were concerned about global warming and other ways in which humans were damaging the environment. Global warming turned out not to be a significant problem, but did this early environmentalist movement have any lasting effects on how things developed? Emily: Environmentalist thinking opened a lot of minds to the idea that we are living on this finite planet and that what is done in one part of the sphere can affect what happens on the other side. "Global conscience", was perhaps the most beneficial lasting effect. Remember that it is only in the last twenty years that widespread undernourishment has finally been eliminated. Of course, bioengineering and nanotechnology were essential tools in achieving that, and this helped reshape environmentalist thinking. At the turn of the century, many environmentalists were very suspicious of new technology. Now most people accept that it is thanks to technological progress that we are able to sustain a world population of eleven billions and still preserve significant parts of the natural environment. Neil: What also happened was that the early focus on the environment gradually widened to include all aspects of human life. Today we have the debate between transhumanists, who are enthusiastic about transforming the human condition, and naturalist conservatives who are happy about technology as long as it doesn’t change human nature too much. While most naturalists will take life-extension supplements, they don’t approve of the use of mood-drugs or cosmetic psychopharmacology. Chun: The emergence of law and order in the international plane - "global conscience" as Emily calls it - is probably the reason why we are still here today. There is still no effective global immune system that could defend against malicious self-replicating nanomachines. Without the steadily improving global surveillance network, some terrorist group or scientifically-literate madman might long ago have made their own nanotech assembler and manufactured a "virus" that could have destroyed the biosphere and eradicated intelligent life from this planet. Neil: Indeed, and this could still happen. It just takes an ill will and a single security breach somewhere. We must just hope the surveillance network improves rapidly enough that this will never happen. Host: How was it possible for the surveillance network to be put in place so quickly? Chun: Well, a lot of the infrastructure was already there. Companies monitored customers, the police monitored public places, the military monitored foreign countries. Once gnat-cameras became cheap and you could link everything up to data bases and face-recognition software, surveillance networks began covering larger and larger territories in more and more detail. The other part of the explanation is that policy-makers could become persuaded by the necessity to regulate nanotechnology. And here I think better collective decision-making institutions played a key role. Idea futures is a good example of this. As you know, idea futures is the market where speculators place bets on hypotheses about future scientific or technological breakthroughs, political events and so forth. Trading prices - the odds - in idea futures markets gradually became recognized as authoritative estimates of the probabilities of possible future events. In the twentieth century, policymakers would sometimes commission the opinion of bodies of experts that they appointed, but it turns out that the market is much better at making these predictions than politically selected committees. Such markets were once banned under anti-gambling laws in most places. Only gradually were exceptions granted, first for a stock market, then for various commodities and derivatives markets, but only in this century did we see the rise of wide-ranging free markets with low transaction costs, where speculators could trade on most any claim. It’s hard to overestimate how important this was in making society’s decision-making a bit more rational. Neil: I agree that this was important. The tragedies of the tens and twenties, where genetically engineered biological viruses were used to kill millions, also helped prepare the world. People have seen the damage that a malicious guy can do with self-replicating organisms. And nanotech is much, much more powerful. I think we have been incredibly lucky so far. Maybe we are witnessing the result of what philosophers call an observational selection effect. Maybe most civilizations in our infinite universe destroy themselves when they develop nanotechnology, but only the lucky ones remain to wonder about their luck. So our success so far should not make us complacent. Emily: It’s amazing how quickly people have got used to the idea that everything they do can now be known by anybody who is interested in finding out. When you are going on a date with someone, you can check out their previous relationships, and so on. If you had suggested this to somebody fifty years ago, they’d have been horrified! They would probably have referred to it as Brave New World, or Orwell’s 1984, with Big Brother watching you all the time. But it’s like a nudist colony: when everybody is naked, the embarrassment quickly wears off. So we had all these little secrets that we thought were so important, little vices. But when we can see that everybody has similar little vices, our standards adapt and we become more tolerant. As long as you’re not doing anything really bad or break the law, you don’t have to worry. And the streets are much safer now. Chun: The key to preventing global surveillance from turning into global suppression is to always insist on basic liberties and to keep government and law-enforcement agencies under constant scrutiny. It is crucial that we make sure that the system is transparent in both directions, so that we can watch who’s watching us. Emily: Yet there are still rumblings about it being a plot by the technologically advanced countries to monopolize the power coming from the nanotech revolution. Chun: The logic is crystal clear to every honest thinker. Since there is as yet no general defense against nanotech attacks that could potentially destroy the biosphere, such attacks must be prevented at any cost. The only way to do that is to limit the number of powers who have nanotechnology. If a single evil madman gets it, then it’s "game over". Emily: I agree that is a good argument, and that we have to limit access to nanotechnology until we have developed reliable defenses. But that doesn’t mean that the leading powers don’t derive an unfair advantage from this technological monopoly. The people who are excluded from building nano-assemblers have a right to be compensated. Current transfer payments are nowhere near what they are entitled to. Chun: It is also a problem of intellectual property laws, which has been a burning issue for quite some time now. - [Emily Brown disappears] Host: It appears that we have lost transmission from Emily - she was on her yacht. Hope she hasn’t run into bad weather - Ok, let’s turn our gaze towards the future. What lies ahead? Neil, I know that you have expressed pessimism about the future - Neil: I’m not a pessimist, but I’m concerned that we might have underestimated the risk that nanotech could proliferate before we can build defenses. There has been a series of incidents already. As technological know-how spreads, it may be harder and harder to make sure that no rogue group develops an assembler. Building an effective global immune system for self-replicating nanomachines is a very, very difficult challenge. It has to be present everywhere, on land, sea, and in the atmosphere. It must avoid attacking biological life forms. And it has to work 100% of the time. We are very far from being able to do that at the present time. Chun: Surveillance technology is improving rapidly, however. Neil: Yes, but what if someone develops a good counter-surveillance technology? Intelligence agencies are now discussing proposals for monitoring and preventing research in that field as well. But the problem is that the task is so ill-defined. Who can say what research might potentially lead to some way of tricking the global surveillance network? Host: Are there any idea futures claims that measure the probability of these things? Neil: Yes, but they don’t really tell us much. You see, who would want to bet on the hypothesis that civilization will be destroyed? If you are right, you would not get paid! It is disconcerting to note that what economists call the time discount factor seems to have been going up. (The time discount factor is a measure of how much more value people place on a present good compared to having the same good at some point later in the future.) That could be interpreted as people being worried about whether they will be around to benefit from their savings. [Emily Brown reappears] Host: Welcome back, Emily! Emily: I’m terribly sorry! I’m on a yacht here, and my son was playing with the antenna - I’m so sorry! Host: Don’t worry. My attention-meter shows that the intermezzo increased our eyeball-count. That’s why we are doing this live. Our viewers like it when things go wrong! You should give your kid a bonus! Emily: I don’t think so! Host: We were talking about the future. Neil was just trying to convince us that he is not a pessimist, and he went on to explain all the excellent reasons we have for thinking that there is a great risk that we will all be turned into gray goo by nanoreplicators running amok. Is that a fair summary, Neil? Neil: I suppose so. But on the other hand, if we do manage to avoid doomsday, things could turn out really nice. That’s why I don’t call myself a pessimist, because if we are careful we might achieve a great outcome. Chun: My grandfather is 102 years old and he is quite active. This is all thanks to life-extension. We can only dream of all the wonderful things that will become possible when we learn to redesign and expand our minds. There is more to lose than ever before. Having got this far, we really must try to make sure that we don’t destroy ourselves with rogue nanotechnology, even if it means holding some types of developments back temporarily. Neil: There are many people who won’t make it. Over a hundred thousand people deanimate everyday, many due to aging-related ailments. More than two thirds of the world’s population still don’t have a cryonics contract! When they deanimate, instead of having the information in their brains preserved by freezing or vitrification, they are just cremated or buried. For worms, there certainly are such things as free meals! And this at a time when in the idea futures market there is more than a 50% chance of reanimation of a human brain within three years! This preventable loss of human life, day after day, year after year, almost makes the two world wars or the Baghdad flu insignificant by comparison. Chun: The difference is that this is largely self-selected death. It’s a price you pay for freedom of religion. Neil: For some people that is true. But there are millions who simply cannot afford a cryonics contract, or don’t have enough education to understand what the options are. Now, I’m not suggesting that we should force anybody to have themselves frozen when they deanimate, but I think we have a duty to provide enough information and financial help so that each individual can at least make an informed decision. Host: Won’t overpopulation be a big problem if all the people who are currently in cryonic suspension are suddenly reanimated? Chun: Actually, there aren’t that many of them. About three billion people have made arrangements to have themselves suspended when they deanimate, but only about 150 million people have been suspended to date. Neil: Idea futures indicate that if these people are reanimated, it will be through being uploaded into computers. It looks much more feasible to disassemble these vitrified brains cell by cell, molecule by molecule, scanning off the neural network, and then run an emulation of that neural network on a computer. In vitro repair is harder than in silico repair - that is, doing it in a computer simulation. And I, for one, would much rather be uploaded than having my biological brain repaired. I already spend most of my time in virtual reality, and I’d like the security of being able to make a back-up copy of my mind every hour or so. If for some reason I want to manipulate physical objects, I would rent a robot body that was suitable for what I wanted to do. Emily: I can understand those who are scared of uploading. I’m scared of it myself. Who can guarantee that a brain emulation would truly be conscious? And even though our virtual reality is pretty good at vision and sound, I still think it can’t compete with the meatspace in the other sensory modalities. Virtual sex is great, but I prefer to touch my husband’s body directly. Neil: The other sensory modalities of virtual reality will work much better for uploads. It is easier to do it for uploads, because you can directly activate neurons in their sensory cortices. You don’t have to build these complicated mechanical devices we have now to stimulate our body surfaces. Emily: I still see it as a last resort. I would rather be an upload than be dead, but I think that as long as my biological body is functioning well, I’ll stick with that. Neil: There could be huge advantages to being among the first uploads. I was attending a neurocomputation conference last week, and I was talking to several groups there who are studying hypothetical ways of enhancing the intellectual abilities of an upload, by adding new neurons and new connections and so forth. If uploads could become more intelligent than humans, I would prefer to be an upload. Also, computing power increases. There is an idea futures claim that within two years of the first human upload, there will be an upload running at a clock speed one hundred times greater than a biological brain. That means the upload will think a hundred times faster than we do! Last time I checked, this claim was trading at better than even odds. Chun: Since uploads are software, they can easily make copies of themselves. We could see a population explosion with a vengeance, and we might have to brush up on Malthus again. Malthus was a late eighteenth century political economist who argued that unless population control is instituted, then population growth will eventually annihilate any improvement in the standard of living for the masses that economic growth has brought about. Even rapid space colonization, which hasn’t been economical so far, but which idea futures indicate is likely to begin within a decade, will not be able to keep up with a free-breeding population of uploads. Emily: Uploading raises huge ethical and political issues. Not to speak of the legal challenges. If you are an upload and you make a copy of yourself, so there are now two almost identical implementations of you, who is married to your husband? We have only scratched the surface of this problematique. I would like to see a moratorium on uploading experiments and some form of artificial intelligence research until we have a better understanding of where we are going. We are playing with fire. Host: I’m afraid we’re out of time. It’s been fascinating. I suppose we’ll see over the next few years how this plays out. Thank you all for participating. Neil, I leave the final word with you. Neil: We are in the process of crossing a road. The one thing we must not do is stop in the middle. We need greater-than-human intelligence to build defenses against nano-attacks. We would not reduce the danger by slowing down; on the contrary, that would make the risks even bigger. The best we can do is to press onward with all possible speed, using as much foresight as we can muster, and hope that there is an other side that we can get to.