Today I was at a talk with Robin Hanson to promote his book THE AGE OF EM hosted by the Bay Area Futurists.
As an academic polymath with interests in physics, computer science, and economics, Hanson draws upon his extensive reading across these fields to try to piece together what such a society will look like.
His argument is that in 30 years to a century, there will be a phase transition as mind uploading takes off and the world economy rapidly becomes dominated by “ems” (emulations); human brains running on a silicon substrate, and potentially millions of times faster. Since transport congestion costs aren’t a factor, this em civilization will live in a few very densely populated cities largely composed of cooling pipes and computer hardware. The economy will double once every month, and in a year or two, it will transition to yet another, cardinally different, growth phase and social structure.
I might or might not eventually do a book review, but for now, here is a link to Scott Alexander’s.
Alternatively, this lecture slide summarizes the main points.
A few observations, arguments, and counterarguments from the meeting:
(1) This struck many people as the most counterintuitive assetion, but I agree that wages in the em world should quickly plummet to subsistence levels (which are much lower than for biological organisms). This is probably what will happen eventually with our civilization if there is no “singularity”/transition to a higher growth phase, since fertility preferences are an aspect of personality, and as such, highly heritable. (Come to think of it this is basically what happens to the Imperium of Man in Warhammer 40k, down to the hive cities in which most citizens eke out “lives of quiet desperation,” though ones which “can still be worth living.”)
Since Ctrl-C Ctrl-V is much easier and quicker than biological reproduction, a regression to the historical (and zoological) norm that that is the Malthusian trap seems – barring some kind of singleton enforcing global restrictions on reproduction – seems inevitable.
(2) A more questionable claim is Hanson’s prediction that ems will tend to be more religious than humans, on the basis that hardworking people – that is, the sorts of people whose minds are most likely to be uploaded and then copied far and wide – tend to be more religious. This is true enough, but there is also a strong and well known negative correlation between religiosity and intelligence. Which wins out?
(3) The marginal return on intelligence is extremely high, in both economics and scientific dynamism (Apollo’s Ascent theory). As such, raising the intelligence of individual ems will be of the utmost priority. However, Hanson makes a great deal of the idea that em minds will be a black box, at least in the beginning, and as such largely impenetrable to significant improvement.
My intuition is that this is unlikely. If we develop technology to a level where we can not only copy and upload human minds but provide them with internally consistent virtual reality environments that they can perceive and interact within, it would probably be relatively trivial to build brains with, say 250 billion neurons, instead of the ~86 billion we are currently endowed with and largely limited to by biology (the circulatory system, the birth canal, etc). There is a moderate correlation between just brain volume and intelligence, so its quite likely that drastic gains on the order of multiple S.D.’s can be attained just by the (relatively cheap) method of doubling or tripling the size of the connectome. The creative and scientific potential of billions of 300 IQ minds computing millions of times faster than biological brains might be greater than the gap between our current world and that of a chimpanzee troupe in the Central African rainforest.
Two consequences to this. First, progress will if anything be even faster than what Hanson projects; direct intelligence amplification in tandem with electronic reproduction might mean going straight to the technological singularity. Second, it might even help ems avoid the Malthusian trap, which is probably a good thing from an ethical perspective. If waiting for technological developments that augment your own intelligence turns out to be more adaptive than making copies of yourself like Agent Smith in The Matrix until us ems are all on a subsistence wage, then the Malthusian trap could be avoided.
(4) I find this entire scenario to be extremely unlikely. In both his book and his lecture, Hanson discusses and then quickly dismisses the likelihood of superintelligence first being attained through research in AI and neural nets.
There are two problems with this assertion:
(a) The median forecast in Bostrom’s Superintelligence is for High Level Machine Intelligence to be attained at around 2050. (I am skeptical about this for reasons intrinsic to Apollo’s Ascent theory, but absolutely the same constraints would apply to developing brain emulation technology).
(b) The current state of AI research is much more impressive than brain emulation. The apex of modern AI research can beat the world’s best Go players, several years ahead of schedule. In contrast, we only finished modeling the 302 neuron brain of the c. elegans worm a few years ago. Even today, we cannot obtain functional models even of 40 year old microchips from scanning them, to say nothing of biological organisms. That the gap will not only be closed but for the brain emulation route to take the lead is a rather formidable leap of faith.
Now to be fair to Hanson, he did explicitly state that he does not regard the Age of Em as a certain or even a highly probable future. His criterion for analyzing a future scenario is for it to have at least a 1% chance of happening, and he believes that the Age of Em easily fulfills that condition. Personally I suspect it’s a lot less than 1%. Then again, Hanson knows a lot more computer science than I do, and in any case even if the predictions fail to pan out he has still managed to provide ample fodder for science fiction writers.
(5) My question to Hanson during the Q&A section of the talk: Which regions/entities do you expect to form the first em communities? And what are the geopolitical ramifications in these last years of “human” civilization?
(a) The big factors he lists are the following:
- Access to cold water, or a cold climate in general, for cooling purposes.
- Proximity to big human cities for servicing human customers (at least in the initial stages before the em economy becomes largely autonomous).
- Low regulations.
So plausible candidates (according to Hanson) would be Scandinavia, or the “northern regions of China.”
As he also noted at another point, in the early stages of em creation technology, mind uploading is likely to be “destructive,” i.e. resulting in the biological death of the person who is to be emulated. So there might be an extra selection filter for state or corporate ruthlessness.
(b) In domestic and social terms, during the transition period, humans can be expected to “retire” as the em economy explodes and soon far exceeds the scope of the old human economy. Those humans who control a slice of the em economy will become very rich, while those who don’t… fare less well.
However, Hanson doesn’t have anything to say on the geopolitical aspects of the transition period because it is much less predictable than the “equilibrium state” of the em economy that he set out to describe. As such, he does not think it is worthwhile for someone who is not a sci-fi writer to delve into that particular issue. That makes sense.
(6) As a couple of people pointed out, atomic weapons can wipe out an entire em “city,” which contain billions of ems.
What would em warfare be like? The obvious answer is cyber-cyber-cyber we gotta hack the mainframe style stuff. But surely, sometimes, the easiest move is to just knock over the table and beat your opponent to death with the chessboard.
If Pinker gets pwned during the em era and global nuclear wars between em hive cities ruled by Gandhi emulations break out, could this make em hive cities unviable and result in a radical decentralization?
(7) How did Hanson become Hanson?
He repeated the Talebian argument (which I sympathize with) that following the news is a pointless waste of time.
It is much more productive to read books, especially textbooks, and to take introductory classes in a wide range of subjects. To try to get a good grasp on our civilization’s system of knowledge, so that you might be able to make productive observations once you reach your 50s.
Confirmation bias? Regardless, it’s one more small piece of evidence in favor of my decision to log off.
This is what Hansons actually believe.
Sounds horrible, I really do hope that this future won’t come to pass.
And I have my doubts about the whole “making productive observations in your 50s” argument…at that age cognitive decline has already begun.
There is a decline in cognitive processing speed with age. On the other hand, you do need a decent inventory or storehouse of mental “stuff” to be able to make productive observations, novel connections and analogies between unrelated things, etc.
All of these ideas trace back to early Soviet thinkers like Fedorov
Maxim Gorky was another one :-
The British commie scientist John Desmond Bernal
Don’t. Some other factors begin to play big time in making those productive observations in 50s.
You are at your best when you write on futurology.
Keep up the good work!
Isn’t this guy the least bit concerned that such “ems” would mean the end of humanity as a species?
Fedorov is a “Russian Orthodox Christian philosopher” who died years before the Bolsheviks came to power. He isn’t rightly considered Soviet.
Interesting quotes though.
Hanson’s goal (as he views it) isn’t to preach or polemicize but to take a plausible future scenario and then apply the various things he knows (economics, thermodynamics, computer science, etc) to draw out that scenario as comprehensively as possible.
After all, if we view this scenario as very bad and/or dystopian, we will have more of an incentive to draft up policies to avoid some of its very worst aspects from becoming manifest. For instance, Hanson suggests an inbuilt kill switch (“right to suicide”) for every em to forestall torture.
Hanson also pointed out that even though there are aspects of this world we view as bad (subsistence wages, expendable lives) there are also plenty of positive aspects (no pain, hunger, disease, no terror of death, incredibly luxurious virtual realities). He points out that hunter-gatherers despised the farming world even as they were driven back by it, and that traditional farmers wouldn’t exactly approve of the modern lifestyle.
So just as it is not appropriate to judge the past by the standards of the present, so likewise it is probably not a great idea to judge the future by the standards of the present.
I bought this book’s Kindle version a couple of weeks ago and just started reading it. Don’t know if I’ll finish it. If I do, I’ll probably write a review for my little blog.
He says in the first few pages that society will choose the best (smartest, hardest-working, most agreeable, etc.) humans to be emulated electronically. That sounds implausible. Steve Sailer talks a lot about the top and bottom being allies against the middle. It’s true. The top needs the bottom. The forces that prevent meat-space eugenics will probably prevent em-space eugenics as well.
Also, in the very beginning (I’m at the 4% point) he paints a forwards-and-upwards picture of societal and tech development. Actually there’ve been long periods of regression and stagnation. A civilizational peak was reached by the Greeks around 200 BC which was not surpassed until about 1400 AD. The Far East had long periods of stagnation as well. The Maya collapse. The Soviet collapse, which caused a retreat of civilization, its regression over a vast area. There’s a powerful worldwide dysgenic trend right now. The current era of tech progress could be a fragile thing that could be eaily undone by political developments.
Faced with the choice of whom to emulate, most modern states would choose proportional representstion based on race, ethnicity and gender. Which would carry modern racial and ethnic politics over into em-space.
The Chinese (the Southern Song dynasty in South China) were the pinnacle of civilization around 1250, before the Mongol conquest. As far as I know the Ming never managed to reach the same level (even though they were probably the most developed country in the world around 1400), nor the Qing.
There could be more such examples of which we (or at least I personally) know very little.
“Em” is the kind of fantasy that people disconnected from reality come up with..It’s actually a nightmare..Why would anyone even want to live as a disembodied brain? Answer, they wouldn’t.
Just one scenario that could stop tech progress within our lifetimes: a victory of libertarianism. Libertarianism is very similar to early, pre-Stalinist communism in its policy prescriptions, subconscious motivations and in the kind of people it attracts. It’s very popular among smart nerds, which gives it a good chance to grab power. All significant scientific and technical progress in history was made possible by government funding. A victory of libertarianism throughout the West will end human progress.
Karlin: It is much more productive to read books, especially textbooks, and to take introductory classes in a wide range of subjects.
The best mathematics teacher I’ve ever had detested the available textbooks.
The best economics educator I know of says that economics textbooks are awful, generally, and that even the best ones misinform on basic points (viz: Samuelson on creation of money).
The best history teacher I have ever dealt with always emphasized his dislike of history textbooks: he says he educated himself by almost completely avoiding such textbooks.
Some of the best teachers I’ve ever had never used textbooks in my classes.
Jeff Schmidt, Roger Schank, Jacques Ranciere, John Gatto, Denis Rancourt and Clive Semmens, – each a person with extensive experience in teaching and distinguished academic results – emphasize the overwhelming importance of living experience, interacting, making your own decisions in education, and two of them are explicitly dismissive of reading as a way to learn.
Just saying.
All significant scientific and technical progress in history was made possible by government funding.
Medieval Europe, virtually without state authorities let alone government funding, was a stage of remarkable technological expansion. (see Medieval Technology and Social Change, Lynn White; The Medieval Machine, Jean Gimpel).
I share most of your estimations of Hanson’s em-future: interesting to think about, very implausible.
I do fear that Hanson misses a big question-mark: will ems have qualia? Functionalists would say of course they will, since there’s no reason they wouldn’t.
However, I would note that the leading quantitative framework for consciousness (IIT) as well as a couple of promising approaches to consciousness not-yet-formalized-into-frameworks (Tegmark’s Perceptronium, Aaronson’s ‘arrow of time’ hypothesis) suggest that whole-brain emulations would not automatically replicate the qualia of the original brain.
And if ems aren’t conscious, Hanson would need to write a significantly different book.
IIT and Perceptronium are searchable, but here’s a link to Aaronson’s hypothesis, which I actually find pretty compelling:
http://www.scottaaronson.com/blog/?p=1951
From what I understand, Hanson doesn’t make huge assumptions about the tech necessary to build these emulations. I think he says that it boils down to “scanning” the brain, like with a photo scanner or Xerox machine, that is scanning very many cross sections of the brain so that you can produce a 3D image out of these very many 2D cross section scans. Then the 3D image displaying all the neurons and connections are replicated with computer chips. Or something along those lines.
Sounds like a perfect recipe for the Butlerian Jihad.
Who needs machines when you have nootropics and genetic manipulation.
To put it mildly, none of those attempts to explain consciousness are impressive. They all amount either to shrouding emptiness in a veil of buzzwords, or throwing disparate mysteries together in a sack and pretending that they must amount to the same thing.
Why are you so emotionally tied to your human body?
It is just a bunch of meat.
Sex…
What is the equivalent of that in “silicon substrate”?
Peace.
Down with thinking machines! Mentats welcome…
Peace.
If we stimulate the right parts of our brains via tech, we might get something better than sex.
‘Concerning life, the wisest men of all ages have judged alike: it is no good’. Intellectuals want to transcend the scarcity and the human condition in order to abolish war, or even inner conflict. So nations have to go, and biologically based minds will get uploaded and run things after somehow leaving behind all the troubles that go with having a personality.
The question of what cities will go first – what an interesting question. It was my favorite of the night. My top guess would be Tianjin or Tangshan in China. They are at a northern latitude, close to humans in Beijing, close to the water for cooling, lax regulatory oversight from the government, and access to a massive existing manufacturing network. Not only that, but the government is in transition and trying to hold on to political legitimacy through economic prosperity. There is additional strength to the China argument in that Ems would almost certainly be militarized (a huge oversight in “Age of Em,” I think, that issues of war were mostly discounted and waved away). The PLA is the world’s largest military force; there’s no way that they wouldn’t want to take advantage of something with such obvious martial potential. If Ems can be tortured to give up their secrets, if they can day dream and take drugs and make love, then they can certainly be brainwashed into being ultimate fighting machines (keeping with the black box assumption). Fun fun!
Hey Mitleser,
1) Why not just legalize opiates – this seems to hit the mark and is available now – just look at that smile!
https://www.youtube.com/watch?v=H632nShBISg
2) I guess in this pornified age it just means better and better virtual sex – which sounds so nerdy that I believe one who brings this up in conversation (enthusiastically) in the presence of a prospective female will end up confining himself to availing of sex only ‘via tech’.
Peace.
But then this means that since dinosaurs didn’t like that we mammals replaced them, we should also be happy to be replaced by ems, or something. This is not a very good argument. Neanderthals were unhappy that we replaced them, so that’s a good argument not to be happy about being replaced by black Africans ourselves. But I’d prefer to be replaced by black Africans (who are, at least, humans, our not too distant relatives), than by ems, which are not even humans, not even mammals.
It seems to me that the logic of your preferences would lead to ems. Your preference seems to be that you prefer whatever is most similar or identical to yourself at the present to persist (which doesn’t seem to make sense, since over time similarity gets overwhelmed by change, and the only relevant factor seems to be lineal descent, but that’s a different issue and beside the point here). If you lost your leg in an accident, that which is most like yourself, yourself, would persist in your new body which includes a prosthetic leg made out of metal. And this would apply further until the rest of your body is a metal prosthesis, as long as the “software” of yourself runs on the new “hardware”.
Becoming an em will change your hardware and software beyond recognition. I don’t welcome death, but living on as a ringwraith seems repulsive to me, losing both my body and my personality in the process.
Your personality is shaped by your experiences, which include the experiences you get from your body. By changing your body (and serum testosterone levels) through working out, you change your personality. Mind you, you won’t have hormones at all. What kind of “identical” “software” are you talking about? My hormonal responses are part of my personality, for example my response to young pretty girls, or children (also hormonal response, probably the sight of my daughter decreases my testosterone levels immediately), etc. That will get lost.
Even if it was retained, the possibility of upgrading your brainpower will change your personality inevitably.
I think the ringwraith metaphor (which I just made up at the beginning of this comment) is spot on: you will become more powerful, immortal, but lose your body, soul and personality in the process.
The ultimate transgender experience!!!
I haven’t read Hanson’s book and don’t know all the details about this stuff. But I thought the assumption is that conscious experience would be identical. We’re also assuming that this mind uploading stuff is possible in the first place. Obviously if we don’t make these assumptions, then our discussion changes.
First, I don’t think it’s possible, because of the hormones.
Second, even if it’s possible, my point was that your personality is slowly changed by your bodily sensations.
Let’s just put ems aside for a moment, and think about what would happen if our brains could live on for another hundred years (and without Alzheimer’s or other types of dementia) in an android body. This would still change the personalities of these brains, because their bodily sensations would be quite different from biological beings. Moreover, if these cyborgs (human brain in a machine body) lived on for like five hundred years, over the four hundred years they’d spend as cyborgs their personalities would change quite a bit. Even though they’d have memories of being biological, those memories would slowly fade.
Now imagine what these ems would be like. Depending on the hardware, they could think many times faster than us. Have you been in situations when someone told you something, and you figured out what you should’ve answered like five minutes later? In a sense, these ems would all become wittier. Then there’s the problem of their experiences shaping their personalities. You said they would be identical to the human? Well, then, their experiences do shape their personalities. But their experiences are totally unlike human experiences. For example, they don’t even have bodies. Not in the real sense of the word. Then, they can create virtual realities to their liking. They can have sex with hot women, etc. So they could become virtual reality addicts. (Mind you, that’s a problem for real humans, too.) Since they can be many times faster than real humans, they’ll definitely have time for any simulations they like.
Then, they’ll be kinda sorta immortal. That will also change their outlook.
Then there’s the important problem that once we understand human brains so much we can upload them we’ll also be able to improve them – and it will be easier to improve the ems than the biological things. So, these ems will be improved. Again, they’ll quickly change beyond recognition.
I’m sure ems will for a long time behave much like humans do. But after centuries or millennia, their personalities would change beyond recognition. That’s inevitable.
So what we’ll have will be a machine that’s very different from a human. A Neanderthal or a chimp or a race of super-intelligent (but still biological) Einsteins (where, obviously, the most intelligent ones will be way more intelligent than Einstein himself) will be much much much closer to us than these ems.
How can we accept to be replaced by them?
As an aside, I don’t think we can become immortal by uploading ourselves onto a computer.
What will happen is there’ll be a copy of ourselves. We can even create many copies.
Imagine that atom by atom, I built a copy of my body. Would it be me? No, it would be an identical copy of myself. (I’d be uncomfortable if such copies existed, but nevermind.) His sensations would be different from mine, because from that moment on, he’d be a different person. (His memories, of course, would be identical. As would be his personality – but all these would start to diverge.)
So an em of myself wouldn’t be me. It’d be a copy of me. And a virtual copy to boot.
We’re assuming all kinds of apparently impossible things here though. If we accept Hanson’s scenario, we’re assuming that mind uploading is possible, that the conscious experience would be the same.
I agree that personalities would change with radically different experiences, although presumably there could be early 21st century virtual reality simulations that maintain similar experiences and thus similar personalities, or android bodies that replicated the human body and experience and housed uploaded minds and provided them with the same experience.
I don’t know if it’s a matter of us accepting to be replaced by them. If this does happen, presumably it’ll start with people using it as a means of life extension.