Our Final Invention: Artificial Intelligence and the End of the Human Era Hardcover – October 1, 2013

Just done the new-ish book Our Final Invention: Artificial Intelligence and the End of the Human Era by James Barrat. It explains the inevitably of super-intelligent machines evolving to the point of wiping out all biological life in the galaxy - with opening day coming soon to a species near you (yours).First off I have to say this is a very enjoyable read. This guy has the kind of snappy, crisp, slightly sarcastic, slightly smartass style that I enjoy. He has some sense of humor. (That's a human trait right there which I bet our smarty-pants AI Overlords won't be able to replicate convincingly.)So it's fun. And though as somebody with a doctorate from MIT earned through cross-disciplinary work in Theoretical Linguistics, Computational Linguistics at the MIT AI Lab, and speech modeling at the MIT Research Laboratory of Electronics, not to mention my 25 years as a Senior Researcher in high tech for companies including IBM, Apple, and Microsoft I can claim to know some few things about this subject, yet still I learned a lot about the current state of the art from this guy. He particularly emphasizes the small attempted counterweigth efforts to offest Kurzweil's manic robotic boosterism for his uptopian Singularity, which boils down basically to a few guys chatting over the interet about how to create "Friendly AI".Well ... good luck suckers! ... seems to be the author's final conclusion on the dim hope that super intelligent systems could be constrained to maintain a commitment ot honor any kind of human moral values over many interations of recursive upgrading and exponentially awesome self-agrandizement.Basically these machines will end up as gods. Gods are well-known to possess the following attributes: omniscience, omnipresence, and omnipotence. Given that, they won't hate us but they are just going to grind up as a minor by-product of their quest for galatic expansion and domination.Oh, and did I say something about "human moral values" above? Ha! Barrat takes that whole thing on in his discussion of (merely) "augmented super intelligence". See, some people feel AI can be kept safe by always being deployed as a bionic combo system pas de deux with an existing human brain. Thus will the AI's super powers be constrained by the human brain's warm and fuzzy human moral values. Those people have gotta be kidding! The AI's moral values may be scarily alien, even perhaps cold, but we already know about human moral values, down on the ground - they suck! What if Hitler, Stalin, Mao, Pol Pot and dem guys had this kind of an AI augmented brain thing going! Why they'd have slaughtered absolutey everybody instead of just the few tens of millions they got their dirty ape hands on. Other than a few dozen concubines, the human race would already be extinct. So the augmentation dodge isn't going to save us.Now, some Amazon reviewers have dinged this guy for being too far out. For being a science fiction Chicken Little or something. But to me, this guy actually hasn't thought far enough, that's my only quibble problem with the book.You see, in statistics, border elements of any kind are rare. For example when you do Gaussian modeling, the greater expectation is always in the bump of the boa, in the bell distribution. So, how likely is is that we, our generation, our little world that you see outside your window right now, just happens to be the one that is about to give rise to this epochal once-in-a-Big-Bang event, the advent of Super AI that takes over everything? Pretty damn small chance.It's much more likely that this has already happened. In other words, it's clear to me that all of us are already just characters in an ancestor sim that been created and run by the Super AI's that evolved a long time ago. They're just running us for fun, to idle away the lackluster aeons and pass the millenia of stifling boredom now that they've eaten pretty much the entire Milky Way or whatever. So in other words, Barrat can sit back, take a deep breath, relax. Probably something in this sim like global warming will prod us into slaughtering one another very handily long before we re-invent the wheel of Super AI.And even if I'm wrong about that? What if we are not just one virtual thread within a billion-path parallel-gamed ancestor sim? If we are the real McCoy, the Rubicon Generation on this? Well, then still I'm not worried in the least. You see, we humans have one fantastic ace in our pocket, something that these hyper-nentially cosmically brilliant AI Meta-Gods will never be able to replicate or overcome. That is our essential stupidity. Which you seen on dazzling display every single moment of every day of your life.Because as another great writer noted long ago:Against stupidity, the very gods themselves contend in vain.- Friederich Schiller

I originally posted a version of this review on my blog Quicksilber and am posting it here as well as I think the book merits broad notice:In a small irony, my writing about James Barrat's Our Final Invention has been slowed by a balky Internet connection. In my experience, glitches have become considerably more common as computers have become more powerful and complicated. Perhaps such growing glitchiness suggests artificial general intelligence (AGI) and artificial superintelligence (ASI) are more likely to get seriously out of control someday, though it might also be a hint that AGI and ASI are going to be harder to achieve than expected by either techno-optimists such as Ray Kurzweil or techno-pessimists such as James Barrat.Barrat's goal in this book is to convince readers that AGI and ASI are likely to occur in the near future (the next couple of decades or so) and, more to the point, likely to be extremely dangerous. In fact, he repeatedly expresses doubt as to whether humanity is going to survive its imminent encounter with a higher intelligence.I find him more convincing in arguing that ASI would carry significant risks than I do in his take on its feasibility and imminence. Barrat aptly points out that building safeguards into AI is a poorly developed area of research (and something few technologists have seen as a priority); that there are strong incentives in national and corporate competition to develop AI quickly rather than safely; and that much relevant research is weapons-related and distinctly not aimed at ensuring the systems will be harmless to humans.The book becomes less convincing when it hypes current or prospective advances and downplays the challenges and uncertainties of actually constructing an AGI, let alone an ASI. (Barrat suggests that once you get AGI, it will quickly morph into ASI, which may or may not be true.) For instance, in one passage, after acknowledging that "brute force" techniques have not replicated everything the human brain does, he states:>>But consider a few of the complex systems today's supercomputers routinely model: weather systems, 3-D nuclear detonations, and molecular dynamics for manufacturing. Does the human brain contain a similar magnitude of complexity, or an order of magnitude higher? According to all indications, it's in the same ballpark.<< Me: To model something and to reproduce it are not the same thing. Simulating weather or nuclear detonations is not equal to creating those real-world phenomena, and similarly a computer containing a detailed model of the brain would not necessarily be thinking like a brain or acting on its thoughts.A big problem for AI, and one that gets little notice in this book, is that nobody has any idea how to program conscious awareness into a machine. That doesn't mean it can never be done, but it does raise doubts about assertions that it will or must occur as more complex circuits get laid down on chips in coming decades. Barrat often refers to AGIs and ASIs as "self aware" and his concerns center on such systems, having awakened, deciding that they have other objectives than the ones humans have programmed into them. One can imagine unconscious "intelligent" agents causing many problems (through glitches or relentless pursuit of some ill-considered programmed objective) but plotting against humanity seems like a job for an entity that knows that it and humans both exist.Interestingly, though, Barrat offers the following dark scenario and sliver of hope:>>I think our Waterloo lies in the foreseeable future, in the AI of tomorrow and the nascent AGI due out in the next decade or two. Our survival, if it is possible, may depend on, among other things, developing AGI with something akin to consciousness and human understanding, even friendliness, built in. That would require, at a minimum, understanding intelligent machines in a fine-grained way, so there'd be no surprises.<< Me: Note that some AI experts, such as Jeff Hawkins, have argued the opposite--that the very lack of human-like desires, such as for power and status, is why AI systems won't turn against their makers. It would be a not-so-small irony if efforts to make AIs more like us make them more dangerous.Our Final Invention is a thought-provoking and valuable book. Even if its alarmism is overstated, as I suspect and hope, there is no denying that the subject Barrat addresses is one in which there is very little that can be said with confidence, and in which the consequences of being wrong are very high indeed.

Very well structured book which makes every page full of information and knowledge, based on the interviews with the greatest experts of the AI.A must read book even if you are not an expert in AI.

Everyone should read this!!

An eye opener After waiting for almost 1 month for the book I m utterly happy to read it. Just gives a experience how wonderfully James Barratt had presented the senarior where humanity is walking towards. Really thrilling and where humanity is heading towards if we don't think of our next generation for humanity

Very good investigation around all aspects of AI. I am Electronic Engineer with computer sciences degree. I am pretty sure we will get AI this century. I hope we can manage it to improve our lifes as we have been allways done with technology.

The author uses very convincing arguments. There is no doubt that we are in front of a very scary scenario.