Your Personal Sim: Pt 4 — Deep Agents (Grokking Deep Learning/Natural Intelligence)

The Brave New World of Smart Agents and their Data

A Multi-Part Series

Part 1 — Your Attention Please
Part 2 — Why Agents Matter
Part 3 — The Agent Environment
Part 4 — Deep Agents (this post)

More soon…
For series email reminders, enter your email address at


Why will our smart agents and sims soon become as indispensible as the web and our smartphones are today? Why will we think of our sims as “our better selves” by 2030? To understand this key aspect of our global future, our next two posts will take a deep look at deep learning, a new paradigm of not only machine learning, but of future computer development.

These two will be long posts, but they are about the technology behind thegreatest story of our collective future, the advent of machines that think and feel like us, so I make no apologies for their length. Plenty of people will write the short versions. But there are many doubts and misconceptions on these topics, so the length will hopefully clear up a few of both.

There are also some rewards at the end, to make up for this post’s length. The first reward is the “Mind Meld” (aka, “Merging With our Sims”, or “Slippery Singularity”) prediction. This is a big reward, as it explores how humanity will use deep learning to solve the greatest tragedy presently inflicting our planet — the inevitable death of each and every one of us, due to the disposable nature of human biology. I think this mind meld future is inevitable, and when it comes later this century, hundreds of millions of us, at the very least, will use it to move easily into postbiology as our biological bodies age and die.

Once mind melding happens at scale, and we see that it works, cultures everywhere will stop pretending that human mental death is a good thing, and we’ll upgrade our religious faiths (which will never go away) to be consistent with a new world of indefinite human lifespan, for all who desire it. The second reward is more prosaic, some powerful investment tips you can implement today in the Calls to Action at the end.

The Rest of this Post (on Medium)

Saving the Titanic – Crowdsourcing to Find Hard Solutions, and Unlearning to Implement Them

A Good AH Collection

Hindsight is often 20/20. When we look to the past, we must guard against hindsight bias, where we assume more predictability to certain past events than actually existed. Nevertheless, rethinking the past remains a very helpful mental practice. Not only does it allow us greater insight and self knowledge, as Mark Freeman argues in Hindsight, 2009, it helps us with foresight as well.

Good hindsight helps us build better models of evolutionary and developmental process in the universe. We gain better ability to discriminate between those evolutionary interventions that could have changed the past, moving us to a new and different branch on the evolutionary tree of possibilities, and those which would not have been likely to change outcomes significantly, as there were developmental processes also at work, pushing the system in one predictable direction.

The alternate history (AH) literature genre is concerned with these issues, looking for the leverage points in historical processes, and imagining “what if?” scenarios where those critical levers had been pulled in a different direction. What would the world be like today if Hitler and the Axis Powers had won WWII? What if Hero’s steam engine of 50 A.D. had been subsidized for R&D by a Roman nobleperson or Emperor, and steam-powered devices of increasing complexity had started competing against human labor two millenia earlier? Uchronia maintains a list over 3,100 AH novels, and lists annual awards. The Collected What If?, Robert Crowley (ed.), 2006 is a great intro to the genre.

Debating alternative histories is good practice for envisioning alternative futures, one method of the foresight profession. The U. Hawaii offers an MS in alternative futures, under futurist Jim Dator. There are of course evolutionary and developmental futures as well, as I argue in Evo Devo Universe?, 2008.  Evolutionary processes are those we can influence to multiple future states, like wars, competitions, or election results. Developmental outcomes are those where our interventions won’t change a future state at all, other than delaying or hastening it.  For example, every planet with intelligent biological life must eventually use rocks, spears, wheels, levers, and engines, if you believe in developmentalism. Each planet just takes different evolutionary paths to these future outcomes. If any of us wanted to prevent the arrival of human-surpassing artificial intelligence, or the further acceleration of the world, there likely isn’t a thing any of us could do to prevent those developmental processes from going forward. We could delay or hasten them, but they are going to happen. That apparent inevitability, on careful consideration, should be influencing us to focus our attention and resources on developing these particular outcomes well. The destination may be inevitable, but the path we take toward it is our evolutionary choice. We can fight it and get dragged to the future or we can ride the wave, its our choice.

As any future becomes increasingly predictable, the evolutionary choices for altering it must shrink and eventually disappear. At a certain point, there’s no choice left, only developmental certainty.  As the solution space shrinks, solutions may also get increasingly uncommon or “hard” — both hard to find and hard to implement. The only good way to quickly find and execute hard solutions may be crowdsourcing, or using collective intelligence, the power of a cognitively- and skills-diverse crowd, as social scientist Scott Page notes in his excellent book, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools and Societies, 2008. At the same time, uncommon solutions may increasingly require unlearning existing habits, traditions, and protocols before they can even be implemented, as futurist Jack Uldrich notes in Higher Unlearning, 2011.

Here’s a quick  test: What could have been done, with a motivated crowd of problem solvers, in the two hours after the collision with the iceberg, to save many more of those who died on Titanic?

Recall the sinking of the RMS Titanic, a tragedy of great interest for over a century now, for so many cultural and psychological reasons that don’t need to be recounted here. She struck the edge of a very large iceberg at 11:40pm on April 15, 1912. The engines were stopped just prior to impact. After the impact, Captain Edward Smith summoned Thomas Andrews, the lead shipbuilder, to assess the damage. Around 12:10am, Andrews told the Captain that since five compartments had been breached, the ship must certainly sink in “an hour, or an hour and a half at most.” They also discussed the fact that there were only lifeboats for 1200 of the 2200 souls aboard. The Captain then sent the first wireless distress call, at 12:15am. PBS has a great new doc, Saving the Titanic, 2012, recounting story of the brave engineers, stokers, and firemen who worked to keep at least one of the boilers operational, for the lights and wireless, as long as possible. The ship sank at 2:20am, over two hours after the leaders knew the sinking was unavoidable.

Captain Edward Smith

Prior to the collision there were a number of small, common choices that could have prevented the tragedy. But after the impact, there were very few strategies left, if any, that might save the large majority of people on board. Captain Smith desperately needed some smart people looking hard for survival strategies around midnight on that fateful night. He needed ways to think beyond the box, to find any uncommon solutions. He needed a group of his best officers, crew, and passengers to work on that task. But Smith did not even advise many of his officers of the situation. As the wikipedia post on the sinking says, “he appeared to have become paralyzed by indecision. He did not issue a general call for evacuation, failed to order his officers to load the lifeboats, did not adequately organize the crew, and withheld crucial information from his officers and crewmen.” He also kept all the passengers in the dark as long as possible, to “avoid panic.” But this mushroom management strategy would just delay and eventually heighten total panic, not avoid it.

Those who’ve seen one of the films or read one of the books knows how poorly this approach worked. The first lifeboat wasn’t lowered until 12:45pm, 30 minutes after they were ordered to be uncovered, and that boat and many others were launched vastly under their capacity of 65. A “woman and children first” evacuation policy was proposed by Second Officer Charles Lightoller, but Captain Smith did not supervise the loading process, and no one had responsibility to maximize or expedite it. Passengers getting into the boats weren’t told the ship was sinking. Many wouldn’t get in the boats, preferring the apparent safety of the ship. Only eighteen of the twenty lifeboats were launched, over two hours. They ran out of time when they got to the last two collapsibles. Ultimately only 700 people were saved in the lifeboats, which had 1200 capacity. Even though Titanic’s crew manned them, only two of eighteen lifeboats went back to rescue survivors, pulling just nine people from floating wreckage. 1,502 people died. There have been countless safety improvements since, and six worse losses of life at sea in peacetime, but Titanic remains the most famous maritime disaster in history.

In addition to fully loading the lifeboats, which would have saved as many as 500 lives, a number of other good solutions for saving many more passengers have been proposed, in over a hundred years of hindsight. Experts and amateurs at sites like Encyclopedia Titanica, who know far more of seamanship, engineering, and Titanic history than I ever will, might heatedly critique what follows. But I think at least three are particularly excellent strategies, even if it may have taken an expert willing to think like an amateur to implement them, as we will discuss. Take a moment and see if you can think of any good solutions yourself, then look below the line.

Let me know if I’ve missed any, thanks.


One of the icebergs found near the Titanic

1. Return to the Iceberg, and Tie Up to It. The iceberg that Titanic struck, perhaps this one found in the vicinity by the Minia shortly after the sinking, had dropped ice on the forward deck on impact. That’s how big it was. So the ship’s officers knew that they could get back to that particular iceberg in perhaps twenty to thirty minutes, and they could see for miles around them. There were also several other larger and smaller bergs (called “growlers”) within view as well. Folks could have been offloaded to those in the calm seas of that bitter cold night.

The captain knew and could have told everyone that the Carpathia was en route to rescue them, and was just hours away. They would have spent only the rest of that night on the bergs. Men could have volunteered to get on the bergs first, taking blankets to sit on, and women and children could have gotten into the lifeboats at the end if they ran out of space on the bergs, a reversal of typical evacuation procedures. The berg Titanic hit was described as like the Rock of Gibraltar, with at least one flat top section. Hundreds of men might have been evacuated onto it. One of the ship’s deck cranes could have been used to drop a landing party onto any of the bergs, even if they weren’t flat-topped. They had lots of steel cables, chains, winches, ropes, drills, chisels, sledge hammers, steel, and rope ladders on board. They may even have had dynamite. They could have used axes to hack holds and steel pitons to anchor men to the ice. One good solution for the landing parties on slippery icebergs would have been to make modifications to their shoes and boots, by nailing small pieces of wood to the bottom, pieces that in turn have nails protruding from their bottoms and a few nail heads from the sides, so their shoes would bite the ice. Notice how deep into the solution space we need to get. Our foresight team needs to know how slippery bergs are, about the specialized boots used for ice climbing, and to have confidence in the ability of carpenters to make a few of these boots for landing parties, in very limited time. The rest of the folks could be pulled onto the bergs from lifeboats by ropes, once the landing party had some pitons in the ice.

National Geographic’s / James Cameron’s Haunting Sinking Simulation

Restarting the engines to navigate to any berg, however, would have required the Captain to unlearn some of his many years of training. Titanic had the ability to move either slow forward or slow reverse for at least an hour after the collision, but the engines were stopped for good just after impact. The original iceberg would have been less than a mile behind them once they realized they were sinking, and there was at least one other large berg in view where they went down, and many smaller ones.

The most ingenious idea I’ve yet heard, and the only solution that, with some probability, might even have saved the ship, would have been to dock back up to the original berg, as in the graphic above (a rendition of the initial impact). They could have lashed the ship to the berg using their steel cables, ropes, and one of the ship’s lighter side (bower) anchors (they had two, in addition to the massive center anchor which they wouldn’t have been able to access). Tying cables to the side anchor and using a team and their winch plus pullies to pull it over the berg and dig it in to its far side might have been a good strategy. Pulling both anchors out over the ice and hooking them together might even have worked (engineers would know).  Failing the use of side anchors, they could have also tied cables, chains, and ropes to the front and midships down by its waterline, and hacked a small channel for them over the top of the berg, securing them with pitons. Such efforts might have provided enough buoyancy to keep the ship from sinking. If they’d tied up at the prow early enough, they’d have little weight to support, and the aft chambers might not have filled with water.

In 1860, a Russian-American barge, the Kad’yak, carying 356 tons of ice to San Francisco, hit a rock off Alaska (Russian-owned at the time), and it filled with water, but didn’t sink for three days, until the ice melted. Ice is incredibly buoyant. If one of the enginers knew that fascinating fact, that could have been enough to turn them back. Even if they decided not to try to lash strongly to the berg by the time they got to it, most of the passengers could have been evacuated to the berg as the ship slowly went down, by lashing lightly to it. But I’m willing to bet that with quick strong lashings they could have kept the Titanic afloat. Anyone want to do the calculations?

Put yourself in the mindset of the brilliant shipbuilder, Thomas Andrews. In the very moment after he had just told Captain Smith that the Titanic would go down “with mathematical certainty” because seawater would eventually run over the tops of the unsealed aft compartments as her nose kept sinking. Now imagine someone telling him about the Kad’yak, and he quickly comes up with the idea to lash her nose against a berg, while evacuating the passengers to the berg at the same time. I’d love to see a physics simulation to see whether or not this might have kept the ship’s nose from going under, if they’d been quick about it. At the very least, a strategy of lashing, combined with the use of the pumps, would have bought them more evacuation time. How much more? A great question. Watch this brief and beautiful scene with Andrews and Smith from James Cameron’s Titanic (1997), then imagine what could have happened next. Foresight matters! We all deserve the best foresight we can reasonably acquire.

Unfortunately, unless a smart crowd were charged with the task of finding solutions, iceberg strategies of any kind were unlikely to be discovered, because of standard protocols, quickly executed by Captain Smith very likely without any discussion or consideration of what he was losing by following them. One protocol would have been not move the ship after damage, for fear it would increase the rate of flooding. Another protocol, and the one that particularly damned them as they engaged in it so quickly and broadly, would have been to empty the hot boilers of their coal so they would not explode when cold seawater hit them. Certainly some of the engines should have been kept running, to keep freedom of movement as long as possible. Tradition, the way we think things must be done, is usually wise, but in uncommon situations, it can also be our greatest limitation.

SS Californian

2. Navigate to the Californian. Lookout Frederick Fleet saw a ship, the SS Californian, a small outline sitting next to a field of ice just five to ten miles away from Titanic, only ten minutes after she struck the iceberg. Fourth officer Joseph Boxhall then tried to signal this ship with Morse lamp and distress flares, and the Californian’s night crew, seeing the flares, almost understood the message, but not quite. The Californian was on the horizon the whole time, stopped for the night because of all those nasty bergs floating about, and they could have reached her, or at least gotten very close, if they had moved toward her, either forward or backward, at five to seven knots. At that slow speed they could have also launched lifeboats on the way there, once they started to really nose down.

Titanic’s crew could have made a bonfire on the top deck on the way there, for a visual signal, once they saw their white rockets, launched every few minutes at one point, weren’t being responded to. They could have used the ship’s horn to blow S.O.S., which would have carried in the calm night air for miles. They could have used their guns to make noise. Once they were around a mile away, they could have even reached the Californian with their bullets. At any point over the next two hours the Californian might have taken notice, heard the distress call on the wireless, and brought its lifeboats into action. Again, for this to work, thinking beyond the protocols against restarting the engines would have had to be done.

Adam and Jamie on a Makeshift Titanic Raft

3. Build Rafts. During the lifeboat evacuation, the crew and hundreds of male passengers could have been directed to find and tie as much floatable material together as possible, to make rafts. This is a truly crowdsourceable solution. In addition to all sorts of wood furniture, the ship had massive numbers of wood barrels, oil and food drums, and wood boxes that could have been emptied and lashed with rope, wire, and cable to make rafts. Anyone who’s made a raft knows how quickly you can get flotation if you have a very large number of wooden objects to pile together, as Titanic had. There was even a Mythbusters in Oct 2012 where Adam Savage and Jamie Hyneman, in picture left, show that if James Cameron’s Rose character had put her lifejacket under her plank, it would have supported Jack as well.

If the ship was not moving, rafts could have been lashed together in the water with rope, wire, and cable, and life vests, by a crew using one of or more of the deck cranes, to make a large floating island. You don’t want a lifejacket on you in the North Atlantic, at 28 °F (−2 °C), because you’ll die with full contact within 20 minutes, unless your lucky and drunk, like Charles Joughin, the baker who treaded water for at least an hour before being pulled onto the overturned Collapsible B lifeboat. You want your lifejacket under you, between you and the water, as part of your raft. Again that realization takes some unconventional thinking. It would have taken a bit more unconventional thinking at that point to realize that the women going into the lifeboats didn’t need the vests they were wearing. The men needed them, for rafts. Even if the ship had been moving toward either the iceberg or the Californian, the raft strategy could still have been done, in parallel with one of the two above. In that case the rafts could have been built on the top deck, and launched as the Titanic slowly sank, bow first.

Could a motivated crowd have led us to any or all of these three solutions? I think so, particularly this one. It is heartbreaking to learn that not a single raft was found in the wreckage. That is because the third class passengers were kept down below until the lifeboats were launched, and very few of the passengers knew the ship was sinking until the very end. The leadership strategy of keeping the masses ignorant always has a terrible cost, as seen in this tragedy.

People are usually very effective at quickly coming up with solutions in tight time deadlines, if you trust them with the full news of the situation, however grim. When I googled “Titanic “return to the iceberg” ” I found that high school students came up with some version of both the raft and iceberg solutions, when challenged in a contest. That’s the power of the crowd. Recall NASA’s clever engineers improvising solutions to bring back Apollo 13 in 1970, or Sept 11, 2001, when just a few of United Airlines 93’s brave passengers, using their wits and their cellphones, identified and stopped the terrorists who had taken over their plane.

Certainly there are situations where the crowd doesn’t have the right mindset or training to handle collective power. Mob panic is real, and leaders may need to deny information for brief periods. But panic can also be managed, and justifications for not crowdsourcing solutions get rarer and briefer every year. Learning more conditions in which it makes sense to find and trust the crowd, and quickly build collective intelligence, is one of our greatest opportunities as managers. Here’s a KMPG and Manchester School of Business 2012 report on using the crowd, tamely defined here as bringing doctors and patients closer together in social networks, as a way to accelerate innovation in eHealth program deployment. This is is how it starts, but we can do so much more.

Innocentive’s Problem Solver Market

Whether we are talking about political, defense, economic, environmental, or social problems,  educated people usually deserve to know how bad the situation is, as quickly as possible, and to be empowered to come up with realistic, incremental, bottom-up solutions. To build their own rafts. For that to happen we need a lot more social transparency. We also need to help people become good raft builders. Think of the self-reliance ethic found in several cultures, such as Utah’s Mormons. We get the latter with better education and personal accountability, where irresponsible actions have consequences that are negative enough to influence behavior, while remaining noncatastrophic.

We all need to empower our crowds to come up with brilliant bottom-up solutions as often as possible. We can do this with our staff, our personal contacts, our customers, and the public. With proper education and guidance, cognitively diverse groups will usually find good solutions much faster than we will.

Henry Chesbrough, one of the pioneers of open innovation, has long advocated this. Think of Innocentive, and their growing community of technical problem solvers. Think of incentive prizes. Consider all the people on the web who self-identify as creative thinkers and problem solvers. How soon will LinkedIn or another dominant social network harness all the innovators and creative types into a general online platform that surfaces problems that need solving, and incentivizes competitions for the best solutions, with part of the payment being the growing reputation of the solver?

Spigit and Bright Idea are two new cloud-based innovation management platforms that use pairwise comparison ranking as a way to generate a better distribution of preferences among a set of competing ideas. Most of the other innovation and ideation platforms use simple voting, a crude algorithm that quickly becomes a popularity contest where early winners emerge, and later and potentially better ideas are rarely seen or evaluated. All of these tools are in their infancy, with little machine learning, collaborative filtering, or semantic analysis involved. Yet a few of them have crossed the chasm, as an early majority of cities and companies are now purchasing them, using them, and finding them valuable. Prediction markets are another collective foresight platform that we can expect to continue to grow in popularity in coming years. There are a few companies making those as well, but beyond clever betting sites like InTrade (now defunct), we don’t find that many being used in organizations at present.

One of the more astonishing facts related to Titanic is that the disaster was eerily predicted in fiction, fourteen years before it actually happened. The fictional version of the sinking is so amazingly similar in several details that some have mistakenly presumed some kind of psychic phenomena must have been involved. In 1898 Morgan Robertson, a novelist with extensive previous naval experience wrote a novella, Futility, in which the largest ocean liner ever built, then billed as “unsinkable,” sinks in the North Atlantic, in April, 400 miles off Newfoundland, after striking an iceberg on the starboard side, all just as the Titanic did in 1912. The fictional ship also had less than half of the lifeboats that it needed, most of the passengers died, and the size and features of the ship were near-identical in several other respects. The name of this ship? The Titan. This seems spooky, until you realize that if you are going to build the biggest ship ever, Titan/Titanic was likely the best name in the cultural lexicon at the time. Everyone with sea knowledge knew the risks of icebergs, and where the iceberg lanes were. Anyone who was safety minded knew the lifeboat laws were horribly inadequate, a disaster waiting to happen. Icebergs will do the most damage on one of two sides, versus a head on collision into the reinforced prow, and April and May are the two months when icebergs are most plentiful, another 50/50 choice. None of this detracts from the brilliance of Robertson’s prediction, it just helps us understand how he did it.  Morgan Robertson got there first, and gave us a prediction of a likely future disaster that wasn’t heeded. Ironically, in Robertson’s version, the hero climbs onto the iceberg to survive the sinking. No such foresight emerged when the Titanic shut down its engines, and kept them off, rather than trying to head toward either a berg or the Californian.

In the future, the better our prediction markets get, the better we’ll be at getting our best foresight to the people who need to see it, and challenging them to address the problems and consider the strategies that such foresight uncovers.  To explore that idea a bit further, futurists Venessa Miemis, Alvis Brigis and I just published an article, Open Foresight, in the Journal of Futures Studies, Sep 2012, 17(1): 91-98, where we argue that the best foresight projects in coming years will be based on open access, network-based, crowdsourced approaches. Using a cognitively diverse crowd will quickly generate a distribution of possible futures, and with good iteration and comparison algorithms, the best can rapidly filter to the top. I’m reasonably hopeful that the best of these innovation management platforms today will turn into the best of the open foresight platforms of tomorrow.

A plethora of ideas to be managed

But there’s another component to serious innovation, unlearning our old habits, that is equally necessary for our older managers and leaders, when you’ve got a good sized crowd presenting some great innovation ideas. Including more smart youth in leadership is one good way to avoid the tradition trap. Let’s hope we get other good tools for unlearning going forward as well.

Additions, critiques? Let me know thanks!

On Human Destiny and the Value of an Acceleration-Aware World View

Below is start of an interview I did last month with Phil Bowermaster and Stephen Gordon, my futurist friends who run and Fast Forward Radio. The topic was human destiny, and the personal and social value of cultivating a big picture, acceleration-aware view of the world and our opportunities within it. You can find the full text interview here, at, and listen to a related Aug 31st, 2011 audio interview, on a panel with fellow futurists Venessa Miemis and Robin Hanson, at WT2: Human Destiny Transformed.

Thoughts? Responses? Please feel free to use the comments section here, at my new blog home, and I’ll do my best to respond.  Thanks!

1. If you were to pick just one current or coming transformation that you would advise people to focus on, which one would that be?

I run the Acceleration Studies Foundation, so I’m a tad biased, but I would say one thing to focus on is to try to understand the meaning, risks, and opportunities presented by accelerating technological change, in your own life and in society.  Accelerating change is causing a whole number of transformations today, and several of these are giving us more options for what to do and how to live than we’ve had at any time in the past. Others (automation, globalization, fossil fuels, IEDs) are causing disruption in more ways than ever before. Read Martin Ford’s Lights in the Tunnel, 2009, for one of many thoughtful works on the way technology improves us in some ways, while disrupting us in others.

When you think carefully about accelerating change, you may conclude, as I have, that just a few technologies, specifically computing, communications, and nanotechnologies, are continually accelerating because every new generation of these particular technologies uses less resources per computation or physical transformation than the previous one, a phenomenon I call STEM compression. So these special technologies continually escape the “limits to growth” we see in traditional technologies. And it is these same technologies, as they perennially accelerate, that increasingly shape our future. We can think of them as the growing framework, or cage which restrains and directs all the most powerful actors today, the corporations, the governments, the ultrawealthy, the terrorists, everyone.

Our parents saw minicomputers, cheap telecom and the PC fuel the growth of multinationals, and at the same time, flatten corporate hierarchies. We’ve seen the iPod and internet disrupt the music industry, tablets are now starting to change the publishing industry, and as I argue in How the Television Will be Revolutionized, 2010, a few years from now  iTV will disrupt the television and film industries in even more powerful ways, bringing millions of channels to every device. Blogs, Wikis, virtual worlds, mirror worlds, social networks, and other facets of the internet are flowering and enhancing our collective intelligence. Twitter gives us a window on the thoughts of humanity. Smartphones connect us 24/7 to each other and the web. Sensors are proliferating across the planet, making every nation a transparent society. Lifelogs, wearable computing, telepresence, and augmented reality are just now emerging.

Today, Facebook and Twitter empower the Arab Spring. Tomorrow, we’ll have a conversational interface to the web, and cybertwins (digital assistants that model our personality and can act and transact for us, at first in simple ways, later in very intelligent ways) assisting us in our information consumption, communication, commerce, and political activities. We can expect a valuecosm to eventually emerge, quantitated versions of the publicly expressed values that each of us all hold on all kinds of topics, allowing us to connect with others who share our values, to work with others on projects that we care about, and helping us to generate a whole new level of specialization and subcultural diversity. [The rest of the interview].

%d bloggers like this: