Total Recall: How the E-Memory Revolution Will Change Everything (10 page)

THE NEW JOB

When you start at a new job, it can take a while to get up to speed and learn the ropes. How intensive and how crucial this process is varies widely. It might not be too big a deal in the case of, say, a new waitress at a theme café, who may take a few weeks to figure out what the style of the place calls for and the predictable orders of various regular patrons. At the other extreme it may have enormous importance, such as when a new president takes the helm of the world’s mightiest nation.

I don’t recommend this movie for students of political science, but in
National Treasure 2
there is a presidential “book of secrets.” The book supposedly contains secrets for a president’s eyes only, and is passed on to each new occupant of the White House. While no such book actually exists, at least as far as I know, there is clearly a vast body of knowledge that must pass from one POTUS to the next. The news media loves to run stories on the recently elected politician who doesn’t know X concerning his or her new duties. Just think of the enormous body of knowledge that an aspiring president is expected to have on the tip of the tongue.

In between the president and the waitress are a million other jobs that have memories to pass on. Total Recall will break new ground in the effectiveness of transferring memories from one occupant of a position to the next.

In order to tolerate staff turnover, many large organizations are structured around clearly defined functions and operating procedures for each member of the team. There is no better example than the military, whose constant movement of personnel demands an extremely modular approach at all levels of its command structure. Soldiers and officers routinely rotate in and out of positions on their tours of duty, and even homeland bases and training facilities regularly shuffle their staff. Whether you are a new base commander, quartermaster, or front-line soldier, you are expected to drop into a position and be effective the moment your boots hit the ground. National defense, with such well-defined roles, is a fertile area for the application of Total Recall.

In early 2003, program director Doug Gage and some of his colleagues from the Defense Advanced Research Projects Agency (DARPA) came out to San Francisco to meet with Jim Gemmell and me. They were interested in MyLifeBits as a model for a research program they were hatching called LifeLog. They had already held a LifeLog workshop and decided they were interested in a system that “captures, stores, and makes accessible the flow of one person’s experience in and interactions with the world” and that “can be applied to a wide spectrum of associate/assistant systems to allow the system to ‘understand’ the user’s state based on knowledge of the user’s history (timeline, routines, habits, etc.), in order to make the user more effective in a wide variety of tasks.” They envisioned that LifeLog technology “could result in far more effective computer assistants for war fighters and commanders because the computer assistant can access the user’s past experiences . . . and result in much more efficient computerized training systems.”

We were excited after the stimulating brainstorming session around our conference table, and the project seemed to be building momentum. Was the U.S. Department of Defense, one of the world’s largest organizations, going to be leading the way to the age of Total Recall? We didn’t realize that LifeLog was headed into the middle of a political minefield.

In June 2003, William Safire wrote a column for
The New York Times
about LifeLog that put the fear of Big Brother into the reader’s heart:

And in the basement of the Pentagon, LifeLog’s Dr. Gage and his PAL, the totally aware Admiral Poindexter, would be dumping all this “voluntary” data into a national memory bank, which would have undeniable recall of everything you would just as soon forget.

Although Safire seemed to finish the piece with tongue in cheek, invoking “Ned Ludd, who in 1799 famously destroyed two nefarious machines knitting hosiery,” the powerful image of Admiral Poindexter (a key figure in the Iran-Contra scandal) in the basement spying on the population was enough to send the political class into hysterics.

Poindexter had also been the face man for another DARPA initiative called TIA, for Total Information Awareness, which was unveiled in the months following the terrorist attacks of September 11, 2001. The aim of TIA was to create a centralized überdatabase incorporating every electronic record, transaction, communication, file, and footprint the government could lay its hands on about every person and organization in the nation. They would then sift through this megadossier with data-mining software in search of patterns that could identify terrorist plottings.

Safire had blown the whistle on TIA as well, in an earlier column from November 2002. There was enough public outcry over the possible abuses of TIA for it be officially scrapped a few months later.

The stink over LifeLog seemed to rest on the fear-driven belief that it amounted to the same thing as TIA. But there was nothing about LifeLog that would have required people to entrust all their personal data to a central server farm in the bowels of the National Security Agency. There was nothing about LifeLog that even implied people would be required to do lifelogging at all. This effort was aimed at helping the individual soldier or officer in a state of information overload.

I keep my nose out of partisan politics. I guess that made me naïve enough to imagine someone would just explain the truth of the situation (Safire hadn’t even spoken to anyone at DARPA) and sort things out. Instead, LifeLog was canceled. If I had cared more about politics, I might have been outraged and suspicious that a lot of political decisions were made based on juicy headlines rather than common sense. In any event, I knew not to waste my outrage on this, because I understood a little trick of technopolitics: Ideas that run into trouble, especially good ones, are often officially dropped only to be resurrected, recycled, and rebranded until they gain acceptance. Technology does not give up or give in.

So LifeLog is dead; long live ASSIST! DARPA created the Advanced Soldier Sensor Information Systems Technology (ASSIST), carefully explaining how it would help just soldiers. No one brought up Admiral Poindexter this time, and the program went ahead.

A great example of the fine work done under the ASSIST umbrella comes from the contextual computing group at Georgia Tech, led by Thad Storter. They have shown the kind of ASSIST daily impact could have for a soldier on patrol:

A platoon goes on a presence patrol in Iraq. Their goal is to be visible to provide a sense of security for Iraqi civilians, encourage goodwill, and look for signs of insurgents. Upon returning from their five-hour patrol, the platoon leader is debriefed by his intelligence officer.

“Anything unusual today?” inquires the intelligence officer.

“Pretty calm except that the children were acting strangely,” he replies.

“How do you mean?”

“It’s kinda hard to describe. . . .”

In Iraq, today’s soldiers are fighting an insurgency that uses civilians for cover. According to the soldiers we interviewed, the most common point of contact with the enemy is the Improvised Explosive Devices (IEDs) used against vehicles and troops. Soldiers are forced to uncover the enemy through everyday patrols and intelligence gathering. However, the soldiers are facing an information shortage; they are not equipped to gather this type of everyday intelligence. Soldiers also need a means to share information with intelligence officers and between patrols. Currently, this information is mostly conveyed orally or through images taken with the soldiers’ personal digital cameras. Georgia Tech’s Soldier Assist System (SAS) attempts to augment this process by automatically capturing a “blog” of a soldier’s patrol and allowing him to rapidly select media from that patrol to share with his intelligence officer.

To understand the goal of SAS, let’s revisit the above scenario. While on patrol, the patrol leader and each of his two squad leaders wear SAS capture hardware. Each system records high-resolution images from a head-mounted camera, two streams of audio (one from a close-talking microphone and one from a chest-mounted microphone to record ambient sound), location using the Global Positioning System, and the soldier’s movement using accelerometers on the wrists, hip, thigh, chest, and weapon. During the patrol, the soldiers can also use their high-resolution manual camera to capture images they feel may be important later. Upon returning from patrol the platoon leader now has the information to answer the intelligence officer’s questions:

“Pretty calm except that the children were acting strangely.”

“How do you mean?”

“It’s kinda hard to describe, but let me show you.”

The platoon leader now looks at a map with his GPS path overlain. He selects the area around a local mosque where he met the children and scans for an appropriate image. Looking for images where the system indicates he was speaking, he quickly finds an image with the children’s faces to the intelligence officer.

“You see, generally the children will come up to us along the road because they know we carry candy for them. But today they are here along the wall of the mosque,” states the platoon leader.

As the debriefing continues, the intelligence officer sees a suspicious white pickup truck in the background of one of the platoon leader’s images. While the platoon leader’s blog does not have a good image of the truck and its environment, he uses SAS’s automatic annotation system to select images from his squad leaders’ blog where they “took a knee” to provide security while he was talking with the children. (Often, when monitoring the environment to provide security, the soldiers support themselves on one knee while maintaining a good field of view.) The platoon leader quickly finds a good image of the truck and shows it to the intelligence officer.

“I bet you the owner of that pickup truck was there just before you and was scouting the area for the insurgency. Let’s record this license plate and give it to the next patrol to look for,” says the intelligence officer, ending the debriefing.

The Georgia Tech team found that, for the soldiers, “there was no such thing as ‘too much information’ for presence patrols.” However, with all the data their system could collect, they didn’t want the soldiers spending many extra hours wading through it all to find relevant parts to report. So, they did some postprocessing to automatically detect and tag activities like raising a weapon, walking, running, crawling, standing, shaking hands, driving, opening a door, and so on. These are not intended to replace the intelligence, intuition, memory, and common sense of soldiers, but to complement and enhance them. By combining these automatic tags with time and location, a soldier can quickly find the desired parts to report on.

When a new soldier rotates into an assignment, these improved, data- and media-rich reports will help him get up to speed, and will provide a new kind of resource to draw on. They will allow him to take over the memories of the assignment as well as the assignment itself. For example, if the new soldier sees a suspicious pickup truck, he can look at images from his predecessors’ reports to see if it is the same one. Intelligence officers can go back to extra footage not included in the report to check out other elements that may not have been considered important at the time—was that old man near the truck spotted in the vicinity the last time the truck was seen?

In addition to DARPA, I’ve also met with CIA contractors to discuss applications for Total Recall. After all, the station chief in Budapest needs to hand on his memories to his successor too. It’s another kind of tour of duty, where memories of faces, locations, vehicles, and so on can make life-or-death differences.

Total Recall is also important for the intelligence analysts at their desks back in the States. An interesting project called Glass Box records everything an analyst does at his PC workstation, literally recording a video of what is on his screen at all times, as well as tracking e-mail, opening documents, keyboard and mouse activity, Web surfing, instant messaging, and copy/paste events. The analyst can also make notes by talking or typing. Glass Box can be used to evaluate what research tools are most valuable to analysts, and possibly to detect traits of star analysts that could be taught to others.

Total Recall isn’t limited to helping soldiers and secret agents do their jobs and fill their assignments; Every line of work will benefit from Total Recall.

Jon Gilmore worked as a Sprint engineer for nine years, designing new cell sites in regions where coverage needed improvement. He drove all over northern California measuring signal strength. He also visited individual cell sites to improve their performance, adjusting their power levels or tweaking the direction of an antenna. On his last day at work, Jon handed over the key to his filing cabinet and a hard drive holding about fifty gigabytes of information.

“I used to travel with a compass, a GPS, and a digital camera,” says Jon. “I would verify the exact location of the site—often they were situated a little differently than what was in the records—and check the direction each beam was pointed in. I’d take digital pictures of the site, and the views from it to show the surrounding terrain.”

“I also made notes and took pictures about access to each site. For instance, we used to call one ‘ankle-biter lane.’ If you read my notes, you knew not to get out of your car between the first two gates, unless you wanted to be bitten by the little dog there.”