The goal of lifelogging: to record and archive all information in one’s life. This includes all text, all visual information, all audio, all media activity, as well as all biological data from sensors on one’s body. The information would be archived for the benefit of the lifelogger, and shared with others in various degrees as controlled by him/her.

Some form of this total recall is inevitable, at least for some people. Partial recall from partial recording seems inevitable for the rest. We are all likely to record more and more of our lives.

The extreme form of lifelogging is very radical at the moment. If you are seriously lifelogging you have microscopic cameras that record all you see, all the time, your computer archives every action you take, all your email, phone calls and conversations are kept and annotated, your location is tracked, and dozens of nano-sensors in your body record your vital functions. However you may be the only one who sees all this information. It is sort of like radical self-surveillance.

The technical specs of recording every moment in your life are entirely plausible right now. In ten years, it will be cheap, too. But the social, legal, ethical, and cultural consequences are gnarly, weird, unpredictable, over-the-top, controversial, and thrilling.

A lifelog would offer these benefits;

* A 24/7/365 monitoring of vital measurements such as body temperature, heart rate, blood pressure, and presence or absence of bio-chemicals. This data could serve as a warning system and also as a personal base upon which to diagnosis illness and to prescribe medicines.

* A digital memory of people you met, conversations you had, places you visited, and events you participated in. This memory would be searchable, retrievable, and shareable.

* A complete archive of your work and play, and your work habits. Deep comparative analysis of your activities could assist your productivity, creativity, and consumptivity.

* A way of organizing, shaping, and “reading” your own life.

To the degree this lifelog is shared, this archive of information can be leveraged to help others work, amplify social interactions, and in the biological realm, shared medical logs could rapidly advance medicine discoveries.

By 2007 several dozen people have experimented with lifelogs. The first person I met doing this was Ted Nelson in the mid-1980s who recorded every conversation he had, no matter where or what importance. To my knowledge his archives have never been revisted, even by him. In the 1990s Steve Mann at MIT outfitted himself with a head-mounted camera and recorded his daily life on video tape. Because there was a camera half-obscuring his face, it was hard to be natural around him. For the past 6 years, beginning in 2000 Gordon Bell at Microsoft Research has been documenting every aspect of his work life. His experiment is called MyLifeBits. Bell wears a special camera around his neck, the SenseCam, which will notice a person’s body heat if they are near and photograph them, or snap a picture of a new place if it detected a change in light. Bell records and archives every keystroke on his computer, every email, every phone conversation transcribed, every face to face conversation, and of course every photo he takes, every movie he watches, every web site visits, every window on his computer and how long it remained after – any and all data he can record he does.

For the past two years Daniel P.W. Ellis, an associate professor of electrical engineering at Columbia University, has audio recorded – or audio lifelogged – his life. (One DVD hold 2 week’s  worth of life’s sound.)

Alex Pentland at the MIT Media Lab built a “socioscope” which continuously tracked the location of 81 volunteers for 9 months, as well as track the pattern of their conversations.(To oversimplify their results they found that if you looked the patterns of people living and working in a city they are indistinguishable from tribal herding and foraging behavior.)

While a student at MIT, Bradley Rhodes invented a “Remembrance Agent,” which “pays attention to everything you type; every few seconds it checks inside your hard drive to see if it can find anything relevant. If it does, it alerts you in the corner of your screen by showing a line or two of the related document.”

So how well does lifelogging work? Journalist Clive Thompson visited Gordon Bells MyLifeBits lab and hung out a while. His report for Fast Company is the best description yet of what lifelogging feels like.

For all of its machine muscle, Bell’s virtual memory wasn’t quite what I imagined. When I first heard about his work, I expected someone who would dazzle me mentally, pulling off feats of recollection like some cyborgian savant….MyLifeBits is now so big that it faces a classic problem of information management: It’s hellishly difficult to search, and Bell often finds himself lost in the forest. He hunts for an email but can’t lay his hands on it. He gropes for a document, but it eludes him. While eating lunch in San Francisco, he tells me about a Paul Krugman column he liked, so I ask him to show it to me. But it’s like pulling teeth: A MyLifeBits search for “Paul Krugman” produces scores of columns, and Bell can’t quite filter out the right one. When I ask him to locate a phone call from one of his colleagues, he hits a bug: He can locate the name of the file, but when he clicks on it the data are AWOL. “Where the hell is this friggin’ phone call?” he mutters to himself, pecking at the keyboard. “I either get nothing or I get too much!”

In the winter of 2006 Scott Carlson, writing in the Chronicle of Higher Education, reported his experiment with simply recording his life on digital audio tape. “My wife, who is also a journalist, banned recording at home for the first week because she said I acted like I was “on stage.” I had noticed that, too. I never really forgot that the recorder was on, and now and then I sensed I was talking differently, as if to a crowd. I consciously avoided saying things that might be deemed politically incorrect or downright gross, although some of that slipped out and into my memex.”

Clive Thompson again:

In spring 2004, [Bell’s co-worker] Jim Gemmell lost a chunk of his memory. The Microsoft senior researcher had built his own personal MyLifeBits database, filling it, like Bell, with oodles of his email, Web surfing, and pictures. But one day, Gemmell’s hard drive crashed, and he hadn’t backed up in four months. When he got his MyLifeBits back up and running, the hole that had been punched in his memories was palpable, even painful. He’d be working at a project and vaguely remember some Web site or document that was important, then begin drilling down to find it–only to discover that it was part of the missing period. “It was like having my memories stolen,” he says. He was amazed to realize his backup brain was no longer some novelty but a regular part of his psychological landscape. “I realized I count on this now. It’s like I expect to drive cars and have flush toilets.”

Lifelogging is not yet ready for prime time. The tools for lifelogging today are scarce. You can’t buy Microsoft’s SenseCam yet. There’s a lame Web 2.0 Nokia site called Lifelog, which is just a fancy blog site. And a curious app. called TimeSnapper which takes a screen shot of your computer every few minutes and allows you to view a sped-up week’s worth. You can throw web sites, emails, memos, pdfs, and other digital documents into DEVONthink, a kind of electronic associative memory, but the input is pretty labor intensive. Google’s free Gmail with its promise of eternal email is the closest consumer actual lifelogging tool available.

Writing in the March 2007 issue of Scientific American, Gordon Bell says,

Another technical challenge will be ensuring that users are able to open their digital files decades after storing them. We have already run into cases where we could not access documents because their formats were obsolete. Digital archivists will have to constantly convert their files to the latest formats, and in some cases they may need to run emulators of older machines to retrieve the data. A small industry will probably emerge just to keep people from losing information because of format evolution.

Still the overriding concern is how to read, retrieve, and use this huge – and I mean huge – ocean of data that your life will generate. There’s one solution not normally offered in discussions of lifelogging that seems reasonable to me. Ten years ago computer scientist David Gelertner envisioned a new organizing metaphor for one’s growing cache of personal data. Instead of boxes and “windows” a lifelog should be centered on well, a life log, a timeline, chronological log of one’s life. “We’re going from an artificial information storage scheme to a far more natural one. The idea of a timeline, a chronology, a diary, a daily journal, or a scrapbook is so much older and so much more organic and ingrained in human culture and history than the idea of a file hierarchy.”

Gelertner calls his new perspective Lifestreams. For a while a startup company headed by him called Mirror Technologies offered a downloadable demo of the beta version of the software, but it no longer appears to be available. However, his vision is still valid and in my opinion the  mostly likely model for lifelogging. In an interview with a Sun computer rep he spells it out:

I can imagine all the electronic information in my life collected into one beam, or (equivalently) one flowing stream. Every electronic document: every email, photo, draft, URL, audio, video, calendar or address note, and so on. Life is a series of events in time — a timeline with a past, present and future. The events of your life and the memories in which they’re recorded aren’t parceled out into directories, or typed shoeboxes. An information beam incorporates documents of all types into one (focussable) beam. The question “where did I put that piece of information?” always has exactly one answer: it’s in my beam.

The stream has a past, present and future. The future flows into the present into the past. If I’ve posted an appointment or reminder in the “future” part of the stream, eventually it flows automatically into the present where I’ll notice it and be reminded, and then into the past where it’s part of the permanent, searchable, browsable archive. When I acquire a new piece of “real-life” (versus electronic) information — a new memory of (let’s say) talking to Melissa on a sunny afternoon outside the Red Parrot — I don’t have to give this memory a name, or stuff it in a directory. I can use anything in the memory as a retrieval key. (I might recall this event when I think about Melissa, or sunny afternoons outside the Red Parrot.) I shouldn’t have to name electronic documents either, or put them in directories. And I ought to be able to use anything in them as a retrieval key.

In our view of the future, users will no longer care about operating systems or computers; they’ll care about their own streams, and other people’s. I can tune in my stream wherever I am. I can shuffle other streams into mine — to the extent I have permission to use other people’s streams. My own personal stream, my electronic life story, can have other streams shuffled into it — streams belonging to groups or organizations I’m part of. And eventually I’ll have, for example, newspaper and magazine streams shuffled into my stream also. I follow my own life, and the lives of the organizations I’m part of, and the news, etc., by watching the stream flow.

Once lifelogging becomes prevalent, the really interesting times begin. Dozens of legal and cultural puzzles immediately surface.

* What part of your life is someone else’s privacy?

* Is remembering a scene with your brain different from remembering with a camera?

* Can the government subpoena your lifelog?

* Is total recall fair?

* Can I take back a conversation I had with you?

* Is it a lie if a single word is different from the record?

* How accurate do our biological memories have to be?

* Can you lifelog children without their “permission”?

For many skeptics the social challenges of lifelogging will doom it to a small minority, or else earn it full prohibition. They don’t want ubiquitous lifelogging, and find it implausible than anyone will once they see it in action. In an interview with Scott Carlson, Daniel P.W. Ellis, the guy audio lifelogging, admits, “You know, the reactions I get are mostly pretty strong and mostly pretty negative. Maybe about 50 percent of people I discuss this with have an immediate negative reaction, which they often have difficulty really explaining. There’s clearly a strong intuitive resistance to having a more detailed record of what they’re saying than memory already provides.”

I believe we’ll invent social norms to navigate the times when lifelogging recording is appropriate or not, but for the most part total recording will become as pervasive as text is to us now. It will be everywhere and we won’t even notice it – except when it is gone.