Curated by: Luigi Canali De Rossi

Friday, November 28, 2003

Personal Knowledge Mapping And The Concept Of Data Emergence

Sponsored Links

Once upon a time, in a far-away land, there was an advanced civilization of intelligent beings who wanted to be able to share their personal know-how, preferences, ideas and visions with those others that were set to work and collaborate with them.

Tired of spending millions of dollars to deploy piles of useless shelfware systems like Motive, Autonomy, Verity, Serviceware, Kana, Siebel that companies bought to create a supposedly "more productive" 'knowledge-enabled' culture, some companies set out a few communication explorers to investigate if the direction taken was appropriate and if there were effective alternative approaches to it.

Communication explorers pointed out that failure to better capture organization knowledge were in fact mainly due to a deep lack of understanding of the importance "tacit knowledge" played over the "explicit knowledge" everyone was so willing to share.

The explorers also brought long rolls of papers documenting the many painful failures of all those large organizations who had been looking to their IT teams to create a knowledge management 'magic bullet' solution to make them "richer", smarter and more effective.

Everyone knew at the time that due to the excessive number of big egos, and to the extraordinary lack of communication skills, no grasp of the 'big picture' was ever available to any organization top management for effective strategic use.

What was even more irritating was the fact that organizations could not leverage and effectively gather, nurture and easily share the best learning practices and experiences emerging in some areas of their organization.

Knowledge and progress remain disconnected, fragmented and unavailable to the organization as a whole to take advantage of.

What was most interesting was the explorers had questioned and uncovered that a key Chinese wall blocking the realization of the classical knowledge management dream was the perennial and deeply rooted competitive friction between various departments and divisions and among the different 'levels' of knowledge workers existing inside an organization.

To be able to break out of the established communication policies, out of the internal censorship and approval workflows, and from the long practiced habit of publishing ONLY what is "official" and institutionalized was probably the hardest challenge of all.

No amount of sophisticated technology could ever solve that which was completely rooted in the organizational attitude to internal knowledge creation and into the individual belief that personal and critically valuable intellectual assets should not be shared and made available to others inside the organization.

The foundation of this vastly erroneous prejudice on the part of the individual was discovered to be simply due to a natural and unquestioned fear that by sharing personal knowledge one would be diluting his value and contribution to the organization while becoming easily disposable of (replaceable).

In this light the communications explorers set out to define, draft and invent tools and approaches that would facilitate a tacit knowledge creation/sharing mechanisms while not adding extra layers of work and responsibility to every knowledge worker in the organization.

By following their search and quest for finding the right path to effective organization knowledge management, a few of them were able to exemplify simple but critical concepts that would then lead the way to the initial development of what we maybe calling personal knowledge mapping tools. One of such key concepts fuelling the development of such potentially serendipitous knowledge gathering tools is an idea which some of you may have heard before: it is called "Data Emergence".

Let's find out more about it from those very communication explorers. Here is what I have found:

"Data Emergence is the incidental creation of personal information through the selfish pursuit of individual goals."

While in no way should Data Emergence be considered a new concept as

"Internet Marketers have been leveraging Data Emergence through the collection and analyzing of click stream data, surveys, purchase histories,"
and personalized product references provides a stellar living example of this, the careful analysis of the fundamental ideas behind it open up a realistic venue for the effective development of new smart tools and approaches supporting effective knowledge mapping and sharing.

Which are the main issues that affect smooth data emergence?

1) When data emerges about me, and even though legally I own the data, I generally have no control over where it is stored, how it is used or who gets access to it

2) Once data is captured, it generally cannot be shared or used for any purpose other than what the party capturing the data decides to use it for.

For example, let's say I go to Amazon to search for a book. Amazon's backend systems capture information about who I am and what types of books I like to read. For me, the creation and capture of that information is passive.

I have very little control over it.

Amazon puts that data in their database and every time I go back to their website they make use of it to personalize my experience. However, when I go to, that information about me does not travel with me. It stays at Amazon's site. cannot use it. I want the information about me to travel with me regardless of the sites I visit. Instead, must recapture that data. Once it does, I have two different collections of data that describe the same aspects of my identity. Neither of which I have any real control over. That sucks.

This is the traditional approach where each individual content provider must capture and collect any information about me it can in order to be able to make a guess as to how I'd like to interact with the information or services they are providing. This is horribly inefficient and their guesses are usually wrong. I've never visited a web site that I've enjoyed interacting with." A better solution:

When I visit a site like Amazon, the data captured about me is stored in a location I control (e.g. on my machine). In fact, rather than my "personalized experience" being generated by Amazon's backend systems based on information about me they're collecting on their systems, I want my "personalized experience" to be generated on my own system based on information about me that I'm collecting on my own system. I want Amazon to do nothing more than provide a catalogue of the products and services."

Source: Snellspace

Here is therefore what a rudimentary personal knowledge mapping system would need to be like:

Any Web site should become nothing more than a set of raw data feeds while knowledge workers would be provided with a personal software tool that would allow to:

1) maintain a database of personal information.

2) selectively share that data with anybody I choose.

3) autodiscover new sources of content.

4) completely control how I view and interact with the content sources I've chosen.

This is the right approach.

Content providers should not be trying to guess how I want to interact with their information.

They should just be providing the information.

I will customize my experience as I see fit.

Source: Snellspace

As Stefano Mazzocchi correctly puts it, the concept of 'data emergence' can be expressed in very simple terms:

"you don't go around bothering people to markup their data as *you* like it, but *you* make an effort to collect their data and make a sense out of it."

Then, I could use that information and share it in any way that I see fit.

We have to face up to the fact, though, that this tussle between markup and emergence is the knotty yin and yang of semi-structured data management. The struggle has deep roots, and neither can nor should produce a victor. The best and only possible outcome is dynamic equilibrium.

That all sounds way too new age, but in practice it boils down to really basic and simple stuff. In a brief incisive essay called The Cornucopia of the Commons, which is chapter four of Peer-to-Peer: Harnessing the Power of Disruptive Technologies, Dan Bricklin drills down to the essence of the Napster phenomenon: using that simple, desirable user interface, you are also adding to the value of the database without doing any extra work. I'd like to suggest that one can predict the success of a particular system for building a shared database by how much the databases is aided through normal, selfish use.

For many years I've been preoccupied with how to empower people to work together more productively. It really does come down to making virtues of laziness and selfishness. We see that happening all around us in blogspace, in ways that I hope (and believe) can transfer to the business enterprise.

Every corporate retreat begins and ends with the theme of communication. You've been there, done that. "We don't talk to each another." "I didn't know you were working on that project." "It's not my department."

The missing ingredient is shared awareness.

Blogspace, of course, is a laboratory in which new modes of shared awareness are being invented every day.

Source: Jon Udell

Jon Udell brings in also two effective examples of data emergence naturally generated by tools we use for selfish purposes. The RSS aggregators ability to create and maintain a dynamic and truly updated blogroll as well as the ability of All Consuming to keep an updated list of all the books that have attracted my attention.

Both greatly simplify the job of making tacit knowledge explicit while offering others the opportunity for serendipitous exploration and learning.

But as Jon Udell also correctly points out

"The activation threshold for doing these things is dramatically lower than it used to be. But it's also dramatically higher than it needs to be."

"Businesses need to work out something like scenario one in a general way. When teams form and work together, the "markup" that enables and documents team formation, and that represents shared work product, needs to arise naturally and invisibly as a consequence of tool use."

Source: Markup and emergence, yin and yang by Jon Udell

The above article was inspired by an intense and very personal tacit email conversation with Judith Meskill. She had kindly asked me to engage each other into a collaborative effort supporting her efforts toward the completion of her personal paper on Personal Knowledge Mapping. I wrote this short article in an attempt to assemble and best summarize my limited knowledge in the field and the best ideas I had found during my limited research.

The merit, insightful opening overview, and precise focus on the enterprise failing approach to knowledge management are all fruits of Judith Meskill long and valuable research.

I truly invite you to follow her rich and disruptive ideas as they unfold on her knowledge notes and to credit her the most for my ability to pull together the above ideas.

Recommended readings and referenced articles:

  • Udell Data Emergence

  • Markup and emergence, yin and yang by Jon Udell

  • Data Emergence, Self-hosted identities, Auto Discovery and the Future of Web Browsing by James Snell

  • Pedantic Web by Sam Ruby

  • Business Case for Personal Productivity Improvement by Dave Pollard

  • Personal Knowledge Mapping

  • judith meskill's knowledge notes...

    Readers' Comments    
    2003-12-09 02:32:25

    John Howard Oxley

    How pleasant it is not to be alone! This is one of the tectonic plates underpinning the information revolution. That our tools are so inadequate to our needs and our potential, while scandalous, is less a matter of technical shortcomings, and more a matter of organizational limitations. Nibbling around the edges of this is as important as shooting it in the heart -- the analogy of the blind men and the elephant is relevant -- and some important feeling is being done in this article.

    2003-12-01 20:26:31

    John Blossom

    A great summary of some key concepts. I disagree with some of the key concepts detailed by James Snell regarding data emergence. Details at:

    I am not sure that even in a perfect world would it be possible - or even desirable - to have a machine that would present content "exactly as I want it". Content and publishing is all about presenting something that is created by an author for an audience, making the essential value of content captive to the creator, the audience and the venues in which it is presented. While I agree with many of the fundamental premises of data emergence, the real-world services emerging from these premises are likely to be notably different from those inferred by these theories.

    2003-11-29 17:34:32

    Ralph Poole

    Good post. You have covered a number of different topics here. Most interesting to me is how your information consuming behavior should be carried with you so that it informs other searches. Currently we must do all the integrative work by ourselves, researching, analyzing what we find, and interpreting it. Often I write about my personal explorations in my blog. From the accretions of my postings, a picture emerges of my struggles to understand how people learn, share their understanding of a topic, and immerse themselves in the overwhelming sea of data and information. But I do value the process and I don’t think it can be automated. Knowledge management can only point to patterns of use and provide a sense of how someone else solved an analogous problem. We can learn by examining examples, but it is never easy, and certainly will only be helped a little by computer mediation.

    posted by Robin Good on Friday, November 28 2003, updated on Tuesday, May 5 2015

    Search this site for more with 








    Curated by

    New media explorer
    Communication designer


    POP Newsletter

    Robin Good's Newsletter for Professional Online Publishers  



    Real Time Web Analytics