Curated by: Luigi Canali De Rossi

Tuesday, March 2, 2004

The RSS NewsMaster

Sponsored Links

The NewsMaster: A New Emergent Socio-Professional Role

In the beginning was Yahoo and AltaVista. Then Google came... then the size and the needs grew to such a level of complexity and need for personalization that none of those tools was appropriate anymore.


We are soon to be under enough evolutionary pressure to improve our ability to access specialized news and information in ways and modalities that will make searching with Google or Yahoo appear like a primitive and much limited approach.

Indeed the key problem we are confronted with is this: how to navigate, filter and reduce to humanly manageable dimensions the gigantic flow of information coming at us in increasing amounts.

The entry of the RSS publishing and distribution format into the fray of media and technologies available for communication has only added to the tidal wave of news and information that has gradually been submerging us.

The proliferation of many alternative online independent news sources and the epidemic expansion of the bloggers community have created a sharp increase in information channels available to us outside the mainstream, established sources.

Further, the Web keeps growing at a steady pace, and the amount of content being made available on a daily basis is simply too much to be managed by our everyday Googles and RSS aggregators. (Nonetheless the interesting feedback and comments made by Jon Udell and the Scobleizer, my argument does not really center about the stretching of our human abilities to scan thousands of titles while pretending that to be an effective way of searching and finding what is relevant. If they like to keep on doing that way, I am not against it.)

Jakob Nielsen wrote in June of 2003: "The easier it is to find places with good information, the less time users will spend visiting any individual website. This is one of many conclusions that follow from analyzing how people optimize their behavior in online information systems. "

And you can bet that with the epidemic RSS is having, the exploding amount of content sources, the likely emergence of a multitude of capable newsmasters and innovative technologies supporting them, the number of information seekers, researchers and normal individuals that used to surf and access tens of sites per day, will, in relative terms, drastically decrease.

If information can come at you in such selected, delicious, ad-free fountains, why would you go surfing or searching the Web?

"Online publishers and e-commerce sites no longer need to see search engines simply as sources of traffic and advertising networks.
For the first time, publishers can use a search engine to filter and deliver relevant content to their readers within their own pages, including customized topic search and syndication"

Scott Rafer, Feedster's CEO said.

We need something of an entirely new order of magnitude to manage all of this information.

Search engines, open directories, and millions of bloggers are not enough.

We need a multi-layered, self-organizing approach that allows the load to be highly distributed and the focus and depth to be guaranteed by the combined result of many highly focused individual efforts.

"The layered mechanism works because at no point is the entire weight of the filtering process concentrated in a single individual or a single resource.

It means that individual agents can work without the need for central control, with the only requirement for a functional system being an open set of connections between the agents.

What RSS does best is that it allows an individual to scan, filter, and pass forward.

That's all it ever has to do.

The network can and will do the rest."

The Issues Catalyzing The Change

  • Too much information coming in.

  • Too many sources too scan.

  • General categorization of sources too limited. By author or publishing house is too limiting. I can't scan 100 authors a day to see if there is some interesting for me to read or write about.

  • Too many new sources coming online daily. Which ones do I pick?

  • Difficulty in separating relevant news from press releases, personal rants and job vacancy announcements

  • Too much redundancy - feeds are replicated, news are duplicated several times, the same PR stories get republished over and over again.

  • Dubious quality of many news sources.

  • Selfish. Our search for relevant content and news is an individual activity, in general with no direct benefit for the community at large

  • Non-reusable. Searching for information is mostly carried out as separated, non-contextual, just-in-time quick personal search.

  • Demand for quality, filtered, spam and ad-free selected content on specialized topics is in very high demand it is at best pretty hard to find.

  • RSS is the only place where you can start intelligently filtering the ocean of info, but the process and tools presently at hand are limited, and the time required to scan thousands of feeds and titles is non-trivial.

  • Google, Yahoo, and all of the other major search engines are not sufficiently powerful to extract and provide quality content on specialized news topics. They are good at finding online resources and specific content but when it comes to producing ongoing specialized information channels these tools do not seem to stand the test of time.

  • Bloggers are a valuable and interesting source of independent and alternative information but it is still very hard to extract valuable content from the thousands of personal rants, superficial notes and other trivial and research-lacking information items.

    RSS newsreaders are still author-based aggregators of information requiring the news gatherer at bay with hundreds of interesting resources to scan. News seekers search information by focused topic and not by author. The approach is flawed at its foundations and needs to be rethought in ways that allow for the content that is relevant to me to surface more easily from the whole pool of sources I select.

    The Network Starts To Organize Itself

    As Stephen Downes mentions in his beautiful post about networks, the network of information needs to organize itself automatically, and we must help it in doing so.

    Filtering of the online content happens at different levels and it needs to be a distributed task.

    As Stephen points out no single resource is capable of handling the oceanic flow of information unaided. And if it does, like Google, it does so often ineffectively.

    "Inherent in the structure of the Internet there are distinct layers of filtering mechanisms, each consisting of a "gather filter forward" mechanism.

    At the most basic level the mechanism is fulfilled by a human agent, as in the case of blogs.

    In others, it is fulfilled by automatic mechanisms, such in new search tools and online specialized aggregators...

    The reason why this system works, while other approaches do not, is that there is no reasonable mechanism which can apply the vast requirements of filtering on a single resource.

    If we use metadata, the indexing soon outweighs the content.

    If we use search engines, each resource must be subject to extensive analysis to determine context.

    What's important here is that each node of each layer need not worry about the rest, and need not be focused on the goal of the system.

    The agent seeks what is available, the way a retinal cell gathers light, and passes on what is relevant, the way a neuron passes on a signal. The filtering occurs not in the individual node, but through the independent actions of the aggregation of nodes."

    RSS Dolby

    Bloggers play an important role in this autonomic filtering process, but they are also adding, in the process, lots of extra noise.

    It may actually be, that to make the filtering process more accurate and precise, the level of noise needs to be increased rather than decreased.

    I have been recently noticing how the universe of RSS-based content is also becoming extremely redundant and rich in duplicates and repetitions. I myself feel guilty, as an explorer and early tester of this polluted environment. But as I reason about it with detachment I realize that it is something indeed inevitable (many others will need to test) and probably beneficial.

    The littering and polluting happens as you test how to create new RSS channels and feeds, and when this channels get picked up, exposed or shared inside a directory or RSS search engine, then it becomes very hard to prune and delete them from the RSS universe.

    While apparently this would seem to baffle traditional logic, you have certainly heard about the Dolby sound system, which achieves though pre-emphasis a cleaner, hiss-free sound, by first adding more noise to the original source and then filtering out with greater ease what superfluous.

    So it may indeed be that bloggers add an ergonomic hiss to the information flow which the upper level of news gatherers and aggregators will actually leverage to filter the actual "valuable content".

    What So Great About This?

    The discovery is that we can now start to really organize information in some meaningful, creative, very useful ways.

    As smoothly as the words in the previous sentence traverse your mind, you must realize that this is quite an impressive turn we are about to make. Nor this is anything we have seen before happening during this civilization.

    No Yahoo, Google, ODP, has or can do what we are about to see.

    It is going to be a blossoming of specialized news channels like nothing we have seen before.

    The discovery is in fact in the unlimited and yet untapped power we now have to search, filter, aggregate and create focused news/information channels with the only support of our know-how, culture, experience and a little unknown free technology called: RSS.

    What's The Great Benefit?

    The real value is in the fact that the effective widespread application of this filtering and redistribution process would be a highly valuable social endeavor.

    As amazing as it may appear, the emergence of the newsmasters shows that individualistic and self-directed work may actually provide great social benefits.

    The creation of dedicated information channels, originated by independent publishers and not by vested commercial interests or mainstream media conglomerates may create the opportunity for a true renaissance of culture, learning and to a multiplication of our abilities to manage large amounts of rapidly changing information.

    Search specialist and librarians who will craft with time investments appropriate queries to get at the information they were looking for, will be generously rewarded with eternal fountains of relevant info for the time to come.

    The act of search evolves from a mere set one-time shots at finding something just-in-time, to the start of a collective refined meta-filtering process that generates better information for everyone.

    The more refined filtering process that the newsmaster creates can benefit private enterprises but can also benefit, for extended time and with great value the research and learning needs of free individuals not connected to direct commercial interests.

    Having a great flow of information that can be tapped, aggregated and filtered accurately is a benefit for all. Its uses and applications are limitless, and they can not only enrich our rapidly evolving global culture but can also give the opportunity for ethical sustainability to those who have the will and skill to leverage them.


    "What's important here is that each node of each layer need not worry about the rest, and need not be focused on the goal of the system.

    The agent seeks what is available, the way a retinal cell gathers light, and passes on what is relevant, the way a neuron passes on a signal.

    The filtering occurs not in the individual node, but through the independent actions of the aggregation of nodes."

    The Network Is Already Doing It!

    At the most basic level, bloggers represent the first generation of self-organizers of the network. If we exclude Yahoo initial honorable effort an similar one (e.g.: ODP), there is nothing else, that comes close in scope, to what the universe of bloggers or noosphere, has been able to achieve.

    Bloggers have been first to scout new independent sources, alternative voices, and who have pointed links to new ways of looking at any issue. They, more than any other group, have brought, among much useless noise, the true emergence of effective meta news sources that originate, filter and aggregate valuable content online.

    "Decisions about selection and classification are made on a case by case basis using very coarse, and unregulated, mechanisms."

    So, what it appears to the many superficial onlookers as a universe of mindless writings (blogs) is nothing less than the initial phase of a complex and orderly process whereby humanity at large takes control of filtering, gathering and re-organizing his own know-how an discoveries.

    Newsmasters may indeed represent the second layer of filtering that we can now apply to this ocean of content and information. ...and Newsmasters, as it appears, have been indeed emerging and quietly working at this nouvelle craft, for quite some time now.

    And with the above, the emergence of a new breed of tools, tuned to provide the specific facilities required by the fascinating job of the Newsmaster, heralds the beginning of a popularization process which will allow, like in the past for DTP (desktop publishing), those who have the skill, to become the true competitors of any mainstream information source.

    Now that we have stripped information from everything that was superfluous to it, the real value and skill is in the true ability to find, pick, select and bring together intelligently the best information bits available out there.

    And due to the unique nature of RSS-based content and of its user-centric character, what Newsmasters create are much like geysers of information, open hoses that generate uninterrupted useful info on a specific topic. Could we say "fountains of knowledge"?

    For what I can infer, it is also likely that in the near future, once Newsmasters are an established profession, end users themselves will start to apply on their own an extra level of more sophisticated aggregation and filtering to the highly specialized feeds they select.

    So What Is A Newsmaster And What Does sHe Do?

    NewsMaster definition:

    The newsmaster is an individual capable of personally crafting RSS-based specialized information channels by utilizing technologies that allow hir to select, aggregate, filter, exclude and identify quality news, information, content, tools and resources from the whole universe of content, news and information available on the Internet.

    Newsmastering is the ability of a human being to concert, orchestrate, edit, and refine quality search formulas that tap into the whole Internet content universe and beyond, and that filter out relevant information through selected keywords, source selection, ranking, heuristics, and many other possible criteria

    The Newsmaster specializes in crafting uniquely powerful search/aggregation/filtering formulas generating continuous RSS feeds on narrowly selected topics by:

    a) Resource identification. Selecting and aggregating valuable resources (like a normal aggregator does)

    b) Query formulation at different levels and on different content bases.
    1) Creating advanced search queries in the blogosphere ( a la Bloglines

    2) Creating advanced search queries on the traditional Web content at large

    3) Creating advanced search queries on the overall pot of content derived at
    points a), b) and c) and generating new highly filtered RSS feeds matching
    specific content and quality criteria.

    4) Creating advanced search queries to the deep/invisible Web with dedicated
    tools and creating RSS content out of the results obtained.

    f) Filtering. Applying exclusion filters based on keywords, keyphrases, date, language, and more.

    g) Splicing (union): I want feed C to be the result of merging feeds A and B.

    h) Intersecting: Given primary feeds A and B, I want feed C to consist of all items that appear in both primary feeds.

    i) Subtracting (difference): I want to remove from feed A all of the items that also appear in feed B. Put the result in feed C.

    l) Splitting (subset selection): I want to split feed D into feeds D1 and D2, according to some binary selection criterion on items.

    g) Heuristics. utilization of heuristics allowing the system to learn from previous choices the patterns characterizing the most appropriate news items from the rest.

    i) Reputation. possible utilization of real-time collaborative filtering mechanisms allowing the newsmaster to benefit also from the rating and reputation given to selected news/information items by other independent newsmasters.

    f) Personal selection. The ability to integrate in the newsmastering workflow the ability to personally select information items or to continually improve and refine the governing search and aggregation formulas by way of personal evaluation.

    Some examples of newsmastering work available out there:

    Here are some living examples of newsmastering work available out there:

    1) Stephen Downes - Edu_RSS

    2) Harry Potter channel

    3) MasterViews - a timid initial approach by the author to newsmastering with the automatic generation of a news page focusing on presentation, information design and PowerPoint resources and skills.

    Over 11 dozen news feeds and over 100 blogs aggregated to give a picture of the information security space. Nothing non-commercial met these needs, i don't think anything else does yet. (by Jose Nazario)

    A personal world news system, aggregating close to 100 news feeds. Through hueristics it identifies interesting news items and sorts the results in that order. Got sick of google news' fluffy feel? Here is the alternative. (by Jose Nazario)

    6) Please suggest more! I know there must be thousands out there!

    The NewsMaster Toolkit - An Opportunity To Support Robin Good

    In an effort to offer you an elegant way to support my further research and reporting in this direction, I have made available a small report entitled the Newsmaster Toolkit.

    It contains now over 40 different selected tools that can be used for experimenting and fully testing the thrill of newsmastering today. With your personal skills and these technologies you can start building dedicated news channels and focussed mini Web sites in little time.

    Once again, don't get me wrong. The toolkit is an opportunity I am offering you to support my reporting in this area. If you want to thank, reward, encourage or simply acknowledge the value of this, please do get yourself a Toolkit today.

    More information about the NewsMaster Toolkit here.

    Personal Thoughts

    Newsmasters and their role are central to my position as an evangelist of intelligent use of new media technologies. Here is a great opportunity to be taken: How to intelligently use the power of RSS-based content to facilitate more effective, learning and communication, by initiating a process of self-organization of all of the content available on the Internet.

    I would suggest the ones of you that like what I am describing to start working and getting skilled at this as soon as possible.

    Being there at beginning and understanding early in the game the principles of this fascinating mechanism can only provide you with greater ability to manage and craft more effective tools rather than having to follow what the market will make available for everyone.

    At any rate, the desired asset will not be the one of only having a better or more evolved technological toolset, but rather at the combined personal ability and skill to filter, select, pick and edit/comment those news, ideas, information and resources that are most valuable to your specialized information channel(s).

    While technology will certainlt greatly contribute to our ability to rapidly select and manage while drawing from a large poll of sources, it will be our personal experience, know-how and intuition that will guide us in customizing the filtering mechanisms we will create.

    So if you are into newsmastering you need both to learn and refine your research skills applied to this new vast set of information sources now available, as well as to facilitate the development and refinement of technologies that ease the tremendous scobleizing task at hand (picking through thousands of RSS-based content feeds and filtering out the pure junk from what may be just waht you are looking for).


    How RSS Can Succeed by Stephen Downes

    RSS Needs by Dina Metha

    The algebra of feeds, or the amateurization of RSS bricolage by Sebastian Paquet

    Research, inspiration and ideas-checking: Massimo Curatella

    Readers' Comments    
    2004-03-10 13:53:34


    Robin, I'm convinced you're on right track. And way ahead of most of the people who are looking at this space. Keep it up!

    2004-03-04 16:14:24


    It is interesting that the links referenced and other similar weblog discussions on this topic avoid the sexist, hegemonic spin, mass'uh

    2004-03-03 22:41:08

    John Blossom

    I found this to be a particularly stimulating piece, a lot of good thoughts. Detailed comments on our weblog:

    All the best,
    John Blossom
    Shore Communications Inc.

    posted by Robin Good on Tuesday, March 2 2004, updated on Tuesday, May 5 2015

    Search this site for more with 








    Curated by

    New media explorer
    Communication designer


    POP Newsletter

    Robin Good's Newsletter for Professional Online Publishers  



    Real Time Web Analytics