The usefulness of aggregating, filtering and eventually publishing RSS newsfeed content is not something everyone, even the insider circles has fully understood yet.

Well, I guess it takes a bit of direct exposure and exploration to fully understand where the meat really is.

Steven M. Cohen, a popular and well-respected reporter for the Information Library professional, writes in his "Library Stuff" blog:

"Microsoft is supporting RSS in their beta search. I'm not the least bit excited about this. Here's why:

The top ten results for a general search engine (like G or Y!) rarely change for general keywords. (Who uses more than 2 keywords anyway, right?)

I monitor numerous queries in over 16 engines for my consulting work and there are rarely any new results that appear in the top 10.

The point of RSS is to get updated with anything new to appear on a page (and more importantly, customized feeds created by the user). Since the information in search engines rarely changes (and remember, you are NOT searching the live web), why bother supporting RSS?

This makes no sense.

Can someone please explain why I would want to add this to my aggregator?"

Here is the answer:

Of course, if you want to monitor the news, and only those sources that have been accepted by Google, MSN and Yahoo as approved news sources (quite a limited number), then I have no problem in agreeing with the above.

But say that relevant sources that have highly valuable content on the topics you care about are not part of that pool of a few thousand sources that Google News and its competitors monitor on a hourly basis, then you are both under-served and left out of a great deal of very useful stuff.

For example, assume that you want to monitor what is being said about new RSS tools and technologies or even better about the very topic we are talking about: RSS web-based search feeds.

RSS search feeds tapping into Google News, Yahoo News and MSN can surely bring up a constant stream of interesting things.

Are you getting the latest from Scoble, Winer or me when you search there?

I doubt it.

Try instead on your disqualified Google Web search that query and see how relevant the results are and how fresh too!

Try that same search on your beloved News engines.

Google News doesn't index the blogosphere and Yahoo News will also bring up mostly results from reliable and established major media sources (e.g.: Washington Post, NY Times, and sometimes Slashdot too).

Of course your mileage may vary. And quite a bit as well.

But my point is not to dismiss one in favor of the other.
I am for inclusiveness.
I am for making it very clear that few, rare results, once in a while, can be invaluable gems.

What's the use in having a feed with tons of new content, of which only very little is truly of good use to me?

So, though it is true, that for certain popular keywords the general Web search results from Google, Yahoo and MSN change rarely, this is just a consequence of the type of query being used, not an actual trait of any Web search.

This is a consequenceand a characteristic of highly competitive industries, in which SEO optimization (the art/science of optimizing pages so that they show up among the highest ranked results for a given query) is driving the first few pages of general search engine results.

But this is true only if you look at generic, popular one- or two-keyword searches.

As a matter of fact, the Web does grow daily with new pages covering most any topic field, and a witness to this is my own constant monitoring of specific keywords and keyphrases that do alert me of relevant new resources, tools and articles becoming available on the Web at large.

I do agree that if my typical query is for e-Learning or Web conferencing, my Web search results (and consequently any RSS query-based feed I create) could bring up very few new updates a day.

But I start to wonder a bit, when someone tells me that he is monitoring certain industry topics by using "general keywords" and no more than two keywords.

What kind of professional, competitive intelligence search is that?

Don't even need RSS to do this, and scores of tools have been making this possible without the complexity of RSS.

Is this really the general approach that search professionals, industry analysts and any independent reporter online uses to monitor key topics?

In my humble view, a) if your query terms are a bit more specific than general keywords (and therefore exempt from the negative results caused by excessive SEO optimization), and b) if you leave behind this false concept that keyphrases longer than two keywords are unpopular (I really wonder where Mr Cohen is getting this assumption from), you can certainly be in for a lot of good surprises.

If your goals, like that of Mr Cohen, are to be constantly updated on new resources, content and articles and to monitor key industries and research areas, there are two basic issues you need to look at:

1.) To do this monitoring job effectively you need to monitor three main content and news-generation areas. If you leave one out, you may be missing truly unique and valuable information. These are:

a) The Web at large - this is the largest content area of all, and through it you can get access to newly indexed content coming from the most disparate sources. From traditional web sites to the latest blog posting. Too valuable not to be used. (Tools to use: GoogleAlert, MSN Search)

b) The news - yes, this is a wonderful universe of new content that is a trove of valuable pointers. Just keep in mind that the number of sources covered is very limited and that this news is strongly skewed toward the points of view of traditional, established media. (Tools to use: Jeremy Zawodny's Yahoo! News RSS URL Generator, MSN beta, Google News RSS Generator and Grouper.

c) The blogosphere - not tapping into the rapidly growing resources of independent genuine content, commentary and analysis is like leaving out the best for who is a true information seeker. The blogosphere is so richly textured, varied and wide, that it is a must-be-used content source for any professional RSS-based search or intelligence gathering project. (Recommended tools: Daypop, Feedster, Pubsub).

All three sources—Web, news and blogosphere—are equally important.

The value is not derived from the "quantity" of relevant content provided but by the quality of it.

While farmers' markets have lots of high-quality local produce and a wide variety of choices, supermarkets have the capacity to carry products that will never be found elsewhere.

It is up to you to determine what exactly it is that you are looking for, and to what depth you want to obtain your results.

2) The newsmaster skill is all based around gardening multiple and, somehow overlapping, precise search queries, the ones that—given their tight scope—filter out only truly targeted content on the selected topic. By definition, these very query searches will bring back very little, but very targeted content. It is in fact through the assembly and intelligent mixing of these many and multiple tight filters that one can effectively generate a very tight, high-quality selection of content coming from all the available digital content sources out there.

Discarding general Web-search queries in favor of RSS feeds based on news-only search queries, is like discarding jewels in favor of pebbles because these last ones are easier to find.

Again: you do need to tap into all three content areas.

And you need to understand that, for now, only by going through multiple very tight query and optimized searches and by tapping multiple news sources you can truly extract the cream we are all looking for.

The value is indeed in creating, crafting and optimizing a multitude of very specific search queries that can extract only high quality, very relevant content for your search needs.

Both Yahoo and Google traditional web-based results can bring up breaking new posts and newly emerged online resources that were not there 48 hrs before.

It is the specificity of the query and the number of them that makes all the difference.

Food for thought.

"Why RSS Feeds For Search Engine Results Are Useless."