Social Networks Meet News Aggregation And Filtering: Information Routing Groups
Social networks meet news aggregation and filtering: social collaborative newsmastering is all around us. But someone got an early view on it just before the first personal computers started to get around us.
As early as 1980 Dave Andrews, an independent writer had started visualizing and researching a number of concepts and ideas that are about to become realities as I write now, twenty-six years after his first writing them.
Mr. Andrews' Information Routing Groups or IRGs are spontaneously emerging small social networks capable of intelligent ly filtering, selecting and aggregating the most relevant news and information for their community goals and interests.
News map credit: Newsmap is an application that visually reflects the constantly changing landscape of the Google News news aggregator. A treemap visualization algorithm helps display the enormous amount of information gathered by the aggregator. - Blog integration by Widgetbox
His vision and ideas are well contrasted with the status quo of information delivery and distribution determined by mass media in the last century. In his views even the unfortunate ecological consequences humanity is facing are sad outcomes of inefficient information distribution imposed by mass media peculiar characteristics. In his view, to promote a more participatory society, it is important to promote participatory media and to challenge, replace and eventually abandon mass media.
In order to better understand today's mass media inherent lack of democracy, it is useful to imagine a communication system that allows and fosters participation by everyone. David Andrews did this with his concept of "information routing groups" or IRGs. His discussion predated the vast expansion of computer networks and is worth outlining again in its original form and with added detail (here is some valuable past coverage of Dave Andrews fascinating views thanks to the work of Prof. Brian Martin:
"The Power Of Open Participatory Media And Why Mass Media Must Be Abandoned" and in "Why Mass Media Are Bad: Weaknesses And Limitations Of Commercial Mainstream Media".
Read this: Dave Andrews imagined a computer network in which everyone is linked to several interest groups, with each group having anywhere from perhaps half a dozen up to several hundred people. An interest group might deal with anything from growing apples to racism. Each time a person made a contribution on a topic, whether a short comment, a picture or a substantial piece of writing, that contribution became accessible to everyone in the group and a person receiving a message could, if it so wished, post it to other groups to which he/she belonged to. Andrews called each of these groups an IRG (information routing group).
In a network of IRGs, everyone can be a writer and publisher at the same time.
But there are no guaranteed mass audiences. If a contribution is really important or exciting to those who receive it, they are more likely to post it to other groups. In this way, a piece of writing could end up being read by thousands or even millions of people.
Which is exactly what is happening today thanks to the spontaneously emerging information routing groups created by delicious, Furl, Digg, Slashdot and the thousands of others, large and small intelligent social filtering and news aggregating networks. That's what they essentially are.
So here I again, and thanks to his own, personal invitation, I have the pleasure to host Dave Andrews original writings on the founding principles of IRGs:
An IRG is one of a semi-infinite set of interlocking and overlapping groups containing individuals who use software and email to automatically mediate and exchange information via lateral communication. Due to the principle of six degrees of separation, a specific message is highly likely to meet any relevant but unknown target by the process of lateral diffusion.
Photo credit: Taro Taylor
IRGs were a concept developed in the book "The IRG Solution. Hierarchic Incompetence and How to Overcome It" by David Andrews- Published by Souvenir Press, London 1984
It began as an article published in The Guardian circa 1980 in which the author promoted the idea of "interlock research". This was based on the notion that the looming environmental and other catastrophes, social, medical, health, and organisations and policy generally not working efficiently, were caused in large part by the failure for active professionals within and across different organisations; or even within the same organisation to communicate at a lateral level, leading to unintended or unseen interactions and consequences.
The article noted how enormous amounts of energy and raw material could be saved, and pollution avoided by interlinking various industrial processes. He claimed most professionals and policy makers were uninterested in these opportunities because they were unaware of them.
Photo credit: Nathan Kendall
A prime example was - and still is - the astonishing and not widely appreciated fact that the UK wastes heat from power stations, equal to and able to replace the entire usage of natural gas for heating, which is an amazing example of what he termed hierarchic incompetence.
The book advanced the notion of the relevance paradox, whereby professionals, or groups of professionals might be unaware of certain inevitable but unintended consequences -, for example a civil engineer, might cause an increase in Bilharzia infection for locals as a result of irrigation schemes not having simple low cost counter measures built in. The book claimed that this relevance paradox can and does apply to entire professional groups and individuals in numerous ways.
The article propounded the need for and a mechanism to create a methodical scheme whereby such lateral self-constructing communications networks could develop and pro-actively inform professionals (engineers, scientists, politicians, health workers) of what it is they needed to know but weren't aware of.
These schemes would enable professionals to ask questions and get answers more rapidly, but often they might be told, unbidden and out of the blue, not only that they were asking the wrong questions but also to supply the correct answers.
Examples cited included the NASA engineers who, having spent a fortune on unsuccessfully developing the complex sliding and articulating inside knee joint needed for space suits, eventually went to the tower of London and ruefully copied the armour of Henry VIII with just such a joint stating "we wish we had known about this earlier!"
Following the article, Souvenir Press of London published the ideas in the book.
Dave Andrews radical ideas, have already in part well occured - the notion of computers in the majority of Western homes, all linked by modem has been - while others, and in particular the one of information routing groups maybe taking shape under our eyes.
Here the basic ideas on which information routing groups are based on:
1. Summary Of Core Ideas From The IRG Solution
- The notion of Central Media (CM)
These are television, newspapers, professional journals etc. - which the book claimed were inherently unable to adequately inform, due their need to simplify complex issues, and only carry issues of particular interest to the majority of the readership.
Specialist journals were also Central Media under this definition and suffered from the same defect of interesting only the specialists so neither media promotes the essential lateral communication and understanding between interacting specialisms, of actual but ignored / unperceived real world links and interactions, which are now coming home to bite us. Another defining feature of CM is that they are essentially monologue, notwithstanding the letters pages and other token feedback mechanisms.
- The notion of Lateral Media (LM)
This would be -gossip groups for example at the informal end of the spectrum, - and at the other end of the spectrum Lateral Access Networks or as later termed Information Routing Groups or IRGs. These would be formalised attempts to improve the lateral communication process in a structured way.
Another feature of Lateral Media is that they are essentially open to a series of dialogues - not just sending out information but often questioning it. Of course now the prime example of a lateral media is quintessentially the Internet.
- The Notion Of Only Six Degrees Of Separation
The book cited this now well known phenomena of how any two randomly chosen people in the West are connected by at most six inter personal contacts or friends of friends. Thus in principal, information, useful ideas, questions, or answers could be propagated from participant to participant and thus span the world with on average only six "friends of friends" or "colleagues of colleagues" in between. The book used this idea as part of the functioning of the IRG concept.
- Tacit Knowledge
This is the idea propounded by Harry Roberts of Bath University, whereby much essential knowledge, whilst it is capable of being learnt and passed on, cannot frequently be specifically articulated, and often the possessors of are unaware of it.
A simple example is riding a bike - few people actually realise that when you turn right on a bike the first thing you do is actually turn left - but this does not stop them learning how to do it, and few people will actually believe it if they are told.
Another example is how to do and use algebra - the rules can be easily articulated and memorised ("Multiply both sides by the same term, take away the same term from each side of the equation ,"etc.) but exactly how to use algebra cannot be reduced to merely a set of explicit rules - it can only be learnt by performing examples and with guidance over time students get the idea. If you don't know algebra, what exactly is an equation?
Or for example we are told that X is the unknown, but is that different when X is written x, does it vary if it is Wednesday or Tuesday, (sometimes it can) can A or B be unknown in the same way, why X not P? Does it matter if X is written in chalk or printed and so on? There are an infinite numbers of such question and issues which a novice would be confused by but which can eventually understood. Simply learning the rules of algebra will not enable someone to do algebra in a meaningful way.
Roberts went on to show in some detail how a particular laser was designed in America and the idea, with specific assistance, was gradually propagated to various other universities world wide. Even when specific instructions were sent, other labs failed to replicate the laser, it only being made to work in each case following a visit to or from the originating lab. It became clear that the originators, while they clearly could make the laser work, they did not know exactly what it was they were doing to make it work, and so could not articulate or specify it...
Photo credit: Taco Meeuwsen
Other examples are the Bessemer steel converter: Bessemer sold a patent and was sued by the purchasers who couldn't get it to work. In the end Bessemer set up his own steel company which became one of the largest in the world.
This may seem a simple idea but the implications are huge and far reaching - all sort of enormous technological and social mistakes have occurred because people pushing certain policies - technical or social have lacked the vital tacit knowledge - either leading to misguided and therefore doomed policies, or misguided implementation of a good policy so still leading to failure.
For example it is clear that politicians, civil servants and managers in general, do not really understand how computers can and can't be successfully applied to problems and which problems are not amenable to computerised solutions.
Also how people can and can't be successfully organised - consider the examples of the old British local court paper based maintenance orders system compared to the fiasco of the centralised and computerised, but now disbanded Child Support Agency, or the disaster which was initially the centralised Driver and Vehicle Licensing Authority only saved by having local checking centres.
Equally in WW2 it was found that having radar plots going directly onto the WAAFS plotting room often gave unsatisfactory results. As a result the Filter Room was introduced where experienced fighter pilot officers were installed to screen the incoming radar plots, which vastly improved operational efficiency - these experienced officers had the necessary tacit knowledge which could not be adequately conveyed to the WAAFS, their controllers, or put into a manual.
The book claimed that Central Media could not transmit tacit knowledge but that Lateral Media could, or at least flag up the lack of and need for.
- Paradigms and World Views
The book also pointed out that professions, professional groups, organisations, political parties even entire cultures tended to get stuck into world views that were effectively reinforced by central media, and that these world views were almost impossible to unlock by the individual action of enlightened individuals.
These persistent world views often had damaging, counter productive and counter intuitive long term consequences, particularly in the environmental or social sphere, where sooner or later the ignored and unknown consequences would nevertheless make themselves felt in unpleasant ways. The book claimed that lateral media of the type envisaged could help overcome these difficulties.
- Hierarchical Incompetence
A key feature of large hierarchic organizations is the simplification of issues and the loss of tacit knowledge about issues as they ascend the hierarchy and the tendency for lateral communication across the various departments, fiefdoms, etc to be stifled either actively by management, or by self imposed isolation.
Conversely, the hierarchies only easily deliver simple messages which are cascaded down and these often tend to be inappropriate or counter productive to the goals of the hierarchy and open to inadvertent mis-interpretation as they arrive at the organisational "sharp end", where reality is inevitably for much more complex than could be modelled at the peak of the hierarchy.
Often unofficial actions based on local tacit knowledge and lateral communications often compensate for these inevitable communications and conceptual failures, no matter how well intentioned the original policies have started out.
Hitler's' Third Reich was famous for bitter inter-departmental rivalries, personal vendettas and lack of co-operation actively fostered by Adolf Hitler himself, but these communications failures may have cost him the war. For example, German fighters should have been fitted with long range fuel tanks to enable them to protect the Luftwaffe bombers for several hours over England, rather than the 20 or so minutes of fuel they in fact had.
But bureaucratic inefficiency / hierarchical incompetence meant the long range fuel tank programme never went ahead with consequences for the world history that could have been enormous had the Luftwaffe won the Battle of Britain.
On the other hand, the Leigh Light, a special searchlight which was significant in defeating the WW2, U-boat menace came about as the result of a personnel officer in the RAF designing and fitting the first light entirely unofficially - if he had tried that in the Luftwaffe he would probably have been shot.
Lateral communication should therefore be encouraged throughout and across organisations in the interests of efficiency and email and all the resources of the Internet are one means of doing this.
However, in order to prevent information overload, Information Routing Groups, systems to automatically encourage, mediate, organise, monitor and control these lateral messages to prevent participants being overloaded, and at the same time help and assist them in their prime professional tasks, can assist by guiding important information in particular tacit knowledge to them.
Thus Lateral Media and IRGs are an essential aid to the hierarchic function of any large organisation and society, defeating the relevance paradox, and helping to minimise unforeseen and undesirable outcomes.
2. Information Routing Groups: how would they be structured and function?
Photo credit: Ahmed Zahid
The idea was that every person could be in several IRGS which each might contain 10 - 100 people. Generally participants might be in a local one - such as neighbours, a social one - friends, and one or more professional ones.
A local IRG for neighbours might share lawn mowers and get help moving heavy objects, a social one for friends might share social engagements, a professional IRG might share professional ideas, but they would all to a greater or lesser degree overlap.
Each member would provide a different description or profile depending on the type of IRG which could vary from being only a name, to key words built up by the software noting the user's actions and interests, or a profile specifically written by them.
IRGists would begin by passing information to each other - this information could be a joke, a quote, an interesting fact, a picture, and an entire article from central media, a scientific paper, a comment or a question.
Software on each of those particular IRGs members' computers would automatically receive and log the information exchanged. Each user could then choose to delete unread, read, forward, comment on, or rate the information, or do nothing. The user (or indeed his software) could do no more than simply read it and pass it to other IRGists in the same IRG or other IRGs the recipient was in but the sender was not.
The IRG software would then automatically rate it by various parameters - was it passed on, had it been read, had he received other similar articles in the past etc., and how were they rated by previous IRGists?
Thus over time the IRG software would build up a profile of what articles the IRGists was currently interested in, who he like getting information from, who he trusted / did not trust etc. Fellow IRGists would meanwhile be doing the same to his postings.
The IRGists would each set a personal daily ration of information, and the IRG software would then deliver that amount. The software could be set so they would always get certain information with a points score above a certain threshold (i.e. stuff they were very interested in) but below that only up to the daily specified quota - so this would automatically limit and reduce "SPAM".
In a way this is similar to an extent to the way we already use newspaper editors, and indeed our choice of newspapers and journals to limit and specify the amount and kind of information we receive.
However in the IRG case, not only do we get information we want, we would also get information we don't actively want until we get it, relevant but whose need was not realised until it arrived - thus is the relevance paradox resolved - "I thought you might like to know this", " You really should be aware of this". Furthermore, participation would inevitably encourage interaction beyond normal organisational boundaries.
Of course not all messages need to be broadcast to all IRGists - they could be small local discussions going on all the time between two or more IRGists, and occasionally the results could be broadcast.
That was the basic IRG model and even at that level it can be seen how the 6 degrees of separation can be mobilised and the relevance paradox defeated - articles or queries can be diffused in a rapid way to almost any person in the world, a bit like a disease, as long as they are in several IRGs without swamping uninterested individuals with information they are not interested in.
Also note you are just as likely to get technically relevant information from your local gardening or social work IRG as you are from your professional engineering or health IRG. This phenomena can be seen at work within the Internet, where current jokes sweep around the Internet at phenomenal speed.
3. Further Development of the IRG concept
Photo credit: Ahmed Zahid
The IRG Solution foresaw that every participant would build up a library of documents, email conversations, and chit chat embodying all kinds of technical, social, philosophic, musical, artistic and humorous information. In general this would be in a semi-public area of the user's computer.
It foresaw the IRG software cataloguing, organising and collating this information, and enabling its use as a resource to guide other IRG seekers to it, without any specific effort from the user, although this could be input as well, if so desired.
These guidances could come from lateral referral from other IRGists - so a seeker's IRG software might send out a software probe saying - "I am interested in how to build irrigation schemes in Egypt", and this would be passed on, perhaps largely unread by his immediate IRGS, and after six transits might end up in the files of such a civil engineering expert.
The originator can then delve away in the engineers files. Questions can be asked - maybe actual personal contact will ensue. At that point the expert might say to the enquirer "by the way have you made due allowance for the Bilharzia issue" "The Bilharzia issue - what is that?". Having met, mutually beneficial exchanges might occur, which may naturally lead to consultancy fees being paid.
Large IRGs would be set up as a result of invitation, some being very exclusive, other not. Because all members would be constantly rating each other - either explicitly or using automatic means, then members would be on best behaviour.
Some IRGs would have high social status and be sought after, and one could only enter after an informal apprenticeship in lower level IRGs and recommendation - but this whole process would be informal and self organising, self vetting and self accreditation as is any social network.
Thus credibility and trust would be based on the recommendations implicit in the IRGs that they inhabit - a key part of The IRG proposal was the need to know and trust who it was you were getting information from.
At the time these notions were extremely difficult to explain and promote. Many people said, "Well why would people want a computer in their house? Why on earth would they want to connect them up?" - this was before the PC and internet revolution. The book had to use many mundane and complex examples and justifications to try and get people interested and explain why everyone else would want to do it, but with little success.
Nowadays it is very easy to explain because people are fully familiar with computer networking email and the Internet. However, the basic idea of IRGs has not at all been fully implemented, although the Internet has gone a long way towards what was envisaged by e.g. Internet, email, bulletin boards, eBay, personal web sites, Wikipedia etc.
But these don't give all the benefits foreseen for IRGs. For example at a mundane level, spam, unwanted pornography, scams all extremely annoying and dangerous information would be avoided by IRGs because of the nature of the interpersonal trust networks built up and mediated automatically, mapping each members needs and values.
Google, whilst a fantastic innovation, only works if you know in advance what you are looking for, i.e. the relevance paradox is often not tackled: you frequently have to know the exact words, date, name, place of what it is you are looking for, and usually you only get told what it is you think you want to know.
Also Google does not readily harness or provide tacit knowledge - this really needs a human interaction, or at least the embodiment of that person's mind and experience, which can happen in the way he chooses to organise or let his software organise his information. Tacit knowledge can be captured in his email exchanges and these, or those that he wants to can be open to selected scrutiny.
The concept envisaged that access would be restricted in various ways to fellow IRGists or only particular IRGs. Again this could all be automatically mediated by software.
It was also envisaged that eventually an entire economy would develop where people could not only charge for any consultancy services that arose, but also charge a fee for allowing them access to perusing their information. Furthermore, a transit fee could be charged for every successful probe that crossed their IRG space.
This mixture of economic and social activity would be drivers for good behaviour, since reputation would be important - as it is in existing social networks - again this is lacking in many areas of the Internet. In other words, experts, with well organised IRGs, and lots of knowledge could earn substantial income by the very natures of what they know, and who they know.
The book saw almost anyone as a potential contributor. For example living in Brighton, means apart from anything else, that a prospective house buyer might ask you and pay you for your views on a particularly area they where might be contemplating a house purchase - is it a good area, are the schools good, etc. A bricklayer might have family in Ireland or Ethiopia and can give good unbiased advice on a holiday there or the climate.
Thus the book saw the development of an IRG economy as not only a means for people to effortlessly earn money, based on their unique attributes and experience as a driver, but also a means for entire groups, organisations and cultures to optimise their modus operandi, and avoid many spectacular disaster which have in part come about over time through our ignorance of certain essential facts about the world we actually live in, the people we are and the ways our organisations work and don't work, and which central media and bureaucratic hierarchies alone are intrinsically unable to address.
About the author
David Andrews works in the area of Energy Conservation and he is currently Energy Manager of Wessex Water controlling a £20m energy budget. Following the three years work at the Open University Energy Research Group, David wrote the book "The IRG Solution. Hierarchic Incompetence and How to Overcome It" based on the experience he gained.
Read more about IRG - Information Groups here:
- The Power Of Open Participatory Media And Why Mass Media Must Be Abandoned"
- Why Mass Media Are Bad: Weaknesses And Limitations Of Commercial Mainstream Media
blog comments powered by Disqus