I have been recently shopping for e-learning and content management tools and technologies that could satisfy the needs of large international organizations in handling the development of a topic-specific knowledgebase as well as the gradual development of distance training resources for their staff.

As the e-learning market is growing at phenomenal rates, and as numerous new companies and products have become recently available, it becomes always more difficult to identify what works and fits your requirements from what could be too expensive, too-complex or too difficult to implement.

Worst than the above is the fact that the e-learning market is experiencing a very high number of deployment failures.

Content that cannot be uploaded to the content management system it is meant to work in, modules that do not integrate as promised, bugs and enormous technical difficulties have embarrasses the major players in the e-learning market including Saba, Smartforce, Click2learn, and many others.

Criteria for Evaluating e-Learning, LMS/LCMS Vendors or Partners

Most present discussions of e-learning vendors focus on product functions, and technological facilities, a mistake.

Organizations should first evaluate any technology vendor or partner (e.g., e-learning, LMS or systems integration vendor) based on their business model and actual position in the industry.

A slick product doesn't deliver value if the company cannot deploy it, support it, or will not be around tomorrow. In this light I also encourage you to evaluate vendors through a set of very specific criteria.

These should be:

1) longevity,
2) customer base,
3) parent-companies,
4) partners,
5) technological sophistication,
6) financial credibility, customer
7) engagement protocol,
8) deployment options and protocols,
9) deployment record,
10) industry experience,
11) conformance with industry standards.

You can find a very thorough reference for such evaluation criteria by going to:
http://www.ltnavigator.com/selection.htm

The above resource "distills analyses of the technology context, knowledge commerce systems (KCS), and evolution of learning technologies into a set of criteria for evaluating vendors.
These criteria capture basic business and economic considerations, as well as issues pertaining to the learning technology space."

Deployment failures are the most serious problem facing the e-learning and distance training industry.

Many of the evaluation criteria for selecting learning technologies vendors or suppliers (such as the ones listed above) offer information about the vendor's ability to avoid this problem, but you should focus increasing attention at the vendor's customer engagement and implementation policies and workflow.

Caveat Emptor

The principle of "caveat emptor" definitely applies to buying learning technologies. Morgan Stanley estimates that U.S. companies wasted $130 billion on unneeded software and other technologies over the last two years.

Estimated IT project failure rates range from 50 to 75 percent (down from 80+ percent) - projects are never completed or do not function properly. Failure rates in eLearning installations mirror those in the larger IT community owing to the technological limitations of most vendors.

The problem is easy to understand. Saba and Docent did not exist 5+ years ago and SmartForce was a content company (CBT Group) 3+ years ago. They may understand learning technologies but their IT experience does not extend to the diversity of runtime platforms they encounter in corporations. Saba and SmartForce have earned reputations for producing deployment failures (e.g., content does not work with the LMS or customer cannot register students) in a minimum of one out of every two installations - (N.B.:no data exists to support this accusation.)

Conclusions

a) Organizations and institutions striving to start implementing learning technologies for the delivery of their courses cannot assume that prominent vendors have the systems integration experience to install software successfully in-house, behind firewalls.

b) They should also not assume that sales representatives will accurately represent their company's technological capabilities or installation record.

c) Finally organizations and institutions purchasing learning technologies contribute to the problem by choosing unwisely. They go into the market saying, "I'm looking for an LMS or LCMS" and their decision hinges on product functions, a big mistake.

Credits

I am thankful to David A. Baucus, Ph.D. for his personal advice and contribution to the above. He is a valuable industry analyst and consultant in knowledge commerce and learning technologies. Mr. Baucus has voluntarily offered his expertise and know-how to me and my colleagues through a set of e-mails, once he has read a quest of ours on the ASTD forums online.

If you are a large organization I would recommend hearing his views on this matters. You can join the Yahoogroup elearning leaders to discuss and exchange further on these topics with him and other qualified learning technologies specialists:

Subscribe to this discussion list by sending an email to:
elearningleaders-subscribe@yahoogroups.com
or by accessing:
http://groups.yahoo.com/group/ elearningleaders/

Some relevant resources

a) JISC TechWatch Report on Content Management Systems:
...............................................
http://www.jisc.ac.uk/techwatch/reports/ index.html
http://www.jisc.ac.uk/techwatch/reports/ tsw_01-02.pdf

b) Content Management Systems (CMS) Directory - Free systems
"No Start-up Cost" Solutions
...............................................
http://www.clueful.com.au/cgi-bin/ cmsdirectory/browse/Products: Free%20systems

c) e-Learning Course Authoring Tools
...............................................
http://www.e-learningcentre.co.uk/eclipse/ vendors/authoring.htm

Recently updated find here authoring software tools (21) useful for the production of formal e-learning solutions, i.e. online courses (or WBTs) rather than presentations or other informational solutions.

I have found this to be a well selected set of the best authoring and learning content management systems available on-line. Recommended when you are initially looking at which tools or systems to consider.

These tools are to be considered one generation ahead of the WebCTs, BlackBoard and similar systems that have made the history of the first decade of e-learning.

 
d) Learning Technologies Vendors
...............................................
http://www.ltnavigator.com/learning_cmps.htm

Find here vendors that embrace a broader definition of the learning tech space and partner with firms outside the industry. They integrate content and knowledge management functions into their software. They partner with relationship management vendors to include cutting edge data analytic tools (i.e., a precursor to profiling functions) in their products. Indeed, the very best firms in this industry use the term "learning infrastructure software and services" (Learning IS2)™ to describe the value they deliver to knowledge commerce systems.