Monitoring "live" your web site traffic
July, 2001
1. Browserhawk
2. Hitbox
3. Superstats
4. Extreme-DM
If you have a web site, monitoring your online traffic is a crucial and highly insightful activity.
Monitoring traffic on the Web means the ability to analyze the "trails" that each online visitor leaves behind, when she comes to see your web site.
All Internet web servers save systematically on their hard disks a so-called "log file" containing all of the visitors' trails. By utilizing a program that can churn this large amount of data and convert it into readable statistics, you have the embryo of a so-called log-analysis tool.
Evidently, this process takes place after your visitors have come, and you have fully downloaded the log files that your web server or hosting provider have automatically generated.
There is also an alternative approach to monitoring your web traffic which allows for a much simpler setup and for almost real-time tracking of your online visitors. The tools that make this possible are called Live Trackers, and they allow even non-technical users to easily gather Internet traffic information and demographics about their web site.
Live Trackers usually require the installation of a Javascript snippet (short piece of programming code) within the HTML code of the web page you want to be "tracked".
While this is a pretty simple procedure, you do not want to mess with it personally, unless you normally edit HTML code (not with FrontPage, Dreamweaver or Golive).
The advantage of using live trackers versus log analysis tools is ease of setup and use, as well as immediacy of access to traffic statistics.
For example, if I wanted to see now how many visitors have passed today on the isitQ.org web site (I very recent web site project I contributed to) I can almost instantly give you an answer, which includes visitors that have passed by only a few minutes ago.
Try it for yourself and see how difficult is to look at a traffic on a web site. Go to:
http://t.extreme-dm.com/?login=isitqnew
Had I used a log analysis tool, I would have had access probably to much broader sets of data, statistics and reports, covering also very long time sets (6 months, 1 year or more), but only after a few hours. Well the time depends on how many days or months of data you will be looking at, and how many visitors you have actually had, but from experience I can tell you this is not a 10-minute process.
So, you can either opt for ease of use and immediacy, or for delayed response with much greater depth and breadth of data.
I guess a combination of the two would really work best for many web sites, as the need for immediacy of reports can be easily covered by only a few trackers placed on key strategic pages.
The overall traffic analysis, if you are serious about getting anywhere on the Web, must be executed through a log analysis tool. My suggestion is to do a good log analysis every 2-3 months, so you can better identify the trends and the "large picture" of what is happening to your web site.
Use instead live trackers to monitor tactical promotional actions on your part, and for example to verify the peaks of traffic you may receive after a certain announcement, or after you have sent out your newsletter or ezine.
What can Traffic Analysis report do for you?
Having had much exposure to the use of these tools I have become curious with time with the amount of indicators and variables that could be tracked and monitored through the use of these tools.
Besides the expected demographics, which can only report on country of origin and time/date/duration of visit, live online trackers are particularly apt at gathering also "technographics" information.
By technographics, the industry defines the specification of the technology characteristics of each visitor, rather than attempting to know his name, age, date and place of birth. Through the specific use of javascript code, live trackers can in fact detect a great number of detailed information about the system being used by each visitor. Among the variables that can be monitored are the number of colors to which your monitor is set (256, thousands or 16-bit, 24-bit, 32-bit or millions), the resolution to which your screen is set to (VGA 640*480, SVGA 800*600, XGA 1024*768 or higher), the height and width of your browser window, if javascript and java are turned on or off, and if other plug-ins like Flash, Realaudio or Acrobat are already installed.
Many people have asked me specifically if the speed at which the connection is made can be tracked and monitored. The answer is a full yes. There are several solutions to this. For one you can use a log analysis tool like Summary.net, also reviewed in this issue of MasterMind and take advantage of
its specific proxy-based facility that checks exclusively for connection speeds for each one visitor coming to your web site.
Alternatively, you can look into a great server-based application called Browserhawk.
..........................................
Browserhawk
http://www.cyscape.com
= breakthrough tool
"Does your site look best when viewed at a certain screen size, color depth, or font setting? Whatever the requirements, you can easily use BrowserHawk to notify users if their browser is not up to snuff and provide instructions on how to make the changes needed to get the best experience on your site.
BrowserHawk enables you to be proactive by informing visitors of potential problems related to their browser up-front, rather than leaving them staring at error messages or blank pages wondering "What's going on?!". Also included in Browserhawk is our FREE browser troubleshooting script, which automates the collection of detailed browser information from the user and emails the results to your support staff."
(From Browserhawk features web page http://www.cyscape.com/products/ bhawk/features.asp)
This is a uniquely powerful tool used by professional webmasters. Here it is what Browserhawk can do, besides what you read above and beyond (!) what all the other tracking and monitoring tools can:
1) Detection of numerous security settings, such as disabled cookies (both session and persistent), disabled JavaScript, disabled applets, disabled ActiveX controls, disabled or corrupted VBScript, and disabled scripting of Java applets
2) Detection of all types of devices, including standard browsers (both new and old), wireless devices, WAP and i-Mode phones, PDAs and other non-PC based browsers
3) Detection of connection type (LAN vs. modem) and connection speed throughput, measured in bits per second (!!!)
4) Detection of SSL key size (i.e. 40 Vs. 128 bit encryption)
5) Detection of numerous display properties, including screen size, available browser window size, font sizes (!!!)color depth, mouse-over effects, PNG graphics, font smoothing and more
6) Detection of numerous layout properties, including support for frames, tables, images, and colors and images in table cells
7) Detection of installed build of JavaScript, VBScript, and the browser build itself (i.e. IE 5,5,121,1800)
8) Detection of installed Java virtual machine version and vendor, and Microsoft JVM build number
9) Detection of the user's language settings, local time, and time zone difference between you and them
10) Detection of support for advanced client side scripting, such as JavaScript, style sheets, XML, DHTML, layers, frames and more
11) Detection of the user's host name based on IP address (reverse DNS look up) and referring web page
This tool is also particularly good at "sniffing" the visitor's computer/browser settings and at providing an advance notice of incompatibilities where there is a need.
For example, the web site project DAD-IS (http://dad.fao.org), to which I have extensively contributed, is best viewed with Netscape browsers 4.x. This is due to the fact that at the time of development (1998), this was the most popular browser, and that the technical budget run out when we realized we needed extra code to provide full compatibility with all potential visitors to the site.
In such cases, it would be much more elegant, ethical and user-friendly to utilize a tool such as Browserhawk to not only detect if the user is coming to the web site unequipped to best view it, but to also provide her with immediate advice and explanation on what is required and on what she will not be able to see correctly. (!)
Finally, not only can Browserhawk detect such useful technographics and settings but it can also act as a bridge with your server application, instructing it to offer specific web pages according to pre-defined set of rules (e.g.: if the visitor has a Mac with Netscape 3.x display this page, in all other cases display this other one, ...)
You can check out the live demo at:
http://www.cyscape.com/showbrow.asp
or you can consult the Browserhawk Property Reference Guide for all the details on all variables reported through this tool at:
http://www.cyscape.com/docs/ showhelp.asp?topic= About_The_Property_Reference_Guide
To see the three different versions of Browserhawk offered go to: http://www.cyscape.com/products/ bhawk/editions.asp
Prices range from USD $ 99 to USD $ 429 depending on the features you need. (To get all of the statistics I have listed above and more, going well beyond what other live online trackers and log analysis tools can do, you need the highest priced version.)
For all other information relating to ordering and pricing go to: https://www.cyscape.com/products/ order/index.asp?bhcp=1
System requirements: Windows 9x/NT/2000/Me Any scripting or program language that supports the creation of ActiveX/COM objects, such as ASP, ColdFusion, C++, and Visual Basic.
Alternatively, BrowserHawk4J runs on ANY system with the following characteristics: *any* platform, that supports the ability to create JavaBeans. Any scripting or programming language that supports the creation of JavaBeans, such as JavaServerPages (JSP), servlets, Java, and PHP. A Java Virtual Machine version 1.1 or higher
To download a FREE evaluation version go to:
http://www.cyscape.com/products/ bhawk/download.asp
Recommended Resource
--------------------------------------
I invite you to give a look at MasterView, another monthly newsletter I contribute passionately to. It is out always around the 15th of the month and is all about effectively creating and managing PowerPoint presentations for international audiences.
If you are on of those frequent travellers to conferences and seminars, this is going to be a useful resource for you. No advertising and plenty of down right practical advice.
Each issue has a theme and the next one will be on "Presentation Looks" (preceding issues have been on
"Managing presentation file size" and on "Running your presentation on ANY computer").
Go subscribe to it at:
http://masterview.ikonosnewmedia.com
-------------------------------------------
Back to our issue of what Traffic Analysis reports can do for you.
What you have seen is so incredibly beneficial, is the automatic ability to know so much detailed info about your visitors without ever bothering them with questions.
In order to provide some introduction and clarification on what Traffic Analysis can report on, I have decided to list here for you the most relevant "indicators" or traffic statistics that can be actually gathered by using any of the technologies discussed above and below.
This may serve you both as general information, and also as a reference guide in your evaluation of future services and tools.
User-related info that can be gathered:
*Country of origin (not always possible and precise as the user may log in through an international ISP provider) and domain or organization/institution.
*Browser type, version, plugins used, Javascript and Java settings, operating system, type and version, number of screen colors set, and screen resolution.
*Date, time and duration of each page visited on the web site. Path taken through the web site. Entry and exit pages.
*Referral info: from which search engines or directories users come from, what keywords have been typed to find the web site, what other web sites provide a link to your web site that people have clicked on. Which emails people have clicked to get to your web site and in which email program they have been opened.
Secondary indicators
*Loyalty
This secondary indicator measures how many of your web site visitors are coming back vs. first time users coming for the first time.
*Circulation
This indicator says how frequently your loyal web site visitors come back to your site.
*Depth of visit
How much time did visitors spend on the web site and how many pages did they view while there?
Credibility and trust issue
Traffic reports, can be in my opinion a greatly untapped credibility resource. By allowing access to the public to key statistics of your web site traffic you can only gain extra credibility and trust from your own visitors.
Possibly this may not be a recommended solution for new web sites, or for web sites which have yet to get their act together.
To make your life easier, I am reviewing here for you my top three selections on the live online tracker camp. In the next MasterMind issue (#10) I will do the same with the log analysis tools.
Selected Online Live Trackers:
a) Hitbox
b) Superstats
c) Extreme-DM
..........................................
Hitbox
http://www.hitbox.com/
= interesting, promising
Hitbox is a capable and very reliable live tracker which utilizes a simple javascript piece of code to keep track of your selected web pages.
Hitbox offers extensive reporting features including the ability to provide detailed statistics on most of the above listed traffic analysis indicators.
One of the pro elements of this service, even in its free version, is the archiving of all the statistics collected for the complete life of the account. Few services offer this valuable feature, because they keep only summary data of previous weeks and months statistics.
Among its unique reports and statistics is one that forecasts the number of visitors on the web site according to the historic trend tracked so far.
The paid service, which does not require to display a Hitbox counter, extends the number of reports and number of pages that can be tracked (5000) on your web site. Its price starts at USD $ 23.95 per month.
The graphic display of the statistics and the organization of their reports is the best among the three services reviewed here.
Unfortunately, the complexity of most of the report pages and the extensive use of javascripts in the interface of the Hitbox reports make this service a slow mover especially if you are not based in the US and if you do not have a good connection (33.6 or better).
The information design of the reports is well executed and allows for easy switching of views. You can easily see trends through the appropriate use of bar charts.
In the Pro version you have a number of extra reports as well as more sophisticated graph charts. This allows for an even more comprehensive view of the traffic as well as clear indications of all trends. In the Pro version in fact, nice trend-line graphs are superimposed on bar charts showing visually rich traffic plots.
HitBox Professional provides more useful statistics and features, including SSL page tracking, content grouping, and new statistics such as Visits, Cookies, and Most Requested Content. With HitBox Professional, you can group site pages together that contain similar types of content, such as download pages, product pages, corporate information pages, and so on. The new Most Requested Content statistic lists the most frequently viewed content types of your Web site, helping you track the popularity of your site content.
The charts also display information dynamically. As one passes the mouse over the individual graph elements, more detailed information is displayed. Each element is also clickable, allowing the "analyst" to dig deeper into any point of information at will.
You can also plot custom charts by specifying different indicators to be compared against each other over a set period of time. I find this a very useful feature, as charts can also be plotted against world data gathered by the Hitbox service across the over 200,000 web sites utilizing this system.
If you want to get a good idea of much of the terminology utilized in the traffic monitoring industry and see an exact description of each one of the statistical reports generated by the Pro version, go check out this link:
http://hitboxprofessional.com/cgi-bin/ pro.cgi?pro&productinfo&productinfo_faqs
It contains the Installation Support manual of Hitbox Pro, but believe me, it is a great resource to study what traffic monitoring is in general. Recommended.
*PROS*:
In the FREE version, up to 100 pages can be tracked.
For global organizations, interested in seeing visually the amount of traffic from the different parts of the world, the Time Zones feature allows for a good view of a world map with a superimposed bar chart showing traffic by time zone.
By virtue of the operating and browser system settings of each visitor, the preferred language of each one can be "sniffed", offering a much truer portrait of which languages should be addressed to best serve your audience(s).
Hitbox tracks visitors' locations, how long they stay, and whether they are unique or repeat visitors.
*CONS*:
The reports pages in the free version display banner ads in multiple locations.
The javascript interface is not fully compatible with Mac browsers and therefore I do not advise Mac users to utilize this service.
Report pages are heavy too load, and unless you have a good internet connection, they may take too much time. The extensive use of javascript to create pop-up submenus, plus the use of multiple ad banners on each, creates a significant performance penalty for those pages. (This does not happen, of course, on the paid service. It is to be said that pages remain however heavy to load because of their extensive use of javascript code and a very high load of customer traffic on the Hitbox servers).
The javascript code to be added to your HTML pages adds about 5KB of code to them. This negatively affects the performance and display time of your web page by a number of seconds, depending on the speed of the visitor's connection, distance and connection reliability.
The free will display ads on your free tracked pages, including some online casinos ads.
See a full free version of HitBox working online at:
http://get.hitbox.com/content/ demo_frame.html
See a full working version of the Professional version at:
http://get.hitbox.com/pro/content/ prodemo.html
..........................................
Superstats
http://www.mycomputer.com
= worth knowing
Among the several online services offered by Mycomputer.com, Superstats is one of the best and most complete online live tracker service.
Superstats offers a basic free service just like Hitbox and Extreme-DM, and a great array of statistical reports.
SuperStats is available in 3 service levels:
Standard, priced from $9.95 per month for sites with up to 10,000 monitored monthly page views, includes 4 reports and offers basic page view and visitor reporting.
Professional, priced from $29.95 per month for sites with up to 20,000 monitored monthly page views, includes over 100 reports and offers all features of Standard plus visitor profile reports and site path reporting.
Premium, priced from $99.95 per month for sites with up to 50,000 monitored monthly page views, includes over 500 reports and offers all features of Professional plus ecommerce, HTML analysis, and server monitoring.
Half a million hits a month will cost you $279.95.
Working on an analysis method very similar to that of HitBox Pro, SuperStats requires you to insert code on the Web pages you wish to monitor. This code, in turn, reports site activity back to the SuperStats server for future analysis.
Nonetheless several shortcomings, Superstats is one of the most popular live trackers available today. To see why, review this list of the many real time reports and graphs available through SuperStats Professional and Premium:
Site Traffic Reports: Page Views by Hour, Page Views by Day, Page Views by Month, Page Views by Year, Hourly Unique Visitors, Daily Unique Visitors, Monthly Unique Visitors, Yearly Unique Visitors, and Most Popular Pages.
Marketing Reports: Referrers, Most Active Organizations, Domains, Search Engines, Search Keywords, Time Spent on Page, Time Spent on Site, Return Frequency, Daily Return Visitors, and Return Visitors.
Visitor Profile Reports: Browsers, Browser Height, Browser Width, Operating Systems, Monitor Color Depths, Monitor Resolutions, Countries, Languages, Netscape Plug-Ins, Java, JavaScript, JavaScript Version, Cookies, Visitor Detail, and Last 100 Visitors.
Site Path Reports: Site Paths, Entry Page, Exit Page, Page Reloads, Previous Page, Next Page, Page Depth, Visit Depth, Single Access Pages, Site Path by Visit Number, and Page Depth by Visit Number.
SuperStats monitors activity in real time, making your reports available to you in a password-protected account.
The Premium version includes site analysis and e-commerce reports including:
Customer acquisition Customer detail
HTML validation Link Check Site Popularity Browser Compatibility
To see demo versions of the Premium version go to:
http://www.superstats.com/ demo_premium.html?mycSession =hTkzfOk%3Dl-5-%2522% 2522_& mycSession_sig=2098891861
To see demo versions of the Professional version go to:
http://www.superstats.com/ demo_professional.html? mycSession=hTkzfOk%3Dl-5-% 2522%2522_&mycSession_sig =2098891861
*PROS*
Can export any report into either .doc (MS Word), comma delimited, .xsl (Excel) or even Acrobat PDF format!
Can access data of previous months in detail.
*CONS*
Reports are not easy to consult and navigate through.
It is difficult to find the reports one is looking for.
Sparse use of graphics and charts.
SuperStats does not provide the large variety of reports offered by Hitbox Pro, nor reports offering much in-depth analysis.
Heavy menu javascripting make Superstats report pages heavy to load and slowly performing.
It is not self-evident when you are looking at daily versus monthly or yearly data.
SuperStats does not track visitors' locations, how long they stay, and whether they are unique or repeat visitors.
HitBox Pro can do all of those functions.
Go see a full demonstration version of the FREE Superstats service at:
http://www.superstats.com/ demo_standard.html? mycSession=hTkzfOk%3Dl-5-% 2522%2522_&mycSession_sig =2098891861
..........................................
Extreme-DM
http://www.extreme-dm.com
= must have
Extreme-DM is the simplest and most basic of the services reviewed here. Still, it makes for my favorite choice after all elements have been taken into account.
Its best characteristic is its simplicity and ease of use. This makes it also a speedy performer when you are accessing your report pages.
While it does not provide any of the advanced features found on the more expensive and competing services, it does provide all of the basic information a serious webmaster or marketer would need.
Here is the list of statistics reports it can provide:
Summary
Totals and Averages
o Basic Tracking 1
Unique Visitors:
- Days
- Weeks
- Months
- Hours of the day
- Days of the Week
o Basic Tracking 2
Incl., Excl. Reloads:
- Days
- Weeks
- Months
o Geo Tracking
Domains
Countries
Continents
o System Tracking
Browsers
JavaScript Enabled
Operating Systems
Screen Resolutions
Screen Colors
o Referrer Tracking 1
Last 20
Last 20 from Email
Last 20 from Searchengines
Last 20 Queries
Last 20 from Usenet
Last 20 from Harddisk
o Referrer Tracking 2 Totals by Source: - Website -
Searchengine - Email - Usenet - Harddisk Totals by Searchengine: - 24 most popular engines All Keywords All
Website Referrers
The most missed feature is the ability to see visitors paths through the site, which I deem a very valuable information.
This feature is available on the free Hitbox service and on the paid version of Superstats (Superstats Professional).
Extreme-DM provides clear reports on both Search Engine referrals as well as on the keywords utilized by your visitors to find your web site.
It also tracks and compares referrals from web sites, email messages and from any hard disk files.
On the good side Extreme-DM also keeps in its own archive all of the keywords and the referring web sites that have ever been recorded.
*PROS*:
Interface is very simple and straightforward to use.
Performance is great. Compared with the other live trackers services, this is by far the speediest and most responsive one.
Javascript code to insert in the HTML code of your pages is highly optimized.
Basic summary reports on daily, weekly and monthly visitors, plus all technographics, referrer and keyword info you may want to see.
It provides IP blocking allowing you the option of excluding your own IP address (and maybe your webmaster's too), to avoid influencing traffic results with the home team traffic work on the web site.
*CONS*:
Limited reporting detail.
No detailed history of user's demo and technographics.
Customer support has good margins of improvement. (If you loose the email in which they confirm to you your access username and password, you can pray them in any position, but they will not give it back to you!)
To see a real working example of Extreme-DM at work, go to:
http://www.extreme-dm.com/tracking/ sample/nph-summary.cgi?tag=example
blog comments powered by Disqus