Sudden Traffic Loss? Google Doesn't Send You Anymore Visitors? Here Is What To Check

A few days ago I was to wake up to a pretty strange surprise. All of the traffic coming to MasterNewMedia from Google had suddenly disappeared. From over 14,000 visitors to 4,000 and less is a pretty big change from one day to the next. All the worst when you make your living on it, as slashing your daily revenue by three is certainly going to cause you some trouble.

Traffic stats showing unique visitors for the first days of August 2007 for MasterNewMedia as recorded by Hitbox Professional

But I have had similar situations before, mainly due to comment spam, to reporting bugs with your analytics service, or to issues on the server like search engine spiders overly exploiting my web site resources to update their indexes.

But after having done all the checks and verifications I knew this time it was something different.

I could tell because my webmaster Drazen had noticed that while traffic from search engine Google had dropped drastically from one hour to the next, visitors coming from their bookmarks or by simply typing the name of this site hadn't changed at all.

Google itself, for some unknown reason to me, was not sending traffic anymore to my main web site. Why?

God, I said to myself, if this lasts more than a few days I am in serious, deep shit.

How could I make my online publishing business survive if Google changes were permanent and irreversible?

As over 80% of my revenue is derived from Google AdSense and its ability to convert web surfers into potential customers to one of Google many text-based advertisers messages, with no traffic coming from the Google search engine anymore, I would need to re-evaluate completely my content and monetization strategy. But how?

As I first stated jotting some ideas about how I would have needed to revolutionize my content strategy to survive this nuclear impact, a sudden thought about what could have been causing all of my traffic troubles went to the Google Dance.

Traffic stats showing page views - the yellow line shows the average reference traffic level for MasterNewMedia - stat graphs by Hitbox Professional

If you have never heard about the Google Dance, and you are an online publisher, you should stop and learn a bit more about this key periodical event that may affect your web site as well.

Basically, the Google Dance is a process lasting several days and taking place on Google multiple distributed data-centers around the world. During this process the Google indexes containing all of the world web sites, their links and contents are updated and refreshed in a complete way. If your web site happens to be part of the re-shuffling of ranking that takes place during this "dance", your site may lose temporarily its PageRank only to regain it a few days later. Losing PageRank may mean falling from page one of the Google search results page to page ten or more. The Google Dance happens typically every three months but there is no official news from Google announcing when this actually takes place.

Therefore, as soon as all of the technical checks on the server and on the network confirmed multiple times that there was nothing out of ordinary happening on our machines or on relevant Internet pipes I started thinking that, nonetheless, this had never happened in over seven years of publishing on this web site, maybe the day had come for the magic Google Dance to get MasterNewMedia to do a little rock and roll.

The thing is, you really have no easy way to tell whether you are victim of the Google Dance, outside of following some not too scientific advice available online and which advises you to go and check periodically several Google data-centers to verify whether their indexing records for your site vary from server to server, possibly indicating, in that case, that your site is participating in the Google Dance.

Outside of that, it would appear as all you can do is to sit down and wait for the typical duration of a Google Dance (supposedly three to five days) and to cross fingers that things go back like they were traffic disaster striked.

And so, you may wait for three, four, five days while your traffic is splatted to near-zero numbers and with it most of your daily revenues.

But, as I just discovered for myself, that is sitting back and waiting, may not be your very best strategy.

Waiting when you are in emergency situation is not always the best option to take. And in fact, this morning, the fourth day in a row that the Google search hose dripped near zero visitors to MasterNewMedia that I decided to look somewhere else I had not been looking at before.

Yesterday, as I was scratching my head around the issue, I consciously directed myself to Webmaster World forums, one of the Internet oldest and most popular public forums discussing webmaster related issues and therefore also problems very similar to the ones I was victim of. At WWF I read several times people referring to GWT as something they had either used or verified before arriving at some conclusion relating to the cause of their web site problems, but to me that acronym meant nothing.

As it often happens in these situations, I didn't give much weight to it and proceeded searching for other comments and reports from individuals having situations similar to mine without any significant results. I only further convinced myself that I was probably a Google Dance victim and that the only thing I could do was to be patient and wait.

And so I did. But only for a few hours more. As my life and the one of several other loved ones here and around the world depends on my online publishing business making a sufficient amount of revenues, you really can't sit back and not think about this when it all comes down to hundreds of dollars per day that suddenly disappear.

In fact this morning at 7am, the first thing that comes to mind as I move my feet out of the cozy sheets of my bed is: Is it over? Are my visitors back?

But unfortunately nothing.

My traffic stat trend line is a flat curve with no sign of life. I don't even check my earnings for the previous day as seeing those numbers would only make me feel more pain.

But... wait a minute!

My own, "internal Google", had taken good advantage over the night sleep, and to my advantage flashed into my internal visual brain a search result I had put on query since yesterday: GWT = Google Webmaster Tools, something that I had indeed seen before.

Several idea lights suddenly switched on in my head as I headed off to Google Webmaster Tools, an open and free resource for all online publishers wanting to check and verify whether there are any issues or technical problems with their sites and Google's ability to properly index it.

Unfortunately, you can't go to GWT and do an immediate check on your site yourself. GWT requires you to add a tag or an HTML file with a specific name to your web site home page to make sure you are the effective owner/webmaster of the site to be verified. Armed with patience and time I uploaded the file Google had specified and got immediate access to my site stats as seen by the Google crawler.

Blam. The error causing all of my trouble was sitting right there in front of me, waiting for action.

Shit, shit, shit... I screamed at me repeatedly.

Google had identified it, dissected it, reported it and suggested how to fix it. It was just waiting for me there.

A bad formed URL of an article was causing the Google spider to stop its crawling process and that caused Google to go in a standby mode with our web site and to basically withdraw our content from its search results until the issue had been fixed.

What did I need more? Nothing. Only to blame myself for not having systematically checked this tool and for having sat back too long thinking this was just another Google Dance.

Conclusions

What I can recommend everyone who is managing one or more web sites who heavily depend on Google's traffic and AdSense-generated revenues is to make a point to systematically use Google Webmaster Tools if not to find a way to automate this process (do you know of any way to do this?) so that it can be carried out thoroughly on a daily basis.

Losing traffic is bad, but losing traffic and not knowing why is suicidal. Google Webmaster Tools has proved in this case to be a strategically valuable tool in helping me spot a critical technical issue which was blocking all Google search traffic from finding my site.

The Google Webmaster Tool is also the key facility to periodically submit your "sitemaps" to Google greatly facilitating its job of fully indexing your relevant content in the fastest and most reliable way possible.

N.B.: One curious thing I noticed is that the example XML sample sitemap file which Google provides with a link to the W3C validator, doesn't validate even as originally prepared by Google. I don't say to blame to Google but to save you time in preparing and troubleshooting your sitemaps to Google. The only true relevant validator is Google itself! Do not loose time trying to have the W3C validator approve it. As long as Google likes it you are going to be happy man.

Originally written by Robin Good for Master New Media and titled: "Sudden Traffic Loss? Google Doesn't Send You Anymore Visitors? Here Is What To Check"