Yes, I sang victory way too early. Nonetheless I had hoped to have come out of the terrible traffic loss that started last Friday, I instead discovered that Google has fully penalized Master New Media and dropped all of my articles out of Google search results.
Photo credit: Andres Rodriguez and Robin Good
I don't know if you have read my article of this past Monday, where I recounted the stress and apparent resolution of a major problem affecting the flow of web visitors to this very site. Starting last Friday in fact, all of the stats reporting to me the number of visitors to Master New Media crumbled to 1/4 or more of their normal level. Visitors went from about 15,000/day to 3-4,000/day. Even worst was the deeper slash in daily revenues which saw an analogous drop.
As I wrote in that article, this can prove to be one of the most difficult situations that an independent publisher may find itself in. Not only, but the bigger you have become, especially in terms of collaborators and staff, without the help of external financiers or other partners, the larger and more difficult to manage the consequent problems.
Fact is, that while on Monday I posted an apparently victorious article explaining the trouble I had run into and what I thought at the time to have been its cause and resolution, reality has settled in again with quite a different face.
The new discoveries I made afterwards with Drazen, my old-time trusted and untiring webmaster, left me really without words, as our whole micro publishing enterprise had fallen into a situation we had never planned for.
For some reason, yet unknown to us, Google has actually penalized all of my web pages, to the point that they could not be found anymore for any of the terms for which they were so highly ranked until a few days ago. This is not only disaster from a standard web publisher viewpoint, but it is triple tragedy because the very visitors coming from Google, and not out of their own bookmarks or of a friend recommending them, are the very ones that make my site sustainable.
Master New Media is based on a contextual advertising business model, in which text-based ads carrying advertising info for products and services that relate to the very content appearing in our articles. The moment you eliminate the majority of the visitors coming from search engines, you are wiping out your greatest potential to monetize your content via advertising. Read on and I will explain you why.
By looking at this often not immediately visible but very critical aspects of an independent publisher life, you can learn a lot more about the obstacles and challenges awaiting you ahead on this path.
This is why, like many other times before, I have chosen to fully share my experiences, errors and lost battles. To further help and support my very mission: helping the small guy grow smarter, bigger and more sustainable (that is = independent).
How? By being a new media explorer and an online experimenter ahead of those wanting to walk this same path, as well as by being an honest reporter who shares and reports his discoveries and errors all along the way.
This is why I do it.
Look at my traffic stats now:
Unique daily visitors for MasterNewMedia for August 2007 - stats by HitboxProfessional
Let me take you then where I left off this past Monday.
After three days of relatively zero traffic, from last Friday to Monday morning, and a million checks we did in every direction, I realized that we needed to look somewhere else. This is when I first remembered of my Google Webmaster Tools account and discovered that Google itself was reporting errors for my sites that I was not aware about.
The errors were in fact mostly relate with the format and syntax of the sitemap file one needs to periodically resubmit to Google, to get maximum indexing coverage from its crawlers.
I and Drazen thought we had found the culprit and that's when I wrote Sudden Traffic Loss? Google Doesn't Send You Anymore Visitors? Here Is What To Check, fully convinced that having cleared that things would have gone back to normal.
Was I wrong!
What I had not paid attention to was a notice inside GWT that clearly said that while the errors it had reported for Master New Media did limit Google ability to fully index our site, these errors would have not impacted our Google search results or rank. In fact nonetheless visitors were still a small fraction of what we normally see, all of Master New Media pages were properly indexed by Google and the official PageRank was also firmly set at 7.
The thing I stupidly had completely missed over, showing how much I give for granted what granted is not, is that yes Master New Media was still fully indexed and ranked, but if you searched for any of the hundreds of keywords for which we ranked in the first page of Google search results, our pages were nowhere to be seen anymore. You can try it yourself. Go to Google and search for "independent publishing" for which Master New Media has ranked first for a very long time now (several years). Master New Media is nowhere to be found in the first ten pages of results!
What does that mean?
Google, for some reason, has penalized Master New Media in the most significant way possible: it has dropped our pages from their search results.
Why?
Well, this is not something as easy to answer, but obviously I have spent most of my time yesterday trying to figure out this very issue and here it is what I have found out.
As you probably already know, Google will not come to you or send you an email saying "Hey Robin, look there are some things not Ok with your site, can you kindly fix them or we will have to ban you.". If their system senses and reports specific issues on a site, it looks like they just go ahead and penalize it. I don't know whether this is automatic or humanly-handled but fact is that you get no notifications or warning that this is about to happen.
Therefore the first and foremost lesson I have already learned from this is that, at least from now on, you and I need to be way more proactive in making sure that our sites are fully compliant with Google requirements and guidelines.
One other realization I made is that it is likely that Google keeps changing and refining its own approach and policies to improve the quality of its results and in this effort, it appears that it may have become more restrictive and intolerant of issues and deviations from the official standards that it didn't bother about before. I say this because Master New Media has never done any active tweaking of its web pages to rank higher on Google search results nor has tried to trick Google or any other search engine to better exploit ranking positions and visibility. Our strengths on this front were all organic and due to lots of good in-depth writing work.
But back to the main question: what could I have done to trigger Google into censoring all of my content pages?
We spent a lot of time looking at this, asking people and by reading comments and answers in the Google Webmaster Help forums.
In the end, the items we found that were clearly in contrast with Google guidelines were the following:
1) Hidden text - I had an H1 title on the home page of MasterNewMedia English edition as well as on the home pages of the other international editions (IT, ES, PT).
2) Hidden text - I had since the longest time a unique 5-digit numerical code hidden in every article page I publish. It is a kind of unique ID we have manually created for all of our article pages so that we can more easily reference them inside our content management system. We have had this for years, and it is obviously not something to trick anyone. But if Google monitors and penalizes pages via a program that does this automatically I doubt that it may be able to tell "good" from "bad" invisible text. So, the only solution here is not to try to explain to Google the why but rather to just avoid using any kind of solution that may jeopardize your compliance with their guidelines.
3) Non-compliant Text Links - I did have several text link ads appearing on many pages of Master New Media as well as on some of my other sites. These are those small text only ads often appearing at bottom of web pages or in other not very visible locations. (As I will write a whole article tomorrow about this very issue I reserve for that the going into more details about the ins and outs of these strange type of ads.) Google does not prohibit the use of "paid for" text link ads but it requires that you as a site publisher clearly label these ads as such and that a "no-follow" tag is assigned to them so that they cannot pass "Google juice" (read PageRank) to other sites. I was clearly not in compliance with these guidelines, as some did not display a clear detectable text label, nor they were all assigned a "no-follow" tag.
Without hesitation I decided to correct or take down all of these self-identified problems, no matter what the cost or loss in further revenue, and to immediately utilize Google official request form to be re-instated fully in its search results. While I have no guarantee that I did spot what Google didn't like and that they will re-instate all of my pages in their search results, this is my best effort to do so, and without further indications from Google itself, it is all I in fact can do.
There is also a lot more to say about text links, the universe that surrounds them, Google view and my own ethics, but as I have mentioned, I want to devote to a full article to this, as I think it is subject prey of lots of supposed-expert advice but for which I have had personally lots of difficulties in finding answers I could truly trust.
Obviously you should read the popular articles by Google's own Matt Cutts on this, as well as other relevant reports, but in my experience, they are not enough.
And while I don't know yet whether text links were the true case of my Google penalization, I have certainly learned lots about them and I think I have some very valuable insight to share with you about whether you should consider them your friend or foe.
Having cleared, corrected and dropped all of the above deviations from the Google official guidelines from Master New Media, I then went on to submitting a formal request to Google for having my pages fully re-instated in their search engine results.
As I have mentioned there is a dedicated form in your Google Webmaster Tools account that serves just this purpose.
In theory there is nothing else I can do right now. Just sit back and wait until a Google revisor sees my request, evaluates it and puts the site back in the search results.
According to what I have read in forums yesterday this can easily take several weeks if not months.
And this is where my heart crumbles. Without Google visitors, my revenues are 1/4 or less than our average monthly gross income. Without them I can't pay my office space, editors, writers, webmaster or server space, let alone my own monthly expenses and the taxes I am due at the end of the month.
It is a tough situation.
What will I do?
I don't know yet.
I consider still the situation in flux and will continue do so at least until the end of this week. I expect to discover new things as I go along and to hopefully learn from others interesting options on how to get out of this situation fast.
For now, I am doing my best to make sure that everything that could be possibly done to guarantee full Google compliance is done.
I have also already alerted most of my staff in a meeting yesterday, about how bad the situation is. Most of them have reacted with strong signals of support and desire to contribute even without being paid. "We have married your cause Robin," one said, "and we will not let you down now that you need us most.". That was an injection of love, thank you.
But evidently not everyone can be a martyr in this situation, and others, like me, will need to find a way to be able to pay their bills. So while I was greatly lifted by the positive moral support received I felt great respect for those who may need to take on other roads to compensate for this unexpected situation.
The next hours and the rest of the week are going to be devoted to brainstorm further the situation I am in, while evaluating all possible roads I have available.
If it was not for my smaller sister sites Kolabora and MasterViews, my final revenue at the end of August would be next to useless, but thanks to these two additional sites I publish I hope to be able to at least pay my rent and food until things do get better.
And obviously, this is so assuming these other sites also don't get banned or penalized by Google. I am not trying to be paranoid. Fact is that there is a number of posts and articles out there that suggests in fact, that if you are linked by sites that are banned or penalized then your site as well may be penalized itself. While the logic of this is all to be understood, I link first and foremost my own other sites! Therefore it goes without even thinking that all of my sites may risk being penalized if I don't do everything I can to put myself in safe waters.
That is why our next step is to take every single one of our sites and to systematically check and clear out any item that may further trigger Google sensitive probes.
Notwithstanding the apparent disaster I am in, and which I have no idea how long it is going to last, I have a natural drive to turn negative events into opportunities.
In this respect I am glad to have fallen prey, before others, to this situation, strongly hoping that my story and example could help many others avoid doing my same mistakes.
As my mission is one of helping small, online independent publishers find their own way, I am grateful, nonetheless the pain this carries, of the unique opportunity I have to share some of the true difficulties and risks that this new profession has in store.
If you have gone yourself through such a situation, and wish to share, even anonymously, your experience and lessons learned, please feel free to do so by either using the comments section here below or by writing to me directly at Robin dot Good at masternewmedia dot org.
I don't know whether Master New Media will survive this jolt, and if and how it will come out of it. But of one thing you can be certain: until someone comes here and locks my keyboard I am going to keep sharing what I learn and discover each single day.
Recent related resources:
Originally written by Robin Good for Master New Media and entitled "Penalized by Google: Master New Media Is Out Of Google Search Results"
