Feb 12, 2011 at 6:29pm ET by Vanessa Fox
Today, the New York Times published an article about a search engine optimization investigation of J.C. Penney. Perplexed by how well jcpenney.com did in unpaid (organic) search results for practically everything the retailer sold, they asked someone familiar with the world of search engine optimization (SEO) to look into it a bit more. The investigation found that thousands of seemingly unrelated web sites (many that seemed to contain only links) were linking to the J.C. Penney web site. And most of those links had really descriptive anchor text. It was almost like someone had arranged for all of those links in order to get better rankings in Google.
The New Yorks Times presented their findings to Google. Googler Matt Cutts, head of webspam, confirmed that the tactics violated the Google webmaster guidelines and shortly after, the J.C. Penney web site was nowhere to found for the queries they had previously ranked number one for. Matt tweeted that “Google’s algorithms had started to work; manual action also taken”.
J.C. Penney, when contacted by the New York Times, said that they didn’t know anything about the links and promptly fired their SEO firm, SearchDex.
So where did J.C. Penney go wrong? Why did they do it? What have they lost? And how do they get it back? Read on to learn more and make sure this doesn’t happen to you.
“Link Schemes” and the Google Webmaster Guidelines
The web is big. Like more than a trillion pages big. When a searcher types a couple of words into the search box, Google has to be able to sort through all the pages on the web and show the searcher the ones that are the most relevant to the search, as well as the most useful. That’s a tough challenge. Google uses hundreds of signals to figure this out in an automated way. Historically, the link graph has factored fairly heavily into these relevance and value algorithms.
Google was launched on a foundation of PageRank: the idea that people link to things they like and find valuable, so a page with a lot of links to it is probably more useful than a page without very many links. How people link comes into play too. If a bunch of links to a page use the anchor text “watch the latest episode of Glee online”, then it’s reasonable to assume that the page being linked to has a video of the TV show Glee. (Note: Google has evolved well beyond this simplistic explanation as the web itself has evolved.)
Over time, as site owners realized how valuable it was to rank well in Google search results, some began hatching “link schemes”. Rather than just hope people find our content valuable enough to link to and raise awareness of the content through traditional marketing techniques, how about we just make a deal and agree to link to each other? We both get links with the anchor text of queries we want to rank for and everyone’s happy!
Everyone, that is, except Google. And searchers. Because these types of back room deals break the PageRank algorithm, which was based on a link equating to someone finding the content valuable. With a link exchange, the link simply equates to a deal being made. Less useful results could rise to the top, causing the search results to be of lower quality.
Hence, the Google webmaster guidelines, which at their core say this: our algorithms are meant to surface the best possible results to searchers. If you try to manipulate those algorithms, we might lower your site’s rankings or take your site out of our index entirely.
Matt Cutts manages a very large team dedicated to finding violations to the guidelines and then refining the algorithms to catch them in a wide swath as well as manually remove sites or lower their rankings.
What if you want a bunch of links to get to the top of the rankings quickly but you don’t want to link back (maybe because you think exchanging links would be too obvious to Google and you’d get caught or because you don’t want your site to lose credibility with visitors by linking to a bunch of random, irrelevant sites)? Why not just buy links? Note that buying links for PageRank purposes is very different from buying online advertising. Advertising links generally include code that cause them to be seen differently by search engines so that they’re not counted like editorial links would be. As you might imagine, Google doesn’t like links purchased for PageRank manipulation any more than they like link exchanges.
You can read more about both of these kinds of links directly from Google:
Why Do Organizations Purposely Violate the Guidelines?
If all of this artificial linking is against the Google webmaster guidelines and could get a site removed from the index, wouldn’t sites that rely on unpaid search traffic be very careful that they adhere to the guidelines? You would think. And most do.
But some companies think they can outsmart Google. They think they won’t get caught. They see that it works for competitors. They try it and it works for them too. So they say think well, the guidelines say not to do it, but it works! I’m getting more traffic and sales than ever! Why would I stop?
In other cases (as seems to be the case with J.C. Penney), a company hires the wrong search engine optimization firm — one that that engage in tactics that violate the guidelines (and guidelines involving linking are only a small part). Firms that implement these types of tactics want to show the client results quickly. Most of them think they won’t get caught. A few of the more unscrupulous ones don’t care. They figure they’ll collect their money and move on. If the site in question gets burned later, that won’t impact the SEO firm.
High search rankings for targeted queries can be very valuable to companies. We in Search Engine Land talked to the New York Times reporter who wrote this story while he was researching just how valuable a number one ranking really is. It’s impossible to pin that down exactly, but it’s certainly the case that most of us use search engines for product research and many of us make those purchases online. A recent PEW Internet study found that 88% of online Americans who make over $75,000 a year conduct online product research and 81% purchase products and services online. Google released some data just before the holidays that indicated that nearly 90% of consumers research online. Many of them then buy online, whereas others then buy at a store.
Undoubtedly, J.C. Penney found that ranking well for so many queries over the holiday season helped them with both online and offline sales. A spokesperson downplayed this, saying that just seven percent of traffic to the site comes from unpaid search results. But between online sales and online research that drove buyers into the stores, it’s likely all of this visibility was indeed valuable.
The trouble is that generally, these tactics don’t work forever. And if you’re basing your business on them, you’re building on a shaky foundation that could cause things to come crashing down at any moment.
What Happened with J.C. Penney?
Doug Pierce, who worked with New York Times to uncover what was happening with J.C. Penney (and wrote about his experience) found that the site had a LOT of links pointing to it. From peculiar sites. With very descriptive anchor text. You can see this for yourself using free online tools. Take, for instance, a search for [comforter sets]. Bing still ranks J.C. Penney as the second unpaid result:
Yahoo Site Explorer shows that 774 pages link to that URL:
Music teams? Piano players for hire? Car modification? Why would those sites link to a J.C. Penney page listing comforter sets for sale? Open Site Explorer shows that the anchor text from these random pages is amazingly descriptive:
The likeliest answer here is that someone (J.C. Penney possibly suspects their SEO firm, since they quickly fired them) bought into a paid link network. (J.C. Penney said they would work to get these links removed and indeed many of them are already.)
Branko Rihtman ran some of the URLs through Majestic SEO‘s link reporting tool and found that it appears that links were acquired in two bursts that coincided with the holiday seasons at the end of 2009 and 2010. (This graph aligns with the statement from Matt Cutts that the recent paid links campaign seems to have been running for the last three to four months.) Below are trends of new links (from pages and domains) to that comforter sets page just before the 2009 and 2010 holiday seasons.
What Is the Impact?
Doug Pierce said it was “the most ambitious attempt [of link spamming he’s] ever heard of”. Sadly, it’s not the most ambitious attempt I’ve ever heard of. When I worked at Google managing Webmaster Central (where the webmaster guidelines reside), I saw this kind of thing all of the time. And I saw the impact it had first hand. Now that I’m no longer at Google, I regularly field emails and phone calls from companies, large and small, panicked because they’ve lost their major source of revenue due to lost rankings in Google. More than one company has told me they’d have to close down entirely if they weren’t able to get their traffic from Google back (and a site can’t always get its rankings back). I showed the following slide at a keynote I gave recently, showing the impact it can have on a site’s traffic to violate the guidelines:
When I caution companies against tactics that violate the guidelines, they sometimes say that I’m being a goody goody or as an ex-Googler simply have strong allegiance to Google. Some tell me that they have an obligation to use all of the tools available to them to gain an audience and revenue. But the truth is just that I’ve seen too many analytics reports with traffic down and to the right. And I feel my obligation is to help companies build sustainable audiences and revenue.
Building a long term search strategy that adheres to the search engine guidelines may mean that it takes longer before you start seeing traffic from search engines, but that traffic is not at risking of drying up at any moment.
What If This Happens to You?
The best scenario is not to go down the path of manipulating search engine algorithms, but instead focus on solid search acquisition principles (building a site search engines can access, creating content your audience wants, raising awareness of that content…). Take care when hiring a search engine optimization firm. You want a firm who will work with you to improve your site and content. If the firm says you don’t need to be involved or won’t give you specifics on how they work, be wary. (Google has some tips on hiring an SEO firm.)
But it still can be tough. SearchDex devotes an entire page of their site to their adherence to the webmaster guidelines, including a statement that “SearchDex does not support link scheme architectures”. (To be clear, I have no knowledge on whether or not they were involved with the placement of the links.)
But if you find that you’ve lost your search traffic and that a likely cause is a violation of the webmaster guidelines, the first thing you should do is fix everything. Review your site and make sure nothing suspicious remains. Then log into Google Webmaster Tools and file a reconsideration request. (Note, if you have a Webmaster Tools account, Google may let you know when they’ve found violations on your site and give you a chance to fix them.) In the request form, explain everything that happened and how you fixed it. Google will manually review the situation and if they find that everything is cleaned up, they may let you back in.
What’s Going to Happen to jcpenney.com?
They’ve started cleaning up the links so as long as no other problems remain (Doug Pierce, who helped the New York Times with the investigation noted at least two other issues that will need to be fixed), the following will likely happen:
- Any lowered rankings due to algorithmic factors will likely continue as these algorithms find these kinds of links and ensuring they aren’t sending value signals. Matt Cutts told the New York Times that this change impacted how Google trusts links.
- Any manual actions that have lowered their rankings will likely be manually be removed once everything is cleaned up.
The New York Times article points out that Google has been public about action they’ve taken due to violations guidelines before. In 2006, Matt Cutts posted on his blog about removal of bmw.de. That site was removed entirely from Google, but once BMW fixed things, they were let back in.
Does this mean that jcpenney.com will start ranking number one for all of those searches again? Probably not. For instance, the New York Times article notes that the site was number one for [samsonsite carry on luggage] and has dropped to 78. It was number one in part due to all of those links. It’s 78 due to Google’s action. Its “real” ranking is somewhere in between and that’s likely where it will end up once the smoke clears.