Google Killing Keyword Tracking

Google Killing Keyword Tracking

Google Killing Keyword Tracking

If you’ve ever done “keyword research” for your websites, you know it’s a minefield. Unless you have a Ph.D. in keywordology, you’ll quickly be bamboozled by comparing “visitor volume”, “seasonal trends”, “exact versus phrase match” and all the other related variables that go into finding “good” keywords to target and write articles for.

One of the easiest ways to shortcut the entire process is to search your webserver logs for the keywords that people actually USE when searching Google (or other search engines) before visiting your site. I even created a product called “Niche Treasure Hunter” around this keyword research topic. All you have to do is trawl your weblogs for these “Bonus” keywords. You can do it manually, or use software to do it automatically. I do a bit of both, with Google Analytics doing the automation for me.

When you discover a keyword that someone used to find and visit your site, it’s a great feeling. If it’s a keyword you wrote an article around, it’s proof that your SEO is working. If it’s a keyword you DIDN’T write an article around, that’s even better because it shows that Google trusts your site enough to send you traffic for a Golden keyword you didn’t actually optimise for… and now you can write an article around it to get more visits!

For example, say your main keyword is “toy electric train”, but, one day, you notice someone visited your site for the keyword, “electric model train”. When you check at Google, you’re only on page two for that keyword. How hard would it be to get more visitors for “electric model train”, when you know that, without trying, you’ve generated traffic for it from page two of Google? Pretty easy, right? That’s why I call them bonus keywords!

But now Google is starting to hide the data containing the keywords from webmasters. When people do a Google search while logged in to their Google account, the search is done at a secure page, an https request. The results of the secure search are show to the Google user as normal, and the click on the chosen search result takes them to the website they want to visit… BUT the search keyword is NOT passed to the destination web-server.

It’s estimated that about 10-12% of searches done at Google are secure ones, which doesn’t seem like a problem, until you consider what would happen if Google switched ALL searches to the secure system. All referral keyword information would be hidden resulting in webmasters losing “Bonus” keyword data, third party analytics software being unable to track “keywords to sales” and advertising platforms that display ads based on the incoming keyword searches would stop working.

So why would Google make the switch to secure searches? Well, it would be an obvious way to let everyone know (including lawmakers and regulators) that Google cares about “user privacy”. Not only cares, but is actively switching people to secure searches. Who could argue that making the switch would be a good thing? Of course, the fact that it kills “bonus keywords” and “keyword to sale tracking” for webmasters as well as certain advertising methods would be just coincidental. Interesting, Adwords ads still reveals the search query to the advertisers! I guess Google is assuming that, 1: people can tell the ads from the organic results and, 2: are happy to give up a measure of their privacy to advertisers that they’re not happy to give up to the websites appearing in the organic results. That’s quite a big assumption…

Of course, Google says that webmasters need not fear! If you use Google’s Webmaster Tools (GWT) you can, “receive an aggregated list of the top 1,000 search queries that drove traffic to their site for each of the past 30 days”. Woohoo. I don’t know about anyone else, but I don’t like GWT at all. I’d much rather do my own digging through my weblogs or via Analytics than trust the “aggregate list” from GWT.

What do you think? Is this a deliberate ploy by Google to move webmasters to Adwords, or just a privacy -related action ahead of a regulatory requirement? Will it affect keyword tracking? Will it affect you as a user of Google, or as a webmaster?

[comments are now closed. When the "spam to genuine" ratio exceeds 50:1 it's time to close the comments!]

About Neil_Shearing

I've been an Internet Marketer since creating and selling my first ebook in 1997. As a former scientist I like to test money-making ideas, then simplify and share them. Apart from this blog, I can also be found on many social media sites. Click here for a list.
This entry was posted in Search Engine Optimisation. Bookmark the permalink.

49 Responses to Google Killing Keyword Tracking

  1. Steven says:

    Neil …
    It seems like almost everyday there is more dread from the search engines. However there are other methods for doing keyword research (thank goodness) … the bottom line is we have to figure some of this stuff out on our own. Become students of observation.

    Thanks for the insight…

  2. Tom says:

    Wow! That is the second most horrible thing I have seen today. I use those secret gems almost every day to write new webpages for my site. I have found that they are absolutelly invalualbe… the first worse thing that happened to me today is that I have been doing a 301 blanket permanent transfer on my website and someone told me that since any url no matter what it is will forward.. its possible that google might ban me for cloaking.. or have a page when one does not exist.. but I have no choice.. I cannot 301 all 50k pages… woe is me what is a soul to do..

  3. Thanks for bringing this to my attention Neil. Damn shame as I haven’t even started touching the surface of driving traffic to any of my blogs and now it seems there’s little point anyway if Google aren’t going to make this information available to us via my weblogs and Cpanel stats.

    Have you got any ideas on how to combat this or make the best of a bad situation? Answers on a postcard! (or this blog)

    • Hi Gareth,

      To combat this specific move by Google… no. It’s up to them if they want to pass the data or not. I do have ideas for being less reliant on Google though. :)

      Neil.

  4. Dennis says:

    More and more is Google attempting to isolate starting webmasters from the rest.
    Unless you are well experienced on SEO, with a few yrs of webmastering under your belt, you dont stand a change. To bad because I think the value of the displayed results be more focused on paying advertisers, then real content.

  5. Mike Liebner says:

    If it were up to Google they’d have us all use Adwords and buy advertising and NEVER try to get their golden traffic for free! It’s amazing how quick they’ve gotten out of touch with the people like us with a rebellious spirit from which they originally benefited from when Google came out of nowhere and started riding the big wave of internet search.

  6. Really good post Neil thanks for sharing this. I am really concerned about this move working with SMB businesses where money is tight optimising for organic results is a preferred option. I cannot help feeling that this is geared around pushing more people to adword spending particularly as they are pushing this adword express for local advertising.

    Would be interested to see what other people think?

    Tracy

  7. Ken says:

    Good heads-up on the Big G.

    Yup – this could really bite.
    I think they are definitely trying to make it harder to “reverse engineer” them.
    e.g.- Demise of the Wonderwheel

  8. Mark Montgomery says:

    Hi Neil!

    Thanks for the heads up on googles tactics. My site for local business leads relies heavily on these golden nuggets of bonus keywords. It is one reason why my clients get so many leads. My keyword advertising and optimization gets them results. You can’t blame google for wanting to drive more business to themselves even under the guise of privacy regulation. How much money do they really nead anyway? LOL! Thanks again for the information.

  9. I freely admit it…I’m confused.

    In what way is letting me know the keywords people use to find my site going to tell me any private information about them? I never saw anything that said “SaRita Hartin from IP XX.XXX.XX.XX searched for curling rules” in my analytics.

    Intentionally or not, so many of the big G’s changes seem to be punitive – to everybody. As a consumer, I would be thrilled to know people are optimizing for exactly what I’m looking for instead of a bunch of useless ads or directories that waste my time.

    In answer to your question, I definitely think it’s an effort to move us to invest more heavily in Adwords because I don’t see what privacy is protected by hiding this data.

    Or maybe I just missed something?

  10. David says:

    Hi Neil,

    I think we will just see this sort of thing continue. It is going to get harder and harder.
    The answer is legitimate businesses and branding. More and more scammers are going to be weeded out. I’ll leave the definition of what a scammer is to each person that reads this. But I know what mine is and it is a pretty narrow definition.

    Google is going to support brands from this point forward.

  11. Lee says:

    Google’s Adwords is nothing more than advertising. I think that Google has become so popular that it can now afford to screw people. While Google is used as a generic verb for web searches, which really is the crown jewels of marketing and very annoying, it will always be at the forefront of many neitzens’ minds and goal for SEO. Considering that, if Google can forcefully migrate website owners away from free optimisation strategies to merely bidding for page placement for Google’s gain without much resistance they will. Social networks are also a breeding ground for antisocial advertising. These companies do need to generate an income from somewhere, but I believe that people are getting despondent at the sheer level of advertising that is thrust into their faces. Google’s Adwords is advertising not search results.

  12. Bill Harzer says:

    I wouldn’t worry too much about their “hiding” of referring keywords. Honestly, it’s really only a small minute percentage of actual search traffic to your site–and people have other options to hide referrers. You can do that right in your web browser, for example, using various add-ons and plugins to hide your referrers.

    If you’re really worried about the 2 percent of traffic that you’re not getting the referral keyword for, then you actually need to start looking elsewhere for those keywords.

  13. Trevor says:

    Hi Neil, I would say do not be afraid of Google my auto blogs are still working a treat. I will even say they have done better since Google Panda. So who is scared of the next change? not me they need content even if its only a Tag the show up great!

    Thanks
    Trev
    Bye the way, the people working at Google also use water to cook :)

  14. I suspect someone at Google has forgotten that what makes the web so useful is to be able to see where you are making your money in detail – unlike magazines/tv/radio.. did I say TV… crikey maybe they don’t want to cannibalise the potential of their TV advertising business…?

  15. Dave Lucas says:

    I’ve been blogging for so long that I’m ready for anything!

  16. Guys

    I am going to weigh in on google’s side for fro a number of reasons, in the UK and Europe especially with the cookie privacy laws coming in referral cookies and tracking cookies are going to be something that will have to be sanitised, part of this can be done using a browser but the software makers need to sign up to this.

    The other way is to enable SSL although I can see that keywords will be more of an issue there is an overiding issue of the level of data that can be extracted from profile info which is not encrypted by SSL and can provide information to expose a user by their keyword research and tracking cookies.

    I have seen google profiles unencrypted that showed a substantial amount of information, although the profile was anonymised the level of information was eye opening. This will also put search companies in positions of Data Protection breaches for not only UK and EU as well as other countries. To illistrate this point there was a case of Yahoo search results which were anonymised and published. Some enterprising researcher looked at the data and built out a profile to the point that he was able to personally identify someone with search info and tracking cookies. Using SSL will help protect this info and prevent search companies from facing federal and other national or area wide prosecution. With the amount of personal info being exposed using google searches and by script kiddie groups this step is inevitable.

    Does this put pressure on webmasters? Yes but it means we have to be smarter, that is why we are in this business, the lazy and easy way out people fade out the strong find new ways to work to their advantage. I am not the eternal optimist just a pragmatist with how online and personal info exposure has put pressure on everyone.

    People said the same thing about speed liimits and seatbelts but they do save lives and are for our benefit and everyone is used to this now.

    The open trust that used to exist online has been abused and now these are steps that are being taken to to prevent a larger issue or be put in a financially puntive situation due to regulators wading in.

    • I would agree, if Google passed the keyword info to SSL pages, which was the case in the past. Now they strip out the keyword information regardless of whether they’re passing data to a secure or non-secure website.

      Neil.

  17. John Edgley says:

    I think the entire issue of G being able to do whatever they want is worrying, however whilst I used to believe that they championed “small” sites (with value) I think that courtesy of article spinning, doorway pages, and keyword “optimisation” etc etc they are now unable to distinguish and discriminate sufficiently based on content alone.

    Having recognised that IM’ers are amongst the main culprits in the use (abuse?) of “statistics”, keywords included, in pursuit of sales, they actively wish to at least discourage them if not outright penalise them for it. Since they are, by definition, commercial sites they can be differentiated and singled out by their SSL connections, whereas pure “info” sites have no need for such.

    In short, if you’re in business then no free traffic for you my boy (or girl) – use adwords and pay for it!

  18. Titus says:

    Great Post Neil,

    Can’t believe I am still making a full time living at search engine marketing, this year Google has thrown up more obstacles than they did in past 10 years put together. If they go with 100% “keyword deadlocking” – it will be a major blow for free organic search… some of my most profitable keywords have come from my daily traffic logs. But we all forget Google is a business, their goal is to make money… not make the world a better place. A publicly traded company like Google, can’t be concerned about such silly notions for too long… they have their shareholders to worry about now. And that search data is pure gold…

  19. George says:

    I thought Google was supposed to “Do No Harm”! I think everyone should join forces and start our own search engine… I have a good domain: (Greenscious.com – which means being Green Conscious)… which if you think about it, everything is heading towards being Green Conscious!.

    I’m willing to share and develop this domain so we all (non-corporates) can create our own search engine – as a community…

    What do you think?

  20. Paul says:

    I am beginning to believe Google does not like Internet Marketing Folks. Particularly those who do not advertise on adwords. Your post has now solidified my opinion.

    Paul

  21. Rose says:

    I’m getting sick and tired of researching good keywords, writing decent content, and then seeing forums at the top of the search results for the keywords. I reckon in the next few years we will all need to have ‘authority sites’ to rank anywhere on Google.

  22. Brian Garvin says:

    Hey Neil,

    I’ve been doing SEO for about 4 years now and wasn’t aware of this change until today, so thanks for letting me know. What I’ve been doing lately with blogs is writing about 5 articles each day based on useful long-tails I found around a particular niche. Then I’ve been submitting each article to 5 of the top article directories (not ezine articles though) that pass page rank and even post panda it’s been working well for me. The original article goes on the blog, then the same article goes to these directories. One author bio link points to the section page of the blog and the other to the actual blog post. I get about 40 links per day doing this and have lots of inner links pointing all over the blog.

    Even post panda it’s been working wonders, my keyword list that people find my blogs under grows pretty fast. In a month or so I can get a blog up and get found on about 800 keywords or so. I think Steve Weber is doing basically the same thing but a little different. I still think in the end there will be hurdles but Google still loves lots of original content.

    For the bonus keywords like you mentioned Neil, I’ve been getting those from C-Panel and have been writing articles around them, and often times (about 50% of the time) I get double listings for that keyword.

    This doesn’t work as fast as paid traffic, but it’s still predictable and I don’t think it’s too late to build a solid content business.

    Brian

  23. Kevin Polley says:

    Hi Neil,
    Just as Hoover became synonymous for the term Vacuum Cleaner, G has become the default for search, we can only guesstimate when search will have a Dyson to contend with. I’m in the camp that says semantic find and filtering tech is worth watching as the various forms of descriptive data markup such as GoodRelations get exposure becoming main stream and we start searching the web direct, not the SE’s (it’s not a co-incidence that the other search players have banded together with G to try and dominate with schema either).

    The day after G made that announcement the security sector released information that SSL had been hacked (see: which kind of put’s a hole in one side of G’s reasoning. Yep I know that there is ssl and there is SSL and G will be secure but hasn’t there always been a battle between SE’s and ‘marketers’?

    For my money you’re bang on, G do want more ppl to use adwords to increase profits but where they have been banning so many ‘small’ marketers who fell foul of ‘T&C’s’ us little people really need to start reading and acting on the writing they have been putting on the wall. Understanding they (G) want to be a big boy’s (with big budgets) toy plain and simple will help. Heck they are even going to start charging ppl for using Google Maps/Places (http://www.seroundtable.com/google-maps-api-fees-14235.html). The ‘free line’ is moving for sure.

    I’m thankful we’ll only lose ‘some’ of the G keyword data, yes it’s a pain but do well promoted business only get traffic from G these days? If the social circles do the same then we will have an issue.

  24. Chris says:

    Google is all about user experience and relevance. This is probably a move to be ahead of new regulations that the FTC is putting on the internet.

  25. Lana says:

    Initially I was irritated by this. I’ve seen the missing results in my Analytics.

    Maybe we just need to use another tracking program for our keywords. I’m surprised that I have not seen someone promoting something in my inbox for this yet. I’m pretty sure your hosting program has all that data for your sites anyway.

    Google can do whatever they want. We don’t own them.

  26. zmodo says:

    I believe the ecommerce model is getting the nod over thin affiliate sites, as Google knows that people are looking either for actual physical products and or information about subjects that doesn’t necessarily have physical products to offer.
    Or the WWW is just heading towards the GooGLaZoN or AmAgOOgLe.

  27. V K Rajagopalan says:

    Hi Neil,

    There is absolutely no doubt about it. G at the end of the day is a cut throat business establishment. And, they don’t do anything for philanthropy or for good of the community.

    As SaRita Hartin rightly said what privacy are we talking about. It is just that Big G wants to push adwords aggressively and they will do anything for that.

    Raj

  28. Chris says:

    To look at this another way, what percentage of people use Google while logged in to Facebook? Quite a high percentage I’d imagine.

    Now, what if Google+ became really popular? That 10% would very much be at the bottom end. Add in Gmail and you’re looking at savvy users – i.e. logged in users – being probably 20% or more of incoming searches, particularly those sent via social contact – exactly the ones we want more of. All of those visitors would have no query info attached to them – we’d be left with just the ‘floater’ data.

    This move is way more important than any of the coverage I’ve seen it receive. We could be looking at useful search data becoming almost non-existent within a year.

  29. Danica says:

    Hi Neal,
    You make a good point but then as business owners, online or elsewhere – why rely on one source of traffic to your business? Something seems skewed to me when everyone is pushing build your list and give people a reason to buy from you, yet here everyone is relying on Google to bring people to their door?

    I take Google, or any search engine as part of the cost of doing business but if they died tomorrow, my business wouldn’t go with them. What did you do before keywords? People were making money before they learned about keywords, and there are those who are doing quite well without stressing over what the search engines next move will be.

    Maybe instead of relying on something as iffy as keywords to get traffic or blaming Google for doing whatever is going to make the business profitable, we start thinking about those boring things that make our own business sustainable long term. Just a thought. And it’s not keywords.

  30. Avent says:

    Google is like the school bully – play by his rules or don’t play at all (unless you are smart enough ;)

  31. David says:

    The key is not to be reliant on Google. There are other good search engines out there. And one day, the mighty will eventually fall from its strong tower.

  32. FS says:

    Yes, they are tooo big and they can do what they like.

  33. Charlie says:

    Hello Neil
    If the worst was to happen and G switched all searches to the secure system, then as IMers we should continue to THINK OUTSIDE OF THE BOX

    One idea would be to simply ask the visitor how they landed on your website, you could sweeten it up by placing an Aweber form on the page, which stated for example:

    “We are constantly striving to ensure your visits to “yourwebsite” are both enjoyable and beneficial for you. We are currently analysing our metrics and hope you could spare us a couple of minutes of your time, by filling the form below.
    Simply place the search term you entered in Google in box1. So we can reward you for your time, we would love to send you “niche ebook/video etc” so please also place your name and email in the other boxes, so we can send you your gift. Thanks…

    So you get your search term and a bonus opt-in…

  34. Bryan says:

    This looks like an excellent opportunity for facebook to step into the void google are creating.
    They combine the keyword and demographics for a more stable and affordable PPC to the sites that reside on facebook at a much lower price.

    If they could monitor the number of refunds and warrior reviews on these promotions then they could weed out and protect people from wasting there time and money.

    The guru season is long gone.

  35. John Corrick says:

    Anything Google is doing is purely driven by profit for Google, not privacy. To believe otherwise is foolish. True privacy means that data is hidden from all and does not make exceptions for those that Google receives money or benefit from.

    This is a volatile time when internet development is going ahead in leaps and bounds and the face of the web about to change forever. Competition is increasing for Google and now more than anytime during internet history, maintaining market share and loyalty for Google will be more critical than ever. Google would do well to remember this and think who, what sector and how many will be alienated by their decisions.

  36. Alex says:

    Horribly transparent ploy to sell Adwords; speaks volumes about the type of company Gooogle has become. But regardless, the traffic will still be there :)

  37. Graham in UK says:

    Hi Neil
    “More Confusion for the Masses”!!
    I sometimes laugh at my web logs when I see what they have entered as ‘keywords’. Unbelieable sometimes, I wonder how the search-engines manage to get them to the right page.
    I also find it strange that I’m in the no1 position for a few weeks or more in the top 3 S.E. then the following week Google gives me a position 0 – doesn’t make any sense to me at all? I think they are a law unto themselves and we just have to go with it.
    Paid search? – don’t make me laugh!

  38. Mark Antony says:

    Thanks for the warning Neil. Many never recovered from last years change to SEO, it halved my traffic. I guess it all comes down to Adwords as far as Google are concerned.

  39. Hi Neil

    Yes, I noticed a few days ago that the Google US search engine is now searching with https in the browser address bar. Has this got to do with what you are talking about?

    Google Australia, and Google UK are still using an unsecured http address though. I guess they will be next.

    -Richard

  40. Hi Neil

    Great heads up but not really something that surprises me as I have noticed over the last few months a change in Google’s postioning of its products to a more monetised place. If you spend money with them then you now get more obvious preferential treatment.

    With Google plus the circle is complete, they have all the tools and the social media site to ensure there is no need to ever venture too far from their world.

    I wrote about the relationship that we as marketers need to adopt with Google, we are their suppliers and as such, need to ensure we are supplying a quality product, unfortunately far too many focus on the seo and forget about what our customer wants, ie Google.

    igor Griffiths

  41. Robert Boyd says:

    Google can’t stop the little guy who truly has the “Law of Attraction” on his side, there is a way around everything that Google does as well. Karma has a way of biting those that try to harm the innocent and if Google is not carefull a user friendly search engine that promotes itself as such will rise and slap them back into submition, pun intended. :-) I’m with the gentleman who stated, “I have been blogging for a long time and I’m ready for everything.”

Comments are closed.