marketing you can measure™
020 7061 6426020 7061 6426
The Adwords spider bot is used to analyse the content of advertising landing pages, which helps determine the Quality Score that Google assigns to your PPC ads.
Google takes this Quality Score into account, as well as the amount of money you are willing to bid, when it determines the position of your ads. Therefore, ads with a high Quality Score can rank higher than competitors' - even if they are paying more than you.
It clearly makes sense to ensure the Adwords spider bot gives your landing pages as high a score as possible. To help you, I've put together some frequently asked questions about how the bot works.
What is the user agent name of the PPC bot that checks relevancy on a landing page?
Adsbot.
Does this bot obey robots.txt?
The AdsBot is not a "search crawler" in the strict sense, so robots.txt is irrelevant (i.e. it doesn't visit all the links in the landing page). It only crawls the URLs of keywords with an 'Active' status.
Does this bot support wildcards e.g /filename*
The wildcards would be in robots.txt and therefore not relevant.
Does this Adbot run JavaScript and drop a cookie?
No.
Is keyword (KW) relevancy on landing page used as a factor that determines Quality Score?
The keywords have to be relevant to the landing page. Relevance refers to the usefulness of information to a user (such as an ad, keyword, or landing page). The quality of this relevance is reflected by a keyword's Quality Score.
Is KW relevancy on landing page used as a factor in determining KW editorial approval?
The keyword policy states: "Target relevant and specific keywords. Use specific keywords that accurately reflect your products, services, or the site you're promoting. If you offer a location-specific product or service, you might consider using keywords that reflect your location."
An example: A New York apartment rental agency would not be allowed to run on only the overly general keyword 'rentals.' The agency should instead use keywords such as 'New York rental agency' or 'find NY apartments.'
When matching KW relevancy, which is used: the advert destination URL or the KW destination URLS?
Keyword Relevancy is solely based on Click-through rate (CTR). High CTR = High Relevancy.
When matching KW relevancy, are mis-spellings such as ‘credit carRd’ accounted for?
Since the KW relevancy is based solely on CTR then the mis-spelling will have a high relevancy if it has a high CTR.
When matching KW relevancy does the bidword have to be an [exact] or [standard] match with the keyword on the landing page?
It will have a high relevancy if it has a high CTR. See the point above.
When matching KW relevancy, where does the bot check on the landing page (e.g does it look in title, meta KW, meta description, alt tags, anchor text, or only the text content on the page)?
Again the KW Relevancy is solely based on CTR. High CTR = High Relevancy.
I hope these questions and answers have given you an insight into the ways in which the Adwords spider operates. I'm very happy to take any questions in the comments.
Hannah is an online marketing executive in Coast Digital's online marketing team. Her work combines SEO, PPC, eBay marketing & blog management.
Hannah’s combined experience of sales, online marketing and account management has given her an excellent insight into, and understanding of businesses processes. She has a passion for SEO and a strong thirst for knowledge - so the fast paced and continuously changing industry of Online marketing suits her well. She is currently studying for her Google Adwords Professional Exam.
(2) comments
Comments
I would suggest that Screen scraping tools like those mentioned are bad for the industry in general, as they inflate the impression rate of adverts, which falsely increases impression numbers. They don’t take into account the nuances of quality PPC management – don’t forget that bid price is only one factor in assessing what you actually pay for a click – the Ad text, Google’s calculated keyword relevance and landing page quality as well as account history are all extremely relevant factors which these scraping tools don’t have access to. The Google bid auction is well documented online, and you can see the various influences if you research it a bit. It raises the question, is there really any point in finding out what competitor X is paying for top spot, when there are so many other factors involved?
The alternative, to obtain a good overall picture of what competitors and searchers are doing, is to carry out some very thorough keyword research, to use the Google keyword tool, Google trends and Google insight to learn about the market behaviour and run some test campaigns. The accuracy of those screen scraping tools, which haven’t any solid basis for budget estimations is certainly in question, the only way to know about real search volume is to use a competitive intelligence tool like Hitwise, which is comparatively expensive, but considerably more accurate than screen scraping tools, as it is based on a very large segment of actual ISP traffic data.
I hope this helps but let me know if you need any other information.