1. Home
  2. Use Cases
  3. Detect Robots Meta Tag with Onpage Hero

Detect Robots Meta Tag with Onpage Hero

Onpage Hero extends Google Analytics functionality with unique custom dimensions. This time, we are focussing on automatically detecting the robots meta tag of each page using the Robots Onpage Hero custom dimension. 

Firstly, make sure you have the Robots dimension selected as part of your Onpage Hero plan. Free plans allow you to add up to 3 datapoints, whereas paid plans allow up to 20.

Upgrade today to access up to 20 datapoints

To check which datapoints you have access to, log in to your Onpage Hero account and click the little settings icon on the right. Now, select Change Datapoints to display your current setup.

Change Data Points Onpage Hero

The options on the left show the datapoints applied to this property. Select the Robots dimension, click NEXT, then VIEW ANALYTICS to jump into your Google Analytics property.

Select Custom Dimension Robots

What is a robots meta tag?

The robots meta tag is a HTML element that gives instructions to search engine crawlers such as Googlebot. This tag is placed in the header of a page and can give instructions to all crawlers name=”robots” or specific name =”Googlebot”. Typically instructions include either to index content (content=”index”) or not to index (content=”noindex”).

robots meta tag source code

Why are robots meta tags important?

Robot meta tags are important because they give or remove access from search engine crawlers. For example, if you were performing maintenance or major SEO efforts on part of a site and wanted to hide if from crawlers, then you could add  <meta name=”robots” content=”noindex”> to the robots.txt file. This allows the search engine to crawl this page but not to display it in the search results.

Robots meta tags best practices

  • If you don’t want search engines to crawl part of your site use “noindex” meta tags
  • Make sure search engines can crawl all the pages you want to rank

Detecting robots meta tags with Onpage Hero

To find the Robots custom dimension in Onpage Hero, simply navigate to your default view within in your Google Analytics property.

Firstly, enter your Google Analytics property and navigate to:

Behaviour > Events > Top Events

Then, set Event Label as the primary dimension. This is your standard view for using Onpage Hero custom dimensions.

Not seeing any Onpage Hero data? Check that you have set up Onpage Hero correctly following this guide.

Onpage Hero Default View

Ok, let’s take a look at our robots data. So, type “robots” as a secondary dimension to display the results.

Select Robots custom dimension

Onpage Hero now displays each robot meta tag set per URL. Check for pages that are mistakenly set to “noindex” or “nofollow” and update your robots.txt file accordingly.

Robots meta tag results

Onpage Hero custom dashboards: Meta Tags

Robots meta tags are just one of the meta tags that Onpage Hero allows you to see in Google Analytics. Others include the hreflang tags and canonical URLs. To see these three meta tags all together head over to our dashboards page and grab the Meta Tags custom report for Google Data Studio. Simply select the property for which you wish to see the data for from the drop-down menu.

Custom report meta tags

Key Takeaways: robot meta tags

  • Use Robots dimension to detect robot meta tags
  • Fix and falsely set tags
  • Use the Content Data dashboard to view other meta tags side-by-side

Further reading

Time to optimize these other meta tags:

Any issues or feedback? Let us know about your issues and product feedback via support@onpage-hero.com and we’ll be happy to help you.

Updated on September 25, 2018

Was this article helpful?

Related Articles

Add A Comment