When To Use Noindex vs. Disallow [Video]

In a recent YouTube video, Google’s Martin Splitt explained the differences between the “noindex” tag in robots meta tags and the “disallow” command in robots.txt files.

Splitt, a Developer Advocate at Google, pointed out that both methods help manage how search engine crawlers work with a website.

However, they have different purposes and shouldn’t be used in place of each other.

When To Use Noindex

The “noindex” directive tells search engines not to include a specific page in their search results. You can add this instruction in the HTML head section using the robots meta tag or the X-Robots HTTP header.

Use “noindex” when you want to keep a page from showing up in search results but still allow search engines to read the page’s content. This is helpful for pages that users can see but that you don’t want search engines to display, like thank-you pages or internal search result pages.

When To Use Disallow

The “disallow” …