Skip to main content

You Cannot Index Label Searches, Productively

The subject of indexing of label searches comes up, from time to time, in Blogger Help Forum: Learn More About Blogger.
I want the label content to appear, in search engine hit lists.
Some blog owners cannot understand what gets indexed, by the search engines.

Label searches are not unique content - and should not be indexed. Posts are content - and posts are best indexed, as post pages - directly from the automatically generated sitemap.

Post content indexed both using a normal post page URL, and using one (or many) label search URLs, will be detected as duplicated content.

Duplicated content penalties will be applied to content simultaneously indexed using both label search URLs, and post page URLs. We've seen this happen - and it was not a time of enjoyment, for many blog owners.

A few months after labels were added to our blogs, in 2007, blog owners were complaining about lack of search engine activity. Blogger blog owners, in massive amounts, reported that page rank had dropped to zero - and search engines had stopped indexing their posts.

Investigation revealed that Blogger blogs, with newly added label search indexes, were being indexed twice - one using the normal, post page URLs, and a second, using the label search URLs. Blogs with multiple labels / post were rampantly indexed.

And all of the indexing was for nothing. Why? Because the search engines were detecting the duplicated indexing - and applying penalties to both the post page indexed content, and the label search indexed content. And nobody was happy.

Blogger Engineering installed an emergency update, to our "Robots.Txt" files.
User-agent: * Disallow: /search Allow: /
Since a label search has the URL "/search/label/whatever", "Disallow: /search" blocks obedient robots from searching, using label search URLs - and "Allow: /" permits all other access.

And there we are. Everybody who understands search engine indexing leaves the "Robots.Txt" code intact, and publishes informative, interesting, and unique content.

If you wish, you may remove that portion - or any other portion - of your file. It's your blog, and your file.

However, if you do remove the necessary entries, and you come later to Blogger Help, or maybe Webmaster Central, complaining of reduced search engine activity, or of no visitors, you'll be instructed to restore to a standard "Robots.Txt" file. And, to spend more time publishing content, in your blog.

Comments

Hot guys said…
Very helpful. :)

Popular posts from this blog

What's The URL Of My Blog?

We see the plea for help, periodicallyI need the URL of my blog, so I can give it to my friends. Help!Who's buried in Grant's Tomb, after all?No Chuck, be polite.OK, OK. The title of this blog is "The Real Blogger Status", and the title of this post is "What's The URL Of My Blog?".

Leave Comments Here

Like any blogger, I appreciate polite comments, when they are relevant to the blog, and posted to the relevant article in the right blog. If you want to ask me a question thats relevant to blogging, but you can't find the right post to start with (I haven't written about everything blogger related, yet, nor the way things are going I don't expect to either), ask your questions here, or leave an entry in my guestbook.

As noted above, please note my commenting policy. If you post a comment to this post, I will probably treat it as a "Contact Me" post. If you have an issue that's relevant to any technical issue in the blog, please leave a comment on the specific post, not here. This post is for general comments, and for non posted contact to me.

If the form below does not work for you, check your third party cookies setting!

For actual technical issues, note that peer support in Blogger Help Forum: Something Is Broken, or Nitecruzr Dot Net - Blogging is, almos…

What Is "ghs.google.com" vs. "ghs.googlehosted.com"?

With Google Domains registered custom domains becoming more normal, we are seeing one odd attention to detail, expressed as confusion in Blogger Help Forum: Learn More About Blogger.My website uses "ghs.google.com" - am I supposed to use "ghs.googlehosted.com", instead?It's good to be attentive to detail, particularly with custom domain publishing. This is one detail that may not require immediate attention, however.