Skip to main content

Blogger Magic - The "Robots.txt" File

Some new blog owners spend time examining the various settings provided by Blogger, in the dashboard.

Some dashboard settings inspire questions, in Blogger Help Forum: Learn More about Blogger. As helpers there, we know from experience that some questions, asked and answered there, will lead to later questions in Blogger Help Forum: Get Help with an Issue.

The "Robots.Txt" file is one feature which inspires these types of questions.
I was updating my settings on Blogger, and I discovered some settings that I didn't understand. What do I add for "Custom robot text" and "Custom robot header tag"?
When I was young, my mother used to provide advice "That is a well enough!" - as the proper answer, to this question, is to "Leave well enough alone!".

The "Robots.txt" file is a collection of various settings - and the dashboard "Search preferences" wizard provides useful options, for Blogger blog owners.

You may edit "robots.txt", if you wish - but be mindful of possible consequences.

When used properly, "Robots.Txt" provides us several possibilities. That said, we should heed the warning.
Warning! Use with caution. Incorrect use of these features can result in your blog being ignored by search engines.
The section "Crawlers and indexing" is a dashboard feature which should probably be left alone by 99% of all Blogger blog owners - except in specific, documented examples.

All "robots.txt" entries are carefully designed, by Blogger / Google Engineers.

Some portions of "Robots.Txt" are maintained by various well documented Blogger and Webmaster Tools features.
Other sections of "Search preferences" have similar value. Unless you understand "Robots.Txt" functionality, however, you should leave settings in "Crawlers and indexing" alone.

If you are not familiar with the settings, it's best to not play with them.

Leave the magic spells, to the magicians and wizards. See Blogger Help: Help people find your blog on search engines, and Google Developers - Webmasters: Robots.txt Specifications, for details.

Use the Blogger dashboard "Privacy" wizard - and stop there - unless you are prepared to deal with the consequences.

Work on publishing blog content - informative, interesting, and unique. Learn more, as you publish your blog. Indexable blog content will get you more search reputation, than tweaking "robots.txt".

Comments

Popular posts from this blog

Add A Custom Redirect, If You Change A Post URL

When you rename a blog, the most that you can do, to keep the old URL useful, is to setup a stub post , with a clickable link to the new URL. Yo! The blog is now at xxxxxxx.blogspot.com!! Blogger forbids gateway blogs, and similar blog to blog redirections . When you rename a post, you can setup a custom redirect - and automatically redirect your readers to the post, under its new URL. You should take advantage of this option, if you change a post URL.

Jump Break Is Visible Only In An Index Page

Some blog owners are unsure what a Jump Break looks like, when applied to their blogs. Occasionally, in Blogger Help Forum: Get Help with an Issue , we see the confusion. Where's the Jump Break? or I added a Jump Break to several posts, but it never shows up! When asked for a screen print of what they're seeing, they may provide a image of the post, in the Post Editor Preview window - or possibly, the published post, but in post page view.

Leave Comments Here

Like any blogger, I appreciate polite comments, when they are relevant to the blog, and posted to the relevant article in the right blog . If you want to ask me a question thats relevant to blogging, but you can't find the right post to start with (I haven't written about everything blogger related, yet, nor the way things are going I don't expect to either), ask your questions here, or leave an entry in my guestbook . As noted above, please note my commenting policy . If you post a comment to this post , I will probably treat it as a "Contact Me" post . If you have an issue that's relevant to any technical issue in the blog, please leave a comment on the specific post , not here. This post is for general comments, and for non posted contact to me. If the form below does not work for you, check your third party cookies setting! For actual technical issues, note that peer support in Blogger Help Forum: Something Is Broken , or Nitecruzr Dot Net - Bloggin...