The file "Robots.Txt", used to manage search engine access to your blog, is found in the blog root folder.
You can't edit "Robot.Txt" directly - but you can upload a new file. Long ago, you would use Webmaster Tools - which is now called Search Console.
Now, we can update "robots.txt" directly, using the Blogger dashboard "Custom robots.txt" wizard, in Settings - "Search preferences".
Here's the link to the "Robots.Txt" file, for this blog.
To view / copy the current contents, just click on the link. You can do the same with the file, for your blog.
If I want to change the file for this blog, I start with the current file. Excepting one small detail (my blog is published to "blogging.nitecruzr.net", yours is published to your chosen URL), your file will be identical - if you have not tweaked it.
Let's look at a simple example - adding a sitemap.
Some people would like to index the static pages, in their blog. That's a simple enough task - just add the sitemap, for static pages, to your file.
"sitemap-pages.xml", like "sitemap.xml", is automatically generated for each blog.
Start from Settings - Search preferences.
Start with an empty ("[Disabled]") "Custom robots.txt".
Click on "Edit".
Click on "Yes".
Paste in your edited file content. Note that you will overlay the current contents with what you paste - so paste the entire file, as you wish to have.
Click on "Save changes".
Select "Yes" for "Enable custom robots.txt content?", if necessary.
Right now, though, I don't want to change my file. So, I'll leave my custom file in the wizard, and leave "No" selected.
And the custom file is
"[Disabled]", again.
Even if disabled, I can get it back again, easily enough. Just click on "Yes", and there it is, again. For right now, though, I'll leave it "[Disabled]".
Just be aware of the dangers of editing, without knowing what effect a given entry has, on search engine activity. Remember that you can get the default file back, with a couple mouse clicks.
You can't edit "Robot.Txt" directly - but you can upload a new file. Long ago, you would use Webmaster Tools - which is now called Search Console.
Now, we can update "robots.txt" directly, using the Blogger dashboard "Custom robots.txt" wizard, in Settings - "Search preferences".
Here's the link to the "Robots.Txt" file, for this blog.
http://blogging.nitecruzr.net/robots.txt
To view / copy the current contents, just click on the link. You can do the same with the file, for your blog.
If I want to change the file for this blog, I start with the current file. Excepting one small detail (my blog is published to "blogging.nitecruzr.net", yours is published to your chosen URL), your file will be identical - if you have not tweaked it.
User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search Allow: / Sitemap: http://blogging.nitecruzr.net/sitemap.xml
Let's look at a simple example - adding a sitemap.
Some people would like to index the static pages, in their blog. That's a simple enough task - just add the sitemap, for static pages, to your file.
"sitemap-pages.xml", like "sitemap.xml", is automatically generated for each blog.
User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search Allow: / Sitemap: http://blogging.nitecruzr.net/sitemap.xml Sitemap: http://blogging.nitecruzr.net/sitemap-pages.xml
Start from Settings - Search preferences.
Start with an empty ("[Disabled]") "Custom robots.txt".
Click on "Edit".
Click on "Yes".
Paste in your edited file content. Note that you will overlay the current contents with what you paste - so paste the entire file, as you wish to have.
Click on "Save changes".
Right now, though, I don't want to change my file. So, I'll leave my custom file in the wizard, and leave "No" selected.
And the custom file is
"[Disabled]", again.
Even if disabled, I can get it back again, easily enough. Just click on "Yes", and there it is, again. For right now, though, I'll leave it "[Disabled]".
Just be aware of the dangers of editing, without knowing what effect a given entry has, on search engine activity. Remember that you can get the default file back, with a couple mouse clicks.
Comments
Bryan
I tried to do this in my blog but it is showing only 151 submitted link and 126 indexed only knowing that i have 304 posts published.
http://chem2tech-en.blogspot.com
can you please check it, and help me to adjust this error.
Regards
Tamer
But a Custom Search does not find any words that are indexed on the four pages {I assume they are static) that I have. So obviously I have missed a step, or a concept somewhere. So I would appreciate a little guidance.
Harry