With (static) pages becoming a popular component in our blogs, we are seeing a few queries about getting pages content indexed.
We see an occasional query, in Blogger Help Forum: Learn More About Blogger, which leads to using the new pages sitemap.
Blogger provides two automatically generated sitemaps, for each blog. This blog has
Most blogs use the posts sitemap - and ignore the pages sitemap.
If you look at the "robots.txt" file for this blog, you see the first sitemap in use. The file for your blog, if standard, should be similar.
Note the presence of "sitemap.xml" - and the absence of "sitemap-pages.xml". This blog has no indexable content, in the static pages.
If I wanted to index my static pages- and there are significant differences between pages and posts, that generally makes indexed static pages not so useful - I could add "sitemap-pages.xml", to complement "sitemap.xml".
For details on adding "sitemap-pages.xml", to your "robots.txt" file, see here.
Static pages are not indexed, deliberately - but random indexing will take place.
Static pages are not indexed, deliberately - though there can be some random links that lead to indexing. With blogs, search engines index dynamic content - aka posts, deliberately.
If you publish a website as a Blogger blog, you may want the website indexed - and that is why you might use the pages sitemap. Be aware of possible confusion, from a blog built with static pages.
This blog has no indexable static content.
Besides having no indexable static content, this blog has some non indexable static content. I wrote one article, which is carefully mirrored between a page ("static page") and a complementary post ("dynamic page") - which discusses the differences between pages and posts.
Were both the dynamic and static pages of that specific article to be indexed, in this blog, the search engines might levy a duplicated content penalty. Your blog may differ - and that is why Blogger provides "sitemap-pages.xml", and the option to index static pages.
It's your blog - and your choice to make.
Most blog owners will publish posts (dynamic pages) as blog content, and use static pages sparingly, as static (non indexed) accessories. Blogger, as a default, leaves "sitemap-pages.xml" out of "robots.txt" - but you can add pages, as indexed content, if you wish.
We see an occasional query, in Blogger Help Forum: Learn More About Blogger, which leads to using the new pages sitemap.
How do I get pages indexed?Aside from the design issue, where not all blog owners should want pages indexed, there should not be an issue here.
Blogger provides two automatically generated sitemaps, for each blog. This blog has
http://blogging.nitecruzr.net/sitemap.xml
http://blogging.nitecruzr.net/sitemap-pages.xml
Most blogs use the posts sitemap - and ignore the pages sitemap.
If you look at the "robots.txt" file for this blog, you see the first sitemap in use. The file for your blog, if standard, should be similar.
http://blogging.nitecruzr.net/robots.txt
User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search Allow: / Sitemap: http://blogging.nitecruzr.net/sitemap.xml
Note the presence of "sitemap.xml" - and the absence of "sitemap-pages.xml". This blog has no indexable content, in the static pages.
If I wanted to index my static pages- and there are significant differences between pages and posts, that generally makes indexed static pages not so useful - I could add "sitemap-pages.xml", to complement "sitemap.xml".
For details on adding "sitemap-pages.xml", to your "robots.txt" file, see here.
User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search Allow: / Sitemap: http://blogging.nitecruzr.net/sitemap.xml Sitemap: http://blogging.nitecruzr.net/sitemap-pages.xml
Static pages are not indexed, deliberately - but random indexing will take place.
Static pages are not indexed, deliberately - though there can be some random links that lead to indexing. With blogs, search engines index dynamic content - aka posts, deliberately.
If you publish a website as a Blogger blog, you may want the website indexed - and that is why you might use the pages sitemap. Be aware of possible confusion, from a blog built with static pages.
This blog has no indexable static content.
Besides having no indexable static content, this blog has some non indexable static content. I wrote one article, which is carefully mirrored between a page ("static page") and a complementary post ("dynamic page") - which discusses the differences between pages and posts.
Were both the dynamic and static pages of that specific article to be indexed, in this blog, the search engines might levy a duplicated content penalty. Your blog may differ - and that is why Blogger provides "sitemap-pages.xml", and the option to index static pages.
It's your blog - and your choice to make.
Most blog owners will publish posts (dynamic pages) as blog content, and use static pages sparingly, as static (non indexed) accessories. Blogger, as a default, leaves "sitemap-pages.xml" out of "robots.txt" - but you can add pages, as indexed content, if you wish.
Comments
thanks,
Thanks for the question.
Any "page" - whether dynamic or static - can be linked, and any linked page can be randomly indexed when the blog is being indexed. So the term "not looked at" is rather extreme.
Deliberate indexing, using a sitemap, is the best way to ensure indexing of specific content. If you want your pages indexed, because the blog only contains pages, you have to add a pages sitemap.
If you want your pages indexed, and the blogs contains posts, and possible links to the pages, the pages may be indexed - but again, I'd add a pages sitemap to ensure indexing.
If you don't want the pages indexed, at all, I'd add "nofollow" to all pages links.
So, it's just up to you, what you want.
http://blogging.nitecruzr.net/2010/06/blogger-magic-search-engine-indexing.html
When you say "Blogger provides two automatically generated sitemaps, for each blog. This blog has
http://blogging.nitecruzr.net/sitemap.xml
http://blogging.nitecruzr.net/sitemap-pages.xml
My robots.txt shows only one and it's the sitemap-pages.xml I am particularly interested in soi that pages are ultimately searchable in my blog - currently they are not.
So how do I get sitemap-pages.xml into my robots.txt file?
Thanks for the question.
You make a custom "robots.txt" file, using the the "Custom robots.txt" wizard on the dashboard Settings - "Search preferences" page.
http://blogging.nitecruzr.net/2015/07/customise-robotstxt-file-for-your-blog.html