Thursday, June 03, 2010

Blogger Magic - Search Engine Indexing

A lot of blog owners are confused about how their blogs get indexed by the search engines.
Why do many SERP hits lead to my main page, where there is no content?
is a normal question. Many blog owners think that the search engines, as they index a blog, are executing one big magic trick.

Some blog owners think that their blogs will be indexed, and will appear in Search Engine Results Pages (aka "SERPs"), automatically. They are wrong, though - there is no magic here.
  • Main page indexing, from "Index my blog!" requests. The more often you request re indexing, the more often your blog will be re indexed.
  • Random indexing, from inlinks from other websites. The more inlinks you have, the more often your blog will be re indexed.
  • Scheduled indexing, using a sitemap. The more often you publish, the more often your sitemap gets updated, and the more likely that the search engines will see a need to re index in the future.
All of these factors affect the others. The more often your blog gets re indexed, the quicker any updates that you make will appear in SERPs, which will lead to more readers, and to more inlinks. The more inlinks your blog has, the more often the search engines will schedule the next re indexing. And so on, forever.

You start out by publishing a blog with interesting, useful, and unique content.

Once you have a blog, you spend time publicising your blog. One of the key steps in publicising involves submitting indexing requests to the search engines - either directly, or through one or more search engines submission services. Your blog is placed into a queue, and the main page is indexed. Until your blog has a reputation, the blog is indexed, using the main page - and that's it.

With a Blogger blog, you provide a sitemap, which uses the blog posts newsfeed. The search engines index the blog using the newsfeed, which is the text content of the posts. The feed type selection will have a major effect on sitemap related search indexing.

Please, always remember that your blog gets indexed faster, when the search engines can read the content. If your blog is published to a custom domain, it's in your best interest to use righteous DNS addressing.

There are many search engines on the Internet. Each search engine indexes the websites on the Internet, so it can serve its customers - the people who want to read the websites. Each search engine wants to improve its own reputation for providing good results.

For an insightful reference into Google Search, you may enjoy Google: How Search Works. Remember that there are subtle differences between how Blogger blogs, and websites in general, are treated. Be selective, in accepting advice outside the Blogger environment, when asking about a Blogger blog.

Each search engine serves its own interests - and provides better visibility to blogs and websites with higher reputation, when people search for blogs and websites. Your blog gets a higher reputation, as more blogs and websites link to it. With better visibility, your blog gets more new readers, and more inlinks.

A link from a low reputation website - to your blog - gives your blog little reputation. A link from a high reputation website - to your blog - gives your blog more reputation. When that website is being indexed, and the search engine finds a link to your blog, your blog is indexed. This may be a main page link, or an individual post link.

As your blog gains reputation, it gets indexed regularly, by a search engine, on its own. This is where providing a sitemap may be a key ingredient to getting your blog indexed - and this is where your individual posts are indexed, possibly based on a sitemap.

Just remember that your blog needs a good reputation, to get more readers - but your good reputation won't be yours, forever.

Until your individual posts are indexed, and they get reputation, most SERPs that reference the posts will link to the main page. You'll see similar latency issues when you rename your blog without properly planning for the change - you'll have search hit entries referencing the old blog name (URL) long after the blog is renamed.

Neither issue is magic, it's all logic. None of this is magic, in reality.

>> Top


Alanna Kellogg said...

It also makes it even more important to have a search function available high on each page - so motivated searchers can find what they are looking for.

Antonia Noël said...

And until you get a significant amount of traffic, linking your blog on forums, twitter, facebook, myspace, technorati, etc. on a regular basis will increase your blog's reputation. I would do this anyway even if you already have a lot of readers. At least that's how I believe it works.

-Antonia Noel speaking on behalf of the earth