Referer Spam Cannot Be Blocked, Immediately

That is the unfortunate truth.

Every day, some new member of Blogger Help Forum: Something Is Broken asks, innocently

What is all this traffic from dodgy websites?

and after we explain what the dodgy traffic is, and why it does not reflect real traffic, the next question is

So why doesn't Google block it? Why should I have it polluting my Stats displays, and be unable to find actual traffic in my counts?

and the unfortunate truth is simply that Google cannot block it, because it's not significantly different from normal traffic - and the insignificant difference is not easily detected.

At the risk of being politically insensitive, I'll ask an old joke from American low class humour

Do you know how to keep a moron in suspense?

and if any of you have never heard the joke, and want to know why it's so funny, I'll simply suggest that you let me know when you decide that you don't know the answer. If I was asking you in person, I would simply continue talking, and change the subject of the conversation completely.

Now, I'll apologise to the American Autism Foundation, or whatever non profit might be offended by my use of the term "moron" - and I'll point out to you that, if you could not answer my joke, then you are - obviously - a moron.

And that is a non technical example of how referer spam works, and why it cannot be blocked. Now, we will attempt a (highly simplified) technical definition.

When you click on a link from, say, a post in Blogger Help Forum: Something Is Broken, to this article, your computer sends a single message to the Blogger server, containing three essential details.

1. The IP address of your computer.
2. The URL of a forum discussion, which contains a link to this article.
3. The URL of this article.

From the message, the Blogger server creates a server activity record.

1. An IP address.
2. The URL of the page containing a link to the webpage requested.
3. The URL of the webpage requested.

That server activity record, in the Stats display for the blog, is known as a "pageview".

Finally, the Blogger server starts sending web page content back to your computer, so your computer can display this article to you. As the webpage content is sent back to your computer, your computer receives, and displays, the received content - and asks for more content.

Simple enough? So, what is referer spam? Simply, a single message from a spammer computer, to the Blogger computer, containing three essential details.

1. An IP address - possibly, but not predictably, of their computer.
2. The URL of the website being pimped (the spammed website).
3. The URL of the blog being spammed (your blog).

From the message, the Blogger server creates a server activity record.

1. An IP address.
2. The URL of the page (supposedly) containing a link to the webpage requested.
3. The URL of the webpage (supposedly) requested.

That server activity record, in the Stats display for the blog, is also known as a "pageview".

Finally, the Blogger server starts sending web page content back to the IP address provided. If the IP address does refer to the spammers computer, what is received is simply ignored. The spammer computer moves on, and sends another spam message to another server.

The problem is simply that no web server can detect a message from a client computer, that results in a response that is just ignored. Web traffic is lossy, and clients drop offline constantly. Even if the response could be detected as ignored, the ignored request might still reflect legitimate activity, initiated by a client that immediately went offline.

There is simply no way for Google to block the spam - because the spam is simply one message that results in a response, by the Blogger server, that is subsequently ignored by the client computer.

That's it.

Are you in suspense yet?

So why can't Google block the numbers generated by referer spam, as the referer spam hits the servers? Simply because the numbers may not really represent actual spam. They can, just as easily, reflect intense, legitimate activity - or possibly a secondary attack against a legitimate website. Specific numbers are observed in context - are blocked only after the same activity is observed against multiple blogs, over long periods of time - and the numbers are removed, retroactively.

All of this is a simple unavoidable side effect, of blog owners needing site activity figures that are not affected by script filtering by the blog readers, complicated by fraudulent activity by hackers and spammers.

D'Ohh!

>> Top