GoogleBot ate my bandwidth!

Yesterday I got an automatic e-mail from my web host that this site has consumed 80% of the bandwidth.

I was thinking that it was very stange, it is just in the beginning of the month and I don't get so many visitors. My bandwidth is on 2 GB.

I checked the my statcounter but no increase in visitors. Hmmm. I checked in AW Stats and there I see that 1,6 GB was used in the last days coming from one IP. And it is not a SE spider.

Ok I go and use the IP look-up in statcounter but that IP was not recorded ...

I go to DNSstuff.com to check who is this IP and I find some kind of company in the US ...

I checked the IPs of some sites and found out that my site AddURL-Free.com has the same IP, so this is the IP of the servers of the hosting company.

After a moment of figuring I put two and two together I came to the conclusion that Google has been spidering the ODP section of addurl-free.com like crazy and I had a RSS feed there from this site that displayed 8 items on each subpage. So the spidering only costed addurl-free.com about 50 MB but the generation of the RSS feed costed this site 1,6 GB. I have now turned that off.

I have sent an e-mail to my hosting company to see how to solve this, perhaps I will have to upgrade again so that I have 5 GB.

So it could happen that this site goes down for a while "bandwidth limit exceeded" but I hope not.

Anyway I should see it from the good side, I leared that RSS feed consumes bandwidth.

Other posts

About the Author

Jim Westergren is a Swedish web entrepreneur currently living in Spain. He is happily married and has three lovely children. Some of his interests are web development, SEO and writing.
He is the Founder of DomainStats and N.nu. Read his full about page.