|
| |
|
|
ou812
Posts: 1603
Joined: 1/5/2002
From: San Diego
Status: offline
 |
Crawling - 2/27/2005 23:12:23
A friend of mine has a very big homestyle webpage. It probably has well over 200 pages and 90 completely different subject areas of content on the different pages. He just started to get statistics (awstats) for his site for fun and has gotten crawled 1300 times by googlebot this month and 350 by msnbot, as well as many others.
His site, and many of the site's pages, have many other sites that link to his. He has a google search on his pages too. He gets about 5000 unique visitors a month. I also see he has had 2000 refered this month from Google. This isn't included in the bot, is it? He sells and advertises nothing. All the pages are pure information on the many different subjects.
I'm just curious, is this normal for google, msn, and other crawlers, to visit so often? This seems ridiculous to me. Why so much?
Any thoughts?
_____________________________
-brian
EnterpriseDB: Enterprise-class relational database management system
PostgreSQL: The world's most advanced open source database
|
|
|
|
ou812
Posts: 1603
Joined: 1/5/2002
From: San Diego
Status: offline
|
RE: Crawling - 3/1/2005 12:11:39
Well, nice to know it "may" not be uncommon. But wow, 50 per day! sure seems extreem. Maybe a few times a month would make more sense.
_____________________________
-brian
EnterpriseDB: Enterprise-class relational database management system
PostgreSQL: The world's most advanced open source database
| |
|
 New Messages |
 No New Messages |
 Hot Topic w/ New Messages |
 Hot Topic w/o New Messages |
 Locked w/ New Messages |
 Locked w/o New Messages |
|
Post New Thread
Reply to Message
Post New Poll
Submit Vote
Delete My Own Post
Delete My Own Thread
Rate Posts
|
|
|
Printable Version 
