Unusual Problem (Full Version)

All Forums >> [Web Development] >> Search Engine Optimization and Web Business



Message

Thomas Brunt -> Unusual Problem (11/21/2005 15:27:53)

I have a hosting client with a very successful site. She has a popular page that is very important to her site. This page has lots of large images. She feels it's important to have all of those images in there at full size, and she has numbers to back that up.

The problem is that this page is exploding with traffic. Most of the traffic doesn't do her any good, but some does. And it's so heavy with big images that it's using more bandwidth than all of the rest of her site.

The bad traffic is coming from image searches on Yahoo and Google, and the traffic has more than doubled each month for quite a few months. It's getting really huge.

I have thought of telling her to put some Adsense ads on that page, and that might be good to pay for some more bandwidth, but there is no end in site for the usage growth. I've really never seen anything like it.

I'm hesitant to use a robots.txt file to exclude the problem page because that page does send her lots of on-target traffic. I'm unsure of where to go from here. Any ideas?

t




slappyJmarsden -> RE: Unusual Problem (11/21/2005 15:33:04)

How about a link? lol



Reflect -> RE: Unusual Problem (11/21/2005 15:41:05)

quote:

The bad traffic is coming from image searches on Yahoo and Google


Ban Froogle only?

User-agent: Googlebot-Image
Disallow: /

Took me a few to locate the bots name as I have never payed attention to it.

Take care,

Brian



Kitka -> RE: Unusual Problem (11/21/2005 18:02:54)

As a matter of course, I ban Google-Image, and Picsearch in all my sites' robots.txt as the traffic from them is never related to a business sites' intent (raison d’ętre), and wastes bandwidth, as you have found.

It also helps to ban indexing the images. So place them all in folders and bar that/those.

Here is a small portion of my robots.txt that relates to images: 

User-agent: Googlebot-Image 
User-agent: psbot 
User-agent: Web.Image.Collector 
Disallow: /

User-agent: * 
Disallow: /images/


You might also wish to check the logs/stats and see if anyone is hotlinking to them, and fix that with .htaccess

HTH



Kitka -> RE: Unusual Problem (11/21/2005 19:04:14)

A couple more things: if she has images already listed on Picsearch, she might have to email them to get them removed, citing the disallow in robots.txt and maybe claiming copyright issues. I had to do that with one site. Once they already have them indexed, they appear to ignore any subsequent changes to robots.txt.

For Google, once the robots.txt is in place, use their removal tool to get the image folders removed sooner rather than later:

http://services.google.com/urlconsole/controller



Thomas Brunt -> RE: Unusual Problem (11/22/2005 9:16:01)

Thanks very much!

t



jaybee -> RE: Unusual Problem (11/22/2005 11:23:05)

Well I've just checked and what seems to have worked for me is as Kitka says, put all the images in a folder called images and then stick

Disallow: /images/

in the robots.txt file.

The only images appearing for my site are not in the images folder.



jaybee -> RE: Unusual Problem (11/22/2005 11:26:21)

quote:

You might also wish to check the logs/stats and see if anyone is hotlinking to them, and fix that with .htaccess


She may start getting emails from users complaining that they can't see any of the images. Pound to a penny they'll be using Norton Internet Security and they need to adjust their settings.



Page: [1]

Valid CSS!




Forum Software © ASPPlayground.NET Advanced Edition 2.4.5 ANSI
0.0625