prevent your sites from being raped!!

[playa]

my host sent me this info..
i didn't realize that those offline browsers were so popular....
here is an email they sent me...
looks like it was copied from one of andy's sexswap newsletters.....i use to get those emails all the time..but ever since i removed their bannerswaps i haven't gotten any... well here is the .htaccess file if some of you like me didn't get it..

PRO-LEVEL HELLA BANDWIDTH THIEF BLOCKING by Joel
Joel wrote me with some great info on apache htaccess with a terrific list of know
offline webbrowsers that chew up bandwidth like there is no tomorrow. Joel writes:

I've been reading your newsletter as long as I can remember, and
you've even posted a few of my emails in the past, now I'm asking for some
help from you and/or the readers to put together a air tight .htaccess
file using mod_rewrite to block people downloading images or hotlinking
without actually going to the site. As we all know, someone who uses an
offline browser like WebZIP or Internet Ninja is only out for the images,
they'll never see our advertisements, they'll never join our paysites,
they simply rape our sites of everything we have to offer, and move on.
These programs are nothing new, but they're becoming a big concern for me.
After analizing my logs for my free sites for the past ~1 month, I've seen
over 1.4 million requests made by these offline browsers alone. What I'm
attempting to do is to grab the user_agent tags from each of these offline
browsers and essentially bounce them away from my images using .htaccess
with mod_rewrite.

Second part that I'm trying to protect myself against, which is a
growing concern for me, is all of these chat boards that hotlink your
images without even a shred of concern that they're eating up your
bandwidth without even providing you with a link. (But hey, I don't like
paying for someone else to get a free ride). I'm going to attach my
current .htaccess file, with hopes that the Sexswappers will append any
sites they have discovered in their logs so I may come up with a complete
listing to protect us from "Joyriders".

For anyone who isn't familiar with mod_rewrite and Apache
webservers, you may want to check out their documentation so you'll
understand it a little better before you dip into this:

Module Rewrite URL Rewriting Engine http://www.apache.org/docs/mod/mod_rewrite.html

Apache 1.3+ URL Rewriting Guide (Basic and complex examples) http://www.apache.org/docs/misc/rewriteguide.html

RewriteEngine On

RewriteCond %{HTTP_USER_AGENT} ^DISCo\Pump.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Drip.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Gets.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^IBrowse.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\Ninja.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^JustView.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\tool.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\PiX.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\Explorer.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\Foto.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Pockey.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Slurp.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^SpaceBison.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\Image\Collector.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\Sucker.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Webster.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^ia_archiver.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^lftp.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut.*
RewriteRule .[Jj][Pp][Gg]*$ /leeches.html [L]

RewriteCond %{HTTP_REFERER} ^http://.*adfilter.com.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://207.198.147.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://www.neocities.* [OR]
RewriteCond %{HTTP_REFERER} ^http://chat.passagen.s.* [OR]
RewriteCond %{HTTP_REFERER} ^http://volpi.sti.com.* [OR]
RewriteCond %{HTTP_REFERER} ^http://batepapo0.uol.com.* [OR]
RewriteCond %{HTTP_REFERER} ^http://batepapo1.uol.com.* [OR]
RewriteCond %{HTTP_REFERER} ^http://batepapo2.uol.com.* [OR]
RewriteCond %{HTTP_REFERER} ^http://batepapo3.uol.com.* [OR]
RewriteCond %{HTTP_REFERER} ^http://batepapo4.uol.com.* [OR]
RewriteCond %{HTTP_REFERER} ^http://batepapo6.uol.com.* [OR]
RewriteCond %{HTTP_REFERER} ^http://batepapo7.uol.com.* [OR]
RewriteCond %{HTTP_REFERER} ^http://209.2.137.* [OR]
RewriteCond %{HTTP_REFERER} ^http://207.126.121.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://.*bianca.com/.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://.*angelfire.com/.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://.*hotmail.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://.*yahoo.com.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://.*geocities.com.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://www.partyhouse.*$
RewriteRule .[Jj][Pp][Gg]*$ /leeches.html [L]



.HTACCESS TO ALLOW/DENY from certain domains (for AVS)

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://www.<CURRENT_DOMAIN> [NC]
RewriteCond %{HTTP_REFERER} !^http://<CURRENT_DOMAIN> [NC]
RewriteCond %{HTTP_REFERER} !^http://<IP OF CURRENT_DOMAIN> [NC]
RewriteCond %{HTTP_REFERER} !^http://www.<GOOD_REFERER> [NC]
RewriteCond %{HTTP_REFERER} !^http://<GOOD_REFERER> [NC]
RewriteCond %{HTTP_REFERER} !^http://<IP OF GOOD_REFERER> [NC]
RewriteRule /* http://www.<CURRENT_DOMAIN>


NOTES:
<CURRENT_DOMAIN>:
This is the domain name of the site you wish to protect. (xxxwebhosting.com)


<IP OF CURRENT_DOMAIN>:
This is the IP address of the site you wish to protect. (63.168.246.202)

<GOOD_REFERER>:
This is the domain name of a site you wish to allow to refer people to this
directory from (reliablehosting.com).

<IP OF GOOD_REFERER>:
This is the IP address of the site you wish to allow to refer people to this
directory from (198.172.12.96)

- Save this file as .htaccess in the directory you wish to protect.

- If you save it as /images/.htaccess , it will protect everything under images.

Meaning, there is no reason to save another one in /images/images2/.

[Jimbo]

personnally I like when I find someone hotlinking my pictures, I then edit the image and write my URL on it saying it was stolen http://adultwebmasterinfo.com/ubb/smile.gif then rename the real image for my site http://adultwebmasterinfo.com/ubb/smile.gif

[Lizard]

Hey,

for leach/snake agents like teleport, instead of redirecting
to a page, send em off to a place like www.microsoft, (http://www.microsoft,) some
people let them run unattended, imagine their faces when they
come back later and find that they are trying to download
a copy of microsoft http://adultwebmasterinfo.com/ubb/smile.gif

Oh, and the url for Andy's trix archives is:
http://trix.sexswap.com/

There is still some good stuff sometimes.

[moses]

While it's true that some of those offline browsers can bring a cgi driven site to its knees, balance your use of rewrite rules with the fact that for each one of those rules, the webserver has to do a comparison to see if they match. While apache is fast, and cpu is cheap, there's no way a hit with 100 rules to match can be as fast as a hit with no rules.

Moses

[Lizard]

Hey,

I suck at *nix stuff, but for the user_agent
cant you just look for the ones you DO allow
so all others are denied access?

Something like:

if not *mozilla*
or not *opera*
then redirect to somewhere...

It should be possible to write a regexp
that matches the above...

That would cut down on the cpu useage since
you would only have to look for the 2-3 major
browser agent strings...

[Muff]

Why dont you just rewrite the htaccess so when someone trys too access an image outside of your domain or a good referer they are redirected to your sponsor. heck maybe even make a buck from the hotlinkers.

Sorry dont have the code but there is one just cant remeber where.

Muff

[Rod]

There is one there: http://www.webmasterlounge.com