Saving Bandwidth and Preventing Hotlinking With Coral Cache

A while ago Jeff Artwood wrote about “outsourcing” hosting your images to a 3rd party service as a method for minimizing bandwidth costs. This a very practical advice not just for high traffic sites but for just about anyone. You never know when you might get on digg or slashdot and get a significant traffic spike. Jeff outlined various image hosting services in his post, and all of them have certain flaws:

  1. Image Shack and Photo Bucket have prohibitive storage and bandwidth caps
  2. Flickr has an upload cap and a restrictive TOS
  3. S3 is cheep but not free

There is an alternative to offsite image hosting though: Coral Cache.

Coral Cache Logo

The image above is a good example of Coral Cache-ing your images. It is both hosted on my server, and distributed by the Coral Cache service. How is it done? Click on that image and check out the URL. Every time you want to Coral Cache something simply add .nyud.net to the domain name. That’s all there is to it.

If you do it manually for all your larger images you get a fine grained control on which sections of your website are cached and which are not. On the other hand, this approach can become difficult to maintain. If one day you decided you no longer want to use Coral Cache you will have to search and change the URL for each image.

What you really want is to have normal URL’s for all the locally stored files, and then use mod_rewrite to automagically alter all of them to use Coral Cache. For example, if you are like me and you usually upload image files via the browser using WordPress’ default upload dialog you can use the script below:

RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx
RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$
RewriteRule wp-content/uploads/(.*)$ http://www.yoursite.com.nyud.net/blog/wp-content/uploads/$1 [R,NC]

The first two lines are recommended on the Coral Cache wiki. The last line does the actual rewriting.

Put this in a .htaccess file in the root directory of your blog and observe the magic. The first time you load a page it might take a bit longer than usual, but with high traffic, this trick should quickly pay off. I’m not sure how often Coral flushes their cache, but if you have a low traffic site, this might actually slow down the loading a bit if the time between visits is larger than the lifetime of a file in the cache.

Of course the best part about using url rewriting is that you can easily switch it off at any time. When the digg effect subsides, or traffic goes down significantly, and you no longer need the cache you can simply comment out these 3 lines from your .htaccess file. In fact, you can just commend out the last line, as the two conditions can’t really do any damage by themselves.

Note that side effect of this trick is that you no longer need to worry about the hotlinking issue. Since all images will be redirected to the Coral Cache the lazy assholes can hotlink to you all they want. In fact you might as well stick the following two lines right after the Coral Cache conditionals:

RewriteCond %{HTTP_REFERER}!^$ 
RewriteCond %{HTTP_REFERER}!^http://(www\.)?yoursite\.com.*$ [NC]

First line excludes requests without a referrer, while the second one excludes the local requests. You may also want to add few lines here that will exclude Googlebot and other similar services which you may or may not want to index your images. But proper hotlinking prevention strategies are a a topic for a whole new discussion.

Then, when your server gets dugg comment out all the hotlinking related lines, and you go back into ultra-bandwidth saving mode.

Also, please feel free to correct or improve my mod_rewrite code. It is definitely not perfect, but it is functional. I would greatly appreciate any input or constructive criticism here. :mrgreen:

Update 08/17/2007 06:24:31 PM

Damn it! Phil Haack beat me to it. I wrote this post on the 14th and 15th completely independently from Phil. I was also inspired by the Yslow plugin to do some research on this topic.

Of course I decided to queue this post to appear on Monday the 20th (because I had some other stuff in the pipe) while Phil posted it right away. Oh well…

Update 08/20/2007 11:40:34 AM

Oops. It seems that Coral Cahce wiki got vandalized. Too bad. Let’s hope they get it fixed soon. I tried to restore it back to normal, but it seems that it has been edited by spammers so many times their wiki software “forgot” the good article already. :(

Also it’s probably worth mentioning that this caching method should mostly be used for heavy lifting during immense traffic spikes. Using it to cache your images on every-day basis is probably not the best idea as it might and will slow down your page – see Phill Haacks post.

Also, imagered.com is another valid alternative here. You could use the same techniques outlined here to cache your images with their site.

[tags]bandwidth, hotlinking, coral cache, mod_rewrite, nyud.net, cache, cacheing[/tags]

This entry was posted in programming and tagged , . Bookmark the permalink.



6 Responses to Saving Bandwidth and Preventing Hotlinking With Coral Cache

  1. Ricardo INDIA Mozilla Firefox Windows says:

    Hey Luke, sorry for the off-topic but this is just to let you know that I finally had enough courage to setup a blog in English… ;-)

    Reply  |  Quote
  2. Luke UNITED STATES Mozilla Firefox Windows says:

    Nice! :) I went and posted a comment. :)

    Reply  |  Quote
  3. Sameer CANADA Internet Explorer Windows says:

    Dude, your images are not loading for me – nyud.net “No answer from the server”

    Reply  |  Quote
  4. Ricardo INDIA Mozilla Firefox Windows says:

    It is working for me…

    Reply  |  Quote
  5. Luke UNITED STATES Mozilla Firefox Ubuntu Linux says:

    Yeah, the Coral Cache can be funky sometimes. See the linked post by Phill Haack for details.

    It’s working for me now too btw.

    Reply  |  Quote
  6. dennyhalim INDONESIA Mozilla Firefox Windows says:

    try automatic redirecting all readers came from slashdot/dugg/fark/etc…..

    http://wiki.dennyhalim.com/htaccess

    Reply  |  Quote

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>