Compress JavaScript and CSS without touching your application code
by James
UPDATE: Safari doesn’t seem to play well with this — I’ve modified the code so that Safari is served regular uncompressed files. Also, I realised that the code went about things in a slightly roundabout way, so I’ve shuffled it around a little.
What with the fact that optimising Sylvester for speed is causing its brief, legible internals to balloon into reams of hard-coded arrays and unrolled loops, and the fact the fact that Prototype’s latest release has put on a few extra chins (1.5.1 is 97kb compared to 1.5.0’s 70kb), I thought it was about time I figured out how to serve my JavaScript like a grown-up. Most of what’s written online seems to be focused on specific platforms (PHP, mostly) and I thought something a little more generally useful was needed.
The general approach to gzipping your web content seems to run as follows:
- Check the user agent will accept gzip-encoded content.
- Have your application code filter its output via a gzip function, or have Apache do this for you.
- Fiddle around with .htaccess to add the correct content type.
The problem with doing all that is that, well, you have to do all that. Compressing assets on the fly is probably not the most efficient way of doing things, and I’m not a huge fan of having your application deal with content delivery — the server should be doing that.
There is a way to have all this handled in a few lines of .htaccess, providing you put in a couple of minutes effort compressing your files by hand. Let’s say you have prototype.js sitting on your server. Gzip it (use 7-zip or similar if you’re running Windows) to give you a file called prototype.js.gz. Stick this in the same directory on your server. If you’re comfotable with the command line, you can just SSH into your box and do, for example:
gzip foo.js -c > foo.js.gz
Then, add the following to .htaccess in your document root (preceed this with RewriteEngine On if you don’t have it somewhere already):
AddEncoding gzip .gz
RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{HTTP_USER_AGENT} !Safari
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.*)$ $1.gz [QSA,L]
The first line tells the server that files with .gz extensions should be served with the gzip encoding-type, so the browser knows what to do with them. The second line checks that the browser will accept gzipped content — the follwing lines will not be executed if this test fails. We exclude Safari as it doesn’t interpret the gzipped content correctly. We check that this gzipped version of the file exists (fourth line), and if it does, we append .gz to the requested filename.
With this in place, all you have to do is upload gzipped copies of your files to your server and Apache will serve whichever version of the file is most appropriate — you don’t have to change your <script> tags or any application code.
One final platform-specific extra for those of you on Rails: I cannot recommend AssetPackager highly enough. The base package for an app I’m currently working on includes Prototype and various commonly used widgets of mine, some of which are packed. The package totals 125kb as separate files, 99kb using AssetPackager, and a measly 26kb when you throw gzip into the pot.
Comments
very useful, but you piqued my curiousity– why do you “pack” some of your javascript but not others? and if you do “pack” some, how do you manage development/production cycles with this? i.e. if you do development with packed files, you’d have to edit the unpacked version and then pack for every single change you wanted to test (very annoying). otherwise if you do development with unpacked versions, you have to use some method to get production to use the packed versions, as well as manually creating the packed version on every production push. (also annoying). run into this issue? any ideas? maybe worth another article..
What I tend to do when writing JavaScript is just write unpacked while making significant changes. I write code that I know will pack nicely, although I sometimes pack up what I’ve written every so often just to make sure. Usually though, I’ll only make a packed copy once I’m done making a whole set of changes to a file. It might be nice to automate the task but I really don’t find it that onerous.
As regards automating deployment, I did write about getting Capistrano (and AssetPackager) to compress everything to gzip format when you deploy code. I’ve been considering porting Dean Edwards’ packer to Ruby so I could incorporate it into my deployment process, although I’ve not the time at the moment. Plus, it can be a bad idea to rely on automated packing, as it can break your code if you’ve missed a semicolon or curly bracket somewhere. Auotmated gzip is fine, and, if you could run ShrinkSafe or some other packer built on a JS engine rather than regular expressions, that would work fine too. For the moment, I’ll stick with making and testing my packed code by hand.
For some reasons I’ve failed to correctly setup -f rule (may be due to all the other RewriteRules, don’t know). So my solution is the following. First part redirects to normal files if browser doesn’t support gzip or is Safari, second part delivers correct gzip content to browser. So I also have two versions of the file: gzipped and normal, on page always gzipped version is called. Also I may change the “version” of the file (foo.js?v3.54), all works correctly.
AddEncoding gzip .gz
RewriteCond %{HTTP:Accept-encoding} !gzip
RewriteRule ^(.*)\.gz(\?.+)?$ $1 [QSA,L]
RewriteCond %{HTTP_USER_AGENT} Safari
RewriteRule ^(.*)\.gz(\?.+)?$ $1 [QSA,L]
AddType text/javascript .js
AddType text/css .css
ForceType text/javascript
Header set Content-Encoding: gzip
ForceType text/css
Header set Content-Encoding: gzip
second part
<FilesMatch .*\.js.gz$>
ForceType text/javascript
Header set Content-Encoding: gzip
</FilesMatch>
<FilesMatch .*\.css.gz$>
ForceType text/css
Header set Content-Encoding: gzip
</FilesMatch>
James, what’s the problem with Safari’s gzip handling? Do you have any more details?
I think it’s legacy bug of the KHTML engine becase Konqueror also has such troubles (and it should be added to RewriteRule User Agent Condition)
Maybe this will save some others time. If you go with dreamwind’s method, make sure you have mod_headers enabled in apache. I had to recompile with the ‘–enable-headers’ to get it to work.
Great article. Solved my problem perfectly.
hey,
I am using Rails. I copied these rules:
AddEncoding gzip .gz
RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{HTTP_USER_AGENT} !Safari
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.*)$ $1.gz [QSA,L]
at the end of htaccess file. And I restarted httpd. I have a gipped version of prototype.js.gz in public/javascripts/ as well.
But I am not seeing compression happening although normal html file is getting compressed.
Am i missing something here?
[...] section below enhances the configuration suggested by The If Works [...]
note for newbies like me:) mode_headers has to be enabled (this line LoadModule headers_module modules/mod_headers.so in apache conf) for
ForceType text/javascript
Header set Content-Encoding: gzip
ForceType text/css
Header set Content-Encoding: gzip
great it work
i mean for second part of dreamwind
awesome stuff! Got it working in our ec2onrails setup in half an hour and it dropped load times for our js 50%
some servers have problem with this solution as mentioned here for example:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=503069
some servers treat rewriten path as absolute and looks for out .js.gz file in system dirs.. to fix that just add “/”
Complete (Fixed) Working Solution for JS/CSS gzipped files:
RewriteEngine On
AddType “text/javascript” .gz
AddType “text/css” .gz
AddEncoding gzip .gz
RewriteCond %{REQUEST_FILENAME} \.(js|css)$
RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{HTTP_USER_AGENT} !Safari
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.*)$ /$1.gz [QSA,L]
Hi Guys,
This script works great first of all, since I’ve been able to pack down jscript to only 62KB. Where this breaks is if you have image references within the javascript, so URLs like “/images/logo.png” don’t come across.
Is there anything that needs to be done for enabling the images?
Thanks,
Sjs
Google Chrome 1.0 supports this function.
Newly released Google Chrome 2.0 does NOT support this function.
You should exclude Google Chrome 2.0 as well as Safari.
i’m using this code so if the gzip version of the file dosen’t exist apache does it on the fly.
———————————–
AddType “text/html” .gz
AddEncoding gzip .gz
AddType “text/javascript” .gz
AddEncoding gzip .gz
AddType “text/css” .gz
AddEncoding gzip .gz
RewriteEngine on
ReWriteCond %{HTTP:accept-encoding} gzip
RewriteCond %{HTTP_USER_AGENT} !Safari
ReWriteCond %{REQUEST_FILENAME} !^.+\.gz$
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.+) $1.gz [QSA,L]
AddOutputFilterByType DEFLATE text/html
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
—————————–
how can i exclude these:
Netscape 4.06-4.08
IE6
chrome 2.0
like safari?
RewriteCond %{HTTP_USER_AGENT} !Safari
I recently wrote a mod_perl output filter which sits inside Apache. It intercepts requests for .css files and then “compresses” them on the fly before sending. It’s not gzip compression, what it does is strip whitespace, comments, newlines etc. Check it out here: https://secure.grepular.com/blog/index.php/2009/10/28/compressing-css-on-the-fly/