dean.edwards.name/weblog/2007/08/js-compression/

Notes on JavaScript Compression

The most efficient way to serve compressed JavaScript is to first run your code through a JavaScript compressor that shrinks variable and argument names, and then serve the resulting code using gzip compression.

These JavaScript compressors are capable of shrinking variable and argument names:

Both the YUI compressor and Packer use scoping wisely so that variable names are substituted with “a”, “b” and “c” etc all the way down. This produces patterns that lead to more optimal gzip compression rates.

Packer

If you can’t use gzip compression for some reason, then Packer with base62 encoding turned on, is the next best thing. Scripts compressed with Packer are unpacked using one pass of a RegExp on most platforms. However, this still introduces an overhead for larger scripts. The “Decode” button on the Packer page will tell you how long the decode time is, typically 0 – 200 milliseconds. The next release of Packer will improve the decode time by about 10%. Compressors with more complex compression algorithms can have very high decode times.

The strictness of Packer trips some people up, especially function declarations requiring a terminating semi-colon. This is not a habit for many JavaScript programmers. In this case I recommend using Julien Lecomte’s compressor first and then running the code through Packer for base62 compression. Remember that base62 compression is only necessary if you can’t use gzip.

Update: Check out CompressorRater!

Comments (47)

Leave a comment

You forgot the qooxdoo generator. It is also capable in renaming variables to shorter ones like the other tools listed. The generator is a python script which is easily installable cross-platform (just require a standard python installation). As it is based on a full-blown tokenizer and treegenerator it also safely supports semicolon-less code and regular expressions – areas where others often fail. Another great feature is the string optimizer which lay out strings to a centrally stored array. This dramatically improves the runtime performance in IE. For further details have a look in the corresponding documentation.

“it also safely supports semicolon-less code and regular expressions”

The page you link to says “The current implementation automatically removes comments, new lines, indents and other additional whitespaces from your files. It keeps advanced code fragments like regular expressions, strings, … You just to make sure to add semicolons after each command.

  • Comment by: Observer
  • Posted:

I’d love to see a comparison matrix among these three stripping libraries, various code bases and sizes, and their resultant gzip compressed sizes. I’ve been using Dojo’s ShrinkSafe (since it’s as agnostic about my syntax as a browser), and it does variable renaming with an _hexadecimal strategy. I’m curious how it stands against Packer and YUI compressor. I can offer my assistance with the ShrinkSafe part.

  • Comment by: Kris Kowal
  • Posted:

@Kris – This partly addresses your curiosity:

http://www.julienlecomte.net/blog/2007/08/21/gzip-your-minified-javascript-files/

  • Comment by: -dean
  • Posted:

What was the problem with GZIP’ing JavaScript again? It breaks in IE, right?

  • Comment by: Johann
  • Posted:

@Kris – And this completely addresses it:

http://compressorrater.thruhere.net/

  • Comment by: -dean
  • Posted:

@Kris

Compressing JavaScript is another test that compares cruncher performance (/packer/, JSCD and MemTronic).

  • Comment by: Johann
  • Posted:

@Johann – There are problems with gzip in MSIE. I’m not sure how they affect JavaScript:

http://support.microsoft.com/default.aspx?scid=kb;en-us;823386&Product=ie600

Can someone else comment on this?

  • Comment by: -dean
  • Posted:

Note: there are some trap in the shrinking of variable name. We should pay attention to the “eval” and “with” statement. In generally, “eval” may refer to any variable in the outer functions which make all those names unchangable, and all var name in the scope of “with” statement can not be shrink, because it may refer to the property of the withObj.

Some compression tools have problems in eval and with, so please consider JSA — a great product form my friend Jingdw. http://www.xidea.org/project/jsa/

  • Comment by: hax
  • Posted:

FYI: I have a LZ77-algorithm-based JavaScript compressor: http://demo.java2script.org/lz77js/

In most cases, this compressor compresses JavaScript at the ratio of 40%~60%. It can be used as a general compressor with JavaScript options turned off.

It is not designed to compressed large JavaScript (>200k). Large JavaScript’s decompressing takes too much time on concatenating the original string. gzip is recommended for large JavaScript.

@Observer: Yes, this is not really needed in nearly any cases. Infact I don’t know about a case where a missing semicolon results into problems. It is always better to have some. This is more or less meant as a suggestion.

I’ve no problem with the gzip in IE. And yes gzip is the best for larger js.

[…] Dean Edwards: Notes on JavaScript Compression Both the YUI compressor and Packer use scoping wisely so that variable names are substituted with “a”, “b” and “c” etc all the way down. This produces patterns that lead to more optimal gzip compression rates. (tags: javascript performance compression gzip) […]

A feature request I have for all JS compressors would be the ability to upload multiple files so that variables declared in one may map up to variables defined in other files when the compressors automatically rename the variables.

Yes, I know that for optimal compression one should have only one JavaScript file per page, but this is not always possible. Where I work we use the Prototype Window Class, which creates stylized iframes to give our popup an XP-like look. These are instances where an iframe will reference a variable defined in the main window using top, which would obviously be renamed to a different variable name if we ran each JS file through the compressor on an individual basis.

  • Comment by: Brian LePore
  • Posted:

I use a lint-tool to check for comma’s. I know of two:

The latter is integrated in my text editor (TextMate), and every time I save it comes up with a tooltip when there are errors or warnings. It’s called Javascript Tools. It should also work with the Windows E editor for the Pl3b31an hackers;-)(no offence intended, just joking).

(not advocating any editor: it probably can be integrated in any editor, except notepad.exe, so take your own pick)

  • Comment by: Doekman
  • Posted:

Anyone using Maven who needs JS compressed could use this: http://sourceforge.net/projects/maven-js-plugin

I’ve used it for a few projects and it’s worked well.

  • Comment by: Adam
  • Posted:

A reddit discussion about gzip compression and JavaScript:

http://programming.reddit.com/info/2jkbz/comments/c2jnox

  • Comment by: -dean
  • Posted:

[…] Mit der Kompression von Javascripten habe ich in letzter Zeit ein wenig Erfahrung gesammelt, aus gegebenem Anlass sozusagen. Dean Edwards sammelt hierzu nocheinmal die Fakten, und hat ein paar gute Tips parat. Es bleibt aber eine Krux: Komprimierung ist super, besser ist es jedoch, von Anfang an mit kleinerem Datenaufkommen arbeiten zu können (das kann man dann immer noch komprimieren). Mit dem Mißverständnis, seit »Web2.0« könne man mit ein wenig Javascript auf Webseiten so ziemlich alles veranstalten, haben zwei wesentlich Fehler Einzug in die Aufgabenstellung am Frontend genommen: […]

@12

Some gzipped Files don’t working in IE. Then a failure message appears and the window closes.

  • Comment by: Rene
  • Posted:

I pulled together this nice simple script that is based off the PHP port of the packer, and the web application demo that comes with YUI-Compressor, to make something that i dubbed, “yui-packer”.

It’s a simple script, that runs your scripts through the yui compressor which magically compresses your scripts (to some extent) before running it through the packer algorithim.

http://antimatter15.110mb.com/misc/yuipacker.zip

Another idea for making packer “safer”

eval(“function(){“+script+”}”).toSource().substring(11)

i just tested it on opera, ie, and safari, and it only works on firefox.

function safescript(script){return String(eval(“function(){“+script+”}”).toSource().substring(0,eval(“function(){“+script+”}”).toSource().length-1).substring(13))}

[…] Dean Edwards: Notes on JavaScript Compression (tags: javascript compression performance tools gzip web programming ajax **) […]

@Johann There are two basic problems that one might encounter:
– HTML pages may only partially appear, or the pages may not appear at all.
– Your HTTP connection may stop responding.

  • Comment by: Sebie
  • Posted:

@antimatter15:

The following safeScript() function I’ve put together does what I think you were aiming for and is fully cross-browser (including IE 4 & NS 4):

function safeScript(code)
{
      return (/\(.*\)\s*\{([\w\W\s]*)\}/).exec(new Function(code).toString())[1];
}

Nice work, but it’s got a flaw (though not an outright bug). This version of Packer does not correctly handle for/in syntax, like the following:

for (key in data)
    // do something with key

“key” is not recognized as a new variable name and left as is.

Nice tips.

I only moderately use JavaScript. Do you normally compress all of your code on only when it reaches a certain size?

Seems like compression would add more overhead then its worth with smaller files.

  • Comment by: ADAC
  • Posted:

Hello Dean,

I have used MooTools compressed with your packer using eval. Now I experienced Unresponsive script prompts occasionally on my site in FireFox. Since I just used the variable shrinking and remove unnecessary whitespace/comments (without eval), I have not since seen the error.

So does this mean eval can mean longer execution times, even though the file is smaller?

Thanks!

Please check out jsutility.pjoneil.net for a set of JavaScript development tools. It will compact and compress JavaScript substantially better than any other tools mentioned in these articles.

It also has a number of useful tools for the Javascript developer and it’s completely free.

Regards Pat

  • Comment by: Pat
  • Posted:

I owe all of you an apology. The web site jsutility.pjoneil.net was not ready for prime time when I made my original post. There were simply too many problems.

I have fixed all of the problems reported and have successfully tested against a number of frameworks. Hopefully, it is more stable now. I have posted the results on my web site.

Thanks for your patience. Pat

  • Comment by: Pat
  • Posted:

[…] Nicked this from Dean Edwards’ blog entries, but thought you’d all want to see it! […]

You say those three packers are capable of shrinking variables, but when inputting a simple line of javascript:

function t() { var blah=4; }

None of them (not even packer 3.0!) shrink the “blah” variable?? I would expect it to become:

var a=4;

Or something like that…

  • Comment by: Thany
  • Posted:

@Thany – Packer does shrink the blah variable. The output is:

function t(){var a=4}
  • Comment by: -dean
  • Posted:

Jordy/Jediknil.

Do you mean that it’s not rewriting the key variable to a shorter (a) variable? In that case, it is because you’re not using “var” to create the variable. As such the variable you’re making is global, and therefore not shrunk. A variable defined in the start of a for loop is also accessible to the rest of the script on its level, so something like:

var test = [0,1,2];
for(var key in test) {};
alert(key);

Will result in an alert with a 2.

In my experience, I’ve not come upon a case where Packer appears to handle these incorrectly. But then I’ve not used it that extensively.

  • Comment by: danielP
  • Posted:

[…] Dean Edwards hat bereits vor einiger Zeit einen sehr interessanten Artikel zur Javascript Kompression geschrieben. Ein guter Javascript Kompressor arbeitet folgendermaßen am effizientesten: […]

Hi,

I am in the process of creating a shrinker of javascript in php. I have some problems to find the difference between a regular expression and operator of a division.

A simple example : if( a /b>1)a=a/b;

How I can to know /b>1)a=a/ is not a regexp ?

Should I the find the char after this (if is not a comment) and, if it’s not a quotes, number or var, I accept this as a regexp ? or is there a way shorter ?

  • Comment by: XoraX
  • Posted:

Another website using dean edwards packer algorithm: http://www.compressjavascript.com

maybe a bit late, but the following piece of code is not being compressed:

var TA=new Array();
var D=document;
var W=window;
var M=Math;
var z;
var y;
var x;
var w;
var h;
var f;
var s;
var t=0;
var C;
var o;
var b=0;
var g=0;

I expect it to become something like:

var TA=new Array(),D=document,W=window,M=Math,z,y,x,w,h,f,s,t=b=g=0,C,o;
  • Comment by: Woudloper
  • Posted:

@Woudloper – what you expect and what you get are not always the same thing.;-)That cheeky remark aside, it’s not a bad idea. I’ll consider it.

  • Comment by: -dean
  • Posted:

Great code! I just did some test. I was looking for best compression / obfuscation. I tested the following freeware apps:

1) ObfuscateJS v0.3.6

2) ShrinkSafe

3) yuicompressor v2.3.5

4) packer2.net

5) packer2.wsh

6) http://dean.edwards.name/packer/

Following are the results (ajax.js is original file):

02/25/2008 01:19 PM 119,574 ajax.js

02/25/2008 01:49 AM 87,323 ajax1.js

02/25/2008 01:51 AM 92,522 ajax2.js

02/25/2008 02:21 AM 84,519 ajax3.js

02/25/2008 01:24 PM 39,330 ajax4.js

02/25/2008 01:26 PM 39,331 ajax5.js

02/25/2008 01:21 PM 43,198 ajax6.js

ajax.js is pretty huge JavaScript file with 3294 lines. All packages worked perfectly. Best results were achieved with packer2.net and packer2.wsh.

Great work Dean and people that did the porting!

  • Comment by: no1
  • Posted:

here is a javascript compressors comparison. http://cssgallery.info/javascript-compressor/
it takes mootools v.1.11 and compress with different tools
hope is helpful
rborn

  • Comment by: rborn
  • Posted:

I used to be one of those people that wrote code with lines of 100+ characters long. Just putting all on one line so that every line break would be preceded by a semicolon. Recently I finally bought the argument of that being a bad habit because of readability;-)

So now my scripts contain multiple string concatenations that could be merged easily saving a hefty (ahum) 3 bytes each. Perhaps packer 3.1 could support this:-D

Example:

var blah = ‘Some string that spans ‘ + ‘multiple lines but could be ‘ + ‘merged into one long string when ‘ + ‘packed…’;

To become:

var blah = ‘Some string that spans multiple lines but could be merged into one long string when packed…’;

Thanks

  • Comment by: Marijn
  • Posted:

Does anyone have any experience with Performance Testing an online site with new code that is now compressed JS? Should I be expecting Performance that is alot better than before?

Are there any other ways I can improve Performance without buying new hardware?

Thanks Felix

  • Comment by: Felix
  • Posted:

I have compressed a js file using dojo shrink safe. Is there a way I can decompress it to get back the code in it’s original form???

Thanks in advance

  • Comment by: Ram
  • Posted:

[…] are several public algorithms for minifying Javascript, and Dean Edwards gives a nice overview of those on his […]

IE indeed comes across with problems with shorter chunks of code. There is also a handy compression tool http://jscompress.com/ You can compress by means of Packer or JSMin. Also, it is able to put together many files into one apart from taking direct code input.

  • Comment by: Marius
  • Posted:

Packer only compresses the first variable declared by a var statement.

example: function test(){var MyFirstVar, MySecondVar, MyThirdVar}

becomes: function test(){var a,MySecondVar,MyThirdVar}

  • Comment by: Peter Holtan
  • Posted:

Comments are closed.