dean.edwards.name/weblog/2007/04/packer3b2/

Packer version 3.0 beta 2

A couple of bugs in the last release of packer. If you were experiencing problems then please try the new version.

Comments (51)

Leave a comment

Hi,

I often have problems with my CVS when checking in packed JavaScript. They complain about at least 1 line in the document being too long.

Could you add the option to ‘wrap’ the output?

Thanks for this great script!

  • Comment by: Jeffrey Ridout
  • Posted:

Thanks! I have a few questions.

1. Packer v3 relies on base2 so it’s not as easy as before to make a WSH port. Are you planning to release an “official” WSH port or a rhino port?

2. We can pack the code directly or rhino it first. Which do you recommend?

3. Missing ; errors break the packed code but are hard to find in source code. Is it possible for packer to complain and tell the line number?

I also noticed that V3 is much slower than v2 in WSH.

  • Comment by: Arrix
  • Posted:

@Arrix

  1. I’ve written Rhino and WSH ports already. I’ll make them available for download soon.
  2. I assume you mean Shrinksafe? With packer’s new variable shortnening option you don’t really need ShrinkSafe
  3. No.
  4. Yes it is a bit slower. Does that matter?
  • Comment by: -dean
  • Posted:

Javascript – versione 3.0 beta 2 del packer di Dean Edwards Come si legge dal blog di Dean Edwards, è stata rilasciata una nuova versione beta del packer. Risolti i bug verrà rilasciata la versione 3 finale. Per chi non lo sapesse il packer permette di rendere uno script javascript più vel

Thanks for the release to get rid of the bugs! What are the plans for the next version?

  • Comment by: devmag
  • Posted:

Dean, The reason why I use custom_rhino (Shrinksafe) is that the code processed by custom_rhino is standard and strict so it’s less likely that the packed code will fail. Custom_rhino also fixes all the missing ; errors.

The slight speed drop does not matter at all when you think of all the benefits!

  • Comment by: Arrix
  • Posted:

@Arrix – If Custom_rhino adds missing semi-colons (which I don’t think it does) then use it. Otherwise there is no point.

  • Comment by: -dean
  • Posted:

I’m not sure. I downloaded custom_rhino.jar from the dojo site. It really does tidying up the code for me – adding missing semi-colons, adding line-breaks and even adding curly braces for if blocks, etc.

Here is my usage:
test.js
————–
a = 1
b = function(){}
if (true) a = b
————–
Run the command in cmd (Windows)
java -jar custom_rhino.jar -c “test.js”

The output is
————–
a=1;
b=function(){
};
if(true){
a=b;
}

Now I’m trying to use packer directly to achieve best compression rate.

  • Comment by: Arrix
  • Posted:

@Arrix – I didn’t know that Shrinksafe added missing semi-colons.:-)In that case you can run through Shrinksafe first and then packer. Packer will remove any unnescessary semi-colons anyway. The best of both worlds.

  • Comment by: -dean
  • Posted:

I covet a tasty command-line version of packer once more. Covet, I say.

  • Comment by: Seth
  • Posted:

@Seth – As soon as packer3 is out of beta I will release the command line version. I haven’t had any bug reports since the last beta. So fingers crossed. Another week or so of beta then I will release the WSH and Rhino versions.

  • Comment by: -dean
  • Posted:

Is the ;;; feature working correctly?

Running packer 3.0 beta 4 on your website.

———-
function test() {
var asdf;

;;; alert(“debug”);
alert(“incorrect password”);
}
———–
function test(){var asdf;alert(“debug”);alert(“incorrect password”)}
———–

Sometimes it did correctly remove the debug alert though. I was just testing the feature since I was consider using it.

  • Comment by: david_kw
  • Posted:

works with Prototype 1.5.1_rc2. Using the short var name packed version and then gzipping it. I got the filesize down to 15.4kb which is awesome considering it was 91.8kb.

  • Comment by: j
  • Posted:

What about .htc files ?

le body is in javascript (or jscript). It will be usefull to have packer running directly whith a .htc file ?

  • Comment by: Eric
  • Posted:

Dean,

I’m having problems compressing Lytebox. Apparently there is a semicolon missing in the generated output.

  • Comment by: Johann
  • Posted:

I guess we need to shrink the function names too, we can alert user for new function names in a comment or something so he changes it in other files.

If you want better compression ratio for base62 encoding you can replace the

var WORDS = /\w+/g;

with this:

var WORDS = /(function|if|for|while)\(|(else|\))\{|window\.|return |\w+/g;
  • Comment by: Pusztai Tibor
  • Posted:

Only issue i met:

name.prototype.fn=function(){
/*...*/
}
if(/*...*/){/*...*/}

semicolon mising. it’s correct. i fixed all my objects in a minute, but anyway it can be added automaticaly – it’s the only issue.

If it’s hard to find a place of error in packed file I have an advice: get Microsoft cscript attached to SciTE.

command.go.*.js=cscript /nologo $(FileNameExt)
and F5 produce
>cscript /nologo _obf.den.js
D:\ MyDocs\***\js\_obf.den.js(1, 53674) Microsoft JScript compilation error: Expected ';'

I press Ctrl+G and go to that column. With bracket highlightning feature of SciTE its easy to find any error.

  • Comment by: DPP
  • Posted:

Oh sorry. I’ve found talk about that missing semicolon here.

  • Comment by: DPP
  • Posted:

Hi,

Thanks for this great script. I use it on my project to speed up loading. And that is how I found a bug in version 3 (beta 8 ) and I was not sure where to post it so I’ll just put it here. When using the “shrink variables” option, stuff like:

var path = options.path ? '; path=' + options.path : '';

gets converted to:

var f=c.path?'; path='+c.f:'';

Obviously, c.f does not exist and you get an ‘undefined’.

  • Comment by: Stephan
  • Posted:

@Stephan – this is fixed now.

  • Comment by: -dean
  • Posted:

[…] As you can see in the build file I’m executing ‘cmd’ with some parameters to pack the sol_uncompressed.js file. I used the WSH version of packer. This version can be executed from the commandline. It seems this WSH port of packer is not yet version 3.0, but Dean said on his blog the ports of the latest version are ready to be released. […]

Hi Dean, I was wondering if the WSH port is already at version 3.0. I wrote an article on how to automate the packing of javascript files using Apache Ant. I use the WSH port because this one can be executed by Ant through the commandline.

If you wanted to use this to make code hard to read …is there a way around someone just putting document.write instead of eval in the packed code and thereby unpacking the code?

  • Comment by: David
  • Posted:

Cool

  • Comment by: Stephan
  • Posted:

Hi Dean, I’m a bit late to the game, but I’ve recently been using your excellent packer and came across the following problem:

When I use the following valid input:

   function forTest()
   {
      for(;;);
   }

Packer returns invalid code:

function forTest(){for(;;)}

As you probably know, it doesn’t matter if I use Base62, but for your debugging purposes this Base62 output is invalid as well when evaluated:

eval(function(p,a,c,k,e,r){e=String;if(!''.replace(/^/,String)){while(c--)r[c]=k[c]||c;k=[function(e){return r[e]}];e=function(){return'\\w+'};c=1};while(c--)if(k[c])p=p.replace(new RegExp('\\b'+e(c)+'\\b','g'),k[c]);return p}('0 1(){2(;;)}',3,3,'function|forTest|for'.split('|'),0,{}))

The problem is that Packer strips the trailing semi-colon after the for(;;); which creates a syntax error when evaluated.

This sample is trivial, but it demonstrates something that becomes a real problem with more code.

Keep up the great work. It’s fun reading about and trying your creations:-)

@Michael – I realise that for(;;); is valid JavaScript but is anyone going to actually write any empty for loop?

  • Comment by: -dean
  • Posted:

@Dean – Maybe I over simplified the example, of course empty for loops don’t serve much purpose;-)

Here’s a more realistic, though verbose example:

function disableAllLinks()
{
   var links = document.getElementsByTagName("a");

   for(var i = links ? links.length : -1; --i >= 0; links[i].disabled = true);
}

When packed it becomes the following invalid code:

function disableAllLinks(){var links=document.getElementsByTagName("a");for(var i=links?links.length:-1;--i>=0;links[i].disabled=true)}

@Michael – OK. I see what you mean. To avoid this bug, code the loop like this instead:

for(var i = links ? links.length : -1; --i >= 0; links[i].disabled = true) continue;

I’m still not sure that this is worth writing a specific fix for.

  • Comment by: -dean
  • Posted:

@Dean – It’s your call on whether your packer tool should handle this scenario, but packer is widely used and its otherwise high quality output is put in question by this one shortcoming.

I’ll make one last effort to convince you of the value in properly handling the described scenario, hopefully you’ll come around ;-):

This bug kills the code integrity expectation in point #2. Anyone who tries to pack a script containing code with this empty statement for/while loop technique will be sorely disappointed after possibly spending hours trying to debug a problem caused not by them, but a tool that is normally very effective.

Is this bug really that difficult to fix?

Is this bug really that difficult to fix?

Yes!:-)

Potentially anything can appear between the brackets of a for loop:

for (var i=0, j=next(i); obj.test(fetch(j),/(hard[\(regexp])/); i++);

It is as complex as the variable shrinking option. It’s not impossible but I just don’t think it is worth the effort. Use continue instead.;-)

  • Comment by: -dean
  • Posted:

Minor issue: Packer does not treat unescaped “/” characters within character classes within regex literals as literal characters. Examples:

Original: x=/[///]/g;
Packed: x=/[/

Original: x=/[/\//]/g;
Packed: x=/[/\

IE 7, Firefox 2, Safari 3, and Opera 9 all return true for the code /[/]/.test("/");.

These examples are contrived, but there might be cases where it has potential for impact with non-contrived scripts. I don’t have any knowledge of Packer’s inner workings, so I’m not sure.

@Steven – Fair enough. You’ll just have to escape that character.

  • Comment by: -dean
  • Posted:

Is the JavaScript version of this available for download as a package or just the ports? I’ve looked at the downloads page a couple of times and can’t find it, though admittedly I can kind of blind about obvious things sometimes. Regardless, love the work you do here great stuff.

  • Comment by: Bryan
  • Posted:

@Bryan – I’ll build a downloadable bundle soon.

  • Comment by: -dean
  • Posted:

Great Tool. But I think I found a bug, though I don’t know if anyone would write such JavaScript Code (because it would have no sense I think): If a regular expression is at the beginning of a line (with only whitespace before it), it is not preserved:

/abc /;

results in

/abc/;

  • Comment by: Christian S.
  • Posted:

@Christian – parsing RegExps is a real pain. I think I’ve got it 99.9% right. The remaining 0.1% are unrealistic cases like the example you showed and the example in comment 32.

  • Comment by: -dean
  • Posted:

Hello, any update on the Rhino version? Thanks. I have managed to get javascript compression down to a fine art using custom_rhino, a few scripts to merge and import scripts, and the rhino version of packer. 311Kb of JS in 17 files down to 88 KB in 2.:)

  • Comment by: Kroc Camen
  • Posted:

@dean – I noticed that Packer is no longer in beta, so I prettied up a .NET GUI to use to obfuscate my js files. I saw while browsing the source that you created a WSC for Packer, so I decided to use (and modify) it and include it in the app as an ActiveX component. Works pretty well as a server-side on-demand obfuscater that way too!:D

Anyway, here it is: http://beta.trakit.ca/packer3.zip

If you have any issue with the way I’ve packaged and/or modified your original source, just let me know and I’ll change it. You can serve it from your site, or I can delete it from my server if you don’t like it… your call. I just like making tools and giving them away free!

ps; http://www.opensource.org/licenses/mit-license is 404, http://www.opensource.org/licenses/mit-license.php is 200

  • Comment by: Alex Lein
  • Posted:

@Alex – great work! Do you mind if I include this in the Packer 3 downloadable bundle? No answer signals consent.;-)

Thanks for the heads-up regarding the license URL.

  • Comment by: -dean
  • Posted:

@dave – yeah, that’s why I wrote it. I don’t have my own permanent webhost at the moment, so if you can host it yourself, please do so.

I found a tiny UI bug (ctrl+A doesn’t work) but other than that, its been working for me just fine! (still going to try to resolve that bug though)

  • Comment by: Alex Lein
  • Posted:

@dean – I think I just called you dave… it was early, and I was tired.

  • Comment by: Alex Lein
  • Posted:

Hi, I notice the first line of packed scripts start with “eval(function(p,a,c,k,e,r)”

I was wondering if it would be possible to change the script so that that line could be made a bit more random/obfuscated so that it cannot be easily determined which packer (this one) was used?

  • Comment by: cameron
  • Posted:

@dean – I fixed the “select all” bug and added shortcut keys (crtl+O, crtl+S). It’s done. I don’t think I need to spend any more time on it myself, but if anyone wants, the source is included in the zip. It does everything you’d want, but anyone is free to use or hack the code as they see fit.

Same filename, new package: http://beta.trakit.ca/packer3.zip

The previous package was here in case there are any problems with the new one: http://beta.trakit.ca/packer3_old.zip

  • Comment by: Alex Lein
  • Posted:

Dean, I just wanted to point out that Packer doesn’t seem to shrink local variables that are defined with a comma seperator.

This:

function foo (zero) {
    var one   = 1,
        two   = 2,
        three = 3;
    return;
};

…which becomes:

function foo(a){var b=1,two=2,three=3;return};
  • Comment by: Már
  • Posted:

Thanks for the new version Dean. I am also having some regular expression problems, but all in all great work again!

Hello, I have a problem, my packed Javascript doesn’t work. Firefox says “missing ; before statement”.

I reduce the code, and the error is in this part of the code:

function disc (f)
{
	try
	{
		connexion = new XMLHttpRequest();
	} catch (e) { return true; }

	connexion.open("POST", "discussions_v.php");
	connexion.setRequestHeader('Content-Type','application/x-www-form-urlencoded');
	connexion.send("reaction=" + f.elements['reaction'].value);

	connexion.onreadystatechange = function()
	{
		if (connexion.readyState == 4)
		{
			alert ('ok');
		}
	}

	return false;
}

Try to pack this and add to a web page, you will get an error. (Unpacked, of course it works!)

If I don’t check Base62, I can add a new line before «return false;» and it works. But with Base62, there’s nothing I can do.:-(

Sorry, my fault. I didn’t read I must end all functions with a ;. :-[

Is there a port of the packer 3.0 to php??? I know there’s a packer 2 port, but I want the shrink variables option…

  • Comment by: Ionatan
  • Posted:

Hi Dean,

Thanks for the packer.

I just want to know,When you are going to release .net version of PACKER 3.0?

Thanks && Regards, Mohit Mishra

  • Comment by: Mohit Mishra
  • Posted:

hi Dean,

I’m not seeing the downloadable version? Would love to run Packer 3 via the command line on OS X. Thanks!

  • Comment by: felix
  • Posted:

Comments are closed.