You are viewing a plain text version of this content. The canonical link for it is here.
Posted to xap-commits@incubator.apache.org by mt...@apache.org on 2006/08/16 21:57:12 UTC
svn commit: r432045 [3/6] - in /incubator/xap/trunk/buildscripts: ./
doctool/ doctool/narcissus/ doctool/narcissus/CVS/ lib/ profiles/
Added: incubator/xap/trunk/buildscripts/jslink.pl
URL: http://svn.apache.org/viewvc/incubator/xap/trunk/buildscripts/jslink.pl?rev=432045&view=auto
==============================================================================
--- incubator/xap/trunk/buildscripts/jslink.pl (added)
+++ incubator/xap/trunk/buildscripts/jslink.pl Wed Aug 16 14:57:09 2006
@@ -0,0 +1,1379 @@
+#!/usr/bin/perl -w
+
+=head1 NAME
+
+ jslink.pl - eliminate unused code from a javascript library
+
+=head1 SYNOPSIS
+
+ jslink.pl -pre cat -i myapp.js -l lib.js -o -
+
+Options are:
+
+ -pre cat # preprocessor to apply to input files (but not -e).
+ -e 'myfunc(3); foobar;' # an "anchor" expression.
+ -h index.html # an "anchor" html file, whose script will be used. unimplemented.
+ -i myapp.js # an "anchor" script file, which will pull in things from library files.
+ -l lib.js # a library file.
+ -o output.js # the -l files with unneeded code removed. defaults to '-' (stdout).
+ -debug debug # default is none (no debugging)
+ -warn functionmatch,instmeth,ambigs,dups # default is none (no warnings about things that may be acceptable). 'all' is also supported.
+ -dump used,unused,usedby,refs,undefs # default is none (no dumping). 'all' is also supported.
+ -print used,filemarker,skipped,sourcelines # default is 'used,filemarker' (print used code from libraries)
+ -trace symname # issue debug output every time symname is seen. unimplemented.
+ -tabwidth 4 # set the number of spaces that tabs are interpreted as, if different from default of 8.
+ -nestedassigns 0 # whether to attempt to track assignments of nested function definitions to other symbols.
+
+All output except for -o (from -debug, -warn, or -dump) is sent to STDERR.
+
+=head1 DESCRIPTION
+
+This determines "dead" code based on following the transitive closure of references to
+symbols in one or more "anchor" files.
+
+It eliminates whole definitions only; it does not do anything even
+approaching full dead code elimination, as might be done within function bodies.
+
+It will eliminate nested functions if they are not used.
+
+It has knowledge of the builtin ECMAScript objects and their method names, as well as many DOM objects and their methods. It assumes that
+any parameter which calls a method with a name matching a builtin method name is indeed that kind of object. It does not currently
+do any sort of detailed static analysis that might actually prove this.
+
+It also has knowledge of all builtin ECMAscript global functions, and so knows not
+to try to find their definitions. Note that this means that it is up to you to
+determine if you need to provide definitions for missing builtin function or
+missing builtin object methods (for example, Array.prototype.push for IE 5.0).
+
+
+=head2 Treatment of Top-Level Statements
+
+If it finds any references at all to a symbol in some library file (global data or function),
+it will not only pull in the definitions of those symbols, it will then also include
+*all* of the top-level statements in that file.
+That is because we don't want to try to analyze the necessity of the load-time
+statements; it is all or nothing for them.
+If any of those load-time statements have references to functions in
+the same file, then those functions are pulled in too.
+For this reason, it is better to supply individual smaller javascript files
+(or just have few load-time statements).
+
+=head1 IMPLEMENTATION
+
+=head2 Unnecessary Limitations
+
+This implementation is based on a crude parser using regular expressions,
+which makes assumptions about indentation in order to identify the beginnings
+and endings of function definitions.
+
+The assumptions about indentation are valid if a pretty-printing
+preprocessor is used. The default preprocessor is just 'cat'.
+(We also have a Rhino-based preprocessor that we hacked together,
+but it is dependent on Rhino patches we have not yet organized.)
+
+Even if the indentation is regular, there are still lots of problems
+with this implementation: not properly skipping literals (String and RegExp),
+not properly parsing all expressions that might be a function call,
+etc.
+
+There are a variety of alternative implementation approaches, any of which
+would be more interesting and probably better, such as:
+
+ - based on a real ECMAScript grammar, or
+ - based on extending some real ECMAScript interpreter (such as SpiderMonkey or
+ Rhino, which do not explicitly use a grammar), or
+ - implemented in ECMAScript itself, such as something based on Narcissus, or
+ - implemented by analyzing the result of a "real" ECMAScript/JScript linker
+ (which would work with a "real" ECMAScript compiler), to see what symbols it pulls in.
+ - perform source translation to some other programming language that has
+ more mature tools.
+
+=head2 Fundamental Limitations
+
+Given the possible uses of eval(), function lookup tables, computed function names,
+dynamically added object methods, redefinition of functions, and so on,
+it is practically impossible to do this job fully automatically and correctly.
+
+These challenges can result in both false negatives and false positives.
+We can for example entirely miss a dependence on some code
+whose entry point is a string that might even come from
+the outside environment.
+
+On the other hand, when applied to an application that has
+a "driver" or "plugin" model, we might end up pulling in all
+available drivers/plugins, just because their entry points show
+up in some global registry table.
+
+We will also tend to pull in code even if the reference
+is protected by a conditional:
+
+ if (typeof someFunc != 'undefined') someFunc()
+
+The intent of the programmer in such code is typically to call
+someFunc() only if the code that defines it has been loaded
+for some other reason (vz. "weak references" in compiled
+languages).
+
+It is impossible really to solve all these situations automatically.
+It is necessary to get some guidance from the programmer, such
+as pragmas in the code itself, or by some other external
+configuration.
+
+=head2 Internals
+
+As we parse the input, we make up a list of definition objects, which
+have these keys:
+
+ deftype # one of: 'ctormeth', 'protometh', 'instmeth', 'globalfunc', 'localfunc', 'assign', 'singleton'
+ is_global # whether this is a top-level expression in a file (versus a function definition).
+ actualname # the string the actually occurred in the source code for the definition, such as 'MyClass.prototype.sort'.
+ justname # the last identifier name in the dotted name of the definition, such as 'sort'. key for $DEFS_BY_UNQUAL.
+ qualname # the fully qualified name for the defined symbol, which may be more explicit than appeared in source code. key for $DEFS_BY_QUAL.
+ parentqual # the value of qualname of the parent definition (in nesting level), if any.
+ protoname # the name of the prototype class, such as 'MyClass' (if any -- just for deftype 'protometh')
+ aliasto # the name of another symbol that this is an alias for (as in "Foo.prototype.meth = aliasfunc").
+ level # the nesting level of the definition
+ filename # name of the file (or expression) this came from.
+ startline # zero-based line number this definition starts with.
+ lastline # zero-based line number of last line of this definition.
+ lines # array ref of source lines corresponding to startline to lastline.
+ params # a hash ref with keys which are the parameter names of the defined function.
+ refs # the references from this definition to other symbols. hash ref from qualnames to [$reftype, $lineno]
+ undefs # array ref of symbols in refs that are apparently not defined anywhere.
+ usedby # array ref of other definition objects which refer (directly) to this one. opposite of refs.
+ used # boolean to indicate whether it is part of the transitive closure (actually it is the loopcount).
+
+=head1 TODO
+
+=head2 Data Tracking
+
+Track definitions of global data variables, and references to them.
+
+ # global data
+ ^var $NAMERE =
+ # member data
+ ^\ *this.$NAMERE =
+ # local data
+ ^\ *var $NAMERE =
+
+=head2 Aliasing and Scoping
+
+Make sure aliased definitions are not used to resolve references across lexical blocks.
+
+Warn about shadowed variable/function names. Implement 'localfunc' and 'assign' properly
+(see $ANALYZE_NESTED_ASSIGNMENTS).
+
+Track alias definitions (in 'assign' case, LHS inherits all the methods available from RHS).
+Things like "a = b.c.d;".
+
+Track creation of global instances, things like "var foo = new SomeFunc();".
+
+Ultimately, properly tracking of aliases could allow for tightening up of typeing.
+
+=head2 Typing
+
+Right now, if we can't resolve a reference using the fully qualified name, we'll
+match to any definition of a function with the same unqualified name.
+This has the unfortunate consequence of potentially pulling in the wrong code.
+
+We also currently ignore all references to methods that share a name with
+any DOM or builtin ECMAScript object method. This means we can miss undefined
+symbols.
+
+A proper fix would involve attempts at full data flow analysis to determine
+and variable argument types, and/or capitalize on some declaration mechanism
+(such as commented out types, or reliance on JScript .NET source code prior
+to down translation).
+
+Other benefits would be obtained as well, such as catching such errors as
+calling getElementById on a Window object instead of a Document object....
+
+=head2 Subclass Method Calls
+
+Track calls to prototype functions from within subprototype method bodies (based on
+remembering the set of the subprototype's prototype object).
+
+=head2 Function References
+
+Do something about passing in named function references (no parens). Though
+we win somewhat with "justname" look up on the other side.
+
+Scope locally bound functions such as "OuterFunction.inner."
+
+Extend $QUALRE to handle function calls with parameters in dotted expressions: "foobar(1,2).println()"
+
+Handle (ignore) calls from literals, such as '00000'.substring(2)
+
+=head2 Predefined Symbols
+
+Make complete list of ECMAScript functions, variables, and builtin object methods.
+
+Make complete (or somewhat complete) list of DOM object methods.
+
+=head2 Top-Level Statements
+
+Break up top-level statements into multiple continuous sequences within the file,
+or even per whole statement. This would make for better diagnostics (tracking
+line number for the definition/reference instead of just line -1). It would
+also pave the way for perhaps excluding more blocks from the
+final code.
+
+=head2 Preprocessing
+
+Get Rhino to issue original file line numbers.
+Also maybe get Rhino to mangle/change names, or do other transforms (such as conditional "in").
+
+Exclude pattern matches within quoted strings and regexp literals.
+
+Regular expressions to match for strings containing references that should be
+considered (vs. ignore all strings).
+
+Support for -h: allow anchor refs to come from an html page.
+
+Support -D of expressions known to be false or true, to exclude references in:
+
+ if (FALSEEXPR) {...}
+ if (!FALSEEXPR) {} else {...}
+ if (TRUEEXPR) {} else {...}
+
+=head2 Pragmas
+
+Support external declaration of other builtin objects and functions to assume are defined.
+
+Support some special inline comment syntax to indicate that something should be
+considered defined.
+
+=head2 Debugging and Reporting Features
+
+Implement -trace.
+
+Report on line counts in used and unused code.
+
+=head2 Lint-Like Features
+
+Provide warnings on:
+
+ calls to eval
+ computed apply and computed call
+ js file names
+ unknown method on builtin object
+ redefinition of builtin object method
+
+=head1 AUTHOR
+
+Copyright 2004, Mark D. Anderson, mda@discerning.com.
+
+This is free software; you can redistribute it and/or
+modify it under the same terms as Perl itself.
+
+Alternatively, this is licensed under Academic Free License version 1.2.
+
+=cut
+
+use Data::Dumper;
+
+################################################################
+# constants
+
+# including 'this'
+my @KEYWORDS = qw(break else new var case finally return void catch for switch while continue function with this default if throw delete in try do instanceof typeof);
+my %KEYWORDS = map {$_=>1} @KEYWORDS;
+my @BUILTIN_OBJECTS = (qw(Number String Boolean Date RegExp Array Math Object Error Function),
+ qw(XMLHttpRequest ActiveXObject DOMParser XMLSerializer),
+ qw(arguments NaN Infinity undefined));
+my %BUILTIN_OBJECTS = map {$_=>1} @BUILTIN_OBJECTS;
+my @BUILTIN_FUNCTIONS = (qw(eval parseInt parseFloat isNaN isFinite decodeURI decodeURIComponent encodeURI encodeURIComponent escape unescape),
+ qw(ScriptEngineMajorVersion ScriptEngineMinorVersion));
+my @WINDOW_FUNCTIONS = qw(alert blur clearTimeout close focus open print setTimeout);
+my %BUILTIN_FUNCTIONS = map {$_=>1} (@BUILTIN_FUNCTIONS, @WINDOW_FUNCTIONS);
+
+# TODO: Math static methods, rest of methods for RegExp and Date
+my @BUILTIN_METHODS = (
+ qw(toString toLocaleString valueOf hasOwnProperty isPrototypeOf propertyIsEnumerable), # Object
+ qw(apply call), # Function
+ qw(charAt charCodeAt fromCharCode concat indexOf lastIndexOf localeCompare match replace search slice split substring substr toLowerCase toUpperCase toLocaleLowerCase toLocaleUpperCase), # String
+ qw(test match exec), # RegExp
+ qw(concat join push pop reverse shift slice sort splice unshift), # Array
+ qw(toFixed toExponential toPrecision), # Number
+ qw(parse toDateString toTimeString getDate getDay getFullYear getHours getMilliseconds getMinutes getMonth getSeconds getTime getTimezoneOffset getYear), # Date
+ qw(setDate setHours setMilliseconds setMinutes setMonth setSeconds setYear toLocaleTimeString), # more Date
+ qw(caller), # Arguments
+ );
+
+my @DOM_METHODS = (
+ qw(clear createDocument createDocumentFragment createElement createEvent createEventObject createRange createTextNode getElementsByTagName getElementById write), # Document, Document.implementation
+ qw(addEventListener appendChild attachEvent cloneNode createTextRange detachEvent dispatchEvent fireEvent getAttributeNS getAttributeNode hasChildNodes hasAttribute hasAttributes insertBefore removeChild removeEventListener replaceChild scrollIntoView), # Node
+ qw(submit), # Form
+ qw(item), # DOM collections
+ qw(collapse createContextualFragment moveEnd moveStart parentElement select setStartBefore), # Range
+ qw(getPropertyValue setProperty), # Style
+ qw(initEvent preventDefault stopPropagation), # Event
+ qw(serializeToString), # XMLSerializer
+ qw(open send), # XMLHTTP
+ qw(loadXML), # XMLDOM
+ qw(parseFromString), # DOMParser
+);
+# put @WINDOW_FUNCTIONS both in methods and functions, since might be qualified or not
+my %BUILTIN_METHODS = map {$_=>1} (@BUILTIN_METHODS, @DOM_METHODS, @WINDOW_FUNCTIONS);
+my $NAMERE = '([\w_]+)';
+# TODO: need to handle "foobar().baz", but not "switch(a)"
+my $QUALRE = '([\w_][\w_\.]*)';
+my $GLOBALNAME = 'GLOBAL '; # fake symbol name for top-level statements in some file. trailing space is deliberate to match any real symbol.
+
+################################################################
+# internal variables
+my $DEFS_BY_QUAL = {};
+my $DEFS_BY_UNQUAL = {};
+my $DEFS_BY_FNAME = {};
+
+################################################################
+# configuration variables (or at least potentially configurable)
+
+# the presumed indent on input to indicate a whole definition level.
+my $PREINDENT = 1;
+my $TABWIDTH = 8;
+my $TABSPACES;
+my $PREPROCESSOR = 'cat';
+my $RHINOJAR = '/Users/mda/workspaces/rhino1_5R5/js.jar';
+my $JSLINKERDIR = '.';
+
+my $DEBUGOPTS = {debug => 0};
+my $WARNOPTS = {functionmatch => 0, instmeth => 0, ambigs => 0, dups => 0};
+
+my $DUMPOPTS = {used => 0, unused => 0, usedby => 0, undefs => 0, refs => 0};
+my $DUMPINDENT = ' ';
+
+my $SYMSEP = " ";
+
+my $OUTOPTS = {skipped => 0, sourcelines => 0, used => 1, filemarker => 1};
+
+my $OUTFILE = '-';
+
+my $TRACE_SYMS = {};
+
+my $FIND_UNDEFINED_IN_UNUSED = 1;
+my $ANALYZE_NESTED_ASSIGNMENTS = 1;
+
+################################################################
+# output routines
+
+sub debug {
+ print STDERR 'DEBUG: ', @_, "\n" if $DEBUGOPTS->{debug};
+}
+
+sub info {
+ my $mess = "\n" . join('', @_);
+ $mess =~ s/\n$//;
+ $mess =~ s/\n/\n /g;
+ print STDERR 'INFO:', $mess, "\n";
+}
+
+# set in parse_file()
+my $CURRENT_FNAME = '';
+
+sub parse_warn {
+ my ($mess, @rest) = @_;
+ my $line = $_;
+ my $lineno = $.;
+ my $fname = $CURRENT_FNAME;
+ print STDERR "PARSE WARNING: At line $fname\:$lineno : '$line'\n $mess", @rest, "\n";
+}
+
+sub warning {
+ print STDERR "WARNING: ", @_, "\n";
+}
+
+sub trace_sym {
+ my ($sym, @rest) = @_;
+ my $is_trace = 0;
+ if (ref($sym) eq 'ARRAY') {$is_trace = grep{$TRACE_SYMS->{$_}} @$sym}
+ else {$is_trace = $TRACE_SYMS->{$sym};}
+ debug @rest if $is_trace;
+}
+
+################################################################
+
+my $FILEOBJS_BY_NAME = {};
+
+sub usage {
+ my ($mess) = @_;
+ print STDERR "ERROR: $mess\n" if $mess;
+ print STDERR "See 'perldoc $0' for usage\n";
+ exit(1);
+}
+
+sub process_commandline {
+ my $fileobjs = [];
+ my $i = 0;
+ my $expr_count = 0;
+ while (@ARGV) {
+ my $opt = shift @ARGV;
+ warning "skipping empty argument", next unless $opt;
+ usage("unexpected non-option $opt") unless $opt =~ m/^-/;
+ my $optarg = shift @ARGV;
+ usage("no option argument following '$opt'") unless $optarg;
+ my $fileobj;
+ if ($opt eq '-e') {
+ $fileobj = {
+ filename => ('expr' . ($expr_count++)),
+ expr => $optarg,
+ is_anchor => 1,
+ };
+ }
+ elsif ($opt eq '-h') {
+ usage("unimplemented: reading script out of an html file");
+ }
+ elsif ($opt eq '-i') {
+ $fileobj = {filename => $optarg, is_anchor => 1};
+ }
+ elsif ($opt eq '-l') {
+ $fileobj = {filename => $optarg};
+ }
+ elsif ($opt eq '-o') {
+ $OUTFILE = $optarg;
+ }
+ elsif ($opt eq '-debug') {
+ $DEBUGOPTS = {map {$_ => 1} split(',',$optarg)};
+ }
+ elsif ($opt eq '-trace') {
+ $TRACE_SYMS->{$optarg} = 1;
+ }
+ elsif ($opt eq '-warn') {
+ if ($optarg eq 'all') {
+ while(my($k,$v) = each %$WARNOPTS) {$WARNOPTS->{$k} = 1;}
+ }
+ else {
+ $WARNOPTS = {map {$_ => 1} split(',',$optarg)};
+ }
+ }
+ elsif ($opt eq '-dump') {
+ if ($optarg eq 'all') {
+ while(my($k,$v) = each %$DUMPOPTS) {$DUMPOPTS->{$k} = 1;}
+ }
+ else {
+ $DUMPOPTS = {map {$_ => 1} split(',',$optarg)};
+ }
+ }
+ elsif ($opt eq '-pre') {
+ $PREPROCESSOR = $optarg;
+ }
+ elsif ($opt eq '-tabwidth') {
+ $TABWIDTH = $optarg;
+ }
+ elsif ($opt eq '-nestedassigns') {
+ $ANALYZE_NESTED_ASSIGNMENTS = (!$optarg || $optarg eq '0') ? 1 : 0;
+ }
+ else {
+ usage("unknown option '$opt'");
+ }
+ if ($fileobj) {
+ push(@$fileobjs, $fileobj) ;
+ $FILEOBJS_BY_NAME->{$fileobj->{filename}} = $fileobj;
+ }
+ $i++;
+ }
+ $TABSPACES = ' 'x$TABWIDTH;
+ return $fileobjs;
+}
+
+# used for 'ctormeth' definition, for example "this.foobar = function ...".
+# It figures out what "this" is from the $parentdef
+sub qualify_def_this {
+ my ($name, $parentdef) = @_;
+ my $pname = $parentdef->{qualname};
+ my $pdeftype = $parentdef->{deftype};
+ if ($pdeftype eq 'globalfunc') {
+ debug("qualifying name '$name' with parent '$pname' of type '$pdeftype'");
+ $name = "$pname\.constructor\.$name";
+ }
+ else {
+ if ($pdeftype eq 'instmeth' || $pdeftype eq 'localfunc') {
+ if ($WARNOPTS->{instmeth}) {
+ parse_warn("instance method definition of '$name' in parent '$pname'; if parent is a singleton, defining a method in the constructor is ok")
+ }
+ }
+ elsif ($pdeftype eq 'singleton') {
+ }
+ else {
+ parse_warn("member function definition of '$name' with parent '$pname', parent deftype '$pdeftype'");
+ }
+ }
+ return $name;
+}
+
+# used when a reference string for a function call starts with 'this.'
+# we try to determine what "this" means based on deftype we are in :
+# 'ctormeth' - in a "this.foobar = function" (itself inside a global function)
+# 'protometh' - in a "FooBar.prototype.methname = function"
+# 'instmeth' - in a "whatever.methname = function"
+# 'globalfunc' - inside body of global function (constructor)
+sub qualify_ref_this {
+ my ($refname, $def) = @_;
+
+ # we this could have been from the body of a constructor, or from the body of another method.
+ my ($restname) = ($refname =~ m/^this\.(.*)/);
+ my $deftype = $def->{deftype};
+
+ my $refqual = undef;
+ my $is_funny = 0;
+
+ if ($deftype eq 'ctormeth' || $deftype eq 'protometh' || $deftype eq 'instmeth' || $deftype eq 'singleton') {
+ ($refqual) = ($def->{qualname} =~ m/(.*)\./);
+ if (!$refqual && $def->{parentqual}) {
+ debug("giving reference '$refname' the parent qualifier of '", $def->{parentqual}, "'");
+ $refqual = $def->{parentqual};
+ }
+ }
+ elsif ($deftype eq 'globalfunc') {
+ $refqual = $def->{qualname} . '.prototype';
+ }
+ else {
+ $is_funny = 1;
+ parse_warn("reference '$refname' in a definition of unknown type '$deftype'");
+ }
+
+ if ($refqual) {
+ debug("qualifying reference '$refname' in a '$deftype' definition as '$refqual' + '.' + '$restname'");
+ $refname = "$refqual\.$restname";
+ }
+ elsif (!$is_funny) {
+ parse_warn("could not figure out what 'this' means in reference, inside definition: ", Dumper($def));
+ }
+ return $refname;
+}
+
+# create and register a definition object.
+sub add_def {
+ my ($qualname, $actualname, $deftype, $fname, $lineno, $level, $parentdef, $protoname, $aliasto) = @_;
+ die "add_def: wrong number args: @_" unless scalar(@_) == 9;
+ # we are currently at the next line, so subtract 1 from $. for zero-based line number.
+ $lineno--;
+
+ my $is_global = 0;
+ if ($qualname eq $GLOBALNAME) {
+ $is_global = 1;
+ $qualname = "$GLOBALNAME$fname";
+ }
+
+ # parse out params, unless type 'assign'
+ my $params = {};
+ if (!$is_global && $deftype ne 'assign') {
+ my ($funcname, $paramstr) = m/function ([\w_]*?)\s*\((.*?)\)/ ;
+ if (!$funcname) {
+ $funcname = '';
+ ($paramstr) = m/function\s*\((.*?)\)/;
+ }
+ if (defined($paramstr)) {
+ trace_sym($funcname, "funcname=$funcname, paramstr=$paramstr in '$_'");
+ # $paramstr ||= '';
+ # if (!defined($paramstr)) { parse_warn("did not match function params, 1='$1'"); $paramstr = ''}
+ my @parms = split(/\s*,\s*/, $paramstr);
+ $params = {map {$_ => 1} @parms};
+ trace_sym($funcname, "got params '$paramstr', ", Dumper($params));
+ }
+ else {
+ parse_warn("no function params to parse in: ", $_);
+ }
+ }
+
+ my $parentqual = $parentdef->{qualname};
+ my $justname = undef;
+ if (!$is_global) {
+ ($justname) = ($qualname =~ m/$NAMERE$/);
+ parse_warn("no match to m/$NAMERE\$/ in '$qualname'") unless $justname;
+ }
+ my $def = {
+ qualname => $qualname,
+ is_global => $is_global,
+ actualname => $actualname,
+ justname => $justname,
+ deftype => $deftype,
+ filename => $fname,
+ startline => $lineno,
+ linenos => [], # used only if $is_global, a list of line numbers
+ params => $params,
+ level => $level,
+ parentqual => $parentqual,
+ protoname => $protoname,
+ aliasto => $aliasto,
+ refs => {}, # all references found in body of this definition. hash from $qualname to [$reftype, $lineno]
+ undefs => [], # list of keys from refs which are not defined.
+ usedby => [], # array of other $def's which point to this one.
+ used => 0,
+ added => 0,
+ };
+ my $existing = $DEFS_BY_QUAL->{$qualname};
+ my $do_replace = 1;
+ if ($existing) {
+ # don't warn if either of type 'assign' (and same file?)
+ if ( # $existing->{filename} eq $def->{filename} &&
+ ($existing->{deftype} eq 'assign' || $def->{deftype} eq 'assign')) {
+ # don't replace a non-assign with an assign
+ if ($def->{deftype} eq 'assign') {
+ $do_replace = 0;
+ parse_warn("preventing replacement of existing definition of '$qualname' at ", $existing->{filename}, ":", $existing->{startline},
+ " with an 'assign' definition") if $WARNOPTS->{dups};
+ }
+ else {
+ parse_warn("allowing replacement of an existing 'assign' definition of '$qualname' at ", $existing->{filename}, ":", $existing->{startline},
+ " with another") if $WARNOPTS->{dups};
+ }
+
+ }
+ # don't warn if a localfunction
+ elsif ($existing->{deftype} eq 'localfunc') {
+ debug("overriding previous definition of '$qualname' because localfunc");
+ }
+ else {
+ parse_warn("duplicate definition of '$qualname': ", Dumper($existing), Dumper($def)) if $WARNOPTS->{dups};
+ }
+ }
+ $DEFS_BY_QUAL->{$qualname} = $def if $do_replace;
+
+ # if $do_replace of $existing, then we better make sure that $existing knows not to
+ # complain later....
+ $existing->{replacedby} = $def if $do_replace && $existing;
+
+ trace_sym($qualname, "storing def on '$qualname' with justname=$justname") if $justname;
+
+ if ($justname) {
+ my $a = $DEFS_BY_UNQUAL->{$justname};
+ $DEFS_BY_UNQUAL->{$justname} = $a ? [@$a, $def] : [$def];
+ }
+ my $filedefs = $DEFS_BY_FNAME->{$fname};
+ my ($already) = grep {$_->{qualname} eq $qualname} @$filedefs;
+ parse_warn("The file $fname already has a definition for '$qualname' at ", $already->{filename}, ':', $already->{startline})
+ if $WARNOPTS->{dups} && $already && ($already->{deftype} ne 'localfunc' || $def->{deftype} ne 'localfunc') ;
+ push(@$filedefs, $def);
+
+ debug("starting definition of '", def_name($def), "' with type '$deftype' at $fname:$lineno, level $level");
+ return $def;
+}
+
+# used when displaying messages about a definition.
+sub def_name {
+ my ($def) = @_;
+ my $q = $def->{qualname};
+ if ($def->{is_global}) {
+ return "(global expr)";
+ my $sl = $def->{startline};
+ my $ll = $def->{lastline};
+ return "(expr lines $sl-$ll)"; # in $1)";
+ }
+ # return $def->{protoname} . ".prototype.$q" if $def->{protoname};
+ return $q;
+}
+
+# note that this definition $def is referring to qualified symbol $refname.
+# a single definition might refer to some other symbol multiple times; we only record one such case.
+sub add_ref {
+ my ($def, $refname, $reftype, $fname, $lineno) = @_;
+ # we are currently at the next line, so subtract 1 from $. for zero-based line number.
+ $lineno--;
+ my ($startname) = ($refname =~ m/^(\w+)/);
+
+ if ($refname eq $def->{qualname} || $refname eq $def->{actualname}) {
+ debug("skipping recursive reference to '$refname' in ", def_name($def));
+ return;
+ }
+
+ if ($def->{params}->{$refname}) {
+ debug("skipping reference '$refname' in ", def_name($def), " because it is a parameter");
+ return;
+ }
+
+ if (!$startname) {
+ warning("reference name '$refname' does not start with word (reftype=$reftype), at $fname\:$lineno");
+ return;
+ }
+
+ my $qualname = $refname;
+ # my $actualname = $refname;
+ if ($startname eq 'this') {
+ if ($refname eq 'this') {debug("skipping 'this' as function"); return;}
+ $qualname = qualify_ref_this($refname, $def);
+ }
+ elsif ($KEYWORDS{$startname}) {
+ # debug("skipping keyword '$refname' at $fname:$lineno");
+ return;
+ }
+ elsif ($BUILTIN_OBJECTS{$startname}) {
+ # debug("skipping builtin object reference '$refname' at $fname:$lineno");
+ return;
+ }
+ elsif ($BUILTIN_FUNCTIONS{$startname}) {
+ # debug("skipping builtin function reference '$refname' at $fname:$lineno");
+ return;
+ }
+ debug("adding reference '$refname' (qualname='$qualname') of type '$reftype' from ", def_name($def), " at $fname:$lineno : ", $_);
+ # 3rd slot is to hold the definition, once we know it
+ $def->{refs}->{$qualname} = [$reftype, $., undef];
+}
+
+sub is_parameter {
+ my ($currentdef, $actualname) = @_;
+ return ($currentdef && $actualname && $currentdef->{params}->{$actualname});
+}
+
+# parse the provided file (or expression)
+sub parse_file {
+ my ($f, $fileobj) = @_;
+
+ my $fname = $fileobj->{filename};
+ $CURRENT_FNAME = $fname;
+ $DEFS_BY_FNAME->{$fname} = [];
+
+ # the function def we are currently inside of, or the top-level script
+ my $currentdef = add_def($GLOBALNAME, '', 'global', $fname, 0, 0, undef, undef, undef);
+ $fileobj->{globaldef} = $currentdef;
+
+ my $nested = [$currentdef]; # stack of functions being defined.
+ my $in_comment = 0;
+ my $lines = [];
+ while(<$f>) {
+ push(@$lines, "$_");
+ chop;
+ # debug("parsing $fname:$. : '$_'");
+
+ # convert tabs to equivalent number of spaces
+ s/\t/$TABSPACES/g;
+
+ # a dumb C-comment parser, in case preprocessor didn't exclude them.
+ # check for end of multi-line C comment
+ if ($in_comment) {
+ if (m,\*/,) {
+ $in_comment = 0;
+ s,.*?\*/,,;
+ }
+ else {next;}
+ }
+ # remove C++ comment
+ s,//.*$,,;
+ # start of C comment.
+ if (m,^\s*/\*,) {
+ if (m,\*/,) {
+ s,/\*.*?\*/,,;
+ }
+ else {
+ $in_comment = 1;
+ next;
+ }
+ }
+
+ # entirely blank line
+ next if m/^\s*$/;
+
+ # collapse quotes
+ s/"[^\\\"]*"/""/g;
+ s/'[^\\\']*'/''/g;
+
+ # determine indent level.
+ m/^( *)/;
+ my $level = length($1)/$PREINDENT;
+
+ # ugly assume global definitions are ones that start with a zero indent
+ my $is_global = ($level == 0);
+
+ # maybe done defining function
+ my $lastlevel = $currentdef->{level};
+ my $numnested = scalar(@$nested);
+ my $popped = 0;
+ if ($level <= $lastlevel && ($lastlevel > 0 || $numnested > 1)) {
+ debug("finishing definition of ", def_name($currentdef), " at line $. because $level <= $lastlevel\: '", $_, "'");
+ $popped = 1;
+ $currentdef->{lastline} = $.;
+ pop(@$nested);
+ $currentdef = $nested->[$numnested - 2];
+ die "no nested function definition to pop at: $_" unless $currentdef;
+ }
+
+ my $qualname = undef; # full qualified
+ my $actualname = undef; # what actually was found in the file
+ my $deftype = undef;
+ my $protoname = undef;
+ my $aliasto = undef;
+
+ if (m/^ *function $NAMERE/ || m/^ *var $NAMERE = function/ ) {
+ $deftype = ($is_global ? 'globalfunc' : 'localfunc');
+ $qualname = $actualname = $1;
+ }
+ elsif (m/^$NAMERE = new function\(/ || m/^ *var $NAMERE = new function\(/) {
+ $deftype = 'singleton';
+ $qualname = $actualname = $1;
+ }
+ elsif (m/^ *this\.$NAMERE = function/) {
+ $deftype = 'ctormeth';
+ $actualname = "this.$1";
+ $qualname = qualify_def_this($1, $currentdef);
+ }
+ # TODO: Foo.prototype.meth = aliasfunc
+ # TODO: var Foo = {methname : function ...
+ elsif (m/^ *$QUALRE\.prototype\.$NAMERE = function/) {
+ $deftype = 'protometh';
+ $protoname = $1;
+ $actualname = $qualname = "$1\.prototype\.$2";
+ }
+ elsif (m/^ *$QUALRE\.$NAMERE = function/) {
+ $actualname = $qualname = "$1\.$2";
+ $deftype = 'instmeth';
+ }
+
+ # starting a function definition - register and continue loop
+ my $wholebody;
+ if ($deftype) {
+ my $parentdef = $currentdef;
+ $currentdef = add_def($qualname, $actualname, $deftype, $fname, $., $level, $parentdef, $protoname, $aliasto);
+ push(@$nested, $currentdef);
+ # if line ends in an open curly, then there are no body lines to parse, and it will finish on another line
+ if (m/\{[\s\n]*$/) {
+ next;
+ }
+ elsif (m/\{(.*)\}[\;\s\n]*$/) {
+ debug("function begins and ends on same line $.: $_");
+ $wholebody = $1;
+ $_ = $wholebody;
+ }
+ else {
+ parse_warn("definition start line does not end with open or closing bracket");
+ next;
+ }
+ }
+
+ # if global expression, and not a new function definition, add the line to the currentdef
+ if (!$popped && !$wholebody && $currentdef->{is_global} && ! m/^[\s\n\r]*$/) {
+ my $linenos = $currentdef->{linenos};
+ push(@$linenos, $. - 1);
+ # debug("pushing expression line ", $. - 1);
+ }
+ else {
+ debug("not in a global expression at $.: ", $popped, $currentdef->{is_global}, ': ', $_);
+ }
+
+ # deal with assignment. This could be both a definition (LHS) and reference (RHS). could also be an alias.
+ if (!$deftype && (m/^ *$QUALRE = / || m/^ *var $NAMERE = /) && ($is_global || $ANALYZE_NESTED_ASSIGNMENTS) ) {
+ $deftype = 'assign';
+ $actualname = $qualname = $1;
+ if (is_parameter($currentdef, $actualname)) {
+ debug("skipping assignment because it is to parameter '$actualname', at line $.");
+ }
+ elsif (@$nested > 1 && is_parameter($nested->[@$nested - 2], $actualname)) {
+ debug("skipping assignment because it is to parameter '$actualname' of parent function, at line $.");
+ }
+ else {
+ # we don't push a function defining context
+ add_def($qualname, $actualname, $deftype, $fname, $., $level, $currentdef, $protoname, $aliasto);
+ }
+ }
+ else {
+ my $line = $_;
+ trace_sym($1, "did not match assignment") if grep {$line =~ m/$_/} keys %$TRACE_SYMS;
+ $_ = $line;
+ }
+
+ # Maybe warn because not starting a function definition, but contains the string 'function'.
+ if (m/function/) {
+ # matches to the string 'function' that are normal and expected:
+ # closures in function calls: foobar(17, function(a, b) {
+ # returning a closure: return function (o) {
+ # closure on rhs, in level: foobar = function (s) {
+ # matches in string: foobar("what is this function");
+ # matches in regexp: var m = s.match(/function /);
+ # partial matches: var s = foobar.functionName(f);
+ parse_warn("ignoring function defintion starting here") if $WARNOPTS->{functionmatch};
+ }
+
+ # scan this line for function references, using possibly qualified names
+
+ # find all constructor calls (use of "new")
+ my @ctorcalls = m/new $QUALRE/g;
+ for my $csym (@ctorcalls) {
+ if (is_parameter($currentdef, $csym)) {
+ debug "skipping call to constructor '$csym' because a parameter";
+ }
+ add_ref($currentdef, $csym, 'construct', $fname, $.);
+ }
+
+ # find all function calls (normal parens)
+ my @funcalls = m/$QUALRE\(/g;
+ # exclude calls to methods of literal regexps such as /foo/i.exec();
+ # TODO detect other things besides \w and / prior to .
+ @funcalls = grep {! m/apply$/ && ! m/call$/ && !m,^\.,} @funcalls;
+ for my $fsym (@funcalls) {
+ if (is_parameter($currentdef, $fsym)) {
+ debug "skipping call to function '$fsym' because a parameter";
+ next;
+ }
+ # TODO: should really track assignments, and check all ancestor scopes....
+ elsif (@$nested > 1 && is_parameter($nested->[@$nested - 2], $fsym)) {
+ debug "skipping call to function '$fsym' because a parameter to parent function";
+ next;
+ }
+ my ($justname, $firstname) = split_name($fsym);
+ my $skip = 0;
+ if ($justname && $BUILTIN_METHODS{$justname}) {
+ $skip = 1;
+ if ($firstname eq $justname && $BUILTIN_FUNCTIONS{$firstname}) {
+ debug "skipping global builtin function reference to '$fsym'";
+ }
+ elsif ($firstname && is_parameter($currentdef, $firstname)) {
+ debug "skipping method reference to '$fsym' because parameter and builtin method";
+ }
+ elsif ($firstname && ($firstname ne $justname)) {
+ debug "skipping method reference to '$fsym' because builtin method even though object is not a parameter";
+ }
+ # right now, we miss matching expressions like: String(n).replace(/(\d)/, "$1.")
+ # because of the parens in "String(n)", we just see "replace"
+ elsif (m/\.$justname/) {
+ debug "skipping method reference to '$fsym' because builtin method and qualifier is an expression: $_";
+ }
+ else {
+ $skip = 0;
+ warning("adding reference to '$fsym' (justname=$justname, firstname=$firstname) even though a builtin method: ", $_);
+ }
+ }
+
+ add_ref($currentdef, $fsym, 'call', $fname, $.) unless $skip;
+ }
+
+ # find all dynamic calls (use of "apply" or "call")
+ my @applycalls = m/$QUALRE\.apply\s*\(/;
+ my @callcalls = m/$QUALRE\.call\s*\(/; # don't want to match 'caller' etc.
+ my @dyncalls = (@applycalls, @callcalls);
+ for my $dynsym (@dyncalls) {
+ if (is_parameter($currentdef, $dynsym)) {
+ debug "skipping dynamic call to '$dynsym' because a parameter";
+ next;
+ }
+ add_ref($currentdef, $dynsym, 'dynamic', $fname, $.);
+ }
+
+ # if function began and ended on this line, pop it
+ if ($wholebody) {
+ $currentdef->{lastline} = $.;
+ pop(@$nested);
+ $currentdef = $nested->[scalar(@$nested) - 1];
+ die "no nested function definition to pop at line $." unless $currentdef;
+ }
+ }
+ die "did not pop all nested definitions in file $fname: ", Dumper($nested) if scalar(@$nested) > 1;
+ die "still in comment at end of file $fname" if $in_comment;
+ $fileobj->{lines} = $lines;
+}
+
+sub preprocessor_command {
+ my ($fname) = @_;
+ return "cat $fname" if $PREPROCESSOR eq 'cat';
+ return "java -classpath $RHINOJAR:$JSLINKERDIR JsLinker $fname" if $PREPROCESSOR eq 'rhino';
+ die "unknown preprocessor '$PREPROCESSOR'";
+}
+
+# collect definitions and references
+sub process_files {
+ my ($fileobjs) = @_;
+ for my $fileobj (@$fileobjs) {
+ my $fname = $fileobj->{filename};
+ my $cmd;
+ if ($fileobj->{expr}) {
+ pipe PRE, STRING;
+ print STRING $fileobj->{expr};
+ close STRING;
+ }
+ else {
+ die "no such file '$fname'" unless -e $fname;
+ $cmd = preprocessor_command($fname) . ' |';
+ open(PRE, $cmd) || die "can't open $cmd: $!";
+ }
+ parse_file(PRE, $fileobj);
+ close(PRE);
+ }
+}
+
+sub split_name {
+ my ($qualname) = @_;
+ my ($justname) = ($qualname =~ m/$NAMERE$/);
+ warning("no justname in '$qualname'") unless $justname;
+ my ($firstname) = ($qualname =~ m/^$NAMERE/);
+ warning("no firstname in '$qualname'") unless $firstname;
+ return ($justname, $firstname);
+}
+
+sub find_def {
+ my ($refname, $fromdef, $undefs_hash) = @_;
+
+ my ($justname, $firstname) = split_name($refname);
+
+ # look up definition object by full name
+ my $todef = $DEFS_BY_QUAL->{$refname};
+
+ return $todef if $todef;
+
+ # don't bother tracking a builtin method name
+ if ($BUILTIN_METHODS{$justname}) {
+ debug("ignoring reference to builtin method in $refname");
+ return undef;
+ }
+
+ # not found, try to find less strict match.
+ # if find multiple, warn.
+ # if still find none, collect in an 'undefined' collection.
+ my $is_local = $fromdef->{params}->{$firstname} ? 1 : 0 ;
+ trace_sym($refname, "is_local=$is_local, firstname=$firstname, fromdef=", def_name($fromdef), ", params=", Dumper($fromdef->{params}));
+
+ my $unqual_defs = $DEFS_BY_UNQUAL->{$justname};
+ my $mess = "reference '$refname' from " . def_name($fromdef) . " at " . $fromdef->{filename} . ':' . $fromdef->{startline};
+ # attempt to find matches to just the unqualified last part of the name
+ if ($unqual_defs) {
+ my $num_unqual = scalar(@$unqual_defs);
+ if ($num_unqual > 1) {
+ warning("no fully qualified matches, found $num_unqual matches to '$justname', " . $mess) if $WARNOPTS->{ambigs};
+ }
+ elsif ($num_unqual == 0) {
+ die "no unqualified matches for '$justname' yet entry exists in DEFS_BY_UNQUAL";
+ }
+ else {
+ debug("found 1 match to justname='$justname', " . $mess);
+ }
+ $todef = $unqual_defs->[0] || die("no 0 entry in ", Dumper($unqual_defs));
+ }
+ # no matches to unqualified name either, collect it.
+ elsif ($undefs_hash) {
+ my $def_undefs = $fromdef->{undefs};
+ push(@$def_undefs, $refname);
+ $undefs_hash->{$refname} = $fromdef;
+ }
+ return $todef;
+}
+
+# perform transitive closure determining what other functions are required
+sub transitive_closure {
+ my ($start_defs) = @_;
+
+ my $used_defs = [];
+ my $new_defs = $start_defs;
+ my $loopcount = 1;
+ my $used_undefs = {};
+ my $unused_undefs = {};
+
+ # loop as long we we added new definitions in the last pass.
+ # we start with the "anchor" definitions.
+ # note that this will only find undefined symbols among smbols pulled in.
+ while (scalar(@$new_defs) > 0) {
+ debug("starting loop $loopcount with ", scalar(@$new_defs), " definitions");
+ my $current_defs = $new_defs;
+ $new_defs = [];
+
+ # loop over all (new definitions), looking at what they refer to, and pulling those in if not already
+ for my $fromdef (@$current_defs) {
+ my $refs = $fromdef->{refs};
+ my $fromqual = $fromdef->{qualname};
+ # print STDERR "**** $loopcount $fromdef->{used} $fromdef->{added} FROM $fromqual\n";
+
+ # loop over all refs from this definition
+ for my $refname (keys %$refs) {
+ # print STDERR "******** $loopcount REF $refname\n";
+ my $todef = find_def($refname, $fromdef, $used_undefs);
+
+ if ($todef) {
+ # add this reference to the definition's usedby array
+ my $usedby = $todef->{usedby};
+ my ($already) = grep {$_->{qualname} eq $fromqual} @$usedby;
+ if ($already) {
+ warning("attempt to do duplicate add of '$fromqual' to usedby of '", $todef->{qualname}, "' because of refname '$refname'");
+ }
+ else {
+ push(@$usedby, $fromdef);
+ }
+
+ # record it in the refinfo
+ $refs->{$refname}->[2] = $todef;
+
+ # add the definition we have pulled in to $new_defs (if it hasn't already been processed,
+ # and we haven't already added it in this loop).
+ if (!$todef->{used} && !$todef->{added}) {
+ $todef->{added} = 1;
+ push(@$new_defs, $todef);
+
+ # see if $todef is from a file we haven't pulled in before (but isn't the globaldef itself)
+ if (!$todef->{is_global}) {
+ my $fileobj = $FILEOBJS_BY_NAME->{$todef->{filename}} || die "no fileobj for definition filename " . $todef->{filename};
+ my $globaldef = $fileobj->{globaldef} || die "no globaldef in fileobj: ", Dumper($fileobj);
+ if (!$globaldef->{used} && !$globaldef->{added}) {
+ $globaldef->{added} = 1;
+ debug "adding global def for file ", $todef->{filename}, " pulled in by $refname";
+ push(@$new_defs, $globaldef);
+ }
+ }
+ }
+ }
+ } # loop for refs
+
+ # mark this definition as having been processed
+ $fromdef->{used} = $loopcount;
+ } # for $new_defs
+ $loopcount++;
+ push(@$used_defs, @$current_defs);
+ }
+
+ # also track undefined symbols in unused functions
+ if ($FIND_UNDEFINED_IN_UNUSED) {
+ my $all_defs = [values %$DEFS_BY_QUAL];
+ for my $def (@$all_defs) {
+ # just because it is marked as used by one ref doesn't mean that all refs know about it
+ # next if $def->{used};
+ my $refs = $def->{refs};
+ my $defname = $def->{qualname};
+ trace_sym($defname, "checking undefs for $defname");
+ for my $refname (keys %$refs) {
+ my $refinfo = $refs->{$refname};
+ if ($refinfo->[2]) {
+ trace_sym([$defname,$refname], "looking for definition of '$refname' used by '$defname', but it ref already has a definition");
+ next ;
+ }
+ my $todef = find_def($refname, $def, $unused_undefs);
+ trace_sym([$defname,$refname], "looking for definition of '$refname' used by '$defname'; ", ($todef ? "found definition" : "did not find definition"));
+ # record it in the refinfo
+ $refs->{$refname}->[2] = $todef if $todef;
+ }
+ }
+ }
+
+ return ($used_defs, $loopcount, $used_undefs, $unused_undefs);
+}
+
+sub dump_syms {
+ my ($fileobjs, $used_undefs, $unused_undefs) = @_;
+
+ *DUMP = *STDOUT;
+
+ for my $fileobj (@$fileobjs) {
+ my $fname = $fileobj->{filename};
+ my $fdefs = $DEFS_BY_FNAME->{$fname};
+
+ # filename summary
+ my $ndefs = scalar(@$fdefs);
+ my @used = grep {$_->{used}} @$fdefs;
+ my $nused = scalar(@used);
+ my $nunused = $ndefs - $nused;
+ print DUMP "DUMP: symbols in '$fname': ";
+ if ($fileobj->{is_anchor}) {print DUMP "anchor input, with $ndefs definitions\n";}
+ else {
+ if (!$DUMPOPTS->{unused} && !$fileobj->{globaldef}->{used}) {
+ debug("skipping unused file, since all symbols in it will be unused");
+ next;
+ }
+ print DUMP "library input, with $nused used, $nunused unused\n";
+ }
+
+ for my $def (@$fdefs) {
+ next if $def->{deftype} eq 'assign';
+
+ my $usedby;
+ my $nused = 0;
+ if ($def->{used}) {
+ next unless $DUMPOPTS->{used};
+ $usedby = $def->{usedby};
+ $nused = scalar(@$usedby);
+ }
+ else {
+ next unless $DUMPOPTS->{unused};
+ }
+
+ # per definition summary
+ # print DUMP sprintf("%s%-10s %3d %s\n", $DUMPINDENT, $def->{deftype}, $nused, def_name($def));
+ print DUMP sprintf("%s%-30s type=%-10s used=%-3d lineno=%-3d\n", $DUMPINDENT, def_name($def), $def->{deftype}, $nused, $def->{startline});
+
+ my $replacedby = $def->{replacedby};
+ if ($replacedby) {
+ print DUMP $DUMPINDENT, $DUMPINDENT, 'REPLACED BY: ', $replacedby->{filename}, ':', $replacedby->{startline}, "\n";
+ next;
+ }
+
+ my $refs = $def->{refs};
+ my $undefs = $def->{undefs};
+ if ($DUMPOPTS->{refs} && (scalar(keys %$refs) - scalar(@$undefs)) > 0) {
+ print DUMP $DUMPINDENT, $DUMPINDENT, 'DEFINED REFERENCES:', "\n";
+ for my $refname (keys %$refs) {
+ my $refinfo = $refs->{$refname};
+ my ($reftype, $reflineno, $def_for_ref) = @$refinfo;
+ my $def_for_ref2 = $DEFS_BY_QUAL->{$refname};
+ # my $def_for_ref = find_def($refname, $def);
+ if (!$def_for_ref) {
+ warning "no definition for reference '$refname' but not in this def's undefs (@$undefs)" unless grep {$_ eq $refname} @$undefs;
+ warning "undefined reference '$refname' not in global undefs" unless $used_undefs->{$refname} || $unused_undefs->{$refname};
+ warning "but in DEFS_BY_QUAL->{$refname}" if $def_for_ref2;
+ next;
+ }
+ print DUMP $DUMPINDENT, $DUMPINDENT, $DUMPINDENT, $refname,
+ ' defined at ', $def_for_ref->{filename}, ':', $def_for_ref->{startline},
+ ' used at ', $fname, ':', $reflineno, "\n";
+ }
+ }
+
+ # indented list of undefined references from within this function
+ if ($DUMPOPTS->{undefs} && scalar(@$undefs) > 0) {
+ print DUMP $DUMPINDENT, $DUMPINDENT, 'UNDEFINED REFERENCES:', "\n";
+ for my $refsym (@$undefs) {
+ my $refinfo = $def->{refs}->{$refsym};
+ my ($reftype, $reflineno) = @$refinfo;
+ print DUMP $DUMPINDENT, $DUMPINDENT, $DUMPINDENT, $refsym,
+ ' used at ', $fname, ':', $reflineno, "\n";
+ }
+ }
+
+ # if desired, list usedby for this definition
+ if ($DUMPOPTS->{usedby} && $nused > 0) {
+ print $DUMPINDENT, $DUMPINDENT, 'USED BY:', "\n";
+ for my $usedef (@$usedby) {
+ print DUMP $DUMPINDENT, $DUMPINDENT, $DUMPINDENT, def_name($usedef), ' at ', $usedef->{filename}, ':', $usedef->{startline}, "\n";
+ }
+ }
+ }
+ }
+}
+
+sub print_some_ {
+ my ($lines, $lineno_ind, $linenos, $startline) = @_;
+ my $globalcount = 0;
+ while ($lineno_ind < scalar(@$linenos) && ($startline < 0 || $linenos->[$lineno_ind] < $startline)) {
+ if ($globalcount == 0) {
+ print OUT "// some global statements\n" if $OUTOPTS->{sourcelines};
+ $globalcount++;
+ }
+ my $lineno = $linenos->[$lineno_ind];
+ print OUT "// line $lineno\n" if $OUTOPTS->{sourcelines};
+ print OUT $lines->[$lineno];
+ $lineno_ind++;
+ }
+ return $lineno_ind;
+}
+
+sub print_minimal_files {
+ my ($fileobjs) = @_;
+
+ if ($OUTFILE eq '-') {
+ *OUT = *STDOUT;
+ }
+ else {
+ open(OUT, ">$OUTFILE") || die "can't open output file $OUTFILE: $!";
+ }
+ my $count = -1;
+ for my $fileobj (@$fileobjs) {
+ $count++;
+ my $fname = $fileobj->{filename};
+ # skip anchor files; we know they have everything
+ next if $fileobj->{is_anchor};
+
+ my $globaldef = $fileobj->{globaldef};
+ if (! $globaldef->{used}) {
+ debug "skipping file $fname because not used";
+ next;
+ }
+
+ print OUT "// STARTING FILE $fname\n" if $OUTOPTS->{filemarker};
+ my $fdefs = $DEFS_BY_FNAME->{$fname} || die "no definitions for file '$fname'";
+ $fdefs = [sort {$a->{startline} <=> $b->{startline}} @$fdefs];
+
+ my $linenos = $globaldef->{linenos};
+
+ # filename summary
+ my $ndefs = scalar(@$fdefs);
+
+ my $lines = $fileobj->{lines};
+
+ my $lineno_ind = 0;
+ for my $def (@$fdefs) {
+ next if $def->{deftype} eq 'assign';
+ my $usedby;
+ my $nused = 0;
+ if ($def->{used}) {
+ $usedby = $def->{usedby};
+ $nused = scalar(@$usedby);
+ }
+ my $startline = $def->{startline};
+
+ $lineno_ind = print_some_($lines, $lineno_ind, $linenos, $startline);
+
+ if (!$usedby || scalar(@$usedby) == 0) {
+ print OUT "// skipping unused ", def_name($def), " from ", $def->{filename}, ":$startline : ", $lines->[$startline] if $OUTOPTS->{skipped};
+ }
+ else {
+ my $lastline = $def->{lastline};
+ if (!defined($lastline)) {
+ print OUT "// WARNING: no last line for function ", def_name($def), " starting at ", $def->{filename}, ":$startline\n";
+ }
+ else {
+ print OUT "// definition ", def_name($def), " from ", $def->{filename}, ":$startline\n" if $OUTOPTS->{sourcelines};
+ for my $i ($startline..$lastline) {
+ print OUT $lines->[$i];
+ }
+ }
+ }
+ }
+ print_some_($lines, $lineno_ind, $linenos, -1);
+ }
+ close OUT unless $OUTFILE eq '-';
+}
+
+sub dump_undefs {
+ my ($undefs_hash) = @_;
+ my $s = '';
+ for my $refname (keys %$undefs_hash) {
+ my $fromdef = $undefs_hash->{$refname};
+ $s .= $SYMSEP . $refname . ' referenced from ' . $fromdef->{filename} . ':' . $fromdef->{startline} . "\n";
+ }
+ return $s;
+}
+
+sub main {
+ my $fileobjs = process_commandline();
+
+ # collect definitions and references
+ process_files($fileobjs);
+
+ # collect anchor file(s) and/or defs
+ my $start_defs = [];
+ my $anchor_filenames = [];
+ for (@$fileobjs) {
+ my $filename = $_->{filename};
+ next unless $_->{is_anchor};
+ my $anchor_defs = $DEFS_BY_FNAME->{$filename};
+ push(@$anchor_filenames, $filename);
+ push(@$start_defs, @$anchor_defs);
+ }
+ die "no anchor files" unless @$anchor_filenames;
+ die "no starting definitions" unless @$start_defs;
+
+ # perform transitive closure determining what other functions are required
+ my ($used_defs, $loopcount, $used_undefs, $unused_undefs) = transitive_closure($start_defs);
+
+ # overall summary
+ my $number_start_globals = scalar(@$anchor_filenames);
+ my $number_start = scalar(@$start_defs);
+ my $number_used_undefs = (scalar keys %$used_undefs);
+ my $number_unused_undefs = (scalar keys %$unused_undefs);
+ my @used_fileobjs = grep {!$_->{is_anchor} && $_->{globaldef}->{used}} @$fileobjs;
+ my @unused_fileobjs = grep {!$_->{is_anchor} && ! $_->{globaldef}->{used}} @$fileobjs;
+ info("Anchor files: @$anchor_filenames\n",
+ "Number of anchor symbol definitions: ", ($number_start - $number_start_globals), "\n",
+ "Number of anchor global expressions: ", $number_start_globals, "\n",
+ "Number used library definitions: ", (scalar(@$used_defs) - scalar(@$start_defs)), "\n",
+ "Number unused library definitions: ", (scalar(keys %$DEFS_BY_QUAL) - (scalar @$used_defs)), "\n",
+ "Number of library files used: ", scalar(@used_fileobjs), "\n",
+ "Number of library files unused: ", scalar(@unused_fileobjs), "\n",
+ "Number loop iterations required to obtain transitive closure: $loopcount\n",
+ "Number undefined symbols in used code: $number_used_undefs\n",
+ "Undefined symbols in used code:\n", dump_undefs($used_undefs),
+ "Number undefined symbols in unused code: $number_unused_undefs\n",
+ "Undefined symbols in unused code:\n", dump_undefs($unused_undefs),
+ "");
+
+ # list what files are not used at all, and what functions in files are not used
+ dump_syms($fileobjs, $used_undefs, $unused_undefs) if $DUMPOPTS->{used} || $DUMPOPTS->{unused};
+
+ die "not a single library file is used" unless scalar(@used_fileobjs) > 0;
+ print_minimal_files($fileobjs) if $OUTOPTS->{used};
+
+}
+
+
+main();
Propchange: incubator/xap/trunk/buildscripts/jslink.pl
------------------------------------------------------------------------------
svn:eol-style = native
Added: incubator/xap/trunk/buildscripts/lib/ant-apache-bsf.jar
URL: http://svn.apache.org/viewvc/incubator/xap/trunk/buildscripts/lib/ant-apache-bsf.jar?rev=432045&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/xap/trunk/buildscripts/lib/ant-apache-bsf.jar
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/xap/trunk/buildscripts/lib/bsf.jar
URL: http://svn.apache.org/viewvc/incubator/xap/trunk/buildscripts/lib/bsf.jar?rev=432045&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/xap/trunk/buildscripts/lib/bsf.jar
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/xap/trunk/buildscripts/lib/custom_rhino.diff
URL: http://svn.apache.org/viewvc/incubator/xap/trunk/buildscripts/lib/custom_rhino.diff?rev=432045&view=auto
==============================================================================
--- incubator/xap/trunk/buildscripts/lib/custom_rhino.diff (added)
+++ incubator/xap/trunk/buildscripts/lib/custom_rhino.diff Wed Aug 16 14:57:09 2006
@@ -0,0 +1,1058 @@
+Index: src/org/mozilla/javascript/BaseFunction.java
+===================================================================
+RCS file: /cvsroot/mozilla/js/rhino/src/org/mozilla/javascript/BaseFunction.java,v
+retrieving revision 1.57
+diff -u -u -r1.57 BaseFunction.java
+--- src/org/mozilla/javascript/BaseFunction.java 30 Aug 2005 10:05:42 -0000 1.57
++++ src/org/mozilla/javascript/BaseFunction.java 8 Nov 2005 00:05:45 -0000
+@@ -373,6 +373,28 @@
+ return sb.toString();
+ }
+
++ String compress(int indent, int flags)
++ {
++ StringBuffer sb = new StringBuffer();
++ String FuncName = null;
++ boolean justbody = (0 != (flags & Decompiler.ONLY_BODY_FLAG));
++ if (!justbody) {
++ sb.append("function");
++ FuncName = getFunctionName();
++ if(FuncName.length()>0){
++ sb.append(" "+FuncName);
++ }
++ sb.append("(){");
++ }
++ sb.append("[native code, arity=");
++ sb.append(getArity());
++ sb.append("]");
++ if (!justbody) {
++ sb.append("}");
++ }
++ return sb.toString();
++ }
++
+ public int getArity() { return 0; }
+
+ public int getLength() { return 0; }
+Index: src/org/mozilla/javascript/Context.java
+===================================================================
+RCS file: /cvsroot/mozilla/js/rhino/src/org/mozilla/javascript/Context.java,v
+retrieving revision 1.241
+diff -u -u -r1.241 Context.java
+--- src/org/mozilla/javascript/Context.java 26 Jun 2005 22:57:31 -0000 1.241
++++ src/org/mozilla/javascript/Context.java 8 Nov 2005 00:05:46 -0000
+@@ -1166,6 +1166,36 @@
+ }
+ }
+
++ public final String decompileReader(Scriptable scope, Reader in,
++ String sourceName, int lineno,
++ Object securityDomain)
++ throws IOException
++ {
++ Script script = compileReader(scope, in, sourceName, lineno,
++ securityDomain);
++ if (script != null) {
++ // System.err.println(script);
++ return decompileScript(script, 0);
++ } else {
++ return null;
++ }
++ }
++
++ public final String compressReader(Scriptable scope, Reader in,
++ String sourceName, int lineno,
++ Object securityDomain)
++ throws IOException
++ {
++ Script script = compileReader(scope, in, sourceName, lineno,
++ securityDomain);
++ if (script != null) {
++ // System.err.println(script);
++ return compressScript(script, 0);
++ } else {
++ return null;
++ }
++ }
++
+ /**
+ * Check whether a string is ready to be compiled.
+ * <p>
+@@ -1348,6 +1378,12 @@
+ return scriptImpl.decompile(indent, 0);
+ }
+
++ public final String compressScript(Script script, int indent)
++ {
++ NativeFunction scriptImpl = (NativeFunction) script;
++ return scriptImpl.compress(indent, 0);
++ }
++
+ /**
+ * Decompile a JavaScript Function.
+ * <p>
+Index: src/org/mozilla/javascript/Decompiler.java
+===================================================================
+RCS file: /cvsroot/mozilla/js/rhino/src/org/mozilla/javascript/Decompiler.java,v
+retrieving revision 1.19
+diff -u -u -r1.19 Decompiler.java
+--- src/org/mozilla/javascript/Decompiler.java 28 Aug 2005 23:25:22 -0000 1.19
++++ src/org/mozilla/javascript/Decompiler.java 8 Nov 2005 00:05:46 -0000
+@@ -1,3 +1,7 @@
++// FIXME: need to implement a stack of something that allows us to keep track
++// of what was pushed onto the replacement stack at what level and remove it
++// when we're above that level.
++
+ /* -*- Mode: java; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 4 -*-
+ *
+ * The contents of this file are subject to the Netscape Public
+@@ -37,6 +41,10 @@
+
+ package org.mozilla.javascript;
+
++import java.util.ArrayList;
++import java.util.Iterator;
++import java.util.HashMap;
++
+ /**
+ * The following class save decompilation information about the source.
+ * Source information is returned from the parser as a String
+@@ -70,6 +78,135 @@
+ * the final constant pool entry from information available at parse
+ * time.
+ */
++
++class TokenMapper {
++ private ArrayList functionBracePositions = new ArrayList();
++ private ArrayList scopeReplacedTokens = new ArrayList();
++ private int lastTokenCount = 0;
++
++ // FIXME: this isn't the brightest way to accomplish this. Fristly, we need
++ // to be sure we aren't colliding with other things in the namespace!
++ private String getMappedToken(String token, boolean newMapping){
++ String nt = null;
++ HashMap tokens = (HashMap)scopeReplacedTokens.get(scopeReplacedTokens.size()-1);
++ if(newMapping){
++ lastTokenCount++;
++ nt = new String("_"+Integer.toHexString(lastTokenCount));
++ if(nt.length() >= token.length()){
++ nt = token;
++ }
++ // System.out.println(nt);
++ tokens.put(token, nt);
++ return nt;
++ }
++ if(hasTokenMapping(token)){
++ return getTokenMapping(token);
++ }else{
++ return token;
++ }
++ }
++
++ private boolean hasLocalTokenMapping(String token){
++ if(scopeReplacedTokens.size()<1){
++ return false;
++ }
++ HashMap tokens = (HashMap)(scopeReplacedTokens.get(scopeReplacedTokens.size()-1));
++ if(tokens.containsKey(token)){
++ return true;
++ }
++ return false;
++ }
++
++ private boolean hasTokenMapping(String token){
++ String blank = new String("");
++ if(blank.equals(getTokenMapping(token))){
++ return false;
++ }
++ return true;
++ }
++
++ private String getTokenMapping(String token){
++ for(int i=scopeReplacedTokens.size()-1; i>=0; i--){
++ HashMap tokens = (HashMap)(scopeReplacedTokens.get(i));
++ if(tokens.containsKey(token)){
++ return (String)tokens.get(token);
++ }
++ }
++ return new String("");
++ }
++
++ public int printCompressed(String source,
++ int offset,
++ boolean asQuotedString,
++ StringBuffer sb,
++ int prevToken,
++ boolean inArgsList,
++ int currentLevel){
++ boolean newMapping = false;
++ int length = source.charAt(offset);
++ ++offset;
++ if((0x8000 & length) != 0){
++ length = ((0x7FFF & length) << 16) | source.charAt(offset);
++ ++offset;
++ }
++
++ if(sb != null){
++ String str = source.substring(offset, offset + length);
++ String sourceStr = new String(str);
++ if(((prevToken == Token.VAR)&&(!hasLocalTokenMapping(sourceStr)))||(inArgsList)){
++ newMapping = true;
++ }
++
++
++ if(((functionBracePositions.size()>0)&&(currentLevel>=(((Integer)functionBracePositions.get(functionBracePositions.size()-1)).intValue())))||(inArgsList)){
++ if(prevToken != Token.DOT){
++ str = this.getMappedToken(str, newMapping);
++ }
++ }
++ if((!inArgsList)&&(asQuotedString)){
++ if((prevToken == Token.LC)||(prevToken == Token.COMMA)){
++ str = sourceStr;
++ }
++ }
++
++ if(!asQuotedString){
++ sb.append(str);
++ }else{
++ sb.append('"');
++ sb.append(ScriptRuntime.escapeString(str));
++ sb.append('"');
++ }
++ }
++
++ return offset + length;
++ }
++
++ public void enterNestingLevel(int braceNesting){
++ functionBracePositions.add(new Integer(braceNesting+1));
++ scopeReplacedTokens.add(new HashMap());
++ }
++
++ public void leaveNestingLevel(int braceNesting){
++ Integer bn = new Integer(braceNesting);
++ if((functionBracePositions.contains(bn))&&(scopeReplacedTokens.size()>0)){
++ // remove our mappings now!
++ int scopedSize = scopeReplacedTokens.size();
++ /*
++ HashMap tokens = (HashMap)(scopeReplacedTokens.get(scopedSize-1));
++ Iterator titer = (tokens.keySet()).iterator();
++ String key = null;
++ while(titer.hasNext()){
++ key = (String)titer.next();
++ // System.out.println("removing: "+key);
++ tokenMappings.remove(key);
++ }
++ */
++ scopeReplacedTokens.remove(scopedSize-1);
++ functionBracePositions.remove(bn);
++ }
++ }
++}
++
+ public class Decompiler
+ {
+ /**
+@@ -102,6 +239,8 @@
+ // the last RC of object literals in case of function expressions
+ private static final int FUNCTION_END = Token.LAST_TOKEN + 1;
+
++
++
+ String getEncodedSource()
+ {
+ return sourceToString(0);
+@@ -266,6 +405,602 @@
+ return new String(sourceBuffer, offset, sourceTop - offset);
+ }
+
++ private static TokenMapper tm = new TokenMapper();
++
++ private static HashMap levelMappings = new HashMap();
++
++ public static String compress(String source, int flags,
++ UintMap properties)
++ {
++ int length = source.length();
++ if (length == 0) { return ""; }
++
++ int indent = properties.getInt(INITIAL_INDENT_PROP, 0);
++ if (indent < 0) throw new IllegalArgumentException();
++ int indentGap = properties.getInt(INDENT_GAP_PROP, 4);
++ if (indentGap < 0) throw new IllegalArgumentException();
++ int caseGap = properties.getInt(CASE_GAP_PROP, 2);
++ if (caseGap < 0) throw new IllegalArgumentException();
++
++ StringBuffer result = new StringBuffer();
++ boolean justFunctionBody = (0 != (flags & Decompiler.ONLY_BODY_FLAG));
++ boolean toSource = (0 != (flags & Decompiler.TO_SOURCE_FLAG));
++
++ // Spew tokens in source, for debugging.
++ // as TYPE number char
++ if (printSource) {
++ System.err.println("length:" + length);
++ for (int i = 0; i < length; ++i) {
++ // Note that tokenToName will fail unless Context.printTrees
++ // is true.
++ String tokenname = null;
++ if (Token.printNames) {
++ tokenname = Token.name(source.charAt(i));
++ }
++ if (tokenname == null) {
++ tokenname = "---";
++ }
++ String pad = tokenname.length() > 7
++ ? "\t"
++ : "\t\t";
++ System.err.println
++ (tokenname
++ + pad + (int)source.charAt(i)
++ + "\t'" + ScriptRuntime.escapeString
++ (source.substring(i, i+1))
++ + "'");
++ }
++ System.err.println();
++ }
++
++ int braceNesting = 0;
++ boolean afterFirstEOL = false;
++ int i = 0;
++ int prevToken = 0;
++ boolean primeFunctionNesting = false;
++ boolean inArgsList = false;
++ boolean primeInArgsList = false;
++ int topFunctionType;
++ if (source.charAt(i) == Token.SCRIPT) {
++ ++i;
++ topFunctionType = -1;
++ } else {
++ topFunctionType = source.charAt(i + 1);
++ }
++
++ if (!toSource) {
++ // add an initial newline to exactly match js.
++ // result.append('\n');
++ for (int j = 0; j < indent; j++){
++ // result.append(' ');
++ result.append("");
++ }
++ } else {
++ if (topFunctionType == FunctionNode.FUNCTION_EXPRESSION) {
++ result.append('(');
++ }
++ }
++
++ while (i < length) {
++ if(i>0){
++ prevToken = source.charAt(i-1);
++ }
++ // System.out.println(Token.name(getNext(source, length, i)));
++ switch(source.charAt(i)) {
++ case Token.NAME:
++ case Token.REGEXP: // re-wrapped in '/'s in parser...
++ int jumpPos = getSourceStringEnd(source, i+1);
++ if(Token.OBJECTLIT == source.charAt(jumpPos)){
++ i = printSourceString(source, i + 1, false, result);
++ }else{
++ i = tm.printCompressed( source, i + 1, false, result, prevToken,
++ inArgsList, braceNesting);
++ }
++ continue;
++
++ case Token.STRING:
++ i = printSourceString(source, i + 1, true, result);
++ continue;
++
++ case Token.NUMBER:
++ i = printSourceNumber(source, i + 1, result);
++ continue;
++
++ case Token.TRUE:
++ result.append("true");
++ break;
++
++ case Token.FALSE:
++ result.append("false");
++ break;
++
++ case Token.NULL:
++ result.append("null");
++ break;
++
++ case Token.THIS:
++ result.append("this");
++ break;
++
++ case Token.FUNCTION:
++ ++i; // skip function type
++ primeInArgsList = true;
++ primeFunctionNesting = true;
++ result.append("function");
++ if (Token.LP != getNext(source, length, i)) {
++ result.append(' ');
++ }
++ break;
++
++ case FUNCTION_END:
++ // Do nothing
++ break;
++
++ case Token.COMMA:
++ result.append(",");
++ break;
++
++ case Token.LC:
++ ++braceNesting;
++ if (Token.EOL == getNext(source, length, i)){
++ indent += indentGap;
++ }
++ result.append('{');
++ // // result.append('\n');
++ break;
++
++ case Token.RC: {
++ tm.leaveNestingLevel(braceNesting);
++ --braceNesting;
++
++ /* don't print the closing RC if it closes the
++ * toplevel function and we're called from
++ * decompileFunctionBody.
++ */
++ if(justFunctionBody && braceNesting == 0){
++ break;
++ }
++
++ // // result.append('\n');
++ result.append('}');
++ // // result.append(' ');
++ switch (getNext(source, length, i)) {
++ case Token.EOL:
++ case FUNCTION_END:
++ indent -= indentGap;
++ break;
++ case Token.WHILE:
++ case Token.ELSE:
++ indent -= indentGap;
++ // result.append(' ');
++ result.append("");
++ break;
++ }
++ break;
++ }
++ case Token.LP:
++ if(primeInArgsList){
++ inArgsList = true;
++ primeInArgsList = false;
++ }
++ if(primeFunctionNesting){
++ tm.enterNestingLevel(braceNesting);
++ primeFunctionNesting = false;
++ }
++ result.append('(');
++ break;
++
++ case Token.RP:
++ if(inArgsList){
++ inArgsList = false;
++ }
++ result.append(')');
++ /*
++ if (Token.LC == getNext(source, length, i)){
++ result.append(' ');
++ }
++ */
++ break;
++
++ case Token.LB:
++ result.append('[');
++ break;
++
++ case Token.RB:
++ result.append(']');
++ break;
++
++ case Token.EOL: {
++ if (toSource) break;
++ boolean newLine = true;
++ if (!afterFirstEOL) {
++ afterFirstEOL = true;
++ if (justFunctionBody) {
++ /* throw away just added 'function name(...) {'
++ * and restore the original indent
++ */
++ result.setLength(0);
++ indent -= indentGap;
++ newLine = false;
++ }
++ }
++ if (newLine) {
++ result.append('\n');
++ }
++ /*
++ */
++
++ /* add indent if any tokens remain,
++ * less setback if next token is
++ * a label, case or default.
++ */
++ if (i + 1 < length) {
++ int less = 0;
++ int nextToken = source.charAt(i + 1);
++ if (nextToken == Token.CASE
++ || nextToken == Token.DEFAULT)
++ {
++ less = indentGap - caseGap;
++ } else if (nextToken == Token.RC) {
++ less = indentGap;
++ }
++
++ /* elaborate check against label... skip past a
++ * following inlined NAME and look for a COLON.
++ */
++ else if (nextToken == Token.NAME) {
++ int afterName = getSourceStringEnd(source, i + 2);
++ if (source.charAt(afterName) == Token.COLON)
++ less = indentGap;
++ }
++
++ for (; less < indent; less++){
++ // result.append(' ');
++ result.append("");
++ }
++ }
++ break;
++ }
++ case Token.DOT:
++ result.append('.');
++ break;
++
++ case Token.NEW:
++ result.append("new ");
++ break;
++
++ case Token.DELPROP:
++ result.append("delete ");
++ break;
++
++ case Token.IF:
++ result.append("if");
++ break;
++
++ case Token.ELSE:
++ result.append("else");
++ break;
++
++ case Token.FOR:
++ result.append("for");
++ break;
++
++ case Token.IN:
++ result.append(" in ");
++ break;
++
++ case Token.WITH:
++ result.append("with");
++ break;
++
++ case Token.WHILE:
++ result.append("while");
++ break;
++
++ case Token.DO:
++ result.append("do");
++ break;
++
++ case Token.TRY:
++ result.append("try");
++ break;
++
++ case Token.CATCH:
++ result.append("catch");
++ break;
++
++ case Token.FINALLY:
++ result.append("finally");
++ break;
++
++ case Token.THROW:
++ result.append("throw ");
++ break;
++
++ case Token.SWITCH:
++ result.append("switch");
++ break;
++
++ case Token.BREAK:
++ result.append("break");
++ if(Token.NAME == getNext(source, length, i)){
++ result.append(' ');
++ }
++ break;
++
++ case Token.CONTINUE:
++ result.append("continue");
++ if(Token.NAME == getNext(source, length, i)){
++ result.append(' ');
++ }
++ break;
++
++ case Token.CASE:
++ result.append("case ");
++ break;
++
++ case Token.DEFAULT:
++ result.append("default");
++ break;
++
++ case Token.RETURN:
++ result.append("return");
++ if(Token.SEMI != getNext(source, length, i)){
++ result.append(' ');
++ }
++ break;
++
++ case Token.VAR:
++ result.append("var ");
++ break;
++
++ case Token.SEMI:
++ result.append(';');
++ // result.append('\n');
++ /*
++ if (Token.EOL != getNext(source, length, i)) {
++ // separators in FOR
++ result.append(' ');
++ }
++ */
++ break;
++
++ case Token.ASSIGN:
++ result.append("=");
++ break;
++
++ case Token.ASSIGN_ADD:
++ result.append("+=");
++ break;
++
++ case Token.ASSIGN_SUB:
++ result.append("-=");
++ break;
++
++ case Token.ASSIGN_MUL:
++ result.append("*=");
++ break;
++
++ case Token.ASSIGN_DIV:
++ result.append("/=");
++ break;
++
++ case Token.ASSIGN_MOD:
++ result.append("%=");
++ break;
++
++ case Token.ASSIGN_BITOR:
++ result.append("|=");
++ break;
++
++ case Token.ASSIGN_BITXOR:
++ result.append("^=");
++ break;
++
++ case Token.ASSIGN_BITAND:
++ result.append("&=");
++ break;
++
++ case Token.ASSIGN_LSH:
++ result.append("<<=");
++ break;
++
++ case Token.ASSIGN_RSH:
++ result.append(">>=");
++ break;
++
++ case Token.ASSIGN_URSH:
++ result.append(">>>=");
++ break;
++
++ case Token.HOOK:
++ result.append("?");
++ break;
++
++ case Token.OBJECTLIT:
++ // pun OBJECTLIT to mean colon in objlit property
++ // initialization.
++ // This needs to be distinct from COLON in the general case
++ // to distinguish from the colon in a ternary... which needs
++ // different spacing.
++ result.append(':');
++ break;
++
++ case Token.COLON:
++ if (Token.EOL == getNext(source, length, i))
++ // it's the end of a label
++ result.append(':');
++ else
++ // it's the middle part of a ternary
++ result.append(":");
++ break;
++
++ case Token.OR:
++ result.append("||");
++ break;
++
++ case Token.AND:
++ result.append("&&");
++ break;
++
++ case Token.BITOR:
++ result.append("|");
++ break;
++
++ case Token.BITXOR:
++ result.append("^");
++ break;
++
++ case Token.BITAND:
++ result.append("&");
++ break;
++
++ case Token.SHEQ:
++ result.append("===");
++ break;
++
++ case Token.SHNE:
++ result.append("!==");
++ break;
++
++ case Token.EQ:
++ result.append("==");
++ break;
++
++ case Token.NE:
++ result.append("!=");
++ break;
++
++ case Token.LE:
++ result.append("<=");
++ break;
++
++ case Token.LT:
++ result.append("<");
++ break;
++
++ case Token.GE:
++ result.append(">=");
++ break;
++
++ case Token.GT:
++ result.append(">");
++ break;
++
++ case Token.INSTANCEOF:
++ // FIXME: does this really need leading space?
++ result.append(" instanceof ");
++ break;
++
++ case Token.LSH:
++ result.append("<<");
++ break;
++
++ case Token.RSH:
++ result.append(">>");
++ break;
++
++ case Token.URSH:
++ result.append(">>>");
++ break;
++
++ case Token.TYPEOF:
++ result.append("typeof ");
++ break;
++
++ case Token.VOID:
++ result.append("void ");
++ break;
++
++ case Token.NOT:
++ result.append('!');
++ break;
++
++ case Token.BITNOT:
++ result.append('~');
++ break;
++
++ case Token.POS:
++ result.append('+');
++ break;
++
++ case Token.NEG:
++ result.append('-');
++ break;
++
++ case Token.INC:
++ if(Token.ADD == prevToken){
++ result.append(' ');
++ }
++ result.append("++");
++ if(Token.ADD == getNext(source, length, i)){
++ result.append(' ');
++ }
++ break;
++
++ case Token.DEC:
++ if(Token.SUB == prevToken){
++ result.append(' ');
++ }
++ result.append("--");
++ if(Token.SUB == getNext(source, length, i)){
++ result.append(' ');
++ }
++ break;
++
++ case Token.ADD:
++ result.append("+");
++ break;
++
++ case Token.SUB:
++ result.append("-");
++ break;
++
++ case Token.MUL:
++ result.append("*");
++ break;
++
++ case Token.DIV:
++ result.append("/");
++ break;
++
++ case Token.MOD:
++ result.append("%");
++ break;
++
++ case Token.COLONCOLON:
++ result.append("::");
++ break;
++
++ case Token.DOTDOT:
++ result.append("..");
++ break;
++
++ case Token.XMLATTR:
++ result.append('@');
++ break;
++
++ default:
++ // If we don't know how to decompile it, raise an exception.
++ throw new RuntimeException();
++ }
++ ++i;
++ }
++
++ if (!toSource) {
++ // add that trailing newline if it's an outermost function.
++ // if (!justFunctionBody){
++ // result.append('\n');
++ // }
++ } else {
++ if (topFunctionType == FunctionNode.FUNCTION_EXPRESSION) {
++ result.append(')');
++ }
++ }
++
++ return result.toString();
++ }
++
+ /**
+ * Decompile the source information associated with this js
+ * function/script back into a string. For the most part, this
+@@ -301,7 +1036,7 @@
+
+ // Spew tokens in source, for debugging.
+ // as TYPE number char
+- if (printSource) {
++ if(printSource){
+ System.err.println("length:" + length);
+ for (int i = 0; i < length; ++i) {
+ // Note that tokenToName will fail unless Context.printTrees
+Index: src/org/mozilla/javascript/NativeFunction.java
+===================================================================
+RCS file: /cvsroot/mozilla/js/rhino/src/org/mozilla/javascript/NativeFunction.java,v
+retrieving revision 1.62
+diff -u -u -r1.62 NativeFunction.java
+--- src/org/mozilla/javascript/NativeFunction.java 17 Jan 2005 13:06:33 -0000 1.62
++++ src/org/mozilla/javascript/NativeFunction.java 8 Nov 2005 00:05:46 -0000
+@@ -70,6 +70,18 @@
+ }
+ }
+
++ final String compress(int indent, int flags)
++ {
++ String encodedSource = getEncodedSource();
++ if (encodedSource == null) {
++ return super.compress(indent, flags);
++ } else {
++ UintMap properties = new UintMap(1);
++ properties.put(Decompiler.INITIAL_INDENT_PROP, indent);
++ return Decompiler.compress(encodedSource, flags, properties);
++ }
++ }
++
+ public int getLength()
+ {
+ int paramCount = getParamCount();
+Index: src/org/mozilla/javascript/TokenStream.java
+===================================================================
+RCS file: /cvsroot/mozilla/js/rhino/src/org/mozilla/javascript/TokenStream.java,v
+retrieving revision 1.63
+diff -u -u -r1.63 TokenStream.java
+--- src/org/mozilla/javascript/TokenStream.java 31 Jul 2005 13:48:46 -0000 1.63
++++ src/org/mozilla/javascript/TokenStream.java 8 Nov 2005 00:05:47 -0000
+@@ -64,9 +64,12 @@
+ private final static int
+ EOF_CHAR = -1;
+
++ public StringBuffer lastComment;
++
+ TokenStream(Parser parser, Reader sourceReader, String sourceString,
+ int lineno)
+ {
++ this.lastComment = new StringBuffer();
+ this.parser = parser;
+ this.lineno = lineno;
+ if (sourceReader != null) {
+@@ -736,6 +739,8 @@
+
+ case '/':
+ // is it a // comment?
++ // FIXME: RAR: comment, need to set config to optionally keep
++ // instead of skipping!
+ if (matchChar('/')) {
+ skipLine();
+ continue retry;
+Index: toolsrc/org/mozilla/javascript/tools/shell/Main.java
+===================================================================
+RCS file: /cvsroot/mozilla/js/rhino/toolsrc/org/mozilla/javascript/tools/shell/Main.java,v
+retrieving revision 1.64
+diff -u -u -r1.64 Main.java
+--- toolsrc/org/mozilla/javascript/tools/shell/Main.java 2 Sep 2005 14:18:40 -0000 1.64
++++ toolsrc/org/mozilla/javascript/tools/shell/Main.java 8 Nov 2005 00:05:47 -0000
+@@ -237,6 +237,10 @@
+ shellContextFactory.call(iproxy);
+ continue;
+ }
++ if (arg.equals("-c")) {
++ outputCompressed = true;
++ continue;
++ }
+ if (arg.equals("-w")) {
+ errorReporter.setIsReportingWarnings(true);
+ continue;
+@@ -372,10 +376,13 @@
+ String path, Object securityDomain)
+ {
+ Script script;
++ Reader cin = null;
++ String cout = null;
++ String source = null;
+ if (path.endsWith(".class")) {
+ script = loadCompiledScript(cx, path, securityDomain);
+ } else {
+- String source = (String)readFileOrUrl(path, true);
++ source = (String)readFileOrUrl(path, true);
+ if (source == null) {
+ exitCode = EXITCODE_FILE_NOT_FOUND;
+ return;
+@@ -395,11 +402,83 @@
+ }
+ script = loadScriptFromSource(cx, source, path, 1, securityDomain);
+ }
+- if (script != null) {
+- evaluateScript(script, cx, scope);
++ if((script != null)||(source != null)){
++ if(outputCompressed){
++ try {
++ URL url = new URL(path);
++ global.getOut().println(url.toString());
++ InputStream is = url.openStream();
++ cin = new BufferedReader(new InputStreamReader(is));
++ } catch (MalformedURLException mfex) {
++ // global.getOut().println(mfex);
++ cin = null;
++ } catch (IOException ioex) {
++ return;
++ }
++ if (cin == null) {
++ // Try filename as file
++ try {
++ cin = new PushbackReader(new FileReader(path));
++ } catch (FileNotFoundException ex) {
++ // global.getErr().println(ex.toString());
++ return;
++ } catch (IOException ioe) {
++ global.getErr().println(ioe.toString());
++ }
++ }
++ cout = compressScript(cx, scope, cin, source, path, 1, securityDomain);
++ global.getOut().println(cout);
++ }else{
++ evaluateScript(script, cx, scope);
++ }
+ }
+ }
+
++
++ public static String compressScript(Context cx, Scriptable scope,
++ Reader in, String script,
++ String sourceName,
++ int lineno, Object securityDomain)
++ {
++ String result = null;
++ try {
++ if (in != null) {
++ try {
++ try {
++ result = cx.compressReader(scope, in, sourceName,
++ lineno, securityDomain);
++ } finally {
++ in.close();
++ }
++ } catch (IOException ioe) {
++ global.getErr().println(ioe.toString());
++ }
++ } else {
++ // FIXME: this is wrong!!!!
++ // we should probably throw the string into a temp file
++ // and then have it read in again.
++ result = script;
++ }
++ } catch (WrappedException we) {
++ global.getErr().println(we.getWrappedException().toString());
++ we.printStackTrace();
++ } catch (EvaluatorException ee) {
++ // Already printed message.
++ exitCode = EXITCODE_RUNTIME_ERROR;
++ } catch (RhinoException rex) {
++ errorReporter.reportException(rex);
++ exitCode = EXITCODE_RUNTIME_ERROR;
++ } catch (VirtualMachineError ex) {
++ // Treat StackOverflow and OutOfMemory as runtime errors
++ ex.printStackTrace();
++ String msg = ToolErrorReporter.getMessage(
++ "msg.uncaughtJSException", ex.toString());
++ exitCode = EXITCODE_RUNTIME_ERROR;
++ Context.reportError(msg);
++ }
++ return result;
++ }
++
+ public static Script loadScriptFromSource(Context cx, String scriptSource,
+ String path, int lineno,
+ Object securityDomain)
+@@ -592,6 +671,7 @@
+ static private final int EXITCODE_RUNTIME_ERROR = 3;
+ static private final int EXITCODE_FILE_NOT_FOUND = 4;
+ static boolean processStdin = true;
++ static boolean outputCompressed = false;
+ static Vector fileList = new Vector(5);
+ private static SecurityProxy securityImpl;
+ }
Added: incubator/xap/trunk/buildscripts/lib/custom_rhino.jar
URL: http://svn.apache.org/viewvc/incubator/xap/trunk/buildscripts/lib/custom_rhino.jar?rev=432045&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/xap/trunk/buildscripts/lib/custom_rhino.jar
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/xap/trunk/buildscripts/lib/js.jar
URL: http://svn.apache.org/viewvc/incubator/xap/trunk/buildscripts/lib/js.jar?rev=432045&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/xap/trunk/buildscripts/lib/js.jar
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/xap/trunk/buildscripts/lib/jython.jar
URL: http://svn.apache.org/viewvc/incubator/xap/trunk/buildscripts/lib/jython.jar?rev=432045&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/xap/trunk/buildscripts/lib/jython.jar
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/xap/trunk/buildscripts/lib/pyLib.zip
URL: http://svn.apache.org/viewvc/incubator/xap/trunk/buildscripts/lib/pyLib.zip?rev=432045&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/xap/trunk/buildscripts/lib/pyLib.zip
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream