DO NOT REPLY [Bug 20103] New: - FileSet horrible performance when dir has huge number of subdirs

[bu...@apache.org]

DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG 
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://nagoya.apache.org/bugzilla/show_bug.cgi?id=20103>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND 
INSERTED IN THE BUG DATABASE.

http://nagoya.apache.org/bugzilla/show_bug.cgi?id=20103

FileSet horrible performance when dir has huge number of subdirs

           Summary: FileSet horrible performance when dir has huge number of
                    subdirs
           Product: Ant
           Version: 1.5.3
          Platform: Other
        OS/Version: Other
            Status: NEW
          Severity: Major
          Priority: Other
         Component: Core
        AssignedTo: dev@ant.apache.org
        ReportedBy: e17x-o9j5@spamex.com


When I tried to breakout the external jar filenames from the a classpath 
<path> specification into a separate fileset, I ran into a huge performance 
problem.

In my build.xml, the BASE_ENG_DIR is a root directory, e.g. /engs51, of
a project release. There are a huge number of subdirectories (1000s) underneath
this directory. It appears the fileset is scanning all these subdirectories 
because the performance goes to hell.  The worst is if you try setting dir 
to "/".

What I would prefer to do is to not even specify a dir tag, and just
list the files with absolute pathnames, and avoid any filtering/scanning.
Alternativel, if fileset could be optimized so that it doesn't scan directories
if no regexp filters are included.

Ant doesn't seem to have an easy and efficient way to add a list of files that 
are in different directories to a FileList/FileSet..

BTW the main reason I was converting the external jar filelist to a FileSet 
was so that I could use the Copy task to copy them to a different package 
directory.  The Copy task doesn't take a <path> only <fileset>.  

There should be a way to convert between <fileset> and <path> type. <fileset> 
should be more flexible in allowing absolute path names. Only relative 
pathnames should be considered relative to dir. <fileset> should allow dir tag 
to be optional, in which case it defaults to ${basedir} or is just null and 
enfoces all filename/patternet/selectors underneath it use absolute pathnames.

        <--! horrible performance when BASE_ENG_DIR = /engs51 -->
        <fileset id="PN.external.jars" dir=${BASE_ENG_DIR}>
            <filename name="${JAVA_DIR}/jre/lib/rt.jar"/>
            <!-- LOG4J_JAR -->
            <filename name="${APPS3RDPARTY}/log4j1.1.3/log4j.jar"/>
            <!-- JSDK_LIB -->
            <filename name="${JSDK_DIR}/lib/jsdk.jar"/>
            <!-- POP3_JAR -->
            <filename name="${APPS3RDPARTY}/pop3/pop3.jar"/>
            <!-- JSSE_LIB -->
            <filename name="${JSSE_DIR}/lib/jcert.jar"/>
            <filename name="${JSSE_DIR}/lib/jnet.jar"/>
            <filename name="${JSSE_DIR}/lib/jsse.jar"/>
            <!-- ORM_JAR -->
            <filename name="${APPS3RDPARTY}/orm/OROMatcher.jar"/>
            <!-- JCE_LIB -->
            <filename name="${JCE_DIR}/lib/jce.jar"/>
            <!-- ICE_LIB -->
            <filename name="${ICE_DIR}/ice.jar"/>
            <!-- APACHE_JSERV_LIB -->
            <filename name="${APACHE_DIR}/libexec/ApacheJServ.jar"/>
            <!-- XML_APIS -->
            <filename name="${APPS3RDPARTY}/xalan/xml-apis.jar"/>
            <!-- XERCES_LIB -->
            <filename name="${XERCES_DIR}/xercesImpl.jar"/>
            <filename name="${XERCES_DIR}/xmlParserAPIs.jar"/>
            <filename name="${XERCES_DIR}/xercesSamples.jar"/>
            <!-- COLT_LIB -->
            <filename name="${COLT_DIR}/colt.jar"/>
            <!-- JNDI_JAR -->
            <filename name="${APPS3RDPARTY}/jndi/jndi.jar"/>
            <!-- JMS_JAR -->
            <filename name="${STATIC_DIR}/javax/jms.jar"/>
            <!-- JORAM_JAR -->
            <filename name="${STATIC_DIR}/joram2_1_1/joram.jar"/>
            <!-- JORAM_JNDI_JAR -->
            <filename name="${STATIC_DIR}/joram2_1_1/a3jndi.jar"/>
            <!-- SITRAKA_JAR -->
            <filename name="${APPS3RDPARTY}/graphing/sitraka/jcschart.jar"/>
            <!-- JAVAMAIL_JAR -->
            <filename name="${JAVAMAIL_DIR}/activation.jar"/>
            <filename name="${JAVAMAIL_DIR}/mail.jar"/>
            <!-- ELEGANT_JAR -->
            <filename name="${ELEGANT_DIR}/common.jar"/>
            <filename name="${ELEGANT_DIR}/dialgauge.jar"/>
            <!-- KUNSTSTOFF_JAR -->
            <filename name="${APPS3RDPARTY}/kunststoff/kunststoff.jar"/>
   </fileset>
   <path id="PN.class.path">
        <pathelement path="${CLASS_DIR}"/>
        <selector refid="PN.external.jars"/>
   </path>