You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Benoit Montuelle (JIRA)" <ji...@codehaus.org> on 2012/11/27 19:04:14 UTC
[jira] (MDEP-391) Unpack fails on linux with large archives
Benoit Montuelle created MDEP-391:
-------------------------------------
Summary: Unpack fails on linux with large archives
Key: MDEP-391
URL: https://jira.codehaus.org/browse/MDEP-391
Project: Maven 2.x Dependency Plugin
Issue Type: Bug
Components: unpack
Affects Versions: 2.6
Environment: Ubuntu 10.04 LTS
Reporter: Benoit Montuelle
When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
[INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
[INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
... 26 more
Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at java.lang.Runtime.exec(Runtime.java:593)
at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
... 29 more
Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 31 more
This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
I guess 1024+ file pointers open to extract is not a good solution.
Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=314526#comment-314526 ]
Kristian Rosenvold commented on MDEP-391:
-----------------------------------------
@Benoit; I'd be really interested in hearing if 2.7 SNAPSHOT fixes the problem of leaking file handles since that'd give a pretty good pinpoint as to the cause of the problem.
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=314544#comment-314544 ]
Olivier Lamy commented on MDEP-391:
-----------------------------------
done https://svn.apache.org/r1414637
snapshot deployed too.
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
> Attachments: lsof.txt
>
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=314539#comment-314539 ]
Kristian Rosenvold commented on MDEP-391:
-----------------------------------------
Are you running with a special jdk version or a security manager active ?
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
> Attachments: lsof.txt
>
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=314502#comment-314502 ]
Kristian Rosenvold commented on MDEP-391:
-----------------------------------------
Can you use lsof -p <pid> to get some indication of what files are being leaked ?
Please attach the dump here (anonymize if necessary or send me directly if not possible to anonymize)
Also, the stack trace seems to indicate we may have a bug wrt the "useJvmChmod" flag of the plugin, can you try setting it to an explicit "true" value and see if that helps ?
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=314512#comment-314512 ]
Olivier Lamy commented on MDEP-391:
-----------------------------------
@Kristian using the flag ignorePermissions is an other possible option :-)
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=314545#comment-314545 ]
Olivier Lamy commented on MDEP-391:
-----------------------------------
@benoit please test last snapshot
merci :-)
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
> Attachments: lsof.txt
>
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Benoit Montuelle (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=314532#comment-314532 ]
Benoit Montuelle commented on MDEP-391:
---------------------------------------
Thank you,
I attached the dump of lsof juste before the crash (ran within a watch every 5 seconds to get it) it seems to use a lot of pipes.
I've tried to use depency plugin version 2.7-SNAPSHOT but could not get one of its dependencies from public repository. Our local nexus has previous released version artifacts of plexus-archiver, til 2.2, but cant get 2.3-SNAPSHOT
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:2.7-SNAPSHOT:unpack (unpack-symfony) on project ecg-deploy: Execution unpack-symfony of goal org.apache.maven.plugins:maven-dependency-plugin:2.7-SNAPSHOT:unpack failed: Plugin org.apache.maven.plugins:maven-dependency-plugin:2.7-SNAPSHOT or one of its dependencies could not be resolved: Could not find artifact org.codehaus.plexus:plexus-archiver:jar:2.3-SNAPSHOT in nexus (http://repo.<xxx>.local/nexus/content/groups/public) -> [Help 1]
[ERROR]
I think we didnt have plexus snapshots repo configured
I also tried with the useJvmChmod explicitely false on execution options, and got a different trace but same error in the end
[INFO] Expanding: /tmp/tmp8086128828239170014.tar into /var/lib/deploy/build-deploy/ecg/target/classes
org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error setting up environmental variables
at org.codehaus.plexus.util.cli.Commandline.getEnvironmentVariables(Commandline.java:462)
at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:624)
at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
... 26 more
Caused by: java.io.IOException: Cannot run program "env": java.io.IOException: error=24, Too many open files
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at java.lang.Runtime.exec(Runtime.java:593)
at java.lang.Runtime.exec(Runtime.java:431)
at java.lang.Runtime.exec(Runtime.java:328)
at org.codehaus.plexus.util.cli.CommandLineUtils.getSystemEnvVars(CommandLineUtils.java:218)
at org.codehaus.plexus.util.cli.CommandLineUtils.getSystemEnvVars(CommandLineUtils.java:182)
at org.codehaus.plexus.util.cli.Commandline.addSystemEnvironment(Commandline.java:438)
at org.codehaus.plexus.util.cli.Commandline.getEnvironmentVariables(Commandline.java:458)
... 30 more
Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 37 more
With useJvmChmod set to true I still got the very same error as in original ticket.
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
> Attachments: lsof.txt
>
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=314543#comment-314543 ]
Olivier Lamy commented on MDEP-391:
-----------------------------------
there is an other flag possible ignorePermissions to simplify don't set any permissions when unpacking dependencies.
I will add that too with true per default (IMHO won't be an issue with dependency plugin)
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
> Attachments: lsof.txt
>
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=314533#comment-314533 ]
Olivier Lamy commented on MDEP-391:
-----------------------------------
https://oss.sonatype.org/content/repositories/plexus-snapshots
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
> Attachments: lsof.txt
>
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Olivier Lamy (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=314511#comment-314511 ]
Olivier Lamy commented on MDEP-391:
-----------------------------------
http://svn.apache.org/r1414445 use plexus-archiver with useJvmChmod on per default.
try with dependency plugin 2.7-SNAPSHOT
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Benoit Montuelle (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Montuelle updated MDEP-391:
----------------------------------
Attachment: lsof.txt
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
> Attachments: lsof.txt
>
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (MDEP-391) Unpack fails on linux with large archives
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/MDEP-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=314536#comment-314536 ]
Kristian Rosenvold commented on MDEP-391:
-----------------------------------------
Ok, the root cause of your problem seems to be plexus-utils CommandLineUtils.getSystemEnvVars, which does not properly
handle the process streams. (Specifically it does not drain error stream of process which will leak pipes if output appears on stderr)
I created PLXUTILS-155 to deal with this.
The thing about the "useJvmChmod" is not relevant for this issue, but only came up because I saw a stacktrace from your code ;)
> Unpack fails on linux with large archives
> -----------------------------------------
>
> Key: MDEP-391
> URL: https://jira.codehaus.org/browse/MDEP-391
> Project: Maven 2.x Dependency Plugin
> Issue Type: Bug
> Components: unpack
> Affects Versions: 2.6
> Environment: Ubuntu 10.04 LTS
> Reporter: Benoit Montuelle
> Attachments: lsof.txt
>
>
> When unpacking "large " archives with numerous files (more than 1024 on this os) I got the following error trace
> [INFO] Expanding /var/lib/deploy/.m2/repository/com/aprilwaf/wafservice/ihm-packaging-ecg/3.9-finsouscription-SNAPSHOT/ihm-packaging-ecg-3.9-finsouscription-SNAPSHOT-distribution-php.tar.gz to /tmp/tmp5620480517649479513.tar
> [INFO] Expanding: /tmp/tmp5620480517649479513.tar into /var/lib/deploy/build-deploy/ecg/target/classes
> org.codehaus.plexus.archiver.ArchiverException: Error while executing chmod.
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:64)
> at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.extractFile(AbstractZipUnArchiver.java:236)
> at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute(TarUnArchiver.java:92)
> at org.codehaus.plexus.archiver.tar.TarGZipUnArchiver.execute(TarGZipUnArchiver.java:76)
> at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:108)
> at org.apache.maven.plugin.dependency.AbstractDependencyMojo.unpack(AbstractDependencyMojo.java:260)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.unpackArtifact(UnpackMojo.java:122)
> at org.apache.maven.plugin.dependency.fromConfiguration.UnpackMojo.execute(UnpackMojo.java:95)
> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
> at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
> Caused by: org.codehaus.plexus.util.cli.CommandLineException: Error while executing process.
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:652)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:102)
> at org.codehaus.plexus.util.cli.CommandLineUtils.executeCommandLine(CommandLineUtils.java:89)
> at org.codehaus.plexus.archiver.util.ArchiveEntryUtils.chmod(ArchiveEntryUtils.java:45)
> ... 26 more
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/deploy/build-deploy/ecg/target/classes/symfony-1.4/lib/plugins/sfDoctrinePlugin/test/unit/record"): java.io.IOException: error=24, Too many open files
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
> at java.lang.Runtime.exec(Runtime.java:593)
> at org.codehaus.plexus.util.cli.Commandline.execute(Commandline.java:647)
> ... 29 more
> Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
> ... 31 more
> This error reproduce on every server tested, we already override linux file pointer limitations for our developers systems but can't do this in these servers.
> I guess 1024+ file pointers open to extract is not a good solution.
> Btw I noticed the tar.cz if first unzipped to /tmp which can be problematic by eating too much disk space during operation.
> Could you have a look ? I guess we could go through other plugins in the meanwhile to extract the tar.gz with command line
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira