You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Carsten Hammer (JIRA)" <ji...@apache.org> on 2019/01/12 18:39:00 UTC
[jira] [Comment Edited] (SUREFIRE-1622) failure to run tests if
classpath gets too long (?)
[ https://issues.apache.org/jira/browse/SUREFIRE-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16741354#comment-16741354 ]
Carsten Hammer edited comment on SUREFIRE-1622 at 1/12/19 6:38 PM:
-------------------------------------------------------------------
Not sure if I really have useful results so far.
After running with mavens "-X" option to see some debug output there is *no* surefire booter jar in the "target/surefire" folder. Only two files with these names:
{noformat}
surefire_12055272397433082914tmp (size 326 bytes)
surefire_12055272397433082914tmp (size 305.414 bytes{noformat}
Both files contain key=value pairs. The bigger one contains two lines like this that are longer than 256 characters but I did not find any hint that this can be a problem:
{code:java}
..
classPathUrl.717=/var/lib/jenkins/.m2/repository/p2/osgi/bundle/org.eclipse.datatools.enablement.sybase.asa.schemaobjecteditor.examples.nl_de/4.8.0.v20180815020001/org.eclipse.datatools.enablement.sybase.asa.schemaobjecteditor.examples.nl_de-4.8.0.v20180815020001.jar
..
classPathUrl.835=/var/lib/jenkins/.m2/repository/p2/osgi/bundle/org.eclipse.datatools.enablement.sybase.asa.schemaobjecteditor.examples.nl_fr/4.8.0.v20180815020001/org.eclipse.datatools.enablement.sybase.asa.schemaobjecteditor.examples.nl_fr-4.8.0.v20180815020001.jar
..
{code}
What I can find is that the file "target/test-classes/javac.sh" is
{noformat}
/bin/sh -c cd "/var/lib/jenkins/jobs/myproject_trunk svn/workspace/buildhelper" && /var/lib/jenkins/tools/hudson.model.JDK/java8/jre/../bin/javac @/var/lib/jenkins/jobs/myproject_trunk svn/workspace/buildhelper/target/test-classes/org.codehaus.plexus.compiler.javac.JavacCompiler2330981713839822451arguments -J-Xmx712m -J-Xms328m{noformat}
The size of the file "org.codehaus.plexus.compiler.javac.JavacCompiler2330981713839822451arguments" in the same folder is 274.463 Bytes.
Alone the one line with the classpath is 270175 characters long.
It is what is following the "-classpath" parameter. So I guess in fact the classpath is used as parameter of the commandline after translation by the /bin/sh process and there you run against a limit in the operating system because of the long classpath. Interesting is of course that it is not the shell that expands the file to a commandline. The problem is with the java compiler itself and its feature with "@<filename>" to use a file with a classpath that has a undocumented limitation. see [https://download.oracle.com/otn_hosted_doc/jdeveloper/904preview/jdk14doc/docs/tooldocs/windows/javac.html]
The test classpath in my case is of course a little bit longer than the compiling classpath and the module in question is the one with the longest classpath of all modules as it aggregates the other modules for integration tests. So I guess I cannot do anything else than trying to reduce the length of the classpath. I will try to do it now.
Best regards,
Carsten
was (Author: chammer2):
Not sure if I really have useful results so far.
After running with mavens "-X" option to see some debug output there is *no* surefire booter jar in the "target/surefire" folder. Only two files with these names:
{noformat}
surefire_12055272397433082914tmp (size 326 bytes)
surefire_12055272397433082914tmp (size 305.414 bytes{noformat}
Both files contain key=value pairs. The bigger one contains two lines like this that are longer than 256 characters but I did not find any hint that this can be a problem:
{code:java}
..
classPathUrl.717=/var/lib/jenkins/.m2/repository/p2/osgi/bundle/org.eclipse.datatools.enablement.sybase.asa.schemaobjecteditor.examples.nl_de/4.8.0.v20180815020001/org.eclipse.datatools.enablement.sybase.asa.schemaobjecteditor.examples.nl_de-4.8.0.v20180815020001.jar
..
classPathUrl.835=/var/lib/jenkins/.m2/repository/p2/osgi/bundle/org.eclipse.datatools.enablement.sybase.asa.schemaobjecteditor.examples.nl_fr/4.8.0.v20180815020001/org.eclipse.datatools.enablement.sybase.asa.schemaobjecteditor.examples.nl_fr-4.8.0.v20180815020001.jar
..
{code}
What I can find is that the file "target/test-classes/javac.sh" is
{noformat}
/bin/sh -c cd "/var/lib/jenkins/jobs/myproject_trunk svn/workspace/buildhelper" && /var/lib/jenkins/tools/hudson.model.JDK/java8/jre/../bin/javac @/var/lib/jenkins/jobs/myproject_trunk svn/workspace/buildhelper/target/test-classes/org.codehaus.plexus.compiler.javac.JavacCompiler2330981713839822451arguments -J-Xmx712m -J-Xms328m{noformat}
The size of the file "org.codehaus.plexus.compiler.javac.JavacCompiler2330981713839822451arguments" in the same folder is 274.463 Bytes.
Alone the one line with the classpath is 270175 characters long.
It is what is following the "-classpath" parameter. So I guess in fact the classpath is used as parameter of the commandline after translation by the /bin/sh process and there you run against a limit in the operating system because of the long classpath.
The test classpath is of course a little bit longer than the compiling classpath and the module in question is the one with the longest classpath of all modules as it aggregates the other modules for integration tests. So I guess I cannot do anything else than trying to reduce the length of the classpath. I will try to do it now.
Best regards,
Carsten
> failure to run tests if classpath gets too long (?)
> ---------------------------------------------------
>
> Key: SUREFIRE-1622
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1622
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin
> Affects Versions: 2.22.1
> Environment: debian linux
> Linux jenkins 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 (2016-01-17) x86_64 GNU/Linux
> Jenkins 2.157
> Reporter: Carsten Hammer
> Priority: Major
>
> We have an aggregating plugin where the classpath of a lot of different projects are combined for integration tests. We now have problems since one point in time with failing tests:
> {noformat}
> Please refer to /var/lib/jenkins/jobs/myproject_trunk svn/workspace/buildhelper/target/surefire-reports for the individual test results.
> Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
> The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
> Command was /bin/sh -c cd "/var/lib/jenkins/jobs/myproject_trunk svn/workspace/buildhelper" && /var/lib/jenkins/tools/hudson.model.JDK/java8/jre/bin/java -Dfile.encoding=UTF-8 org.apache.maven.surefire.booter.ForkedBooter '/var/lib/jenkins/jobs/myproject_trunk svn/workspace/buildhelper/target/surefire' 2019-01-08T17-09-39_970-jvmRun1 surefire76194600426378498tmp surefire_18239836832627501299tmp
> Error while executing forked tests.Error while executing process.Cannot run program "/bin/sh" (in directory "/var/lib/jenkins/jobs/myproject_trunk svn/workspace/buildhelper"): error=7, Die Argumentliste ist zu langorg.apache.maven.surefire.shade.org.apache.maven.shared.utils.cli.CommandLineException: Error while executing process.
> at org.apache.maven.surefire.shade.org.apache.maven.shared.utils.cli.Commandline.execute(Commandline.java:412)
> at org.apache.maven.plugin.surefire.booterclient.lazytestprovider.OutputStreamFlushableCommandline.execute(OutputStreamFlushableCommandline.java:65)
> at org.apache.maven.surefire.shade.org.apache.maven.shared.utils.cli.CommandLineUtils.executeCommandLineAsCallable(CommandLineUtils.java:229)
> at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:609)
> at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:282)
> at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:245)
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1183)
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011)
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
> at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
> at org.jvnet.hudson.maven3.launcher.Maven35Launcher.main(Maven35Launcher.java:130)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at jenkins.maven3.agent.Maven35Main.launch(Maven35Main.java:176)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at hudson.maven.Maven3Builder.call(Maven3Builder.java:139)
> at hudson.maven.Maven3Builder.call(Maven3Builder.java:70)
> at hudson.remoting.UserRequest.perform(UserRequest.java:212)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Cannot run program "/bin/sh" (in directory "/var/lib/jenkins/jobs/myproject_trunk svn/workspace/buildhelper"): error=7, Die Argumentliste ist zu lang
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
> at java.lang.Runtime.exec(Runtime.java:620)
> at org.apache.maven.surefire.shade.org.apache.maven.shared.utils.cli.Commandline.execute(Commandline.java:407)
> ... 41 more
> Caused by: java.io.IOException: error=7, Die Argumentliste ist zu lang
> at java.lang.UNIXProcess.forkAndExec(Native Method)
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
> at java.lang.ProcessImpl.start(ProcessImpl.java:134)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
> ... 43 more{noformat}
> I suspect that the internal implementation of maven surefire does not allow to work with a really long classpath.
> I tried the suggestion with the ulimit command as described here:
> [https://unix.stackexchange.com/questions/45583/argument-list-too-long-how-do-i-deal-with-it-without-changing-my-command]
> However it does not work.
> Is there a way to switch on a more verbose log to learn what is internally the cause of the problem?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)