You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "orpiske (via GitHub)" <gi...@apache.org> on 2024/11/08 13:31:01 UTC

[PR] (chores) camel-jbang: disable resource intensive tests [camel]

orpiske opened a new pull request, #16200:
URL: https://github.com/apache/camel/pull/16200

   - Prevent resource-intensive tests from running on CI envs
   - Ensure they are only executed if required resources are in place


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] (chores) camel-jbang: disable resource intensive tests [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on PR #16200:
URL: https://github.com/apache/camel/pull/16200#issuecomment-2464903688

   For now I'm going to merge the PR as is to prevent these tests from continuing to abuse ASF CI. In the future, however, it would be good to ensure the transient files are created in the `target` directory.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] (chores) camel-jbang: disable resource intensive tests [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on PR #16200:
URL: https://github.com/apache/camel/pull/16200#issuecomment-2464780369

   @christophd @tdiesler @gansheer @claudio4j Folks, just as a heads up here: I am disabling this for the reasons stated above. 
   
   Tests like these, that download a lot of additional data put an even greater strain on ASF resources, as such, we don't run them there by default. Additionally, I adjusted it to ensure it checks for docker presence before running (to prevent them from even downloading data if docker is not available). 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] (chores) camel-jbang: disable resource intensive tests [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on PR #16200:
URL: https://github.com/apache/camel/pull/16200#issuecomment-2464874830

   @christophd I found a few more problems. 
   
   Considering this as the local path: `$HOME/code/java/camel/dsl/camel-jbang/camel-jbang-plugin-kubernetes`. 
   
   After running the test, I can see it creates two extra hidden directories in the component directory: `.camel-jbang` and `.camel-jbang-run`. 
   
   For instance:
   
   ```
   [adjust-resource-intensive-tests][~/code/java/camel/dsl/camel-jbang/camel-jbang-plugin-kubernetes]$ ls -la
   total 16
   drwxr-xr-x   7 opiske  staff   224 Nov  8 15:09 ./
   drwxr-xr-x@ 14 opiske  staff   448 Nov  8 09:06 ../
   drwxr-xr-x@  3 opiske  staff    96 Nov  8 15:09 .camel-jbang/
   drwxr-xr-x@  4 opiske  staff   128 Nov  8 15:09 .camel-jbang-run/
   -rw-r--r--@  1 opiske  staff  4613 Nov  8 14:30 pom.xml
   drwxr-xr-x@  5 opiske  staff   160 Oct  1 15:15 src/
   drwxr-xr-x@ 15 opiske  staff   480 Nov  8 14:30 target/
   ```
   
   Within these 2 directories, we can find multiple references to [`JIB`](https://quarkus.io/guides/container-image#jib), which is what is used by Quarkus to build containers: 
   
   ```
   ./.camel-jbang-run/route/pom.xml:24:        <quarkus.container-image.builder>jib</quarkus.container-image.builder>
   ./.camel-jbang-run/route/pom.xml:62:            <artifactId>quarkus-container-image-jib</artifactId>
   ./.camel-jbang-run/1731074985270/pom.xml:58:            <artifactId>quarkus-container-image-jib</artifactId>
   ./.camel-jbang/camel-jbang-run.properties:15:camel.jbang.dependencies=io.quarkus\:quarkus-kubernetes,io.quarkus\:quarkus-container-image-jib
   ./.camel-jbang/camel-jbang-run.properties:17:dependency=mvn\:io.quarkus\:quarkus-container-image-jib
   ./src/main/java/org/apache/camel/dsl/jbang/core/commands/kubernetes/traits/model/Builder.java:113:    @JsonPropertyDescription("A list of tasks sorted by the order of execution in a csv format, ie, `<taskName1>,<taskName2>,...`. Mind that you must include also the operator tasks (`builder`, `quarkus-native`, `package`, `jib`, `s2i`) if you need to execute them. Useful only with `pod` strategy.")
   ./src/main/java/org/apache/camel/dsl/jbang/core/commands/kubernetes/KubernetesRun.java:135:                        description = "The image builder used to build the container image (e.g. docker, jib, podman, s2i).")
   ./src/main/java/org/apache/camel/dsl/jbang/core/commands/kubernetes/KubernetesRun.java:582:                args.add("-Dquarkus.jib.platforms=%s".formatted(imagePlatforms));
   ./src/main/java/org/apache/camel/dsl/jbang/core/commands/kubernetes/KubernetesExport.java:108:    @CommandLine.Option(names = { "--image-builder" }, defaultValue = "jib",
   ./src/main/java/org/apache/camel/dsl/jbang/core/commands/kubernetes/KubernetesExport.java:109:                        description = "The image builder used to build the container image (e.g. docker, jib, podman, s2i).")
   ./src/main/java/org/apache/camel/dsl/jbang/core/commands/kubernetes/KubernetesExport.java:296:            if ("docker".equals(imageBuilder) || "jib".equals(imageBuilder)) {
   ./src/main/java/org/apache/camel/dsl/jbang/core/commands/kubernetes/KubernetesExport.java:323:                    imageBuilder = "jib";
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] (chores) camel-jbang: disable resource intensive tests [camel]

Posted by "christophd (via GitHub)" <gi...@apache.org>.
christophd commented on PR #16200:
URL: https://github.com/apache/camel/pull/16200#issuecomment-2464800276

   @orpiske AFAIR these tests do not use Docker but the Kubernetes mock server. They should be working on environments without Docker/Kubernetes being installed, too. The Kubernetes mock server is just a normal Http server that mocks the Kubernetes API.
   
   See https://github.com/fabric8io/kubernetes-client?tab=readme-ov-file#mocking-kubernetes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] (chores) camel-jbang: disable resource intensive tests [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske merged PR #16200:
URL: https://github.com/apache/camel/pull/16200


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] (chores) camel-jbang: disable resource intensive tests [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on PR #16200:
URL: https://github.com/apache/camel/pull/16200#issuecomment-2464828963

   > @orpiske AFAIR these tests do not use Docker but the Kubernetes mock server. They should be working on environments without Docker/Kubernetes being installed, too. The Kubernetes mock server is just a normal Http server that mocks the Kubernetes API.
   > 
   > See https://github.com/fabric8io/kubernetes-client?tab=readme-ov-file#mocking-kubernetes
   
   That's very strange. They seem to be failing for me complaining about my environment with Podman: 
   
   ```
   [WARNING] [io.quarkus.deployment.util.ContainerRuntimeUtil] Command "podman info" exited with error code 125. Rootless container runtime detection might not be reliable or the container service is not running at all.
   [INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] Using podman to run the native image builder
   [WARNING] [io.quarkus.container.image.jib.deployment.JibProcessor] Base image 'registry.access.redhat.com/ubi8/openjdk-17-runtime:1.20' does not use a specific image digest - build may not be reproducible
   [INFO] ------------------------------------------------------------------------
   [INFO] BUILD FAILURE
   [INFO] ------------------------------------------------------------------------
   [INFO] Total time:  4.336 s
   [INFO] Finished at: 2024-11-08T14:55:36+01:00
   [INFO] ------------------------------------------------------------------------
   [ERROR] Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:3.16.1:build (default) on project route: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
   [ERROR] 	[error]: Build step io.quarkus.container.image.jib.deployment.JibProcessor#buildFromJar threw an exception: java.lang.RuntimeException: Unable to create container image
   [ERROR] 	at io.quarkus.container.image.jib.deployment.JibProcessor.containerize(JibProcessor.java:261)
   [ERROR] 	at io.quarkus.container.image.jib.deployment.JibProcessor.buildFromJar(JibProcessor.java:188)
   [ERROR] 	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
   [ERROR] 	at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:856)
   [ERROR] 	at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
   [ERROR] 	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
   [ERROR] 	at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2675)
   [ERROR] 	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2654)
   [ERROR] 	at org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1627)
   [ERROR] 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1594)
   [ERROR] 	at java.base/java.lang.Thread.run(Thread.java:1583)
   [ERROR] 	at org.jboss.threads.JBossThread.run(JBossThread.java:499)
   [ERROR] Caused by: java.util.concurrent.ExecutionException: java.io.IOException: 'docker info' command failed with error: Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM
   [ERROR] Error: unable to connect to Podman socket: failed to connect: dial tcp 127.0.0.1:54124: connect: connection refused
   [ERROR] 
   [ERROR] 	at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:596)
   [ERROR] 	at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:575)
   [ERROR] 	at com.google.common.util.concurrent.FluentFuture$TrustedFuture.get(FluentFuture.java:91)
   [ERROR] 	at com.google.cloud.tools.jib.builder.steps.StepsRunner.run(StepsRunner.java:222)
   [ERROR] 	at com.google.cloud.tools.jib.api.Containerizer.run(Containerizer.java:406)
   [ERROR] 	at com.google.cloud.tools.jib.api.JibContainerBuilder.containerize(JibContainerBuilder.java:610)
   [ERROR] 	at io.quarkus.container.image.jib.deployment.JibProcessor.containerizeUnderLock(JibProcessor.java:332)
   [ERROR] 	at io.quarkus.container.image.jib.deployment.JibProcessor.containerize(JibProcessor.java:254)
   [ERROR] 	... 11 more
   [ERROR] Caused by: java.io.IOException: 'docker info' command failed with error: Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM
   [ERROR] Error: unable to connect to Podman socket: failed to connect: dial tcp 127.0.0.1:54124: connect: connection refused
   [ERROR] 
   [ERROR] 	at com.google.cloud.tools.jib.docker.CliDockerClient.info(CliDockerClient.java:195)
   [ERROR] 	at com.google.cloud.tools.jib.builder.steps.StepsRunner.lambda$loadDocker$18(StepsRunner.java:623)
   [ERROR] 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
   [ERROR] 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:76)
   [ERROR] 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
   [ERROR] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
   [ERROR] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
   [ERROR] 	at java.base/java.lang.Thread.run(Thread.java:1583)
   [ERROR] -> [Help 1]
   [ERROR] 
   [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
   [ERROR] Re-run Maven using the -X switch to enable full debug logging.
   [ERROR] 
   [ERROR] For more information about the errors and possible solutions, please read the following articles:
   [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
   
   org.opentest4j.AssertionFailedError: Main finished with exit code 1:
   Exporting application ...
   Generating fresh run data
   Exporting as Quarkus project to: .camel-jbang-run/route
   Building Kubernetes manifest ...
   Project export successful!
   Building Camel application ...
   Run: .camel-jbang-run/route/mvnw --file .camel-jbang-run/route package
   Build failed!
   Project build failed! ==> 
   Expected :0
   Actual   :1
   <Click to see difference>
   
   
   	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
   	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
   	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
   	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
   	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:563)
   	at org.apache.camel.dsl.jbang.core.commands.kubernetes.KubernetesCommandTest$1.quit(KubernetesCommandTest.java:78)
   	at org.apache.camel.dsl.jbang.core.commands.CamelJBangMain.run(CamelJBangMain.java:175)
   	at org.apache.camel.dsl.jbang.core.commands.kubernetes.KubernetesCommandTest.shouldPrintKubernetesManifest(KubernetesCommandTest.java:49)
   	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
   	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
   	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
   
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] (chores) camel-jbang: disable resource intensive tests [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on PR #16200:
URL: https://github.com/apache/camel/pull/16200#issuecomment-2464834955

   And if I launch Podman on my environment, then it works: 
   
   ```
   [INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] Using podman to run the native image builder
   [WARNING] [io.quarkus.container.image.jib.deployment.JibProcessor] Base image 'registry.access.redhat.com/ubi8/openjdk-17-runtime:1.20' does not use a specific image digest - build may not be reproducible
   [INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] Using base image with digest: sha256:dccda1807142b868566032e4855aa39e1621ac16bb15b2031cc6a1bd11980465
   [INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] Container entrypoint set to [/opt/jboss/container/java/run/run-java.sh]
   [INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] Created container image camel-test/route:1.0-SNAPSHOT (sha256:a8994ace46a8133b23abbc044d4a7ecbaf08db566c3388d008d34d6808f862b7)
   
   [INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 17667ms
   ```
   
   So, maybe not the k8s mock server, but something else may require Docker / Podman for this test to run. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org