You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "squakez (via GitHub)" <gi...@apache.org> on 2023/03/10 11:59:22 UTC
[GitHub] [camel-k] squakez opened a new issue, #4118: Camel K could go offline
squakez opened a new issue, #4118:
URL: https://github.com/apache/camel-k/issues/4118
Ref #1721
I've done some experiment to try a Camel K installation entirely offline. I've developed a script that download the entire Camel Catalog set of dependencies, which would allow Camel K to work completely closed from the Internet. It has to be tested, but the outcome is that we could produce a docker image of about 5 GB:
```
apache/camel-k 2.0.0-SNAPSHOT 5ace9561ac83 2 minutes ago 4.56GB
```
This "fat" installation could be delivered beside the "normal" installation. Draft work available in https://github.com/apache/camel-k/commit/ea2974d1f5a26ef35d6113f1f8310d8296fd6b45
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] squakez commented on issue #4118: Camel K could go offline
Posted by "squakez (via GitHub)" <gi...@apache.org>.
squakez commented on issue #4118:
URL: https://github.com/apache/camel-k/issues/4118#issuecomment-1486586192
If you use that work, you just need to `make images`. This command will push the image to the registry where your local docker daemon is connected (ie, you can `docker image ls` right after you made the images).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] github-actions[bot] commented on issue #4118: Camel K could go offline
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4118:
URL: https://github.com/apache/camel-k/issues/4118#issuecomment-1610397564
This issue has been automatically marked as stale due to 90 days of inactivity.
It will be closed if no further activity occurs within 15 days.
If you think that’s incorrect or the issue should never stale, please simply write any comment.
Thanks for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] so-dewy commented on issue #4118: Camel K could go offline
Posted by "so-dewy (via GitHub)" <gi...@apache.org>.
so-dewy commented on issue #4118:
URL: https://github.com/apache/camel-k/issues/4118#issuecomment-1486552520
Hello, at my work I need to install Camel K on cluster without internet and I'm trying to build the image locally to try out the work you've done. What should be done to make the kamel use this image at installation time?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] squakez commented on issue #4118: Camel K could go offline
Posted by "squakez (via GitHub)" <gi...@apache.org>.
squakez commented on issue #4118:
URL: https://github.com/apache/camel-k/issues/4118#issuecomment-1669775422
I'm trying to refine the experiment script mentioned in the issue description, but I'm hitting the Maven issue https://issues.apache.org/jira/browse/MDEP-82 when we try to run the integration:
I'm using a base Camel K runtime project in order to try to go offline, but the maven procedure is not really getting all the dependencies expected:
```
$ mvn org.apache.maven.plugins:maven-dependency-plugin:2.8:go-offline -Dmaven.repo.local=repo
...
$ mvn verify -o -Dmaven.repo.local=repo
[INFO] Scanning for projects...
[INFO]
[INFO] ---------< org.apache.camel.k.integration:camel-k-integration >---------
[INFO] Building camel-k-integration 2.0.0
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:3.3.0:resources (default-resources) @ camel-k-integration ---
[WARNING] The POM for org.codehaus.plexus:plexus-interpolation:jar:1.26 is missing, no dependency information available
[WARNING] The POM for org.apache.maven.shared:maven-filtering:jar:3.3.0 is missing, no dependency information available
[WARNING] The POM for commons-io:commons-io:jar:2.11.0 is missing, no dependency information available
[WARNING] The POM for org.apache.commons:commons-lang3:jar:3.12.0 is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.101 s
[INFO] Finished at: 2023-08-08T16:45:48+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.3.0:resources (default-resources) on project camel-k-integration: Execution default-resources of goal org.apache.maven.plugins:maven-resources-plugin:3.3.0:resources failed: Plugin org.apache.maven.plugins:maven-resources-plugin:3.3.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] squakez commented on issue #4118: Camel K could go offline
Posted by "squakez (via GitHub)" <gi...@apache.org>.
squakez commented on issue #4118:
URL: https://github.com/apache/camel-k/issues/4118#issuecomment-1488587083
Yeah, that procedure was a quick experiment to see how to fill the docker image and how much it would require in term of space. We need to fine tune in order for the operator to have the dependencies in the right directory. It is also advisable to run the build with `mvn -o` in order to make sure it does not go out looking for external dependencies. Your hack looks fine, so, while we make the feature available I guess you can make use of it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] squakez commented on issue #4118: Camel K could go offline
Posted by "squakez (via GitHub)" <gi...@apache.org>.
squakez commented on issue #4118:
URL: https://github.com/apache/camel-k/issues/4118#issuecomment-1488600500
BTW, feel free to work on the official feature if you want to. We're happy to receive any contribution!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] squakez closed issue #4118: Camel K could go offline
Posted by "squakez (via GitHub)" <gi...@apache.org>.
squakez closed issue #4118: Camel K could go offline
URL: https://github.com/apache/camel-k/issues/4118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] so-dewy commented on issue #4118: Camel K could go offline
Posted by "so-dewy (via GitHub)" <gi...@apache.org>.
so-dewy commented on issue #4118:
URL: https://github.com/apache/camel-k/issues/4118#issuecomment-1488571832
Thanks for pointing to the `make images`, I was able to build the image locally with pre-downloaded dependencies. And install it to the Minikube by loading the image inside:
```
minikube image load docker.io/apache/camel-k:2.0.0-SNAPSHOT
```
But I wasn't able to create an integration because during it's creation it couldn't resolve parent POM `org.apache:apache:pom:29` for `org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT`
```
[ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT: Could not transfer artifact org.apache:apache:pom:29 from/to central (https://repo.maven.apache.org/maven2): transfer failed for https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom @ org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT, /tmp/artifacts/m2/org/apache/camel/k/camel-k-runtime-bom/1.17.1-SNAPSHOT/camel-k-runtime-bom-1.17.1-SNAPSHOT.pom, line 22, column 13
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-direct:jar is missing. @ line 23, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-http:jar is missing. @ line 27, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-jackson:jar is missing. @ line 31, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-json-validator:jar is missing. @ line 35, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-rabbitmq:jar is missing. @ line 39, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.k:camel-k-runtime:jar is missing. @ line 43, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-java-joor-dsl:jar is missing. @ line 47, column 17
```
I took a look around the pre-downloaded dependencies and noticed that there were `org.apache:apache:pom` lying around but just up to 28, so I changed 29 into 28 inside the dowloaded `org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT.pom` manually. It made some progress but still was trying to download more missing dependencies, there was quite a lot of them and it was too tedious to figure them out one by one, although I think eventually I would be able to make it work.
```
[ERROR] Some problems were encountered while processing the POMs:
[ERROR] Non-resolvable import POM: Could not transfer artifact io.quarkus.platform:quarkus-camel-bom:pom:2.16.0.Final from/to central (https://repo.maven.apache.org/maven2): transfer failed for https://repo.maven.apache.org/maven2/io/quarkus/platform/quarkus-camel-bom/2.16.0.Final/quarkus-camel-bom-2.16.0.Final.pom @ org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT, /tmp/artifacts/m2/org/apache/camel/k/camel-k-runtime-bom/1.17.1-SNAPSHOT/camel-k-runtime-bom-1.17.1-SNAPSHOT.pom, line 100, column 25
[ERROR] Non-resolvable import POM: Could not transfer artifact io.quarkus.platform:quarkus-bom:pom:2.16.0.Final from/to central (https://repo.maven.apache.org/maven2): transfer failed for https://repo.maven.apache.org/maven2/io/quarkus/platform/quarkus-bom/2.16.0.Final/quarkus-bom-2.16.0.Final.pom @ org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT, /tmp/artifacts/m2/org/apache/camel/k/camel-k-runtime-bom/1.17.1-SNAPSHOT/camel-k-runtime-bom-1.17.1-SNAPSHOT.pom, line 107, column 25
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-direct:jar is missing. @ line 23, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-http:jar is missing. @ line 27, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-jackson:jar is missing. @ line 31, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-json-validator:jar is missing. @ line 35, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-rabbitmq:jar is missing. @ line 39, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.k:camel-k-runtime:jar is missing. @ line 43, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-java-joor-dsl:jar is missing. @ line 47, column 17
```
So what I decided to do is use a messy hack:
1. Use cluster with internet as staging ground for developing and testing integrations using the stock releases of camel-k, not my locally modified one.
2. Inside the camel-k-operator it stores all the dependencies needed for creating integrations in `/tmp/artifacts`.
3. Copy the whole directory from inside the pod to the host
```
kubectl cp default/CAMEL_K_OPERATOR_POD_NAME:/tmp/artifacts /your/host/path
```
4. Install the camel-k-operator inside the air-gapped cluster without internet, you need to point to your docker registry that is installed inside your cluster (I'm using nexus docker registry) and put `docker.io/apache/camel-k:RELEASE_VERSION` image inside it so that kamel can resolve it without internet. Also you need to put 'adoptopenjdk/openjdk11:slim' there too as it is used for creating integrations
```
kamel install --registry YOUR_REGISTRY:5000 --registry-auth-username USERNAME --registry-auth-password
PASSWORD --registry-insecure true --base-image YOUR_REGISTRY:5000/adoptopenjdk/openjdk11:slim
```
5. Move the directory to the air-gapped cluster and copy it inside the installed camel-k-operator pod at this location `/tmp/artifcats`.
6. After all the steps I was able to create integrations without internet!
As I understand `/tmp/artifacts` is a default location for the local maven repository if you don't provide one when doing `kamel install`. If you do provide it I think you could do the same thing by copying from staging maven repository to air-gapped one.
This feels hacky but at least I was able to make it work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org