You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@karaf.apache.org by Steinar Bang <sb...@dod.no> on 2021/02/19 21:36:04 UTC
"Unknown protocol: wrap" when running in docker image
I have created a docker image that used to work
https://hub.docker.com/repository/docker/steinarb/oldalbum
It's built on top of the 4.2.9 karaf official docker image by adding
mvn:no.priv.bang.oldalbum/karaf/LATEST/xml/features
to featuresRepositories, and adding
jersey-karaf-feature, \
oldalbum-with-memory-db-with-url-init
to featuresToBoot
Doing
feature:repo-add mvn:no.priv.bang.oldalbum/karaf/LATEST/xml/features
feature:install jersey-karaf-feature
feature:install oldalbum-with-memory-db-with-url-init
works perfectly (oldalbum is deployed to OSSRH)
However, the docker image fails with the following stack trace:
20:43:01.429 ERROR [activator-1-thread-2] Error installing boot features
org.apache.karaf.features.internal.util.MultiException: Error:
Error downloading wrap:file:/root/.m2/repository/org/owasp/encoder/encoder/1.2.2/encoder-1.2.2.jar
Error downloading wrap:file:/root/.m2/repository/org/owasp/encoder/encoder/1.2.2/encoder-1.2.2.jar
at org.apache.karaf.features.internal.download.impl.MavenDownloadManager$MavenDownloader.<init>(MavenDownloadManager.java:91)
at org.apache.karaf.features.internal.download.impl.MavenDownloadManager.createDownloader(MavenDownloadManager.java:72)
at org.apache.karaf.features.internal.region.Subsystem.downloadBundles(Subsystem.java:457)
at org.apache.karaf.features.internal.region.Subsystem.downloadBundles(Subsystem.java:452)
at org.apache.karaf.features.internal.region.SubsystemResolver.resolve(SubsystemResolver.java:224)
at org.apache.karaf.features.internal.service.Deployer.deploy(Deployer.java:393)
at org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvision(FeaturesServiceImpl.java:1062)
at org.apache.karaf.features.internal.service.FeaturesServiceImpl.lambda$doProvisionInThread$13(FeaturesServiceImpl.java:998)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Suppressed: java.io.IOException: Error downloading wrap:file:/root/.m2/repository/org/owasp/encoder/encoder/1.2.2/encoder-1.2.2.jar
at org.apache.karaf.features.internal.download.impl.AbstractRetryableDownloadTask.run(AbstractRetryableDownloadTask.java:77)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
... 3 more
Caused by: java.io.IOException: Could not download [wrap:file:/root/.m2/repository/org/owasp/encoder/encoder/1.2.2/encoder-1.2.2.jar]
at org.apache.karaf.features.internal.download.impl.SimpleDownloadTask.download(SimpleDownloadTask.java:92)
at org.apache.karaf.features.internal.download.impl.AbstractRetryableDownloadTask.run(AbstractRetryableDownloadTask.java:60)
... 6 more
Caused by: java.net.MalformedURLException: Unknown protocol: wrap
at java.base/java.net.URL.<init>(Unknown Source)
at java.base/java.net.URL.<init>(Unknown Source)
at java.base/java.net.URL.<init>(Unknown Source)
at org.apache.karaf.features.internal.download.impl.SimpleDownloadTask.download(SimpleDownloadTask.java:63)
... 7 more
Caused by: java.lang.IllegalStateException: Unknown protocol: wrap
at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:373)
... 11 more
[CIRCULAR REFERENCE:java.io.IOException: Error downloading wrap:file:/root/.m2/repository/org/owasp/encoder/encoder/1.2.2/encoder-1.2.2.jar]
Does anyone have an idea what's happending here?
I assume the last part, "CIRCULAR REFERENCE", points to something wrong
with my features...?
But the trick is how to find it?
Is there a features debugging tools somewhere?
Thanks!
Re: "Unknown protocol: wrap" when running in docker image
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Jean-Baptiste Onofre <jb...@nanthrax.net>:
> The parenthesis are there especially for the stage.
> If you have:
> featuresBoot=(A,B,C),D
> D will be installed only once A, B, and C will be installed.
Thanks! I've done this, first in oldalbum, and then later in other
troublesome docker images and now they all start without issues.
(I've had an occasional "missing injectionfactory" error on startup but
once I've included jersey-karaf-feature in the set inside the
parantheses, the problem has completely gone away)
Re: "Unknown protocol: wrap" when running in docker image
Posted by Jean-Baptiste Onofre <jb...@nanthrax.net>.
The parenthesis are there especially for the stage.
If you have:
featuresBoot=(A,B,C),D
D will be installed only once A, B, and C will be installed.
Regards
JB
> Le 21 févr. 2021 à 18:42, Steinar Bang <sb...@dod.no> a écrit :
>
>>>>>> Jean-Baptiste Onofre <jb...@nanthrax.net>:
>
>> Hi,
>> That’s my question about stage: in feature boot, you can define feature started before other.
>
>> For instance:
>
>> featuresBoot=(…, wrap), myfeature
>
> Are the parantheses significant? Or are they just there as an
> illustration?
>
> Is the order of the features in featuresBoot significant?
>
> The featuresBoot value is the standard value, with the , so the wrap
> feature *is* there (but not with parantheses):
>
> featuresBoot = \
> instance/4.2.9, \
> package/4.2.9, \
> log/4.2.9, \
> ssh/4.2.9, \
> framework/4.2.9, \
> system/4.2.9, \
> eventadmin/4.2.9, \
> feature/4.2.9, \
> shell/4.2.9, \
> management/4.2.9, \
> service/4.2.9, \
> jaas/4.2.9, \
> deployer/4.2.9, \
> diagnostic/4.2.9, \
> wrap/2.6.2, \
> bundle/4.2.9, \
> config/4.2.9, \
> kar/4.2.9, \
> jersey-karaf-feature, \
> oldalbum-with-memory-db-with-url-init
>
Re: "Unknown protocol: wrap" when running in docker image
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Jean-Baptiste Onofre <jb...@nanthrax.net>:
> Hi,
> That’s my question about stage: in feature boot, you can define feature started before other.
> For instance:
> featuresBoot=(…, wrap), myfeature
Are the parantheses significant? Or are they just there as an
illustration?
Is the order of the features in featuresBoot significant?
The featuresBoot value is the standard value, with the , so the wrap
feature *is* there (but not with parantheses):
featuresBoot = \
instance/4.2.9, \
package/4.2.9, \
log/4.2.9, \
ssh/4.2.9, \
framework/4.2.9, \
system/4.2.9, \
eventadmin/4.2.9, \
feature/4.2.9, \
shell/4.2.9, \
management/4.2.9, \
service/4.2.9, \
jaas/4.2.9, \
deployer/4.2.9, \
diagnostic/4.2.9, \
wrap/2.6.2, \
bundle/4.2.9, \
config/4.2.9, \
kar/4.2.9, \
jersey-karaf-feature, \
oldalbum-with-memory-db-with-url-init
Re: "Unknown protocol: wrap" when running in docker image
Posted by Jean-Baptiste Onofre <jb...@nanthrax.net>.
Hi,
That’s my question about stage: in feature boot, you can define feature started before other.
For instance:
featuresBoot=(…, wrap), myfeature
Regards
JB
> Le 21 févr. 2021 à 15:04, Steinar Bang <sb...@dod.no> a écrit :
>
>>>>>> Steinar Bang <sb...@dod.no>:
>>>>>> Steinar Bang <sb...@dod.no>:
>
>>> But I had a thought waking up today:
>
>>> The reason it works when running locally is probably that the
>>> troublesome jar file in the stack trace is already in my
>>> $HOME/.m2/repository/ before I and the reason it fails in docker, is
>>> probably that it starts with an empty $HOME/.m2/repository/
>
> I have a new hypothesis now:
>
> wrap is loaded as a part of featuresBoot and that means in a regular
> karaf that wrap will be loaded when one starts installing.
>
> However, in docker the application is added to featuresBoot, and this
> means that wrap may not be loaded when the application loads.
>
> The feature that loads the failing non-bundle jar is shiro-web and the
> shiro-web feature doesn't contain
> <feature prerequisite='true'>wrap</feature>
>
> @JB Question 1: would it have helped if shiro-web required wrap?
>
> @JB Question 2: could it be an idea to have an "applicationBoot" set of
> features that are loaded after featuresBoot have completed?
>
Re: "Unknown protocol: wrap" when running in docker image
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
>>>>> Steinar Bang <sb...@dod.no>:
>> But I had a thought waking up today:
>> The reason it works when running locally is probably that the
>> troublesome jar file in the stack trace is already in my
>> $HOME/.m2/repository/ before I and the reason it fails in docker, is
>> probably that it starts with an empty $HOME/.m2/repository/
I have a new hypothesis now:
wrap is loaded as a part of featuresBoot and that means in a regular
karaf that wrap will be loaded when one starts installing.
However, in docker the application is added to featuresBoot, and this
means that wrap may not be loaded when the application loads.
The feature that loads the failing non-bundle jar is shiro-web and the
shiro-web feature doesn't contain
<feature prerequisite='true'>wrap</feature>
@JB Question 1: would it have helped if shiro-web required wrap?
@JB Question 2: could it be an idea to have an "applicationBoot" set of
features that are loaded after featuresBoot have completed?
Re: "Unknown protocol: wrap" when running in docker image
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
> But I had a thought waking up today:
> The reason it works when running locally is probably that the
> troublesome jar file in the stack trace is already in my
> $HOME/.m2/repository/ before I and the reason it fails in docker, is
> probably that it starts with an empty $HOME/.m2/repository/
> So somewhere in the dependency graph there are two versions of a feature
> that at one point in time pulled in the troublesome jar and no longer
> does (in the newest version) and that this is the loop hinted to in the
> stack trace...?
I found this article for how to examine the file system of a docker
image:
https://www.baeldung.com/ops/docker-container-filesystem
I wanted to examine /root/.m2/repository/ which meant I had to use the
approach to start a shell in a running container:
sb@lorenzo:~$ docker run -p 8101:8101 -p 8181:8181 -e "DATABASE_CONTENT_URL=https://gist.githubusercontent.com/steinarb/8a1de4e37f82d4d5eeb97778b0c8d459/raw/6cddf18f12e98d704e85af6264d81867f68a097c/dumproutes.sql" -d steinarb/oldalbum:latest
8c9cf448ba6e84a087964288b5ba1fd698322c61a2af9d9f94f9d78b23ed3d03
sb@lorenzo:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8c9cf448ba6e steinarb/oldalbum:latest "karaf run" 8 seconds ago Up 6 seconds 0.0.0.0:8101->8101/tcp, 1099/tcp, 44444/tcp, 0.0.0.0:8181->8181/tcp elegant_murdock
sb@lorenzo:~$ docker exec -it 8c9cf448ba6e /bin/bash
root@8c9cf448ba6e:/# ls -al /root/
total 20
drwx------ 1 root root 4096 Feb 20 08:52 .
drwxr-xr-x 1 root root 4096 Feb 20 08:52 ..
-rw-r--r-- 1 root root 3106 Apr 9 2018 .bashrc
drwxr-xr-x 3 root root 4096 Feb 20 08:52 .m2
-rw-r--r-- 1 root root 148 Aug 17 2015 .profile
root@8c9cf448ba6e:/# cd /root/.m2/repository
root@8c9cf448ba6e:~/.m2/repository# grep -r encoder
org/apache/shiro/shiro-features/1.7.0/shiro-features-1.7.0-features.xml: <bundle>wrap:mvn:org.owasp.encoder/encoder/1.2.2</bundle>
Binary file org/owasp/encoder/encoder/1.2.2/encoder-1.2.2.jar matches
root@8c9cf448ba6e:~/.m2/repository#
So, the only reference to the encoder jar, is from shiro. Could it be
that I'm pulling in multiple versions of shiro?
root@8c9cf448ba6e:~/.m2/repository# grep -r "shiro-features"
no/priv/bang/oldalbum/oldalbum.web.security/1.6.4/oldalbum.web.security-1.6.4-features.xml: <repository>mvn:org.apache.shiro/shiro-features/1.7.0/xml/features</repository>
no/priv/bang/oldalbum/oldalbum.web.api/1.6.4/oldalbum.web.api-1.6.4-features.xml: <repository>mvn:org.apache.shiro/shiro-features/1.7.0/xml/features</repository>
no/priv/bang/authservice/authservice.web.security.memorysession/1.12.1/authservice.web.security.memorysession-1.12.1-features.xml: <repository>mvn:org.apache.shiro/shiro-features/1.7.0/xml/features</repository>
no/priv/bang/authservice/authservice.users/1.12.1/authservice.users-1.12.1-features.xml: <repository>mvn:org.apache.shiro/shiro-features/1.7.0/xml/features</repository>
no/priv/bang/authservice/authservice.web.security/1.12.1/authservice.web.security-1.12.1-features.xml: <repository>mvn:org.apache.shiro/shiro-features/1.7.0/xml/features</repository>
no/priv/bang/authservice/authservice.web.security.dbrealm/1.12.1/authservice.web.security.dbrealm-1.12.1-features.xml: <repository>mvn:org.apache.shiro/shiro-features/1.7.0/xml/features</repository>
root@8c9cf448ba6e:~/.m2/repository#
No, it looks like I have 1.7.0 for all of them... hm...
Re: "Unknown protocol: wrap" when running in docker image
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Jean-Baptiste Onofre <jb...@nanthrax.net>:
> Hi,
> Is it a static or dynamic distribution ?
I'm provisioning the application from maven and I'm provisioning from
version LATEST on the top feature.
Unsure of the terms here, but I guess that makes it dynamic...?
> If dynamic, do you have stage in the boot features (to have
> pax-url-wrap before your features that use wrap protocol) ?
I don't know.
But I had a thought waking up today:
The reason it works when running locally is probably that the
troublesome jar file in the stack trace is already in my
$HOME/.m2/repository/ before I and the reason it fails in docker, is
probably that it starts with an empty $HOME/.m2/repository/
So somewhere in the dependency graph there are two versions of a feature
that at one point in time pulled in the troublesome jar and no longer
does (in the newest version) and that this is the loop hinted to in the
stack trace...?
> I will take a look on your features XML.
Thanks!
Let me see:
Here is the Dockerfile and the files it copies into the karaf official
docker image:
https://github.com/steinarb/oldalbum/tree/master/docker/docker
Here is the oldalbum top level (handwritten) karaf feature repository:
Source:
https://github.com/steinarb/oldalbum/blob/master/karaf/src/main/filtered-resources/feature.xml
Latest on maven central:
https://mvnrepository.com/artifact/no.priv.bang.oldalbum/karaf/1.6.4
I have moved many of my handwritten karaf repositories from being
attached to the top pom and into a sub-module of the reactor build.
The reason for moving feature repositories to a sub-module, was that
I've reintroduced karaf pax exam integration tests that load and assert
the features (since they are now so simple to write...:-) ), and
attaching the hadwritten feature to the top pom introduced a dependency
loop: "mvn clean install" would run locally if there already was a
SNAPSHOT of the feature with the original URL, but "mvn release:perform"
would fail
But anyway: I may still have feature dependencies referring to other
feature repositories with their old maven URLs and picking up an older
version.
So finding any leftover old URLs and cleaning them up seems like the way
to go...?
Re: "Unknown protocol: wrap" when running in docker image
Posted by Jean-Baptiste Onofre <jb...@nanthrax.net>.
Hi,
Is it a static or dynamic distribution ?
If dynamic, do you have stage in the boot features (to have pax-url-wrap before your features that use wrap protocol) ?
I will take a look on your features XML.
Regards
JB
> Le 19 févr. 2021 à 22:36, Steinar Bang <sb...@dod.no> a écrit :
>
> I have created a docker image that used to work
> https://hub.docker.com/repository/docker/steinarb/oldalbum
>
> It's built on top of the 4.2.9 karaf official docker image by adding
> mvn:no.priv.bang.oldalbum/karaf/LATEST/xml/features
> to featuresRepositories, and adding
> jersey-karaf-feature, \
> oldalbum-with-memory-db-with-url-init
> to featuresToBoot
>
> Doing
> feature:repo-add mvn:no.priv.bang.oldalbum/karaf/LATEST/xml/features
> feature:install jersey-karaf-feature
> feature:install oldalbum-with-memory-db-with-url-init
> works perfectly (oldalbum is deployed to OSSRH)
>
> However, the docker image fails with the following stack trace:
>
> 20:43:01.429 ERROR [activator-1-thread-2] Error installing boot features
> org.apache.karaf.features.internal.util.MultiException: Error:
> Error downloading wrap:file:/root/.m2/repository/org/owasp/encoder/encoder/1.2.2/encoder-1.2.2.jar
> Error downloading wrap:file:/root/.m2/repository/org/owasp/encoder/encoder/1.2.2/encoder-1.2.2.jar
> at org.apache.karaf.features.internal.download.impl.MavenDownloadManager$MavenDownloader.<init>(MavenDownloadManager.java:91)
> at org.apache.karaf.features.internal.download.impl.MavenDownloadManager.createDownloader(MavenDownloadManager.java:72)
> at org.apache.karaf.features.internal.region.Subsystem.downloadBundles(Subsystem.java:457)
> at org.apache.karaf.features.internal.region.Subsystem.downloadBundles(Subsystem.java:452)
> at org.apache.karaf.features.internal.region.SubsystemResolver.resolve(SubsystemResolver.java:224)
> at org.apache.karaf.features.internal.service.Deployer.deploy(Deployer.java:393)
> at org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvision(FeaturesServiceImpl.java:1062)
> at org.apache.karaf.features.internal.service.FeaturesServiceImpl.lambda$doProvisionInThread$13(FeaturesServiceImpl.java:998)
> at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.base/java.lang.Thread.run(Unknown Source)
> Suppressed: java.io.IOException: Error downloading wrap:file:/root/.m2/repository/org/owasp/encoder/encoder/1.2.2/encoder-1.2.2.jar
> at org.apache.karaf.features.internal.download.impl.AbstractRetryableDownloadTask.run(AbstractRetryableDownloadTask.java:77)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
> at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
> ... 3 more
> Caused by: java.io.IOException: Could not download [wrap:file:/root/.m2/repository/org/owasp/encoder/encoder/1.2.2/encoder-1.2.2.jar]
> at org.apache.karaf.features.internal.download.impl.SimpleDownloadTask.download(SimpleDownloadTask.java:92)
> at org.apache.karaf.features.internal.download.impl.AbstractRetryableDownloadTask.run(AbstractRetryableDownloadTask.java:60)
> ... 6 more
> Caused by: java.net.MalformedURLException: Unknown protocol: wrap
> at java.base/java.net.URL.<init>(Unknown Source)
> at java.base/java.net.URL.<init>(Unknown Source)
> at java.base/java.net.URL.<init>(Unknown Source)
> at org.apache.karaf.features.internal.download.impl.SimpleDownloadTask.download(SimpleDownloadTask.java:63)
> ... 7 more
> Caused by: java.lang.IllegalStateException: Unknown protocol: wrap
> at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:373)
> ... 11 more
> [CIRCULAR REFERENCE:java.io.IOException: Error downloading wrap:file:/root/.m2/repository/org/owasp/encoder/encoder/1.2.2/encoder-1.2.2.jar]
>
>
> Does anyone have an idea what's happending here?
>
> I assume the last part, "CIRCULAR REFERENCE", points to something wrong
> with my features...?
>
> But the trick is how to find it?
>
> Is there a features debugging tools somewhere?
>
> Thanks!
>