You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@karaf.apache.org by Steinar Bang <sb...@dod.no> on 2021/04/03 21:53:45 UTC
Problems making debian package of karaf 4.3.1
I'm trying to make a debian package of karaf 4.3.1[1].
The resulting deb package so far:
1. Builds without any error messages
2. Installs without any error messages
3. Leaves a running process
4. Leaves very few messages in the logs[6]
5. Does not start an SSH server (I think...? at least nothing responds
on port 8101)
According to the git history it's been two years since I last had any
issues upgrading (this was when moving from 4.1.x to 4.2.x).
Since this, all I've had to do is bump the version number and rebuild
the package.
All hints and ideas to what I could look at when debugging
this issue, are, as always, highly appreciated!
What I've done so far[2] is:
1. Bump the karaf version number
2. Embed OSGi 7 instead of getting it from a debian dependency, since
debian stable still has OSGi 6[3] (using a debian package can be
reintroduced when "bullsey" is released[4]. There is an OSGi 7
package in buster-backports[5], but that will have to be installed
explicitly, it won't automatically be installed as a dependency
Thanks!
- Steinar
References:
[1] <https://github.com/steinarb/karaf-debian>
[2] <https://github.com/steinarb/scratch/tree/karaf-debian/use-karaf-43x>
[3] <https://packages.debian.org/buster/libosgi-core-java>
[4] <https://packages.debian.org/bullseye/libosgi-core-java>
[5] <https://packages.debian.org/buster-backports/libosgi-core-java>
[6] <https://gist.github.com/steinarb/36b8478c88427a1ebaa740db1fd58456>
Re: Problems making debian package of karaf 4.3.1
Posted by Jean-Baptiste Onofre <jb...@nanthrax.net>.
I created https://issues.apache.org/jira/browse/KARAF-7094 <https://issues.apache.org/jira/browse/KARAF-7094> related to that.
Regards
JB
> Le 5 avr. 2021 à 23:55, Steinar Bang <sb...@dod.no> a écrit :
>
>>>>>> JB Onofré <jb...@nanthrax.net>:
>
>> Yea it makes sense to use an empty file for 4.3. Actually, I think it makes sense to do it by default on 4.3. I will create the jira related to that.
>
> +1
>
Re: Problems making debian package of karaf 4.3.1
Posted by Steinar Bang <sb...@dod.no>.
>>>>> JB Onofré <jb...@nanthrax.net>:
> Yea it makes sense to use an empty file for 4.3. Actually, I think it makes sense to do it by default on 4.3. I will create the jira related to that.
+1
Re: Problems making debian package of karaf 4.3.1
Posted by JB Onofré <jb...@nanthrax.net>.
Yea it makes sense to use an empty file for 4.3. Actually, I think it makes sense to do it by default on 4.3. I will create the jira related to that.
Regards
JB
> Le 5 avr. 2021 à 20:56, Steinar Bang <sb...@dod.no> a écrit :
>
>
>>
>>>>>> JB Onofré <jb...@nanthrax.net>:
>
>> Hi
>> That’s normal: karaf 4.2.x uses blacklist so it provides the file, especially to blacklist 4.3.x features repositories.
>
>> Karaf 4.3.x doesn’t use blacklist (as the version is newer).
>
> Yes, but that collided with the way I had decided to handle the etc
> directory: to preserve as much of users' config modifications across
> upgrades, as possible, the files in the etc directory are left on
> uninstall.
>
> And since nothing replaced the 4.2.11 blacklist file, it was still
> present after upgrade to 4.3.1.
>
> So what the 4.3.1 debian package now does is to include an empty
> blacklist file, and then that will overwrite the 4.2.11 file.
>
> That worked, I have successfully tested upgrading from 4.2.11 to 4.3.1
> with "apt full-upgrade".
>
> But I won't release the 4.3.1 debian package, until I have bumped all of
> my apps to karaf 4.3.1.
>
> Just recompiling after bumping the BoM would have been enough to make
> things run on 4.3.1, but I started on a bigger dependency cleanup and
> upgrade of dependencies, and that will take a little time to complete.
>
Re: Problems making debian package of karaf 4.3.1
Posted by Steinar Bang <sb...@dod.no>.
>>>>> JB Onofré <jb...@nanthrax.net>:
> Hi
> That’s normal: karaf 4.2.x uses blacklist so it provides the file, especially to blacklist 4.3.x features repositories.
> Karaf 4.3.x doesn’t use blacklist (as the version is newer).
Yes, but that collided with the way I had decided to handle the etc
directory: to preserve as much of users' config modifications across
upgrades, as possible, the files in the etc directory are left on
uninstall.
And since nothing replaced the 4.2.11 blacklist file, it was still
present after upgrade to 4.3.1.
So what the 4.3.1 debian package now does is to include an empty
blacklist file, and then that will overwrite the 4.2.11 file.
That worked, I have successfully tested upgrading from 4.2.11 to 4.3.1
with "apt full-upgrade".
But I won't release the 4.3.1 debian package, until I have bumped all of
my apps to karaf 4.3.1.
Just recompiling after bumping the BoM would have been enough to make
things run on 4.3.1, but I started on a bigger dependency cleanup and
upgrade of dependencies, and that will take a little time to complete.
Re: Problems making debian package of karaf 4.3.1
Posted by JB Onofré <jb...@nanthrax.net>.
Hi
That’s normal: karaf 4.2.x uses blacklist so it provides the file, especially to blacklist 4.3.x features repositories.
Karaf 4.3.x doesn’t use blacklist (as the version is newer).
Regards
JB
> Le 5 avr. 2021 à 19:12, Steinar Bang <sb...@dod.no> a écrit :
>
> I did a diff between the directory contents of the etc directory of the
> karaf 4.3.1 binary tarball, and the etc directory of the debian package:
> https://gist.github.com/steinarb/7a8427ac19b355c27598c945b27863f8
>
> One file looked particularily interesting: org.apache.karaf.features.xml
> which was only in the etc directory of the debian package.
>
> And indeed, what the file seems to be doing is blacklist all 4.3.x
> feature repositories:
> https://gist.github.com/steinarb/c5041feca83d74f93973debedecfd5f9
>
> So, what I did was:
> 1. Uninstall karaf ("apt remove karaf")
> 2. remove the file ("rm /etc/karaf/org.apache.karaf.features.xml")
> 3. Reinstall karaf 4.3.1 ("apt install karaf")
>
> And this time the debian-installed karaf booted normally.
>
> So where did org.apache.karaf.features.xml come from?
>
> The most obvious answer was that the file was installed by an earlier
> version of karaf, and not present in karaf 4.3.x, and therefore not
> overwritten or replaced on an "apt full-upgrade".
>
> I uninstalled karaf 4.3.1 and reinstalled karaf 4.2.11 from the debian
> package and then the org.apache.karaf.features.xml file was back.
>
> I think what I will do is to add an org.apache.karaf.features.xml file
> with empty content to the 4.3.1 debian package, so that the 4.2.11 file
> will be replaced, and "apt full-upgrade" will result in a working karaf
> 4.3.1.
>
Re: Problems making debian package of karaf 4.3.1
Posted by Steinar Bang <sb...@dod.no>.
I did a diff between the directory contents of the etc directory of the
karaf 4.3.1 binary tarball, and the etc directory of the debian package:
https://gist.github.com/steinarb/7a8427ac19b355c27598c945b27863f8
One file looked particularily interesting: org.apache.karaf.features.xml
which was only in the etc directory of the debian package.
And indeed, what the file seems to be doing is blacklist all 4.3.x
feature repositories:
https://gist.github.com/steinarb/c5041feca83d74f93973debedecfd5f9
So, what I did was:
1. Uninstall karaf ("apt remove karaf")
2. remove the file ("rm /etc/karaf/org.apache.karaf.features.xml")
3. Reinstall karaf 4.3.1 ("apt install karaf")
And this time the debian-installed karaf booted normally.
So where did org.apache.karaf.features.xml come from?
The most obvious answer was that the file was installed by an earlier
version of karaf, and not present in karaf 4.3.x, and therefore not
overwritten or replaced on an "apt full-upgrade".
I uninstalled karaf 4.3.1 and reinstalled karaf 4.2.11 from the debian
package and then the org.apache.karaf.features.xml file was back.
I think what I will do is to add an org.apache.karaf.features.xml file
with empty content to the 4.3.1 debian package, so that the 4.2.11 file
will be replaced, and "apt full-upgrade" will result in a working karaf
4.3.1.
Re: Problems making debian package of karaf 4.3.1
Posted by Steinar Bang <sb...@dod.no>.
Here's the difference in the startup of 4.3.1 and 4.2.11 in karaf.log.
As you said, JB, it looks like it doesn't find the startup features...?
4.3.1:
2021-04-03T17:25:45,773 | INFO | CM Configuration Updater (Update: pid=org.ops4j.pax.logging) | EventAdminConfigurationNotifier | 6 - org.ops4j.pax.logging.pax-logging-log4j2 - 2.0.8 | Sending Event Admin notification (configuration successful) to org/ops4j/pax/logging/Configuration
2021-04-03T17:25:46,056 | INFO | features-3-thread-1 | FeaturesServiceImpl | 18 - org.apache.karaf.features.core - 4.3.1 | No deployment change.
2021-04-03T17:25:46,059 | INFO | features-3-thread-1 | FeaturesServiceImpl | 18 - org.apache.karaf.features.core - 4.3.1 | Done.
4.2.11:
2021-04-05T17:03:18,867 | INFO | CM Configuration Updater (Update: pid=org.ops4j.pax.logging) | EventAdminConfigurationNotifier | 6 - org.ops4j.pax.logging.pax-logging-log4j2 - 1.11.9 | Sending Event Admin notification (configuration successful) to org/ops4j/pax/logging/Configuration
2021-04-05T17:03:19,307 | INFO | activator-1-thread-2 | FeaturesServiceImpl | 13 - org.apache.karaf.features.core - 4.2.11 | Adding features: system/[4.2.11,4.2.11],feature/[4.2.11,4.2.11],deployer/[4.2.11,4.2.11],diagnostic/[4.2.11,4.2.11],wrap/[2.6.7,2.6.7],management/[4.2.11,4.2.11],bundle/[4.2.11,4.2.11],ssh/[4.2.11,4.2.11],log/[4.2.11,4.2.11],eventadmin/[4.2.11,4.2.11],instance/[4.2.11,4.2.11],config/[4.2.11,4.2.11],framework/[4.2.11,4.2.11],shell/[4.2.11,4.2.11],kar/[4.2.11,4.2.11],service/[4.2.11,4.2.11],jaas/[4.2.11,4.2.11],package/[4.2.11,4.2.11]
2021-04-05T17:03:19,775 | INFO | features-3-thread-1 | FeaturesServiceImpl | 13 - org.apache.karaf.features.core - 4.2.11 | Changes to perform:
2021-04-05T17:03:19,775 | INFO | features-3-thread-1 | FeaturesServiceImpl | 13 - org.apache.karaf.features.core - 4.2.11 | Region: root
... (and the boot continues)
Re: Problems making debian package of karaf 4.3.1
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
> A little more info.
> The debian installed karaf starts in /var/lib/karaf/ which looks like
> this:
Are there any layout changed in the file/directory layout of karaf 4.3.x
that might cause this structure to no longer work?
Re: Problems making debian package of karaf 4.3.1
Posted by Steinar Bang <sb...@dod.no>.
A little more info.
The debian installed karaf starts in /var/lib/karaf/ which looks like
this:
sb@lorenzo:/var/lib/karaf$ ls -al
total 36
drwxr-xr-x 7 karaf karaf 4096 Apr 3 17:25 .
drwxr-xr-x 77 root root 4096 Nov 15 15:36 ..
drwxr-xr-x 2 karaf karaf 4096 Aug 25 2017 .karaf
drwxr-xr-x 3 karaf karaf 4096 Aug 25 2017 .m2
lrwxrwxrwx 1 karaf karaf 20 Apr 19 2016 bin -> /usr/share/karaf/bin
drwxr-xr-x 8 karaf karaf 4096 Apr 3 17:25 data
drwxr-xr-x 2 karaf karaf 4096 Apr 3 17:25 deploy
-rw-r--r-- 1 karaf karaf 1846 Sep 24 2020 derby.log
drwxr-xr-x 2 karaf karaf 4096 Apr 3 17:25 instances
-rw-r--r-- 1 karaf karaf 5 Apr 3 17:25 karaf.pid
lrwxrwxrwx 1 karaf karaf 20 Apr 19 2016 lib -> /usr/share/karaf/lib
-rw-r--r-- 1 karaf karaf 0 Apr 3 17:25 lock
lrwxrwxrwx 1 karaf karaf 23 Apr 19 2016 system -> /usr/share/karaf/system
sb@lorenzo:/var/lib/karaf$
After uninstalling the debian package, the directory looks like this:
sb@lorenzo:/var/lib/karaf$ ls -al
total 24
drwxr-xr-x 5 karaf karaf 4096 Apr 5 16:50 .
drwxr-xr-x 77 root root 4096 Nov 15 15:36 ..
drwxr-xr-x 2 karaf karaf 4096 Aug 25 2017 .karaf
drwxr-xr-x 3 karaf karaf 4096 Aug 25 2017 .m2
drwxr-xr-x 7 karaf karaf 4096 Apr 5 16:50 data
-rw-r--r-- 1 karaf karaf 1846 Sep 24 2020 derby.log
sb@lorenzo:/var/lib/karaf$
The /etc/karaf/ directory (which is symlinked to from
/var/lib/karaf/etc/ in an installed debian package) is also left in
place, so that karaf config will survive across debian versions.
The /var/lib/karaf/data/ directory is deleted when uninstalling and
upgrading packages, because otherwise the resulting reinstall most
probably wouldn't work.
(The downside of deleting the data directory, is that I need to reinstall
all of my apps after doing an "apt full-upgrade" that pulls in a new
version. But I can live with that)
Re: Problems making debian package of karaf 4.3.1
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
>>>>> Jean-Baptiste Onofre <jb...@nanthrax.net>:
>> Hi Steinar,
>> It’s weird: it looks like the boot feature are not startup.
>> Do you have the features core bundle in etc/startup.properties ?
Yes, at the end of /etc/karaf/startup.properties :
https://gist.github.com/steinarb/d69fb3a7e15b31d65ee3eea46ffc7ca6
The jar file is also in
/usr/share/karaf/system/org/apache/karaf/features/org.apache.karaf.features.core/4.3.1:
-rw-r--r-- 1 root root 2611717 Apr 19 2016 org.apache.karaf.features.core-4.3.1.jar
>> Did you update etc/org.apache.karaf.features.cfg ?
No. The /etc/karaf/org.apache.karaf.features.cfg file is identical to
the etc/org.apache.karaf.features.cfg of the karaf 4.3.1 binary tgz.
Re: Problems making debian package of karaf 4.3.1
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Jean-Baptiste Onofre <jb...@nanthrax.net>:
> Hi Steinar,
> It’s weird: it looks like the boot feature are not startup.
> Do you have the features core bundle in etc/startup.properties ?
> Did you update etc/org.apache.karaf.features.cfg ?
Thanks for the pointers! I will check.
It could be that I have modifications in /etc/karaf (which is where the
.deb package puts the karaf etc directory), and that the .deb package
doesn't track those files, detect modifications and ask if they should
be replaced by the package version!
> What JDK did you try ?
OpenJDK 11, as delivered with debian.
Specifically openjdk-11-jdk:amd64 11.0.9.1+1-1~deb10u2
Re: Problems making debian package of karaf 4.3.1
Posted by Jean-Baptiste Onofre <jb...@nanthrax.net>.
Hi Steinar,
It’s weird: it looks like the boot feature are not startup.
Do you have the features core bundle in etc/startup.properties ?
Did you update etc/org.apache.karaf.features.cfg ?
What JDK did you try ?
Regards
JB
> Le 3 avr. 2021 à 23:53, Steinar Bang <sb...@dod.no> a écrit :
>
> I'm trying to make a debian package of karaf 4.3.1[1].
>
> The resulting deb package so far:
> 1. Builds without any error messages
> 2. Installs without any error messages
> 3. Leaves a running process
> 4. Leaves very few messages in the logs[6]
> 5. Does not start an SSH server (I think...? at least nothing responds
> on port 8101)
>
> According to the git history it's been two years since I last had any
> issues upgrading (this was when moving from 4.1.x to 4.2.x).
>
> Since this, all I've had to do is bump the version number and rebuild
> the package.
>
> All hints and ideas to what I could look at when debugging
> this issue, are, as always, highly appreciated!
>
> What I've done so far[2] is:
> 1. Bump the karaf version number
> 2. Embed OSGi 7 instead of getting it from a debian dependency, since
> debian stable still has OSGi 6[3] (using a debian package can be
> reintroduced when "bullsey" is released[4]. There is an OSGi 7
> package in buster-backports[5], but that will have to be installed
> explicitly, it won't automatically be installed as a dependency
>
> Thanks!
>
> - Steinar
>
>
> References:
> [1] <https://github.com/steinarb/karaf-debian>
> [2] <https://github.com/steinarb/scratch/tree/karaf-debian/use-karaf-43x>
> [3] <https://packages.debian.org/buster/libosgi-core-java>
> [4] <https://packages.debian.org/bullseye/libosgi-core-java>
> [5] <https://packages.debian.org/buster-backports/libosgi-core-java>
> [6] <https://gist.github.com/steinarb/36b8478c88427a1ebaa740db1fd58456>
>