You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@karaf.apache.org by Steinar Bang <sb...@dod.no> on 2022/08/03 13:24:46 UTC
Re: Problems with logging to karaf.log from liquibase >3
>>>>> Steinar Bang <sb...@dod.no>:
>>>>> Steven Huypens <st...@gmail.com>:
>> I had a brief look at your repo and I see you are creating an actual
>> Liquibase instance, which we are not, possibly because of certain
>> issues in the past. But I don't think that should make a difference
>> wrt logging.
> No, I don't think that should make a difference either.
Actually, this could make a big difference, because I think how I'm
creating the Liquibase instance and the resource locator I'm using,
could effect the log-finding problems I'm seeing.
I'm curious: if you're not creating an actual Liquibase instance, how do
you use liquibase?
(I'm using Liquibase the only way I was able to figure out it could be
used, back when I started using liquibse in 2017, and if there is a
different and better way, I'm open to that)
Thanks!
Ann: Version 4.15.0 of the liquibase karaf feature is released (Was: Problems with logging to karaf.log from liquibase >3)
Posted by Steinar Bang <sb...@dod.no>.
I'm happy to say that I've made the first release of my liquibase karaf
feature in nearly three years (the previous release was liquibase 3.8.0
in November 2019).
The current version is 4.15.0
https://github.com/steinarb/liquibase-karaf-feature#release-history
I've summarized the things needed to use Liquibase from an OSGi bundle
here:
https://github.com/steinarb/liquibase-karaf-feature#using-liquibase-from-osgi-bundles
(the copying of XSD files is a workaround for https://github.com/liquibase/liquibase/issues/2817 )
I see the error message "Cannot end scope" sporadically in the karaf.log
files of pax exam tests that loads features that uses this feature.
Possibly related to this issue...?
https://github.com/liquibase/liquibase/issues/2248
Regular feature loads in karaf don't get the "Cannot end scope" errors.
Thanks to Steven Huypens for all help! In particular for pointing out
that the spifly bundle must be loaded and not the feature (to get the
correct start-level of the bundle), because that had me stumped for a
long time.
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
> Heh... looking at the karaf.log of the integration test, it seems to actually work:
> https://gist.github.com/steinarb/445699fc4277c36a0f9b94a799da69b1#file-karaf-log-L415
> But the "No common superclass" still happens, prior to the place where
> it starts working
> https://gist.github.com/steinarb/445699fc4277c36a0f9b94a799da69b1#file-karaf-log-L325
> And there are lots of stack traces happening during the shutdown...
> https://gist.github.com/steinarb/445699fc4277c36a0f9b94a799da69b1#file-karaf-log-L438
Soo... I've merged all of the successful changes I've made so far, into
master, and have pushed master.
This is because the way it is now, is probably working for a normal
application.
Still, I will keep at it for a while, seeing if it is possible to lose
the stacktraces.
First thing to try is newer versions of liquibase 4.x.
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
>> What I was trying to tell you, is that by replacing your
>> spifly-feature with the aries-proxy-feature and a single spifly
>> bundle, the classloading issue might be fixed. Liquibase is able to
>> find the Default Log Service, as far as I can see.
> Ahhh! I see!
I eventually figured out the difference (I think...?): you're starting
the spifly bundle on level 40, which is the same level as the
liquibase-feature is on, and using the spifly feature, the spifly bundle
is started on the default level, which is 80, and possibly isn't present
when liquibase is running?
Other than that it seems the same bundles are started and activated in
both cases (i.e. both using the spifly feature and loading the spifly
bundle directly).
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
>>> (I noticed I've added an empty
>>> src\main\resources\META-INF\servicesliquibase.license.LicenseService as
>>> well, maybe that fixes the encryption exception)
>> Ah! Thanks again! I'll try that!
Heh... looking at the karaf.log of the integration test, it seems to actually work:
https://gist.github.com/steinarb/445699fc4277c36a0f9b94a799da69b1#file-karaf-log-L415
But the "No common superclass" still happens, prior to the place where
it starts working
https://gist.github.com/steinarb/445699fc4277c36a0f9b94a799da69b1#file-karaf-log-L325
And there are lots of stack traces happening during the shutdown...
https://gist.github.com/steinarb/445699fc4277c36a0f9b94a799da69b1#file-karaf-log-L438
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
>>>>> Steven Huypens <st...@gmail.com>:
>> (I noticed I've added an empty
>> src\main\resources\META-INF\servicesliquibase.license.LicenseService as
>> well, maybe that fixes the encryption exception)
> Ah! Thanks again! I'll try that!
Didn't help, unfortunately. So I'll keep debugging and see if I can
find out which keyAndAliases value that makes the difference.
But as you pointed out there are some liquibase pro classes in the stack
trace, and these classes are in the liquibase-core jar, but look like
obfuscated code and contains no source code in the source jar.
So debugging below AbstractMapConf.getProvidedValue() is hard.
Also things seems to work as expected when debugging (aka "heisenbug"),
so I suspect there are some race condition(s) involved.
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steven Huypens <st...@gmail.com>:
> Hi Steinar,
> May I suggest you re-open/recreate the issue on the Liquibase bugtracker ?
I'd rather create a new issue suggesting a more OSGi-approach:
1. Define all of the service interfaces in a separate osgi bundle
2. Instead of using ServiceLoader through the mediator service, create
OSGi components exposing the services that then can be injected into
user components
But I fear that this would sound too drastic to the liquibase people, so
that will be rejected out of hand.
In this scenario the parser(s) would be created inside the liquibase
bundle, and would be able to find the XSD files.
(Hm... can DS components be created in an OSGi fragment bundle? Or are
those code only? If so this could be done outside of the liquibase
project)
> Meanwhile you can build your own liquibase-core bundle using a pom.xml like
> below, you simple add your own version of
> src/main/java/liquibase/parser/core/xml/LiquibaseEntityResolver.java to the
> project
Building my own version of liquibase-core would be the last resort.
And I think that if I build my own version of liquibase-core, it would
be to be able to expose the liquibase services as OSGi services, the way
I've outlined above.
> (I noticed I've added an empty
> src\main\resources\META-INF\servicesliquibase.license.LicenseService as
> well, maybe that fixes the encryption exception)
Ah! Thanks again! I'll try that!
(I'm pretty close to get it working now... debugging exactly when the
exception happens is what I was doing now. Logging still isn't quite
right, but that is next)
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steven Huypens <st...@gmail.com>.
Hi Steinar,
May I suggest you re-open/recreate the issue on the Liquibase bugtracker ?
Meanwhile you can build your own liquibase-core bundle using a pom.xml like
below, you simple add your own version of
src/main/java/liquibase/parser/core/xml/LiquibaseEntityResolver.java to the
project
(I noticed I've added an empty
src\main\resources\META-INF\servicesliquibase.license.LicenseService as
well, maybe that fixes the encryption exception)
----------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------------------------------------------------
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>your.domain.org.liquibase</groupId>
<artifactId>liquibase-core</artifactId>
<packaging>bundle</packaging>
<version>4.9.1.1-SNAPSHOT</version>
<!--
A wrapped bundle to contain liquibase-core
-->
<properties>
<liquibase.version>4.9.1</liquibase.version>
</properties>
<dependencies>
<dependency>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-core</artifactId>
<version>${liquibase.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-release-plugin</artifactId>
<version>3.0.0-M5</version>
</plugin>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>5.1.4</version>
<extensions>true</extensions>
<configuration>
<instructions>
<Bundle-Activator>liquibase.osgi.Activator</Bundle-Activator>
<Bundle-Name>${project.name}</Bundle-Name>
<Bundle-SymbolicName>${project.groupId}.${project.artifactId}</Bundle-SymbolicName>
<Import-Package>
javax.activation*;resolution:=optional,
javax.servlet.*;version="[2.6,4)";resolution:=optional,
javax.sql.*,
org.osgi.*,
org.yaml.snakeyaml.*,
*;resolution:=optional
</Import-Package>
<Export-Package>
liquibase.*;version="${liquibase.version}",
</Export-Package>
<Embed-Dependency>liquibase-core;inline=true</Embed-Dependency>
<Include-Resource>
{maven-resources},
{maven-dependencies},
liquibase=target/classes/liquibase
</Include-Resource>
<Provide-Capability>
osgi.serviceloader;
osgi.serviceloader=liquibase.serializer.ChangeLogSerializer,
osgi.serviceloader;
osgi.serviceloader=liquibase.parser.NamespaceDetails,
osgi.serviceloader;
osgi.serviceloader=liquibase.database.Database,
osgi.serviceloader;
osgi.serviceloader=liquibase.change.Change,
osgi.serviceloader;
osgi.serviceloader=liquibase.database.DatabaseConnection,
osgi.serviceloader;
osgi.serviceloader=liquibase.precondition.Precondition,
osgi.serviceloader;
osgi.serviceloader=liquibase.serializer.SnapshotSerializer,
osgi.serviceloader;
osgi.serviceloader=liquibase.configuration.AutoloadedConfigurations,
osgi.serviceloader;
osgi.serviceloader=liquibase.diff.DiffGenerator,
osgi.serviceloader;
osgi.serviceloader=liquibase.lockservice.LockService,
osgi.serviceloader;
osgi.serviceloader=liquibase.changelog.ChangeLogHistoryService,
osgi.serviceloader;
osgi.serviceloader=liquibase.datatype.LiquibaseDataType,
osgi.serviceloader;
osgi.serviceloader=liquibase.configuration.ConfigurationValueProvider,
osgi.serviceloader;
osgi.serviceloader=liquibase.logging.LogService,
osgi.serviceloader;
osgi.serviceloader=liquibase.snapshot.SnapshotGenerator,
osgi.serviceloader;
osgi.serviceloader=liquibase.parser.ChangeLogParser,
osgi.serviceloader;
osgi.serviceloader=liquibase.servicelocator.ServiceLocator,
osgi.serviceloader;
osgi.serviceloader=liquibase.diff.compare.DatabaseObjectComparator,
osgi.serviceloader;
osgi.serviceloader=liquibase.command.LiquibaseCommand,
osgi.serviceloader;
osgi.serviceloader=liquibase.license.LicenseService,
osgi.serviceloader;
osgi.serviceloader=liquibase.diff.output.changelog.ChangeGenerator,
osgi.serviceloader;
osgi.serviceloader=liquibase.executor.Executor,
osgi.serviceloader;
osgi.serviceloader=liquibase.structure.DatabaseObject,
osgi.serviceloader;
osgi.serviceloader=liquibase.parser.SnapshotParser,
osgi.serviceloader;
osgi.serviceloader=liquibase.hub.HubService,
osgi.serviceloader;
osgi.serviceloader=liquibase.command.CommandStep,
osgi.serviceloader;
osgi.serviceloader=liquibase.sqlgenerator.SqlGenerator
</Provide-Capability>
<Require-Capability>
osgi.extender;
filter:="(osgi.extender=osgi.serviceloader.registrar)",
osgi.extender;
filter:="(osgi.extender=osgi.serviceloader.processor)",
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.serializer.ChangeLogSerializer)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.parser.NamespaceDetails)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.database.Database)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.change.Change)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.database.DatabaseConnection)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.precondition.Precondition)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.serializer.SnapshotSerializer)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.configuration.AutoloadedConfigurations)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.diff.DiffGenerator)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.lockservice.LockService)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.changelog.ChangeLogHistoryService)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.datatype.LiquibaseDataType)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.configuration.ConfigurationValueProvider)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.logging.LogService)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.snapshot.SnapshotGenerator)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.parser.ChangeLogParser)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.servicelocator.ServiceLocator)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.diff.compare.DatabaseObjectComparator)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.command.LiquibaseCommand)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.license.LicenseService)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.diff.output.changelog.ChangeGenerator)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.executor.Executor)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.structure.DatabaseObject)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.parser.SnapshotParser)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.hub.HubService)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.command.CommandStep)";
cardinality:=multiple,
osgi.serviceloader;
filter:="(osgi.serviceloader=liquibase.sqlgenerator.SqlGenerator)";
cardinality:=multiple
</Require-Capability>
</instructions>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>7</source>
<target>7</target>
</configuration>
</plugin>
</plugins>
</build>
<scm>
<!-- TODO -->
</scm>
<distributionManagement>
<!-- TODO -->
</distributionManagement>
</project>
On Sun, Aug 14, 2022 at 5:30 PM Steinar Bang <sb...@dod.no> wrote:
> >>>>> Steven Huypens <st...@gmail.com>:
>
> > Hi Steinar,
> > The xsd stacktrace can be explained by this liquibase-issue I think :
> > https://github.com/liquibase/liquibase/issues/2817
>
> > They closed it incorrectly imho
>
> The "fix" was just changed wording in the exception message...? (unless
> I'm missing something...?
>
> If so, then I agree with you.
>
> I know from googling that Nathan Woxland is reluctant to making XML
> parsing not requiring validation (a stack overflow answer from 2010, I
> think it was), so not requiring XSDs is not an option.
>
> I have managed to make a workaround/hackaround, by slurping the
> offending XSD file out of liquibase-core:
>
> https://gist.github.com/steinarb/96840701be3aa387f670b191f44ac14d#file-pom-xml-L164
> and then adding it to the resource classpath of the bundle liquibase is
> executed from
>
> https://gist.github.com/steinarb/96840701be3aa387f670b191f44ac14d#file-pom-xml-L157
> (liquibase looks in www.liquibase.org/xml/ns/dbchangelog/ on the
> classpath, for the XSD files)
>
> But with the additional require-capability
>
> https://gist.github.com/steinarb/96840701be3aa387f670b191f44ac14d#file-pom-xml-L127
> it does feel to me, like liquibase 4.x isn't very OSGi friendly... all
> of this have to be done in the build config for every bundle that is to
> run liquibase...
>
> Oh well!
>
>
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steven Huypens <st...@gmail.com>:
> Hi Steinar,
> The xsd stacktrace can be explained by this liquibase-issue I think :
> https://github.com/liquibase/liquibase/issues/2817
> They closed it incorrectly imho
The "fix" was just changed wording in the exception message...? (unless
I'm missing something...?
If so, then I agree with you.
I know from googling that Nathan Woxland is reluctant to making XML
parsing not requiring validation (a stack overflow answer from 2010, I
think it was), so not requiring XSDs is not an option.
I have managed to make a workaround/hackaround, by slurping the
offending XSD file out of liquibase-core:
https://gist.github.com/steinarb/96840701be3aa387f670b191f44ac14d#file-pom-xml-L164
and then adding it to the resource classpath of the bundle liquibase is
executed from
https://gist.github.com/steinarb/96840701be3aa387f670b191f44ac14d#file-pom-xml-L157
(liquibase looks in www.liquibase.org/xml/ns/dbchangelog/ on the
classpath, for the XSD files)
But with the additional require-capability
https://gist.github.com/steinarb/96840701be3aa387f670b191f44ac14d#file-pom-xml-L127
it does feel to me, like liquibase 4.x isn't very OSGi friendly... all
of this have to be done in the build config for every bundle that is to
run liquibase...
Oh well!
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steven Huypens <st...@gmail.com>.
Hi Steinar,
The xsd stacktrace can be explained by this liquibase-issue I think :
https://github.com/liquibase/liquibase/issues/2817
They closed it incorrectly imho
Kind regards,
Steven
On Sun, Aug 14, 2022 at 8:31 AM Steinar Bang <sb...@dod.no> wrote:
> >>>>> Steven Huypens <st...@gmail.com>:
>
> > 1) The exception "No Common Superclass:javax/crypto/SecretKey
> > javax/crypto/SecretKeyFactory" is from the liquibase pro package, as
> > you can see in the stacktrace. But I do not understand that, because
> > that package is not part of the 4.9.1 liquibase-core.jar. It might be
> > however that we are not seeing that error because we do not use the
> > Liquibase class.
>
> Thanks for the info.
>
> I'd put that one off for later, since it didn't seem to affect the rest
> of the operation. So I hadn't studied it closesly.
>
> Maybe it'll disappear when I stop using the Liquibase class...? :-)
>
> > 2) The XSDLookUpException is because liquibase cannot find the xsd's
> > located in liquibase-core-4.9.1.jar\www.liquibase.org
> \xml\ns\dbchangelog,
> > it's a classLoader issue I guess.
>
> Yep! The OSGi class loader only loads classes from other bundles. All
> resources are local bundle only.
>
> I've actually used that to good effect in bundles where I've used
> Liquibase, but I've landed on having each project having its own unique
> file naming, when I had to load two liquibase-loading bundles as normal
> jar files in a unit test, and the changelog files suddenly
> overlapping...
>
> > 3) The spifly startlevel seems to be important indeed. Since I lowered
> it's
> > startLevel in our own code, the exception did not occur anymore.
>
> Yep!
>
> > 4) I don't know which approach you're supposed to use, sorry.
>
> I'll figure it out eventually, I'm too stubborn to give up. :-)
>
> > 5) fyi, I'm using this to assert the exception. Maybe it's useful for you
> > as well ..
>
> > Stream<String> lines = Files.lines(Paths.get("data/log/karaf.log"));
> > lines.forEach(l -> System.out.println(l));
> > Assert.assertFalse(lines.anyMatch(line -> line.contains("Exception")));
>
> Ah! Clever! Thanks for the tip!
>
>
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steven Huypens <st...@gmail.com>:
> 1) The exception "No Common Superclass:javax/crypto/SecretKey
> javax/crypto/SecretKeyFactory" is from the liquibase pro package, as
> you can see in the stacktrace. But I do not understand that, because
> that package is not part of the 4.9.1 liquibase-core.jar. It might be
> however that we are not seeing that error because we do not use the
> Liquibase class.
Thanks for the info.
I'd put that one off for later, since it didn't seem to affect the rest
of the operation. So I hadn't studied it closesly.
Maybe it'll disappear when I stop using the Liquibase class...? :-)
> 2) The XSDLookUpException is because liquibase cannot find the xsd's
> located in liquibase-core-4.9.1.jar\www.liquibase.org\xml\ns\dbchangelog,
> it's a classLoader issue I guess.
Yep! The OSGi class loader only loads classes from other bundles. All
resources are local bundle only.
I've actually used that to good effect in bundles where I've used
Liquibase, but I've landed on having each project having its own unique
file naming, when I had to load two liquibase-loading bundles as normal
jar files in a unit test, and the changelog files suddenly
overlapping...
> 3) The spifly startlevel seems to be important indeed. Since I lowered it's
> startLevel in our own code, the exception did not occur anymore.
Yep!
> 4) I don't know which approach you're supposed to use, sorry.
I'll figure it out eventually, I'm too stubborn to give up. :-)
> 5) fyi, I'm using this to assert the exception. Maybe it's useful for you
> as well ..
> Stream<String> lines = Files.lines(Paths.get("data/log/karaf.log"));
> lines.forEach(l -> System.out.println(l));
> Assert.assertFalse(lines.anyMatch(line -> line.contains("Exception")));
Ah! Clever! Thanks for the tip!
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steven Huypens <st...@gmail.com>.
Hi Steinar,
1) The exception "No Common Superclass:javax/crypto/SecretKey
javax/crypto/SecretKeyFactory" is from the liquibase pro package, as you
can see in the stacktrace. But I do not understand that, because that
package is not part of the 4.9.1 liquibase-core.jar. It might be however
that we are not seeing that error because we do not use the Liquibase class.
2) The XSDLookUpException is because liquibase cannot find the xsd's
located in liquibase-core-4.9.1.jar\www.liquibase.org\xml\ns\dbchangelog,
it's a classLoader issue I guess.
3) The spifly startlevel seems to be important indeed. Since I lowered it's
startLevel in our own code, the exception did not occur anymore.
4) I don't know which approach you're supposed to use, sorry.
5) fyi, I'm using this to assert the exception. Maybe it's useful for you
as well ..
Stream<String> lines = Files.lines(Paths.get("data/log/karaf.log"));
lines.forEach(l -> System.out.println(l));
Assert.assertFalse(lines.anyMatch(line -> line.contains("Exception")));
On Sat, Aug 13, 2022 at 12:02 PM Steinar Bang <sb...@dod.no> wrote:
> >>>>> Steinar Bang <sb...@dod.no>:
>
> > Hm... I removed the liquibase-slf4j bundle from the feature.xml and then
> > I got a different karaf.log
> > https://gist.github.com/steinarb/c04967029621509053f0b6c5903f5fcf
>
> The important stack trace here, is:
>
> https://gist.github.com/steinarb/fe342779dfe50b82345e9ac04718f17d#file-karaf-log-L433
>
> This is because of the way I'm using the Liquibase Java API and because
> the Liquibase.update() operation creates a new scope from the root
> scope, when there already is a scope, and that doesn't work.
>
> Possibly releated to this open liquibase issue...? (I googled the error
> message)
> https://github.com/liquibase/liquibase/issues/2248
>
> My approach worked for liquibase 3.x, but I have to rethink what to do
> for liquibase 4.x (I don't feel like waiting until issue 2248 is fixed).
> Maybe look at Steven's pseudi code?
>
>
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
> Hm... I removed the liquibase-slf4j bundle from the feature.xml and then
> I got a different karaf.log
> https://gist.github.com/steinarb/c04967029621509053f0b6c5903f5fcf
The important stack trace here, is:
https://gist.github.com/steinarb/fe342779dfe50b82345e9ac04718f17d#file-karaf-log-L433
This is because of the way I'm using the Liquibase Java API and because
the Liquibase.update() operation creates a new scope from the root
scope, when there already is a scope, and that doesn't work.
Possibly releated to this open liquibase issue...? (I googled the error message)
https://github.com/liquibase/liquibase/issues/2248
My approach worked for liquibase 3.x, but I have to rethink what to do
for liquibase 4.x (I don't feel like waiting until issue 2248 is fixed).
Maybe look at Steven's pseudi code?
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
>> What I was trying to tell you, is that by replacing your
>> spifly-feature with the aries-proxy-feature and a single spifly
>> bundle, the classloading issue might be fixed. Liquibase is able to
>> find the Default Log Service, as far as I can see.
> Ahhh! I see!
>> I got 2 new exceptions, one of which can be ignored.If you manage to
>> fix the other exception, I think you get a little closer to the
>> solution.
> Thanks! I will try! I'll report back in this (long) thread!
I didn't get any exceptions from liquibase pro, but I got 3 exceptions
"No Common Superclass:javax/crypto/SecretKey javax/crypto/SecretKeyFactory":
https://gist.github.com/steinarb/3c3ff4e570bf8c6a380343233ed1e471#file-karaf-log-L342
"liquibase.parser.core.xml.XSDLookUpException: Unable to resolve xml entity http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.5.xsd locally: liquibase.secureParsing is set to 'true' which does not allow remote lookups. Set it to 'false' to allow remote lookups of xsd files."
https://gist.github.com/steinarb/3c3ff4e570bf8c6a380343233ed1e471#file-karaf-log-L439
(this one is possible fixable by disabling XSD validation...?)
java.lang.NullPointerException
https://gist.github.com/steinarb/3c3ff4e570bf8c6a380343233ed1e471#file-karaf-log-L535
(possibly caused by ServiceLoader not finding any services...? But that
is guessing on my part)
Here's the feature.xml used
https://gist.github.com/steinarb/22f4ea2e7eda7018bae0af2ad7f774df
Hm... I removed the liquibase-slf4j bundle from the feature.xml and then
I got a different karaf.log
https://gist.github.com/steinarb/c04967029621509053f0b6c5903f5fcf
Not the "normal" error messages (e.g. "Cannot find default Log
service"), but error messages from the derby shutdown, and no logging
indicating that liquibase is actually doing anything.
Hm... I tried again, and this time it looks like liquibase failing
because the derby database has been shut down...?
https://gist.github.com/steinarb/fe342779dfe50b82345e9ac04718f17d#file-karaf-log-L484
And this time I again got the "No Common
Superclass:javax/crypto/SecretKey javax/crypto/SecretKeyFactory" error.
So the behaviour isn't entirely predictable with this setup...?
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steven Huypens <st...@gmail.com>:
> What I was trying to tell you, is that by replacing your spifly-feature
> with the aries-proxy-feature and a single spifly bundle, the classloading
> issue might be fixed. Liquibase is able to find the Default Log Service, as
> far as I can see.
Ahhh! I see!
> I got 2 new exceptions, one of which can be ignored.If you manage to fix
> the other exception, I think you get a little closer to the solution.
Thanks! I will try! I'll report back in this (long) thread!
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steven Huypens <st...@gmail.com>.
Hi Steinar,
What I was trying to tell you, is that by replacing your spifly-feature
with the aries-proxy-feature and a single spifly bundle, the classloading
issue might be fixed. Liquibase is able to find the Default Log Service, as
far as I can see.
I got 2 new exceptions, one of which can be ignored.If you manage to fix
the other exception, I think you get a little closer to the solution.
Kind regards,
Steven
On Tue, Aug 9, 2022 at 12:51 PM Steinar Bang <sb...@dod.no> wrote:
> >>>>> Steven Huypens <st...@gmail.com>:
>
> > Hi Steinar,
> > As far as I can tell, there is a spifly-issue, which is why Liquibase
> > cannot create the correct Log Services. I tried
>
> > <feature name="liquibase-core" start-level="20"
> > version="${liquibase.version}">
> > <feature>aries-proxy</feature>
>
> >
> <bundle>mvn:org.apache.aries.spifly/org.apache.aries.spifly.dynamic.bundle/1.3.4</bundle>
>
> > <bundle>mvn:org.yaml/snakeyaml/${snakeyaml.version}</bundle>
>
> > <bundle>mvn:org.liquibase/liquibase-core/${liquibase.version}</bundle>
> > </feature>
>
> Yes, I tried to handle this by requiring the
> <feature>spifly</feature>
>
> This is what I've done in the
> liquibase-karaf-feature/use-liquibase-slf4j-410
> branch.
>
> > which leads to new exceptions. The one from Liquibase Pro can be ignored
> > for now, I guess, and the other one might be related to your
> > ResourceAccessor, Good luck, I hope this helps you a little.
>
> Thanks!
>
> The fact that I know that running liquibase 4.x in OSGi works for you,
> is my hope that it should be possible to solve this. :-)
>
> I think I know what's going on now, but what I don't know is how to fix it.
>
> When debugging the init, I saw that a JavaLogService (which implements
> the LogService interface), was created here (I put a breakpoint in the
> baseclass constructor of JavaLogService, and the breakpoint was
> triggered):
>
> https://github.com/liquibase/liquibase/blob/master/liquibase-core/src/main/java/liquibase/Scope.java#L78
>
> And then LogServiceFactory.getDefaultLogService() failed here
>
> https://github.com/liquibase/liquibase/blob/master/liquibase-core/src/main/java/liquibase/Scope.java#L85
>
> (at this point I was already thinking "possible classloader issue...?")
>
> The LogServiceFactory.getDefaultLogService() method called
> AbstractServiceFactory<LogService>.getPlugin(), which returns null,
> because the ServiceLocator doesn't find any instances of LogService
>
> https://github.com/liquibase/liquibase/blob/master/liquibase-core/src/main/java/liquibase/plugin/AbstractPluginFactory.java#L98
>
> And StandardServiceLocator.findInstances(Class<LogService>), used
> java.util.ServiceLoader.load() to find the instances
>
> https://github.com/liquibase/liquibase/blob/master/liquibase-core/src/main/java/liquibase/servicelocator/StandardServiceLocator.java#L21
>
> So, yep, defintely a classloader issue.
>
> The java.util.ServiceLoader is what powers the spring dependency
> injection mechanism, and definitely works best when you have a single
> classloader.
>
> So I googled "ServiceLoader osgi" and immediately found this clarifying
> (but old (february 13 2013)) blog post:
> https://blog.osgi.org/2013/02/javautilserviceloader-in-osgi.html
>
> Liquibase is attempting to use the serviceloader.
>
> They are requiring the necessary capabilities (this is why liquibase now
> needs something like spifly, to load):
>
> https://github.com/liquibase/liquibase/blob/master/liquibase-core/pom.xml#L198
>
> And they are providing the capabilities, which should make it possible
> for a different bundle to have ServiceLoader successfully do a classpath
> scan and find the services.
>
> So to sum it up: the reason LogService isn't found is a classloader
> issue, there is a known fix, and liquibase has implemented the fix.
>
> The only problem is... I haven't been able to make it work...:-/
>
> I have tried the following:
>
> 1. Requiring the LogService feature in the maven-bundle-plugin config,
> based on the example in the feb 2013 blog post:
> <Require-Capability>
> osgi.extender;
> filter:="(osgi.extender=osgi.serviceloader.processor)",
>
> osgi.serviceloader;filter:="(osgi.serviceloader=liquibase.logging.LogService)";cardinality:=multiple
> </Require-Capability>
>
> but that failed in the karaf feature loader, because the
> osgi.serviceloader.processor capability couldn't be found.
>
> 2. So I copied the entire <Require-Capability> from liquibase-core
>
> https://github.com/liquibase/liquibase/blob/master/liquibase-core/pom.xml#L198
> where all of the services are provided by the liquibase-core manifest
>
> https://github.com/liquibase/liquibase/blob/master/liquibase-core/pom.xml#L169
> (the osgi.serviceloader.registrar and osgi.serviceloader.processor
> capabilities would presumably be provided by spifly, same as for the
> osgi-core itself)
>
> And this one have no load failures, but the karaf.log contains the
> dreaded "Cannot find default log service" stack trace
>
> I think that everything created inside the liquibase-core bundle itself,
> using that bundle's classloader(s), would work just fine.
>
> But Liquibase classes I instanciate in my own bundles will be unable to
> find the various services.
>
> I've been thinking of various exotic ideas, such as e.g.
> 1. Define a liquibase factory service interface, and make an OSGi
> bundle fragment that adds an implementation of this service (can
> bundle fragments even provide services...?)
>
> 2. Somehow expose the services created by liquibase init as OSGi
> services, and use DS to inject those services into my bundles.
> No idea how to go about this, except that it would probably involve
> a bundle fragment somehow
>
> But, as I say "exotic ideas". I don't think they will work, and they
> feel like a hack. I'm just throwing them out there, to clear them out of
> my head.
>
> I'm sure that the solution, if there is one, will turn out to be a lot
> simpler...
>
>
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steven Huypens <st...@gmail.com>:
> Hi Steinar,
> As far as I can tell, there is a spifly-issue, which is why Liquibase
> cannot create the correct Log Services. I tried
> <feature name="liquibase-core" start-level="20"
> version="${liquibase.version}">
> <feature>aries-proxy</feature>
> <bundle>mvn:org.apache.aries.spifly/org.apache.aries.spifly.dynamic.bundle/1.3.4</bundle>
> <bundle>mvn:org.yaml/snakeyaml/${snakeyaml.version}</bundle>
> <bundle>mvn:org.liquibase/liquibase-core/${liquibase.version}</bundle>
> </feature>
Yes, I tried to handle this by requiring the
<feature>spifly</feature>
This is what I've done in the liquibase-karaf-feature/use-liquibase-slf4j-410
branch.
> which leads to new exceptions. The one from Liquibase Pro can be ignored
> for now, I guess, and the other one might be related to your
> ResourceAccessor, Good luck, I hope this helps you a little.
Thanks!
The fact that I know that running liquibase 4.x in OSGi works for you,
is my hope that it should be possible to solve this. :-)
I think I know what's going on now, but what I don't know is how to fix it.
When debugging the init, I saw that a JavaLogService (which implements
the LogService interface), was created here (I put a breakpoint in the
baseclass constructor of JavaLogService, and the breakpoint was
triggered):
https://github.com/liquibase/liquibase/blob/master/liquibase-core/src/main/java/liquibase/Scope.java#L78
And then LogServiceFactory.getDefaultLogService() failed here
https://github.com/liquibase/liquibase/blob/master/liquibase-core/src/main/java/liquibase/Scope.java#L85
(at this point I was already thinking "possible classloader issue...?")
The LogServiceFactory.getDefaultLogService() method called
AbstractServiceFactory<LogService>.getPlugin(), which returns null,
because the ServiceLocator doesn't find any instances of LogService
https://github.com/liquibase/liquibase/blob/master/liquibase-core/src/main/java/liquibase/plugin/AbstractPluginFactory.java#L98
And StandardServiceLocator.findInstances(Class<LogService>), used
java.util.ServiceLoader.load() to find the instances
https://github.com/liquibase/liquibase/blob/master/liquibase-core/src/main/java/liquibase/servicelocator/StandardServiceLocator.java#L21
So, yep, defintely a classloader issue.
The java.util.ServiceLoader is what powers the spring dependency
injection mechanism, and definitely works best when you have a single
classloader.
So I googled "ServiceLoader osgi" and immediately found this clarifying
(but old (february 13 2013)) blog post:
https://blog.osgi.org/2013/02/javautilserviceloader-in-osgi.html
Liquibase is attempting to use the serviceloader.
They are requiring the necessary capabilities (this is why liquibase now
needs something like spifly, to load):
https://github.com/liquibase/liquibase/blob/master/liquibase-core/pom.xml#L198
And they are providing the capabilities, which should make it possible
for a different bundle to have ServiceLoader successfully do a classpath
scan and find the services.
So to sum it up: the reason LogService isn't found is a classloader
issue, there is a known fix, and liquibase has implemented the fix.
The only problem is... I haven't been able to make it work...:-/
I have tried the following:
1. Requiring the LogService feature in the maven-bundle-plugin config,
based on the example in the feb 2013 blog post:
<Require-Capability>
osgi.extender; filter:="(osgi.extender=osgi.serviceloader.processor)",
osgi.serviceloader;filter:="(osgi.serviceloader=liquibase.logging.LogService)";cardinality:=multiple
</Require-Capability>
but that failed in the karaf feature loader, because the
osgi.serviceloader.processor capability couldn't be found.
2. So I copied the entire <Require-Capability> from liquibase-core
https://github.com/liquibase/liquibase/blob/master/liquibase-core/pom.xml#L198
where all of the services are provided by the liquibase-core manifest
https://github.com/liquibase/liquibase/blob/master/liquibase-core/pom.xml#L169
(the osgi.serviceloader.registrar and osgi.serviceloader.processor
capabilities would presumably be provided by spifly, same as for the
osgi-core itself)
And this one have no load failures, but the karaf.log contains the
dreaded "Cannot find default log service" stack trace
I think that everything created inside the liquibase-core bundle itself,
using that bundle's classloader(s), would work just fine.
But Liquibase classes I instanciate in my own bundles will be unable to
find the various services.
I've been thinking of various exotic ideas, such as e.g.
1. Define a liquibase factory service interface, and make an OSGi
bundle fragment that adds an implementation of this service (can
bundle fragments even provide services...?)
2. Somehow expose the services created by liquibase init as OSGi
services, and use DS to inject those services into my bundles.
No idea how to go about this, except that it would probably involve
a bundle fragment somehow
But, as I say "exotic ideas". I don't think they will work, and they
feel like a hack. I'm just throwing them out there, to clear them out of
my head.
I'm sure that the solution, if there is one, will turn out to be a lot
simpler...
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steven Huypens <st...@gmail.com>.
Hi Steinar,
As far as I can tell, there is a spifly-issue, which is why Liquibase
cannot create the correct Log Services. I tried
<feature name="liquibase-core" start-level="20"
version="${liquibase.version}">
<feature>aries-proxy</feature>
<bundle>mvn:org.apache.aries.spifly/org.apache.aries.spifly.dynamic.bundle/1.3.4</bundle>
<bundle>mvn:org.yaml/snakeyaml/${snakeyaml.version}</bundle>
<bundle>mvn:org.liquibase/liquibase-core/${liquibase.version}</bundle>
</feature>
which leads to new exceptions. The one from Liquibase Pro can be ignored
for now, I guess, and the other one might be related to your
ResourceAccessor, Good luck, I hope this helps you a little.
Kind regards,
Steven
On Sun, Aug 7, 2022 at 6:02 PM Steinar Bang <sb...@dod.no> wrote:
> >>>>> Steven Huypens <st...@gmail.com>:
>
> > Hi Steinar,
> > I tried the liquibase-karaf-feature/use-liquibase-slf4j-410 branch. After
> > changing the liquibase version to 4.9.1 in the pom.xml, the build is
> > successful.
>
> Yes, but if you take a look at
>
> liquibase-karaf-feature/liquibase-integration-test/karaf.liquibase.tests/target/exam/<somehash>/log/karaf.log
> you will see a stacktrace in the log, caused by "Cannot find default log
> service" and no liquibase logging from setting up the schema and
> populating it with data.
> https://gist.github.com/steinarb/2857d60427d42686299b1d9a388ee420
>
> But if you switch to the master branch and change liquibase.version in
> .m2/settings.xml back to 3.8.0 and re-run "mvn clean install" you will
> find a
>
> liquibase-karaf-feature/liquibase-integration-test/karaf.liquibase.tests/target/exam/<somehash>/log/karaf.log
> that doesn't have the "Cannot find default log service" error, and
> contains the log output from liquibase
> https://gist.github.com/steinarb/d5902ae12bdcc1f96bd66eaf45f9311b
>
> This is the behaviour that has eluded me in all liquibase versions since
> 3.8.0(*), and the reason I haven't updated the liquibase version of the
> feature.
>
> But I've not given up yet! :-)
>
> (*) in 3.x versions >3.8.0 because of various OSGi and JDK 8 vs 11
> related problems, and in 4.x because the log system has changed and
> I haven't been able to make the liquibase logging work right with
> karaf
>
>
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steven Huypens <st...@gmail.com>:
> Hi Steinar,
> I tried the liquibase-karaf-feature/use-liquibase-slf4j-410 branch. After
> changing the liquibase version to 4.9.1 in the pom.xml, the build is
> successful.
Yes, but if you take a look at
liquibase-karaf-feature/liquibase-integration-test/karaf.liquibase.tests/target/exam/<somehash>/log/karaf.log
you will see a stacktrace in the log, caused by "Cannot find default log
service" and no liquibase logging from setting up the schema and
populating it with data.
https://gist.github.com/steinarb/2857d60427d42686299b1d9a388ee420
But if you switch to the master branch and change liquibase.version in
.m2/settings.xml back to 3.8.0 and re-run "mvn clean install" you will
find a
liquibase-karaf-feature/liquibase-integration-test/karaf.liquibase.tests/target/exam/<somehash>/log/karaf.log
that doesn't have the "Cannot find default log service" error, and
contains the log output from liquibase
https://gist.github.com/steinarb/d5902ae12bdcc1f96bd66eaf45f9311b
This is the behaviour that has eluded me in all liquibase versions since
3.8.0(*), and the reason I haven't updated the liquibase version of the
feature.
But I've not given up yet! :-)
(*) in 3.x versions >3.8.0 because of various OSGi and JDK 8 vs 11
related problems, and in 4.x because the log system has changed and
I haven't been able to make the liquibase logging work right with
karaf
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steven Huypens <st...@gmail.com>.
Hi Steinar,
I tried the liquibase-karaf-feature/use-liquibase-slf4j-410 branch. After
changing the liquibase version to 4.9.1 in the pom.xml, the build is
successful.
Best regards,
Steven
On Sat, Aug 6, 2022 at 10:44 PM Steinar Bang <sb...@dod.no> wrote:
> >>>>> Steven Huypens <st...@gmail.com>:
>
> > Hi Steinar,
> > Manually changing the liquibse version to 4.9.1 results in
>
> > ... 15 more
> > Caused by: org.apache.felix.resolver.reason.ReasonException: Unable to
> > resolve org.liquibase.core/4.9.1: missing requirement
> > [org.liquibase.core/4.9.1] osgi.extender;
> > filter:="(osgi.extender=osgi.serviceloader.registrar)"
> > at
> >
> org.apache.felix.resolver.Candidates$MissingRequirementError.toException(Candidates.java:1341)
> > ... 16 more
>
> This is from the master branch with liquibase.version 4.9.1?
>
> If you switch to the liquibase-karaf-feature/use-liquibase-slf4j-410
> branch, it will pull in the built-in karaf feature "spifly", which
> provides the capability osgi.serviceloader.registrar.
>
> > Isn't there a way to chat online? Otherwise this is going to take a
> > lot of time :-)
>
> Sorry, no, I don't have anything like that.
>
> I used to have some chat services back in the early 00-ies, but they
> gradually died and I never got around to replacing them.
>
>
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steven Huypens <st...@gmail.com>:
> Hi Steinar,
> Manually changing the liquibse version to 4.9.1 results in
> ... 15 more
> Caused by: org.apache.felix.resolver.reason.ReasonException: Unable to
> resolve org.liquibase.core/4.9.1: missing requirement
> [org.liquibase.core/4.9.1] osgi.extender;
> filter:="(osgi.extender=osgi.serviceloader.registrar)"
> at
> org.apache.felix.resolver.Candidates$MissingRequirementError.toException(Candidates.java:1341)
> ... 16 more
This is from the master branch with liquibase.version 4.9.1?
If you switch to the liquibase-karaf-feature/use-liquibase-slf4j-410
branch, it will pull in the built-in karaf feature "spifly", which
provides the capability osgi.serviceloader.registrar.
> Isn't there a way to chat online? Otherwise this is going to take a
> lot of time :-)
Sorry, no, I don't have anything like that.
I used to have some chat services back in the early 00-ies, but they
gradually died and I never got around to replacing them.
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steven Huypens <st...@gmail.com>.
Hi Steinar,
Manually changing the liquibse version to 4.9.1 results in
... 15 more
Caused by: org.apache.felix.resolver.reason.ReasonException: Unable to
resolve org.liquibase.core/4.9.1: missing requirement
[org.liquibase.core/4.9.1] osgi.extender;
filter:="(osgi.extender=osgi.serviceloader.registrar)"
at
org.apache.felix.resolver.Candidates$MissingRequirementError.toException(Candidates.java:1341)
... 16 more
Isn't there a way to chat online? Otherwise this is going to take a lot of
time :-)
Best regards,
Steven
On Sat, Aug 6, 2022 at 8:03 PM Steinar Bang <sb...@dod.no> wrote:
> >>>>> Steven Huypens <st...@gmail.com>:
>
> > Hi Steinar,
> > I tried to build the project, but ran into several issues. If you can
> > update it, also updating the liquibase version, I'd be happy to take a
> look.
>
> Ah, right! Sorry about that!
>
> It was too much work, and too confusing, to update the liquibase-version
> using git commits, especially when I wanted to try different versions
> of liquibase against a set of modifications.
>
> So what I'm doing during development, is set the liquibase.version in
> ~/.m2/settings.xml
>
> Note: the version will be correct, if I make a release with a version
> matching the actual liquibase version I'm making the feature for (which
> is what I have been doing up to liquibase version 3.8.0, which is the
> last version I didn't have any issues with.
>
> (If I mess up and need to release a feature with a different version than
> liquibase, I need to commit the liquibase version explicitly in the
> pom. But that is a problem for the future)
>
> Anyway, as I said: for development it is easiest to adjust the version in
> ~/.m2/settings.xml
>
> To get it to build, use the setting
> <settings>
> <profiles>
> <profile>
> <id>non-snapshot-liquibase</id>
> <activation><activeByDefault>true</activeByDefault></activation>
> <properties>
> <liquibase.version>3.8.0</liquibase.version>
> </properties>
> </profile>
> </profiles>
> </settings>
>
> To get the version you're using, bump liquibase.version in
> ~/.m2/settings.xml to 4.9.1.
>
> With the current master HEAD, this should fail with the "Cannot find
> default log service" error in the pax exam karaf log (it is in
> target/exim/<some-hash>/data/log/karaf.log after "mvn clean install" has
> run its course).
>
> I also have locally various experimental branches.
>
> I have pushed the most promising
> liquibase-karaf-feature/use-liquibase-slf4j-410,
> which:
> 1. Removes the rebundling of liquibase-slf4j as a bundle fragment,
> attached to the liquibase-core bundle
> 2. Load liquibase-slf4j 4.1.0 at runtime, see [1] (this version is an
> OSGi bundle fragment that attaches to liquibase-core 4.9.1 and later)
> 3. Load the built-in spifly feature at runtime, which provides the
> osgi.serviceloader.registrar capability (incidentially, the
> serviceloader, is the code that currently fails to find the default
> log service...)
>
> Note: I *think* that the new version of liquibase-slf4j is effectively a
> no-op wrt. to the the way liquibase 4.x finds its logs, but I've left it
> in place for now (as I said: I have various failed experimental branches
> that there is no point in pushing).
>
> References:
> [1] <https://github.com/mattbertolini/liquibase-slf4j/issues/12>
>
>
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steven Huypens <st...@gmail.com>:
> Hi Steinar,
> I tried to build the project, but ran into several issues. If you can
> update it, also updating the liquibase version, I'd be happy to take a look.
Ah, right! Sorry about that!
It was too much work, and too confusing, to update the liquibase-version
using git commits, especially when I wanted to try different versions
of liquibase against a set of modifications.
So what I'm doing during development, is set the liquibase.version in
~/.m2/settings.xml
Note: the version will be correct, if I make a release with a version
matching the actual liquibase version I'm making the feature for (which
is what I have been doing up to liquibase version 3.8.0, which is the
last version I didn't have any issues with.
(If I mess up and need to release a feature with a different version than
liquibase, I need to commit the liquibase version explicitly in the
pom. But that is a problem for the future)
Anyway, as I said: for development it is easiest to adjust the version in
~/.m2/settings.xml
To get it to build, use the setting
<settings>
<profiles>
<profile>
<id>non-snapshot-liquibase</id>
<activation><activeByDefault>true</activeByDefault></activation>
<properties>
<liquibase.version>3.8.0</liquibase.version>
</properties>
</profile>
</profiles>
</settings>
To get the version you're using, bump liquibase.version in
~/.m2/settings.xml to 4.9.1.
With the current master HEAD, this should fail with the "Cannot find
default log service" error in the pax exam karaf log (it is in
target/exim/<some-hash>/data/log/karaf.log after "mvn clean install" has
run its course).
I also have locally various experimental branches.
I have pushed the most promising liquibase-karaf-feature/use-liquibase-slf4j-410,
which:
1. Removes the rebundling of liquibase-slf4j as a bundle fragment,
attached to the liquibase-core bundle
2. Load liquibase-slf4j 4.1.0 at runtime, see [1] (this version is an
OSGi bundle fragment that attaches to liquibase-core 4.9.1 and later)
3. Load the built-in spifly feature at runtime, which provides the
osgi.serviceloader.registrar capability (incidentially, the
serviceloader, is the code that currently fails to find the default
log service...)
Note: I *think* that the new version of liquibase-slf4j is effectively a
no-op wrt. to the the way liquibase 4.x finds its logs, but I've left it
in place for now (as I said: I have various failed experimental branches
that there is no point in pushing).
References:
[1] <https://github.com/mattbertolini/liquibase-slf4j/issues/12>
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steven Huypens <st...@gmail.com>.
Hi Steinar,
I tried to build the project, but ran into several issues. If you can
update it, also updating the liquibase version, I'd be happy to take a look.
Kind regards,
Steven
On Fri, Aug 5, 2022 at 5:22 PM Steinar Bang <sb...@dod.no> wrote:
> >>>>> Steinar Bang <sb...@dod.no>:
> >>>>> Steven Huypens <st...@gmail.com>:
> >> Hi Steinar,
> >> I am not using any scopes indeed, nor any explicit logging
> configuration. I
> >> just realized though, we sometimes see the stacktrace below in our
> >> integration tests, maybe it is related to your problem. It doesn't
> happen a
> >> lot, so I did not have a closer look yet.
>
> >> EventDispatcher: Error during dispatch.
> >> (java.lang.ExceptionInInitializerError)
> >> java.lang.ExceptionInInitializerError
> >> at
> >>
> be.cipal.impl.liquibase.extender.internal.LiquibaseMigrationBundleFactory.loadDatabase(LiquibaseMigrationBundleFactory.java:206)
> >> // skipped part of the trace
> >> Caused by: liquibase.exception.UnexpectedLiquibaseException: Cannot find
> >> default log service
> >> at liquibase.Scope.getCurrentScope(Scope.java:87)
> >> at liquibase.database.DatabaseFactory.<clinit>(DatabaseFactory.java:20)
> >> ... 21 more
>
> > Ah! That's the exact message I'm currently seeing in karaf.log and what
> > I'm trying to get past.
>
> > I get that message consistently (at least I do, in the pax exam
> > integration test I created for the karaf feature).
>
> FYI, this is the integration test:
>
> https://github.com/steinarb/liquibase-karaf-feature/tree/master/liquibase-integration-test
>
> karaf.liquibase.sample is a bundle project defining, and attaching, two
> karaf features:
> 1. sample-db which is a jdbc-config feature defining a derby in-memory
> datasource
> 2. karaf-liquibase-sample which loads a @Component that exposes a
> PreHook service, that plugs into the datasource and sets up a schema
> and populates with some dummy data
>
>
> karaf.liquibase.sample.datasource.receiver is another bundle project,
> thatcontains a @Component that has a Datasource @Reference and is loaded
> by a the attached feature karaf-liquibase-sample-datasource-receiver,
> that requires feature sample-db.
>
> karaf.liquibase.tests is a pax exam karaf test that loads and asserts
> the feature karaf-liquibase-sample-datasource-receiver
>
> Note that it is also possible to load the features of the test manually,
> by building the project, and in a karaf started as my own user (ie. a
> karaf from a binary tar.gz or .zip distribution file), give the
> commands:
>
> feature:repo-add
> mvn:no.priv.bang.karaf/karaf.liquibase.sample.datasource.receiver/LATEST/xml/features
> feature:install karaf-liquibase-sample-datasource-receiver
>
> This is what I have used to debug into the liquibase init code that
> fails to find the log, and give the error messages you've quoted.
>
>
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
>>>>> Steven Huypens <st...@gmail.com>:
>> Hi Steinar,
>> I am not using any scopes indeed, nor any explicit logging configuration. I
>> just realized though, we sometimes see the stacktrace below in our
>> integration tests, maybe it is related to your problem. It doesn't happen a
>> lot, so I did not have a closer look yet.
>> EventDispatcher: Error during dispatch.
>> (java.lang.ExceptionInInitializerError)
>> java.lang.ExceptionInInitializerError
>> at
>> be.cipal.impl.liquibase.extender.internal.LiquibaseMigrationBundleFactory.loadDatabase(LiquibaseMigrationBundleFactory.java:206)
>> // skipped part of the trace
>> Caused by: liquibase.exception.UnexpectedLiquibaseException: Cannot find
>> default log service
>> at liquibase.Scope.getCurrentScope(Scope.java:87)
>> at liquibase.database.DatabaseFactory.<clinit>(DatabaseFactory.java:20)
>> ... 21 more
> Ah! That's the exact message I'm currently seeing in karaf.log and what
> I'm trying to get past.
> I get that message consistently (at least I do, in the pax exam
> integration test I created for the karaf feature).
FYI, this is the integration test:
https://github.com/steinarb/liquibase-karaf-feature/tree/master/liquibase-integration-test
karaf.liquibase.sample is a bundle project defining, and attaching, two
karaf features:
1. sample-db which is a jdbc-config feature defining a derby in-memory
datasource
2. karaf-liquibase-sample which loads a @Component that exposes a
PreHook service, that plugs into the datasource and sets up a schema
and populates with some dummy data
karaf.liquibase.sample.datasource.receiver is another bundle project,
thatcontains a @Component that has a Datasource @Reference and is loaded
by a the attached feature karaf-liquibase-sample-datasource-receiver,
that requires feature sample-db.
karaf.liquibase.tests is a pax exam karaf test that loads and asserts
the feature karaf-liquibase-sample-datasource-receiver
Note that it is also possible to load the features of the test manually,
by building the project, and in a karaf started as my own user (ie. a
karaf from a binary tar.gz or .zip distribution file), give the
commands:
feature:repo-add mvn:no.priv.bang.karaf/karaf.liquibase.sample.datasource.receiver/LATEST/xml/features
feature:install karaf-liquibase-sample-datasource-receiver
This is what I have used to debug into the liquibase init code that
fails to find the log, and give the error messages you've quoted.
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steven Huypens <st...@gmail.com>:
> Hi Steinar,
> I am not using any scopes indeed, nor any explicit logging configuration. I
> just realized though, we sometimes see the stacktrace below in our
> integration tests, maybe it is related to your problem. It doesn't happen a
> lot, so I did not have a closer look yet.
> EventDispatcher: Error during dispatch.
> (java.lang.ExceptionInInitializerError)
> java.lang.ExceptionInInitializerError
> at
> be.cipal.impl.liquibase.extender.internal.LiquibaseMigrationBundleFactory.loadDatabase(LiquibaseMigrationBundleFactory.java:206)
> // skipped part of the trace
> Caused by: liquibase.exception.UnexpectedLiquibaseException: Cannot find
> default log service
> at liquibase.Scope.getCurrentScope(Scope.java:87)
> at liquibase.database.DatabaseFactory.<clinit>(DatabaseFactory.java:20)
> ... 21 more
Ah! That's the exact message I'm currently seeing in karaf.log and what
I'm trying to get past.
I get that message consistently (at least I do, in the pax exam
integration test I created for the karaf feature).
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steven Huypens <st...@gmail.com>.
Hi Steinar,
I am not using any scopes indeed, nor any explicit logging configuration. I
just realized though, we sometimes see the stacktrace below in our
integration tests, maybe it is related to your problem. It doesn't happen a
lot, so I did not have a closer look yet.
EventDispatcher: Error during dispatch.
(java.lang.ExceptionInInitializerError)
java.lang.ExceptionInInitializerError
at
be.cipal.impl.liquibase.extender.internal.LiquibaseMigrationBundleFactory.loadDatabase(LiquibaseMigrationBundleFactory.java:206)
// skipped part of the trace
Caused by: liquibase.exception.UnexpectedLiquibaseException: Cannot find
default log service
at liquibase.Scope.getCurrentScope(Scope.java:87)
at liquibase.database.DatabaseFactory.<clinit>(DatabaseFactory.java:20)
... 21 more
Best regards,
Steven
On Fri, Aug 5, 2022 at 2:23 PM Steinar Bang <sb...@dod.no> wrote:
> >>>>> Steven Huypens <st...@gmail.com>:
>
> > Hi Steinar,
> > Some pseudo-code :
>
> Thanks!
>
> It immediately answers one of my questions, which was what
> resourceaccessor you use, and you use the OsgiResourceAccessor.
>
> Back in 2017 I landed on the ClassLoaderResourceAccessor, based on the
> class using the and it has worked as I expected it to, and wanted it to
> (scan the class path for resources inside the OSGi bundle liquibase is
> run from).
>
> I've tried OsgiResourceAccessor now, to see if that helped with the
> unable to find log issue.
>
> Unfortunately, switching the ResourceAccsessor didn't help. But I have
> debugged through the init, and the two aren't very different
> (ClassLoaderResourceAccessor gets the class loader of the current class,
> which is an OSGi class loader for the current bundle and the
> OSGiResourceAccessor gets a Bundle argument, gets the class loader from
> the bundle and uses it to create a ClassLoaderResourceAccessor), so I
> didn't really expect it to make a difference... but one can hope! :-)
>
> > JdbcConnection connection = new JdbcConnection(myConnection);
> > Database database =
> >
> DatabaseFactory.getInstance().findCorrectDatabaseImplementation(connection);
> > ChangeLogParameters changeLogParameters = new
> ChangeLogParameters(database);
>
> > String liquibaseFilePath = "OSGI-INF/liquibase/master.xml";
> > ResourceAccessor resourceAccessor = new OSGiResourceAccessor(myBundle);
> > DatabaseChangeLog databaseChangeLog =
> ChangeLogParserFactory.getInstance()
> > .getParser(liquibaseFilePath, resourceAccessor)
> > .parse(liquibaseFilePath, changeLogParameters,
> > resourceAccessor);
>
>
> > for (final ChangeSet changeSet : databaseChangeLog.getChangeSets()) {
> > if (database.getRanChangeSet(changeSet) == null) {
> > changeSet.execute(databaseChangeLog, database);
> > database.markChangeSetExecStatus(changeSet,
> > ChangeSet.ExecType.EXECUTED);
> > }
> > }
>
> Hm... doesn't look like you make a child Scope like in the examples
> here...?
>
> https://docs.liquibase.com/workflows/liquibase-community/using-liquibase-java-api.html
>
> That was my second question, because when debugging, I can see a
> JavaLogService be created, and then later not be found by the
> ServiceLocator class.
>
> So I wondered if you had any Scope magic I didn't have, but it looks
> like if you don't.
>
> > I'm sure there are better ways nowadays, but this is how we got it to
> work
> > so many years ago.
>
> Amen to that! :-)
>
> I myself have just continued the way I got working in when I started
> using liquibase in 2017.
>
> Thanks!
>
> I'll keep on debugging the init to see if I can figure out where the log
> service scan fails and why it fails.
>
> And I may also try to change my approach with yours to see if that
> magically makes the logger be found.
>
> PS Do you do any explicit logging configuration inside the bundle, in
> addition the above code? Specifically: Do you create a LogService Java
> object that is not JavaLogServic, inside the bundle?
>
>
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steinar Bang <sb...@dod.no>:
> PS Do you do any explicit logging configuration inside the bundle, in
> addition the above code? Specifically: Do you create a LogService
> Java object that is not JavaLogServic, inside the bundle?
For my own part, I see that in December 2020 I introduced a
src/test/resources/logback.xml file in all projects where I used
liquibase in the unit tests, to limit the log noise when doing "mvn
clean install", with the following content:
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<logger name="liquibase" level="error"/>
<root level="debug">
<appender-ref ref="STDOUT" />
</root>
</configuration>
But I have no log configuration for runtime. There I let karaf provide
whatever it provides.
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steinar Bang <sb...@dod.no>.
>>>>> Steven Huypens <st...@gmail.com>:
> Hi Steinar,
> Some pseudo-code :
Thanks!
It immediately answers one of my questions, which was what
resourceaccessor you use, and you use the OsgiResourceAccessor.
Back in 2017 I landed on the ClassLoaderResourceAccessor, based on the
class using the and it has worked as I expected it to, and wanted it to
(scan the class path for resources inside the OSGi bundle liquibase is
run from).
I've tried OsgiResourceAccessor now, to see if that helped with the
unable to find log issue.
Unfortunately, switching the ResourceAccsessor didn't help. But I have
debugged through the init, and the two aren't very different
(ClassLoaderResourceAccessor gets the class loader of the current class,
which is an OSGi class loader for the current bundle and the
OSGiResourceAccessor gets a Bundle argument, gets the class loader from
the bundle and uses it to create a ClassLoaderResourceAccessor), so I
didn't really expect it to make a difference... but one can hope! :-)
> JdbcConnection connection = new JdbcConnection(myConnection);
> Database database =
> DatabaseFactory.getInstance().findCorrectDatabaseImplementation(connection);
> ChangeLogParameters changeLogParameters = new ChangeLogParameters(database);
> String liquibaseFilePath = "OSGI-INF/liquibase/master.xml";
> ResourceAccessor resourceAccessor = new OSGiResourceAccessor(myBundle);
> DatabaseChangeLog databaseChangeLog = ChangeLogParserFactory.getInstance()
> .getParser(liquibaseFilePath, resourceAccessor)
> .parse(liquibaseFilePath, changeLogParameters,
> resourceAccessor);
> for (final ChangeSet changeSet : databaseChangeLog.getChangeSets()) {
> if (database.getRanChangeSet(changeSet) == null) {
> changeSet.execute(databaseChangeLog, database);
> database.markChangeSetExecStatus(changeSet,
> ChangeSet.ExecType.EXECUTED);
> }
> }
Hm... doesn't look like you make a child Scope like in the examples here...?
https://docs.liquibase.com/workflows/liquibase-community/using-liquibase-java-api.html
That was my second question, because when debugging, I can see a
JavaLogService be created, and then later not be found by the
ServiceLocator class.
So I wondered if you had any Scope magic I didn't have, but it looks
like if you don't.
> I'm sure there are better ways nowadays, but this is how we got it to work
> so many years ago.
Amen to that! :-)
I myself have just continued the way I got working in when I started
using liquibase in 2017.
Thanks!
I'll keep on debugging the init to see if I can figure out where the log
service scan fails and why it fails.
And I may also try to change my approach with yours to see if that
magically makes the logger be found.
PS Do you do any explicit logging configuration inside the bundle, in
addition the above code? Specifically: Do you create a LogService Java
object that is not JavaLogServic, inside the bundle?
Re: Problems with logging to karaf.log from liquibase >3
Posted by Steven Huypens <st...@gmail.com>.
Hi Steinar,
Some pseudo-code :
JdbcConnection connection = new JdbcConnection(myConnection);
Database database =
DatabaseFactory.getInstance().findCorrectDatabaseImplementation(connection);
ChangeLogParameters changeLogParameters = new ChangeLogParameters(database);
String liquibaseFilePath = "OSGI-INF/liquibase/master.xml";
ResourceAccessor resourceAccessor = new OSGiResourceAccessor(myBundle);
DatabaseChangeLog databaseChangeLog = ChangeLogParserFactory.getInstance()
.getParser(liquibaseFilePath, resourceAccessor)
.parse(liquibaseFilePath, changeLogParameters,
resourceAccessor);
for (final ChangeSet changeSet : databaseChangeLog.getChangeSets()) {
if (database.getRanChangeSet(changeSet) == null) {
changeSet.execute(databaseChangeLog, database);
database.markChangeSetExecStatus(changeSet,
ChangeSet.ExecType.EXECUTED);
}
}
I'm sure there are better ways nowadays, but this is how we got it to work
so many years ago.
Steven
On Wed, Aug 3, 2022 at 3:25 PM Steinar Bang <sb...@dod.no> wrote:
> >>>>> Steinar Bang <sb...@dod.no>:
> >>>>> Steven Huypens <st...@gmail.com>:
>
> >> I had a brief look at your repo and I see you are creating an actual
> >> Liquibase instance, which we are not, possibly because of certain
> >> issues in the past. But I don't think that should make a difference
> >> wrt logging.
>
> > No, I don't think that should make a difference either.
>
> Actually, this could make a big difference, because I think how I'm
> creating the Liquibase instance and the resource locator I'm using,
> could effect the log-finding problems I'm seeing.
>
> I'm curious: if you're not creating an actual Liquibase instance, how do
> you use liquibase?
>
> (I'm using Liquibase the only way I was able to figure out it could be
> used, back when I started using liquibse in 2017, and if there is a
> different and better way, I'm open to that)
>
> Thanks!
>
>