You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@karaf.apache.org by Cristiano Costantini <cr...@gmail.com> on 2016/07/13 15:00:15 UTC

Karaf 4: Deadlocks and Livelocks when installing features

Hi all,

Since I have updated to Karaf 4, sometime when I install a feature or when
Karaf starts with some specific combination of bootstrap features,
it happen that Karaf goes either in what it seems a Deadlock (Karaf hang
indefinitely and the feature is not installed) or a Livelock (same as
before but with CPU usage that jumps to 100%)

I'm not able to debug exactly and the effect is not always repeatable, some
time it don't works, some other it works.

Are there any current issue about the feature installation that may be the
cause of this behavior?

thank you for the help!

Cristiano


P.S. I define my features in a file with the namespace,
http://karaf.apache.org/xmlns/features/v1.4.0 so the I understand that
"feature resolver checks the service requirements/capabilities of bundles
for new features (xml schema >= 1.3.0) in order to automatically installs
the required bundles". What does this mean?

Re: Karaf 4: Deadlocks and Livelocks when installing features

Posted by Cristiano Costantini <cr...@gmail.com>.
...while in a "LiveLock" condition, I have inspected with JVisualVM the
status of the JVM and I've taken the following screenshots that show where
more cpu time is spent:

http://imgur.com/vp6zJCl
http://imgur.com/FwIP8Ag
http://imgur.com/ld8QBf4





Il giorno mer 13 lug 2016 alle ore 17:15 Cristiano Costantini <
cristiano.costantini@gmail.com> ha scritto:

> Some more (hopefully) useful info:
>
> in one condition I have a Deadlock, the last log entry recorded is this:
>
> 2016-07-13 17:06:49,231 | INFO  | pool-7-thread-1  | FeaturesServiceImpl
>            | 9 - org.apache.karaf.features.core - 4.0.5 | Adding features:
> instance/[4.0.5,4.0.5], ssh/[4.0.5,4.0.5], webconsole/[4.0.5,4.0.5],
> aries-blueprint/[4.0.5,4.0.5], cxf/[3.1.5,3.1.5], [....]
>
> I've inspected the thread "pool-7-thread-1" and it is in this status:
> "pool-7-thread-1" #81 prio=5 os_prio=31 tid=0x00007fbafc0e6000 nid=0x910b
> waiting on condition [0x0000700002ea0000]
>    java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0x00000007bbbdc608> (a
> java.util.concurrent.FutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvisionInThread(FeaturesServiceImpl.java:1051)
> at
> org.apache.karaf.features.internal.service.FeaturesServiceImpl.installFeatures(FeaturesServiceImpl.java:927)
> at
> org.apache.karaf.features.internal.service.FeaturesServiceImpl.installFeatures(FeaturesServiceImpl.java:820)
> at
> org.apache.karaf.features.internal.service.BootFeaturesInstaller.installBootFeatures(BootFeaturesInstaller.java:112)
> at
> org.apache.karaf.features.internal.service.BootFeaturesInstaller.start(BootFeaturesInstaller.java:92)
> at
> org.apache.karaf.features.internal.osgi.Activator.doStart(Activator.java:259)
> at org.apache.karaf.util.tracker.BaseActivator.run(BaseActivator.java:233)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
> the thread stay in "parked" status for about 120 seconds then it
> terminates and Karaf does not respond to any command. I can only terminate
> it with CTRL-C (note: not CTRL-D to shut it down, exactly CTRL-C)
>
>
>
>
> Il giorno mer 13 lug 2016 alle ore 17:00 Cristiano Costantini <
> cristiano.costantini@gmail.com> ha scritto:
>
>> Hi all,
>>
>> Since I have updated to Karaf 4, sometime when I install a feature or
>> when Karaf starts with some specific combination of bootstrap features,
>> it happen that Karaf goes either in what it seems a Deadlock (Karaf hang
>> indefinitely and the feature is not installed) or a Livelock (same as
>> before but with CPU usage that jumps to 100%)
>>
>> I'm not able to debug exactly and the effect is not always repeatable,
>> some time it don't works, some other it works.
>>
>> Are there any current issue about the feature installation that may be
>> the cause of this behavior?
>>
>> thank you for the help!
>>
>> Cristiano
>>
>>
>> P.S. I define my features in a file with the namespace,
>> http://karaf.apache.org/xmlns/features/v1.4.0 so the I understand that
>> "feature resolver checks the service requirements/capabilities of bundles
>> for new features (xml schema >= 1.3.0) in order to automatically installs
>> the required bundles". What does this mean?
>>
>>
>>

Re: Karaf 4: Deadlocks and Livelocks when installing features

Posted by Cristiano Costantini <cr...@gmail.com>.
Some more (hopefully) useful info:

in one condition I have a Deadlock, the last log entry recorded is this:

2016-07-13 17:06:49,231 | INFO  | pool-7-thread-1  | FeaturesServiceImpl
           | 9 - org.apache.karaf.features.core - 4.0.5 | Adding features:
instance/[4.0.5,4.0.5], ssh/[4.0.5,4.0.5], webconsole/[4.0.5,4.0.5],
aries-blueprint/[4.0.5,4.0.5], cxf/[3.1.5,3.1.5], [....]

I've inspected the thread "pool-7-thread-1" and it is in this status:
"pool-7-thread-1" #81 prio=5 os_prio=31 tid=0x00007fbafc0e6000 nid=0x910b
waiting on condition [0x0000700002ea0000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000007bbbdc608> (a
java.util.concurrent.FutureTask)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
at java.util.concurrent.FutureTask.get(FutureTask.java:191)
at
org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvisionInThread(FeaturesServiceImpl.java:1051)
at
org.apache.karaf.features.internal.service.FeaturesServiceImpl.installFeatures(FeaturesServiceImpl.java:927)
at
org.apache.karaf.features.internal.service.FeaturesServiceImpl.installFeatures(FeaturesServiceImpl.java:820)
at
org.apache.karaf.features.internal.service.BootFeaturesInstaller.installBootFeatures(BootFeaturesInstaller.java:112)
at
org.apache.karaf.features.internal.service.BootFeaturesInstaller.start(BootFeaturesInstaller.java:92)
at
org.apache.karaf.features.internal.osgi.Activator.doStart(Activator.java:259)
at org.apache.karaf.util.tracker.BaseActivator.run(BaseActivator.java:233)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)


the thread stay in "parked" status for about 120 seconds then it terminates
and Karaf does not respond to any command. I can only terminate it with
CTRL-C (note: not CTRL-D to shut it down, exactly CTRL-C)




Il giorno mer 13 lug 2016 alle ore 17:00 Cristiano Costantini <
cristiano.costantini@gmail.com> ha scritto:

> Hi all,
>
> Since I have updated to Karaf 4, sometime when I install a feature or when
> Karaf starts with some specific combination of bootstrap features,
> it happen that Karaf goes either in what it seems a Deadlock (Karaf hang
> indefinitely and the feature is not installed) or a Livelock (same as
> before but with CPU usage that jumps to 100%)
>
> I'm not able to debug exactly and the effect is not always repeatable,
> some time it don't works, some other it works.
>
> Are there any current issue about the feature installation that may be the
> cause of this behavior?
>
> thank you for the help!
>
> Cristiano
>
>
> P.S. I define my features in a file with the namespace,
> http://karaf.apache.org/xmlns/features/v1.4.0 so the I understand that
> "feature resolver checks the service requirements/capabilities of bundles
> for new features (xml schema >= 1.3.0) in order to automatically installs
> the required bundles". What does this mean?
>
>
>