You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@karaf.apache.org by Minto van der Sluis <mi...@xup.nl> on 2013/09/11 12:03:39 UTC
Occasional freeze on startup
Hi Folks,
In my custom karaf based distribution I occasionally experience freezes
during start up. So far I have seen 2 different occurrence:
1) A complete freeze during start up (see log below)
In this situation services like the webconsole and my own CXF based
services are not available. Looking at the log this is not surprising
since these services have not yet started. However I am clueless why
they didn't start.
Maybe this is related to the order of boot features. In my configuration
I have the following configured:
featuresBoot=config,management,http,webconsole,cxf,myfeature
2) Bundles depending on Apache CXF stay in creating phase.
Some of my custom bundles stay in creating phase with the log showing
the following messages:
... | INFO | ... | Bundle tmvplus-proxy is waiting for namespace
handlers [http://cxf.apache.org/blueprint/core]
... | INFO | ... | Bundle tmvplus-remote is waiting for namespace
handlers [http://cxf.apache.org/blueprint/jaxrs,
http://cxf.apache.org/blueprint/jaxws, http://cxf.apache.org/blueprint/core]
When I try to stop these manually they keep hanging in stopping phase.
When done from the SSH console then prompt never returns keeping the SSH
console in a hanging state as well. When stopped from the webconsole
they also hang in the stopping phase. Only after restarting the karaf
container they are fully stopped. After the restart I can start them
again manually without any problems.
Initially I suspected start-level issue, but CXF start <= 40 and my
stuff starts >= 70.
Let me stress again that these freeze ups happen occasionally. Most of
the time the karaf container starts just fine. But sometimes it doesn't.
Unfortunately the logs are in both cases not very informative. Just
information log messages.
Has anyone seen this before? Does anyone know what needs to be done to
get around these issues?
Hope someone knows how I can move forward, since I am sort of stuck.
Regards,
Minto van der Sluis
Log of freeze occurence 1:
=========================
Sep 09, 2013 10:21:47 AM org.apache.karaf.main.SimpleFileLock lock
INFO: locking
2013-09-09 10:21:48,541 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.management.cfg
2013-09-09 10:21:48,567 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.logging.cfg
2013-09-09 10:21:48,575 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.sync.cfg
2013-09-09 10:21:48,583 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.jaas.cfg
2013-09-09 10:21:48,590 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.url.mvn.cfg
2013-09-09 10:21:48,598 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.obr.cfg
2013-09-09 10:21:48,605 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.storage.clerezza.cfg
2013-09-09 10:21:48,613 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.repos.cfg
2013-09-09 10:21:48,617 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.clerezza.rdf.jena.tdb.storage.ScalableSingleTdbDatasetTcProvider.cfg
2013-09-09 10:21:48,623 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.auth.cfg
2013-09-09 10:21:48,626 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.proxy.cfg
2013-09-09 10:21:48,632 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.namedqueries.filesystem.cfg
2013-09-09 10:21:48,640 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.log.cfg
2013-09-09 10:21:48,646 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.cxf.osgi.cfg
2013-09-09 10:21:48,664 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.felix.fileinstall-deploy.cfg
2013-09-09 10:21:48,668 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.cfg
2013-09-09 10:21:48,674 | INFO | FelixStartLevel | fileinstall
| ? ? | 6 -
org.apache.felix.fileinstall - 3.2.6 | Installed
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.shell.cfg
2013-09-09 10:21:48,762 | INFO | FelixStartLevel | BlueprintExtender
| rint.container.BlueprintExtender 138 | 7 -
org.apache.aries.blueprint.core - 1.1.0 | No quiesce support is
available, so blueprint components will not participate in quiesce
operations
2013-09-09 10:21:49,592 | INFO | FelixStartLevel |
BlueprintContainerImpl | container.BlueprintContainerImpl 303
| 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
org.apache.karaf.jaas.modules is waiting for namespace handlers
[http://karaf.apache.org/xmlns/jaas/v1.0.0]
2013-09-09 10:21:49,799 | INFO | FelixStartLevel |
BlueprintContainerImpl | container.BlueprintContainerImpl 344
| 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
org.apache.karaf.management.server is waiting for dependencies
[(objectClass=org.apache.karaf.jaas.config.KeystoreManager)]
2013-09-09 10:21:49,819 | INFO | FelixStartLevel |
BlueprintContainerImpl | container.BlueprintContainerImpl 344
| 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
org.apache.karaf.features.management is waiting for dependencies
[(objectClass=javax.management.MBeanServer),
(objectClass=org.apache.karaf.features.FeaturesService)]
2013-09-09 10:21:49,896 | INFO | rint Extender: 1 |
BlueprintContainerImpl | container.BlueprintContainerImpl 344
| 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
org.apache.karaf.features.management is waiting for dependencies
[(objectClass=org.apache.karaf.features.FeaturesService)]
2013-09-09 10:21:49,964 | INFO | FelixStartLevel | core
| ? ? | 25 -
org.apache.aries.jmx.core - 1.1.1 | Starting JMX OSGi agent
2013-09-09 10:21:50,041 | INFO | FelixStartLevel | core
| ? ? | 25 -
org.apache.aries.jmx.core - 1.1.1 | Registering MBean with ObjectName
[osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf]
for service with service.id [10]
2013-09-09 10:21:50,083 | INFO | JMX OSGi Agent | core
| ? ? | 25 -
org.apache.aries.jmx.core - 1.1.1 | Registering
org.osgi.jmx.service.cm.ConfigurationAdminMBean to MBeanServer
com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
2013-09-09 10:21:50,085 | INFO | JMX OSGi Agent | core
| ? ? | 25 -
org.apache.aries.jmx.core - 1.1.1 | Registering
org.osgi.jmx.framework.PackageStateMBean to MBeanServer
com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
osgi.core:type=packageState,version=1.5,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
2013-09-09 10:21:50,086 | INFO | JMX OSGi Agent | core
| ? ? | 25 -
org.apache.aries.jmx.core - 1.1.1 | Registering
org.osgi.jmx.framework.ServiceStateMBean to MBeanServer
com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
osgi.core:type=serviceState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
2013-09-09 10:21:50,089 | INFO | JMX OSGi Agent | core
| ? ? | 25 -
org.apache.aries.jmx.core - 1.1.1 | Registering
org.osgi.jmx.framework.FrameworkMBean to MBeanServer
com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
osgi.core:type=framework,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
2013-09-09 10:21:50,090 | INFO | JMX OSGi Agent | core
| ? ? | 25 -
org.apache.aries.jmx.core - 1.1.1 | Registering
org.osgi.jmx.framework.BundleStateMBean to MBeanServer
com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
osgi.core:type=bundleState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
2013-09-09 10:21:50,093 | INFO | JMX OSGi Agent | core
| ? ? | 25 -
org.apache.aries.jmx.core - 1.1.1 | Registering
org.osgi.jmx.framework.wiring.BundleWiringStateMBean to MBeanServer
com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
osgi.core:type=wiringState,version=1.1,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
2013-09-09 10:21:50,253 | INFO | rint Extender: 3 |
KarArtifactInstaller | eployer.kar.KarArtifactInstaller 75
| 24 - org.apache.karaf.deployer.kar - 2.3.2 | Karaf archives will be
extracted to /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system
2013-09-09 10:21:50,258 | INFO | rint Extender: 3 |
KarArtifactInstaller | eployer.kar.KarArtifactInstaller 76
| 24 - org.apache.karaf.deployer.kar - 2.3.2 | Timestamps for Karaf
archives will be extracted to
/usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system/.timestamps
2013-09-09 10:21:51,237 | INFO | FelixStartLevel |
BlueprintContainerImpl | container.BlueprintContainerImpl 344
| 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
org.apache.karaf.admin.command is waiting for dependencies
[(objectClass=org.apache.karaf.admin.AdminService)]
2013-09-09 10:21:51,704 | INFO | FelixStartLevel |
BlueprintContainerImpl | container.BlueprintContainerImpl 344
| 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
org.apache.karaf.admin.management is waiting for dependencies
[(objectClass=org.apache.karaf.admin.AdminService)]
Re: Occasional freeze on startup
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Minto,
thanks for the update. Happy to help, anytime ;)
Regards
JB
On 09/20/2013 12:03 PM, Minto van der Sluis wrote:
> JB thanks for you help.
>
> Everything seems to working fine now.
>
> Regards,
>
> Minto
>
> Op 11-9-2013 18:08, Jean-Baptiste Onofré schreef:
>> Yes, it's about the sync or async boot features behaviour.
>>
>> So please with it to false (to be sync).
>>
>> Regards
>> JB
>>
>> On 09/11/2013 04:57 PM, Minto van der Sluis wrote:
>>> Hi JB,
>>>
>>> My etc/config.properties shows:
>>>
>>> org.apache.aries.blueprint.synchronous=true
>>>
>>> My etc/org.apache.karaf.features.cfg show:
>>>
>>> respectStartLvlDuringFeatureStartup=false
>>>
>>> Compared the the standard karaf file I also noticed that in the latter
>>> config file I miss the following entry:
>>>
>>> featuresBootAsynchronous=false
>>>
>>> Compared to what you earlier said this one might also be important if
>>> the default value it 'true'.
>>>
>>> Should I add this as well?
>>>
>>> Regards,
>>>
>>> Minto
>>>
>>> Op 11-9-2013 15:40, Jean-Baptiste Onofré schreef:
>>>> Thanks for the update Minto.
>>>>
>>>> Could you check that blueprint.synchronous=true in
>>>> etc/config.properties and you have respectStartLvl in
>>>> etc/org.apache.karaf.features.cfg ?
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On 09/11/2013 03:35 PM, Minto van der Sluis wrote:
>>>>> Hi JB,
>>>>>
>>>>> Thanks for your response.
>>>>>
>>>>> Sorry, I should have mentioned the Karaf version that I use.
>>>>> Initially I
>>>>> used Karaf 2.3.0. First thing I did when I ran into these issues was
>>>>> upgrading both Karaf and CXF to their latest released versions. For
>>>>> Karaf this is 2.3.2 and for CXF it is 2.7.6. Since then I have NOT
>>>>> noticed the mentioned behaviour on my development machine.
>>>>>
>>>>> Unfortunately the server we intend to use for production has since
>>>>> refused to start up properly due to freeze 1. The attached log in my
>>>>> previous message was from that server.
>>>>>
>>>>> Is there some additional logging that I can turn on to see what is
>>>>> happening under the hood?
>>>>>
>>>>> Regards,
>>>>>
>>>>> Minto
>>>>>
>>>>>
>>>>> Op 11-9-2013 13:21, Jean-Baptiste Onofré schreef:
>>>>>> Hi Minto,
>>>>>>
>>>>>> I guess that you use Karaf 2.2.x or 2.3.0 ?
>>>>>>
>>>>>> Previously, the blueprint containers and boot features were
>>>>>> asynchronous: it means that we didn't have guarantee on the boot
>>>>>> features order and blueprint bundles order.
>>>>>>
>>>>>> In 2.3.1, I changed that:
>>>>>> - in etc/config.properties, you have
>>>>>> org.apache.aries.blueprint.synchronous=true. Aries blueprint now
>>>>>> waits
>>>>>> that a blueprint container is completely started before starting
>>>>>> another one.
>>>>>> - the boot feature are now synchronous. Previously each boot features
>>>>>> installation was in its own thread. So for instance, in your case,
>>>>>> myfeature could start before cxf. Now, it's in the same thread in
>>>>>> order to guarantee the startup order.
>>>>>>
>>>>>> Regards
>>>>>> JB
>>>>>>
>>>>>> On 09/11/2013 12:03 PM, Minto van der Sluis wrote:
>>>>>>> Hi Folks,
>>>>>>>
>>>>>>> In my custom karaf based distribution I occasionally experience
>>>>>>> freezes
>>>>>>> during start up. So far I have seen 2 different occurrence:
>>>>>>>
>>>>>>> 1) A complete freeze during start up (see log below)
>>>>>>>
>>>>>>> In this situation services like the webconsole and my own CXF based
>>>>>>> services are not available. Looking at the log this is not
>>>>>>> surprising
>>>>>>> since these services have not yet started. However I am clueless why
>>>>>>> they didn't start.
>>>>>>>
>>>>>>> Maybe this is related to the order of boot features. In my
>>>>>>> configuration
>>>>>>> I have the following configured:
>>>>>>>
>>>>>>> featuresBoot=config,management,http,webconsole,cxf,myfeature
>>>>>>>
>>>>>>>
>>>>>>> 2) Bundles depending on Apache CXF stay in creating phase.
>>>>>>>
>>>>>>> Some of my custom bundles stay in creating phase with the log
>>>>>>> showing
>>>>>>> the following messages:
>>>>>>>
>>>>>>> ... | INFO | ... | Bundle tmvplus-proxy is waiting for namespace
>>>>>>> handlers [http://cxf.apache.org/blueprint/core]
>>>>>>> ... | INFO | ... | Bundle tmvplus-remote is waiting for namespace
>>>>>>> handlers [http://cxf.apache.org/blueprint/jaxrs,
>>>>>>> http://cxf.apache.org/blueprint/jaxws,
>>>>>>> http://cxf.apache.org/blueprint/core]
>>>>>>>
>>>>>>> When I try to stop these manually they keep hanging in stopping
>>>>>>> phase.
>>>>>>> When done from the SSH console then prompt never returns keeping
>>>>>>> the SSH
>>>>>>> console in a hanging state as well. When stopped from the webconsole
>>>>>>> they also hang in the stopping phase. Only after restarting the
>>>>>>> karaf
>>>>>>> container they are fully stopped. After the restart I can start them
>>>>>>> again manually without any problems.
>>>>>>>
>>>>>>> Initially I suspected start-level issue, but CXF start <= 40 and my
>>>>>>> stuff starts >= 70.
>>>>>>>
>>>>>>>
>>>>>>> Let me stress again that these freeze ups happen occasionally.
>>>>>>> Most of
>>>>>>> the time the karaf container starts just fine. But sometimes it
>>>>>>> doesn't.
>>>>>>>
>>>>>>> Unfortunately the logs are in both cases not very informative. Just
>>>>>>> information log messages.
>>>>>>>
>>>>>>> Has anyone seen this before? Does anyone know what needs to be
>>>>>>> done to
>>>>>>> get around these issues?
>>>>>>>
>>>>>>> Hope someone knows how I can move forward, since I am sort of stuck.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Minto van der Sluis
>>>>>>>
>>>>>>> Log of freeze occurence 1:
>>>>>>> =========================
>>>>>>> Sep 09, 2013 10:21:47 AM org.apache.karaf.main.SimpleFileLock lock
>>>>>>> INFO: locking
>>>>>>> 2013-09-09 10:21:48,541 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.management.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,567 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.logging.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,575 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.sync.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,583 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.jaas.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,590 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.url.mvn.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,598 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.obr.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,605 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.storage.clerezza.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,613 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.repos.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,617 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.clerezza.rdf.jena.tdb.storage.ScalableSingleTdbDatasetTcProvider.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,623 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.auth.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,626 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.proxy.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,632 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.namedqueries.filesystem.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,640 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.log.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,646 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.cxf.osgi.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,664 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.felix.fileinstall-deploy.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,668 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,674 | INFO | FelixStartLevel | fileinstall
>>>>>>> | ? ? | 6 -
>>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.shell.cfg
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:48,762 | INFO | FelixStartLevel |
>>>>>>> BlueprintExtender
>>>>>>> | rint.container.BlueprintExtender 138 | 7 -
>>>>>>> org.apache.aries.blueprint.core - 1.1.0 | No quiesce support is
>>>>>>> available, so blueprint components will not participate in quiesce
>>>>>>> operations
>>>>>>> 2013-09-09 10:21:49,592 | INFO | FelixStartLevel |
>>>>>>> BlueprintContainerImpl |
>>>>>>> container.BlueprintContainerImpl 303
>>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>>> org.apache.karaf.jaas.modules is waiting for namespace handlers
>>>>>>> [http://karaf.apache.org/xmlns/jaas/v1.0.0]
>>>>>>> 2013-09-09 10:21:49,799 | INFO | FelixStartLevel |
>>>>>>> BlueprintContainerImpl |
>>>>>>> container.BlueprintContainerImpl 344
>>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>>> org.apache.karaf.management.server is waiting for dependencies
>>>>>>> [(objectClass=org.apache.karaf.jaas.config.KeystoreManager)]
>>>>>>> 2013-09-09 10:21:49,819 | INFO | FelixStartLevel |
>>>>>>> BlueprintContainerImpl |
>>>>>>> container.BlueprintContainerImpl 344
>>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>>> org.apache.karaf.features.management is waiting for dependencies
>>>>>>> [(objectClass=javax.management.MBeanServer),
>>>>>>> (objectClass=org.apache.karaf.features.FeaturesService)]
>>>>>>> 2013-09-09 10:21:49,896 | INFO | rint Extender: 1 |
>>>>>>> BlueprintContainerImpl |
>>>>>>> container.BlueprintContainerImpl 344
>>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>>> org.apache.karaf.features.management is waiting for dependencies
>>>>>>> [(objectClass=org.apache.karaf.features.FeaturesService)]
>>>>>>> 2013-09-09 10:21:49,964 | INFO | FelixStartLevel | core
>>>>>>> | ? ? | 25 -
>>>>>>> org.apache.aries.jmx.core - 1.1.1 | Starting JMX OSGi agent
>>>>>>> 2013-09-09 10:21:50,041 | INFO | FelixStartLevel | core
>>>>>>> | ? ? | 25 -
>>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering MBean with
>>>>>>> ObjectName
>>>>>>> [osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> for service with service.id [10]
>>>>>>> 2013-09-09 10:21:50,083 | INFO | JMX OSGi Agent | core
>>>>>>> | ? ? | 25 -
>>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>>> org.osgi.jmx.service.cm.ConfigurationAdminMBean to MBeanServer
>>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>>> osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:50,085 | INFO | JMX OSGi Agent | core
>>>>>>> | ? ? | 25 -
>>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>>> org.osgi.jmx.framework.PackageStateMBean to MBeanServer
>>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>>> osgi.core:type=packageState,version=1.5,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:50,086 | INFO | JMX OSGi Agent | core
>>>>>>> | ? ? | 25 -
>>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>>> org.osgi.jmx.framework.ServiceStateMBean to MBeanServer
>>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>>> osgi.core:type=serviceState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:50,089 | INFO | JMX OSGi Agent | core
>>>>>>> | ? ? | 25 -
>>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>>> org.osgi.jmx.framework.FrameworkMBean to MBeanServer
>>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>>> osgi.core:type=framework,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:50,090 | INFO | JMX OSGi Agent | core
>>>>>>> | ? ? | 25 -
>>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>>> org.osgi.jmx.framework.BundleStateMBean to MBeanServer
>>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>>> osgi.core:type=bundleState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:50,093 | INFO | JMX OSGi Agent | core
>>>>>>> | ? ? | 25 -
>>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>>> org.osgi.jmx.framework.wiring.BundleWiringStateMBean to MBeanServer
>>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>>> osgi.core:type=wiringState,version=1.1,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:50,253 | INFO | rint Extender: 3 |
>>>>>>> KarArtifactInstaller |
>>>>>>> eployer.kar.KarArtifactInstaller 75
>>>>>>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Karaf archives
>>>>>>> will be
>>>>>>> extracted to
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system
>>>>>>> 2013-09-09 10:21:50,258 | INFO | rint Extender: 3 |
>>>>>>> KarArtifactInstaller |
>>>>>>> eployer.kar.KarArtifactInstaller 76
>>>>>>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Timestamps for Karaf
>>>>>>> archives will be extracted to
>>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system/.timestamps
>>>>>>>
>>>>>>>
>>>>>>> 2013-09-09 10:21:51,237 | INFO | FelixStartLevel |
>>>>>>> BlueprintContainerImpl |
>>>>>>> container.BlueprintContainerImpl 344
>>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>>> org.apache.karaf.admin.command is waiting for dependencies
>>>>>>> [(objectClass=org.apache.karaf.admin.AdminService)]
>>>>>>> 2013-09-09 10:21:51,704 | INFO | FelixStartLevel |
>>>>>>> BlueprintContainerImpl |
>>>>>>> container.BlueprintContainerImpl 344
>>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>>> org.apache.karaf.admin.management is waiting for dependencies
>>>>>>> [(objectClass=org.apache.karaf.admin.AdminService)]
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Occasional freeze on startup
Posted by Minto van der Sluis <mi...@xup.nl>.
JB thanks for you help.
Everything seems to working fine now.
Regards,
Minto
Op 11-9-2013 18:08, Jean-Baptiste Onofré schreef:
> Yes, it's about the sync or async boot features behaviour.
>
> So please with it to false (to be sync).
>
> Regards
> JB
>
> On 09/11/2013 04:57 PM, Minto van der Sluis wrote:
>> Hi JB,
>>
>> My etc/config.properties shows:
>>
>> org.apache.aries.blueprint.synchronous=true
>>
>> My etc/org.apache.karaf.features.cfg show:
>>
>> respectStartLvlDuringFeatureStartup=false
>>
>> Compared the the standard karaf file I also noticed that in the latter
>> config file I miss the following entry:
>>
>> featuresBootAsynchronous=false
>>
>> Compared to what you earlier said this one might also be important if
>> the default value it 'true'.
>>
>> Should I add this as well?
>>
>> Regards,
>>
>> Minto
>>
>> Op 11-9-2013 15:40, Jean-Baptiste Onofré schreef:
>>> Thanks for the update Minto.
>>>
>>> Could you check that blueprint.synchronous=true in
>>> etc/config.properties and you have respectStartLvl in
>>> etc/org.apache.karaf.features.cfg ?
>>>
>>> Regards
>>> JB
>>>
>>> On 09/11/2013 03:35 PM, Minto van der Sluis wrote:
>>>> Hi JB,
>>>>
>>>> Thanks for your response.
>>>>
>>>> Sorry, I should have mentioned the Karaf version that I use.
>>>> Initially I
>>>> used Karaf 2.3.0. First thing I did when I ran into these issues was
>>>> upgrading both Karaf and CXF to their latest released versions. For
>>>> Karaf this is 2.3.2 and for CXF it is 2.7.6. Since then I have NOT
>>>> noticed the mentioned behaviour on my development machine.
>>>>
>>>> Unfortunately the server we intend to use for production has since
>>>> refused to start up properly due to freeze 1. The attached log in my
>>>> previous message was from that server.
>>>>
>>>> Is there some additional logging that I can turn on to see what is
>>>> happening under the hood?
>>>>
>>>> Regards,
>>>>
>>>> Minto
>>>>
>>>>
>>>> Op 11-9-2013 13:21, Jean-Baptiste Onofré schreef:
>>>>> Hi Minto,
>>>>>
>>>>> I guess that you use Karaf 2.2.x or 2.3.0 ?
>>>>>
>>>>> Previously, the blueprint containers and boot features were
>>>>> asynchronous: it means that we didn't have guarantee on the boot
>>>>> features order and blueprint bundles order.
>>>>>
>>>>> In 2.3.1, I changed that:
>>>>> - in etc/config.properties, you have
>>>>> org.apache.aries.blueprint.synchronous=true. Aries blueprint now
>>>>> waits
>>>>> that a blueprint container is completely started before starting
>>>>> another one.
>>>>> - the boot feature are now synchronous. Previously each boot features
>>>>> installation was in its own thread. So for instance, in your case,
>>>>> myfeature could start before cxf. Now, it's in the same thread in
>>>>> order to guarantee the startup order.
>>>>>
>>>>> Regards
>>>>> JB
>>>>>
>>>>> On 09/11/2013 12:03 PM, Minto van der Sluis wrote:
>>>>>> Hi Folks,
>>>>>>
>>>>>> In my custom karaf based distribution I occasionally experience
>>>>>> freezes
>>>>>> during start up. So far I have seen 2 different occurrence:
>>>>>>
>>>>>> 1) A complete freeze during start up (see log below)
>>>>>>
>>>>>> In this situation services like the webconsole and my own CXF based
>>>>>> services are not available. Looking at the log this is not
>>>>>> surprising
>>>>>> since these services have not yet started. However I am clueless why
>>>>>> they didn't start.
>>>>>>
>>>>>> Maybe this is related to the order of boot features. In my
>>>>>> configuration
>>>>>> I have the following configured:
>>>>>>
>>>>>> featuresBoot=config,management,http,webconsole,cxf,myfeature
>>>>>>
>>>>>>
>>>>>> 2) Bundles depending on Apache CXF stay in creating phase.
>>>>>>
>>>>>> Some of my custom bundles stay in creating phase with the log
>>>>>> showing
>>>>>> the following messages:
>>>>>>
>>>>>> ... | INFO | ... | Bundle tmvplus-proxy is waiting for namespace
>>>>>> handlers [http://cxf.apache.org/blueprint/core]
>>>>>> ... | INFO | ... | Bundle tmvplus-remote is waiting for namespace
>>>>>> handlers [http://cxf.apache.org/blueprint/jaxrs,
>>>>>> http://cxf.apache.org/blueprint/jaxws,
>>>>>> http://cxf.apache.org/blueprint/core]
>>>>>>
>>>>>> When I try to stop these manually they keep hanging in stopping
>>>>>> phase.
>>>>>> When done from the SSH console then prompt never returns keeping
>>>>>> the SSH
>>>>>> console in a hanging state as well. When stopped from the webconsole
>>>>>> they also hang in the stopping phase. Only after restarting the
>>>>>> karaf
>>>>>> container they are fully stopped. After the restart I can start them
>>>>>> again manually without any problems.
>>>>>>
>>>>>> Initially I suspected start-level issue, but CXF start <= 40 and my
>>>>>> stuff starts >= 70.
>>>>>>
>>>>>>
>>>>>> Let me stress again that these freeze ups happen occasionally.
>>>>>> Most of
>>>>>> the time the karaf container starts just fine. But sometimes it
>>>>>> doesn't.
>>>>>>
>>>>>> Unfortunately the logs are in both cases not very informative. Just
>>>>>> information log messages.
>>>>>>
>>>>>> Has anyone seen this before? Does anyone know what needs to be
>>>>>> done to
>>>>>> get around these issues?
>>>>>>
>>>>>> Hope someone knows how I can move forward, since I am sort of stuck.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Minto van der Sluis
>>>>>>
>>>>>> Log of freeze occurence 1:
>>>>>> =========================
>>>>>> Sep 09, 2013 10:21:47 AM org.apache.karaf.main.SimpleFileLock lock
>>>>>> INFO: locking
>>>>>> 2013-09-09 10:21:48,541 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.management.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,567 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.logging.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,575 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.sync.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,583 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.jaas.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,590 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.url.mvn.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,598 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.obr.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,605 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.storage.clerezza.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,613 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.repos.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,617 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.clerezza.rdf.jena.tdb.storage.ScalableSingleTdbDatasetTcProvider.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,623 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.auth.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,626 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.proxy.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,632 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.namedqueries.filesystem.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,640 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.log.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,646 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.cxf.osgi.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,664 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.felix.fileinstall-deploy.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,668 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,674 | INFO | FelixStartLevel | fileinstall
>>>>>> | ? ? | 6 -
>>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.shell.cfg
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:48,762 | INFO | FelixStartLevel |
>>>>>> BlueprintExtender
>>>>>> | rint.container.BlueprintExtender 138 | 7 -
>>>>>> org.apache.aries.blueprint.core - 1.1.0 | No quiesce support is
>>>>>> available, so blueprint components will not participate in quiesce
>>>>>> operations
>>>>>> 2013-09-09 10:21:49,592 | INFO | FelixStartLevel |
>>>>>> BlueprintContainerImpl |
>>>>>> container.BlueprintContainerImpl 303
>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>> org.apache.karaf.jaas.modules is waiting for namespace handlers
>>>>>> [http://karaf.apache.org/xmlns/jaas/v1.0.0]
>>>>>> 2013-09-09 10:21:49,799 | INFO | FelixStartLevel |
>>>>>> BlueprintContainerImpl |
>>>>>> container.BlueprintContainerImpl 344
>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>> org.apache.karaf.management.server is waiting for dependencies
>>>>>> [(objectClass=org.apache.karaf.jaas.config.KeystoreManager)]
>>>>>> 2013-09-09 10:21:49,819 | INFO | FelixStartLevel |
>>>>>> BlueprintContainerImpl |
>>>>>> container.BlueprintContainerImpl 344
>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>> org.apache.karaf.features.management is waiting for dependencies
>>>>>> [(objectClass=javax.management.MBeanServer),
>>>>>> (objectClass=org.apache.karaf.features.FeaturesService)]
>>>>>> 2013-09-09 10:21:49,896 | INFO | rint Extender: 1 |
>>>>>> BlueprintContainerImpl |
>>>>>> container.BlueprintContainerImpl 344
>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>> org.apache.karaf.features.management is waiting for dependencies
>>>>>> [(objectClass=org.apache.karaf.features.FeaturesService)]
>>>>>> 2013-09-09 10:21:49,964 | INFO | FelixStartLevel | core
>>>>>> | ? ? | 25 -
>>>>>> org.apache.aries.jmx.core - 1.1.1 | Starting JMX OSGi agent
>>>>>> 2013-09-09 10:21:50,041 | INFO | FelixStartLevel | core
>>>>>> | ? ? | 25 -
>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering MBean with
>>>>>> ObjectName
>>>>>> [osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf]
>>>>>>
>>>>>>
>>>>>>
>>>>>> for service with service.id [10]
>>>>>> 2013-09-09 10:21:50,083 | INFO | JMX OSGi Agent | core
>>>>>> | ? ? | 25 -
>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>> org.osgi.jmx.service.cm.ConfigurationAdminMBean to MBeanServer
>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>> osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:50,085 | INFO | JMX OSGi Agent | core
>>>>>> | ? ? | 25 -
>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>> org.osgi.jmx.framework.PackageStateMBean to MBeanServer
>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>> osgi.core:type=packageState,version=1.5,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:50,086 | INFO | JMX OSGi Agent | core
>>>>>> | ? ? | 25 -
>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>> org.osgi.jmx.framework.ServiceStateMBean to MBeanServer
>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>> osgi.core:type=serviceState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:50,089 | INFO | JMX OSGi Agent | core
>>>>>> | ? ? | 25 -
>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>> org.osgi.jmx.framework.FrameworkMBean to MBeanServer
>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>> osgi.core:type=framework,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:50,090 | INFO | JMX OSGi Agent | core
>>>>>> | ? ? | 25 -
>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>> org.osgi.jmx.framework.BundleStateMBean to MBeanServer
>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>> osgi.core:type=bundleState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:50,093 | INFO | JMX OSGi Agent | core
>>>>>> | ? ? | 25 -
>>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>>> org.osgi.jmx.framework.wiring.BundleWiringStateMBean to MBeanServer
>>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>>> osgi.core:type=wiringState,version=1.1,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:50,253 | INFO | rint Extender: 3 |
>>>>>> KarArtifactInstaller |
>>>>>> eployer.kar.KarArtifactInstaller 75
>>>>>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Karaf archives
>>>>>> will be
>>>>>> extracted to
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system
>>>>>> 2013-09-09 10:21:50,258 | INFO | rint Extender: 3 |
>>>>>> KarArtifactInstaller |
>>>>>> eployer.kar.KarArtifactInstaller 76
>>>>>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Timestamps for Karaf
>>>>>> archives will be extracted to
>>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system/.timestamps
>>>>>>
>>>>>>
>>>>>> 2013-09-09 10:21:51,237 | INFO | FelixStartLevel |
>>>>>> BlueprintContainerImpl |
>>>>>> container.BlueprintContainerImpl 344
>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>> org.apache.karaf.admin.command is waiting for dependencies
>>>>>> [(objectClass=org.apache.karaf.admin.AdminService)]
>>>>>> 2013-09-09 10:21:51,704 | INFO | FelixStartLevel |
>>>>>> BlueprintContainerImpl |
>>>>>> container.BlueprintContainerImpl 344
>>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>>> org.apache.karaf.admin.management is waiting for dependencies
>>>>>> [(objectClass=org.apache.karaf.admin.AdminService)]
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>
--
ir. ing. Minto van der Sluis
Software innovator / renovator
Xup BV
Mobiel: +31 (0) 626 014541
Re: Occasional freeze on startup
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Yes, it's about the sync or async boot features behaviour.
So please with it to false (to be sync).
Regards
JB
On 09/11/2013 04:57 PM, Minto van der Sluis wrote:
> Hi JB,
>
> My etc/config.properties shows:
>
> org.apache.aries.blueprint.synchronous=true
>
> My etc/org.apache.karaf.features.cfg show:
>
> respectStartLvlDuringFeatureStartup=false
>
> Compared the the standard karaf file I also noticed that in the latter
> config file I miss the following entry:
>
> featuresBootAsynchronous=false
>
> Compared to what you earlier said this one might also be important if
> the default value it 'true'.
>
> Should I add this as well?
>
> Regards,
>
> Minto
>
> Op 11-9-2013 15:40, Jean-Baptiste Onofré schreef:
>> Thanks for the update Minto.
>>
>> Could you check that blueprint.synchronous=true in
>> etc/config.properties and you have respectStartLvl in
>> etc/org.apache.karaf.features.cfg ?
>>
>> Regards
>> JB
>>
>> On 09/11/2013 03:35 PM, Minto van der Sluis wrote:
>>> Hi JB,
>>>
>>> Thanks for your response.
>>>
>>> Sorry, I should have mentioned the Karaf version that I use. Initially I
>>> used Karaf 2.3.0. First thing I did when I ran into these issues was
>>> upgrading both Karaf and CXF to their latest released versions. For
>>> Karaf this is 2.3.2 and for CXF it is 2.7.6. Since then I have NOT
>>> noticed the mentioned behaviour on my development machine.
>>>
>>> Unfortunately the server we intend to use for production has since
>>> refused to start up properly due to freeze 1. The attached log in my
>>> previous message was from that server.
>>>
>>> Is there some additional logging that I can turn on to see what is
>>> happening under the hood?
>>>
>>> Regards,
>>>
>>> Minto
>>>
>>>
>>> Op 11-9-2013 13:21, Jean-Baptiste Onofré schreef:
>>>> Hi Minto,
>>>>
>>>> I guess that you use Karaf 2.2.x or 2.3.0 ?
>>>>
>>>> Previously, the blueprint containers and boot features were
>>>> asynchronous: it means that we didn't have guarantee on the boot
>>>> features order and blueprint bundles order.
>>>>
>>>> In 2.3.1, I changed that:
>>>> - in etc/config.properties, you have
>>>> org.apache.aries.blueprint.synchronous=true. Aries blueprint now waits
>>>> that a blueprint container is completely started before starting
>>>> another one.
>>>> - the boot feature are now synchronous. Previously each boot features
>>>> installation was in its own thread. So for instance, in your case,
>>>> myfeature could start before cxf. Now, it's in the same thread in
>>>> order to guarantee the startup order.
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On 09/11/2013 12:03 PM, Minto van der Sluis wrote:
>>>>> Hi Folks,
>>>>>
>>>>> In my custom karaf based distribution I occasionally experience
>>>>> freezes
>>>>> during start up. So far I have seen 2 different occurrence:
>>>>>
>>>>> 1) A complete freeze during start up (see log below)
>>>>>
>>>>> In this situation services like the webconsole and my own CXF based
>>>>> services are not available. Looking at the log this is not surprising
>>>>> since these services have not yet started. However I am clueless why
>>>>> they didn't start.
>>>>>
>>>>> Maybe this is related to the order of boot features. In my
>>>>> configuration
>>>>> I have the following configured:
>>>>>
>>>>> featuresBoot=config,management,http,webconsole,cxf,myfeature
>>>>>
>>>>>
>>>>> 2) Bundles depending on Apache CXF stay in creating phase.
>>>>>
>>>>> Some of my custom bundles stay in creating phase with the log showing
>>>>> the following messages:
>>>>>
>>>>> ... | INFO | ... | Bundle tmvplus-proxy is waiting for namespace
>>>>> handlers [http://cxf.apache.org/blueprint/core]
>>>>> ... | INFO | ... | Bundle tmvplus-remote is waiting for namespace
>>>>> handlers [http://cxf.apache.org/blueprint/jaxrs,
>>>>> http://cxf.apache.org/blueprint/jaxws,
>>>>> http://cxf.apache.org/blueprint/core]
>>>>>
>>>>> When I try to stop these manually they keep hanging in stopping phase.
>>>>> When done from the SSH console then prompt never returns keeping
>>>>> the SSH
>>>>> console in a hanging state as well. When stopped from the webconsole
>>>>> they also hang in the stopping phase. Only after restarting the karaf
>>>>> container they are fully stopped. After the restart I can start them
>>>>> again manually without any problems.
>>>>>
>>>>> Initially I suspected start-level issue, but CXF start <= 40 and my
>>>>> stuff starts >= 70.
>>>>>
>>>>>
>>>>> Let me stress again that these freeze ups happen occasionally. Most of
>>>>> the time the karaf container starts just fine. But sometimes it
>>>>> doesn't.
>>>>>
>>>>> Unfortunately the logs are in both cases not very informative. Just
>>>>> information log messages.
>>>>>
>>>>> Has anyone seen this before? Does anyone know what needs to be done to
>>>>> get around these issues?
>>>>>
>>>>> Hope someone knows how I can move forward, since I am sort of stuck.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Minto van der Sluis
>>>>>
>>>>> Log of freeze occurence 1:
>>>>> =========================
>>>>> Sep 09, 2013 10:21:47 AM org.apache.karaf.main.SimpleFileLock lock
>>>>> INFO: locking
>>>>> 2013-09-09 10:21:48,541 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.management.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,567 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.logging.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,575 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.sync.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,583 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.jaas.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,590 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.url.mvn.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,598 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.obr.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,605 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.storage.clerezza.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,613 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.repos.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,617 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.clerezza.rdf.jena.tdb.storage.ScalableSingleTdbDatasetTcProvider.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,623 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.auth.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,626 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.proxy.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,632 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.namedqueries.filesystem.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,640 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.log.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,646 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.cxf.osgi.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,664 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.felix.fileinstall-deploy.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,668 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,674 | INFO | FelixStartLevel | fileinstall
>>>>> | ? ? | 6 -
>>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.shell.cfg
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:48,762 | INFO | FelixStartLevel | BlueprintExtender
>>>>> | rint.container.BlueprintExtender 138 | 7 -
>>>>> org.apache.aries.blueprint.core - 1.1.0 | No quiesce support is
>>>>> available, so blueprint components will not participate in quiesce
>>>>> operations
>>>>> 2013-09-09 10:21:49,592 | INFO | FelixStartLevel |
>>>>> BlueprintContainerImpl |
>>>>> container.BlueprintContainerImpl 303
>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>> org.apache.karaf.jaas.modules is waiting for namespace handlers
>>>>> [http://karaf.apache.org/xmlns/jaas/v1.0.0]
>>>>> 2013-09-09 10:21:49,799 | INFO | FelixStartLevel |
>>>>> BlueprintContainerImpl |
>>>>> container.BlueprintContainerImpl 344
>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>> org.apache.karaf.management.server is waiting for dependencies
>>>>> [(objectClass=org.apache.karaf.jaas.config.KeystoreManager)]
>>>>> 2013-09-09 10:21:49,819 | INFO | FelixStartLevel |
>>>>> BlueprintContainerImpl |
>>>>> container.BlueprintContainerImpl 344
>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>> org.apache.karaf.features.management is waiting for dependencies
>>>>> [(objectClass=javax.management.MBeanServer),
>>>>> (objectClass=org.apache.karaf.features.FeaturesService)]
>>>>> 2013-09-09 10:21:49,896 | INFO | rint Extender: 1 |
>>>>> BlueprintContainerImpl |
>>>>> container.BlueprintContainerImpl 344
>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>> org.apache.karaf.features.management is waiting for dependencies
>>>>> [(objectClass=org.apache.karaf.features.FeaturesService)]
>>>>> 2013-09-09 10:21:49,964 | INFO | FelixStartLevel | core
>>>>> | ? ? | 25 -
>>>>> org.apache.aries.jmx.core - 1.1.1 | Starting JMX OSGi agent
>>>>> 2013-09-09 10:21:50,041 | INFO | FelixStartLevel | core
>>>>> | ? ? | 25 -
>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering MBean with ObjectName
>>>>> [osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf]
>>>>>
>>>>>
>>>>> for service with service.id [10]
>>>>> 2013-09-09 10:21:50,083 | INFO | JMX OSGi Agent | core
>>>>> | ? ? | 25 -
>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>> org.osgi.jmx.service.cm.ConfigurationAdminMBean to MBeanServer
>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>> osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:50,085 | INFO | JMX OSGi Agent | core
>>>>> | ? ? | 25 -
>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>> org.osgi.jmx.framework.PackageStateMBean to MBeanServer
>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>> osgi.core:type=packageState,version=1.5,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:50,086 | INFO | JMX OSGi Agent | core
>>>>> | ? ? | 25 -
>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>> org.osgi.jmx.framework.ServiceStateMBean to MBeanServer
>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>> osgi.core:type=serviceState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:50,089 | INFO | JMX OSGi Agent | core
>>>>> | ? ? | 25 -
>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>> org.osgi.jmx.framework.FrameworkMBean to MBeanServer
>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>> osgi.core:type=framework,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:50,090 | INFO | JMX OSGi Agent | core
>>>>> | ? ? | 25 -
>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>> org.osgi.jmx.framework.BundleStateMBean to MBeanServer
>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>> osgi.core:type=bundleState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:50,093 | INFO | JMX OSGi Agent | core
>>>>> | ? ? | 25 -
>>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>>> org.osgi.jmx.framework.wiring.BundleWiringStateMBean to MBeanServer
>>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>>> osgi.core:type=wiringState,version=1.1,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>>
>>>>>
>>>>> 2013-09-09 10:21:50,253 | INFO | rint Extender: 3 |
>>>>> KarArtifactInstaller |
>>>>> eployer.kar.KarArtifactInstaller 75
>>>>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Karaf archives will be
>>>>> extracted to
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system
>>>>> 2013-09-09 10:21:50,258 | INFO | rint Extender: 3 |
>>>>> KarArtifactInstaller |
>>>>> eployer.kar.KarArtifactInstaller 76
>>>>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Timestamps for Karaf
>>>>> archives will be extracted to
>>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system/.timestamps
>>>>>
>>>>> 2013-09-09 10:21:51,237 | INFO | FelixStartLevel |
>>>>> BlueprintContainerImpl |
>>>>> container.BlueprintContainerImpl 344
>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>> org.apache.karaf.admin.command is waiting for dependencies
>>>>> [(objectClass=org.apache.karaf.admin.AdminService)]
>>>>> 2013-09-09 10:21:51,704 | INFO | FelixStartLevel |
>>>>> BlueprintContainerImpl |
>>>>> container.BlueprintContainerImpl 344
>>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>>> org.apache.karaf.admin.management is waiting for dependencies
>>>>> [(objectClass=org.apache.karaf.admin.AdminService)]
>>>>>
>>>>
>>>
>>>
>>
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Occasional freeze on startup
Posted by Minto van der Sluis <mi...@xup.nl>.
Hi JB,
My etc/config.properties shows:
org.apache.aries.blueprint.synchronous=true
My etc/org.apache.karaf.features.cfg show:
respectStartLvlDuringFeatureStartup=false
Compared the the standard karaf file I also noticed that in the latter
config file I miss the following entry:
featuresBootAsynchronous=false
Compared to what you earlier said this one might also be important if
the default value it 'true'.
Should I add this as well?
Regards,
Minto
Op 11-9-2013 15:40, Jean-Baptiste Onofré schreef:
> Thanks for the update Minto.
>
> Could you check that blueprint.synchronous=true in
> etc/config.properties and you have respectStartLvl in
> etc/org.apache.karaf.features.cfg ?
>
> Regards
> JB
>
> On 09/11/2013 03:35 PM, Minto van der Sluis wrote:
>> Hi JB,
>>
>> Thanks for your response.
>>
>> Sorry, I should have mentioned the Karaf version that I use. Initially I
>> used Karaf 2.3.0. First thing I did when I ran into these issues was
>> upgrading both Karaf and CXF to their latest released versions. For
>> Karaf this is 2.3.2 and for CXF it is 2.7.6. Since then I have NOT
>> noticed the mentioned behaviour on my development machine.
>>
>> Unfortunately the server we intend to use for production has since
>> refused to start up properly due to freeze 1. The attached log in my
>> previous message was from that server.
>>
>> Is there some additional logging that I can turn on to see what is
>> happening under the hood?
>>
>> Regards,
>>
>> Minto
>>
>>
>> Op 11-9-2013 13:21, Jean-Baptiste Onofré schreef:
>>> Hi Minto,
>>>
>>> I guess that you use Karaf 2.2.x or 2.3.0 ?
>>>
>>> Previously, the blueprint containers and boot features were
>>> asynchronous: it means that we didn't have guarantee on the boot
>>> features order and blueprint bundles order.
>>>
>>> In 2.3.1, I changed that:
>>> - in etc/config.properties, you have
>>> org.apache.aries.blueprint.synchronous=true. Aries blueprint now waits
>>> that a blueprint container is completely started before starting
>>> another one.
>>> - the boot feature are now synchronous. Previously each boot features
>>> installation was in its own thread. So for instance, in your case,
>>> myfeature could start before cxf. Now, it's in the same thread in
>>> order to guarantee the startup order.
>>>
>>> Regards
>>> JB
>>>
>>> On 09/11/2013 12:03 PM, Minto van der Sluis wrote:
>>>> Hi Folks,
>>>>
>>>> In my custom karaf based distribution I occasionally experience
>>>> freezes
>>>> during start up. So far I have seen 2 different occurrence:
>>>>
>>>> 1) A complete freeze during start up (see log below)
>>>>
>>>> In this situation services like the webconsole and my own CXF based
>>>> services are not available. Looking at the log this is not surprising
>>>> since these services have not yet started. However I am clueless why
>>>> they didn't start.
>>>>
>>>> Maybe this is related to the order of boot features. In my
>>>> configuration
>>>> I have the following configured:
>>>>
>>>> featuresBoot=config,management,http,webconsole,cxf,myfeature
>>>>
>>>>
>>>> 2) Bundles depending on Apache CXF stay in creating phase.
>>>>
>>>> Some of my custom bundles stay in creating phase with the log showing
>>>> the following messages:
>>>>
>>>> ... | INFO | ... | Bundle tmvplus-proxy is waiting for namespace
>>>> handlers [http://cxf.apache.org/blueprint/core]
>>>> ... | INFO | ... | Bundle tmvplus-remote is waiting for namespace
>>>> handlers [http://cxf.apache.org/blueprint/jaxrs,
>>>> http://cxf.apache.org/blueprint/jaxws,
>>>> http://cxf.apache.org/blueprint/core]
>>>>
>>>> When I try to stop these manually they keep hanging in stopping phase.
>>>> When done from the SSH console then prompt never returns keeping
>>>> the SSH
>>>> console in a hanging state as well. When stopped from the webconsole
>>>> they also hang in the stopping phase. Only after restarting the karaf
>>>> container they are fully stopped. After the restart I can start them
>>>> again manually without any problems.
>>>>
>>>> Initially I suspected start-level issue, but CXF start <= 40 and my
>>>> stuff starts >= 70.
>>>>
>>>>
>>>> Let me stress again that these freeze ups happen occasionally. Most of
>>>> the time the karaf container starts just fine. But sometimes it
>>>> doesn't.
>>>>
>>>> Unfortunately the logs are in both cases not very informative. Just
>>>> information log messages.
>>>>
>>>> Has anyone seen this before? Does anyone know what needs to be done to
>>>> get around these issues?
>>>>
>>>> Hope someone knows how I can move forward, since I am sort of stuck.
>>>>
>>>> Regards,
>>>>
>>>> Minto van der Sluis
>>>>
>>>> Log of freeze occurence 1:
>>>> =========================
>>>> Sep 09, 2013 10:21:47 AM org.apache.karaf.main.SimpleFileLock lock
>>>> INFO: locking
>>>> 2013-09-09 10:21:48,541 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.management.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,567 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.logging.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,575 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.sync.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,583 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.jaas.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,590 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.url.mvn.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,598 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.obr.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,605 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.storage.clerezza.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,613 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.repos.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,617 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.clerezza.rdf.jena.tdb.storage.ScalableSingleTdbDatasetTcProvider.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,623 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.auth.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,626 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.proxy.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,632 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.namedqueries.filesystem.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,640 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.log.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,646 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.cxf.osgi.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,664 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.felix.fileinstall-deploy.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,668 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,674 | INFO | FelixStartLevel | fileinstall
>>>> | ? ? | 6 -
>>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.shell.cfg
>>>>
>>>>
>>>> 2013-09-09 10:21:48,762 | INFO | FelixStartLevel | BlueprintExtender
>>>> | rint.container.BlueprintExtender 138 | 7 -
>>>> org.apache.aries.blueprint.core - 1.1.0 | No quiesce support is
>>>> available, so blueprint components will not participate in quiesce
>>>> operations
>>>> 2013-09-09 10:21:49,592 | INFO | FelixStartLevel |
>>>> BlueprintContainerImpl |
>>>> container.BlueprintContainerImpl 303
>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>> org.apache.karaf.jaas.modules is waiting for namespace handlers
>>>> [http://karaf.apache.org/xmlns/jaas/v1.0.0]
>>>> 2013-09-09 10:21:49,799 | INFO | FelixStartLevel |
>>>> BlueprintContainerImpl |
>>>> container.BlueprintContainerImpl 344
>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>> org.apache.karaf.management.server is waiting for dependencies
>>>> [(objectClass=org.apache.karaf.jaas.config.KeystoreManager)]
>>>> 2013-09-09 10:21:49,819 | INFO | FelixStartLevel |
>>>> BlueprintContainerImpl |
>>>> container.BlueprintContainerImpl 344
>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>> org.apache.karaf.features.management is waiting for dependencies
>>>> [(objectClass=javax.management.MBeanServer),
>>>> (objectClass=org.apache.karaf.features.FeaturesService)]
>>>> 2013-09-09 10:21:49,896 | INFO | rint Extender: 1 |
>>>> BlueprintContainerImpl |
>>>> container.BlueprintContainerImpl 344
>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>> org.apache.karaf.features.management is waiting for dependencies
>>>> [(objectClass=org.apache.karaf.features.FeaturesService)]
>>>> 2013-09-09 10:21:49,964 | INFO | FelixStartLevel | core
>>>> | ? ? | 25 -
>>>> org.apache.aries.jmx.core - 1.1.1 | Starting JMX OSGi agent
>>>> 2013-09-09 10:21:50,041 | INFO | FelixStartLevel | core
>>>> | ? ? | 25 -
>>>> org.apache.aries.jmx.core - 1.1.1 | Registering MBean with ObjectName
>>>> [osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf]
>>>>
>>>>
>>>> for service with service.id [10]
>>>> 2013-09-09 10:21:50,083 | INFO | JMX OSGi Agent | core
>>>> | ? ? | 25 -
>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>> org.osgi.jmx.service.cm.ConfigurationAdminMBean to MBeanServer
>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>> osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>
>>>>
>>>> 2013-09-09 10:21:50,085 | INFO | JMX OSGi Agent | core
>>>> | ? ? | 25 -
>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>> org.osgi.jmx.framework.PackageStateMBean to MBeanServer
>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>> osgi.core:type=packageState,version=1.5,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>
>>>>
>>>> 2013-09-09 10:21:50,086 | INFO | JMX OSGi Agent | core
>>>> | ? ? | 25 -
>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>> org.osgi.jmx.framework.ServiceStateMBean to MBeanServer
>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>> osgi.core:type=serviceState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>
>>>>
>>>> 2013-09-09 10:21:50,089 | INFO | JMX OSGi Agent | core
>>>> | ? ? | 25 -
>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>> org.osgi.jmx.framework.FrameworkMBean to MBeanServer
>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>> osgi.core:type=framework,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>
>>>>
>>>> 2013-09-09 10:21:50,090 | INFO | JMX OSGi Agent | core
>>>> | ? ? | 25 -
>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>> org.osgi.jmx.framework.BundleStateMBean to MBeanServer
>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>> osgi.core:type=bundleState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>
>>>>
>>>> 2013-09-09 10:21:50,093 | INFO | JMX OSGi Agent | core
>>>> | ? ? | 25 -
>>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>>> org.osgi.jmx.framework.wiring.BundleWiringStateMBean to MBeanServer
>>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>>> osgi.core:type=wiringState,version=1.1,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>>
>>>>
>>>> 2013-09-09 10:21:50,253 | INFO | rint Extender: 3 |
>>>> KarArtifactInstaller |
>>>> eployer.kar.KarArtifactInstaller 75
>>>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Karaf archives will be
>>>> extracted to
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system
>>>> 2013-09-09 10:21:50,258 | INFO | rint Extender: 3 |
>>>> KarArtifactInstaller |
>>>> eployer.kar.KarArtifactInstaller 76
>>>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Timestamps for Karaf
>>>> archives will be extracted to
>>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system/.timestamps
>>>>
>>>> 2013-09-09 10:21:51,237 | INFO | FelixStartLevel |
>>>> BlueprintContainerImpl |
>>>> container.BlueprintContainerImpl 344
>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>> org.apache.karaf.admin.command is waiting for dependencies
>>>> [(objectClass=org.apache.karaf.admin.AdminService)]
>>>> 2013-09-09 10:21:51,704 | INFO | FelixStartLevel |
>>>> BlueprintContainerImpl |
>>>> container.BlueprintContainerImpl 344
>>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>>> org.apache.karaf.admin.management is waiting for dependencies
>>>> [(objectClass=org.apache.karaf.admin.AdminService)]
>>>>
>>>
>>
>>
>
--
ir. ing. Minto van der Sluis
Software innovator / renovator
Xup BV
Mobiel: +31 (0) 626 014541
Re: Occasional freeze on startup
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks for the update Minto.
Could you check that blueprint.synchronous=true in etc/config.properties
and you have respectStartLvl in etc/org.apache.karaf.features.cfg ?
Regards
JB
On 09/11/2013 03:35 PM, Minto van der Sluis wrote:
> Hi JB,
>
> Thanks for your response.
>
> Sorry, I should have mentioned the Karaf version that I use. Initially I
> used Karaf 2.3.0. First thing I did when I ran into these issues was
> upgrading both Karaf and CXF to their latest released versions. For
> Karaf this is 2.3.2 and for CXF it is 2.7.6. Since then I have NOT
> noticed the mentioned behaviour on my development machine.
>
> Unfortunately the server we intend to use for production has since
> refused to start up properly due to freeze 1. The attached log in my
> previous message was from that server.
>
> Is there some additional logging that I can turn on to see what is
> happening under the hood?
>
> Regards,
>
> Minto
>
>
> Op 11-9-2013 13:21, Jean-Baptiste Onofré schreef:
>> Hi Minto,
>>
>> I guess that you use Karaf 2.2.x or 2.3.0 ?
>>
>> Previously, the blueprint containers and boot features were
>> asynchronous: it means that we didn't have guarantee on the boot
>> features order and blueprint bundles order.
>>
>> In 2.3.1, I changed that:
>> - in etc/config.properties, you have
>> org.apache.aries.blueprint.synchronous=true. Aries blueprint now waits
>> that a blueprint container is completely started before starting
>> another one.
>> - the boot feature are now synchronous. Previously each boot features
>> installation was in its own thread. So for instance, in your case,
>> myfeature could start before cxf. Now, it's in the same thread in
>> order to guarantee the startup order.
>>
>> Regards
>> JB
>>
>> On 09/11/2013 12:03 PM, Minto van der Sluis wrote:
>>> Hi Folks,
>>>
>>> In my custom karaf based distribution I occasionally experience freezes
>>> during start up. So far I have seen 2 different occurrence:
>>>
>>> 1) A complete freeze during start up (see log below)
>>>
>>> In this situation services like the webconsole and my own CXF based
>>> services are not available. Looking at the log this is not surprising
>>> since these services have not yet started. However I am clueless why
>>> they didn't start.
>>>
>>> Maybe this is related to the order of boot features. In my configuration
>>> I have the following configured:
>>>
>>> featuresBoot=config,management,http,webconsole,cxf,myfeature
>>>
>>>
>>> 2) Bundles depending on Apache CXF stay in creating phase.
>>>
>>> Some of my custom bundles stay in creating phase with the log showing
>>> the following messages:
>>>
>>> ... | INFO | ... | Bundle tmvplus-proxy is waiting for namespace
>>> handlers [http://cxf.apache.org/blueprint/core]
>>> ... | INFO | ... | Bundle tmvplus-remote is waiting for namespace
>>> handlers [http://cxf.apache.org/blueprint/jaxrs,
>>> http://cxf.apache.org/blueprint/jaxws,
>>> http://cxf.apache.org/blueprint/core]
>>>
>>> When I try to stop these manually they keep hanging in stopping phase.
>>> When done from the SSH console then prompt never returns keeping the SSH
>>> console in a hanging state as well. When stopped from the webconsole
>>> they also hang in the stopping phase. Only after restarting the karaf
>>> container they are fully stopped. After the restart I can start them
>>> again manually without any problems.
>>>
>>> Initially I suspected start-level issue, but CXF start <= 40 and my
>>> stuff starts >= 70.
>>>
>>>
>>> Let me stress again that these freeze ups happen occasionally. Most of
>>> the time the karaf container starts just fine. But sometimes it doesn't.
>>>
>>> Unfortunately the logs are in both cases not very informative. Just
>>> information log messages.
>>>
>>> Has anyone seen this before? Does anyone know what needs to be done to
>>> get around these issues?
>>>
>>> Hope someone knows how I can move forward, since I am sort of stuck.
>>>
>>> Regards,
>>>
>>> Minto van der Sluis
>>>
>>> Log of freeze occurence 1:
>>> =========================
>>> Sep 09, 2013 10:21:47 AM org.apache.karaf.main.SimpleFileLock lock
>>> INFO: locking
>>> 2013-09-09 10:21:48,541 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.management.cfg
>>>
>>> 2013-09-09 10:21:48,567 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.logging.cfg
>>>
>>> 2013-09-09 10:21:48,575 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.sync.cfg
>>>
>>> 2013-09-09 10:21:48,583 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.jaas.cfg
>>>
>>> 2013-09-09 10:21:48,590 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.url.mvn.cfg
>>>
>>> 2013-09-09 10:21:48,598 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.obr.cfg
>>>
>>> 2013-09-09 10:21:48,605 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.storage.clerezza.cfg
>>>
>>> 2013-09-09 10:21:48,613 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.repos.cfg
>>>
>>> 2013-09-09 10:21:48,617 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.clerezza.rdf.jena.tdb.storage.ScalableSingleTdbDatasetTcProvider.cfg
>>>
>>> 2013-09-09 10:21:48,623 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.auth.cfg
>>>
>>> 2013-09-09 10:21:48,626 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.proxy.cfg
>>>
>>> 2013-09-09 10:21:48,632 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.namedqueries.filesystem.cfg
>>>
>>> 2013-09-09 10:21:48,640 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.log.cfg
>>>
>>> 2013-09-09 10:21:48,646 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.cxf.osgi.cfg
>>>
>>> 2013-09-09 10:21:48,664 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.felix.fileinstall-deploy.cfg
>>>
>>> 2013-09-09 10:21:48,668 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.cfg
>>>
>>> 2013-09-09 10:21:48,674 | INFO | FelixStartLevel | fileinstall
>>> | ? ? | 6 -
>>> org.apache.felix.fileinstall - 3.2.6 | Installed
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.shell.cfg
>>>
>>> 2013-09-09 10:21:48,762 | INFO | FelixStartLevel | BlueprintExtender
>>> | rint.container.BlueprintExtender 138 | 7 -
>>> org.apache.aries.blueprint.core - 1.1.0 | No quiesce support is
>>> available, so blueprint components will not participate in quiesce
>>> operations
>>> 2013-09-09 10:21:49,592 | INFO | FelixStartLevel |
>>> BlueprintContainerImpl | container.BlueprintContainerImpl 303
>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>> org.apache.karaf.jaas.modules is waiting for namespace handlers
>>> [http://karaf.apache.org/xmlns/jaas/v1.0.0]
>>> 2013-09-09 10:21:49,799 | INFO | FelixStartLevel |
>>> BlueprintContainerImpl | container.BlueprintContainerImpl 344
>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>> org.apache.karaf.management.server is waiting for dependencies
>>> [(objectClass=org.apache.karaf.jaas.config.KeystoreManager)]
>>> 2013-09-09 10:21:49,819 | INFO | FelixStartLevel |
>>> BlueprintContainerImpl | container.BlueprintContainerImpl 344
>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>> org.apache.karaf.features.management is waiting for dependencies
>>> [(objectClass=javax.management.MBeanServer),
>>> (objectClass=org.apache.karaf.features.FeaturesService)]
>>> 2013-09-09 10:21:49,896 | INFO | rint Extender: 1 |
>>> BlueprintContainerImpl | container.BlueprintContainerImpl 344
>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>> org.apache.karaf.features.management is waiting for dependencies
>>> [(objectClass=org.apache.karaf.features.FeaturesService)]
>>> 2013-09-09 10:21:49,964 | INFO | FelixStartLevel | core
>>> | ? ? | 25 -
>>> org.apache.aries.jmx.core - 1.1.1 | Starting JMX OSGi agent
>>> 2013-09-09 10:21:50,041 | INFO | FelixStartLevel | core
>>> | ? ? | 25 -
>>> org.apache.aries.jmx.core - 1.1.1 | Registering MBean with ObjectName
>>> [osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf]
>>>
>>> for service with service.id [10]
>>> 2013-09-09 10:21:50,083 | INFO | JMX OSGi Agent | core
>>> | ? ? | 25 -
>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>> org.osgi.jmx.service.cm.ConfigurationAdminMBean to MBeanServer
>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>> osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>
>>> 2013-09-09 10:21:50,085 | INFO | JMX OSGi Agent | core
>>> | ? ? | 25 -
>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>> org.osgi.jmx.framework.PackageStateMBean to MBeanServer
>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>> osgi.core:type=packageState,version=1.5,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>
>>> 2013-09-09 10:21:50,086 | INFO | JMX OSGi Agent | core
>>> | ? ? | 25 -
>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>> org.osgi.jmx.framework.ServiceStateMBean to MBeanServer
>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>> osgi.core:type=serviceState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>
>>> 2013-09-09 10:21:50,089 | INFO | JMX OSGi Agent | core
>>> | ? ? | 25 -
>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>> org.osgi.jmx.framework.FrameworkMBean to MBeanServer
>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>> osgi.core:type=framework,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>
>>> 2013-09-09 10:21:50,090 | INFO | JMX OSGi Agent | core
>>> | ? ? | 25 -
>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>> org.osgi.jmx.framework.BundleStateMBean to MBeanServer
>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>> osgi.core:type=bundleState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>
>>> 2013-09-09 10:21:50,093 | INFO | JMX OSGi Agent | core
>>> | ? ? | 25 -
>>> org.apache.aries.jmx.core - 1.1.1 | Registering
>>> org.osgi.jmx.framework.wiring.BundleWiringStateMBean to MBeanServer
>>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>>> osgi.core:type=wiringState,version=1.1,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>>
>>> 2013-09-09 10:21:50,253 | INFO | rint Extender: 3 |
>>> KarArtifactInstaller | eployer.kar.KarArtifactInstaller 75
>>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Karaf archives will be
>>> extracted to
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system
>>> 2013-09-09 10:21:50,258 | INFO | rint Extender: 3 |
>>> KarArtifactInstaller | eployer.kar.KarArtifactInstaller 76
>>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Timestamps for Karaf
>>> archives will be extracted to
>>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system/.timestamps
>>> 2013-09-09 10:21:51,237 | INFO | FelixStartLevel |
>>> BlueprintContainerImpl | container.BlueprintContainerImpl 344
>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>> org.apache.karaf.admin.command is waiting for dependencies
>>> [(objectClass=org.apache.karaf.admin.AdminService)]
>>> 2013-09-09 10:21:51,704 | INFO | FelixStartLevel |
>>> BlueprintContainerImpl | container.BlueprintContainerImpl 344
>>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>>> org.apache.karaf.admin.management is waiting for dependencies
>>> [(objectClass=org.apache.karaf.admin.AdminService)]
>>>
>>
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Occasional freeze on startup
Posted by Minto van der Sluis <mi...@xup.nl>.
Hi JB,
Thanks for your response.
Sorry, I should have mentioned the Karaf version that I use. Initially I
used Karaf 2.3.0. First thing I did when I ran into these issues was
upgrading both Karaf and CXF to their latest released versions. For
Karaf this is 2.3.2 and for CXF it is 2.7.6. Since then I have NOT
noticed the mentioned behaviour on my development machine.
Unfortunately the server we intend to use for production has since
refused to start up properly due to freeze 1. The attached log in my
previous message was from that server.
Is there some additional logging that I can turn on to see what is
happening under the hood?
Regards,
Minto
Op 11-9-2013 13:21, Jean-Baptiste Onofré schreef:
> Hi Minto,
>
> I guess that you use Karaf 2.2.x or 2.3.0 ?
>
> Previously, the blueprint containers and boot features were
> asynchronous: it means that we didn't have guarantee on the boot
> features order and blueprint bundles order.
>
> In 2.3.1, I changed that:
> - in etc/config.properties, you have
> org.apache.aries.blueprint.synchronous=true. Aries blueprint now waits
> that a blueprint container is completely started before starting
> another one.
> - the boot feature are now synchronous. Previously each boot features
> installation was in its own thread. So for instance, in your case,
> myfeature could start before cxf. Now, it's in the same thread in
> order to guarantee the startup order.
>
> Regards
> JB
>
> On 09/11/2013 12:03 PM, Minto van der Sluis wrote:
>> Hi Folks,
>>
>> In my custom karaf based distribution I occasionally experience freezes
>> during start up. So far I have seen 2 different occurrence:
>>
>> 1) A complete freeze during start up (see log below)
>>
>> In this situation services like the webconsole and my own CXF based
>> services are not available. Looking at the log this is not surprising
>> since these services have not yet started. However I am clueless why
>> they didn't start.
>>
>> Maybe this is related to the order of boot features. In my configuration
>> I have the following configured:
>>
>> featuresBoot=config,management,http,webconsole,cxf,myfeature
>>
>>
>> 2) Bundles depending on Apache CXF stay in creating phase.
>>
>> Some of my custom bundles stay in creating phase with the log showing
>> the following messages:
>>
>> ... | INFO | ... | Bundle tmvplus-proxy is waiting for namespace
>> handlers [http://cxf.apache.org/blueprint/core]
>> ... | INFO | ... | Bundle tmvplus-remote is waiting for namespace
>> handlers [http://cxf.apache.org/blueprint/jaxrs,
>> http://cxf.apache.org/blueprint/jaxws,
>> http://cxf.apache.org/blueprint/core]
>>
>> When I try to stop these manually they keep hanging in stopping phase.
>> When done from the SSH console then prompt never returns keeping the SSH
>> console in a hanging state as well. When stopped from the webconsole
>> they also hang in the stopping phase. Only after restarting the karaf
>> container they are fully stopped. After the restart I can start them
>> again manually without any problems.
>>
>> Initially I suspected start-level issue, but CXF start <= 40 and my
>> stuff starts >= 70.
>>
>>
>> Let me stress again that these freeze ups happen occasionally. Most of
>> the time the karaf container starts just fine. But sometimes it doesn't.
>>
>> Unfortunately the logs are in both cases not very informative. Just
>> information log messages.
>>
>> Has anyone seen this before? Does anyone know what needs to be done to
>> get around these issues?
>>
>> Hope someone knows how I can move forward, since I am sort of stuck.
>>
>> Regards,
>>
>> Minto van der Sluis
>>
>> Log of freeze occurence 1:
>> =========================
>> Sep 09, 2013 10:21:47 AM org.apache.karaf.main.SimpleFileLock lock
>> INFO: locking
>> 2013-09-09 10:21:48,541 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.management.cfg
>>
>> 2013-09-09 10:21:48,567 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.logging.cfg
>>
>> 2013-09-09 10:21:48,575 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.sync.cfg
>>
>> 2013-09-09 10:21:48,583 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.jaas.cfg
>>
>> 2013-09-09 10:21:48,590 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.url.mvn.cfg
>>
>> 2013-09-09 10:21:48,598 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.obr.cfg
>>
>> 2013-09-09 10:21:48,605 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.storage.clerezza.cfg
>>
>> 2013-09-09 10:21:48,613 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.repos.cfg
>>
>> 2013-09-09 10:21:48,617 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.clerezza.rdf.jena.tdb.storage.ScalableSingleTdbDatasetTcProvider.cfg
>>
>> 2013-09-09 10:21:48,623 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.auth.cfg
>>
>> 2013-09-09 10:21:48,626 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.proxy.cfg
>>
>> 2013-09-09 10:21:48,632 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.namedqueries.filesystem.cfg
>>
>> 2013-09-09 10:21:48,640 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.log.cfg
>>
>> 2013-09-09 10:21:48,646 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.cxf.osgi.cfg
>>
>> 2013-09-09 10:21:48,664 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.felix.fileinstall-deploy.cfg
>>
>> 2013-09-09 10:21:48,668 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.cfg
>>
>> 2013-09-09 10:21:48,674 | INFO | FelixStartLevel | fileinstall
>> | ? ? | 6 -
>> org.apache.felix.fileinstall - 3.2.6 | Installed
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.shell.cfg
>>
>> 2013-09-09 10:21:48,762 | INFO | FelixStartLevel | BlueprintExtender
>> | rint.container.BlueprintExtender 138 | 7 -
>> org.apache.aries.blueprint.core - 1.1.0 | No quiesce support is
>> available, so blueprint components will not participate in quiesce
>> operations
>> 2013-09-09 10:21:49,592 | INFO | FelixStartLevel |
>> BlueprintContainerImpl | container.BlueprintContainerImpl 303
>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>> org.apache.karaf.jaas.modules is waiting for namespace handlers
>> [http://karaf.apache.org/xmlns/jaas/v1.0.0]
>> 2013-09-09 10:21:49,799 | INFO | FelixStartLevel |
>> BlueprintContainerImpl | container.BlueprintContainerImpl 344
>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>> org.apache.karaf.management.server is waiting for dependencies
>> [(objectClass=org.apache.karaf.jaas.config.KeystoreManager)]
>> 2013-09-09 10:21:49,819 | INFO | FelixStartLevel |
>> BlueprintContainerImpl | container.BlueprintContainerImpl 344
>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>> org.apache.karaf.features.management is waiting for dependencies
>> [(objectClass=javax.management.MBeanServer),
>> (objectClass=org.apache.karaf.features.FeaturesService)]
>> 2013-09-09 10:21:49,896 | INFO | rint Extender: 1 |
>> BlueprintContainerImpl | container.BlueprintContainerImpl 344
>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>> org.apache.karaf.features.management is waiting for dependencies
>> [(objectClass=org.apache.karaf.features.FeaturesService)]
>> 2013-09-09 10:21:49,964 | INFO | FelixStartLevel | core
>> | ? ? | 25 -
>> org.apache.aries.jmx.core - 1.1.1 | Starting JMX OSGi agent
>> 2013-09-09 10:21:50,041 | INFO | FelixStartLevel | core
>> | ? ? | 25 -
>> org.apache.aries.jmx.core - 1.1.1 | Registering MBean with ObjectName
>> [osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf]
>>
>> for service with service.id [10]
>> 2013-09-09 10:21:50,083 | INFO | JMX OSGi Agent | core
>> | ? ? | 25 -
>> org.apache.aries.jmx.core - 1.1.1 | Registering
>> org.osgi.jmx.service.cm.ConfigurationAdminMBean to MBeanServer
>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>> osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>
>> 2013-09-09 10:21:50,085 | INFO | JMX OSGi Agent | core
>> | ? ? | 25 -
>> org.apache.aries.jmx.core - 1.1.1 | Registering
>> org.osgi.jmx.framework.PackageStateMBean to MBeanServer
>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>> osgi.core:type=packageState,version=1.5,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>
>> 2013-09-09 10:21:50,086 | INFO | JMX OSGi Agent | core
>> | ? ? | 25 -
>> org.apache.aries.jmx.core - 1.1.1 | Registering
>> org.osgi.jmx.framework.ServiceStateMBean to MBeanServer
>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>> osgi.core:type=serviceState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>
>> 2013-09-09 10:21:50,089 | INFO | JMX OSGi Agent | core
>> | ? ? | 25 -
>> org.apache.aries.jmx.core - 1.1.1 | Registering
>> org.osgi.jmx.framework.FrameworkMBean to MBeanServer
>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>> osgi.core:type=framework,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>
>> 2013-09-09 10:21:50,090 | INFO | JMX OSGi Agent | core
>> | ? ? | 25 -
>> org.apache.aries.jmx.core - 1.1.1 | Registering
>> org.osgi.jmx.framework.BundleStateMBean to MBeanServer
>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>> osgi.core:type=bundleState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>
>> 2013-09-09 10:21:50,093 | INFO | JMX OSGi Agent | core
>> | ? ? | 25 -
>> org.apache.aries.jmx.core - 1.1.1 | Registering
>> org.osgi.jmx.framework.wiring.BundleWiringStateMBean to MBeanServer
>> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
>> osgi.core:type=wiringState,version=1.1,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
>>
>> 2013-09-09 10:21:50,253 | INFO | rint Extender: 3 |
>> KarArtifactInstaller | eployer.kar.KarArtifactInstaller 75
>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Karaf archives will be
>> extracted to
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system
>> 2013-09-09 10:21:50,258 | INFO | rint Extender: 3 |
>> KarArtifactInstaller | eployer.kar.KarArtifactInstaller 76
>> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Timestamps for Karaf
>> archives will be extracted to
>> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system/.timestamps
>> 2013-09-09 10:21:51,237 | INFO | FelixStartLevel |
>> BlueprintContainerImpl | container.BlueprintContainerImpl 344
>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>> org.apache.karaf.admin.command is waiting for dependencies
>> [(objectClass=org.apache.karaf.admin.AdminService)]
>> 2013-09-09 10:21:51,704 | INFO | FelixStartLevel |
>> BlueprintContainerImpl | container.BlueprintContainerImpl 344
>> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
>> org.apache.karaf.admin.management is waiting for dependencies
>> [(objectClass=org.apache.karaf.admin.AdminService)]
>>
>
--
ir. ing. Minto van der Sluis
Software innovator / renovator
Xup BV
Mobiel: +31 (0) 626 014541
Re: Occasional freeze on startup
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Minto,
I guess that you use Karaf 2.2.x or 2.3.0 ?
Previously, the blueprint containers and boot features were
asynchronous: it means that we didn't have guarantee on the boot
features order and blueprint bundles order.
In 2.3.1, I changed that:
- in etc/config.properties, you have
org.apache.aries.blueprint.synchronous=true. Aries blueprint now waits
that a blueprint container is completely started before starting another
one.
- the boot feature are now synchronous. Previously each boot features
installation was in its own thread. So for instance, in your case,
myfeature could start before cxf. Now, it's in the same thread in order
to guarantee the startup order.
Regards
JB
On 09/11/2013 12:03 PM, Minto van der Sluis wrote:
> Hi Folks,
>
> In my custom karaf based distribution I occasionally experience freezes
> during start up. So far I have seen 2 different occurrence:
>
> 1) A complete freeze during start up (see log below)
>
> In this situation services like the webconsole and my own CXF based
> services are not available. Looking at the log this is not surprising
> since these services have not yet started. However I am clueless why
> they didn't start.
>
> Maybe this is related to the order of boot features. In my configuration
> I have the following configured:
>
> featuresBoot=config,management,http,webconsole,cxf,myfeature
>
>
> 2) Bundles depending on Apache CXF stay in creating phase.
>
> Some of my custom bundles stay in creating phase with the log showing
> the following messages:
>
> ... | INFO | ... | Bundle tmvplus-proxy is waiting for namespace
> handlers [http://cxf.apache.org/blueprint/core]
> ... | INFO | ... | Bundle tmvplus-remote is waiting for namespace
> handlers [http://cxf.apache.org/blueprint/jaxrs,
> http://cxf.apache.org/blueprint/jaxws, http://cxf.apache.org/blueprint/core]
>
> When I try to stop these manually they keep hanging in stopping phase.
> When done from the SSH console then prompt never returns keeping the SSH
> console in a hanging state as well. When stopped from the webconsole
> they also hang in the stopping phase. Only after restarting the karaf
> container they are fully stopped. After the restart I can start them
> again manually without any problems.
>
> Initially I suspected start-level issue, but CXF start <= 40 and my
> stuff starts >= 70.
>
>
> Let me stress again that these freeze ups happen occasionally. Most of
> the time the karaf container starts just fine. But sometimes it doesn't.
>
> Unfortunately the logs are in both cases not very informative. Just
> information log messages.
>
> Has anyone seen this before? Does anyone know what needs to be done to
> get around these issues?
>
> Hope someone knows how I can move forward, since I am sort of stuck.
>
> Regards,
>
> Minto van der Sluis
>
> Log of freeze occurence 1:
> =========================
> Sep 09, 2013 10:21:47 AM org.apache.karaf.main.SimpleFileLock lock
> INFO: locking
> 2013-09-09 10:21:48,541 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.management.cfg
> 2013-09-09 10:21:48,567 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.logging.cfg
> 2013-09-09 10:21:48,575 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.sync.cfg
> 2013-09-09 10:21:48,583 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.jaas.cfg
> 2013-09-09 10:21:48,590 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.ops4j.pax.url.mvn.cfg
> 2013-09-09 10:21:48,598 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.obr.cfg
> 2013-09-09 10:21:48,605 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.storage.clerezza.cfg
> 2013-09-09 10:21:48,613 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.repos.cfg
> 2013-09-09 10:21:48,617 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.clerezza.rdf.jena.tdb.storage.ScalableSingleTdbDatasetTcProvider.cfg
> 2013-09-09 10:21:48,623 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.auth.cfg
> 2013-09-09 10:21:48,626 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/tmvplus.proxy.cfg
> 2013-09-09 10:21:48,632 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/astore.namedqueries.filesystem.cfg
> 2013-09-09 10:21:48,640 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.log.cfg
> 2013-09-09 10:21:48,646 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.cxf.osgi.cfg
> 2013-09-09 10:21:48,664 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.felix.fileinstall-deploy.cfg
> 2013-09-09 10:21:48,668 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.features.cfg
> 2013-09-09 10:21:48,674 | INFO | FelixStartLevel | fileinstall
> | ? ? | 6 -
> org.apache.felix.fileinstall - 3.2.6 | Installed
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/etc/org.apache.karaf.shell.cfg
> 2013-09-09 10:21:48,762 | INFO | FelixStartLevel | BlueprintExtender
> | rint.container.BlueprintExtender 138 | 7 -
> org.apache.aries.blueprint.core - 1.1.0 | No quiesce support is
> available, so blueprint components will not participate in quiesce
> operations
> 2013-09-09 10:21:49,592 | INFO | FelixStartLevel |
> BlueprintContainerImpl | container.BlueprintContainerImpl 303
> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
> org.apache.karaf.jaas.modules is waiting for namespace handlers
> [http://karaf.apache.org/xmlns/jaas/v1.0.0]
> 2013-09-09 10:21:49,799 | INFO | FelixStartLevel |
> BlueprintContainerImpl | container.BlueprintContainerImpl 344
> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
> org.apache.karaf.management.server is waiting for dependencies
> [(objectClass=org.apache.karaf.jaas.config.KeystoreManager)]
> 2013-09-09 10:21:49,819 | INFO | FelixStartLevel |
> BlueprintContainerImpl | container.BlueprintContainerImpl 344
> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
> org.apache.karaf.features.management is waiting for dependencies
> [(objectClass=javax.management.MBeanServer),
> (objectClass=org.apache.karaf.features.FeaturesService)]
> 2013-09-09 10:21:49,896 | INFO | rint Extender: 1 |
> BlueprintContainerImpl | container.BlueprintContainerImpl 344
> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
> org.apache.karaf.features.management is waiting for dependencies
> [(objectClass=org.apache.karaf.features.FeaturesService)]
> 2013-09-09 10:21:49,964 | INFO | FelixStartLevel | core
> | ? ? | 25 -
> org.apache.aries.jmx.core - 1.1.1 | Starting JMX OSGi agent
> 2013-09-09 10:21:50,041 | INFO | FelixStartLevel | core
> | ? ? | 25 -
> org.apache.aries.jmx.core - 1.1.1 | Registering MBean with ObjectName
> [osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf]
> for service with service.id [10]
> 2013-09-09 10:21:50,083 | INFO | JMX OSGi Agent | core
> | ? ? | 25 -
> org.apache.aries.jmx.core - 1.1.1 | Registering
> org.osgi.jmx.service.cm.ConfigurationAdminMBean to MBeanServer
> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
> osgi.compendium:service=cm,version=1.3,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
> 2013-09-09 10:21:50,085 | INFO | JMX OSGi Agent | core
> | ? ? | 25 -
> org.apache.aries.jmx.core - 1.1.1 | Registering
> org.osgi.jmx.framework.PackageStateMBean to MBeanServer
> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
> osgi.core:type=packageState,version=1.5,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
> 2013-09-09 10:21:50,086 | INFO | JMX OSGi Agent | core
> | ? ? | 25 -
> org.apache.aries.jmx.core - 1.1.1 | Registering
> org.osgi.jmx.framework.ServiceStateMBean to MBeanServer
> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
> osgi.core:type=serviceState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
> 2013-09-09 10:21:50,089 | INFO | JMX OSGi Agent | core
> | ? ? | 25 -
> org.apache.aries.jmx.core - 1.1.1 | Registering
> org.osgi.jmx.framework.FrameworkMBean to MBeanServer
> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
> osgi.core:type=framework,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
> 2013-09-09 10:21:50,090 | INFO | JMX OSGi Agent | core
> | ? ? | 25 -
> org.apache.aries.jmx.core - 1.1.1 | Registering
> org.osgi.jmx.framework.BundleStateMBean to MBeanServer
> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
> osgi.core:type=bundleState,version=1.7,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
> 2013-09-09 10:21:50,093 | INFO | JMX OSGi Agent | core
> | ? ? | 25 -
> org.apache.aries.jmx.core - 1.1.1 | Registering
> org.osgi.jmx.framework.wiring.BundleWiringStateMBean to MBeanServer
> com.sun.jmx.mbeanserver.JmxMBeanServer@9eae470 with name
> osgi.core:type=wiringState,version=1.1,framework=org.apache.felix.framework,uuid=ae51c7ab-e8d1-4ffa-bcce-cff4f3ac2bcf
> 2013-09-09 10:21:50,253 | INFO | rint Extender: 3 |
> KarArtifactInstaller | eployer.kar.KarArtifactInstaller 75
> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Karaf archives will be
> extracted to /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system
> 2013-09-09 10:21:50,258 | INFO | rint Extender: 3 |
> KarArtifactInstaller | eployer.kar.KarArtifactInstaller 76
> | 24 - org.apache.karaf.deployer.kar - 2.3.2 | Timestamps for Karaf
> archives will be extracted to
> /usr/local/spg/versions/tmvplus-distribution-0.6.6.53/system/.timestamps
> 2013-09-09 10:21:51,237 | INFO | FelixStartLevel |
> BlueprintContainerImpl | container.BlueprintContainerImpl 344
> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
> org.apache.karaf.admin.command is waiting for dependencies
> [(objectClass=org.apache.karaf.admin.AdminService)]
> 2013-09-09 10:21:51,704 | INFO | FelixStartLevel |
> BlueprintContainerImpl | container.BlueprintContainerImpl 344
> | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle
> org.apache.karaf.admin.management is waiting for dependencies
> [(objectClass=org.apache.karaf.admin.AdminService)]
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com