You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by "Aled Sage (JIRA)" <ji...@apache.org> on 2016/08/02 21:28:20 UTC

[jira] [Created] (BROOKLYN-320) Tomcat8Server entity fails to start: using nss 3.15 on CentOS 6.5

Aled Sage created BROOKLYN-320:
----------------------------------

             Summary: Tomcat8Server entity fails to start: using nss 3.15 on CentOS 6.5
                 Key: BROOKLYN-320
                 URL: https://issues.apache.org/jira/browse/BROOKLYN-320
             Project: Brooklyn
          Issue Type: Bug
         Environment: CentOS 6.5, with nss 3.15, in aws-ec2 singapore
            Reporter: Aled Sage


Deploying Tomcat8Server failed (when waiting for service-up), using Brooklyn 0.10.0-SNAPSHOT. This was in AWS-singapore, using RightImage_CentOS_6.5_x64_v13.5.2.2_EBS (ami-42bfe910).

TL;DR: the problem is fixed by upgrading nss (manually on the target VM).

Looking on the VM, the file {{/home/users/amp/brooklyn-managed-processes/apps/GfmrLSwE/entities/Tomcat8Server_pEKbtDGD/logs/catalina.out}} had the following contents:

{noformat}
JmxmpAgent active at: service:jmx:jmxmp://ip-10-167-0-52.ap-southeast-1.compute.internal:31001
Aug 02, 2016 5:03:10 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
Exception in thread "main" java.lang.InternalError
        at sun.security.ec.SunEC.initialize(Native Method)
        at sun.security.ec.SunEC.access$000(SunEC.java:49)
        at sun.security.ec.SunEC$1.run(SunEC.java:61)
        at sun.security.ec.SunEC$1.run(SunEC.java:58)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.security.ec.SunEC.<clinit>(SunEC.java:58)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at java.lang.Class.newInstance(Class.java:383)
        at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:221)
        at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:206)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:206)
        at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:187)
        at sun.security.jca.ProviderList.loadAll(ProviderList.java:282)
        at sun.security.jca.ProviderList.removeInvalid(ProviderList.java:299)
        at sun.security.jca.Providers.getFullProviderList(Providers.java:173)
        at java.security.Security.getProviders(Security.java:456)
        at org.apache.catalina.core.JreMemoryLeakPreventionListener.lifecycleEvent(JreMemoryLeakPreventionListener.java:393)
        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
        at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
        at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:402)
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:99)
        at org.apache.catalina.startup.Catalina.load(Catalina.java:576)
        at org.apache.catalina.startup.Catalina.load(Catalina.java:599)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:310)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:484)
{noformat}

The process was still running:

{noformat}
amp       2104  0.2  2.7 1941916 107360 ?      Sl   17:03   0:09 /etc/alternatives/jre/bin/java -Dnop -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -javaagent:/home/users/amp/brooklyn-managed-processes/apps/GfmrLSwE/entities/Tomcat8Server_pEKbtDGD/brooklyn-jmxmp-agent-shaded-0.10.0-20160609.1043.jar -Xms200m -Xmx800m -XX:MaxPermSize=400m -Dcom.sun.management.jmxremote -Dbrooklyn.jmxmp.rmi-port=1099 -Dbrooklyn.jmxmp.port=31001 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.endorsed.dirs=/home/users/amp/brooklyn-managed-processes/installs/Tomcat8Server_8.0.22/apache-tomcat-8.0.22/endorsed -classpath /home/users/amp/brooklyn-managed-processes/installs/Tomcat8Server_8.0.22/apache-tomcat-8.0.22/bin/bootstrap.jar:/home/users/amp/brooklyn-managed-processes/installs/Tomcat8Server_8.0.22/apache-tomcat-8.0.22/bin/tomcat-juli.jar -Dcatalina.base=/home/users/amp/brooklyn-managed-processes/apps/GfmrLSwE/entities/Tomcat8Server_pEKbtDGD -Dcatalina.home=/home/users/amp/brooklyn-managed-processes/installs/Tomcat8Server_8.0.22/apache-tomcat-8.0.22 -Djava.io.tmpdir=/home/users/amp/brooklyn-managed-processes/apps/GfmrLSwE/entities/Tomcat8Server_pEKbtDGD/temp org.apache.catalina.startup.Bootstrap start
{noformat}

The output of {{netstat -antp | grep 2104}} is shown below:

{noformat}
tcp        0      0 0.0.0.0:1099                0.0.0.0:*                   LISTEN      2104/java           
tcp        0      0 0.0.0.0:31001               0.0.0.0:*                   LISTEN      2104/java           
tcp        0      0 0.0.0.0:59804               0.0.0.0:*                   LISTEN      2104/java           
tcp        0      0 0.0.0.0:56292               0.0.0.0:*                   LISTEN      2104/java           
tcp        0      0 10.167.0.52:31001           161.202.25.230:41069        ESTABLISHED 2104/java           
tcp        0      0 10.167.0.52:31001           161.202.25.230:41068        ESTABLISHED 2104/java           
{noformat}

The output of {{java -version}} is:

{noformat}
java version "1.7.0_111"
OpenJDK Runtime Environment (rhel-2.6.7.2.el6_8-x86_64 u111-b01)
OpenJDK 64-Bit Server VM (build 24.111-b01, mixed mode)
{noformat}

These bug reports look relevant:
* https://bugzilla.redhat.com/show_bug.cgi?id=1332456
* https://bugzilla.redhat.com/show_bug.cgi?id=1332867#c5

Looking at the version of nss, the output of {{yum list installed nss*}} is:

{noformat}
Loaded plugins: security
Error: Error accessing file for config file://///etc/yum.repos.d/CentOS-Base.repo
[amp@ip-10-167-0-52 Tomcat8Server_pEKbtDGD]$ sudo yum list installed nss*
Loaded plugins: security
Repository epel is listed more than once in the configuration
rightscale-epel                                                                                                                                                                                                                                      | 2.9 kB     00:00     
Installed Packages
nss.x86_64                                                                                                                         3.15.1-15.el6                                                                                                           @base/$releasever
nss-softokn.x86_64                                                                                                                 3.14.3-9.el6                                                                                                            @base/$releasever
nss-softokn-freebl.i686                                                                                                            3.14.3-9.el6                                                                                                            @base            
nss-softokn-freebl.x86_64                                                                                                          3.14.3-9.el6                                                                                                            @base/$releasever
nss-sysinit.x86_64                                                                                                                 3.15.1-15.el6                                                                                                           @base/$releasever
nss-tools.x86_64                                                                                                                   3.15.1-15.el6                                                                                                           @base/$releasever
nss-util.x86_64                                                                                                                    3.15.1-3.el6                                                                                                            @base/$releasever
{noformat}


After running {{sudo yum upgrade nss}}, it has versions:

{noformat}
Loaded plugins: security
Repository epel is listed more than once in the configuration
Installed Packages
nss.x86_64                                                                                                                            3.21.0-8.el6                                                                                                                  @base   
nss-softokn.x86_64                                                                                                                    3.14.3-23.3.el6_8                                                                                                             @updates
nss-softokn-freebl.i686                                                                                                               3.14.3-23.3.el6_8                                                                                                             @updates
nss-softokn-freebl.x86_64                                                                                                             3.14.3-23.3.el6_8                                                                                                             @updates
nss-sysinit.x86_64                                                                                                                    3.21.0-8.el6                                                                                                                  @base   
nss-tools.x86_64                                                                                                                      3.21.0-8.el6                                                                                                                  @base   
nss-util.x86_64                                                                                                                       3.21.0-2.el6                                                                                                                  @base   
{noformat}

This fixes the issue - tomcat restarts correctly.

I notice that https://issues.apache.org/jira/browse/BROOKLYN-218 was also caused by a need to upgrade nss, but that time it was on the Brooklyn server rather than the target VM.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)