[jira] [Created] (BROOKLYN-201) Failure retrieving port sensor value for TomcatServer

["Sam Corbett (JIRA)" <ji...@apache.org>]

Sam Corbett created BROOKLYN-201:
------------------------------------

             Summary: Failure retrieving port sensor value for TomcatServer
                 Key: BROOKLYN-201
                 URL: https://issues.apache.org/jira/browse/BROOKLYN-201
             Project: Brooklyn
          Issue Type: Bug
            Reporter: Sam Corbett


I observed a strange problem in which a Tomcat server appeared to be missing most sensor values.

Context:

Two-member dynamic cluster of Tomcat servers with identical member specs.
Member 1: GZKUwB3x
Member 2: l1LEYWIA

Each member has a mirror entity:
Member #1 mirror: cn7VfAGi
Member #2 mirror: nwVhOYzr  

#1 deployed normally.

#2 failed to start:
{code}
Failed after 4ms: Execution failed, invocation error for JMX_SETUP_PREINSTALL Tomcat8ServerImpl{id=l1LEYWIA}: jmx port must not be null for Tomcat8ServerImpl{id=l1LEYWIA}: java.lang.NullPointerException: jmx port must not be null for Tomcat8ServerImpl{id=l1LEYWIA}

java.lang.IllegalStateException: Execution failed, invocation error for JMX_SETUP_PREINSTALL Tomcat8ServerImpl{id=l1LEYWIA}: jmx port must not be null for Tomcat8ServerImpl{id=l1LEYWIA}
	at org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.logWithDetailsAndThrow(ScriptHelper.java:387)
	at org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.executeInternal(ScriptHelper.java:370)
	at org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper$8.call(ScriptHelper.java:287)
	at org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper$8.call(ScriptHelper.java:285)
	at org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:342)
	at org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:468)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException: jmx port must not be null for Tomcat8ServerImpl{id=l1LEYWIA}
	at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:253)
	at org.apache.brooklyn.entity.java.JmxSupport.applyJmxJavaSystemProperties(JmxSupport.java:275)
	at org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJmxJavaSystemProperties(JavaSoftwareProcessSshDriver.java:258)
	at org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJavaSystemProperties(JavaSoftwareProcessSshDriver.java:205)
	at org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJavaOpts(JavaSoftwareProcessSshDriver.java:125)
	at org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getShellEnvironment(JavaSoftwareProcessSshDriver.java:105)
	at org.apache.brooklyn.entity.webapp.tomcat.TomcatSshDriver.getShellEnvironment(TomcatSshDriver.java:139)
	at org.apache.brooklyn.entity.software.base.AbstractSoftwareProcessSshDriver.execute(AbstractSoftwareProcessSshDriver.java:261)
	at org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.executeInternal(ScriptHelper.java:366)
	... 8 more
{code}

This is the command that failed:
{code}
Integer jmxPort = Preconditions.checkNotNull(entity.getAttribute(JMX_PORT), "jmx port must not be null for %s", entity);
{code}

So for some reason the Tomcat entity with id l1LEYWIA has no jmx.port sensor.

In the debug log, before the exception is thrown:

{code}
2015-12-08 15:23:15,994 DEBUG o.a.b.c.s.PortAttributeSensorAndConfigKey [brooklyn-execmanager-C52UZvVL-147269]: Tomcat8ServerImpl{id=l1LEYWIA}: choosing port 31001 for jmx.direct.port
2015-12-08 15:23:16,116 DEBUG o.a.b.c.entity.AbstractEntity [brooklyn-execmanager-C52UZvVL-147272]: Emitting sensor notification entity.sensor.added value Sensor: jmx.direct.port (java.lang.Integer) on DelegateEntityImpl{id=nwVhOYzr}
2015-12-08 15:23:19,537 DEBUG o.a.b.c.s.PortAttributeSensorAndConfigKey [brooklyn-execmanager-C52UZvVL-147228]: Tomcat8ServerImpl{id=GZKUwB3x}: choosing port 31001 for jmx.direct.port
2015-12-08 15:23:19,537 DEBUG o.a.b.c.entity.AbstractEntity [brooklyn-execmanager-C52UZvVL-147136]: Emitting sensor notification entity.sensor.added value Sensor: jmx.direct.port (java.lang.Integer) on DelegateEntityImpl{id=cn7VfAGi}
{code}

But the attribute _had_ been set on the entity.

To make things more confusing, refer to the images attached to this issue.

Image 1 shows the senors for the running entity (GZKUwB3x)
Image 2 shows all sensors for the failed entity (l1LEYWIA, including those with no value)
Image 3 shows only the sensors on the failed entity (l1LEYWIA) that have a value set
Image 4 shows only the sensors on the failed entity's delegate (nwVhOYzr) that have a value set.
Image 5 shows the failed task.

3 and 4 are the most interesting. The real entity has no sensors set but its delegate does?! Note that the delegate entity has a value for jmx.direct.port and has none of the Clocker attributes like docker.host or docker.infrastructure.

As far as I can tell from the logs the sensor value for jmx.direct.port was not subequently unset. No sensor notifications were emitted for it after it was first set.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)