[jira] [Commented] (SOLR-13712) JMX MBeans are not exposed because of race condition between creating platform mbean server and registering mbeans

["Shalin Shekhar Mangar (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-13712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913947#comment-16913947 ] 

Shalin Shekhar Mangar commented on SOLR-13712:
----------------------------------------------

I tracked down the problem to SolrJmxReporter.

In the absence of any agentId or serviceUrl, the platform mbean server is supposed to be used. The SolrJmxReporter.init (or doInit() method in later versions) first tries to get the first mbean server and registers mbeans to it. However, it exits early if no mbean server is found. Attaching a debugger to the solr processes I found that the platform mbean server is always found for the solr node running on port 8983 but not for solr on port 7574.

The problem is that the platform mbean server is created on the first invocation of {{ManagementFactory.getPlatformMBeanServer()}} and SolrJmxReporter calls this method in JmxMetricsReporter.build() which happens after the mbean server is looked up. If no mbean server is found then the call to build() never happens and therefore no mbeans are registered with the mbean server.

So why does the mbean server exist on 8983? It is because of the embedded zookeeper which calls {{ManagementFactory.getPlatformMBeanServer()}} before solr tries to initialize SolrJmxReporter.

The code is virtually unchanged in Solr 7.x and 8.x. Then why do those versions seem fine? It is because of log4j2 which registers its mbeans with the platform mbean server before Solr initializes.

> JMX MBeans are not exposed because of race condition between creating platform mbean server and registering mbeans
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-13712
>                 URL: https://issues.apache.org/jira/browse/SOLR-13712
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: JMX
>    Affects Versions: 6.6, 6.6.2, 6.6.5, 7.7.2, 8.2, 8.1.1
>            Reporter: Shalin Shekhar Mangar
>            Priority: Major
>
> This is quite easy to reproduce. 
> {code}
> wget https://archive.apache.org/dist/lucene/solr/6.6.0/solr-6.6.0.tgz
> tar xvf solr-6.6.0.tgz
> cd solr-6.6.0
> {code}
> Enable jmx reporting by editing the server/solr/solr.xml and adding the following under the "<solr>" tag:
> {code}
> <metrics>
>         <reporter id="jmx" class="org.apache.solr.metrics.reporters.SolrJmxReporter" />
>   </metrics>
> {code}
> Start solr with:
> {code}
> ./bin/solr start -e cloud -noprompt
> {code}
> Open jconsole and inspect mbeans for solr nodes running on port 8983 and 7574. You'll find that all mbeans (node, jvm, jetty, solr) are present for the solr on port 8983 but completely absent for the solr node running on port 7574.
> Same behavior is on 6.6.2 and 6.6.6. However, Solr 7.x and 8.x seem to be fine.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org