You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@solr.apache.org by David Smiley <ds...@apache.org> on 2022/01/09 22:45:55 UTC
Disable metrics reporting to JMX
I noticed Solr auto-creates a metrics SolrJmxReporter if there is a
platform "MBeanServer" that exists, which AFAICT is always. Thanks? Ehh,
no thanks. It's not evident how to disable JMX after some fruitless google
searches. Don't get me wrong, I like jconsole, jvisualvm, JFR etc and I
think some of these things may rely on JMX but I don't particularly need
Solr to expose its metrics to these tools ever since Solr gained pretty
excellent /admin/metrics support that is easier to get at.
I see Solr's code that makes this decision in
SolrXmlConfig.getMetricReporterPluginInfos and I could see that I could
enhance it with a few lines of code to check pluginInfo.isEnabled(). Thus
to disable JMX reporting, one would configure it with the enable="false"
XML attribute. Or maybe we just remove the automatic enablement.
BTW what's driving me to look at this is that there is some time spent
registering and unregistering SolrCore level metrics to JMX when SolrCores
are loaded and unloaded, and logs to this effect likewise. Not a big deal
but it's something.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
Re: Disable metrics reporting to JMX
Posted by David Smiley <ds...@apache.org>.
Thanks for sharing that Matthias; point taken. I know it's useful for some
users, it isn't going away.
I filed a JIRA issue: https://issues.apache.org/jira/browse/SOLR-15905
It's debatable wether's solr.xml should come with it enabled by default or
not. I don't have a strong opinion there.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Mon, Jan 10, 2022 at 10:22 AM Matthias Krueger <mk...@mkr.io> wrote:
> JMX has its issues but we should be aware that it currently provides a
> relatively generic and easy to use integration point and is used by, for
> example, the Datadog Solr integration:
> https://github.com/DataDog/integrations-core/tree/master/solr (and maybe
> others).
> On 10.01.22 14:30, Andrzej Białecki wrote:
>
> I agree, we should disable it by default (8x probably still needs to
> enable it by default for back-compat?)
>
>
> On 10 Jan 2022, at 13:56, Eric Pugh <ep...@opensourceconnections.com>
> wrote:
>
> Seems like if folks are not using it as much, maybe it should be disabled
> by default?
>
> In SOLR-15887 I removed the <jmx/> from the solrconfig.xml files, and
> added a commented out setup in solr.xml:
>
> https://github.com/apache/solr/blob/main/solr/server/solr/solr.xml#L61
>
> I wonder if it should be NOT commented out, but enabled=“false” ? Or, if
> it isn’t enabled, then that would imply that JMX reporting would be
> disabled?
>
> Or am I misunderstanding how org.apache.solr.metrics.reporters.SolrJmxReporter
> works?
>
>
>
> On Jan 9, 2022, at 7:40 PM, Mark Miller <ma...@gmail.com> wrote:
>
> JMX is really a toy metric system and comes with potential security
> concerns that have to be considered and managed over time.
>
> The cost in the case you are seeing has also been potentially much worse
> in the past - a variety of expensive metrics are now cached I believe - but
> as it iterated over each objects metrics it would rapidly gather all of the
> metrics for the object once for each metric the object had. If you had many
> large cores, each with many index files for example, this was not good to
> say the least.
>
> I would certainly not want to be exposed to these types of things when I
> was not using the metrics or using the more scalable and logical metrics
> api.
>
> *Mark Miller* - Chat @ Spike
> <https://spikenow.com/r/a/?ref=spike-organic-signature&_ts=1dg8vz> [image:
> 1dg8vz]
>
> On January 9, 2022 at 22:46 GMT, David Smiley <ds...@apache.org> wrote:
>
>
> I noticed Solr auto-creates a metrics SolrJmxReporter if there is a
> platform "MBeanServer" that exists, which AFAICT is always. Thanks? Ehh,
> no thanks. It's not evident how to disable JMX after some fruitless google
> searches. Don't get me wrong, I like jconsole, jvisualvm, JFR etc and I
> think some of these things may rely on JMX but I don't particularly need
> Solr to expose its metrics to these tools ever since Solr gained pretty
> excellent /admin/metrics support that is easier to get at.
>
> I see Solr's code that makes this decision in
> SolrXmlConfig.getMetricReporterPluginInfos and I could see that I could
> enhance it with a few lines of code to check pluginInfo.isEnabled(). Thus
> to disable JMX reporting, one would configure it with the enable="false"
> XML attribute. Or maybe we just remove the automatic enablement.
>
> BTW what's driving me to look at this is that there is some time spent
> registering and unregistering SolrCore level metrics to JMX when SolrCores
> are loaded and unloaded, and logs to this effect likewise. Not a big deal
> but it's something.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> _______________________
> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
> | http://www.opensourceconnections.com | My Free/Busy
> <http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.
>
>
>
Re: Disable metrics reporting to JMX
Posted by Matthias Krueger <mk...@mkr.io>.
JMX has its issues but we should be aware that it currently provides a
relatively generic and easy to use integration point and is used by, for
example, the Datadog Solr integration:
https://github.com/DataDog/integrations-core/tree/master/solr (and maybe
others).
On 10.01.22 14:30, Andrzej Białecki wrote:
> I agree, we should disable it by default (8x probably still needs to
> enable it by default for back-compat?)
>
>
>> On 10 Jan 2022, at 13:56, Eric Pugh <ep...@opensourceconnections.com>
>> wrote:
>>
>> Seems like if folks are not using it as much, maybe it should be
>> disabled by default?
>>
>> In SOLR-15887 I removed the <jmx/> from the solrconfig.xml files, and
>> added a commented out setup in solr.xml:
>>
>> https://github.com/apache/solr/blob/main/solr/server/solr/solr.xml#L61
>>
>> I wonder if it should be NOT commented out, but enabled=“false” ?
>> Or, if it isn’t enabled, then that would imply that JMX reporting
>> would be disabled?
>>
>> Or am I misunderstanding how
>> org.apache.solr.metrics.reporters.SolrJmxReporter works?
>>
>>
>>
>>> On Jan 9, 2022, at 7:40 PM, Mark Miller <ma...@gmail.com> wrote:
>>>
>>> JMX is really a toy metric system and comes with potential security
>>> concerns that have to be considered and managed over time.
>>>
>>> The cost in the case you are seeing has also been potentially much
>>> worse in the past - a variety of expensive metrics are now cached I
>>> believe - but as it iterated over each objects metrics it would
>>> rapidly gather all of the metrics for the object once for each
>>> metric the object had. If you had many large cores, each with many
>>> index files for example, this was not good to say the least.
>>>
>>> I would certainly not want to be exposed to these types of things
>>> when I was not using the metrics or using the more scalable and
>>> logical metrics api.
>>>
>>> *Mark Miller*- Chat @ Spike
>>> <https://spikenow.com/r/a/?ref=spike-organic-signature&_ts=1dg8vz>
>>> 1dg8vz
>>>
>>>
>>> On January 9, 2022 at 22:46 GMT, David Smiley <ds...@apache.org>
>>> wrote:
>>>
>>>
>>> I noticed Solr auto-creates a metrics SolrJmxReporter if there
>>> is a platform "MBeanServer" that exists, which AFAICT is always.
>>> Thanks? Ehh, no thanks. It's not evident how to disable JMX
>>> after some fruitless google searches. Don't get me wrong, I
>>> like jconsole, jvisualvm, JFR etc and I think some of these
>>> things may rely on JMX but I don't particularly need Solr to
>>> expose its metrics to these tools ever since Solr gained pretty
>>> excellent /admin/metrics support that is easier to get at.
>>>
>>> I see Solr's code that makes this decision in
>>> SolrXmlConfig.getMetricReporterPluginInfos and I could see that
>>> I could enhance it with a few lines of code to check
>>> pluginInfo.isEnabled(). Thus to disable JMX reporting, one
>>> would configure it with the enable="false" XML attribute. Or
>>> maybe we just remove the automatic enablement.
>>>
>>> BTW what's driving me to look at this is that there is some time
>>> spent registering and unregistering SolrCore level metrics to
>>> JMX when SolrCores are loaded and unloaded, and logs to this
>>> effect likewise. Not a big deal but it's something.
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>
>> _______________________
>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC |
>> 434.466.1467 |http://www.opensourceconnections.com
>> <http://www.opensourceconnections.com/> | My Free/Busy
>> <http://tinyurl.com/eric-cal>
>> Co-Author:Apache Solr Enterprise Search Server, 3rd Ed
>> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>> This e-mail and all contents, including attachments, is considered to
>> be Company Confidential unless explicitly stated otherwise,
>> regardless of whether attachments are marked as such.
>>
>
Re: Disable metrics reporting to JMX
Posted by Andrzej Białecki <ab...@getopt.org>.
I agree, we should disable it by default (8x probably still needs to enable it by default for back-compat?)
> On 10 Jan 2022, at 13:56, Eric Pugh <ep...@opensourceconnections.com> wrote:
>
> Seems like if folks are not using it as much, maybe it should be disabled by default?
>
> In SOLR-15887 I removed the <jmx/> from the solrconfig.xml files, and added a commented out setup in solr.xml:
>
> https://github.com/apache/solr/blob/main/solr/server/solr/solr.xml#L61 <https://github.com/apache/solr/blob/main/solr/server/solr/solr.xml#L61>
>
> I wonder if it should be NOT commented out, but enabled=“false” ? Or, if it isn’t enabled, then that would imply that JMX reporting would be disabled?
>
> Or am I misunderstanding how org.apache.solr.metrics.reporters.SolrJmxReporter works?
>
>
>
>> On Jan 9, 2022, at 7:40 PM, Mark Miller <markrmiller@gmail.com <ma...@gmail.com>> wrote:
>>
>> JMX is really a toy metric system and comes with potential security concerns that have to be considered and managed over time.
>>
>> The cost in the case you are seeing has also been potentially much worse in the past - a variety of expensive metrics are now cached I believe - but as it iterated over each objects metrics it would rapidly gather all of the metrics for the object once for each metric the object had. If you had many large cores, each with many index files for example, this was not good to say the least.
>>
>> I would certainly not want to be exposed to these types of things when I was not using the metrics or using the more scalable and logical metrics api.
>>
>> Mark Miller - Chat @ Spike <https://spikenow.com/r/a/?ref=spike-organic-signature&_ts=1dg8vz>
>>
>> On January 9, 2022 at 22:46 GMT, David Smiley <dsmiley@apache.org <ma...@apache.org>> wrote:
>>
>> I noticed Solr auto-creates a metrics SolrJmxReporter if there is a platform "MBeanServer" that exists, which AFAICT is always. Thanks? Ehh, no thanks. It's not evident how to disable JMX after some fruitless google searches. Don't get me wrong, I like jconsole, jvisualvm, JFR etc and I think some of these things may rely on JMX but I don't particularly need Solr to expose its metrics to these tools ever since Solr gained pretty excellent /admin/metrics support that is easier to get at.
>>
>> I see Solr's code that makes this decision in SolrXmlConfig.getMetricReporterPluginInfos and I could see that I could enhance it with a few lines of code to check pluginInfo.isEnabled(). Thus to disable JMX reporting, one would configure it with the enable="false" XML attribute. Or maybe we just remove the automatic enablement.
>>
>> BTW what's driving me to look at this is that there is some time spent registering and unregistering SolrCore level metrics to JMX when SolrCores are loaded and unloaded, and logs to this effect likewise. Not a big deal but it's something.
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley <http://www.linkedin.com/in/davidwsmiley>
>>
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>
Re: Disable metrics reporting to JMX
Posted by Eric Pugh <ep...@opensourceconnections.com>.
Seems like if folks are not using it as much, maybe it should be disabled by default?
In SOLR-15887 I removed the <jmx/> from the solrconfig.xml files, and added a commented out setup in solr.xml:
https://github.com/apache/solr/blob/main/solr/server/solr/solr.xml#L61 <https://github.com/apache/solr/blob/main/solr/server/solr/solr.xml#L61>
I wonder if it should be NOT commented out, but enabled=“false” ? Or, if it isn’t enabled, then that would imply that JMX reporting would be disabled?
Or am I misunderstanding how org.apache.solr.metrics.reporters.SolrJmxReporter works?
> On Jan 9, 2022, at 7:40 PM, Mark Miller <ma...@gmail.com> wrote:
>
> JMX is really a toy metric system and comes with potential security concerns that have to be considered and managed over time.
>
> The cost in the case you are seeing has also been potentially much worse in the past - a variety of expensive metrics are now cached I believe - but as it iterated over each objects metrics it would rapidly gather all of the metrics for the object once for each metric the object had. If you had many large cores, each with many index files for example, this was not good to say the least.
>
> I would certainly not want to be exposed to these types of things when I was not using the metrics or using the more scalable and logical metrics api.
>
> Mark Miller - Chat @ Spike <https://spikenow.com/r/a/?ref=spike-organic-signature&_ts=1dg8vz>
>
> On January 9, 2022 at 22:46 GMT, David Smiley <dsmiley@apache.org <ma...@apache.org>> wrote:
>
> I noticed Solr auto-creates a metrics SolrJmxReporter if there is a platform "MBeanServer" that exists, which AFAICT is always. Thanks? Ehh, no thanks. It's not evident how to disable JMX after some fruitless google searches. Don't get me wrong, I like jconsole, jvisualvm, JFR etc and I think some of these things may rely on JMX but I don't particularly need Solr to expose its metrics to these tools ever since Solr gained pretty excellent /admin/metrics support that is easier to get at.
>
> I see Solr's code that makes this decision in SolrXmlConfig.getMetricReporterPluginInfos and I could see that I could enhance it with a few lines of code to check pluginInfo.isEnabled(). Thus to disable JMX reporting, one would configure it with the enable="false" XML attribute. Or maybe we just remove the automatic enablement.
>
> BTW what's driving me to look at this is that there is some time spent registering and unregistering SolrCore level metrics to JMX when SolrCores are loaded and unloaded, and logs to this effect likewise. Not a big deal but it's something.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley <http://www.linkedin.com/in/davidwsmiley>
>
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: Disable metrics reporting to JMX
Posted by Mark Miller <ma...@gmail.com>.
JMX is really a toy metric system and comes with potential security concerns that have to be considered and managed over time.
The cost in the case you are seeing has also been potentially much worse in the past - a variety of expensive metrics are now cached I believe - but as it iterated over each objects metrics it would rapidly gather all of the metrics for the object once for each metric the object had. If you had many large cores, each with many index files for example, this was not good to say the least.
I would certainly not want to be exposed to these types of things when I was not using the metrics or using the more scalable and logical metrics api.
[Mark Miller - Chat @ Spike](https://spikenow.com/r/a/?ref=spike-organic-signature&_ts=1dg8vz) [1dg8vz]
On January 9, 2022 at 22:46 GMT, David Smiley <ds...@apache.org> wrote:
I noticed Solr auto-creates a metrics SolrJmxReporter if there is a platform "MBeanServer" that exists, which AFAICT is always. Thanks? Ehh, no thanks. It's not evident how to disable JMX after some fruitless google searches. Don't get me wrong, I like jconsole, jvisualvm, JFR etc and I think some of these things may rely on JMX but I don't particularly need Solr to expose its metrics to these tools ever since Solr gained pretty excellent /admin/metrics support that is easier to get at.
I see Solr's code that makes this decision in SolrXmlConfig.getMetricReporterPluginInfos and I could see that I could enhance it with a few lines of code to check pluginInfo.isEnabled(). Thus to disable JMX reporting, one would configure it with the enable="false" XML attribute. Or maybe we just remove the automatic enablement.
BTW what's driving me to look at this is that there is some time spent registering and unregistering SolrCore level metrics to JMX when SolrCores are loaded and unloaded, and logs to this effect likewise. Not a big deal but it's something.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley