You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Otis Gospodnetić <ot...@gmail.com> on 2017/03/02 15:45:44 UTC
Solr JMX changes and backwards (in)compatibility
Hi,
While I love all the new metrics in Solr, I think metrics should be treated
like code/features in terms of how backwards compatibility/deprecation is
handled. Otherwise, on upgrade, people's monitoring breaks.... and
monitoring is kind of important...
Note: Looks like recent Solr metrics changes broke/changed
previously-existing MBeans...
Don't have the details about what was changed and how exactly, but I see
people using Sematext SPM for monitoring Solr are reporting this with Solr
6.4.1.
Otis
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
Re: Solr JMX changes and backwards (in)compatibility
Posted by David Smiley <da...@gmail.com>.
Agreed. I tried to do my share of being a voice of maintaining backwards
compatibility (register MBeans in old place still along with the new place)
but apparently those changes weren't enough? Andrzej attempted to get your
input
https://issues.apache.org/jira/browse/SOLR-9947?focusedCommentId=15815089&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15815089
but
you didn't respond or notice. If he hadn't '@' you, I would have as it's
obvious you might have opinions on this matter ;-)
And also related: https://issues.apache.org/jira/browse/SOLR-10035
On Thu, Mar 2, 2017 at 10:45 AM Otis Gospodnetić <ot...@gmail.com>
wrote:
> Hi,
>
> While I love all the new metrics in Solr, I think metrics should be
> treated like code/features in terms of how backwards
> compatibility/deprecation is handled. Otherwise, on upgrade, people's
> monitoring breaks.... and monitoring is kind of important...
>
> Note: Looks like recent Solr metrics changes broke/changed
> previously-existing MBeans...
>
> Don't have the details about what was changed and how exactly, but I see
> people using Sematext SPM for monitoring Solr are reporting this with Solr
> 6.4.1.
>
>
> Otis
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
> --
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com
Re: Solr JMX changes and backwards (in)compatibility
Posted by Walter Underwood <wu...@wunderwood.org>.
I did report it a week ago. This is a reply to that email.
New Relic displays the cache hit rate for each collection, showing the query result cache, filter cache, and document cache.
With 6.5.0, that page shows this message:
New Relic recorded no Solr caches data for this application in the last 24 hours
If you think there should be Solr data here, first check to see that JMX is enabled for your application server. If enabled, then please contact support.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Apr 19, 2017, at 2:28 AM, Andrzej Białecki <an...@lucidworks.com> wrote:
>
>
>> On 18 Apr 2017, at 19:15, Walter Underwood <wunder@wunderwood.org <ma...@wunderwood.org>> wrote:
>>
>> Pretty sure the back-compat did not work, because New Relic cannot find the MBeans in our 6.5.0 cluster.
>>
>
> I don’t use New Relic and nobody reported this until now, which is a pity… Could you please be more specific about what MBeans can’t be found? Does this affect all caches or just some of them?
>
>>
>>> On Apr 11, 2017, at 2:28 PM, Walter Underwood <wunder@wunderwood.org <ma...@wunderwood.org>> wrote:
>>>
>>> We are running 6.5.0 in prod and New Relic is not showing cache stats. I think this means it cannot find the MBeans.
>>>
>>> I gleaned that from the discussion here:
>>>
>>> https://discuss.newrelic.com/t/solr-data-not-appearing-in-apm-solr-tabs-caches-updates/37507/4 <https://discuss.newrelic.com/t/solr-data-not-appearing-in-apm-solr-tabs-caches-updates/37507/4>
>>> https://docs.newrelic.com/docs/agents/java-agent/troubleshooting/solr-data-not-appearing-apm-solr-tab-java <https://docs.newrelic.com/docs/agents/java-agent/troubleshooting/solr-data-not-appearing-apm-solr-tab-java>
>>>
>>> wunder
>>> Walter Underwood
>>> wunder@wunderwood.org <ma...@wunderwood.org>
>>> http://observer.wunderwood.org/ <http://observer.wunderwood.org/> (my blog)
>>>
>>>> On Mar 3, 2017, at 7:26 AM, Andrzej Białecki <andrzej.bialecki@lucidworks.com <ma...@lucidworks.com>> wrote:
>>>>
>>>>
>>>>> On 2 Mar 2017, at 16:45, Otis Gospodnetić <otis.gospodnetic@gmail.com <ma...@gmail.com>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> While I love all the new metrics in Solr, I think metrics should be treated like code/features in terms of how backwards compatibility/deprecation is handled. Otherwise, on upgrade, people's monitoring breaks.... and monitoring is kind of important...
>>>>> Note: Looks like recent Solr metrics changes broke/changed previously-existing MBeans...
>>>>> Don't have the details about what was changed and how exactly, but I see people using Sematext SPM for monitoring Solr are reporting this with Solr 6.4.1.
>>>>>
>>>>
>>>> Otis,
>>>>
>>>> Yes, we’ll be more careful, but we need proper feedback too. My understanding was that SOLR-10035 addressed this by adding back-combat registration under old names. Are you saying there are still some issues in 6.4.1? Can you please be more specific? 6.4.2 is almost out, but if it’s something serious then we should fix it.
>>>>
>>>>>
>>>>> Otis
>>>>> --
>>>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/ <http://sematext.com/>
>>>>>
>>>>
>>>
>>
>
Re: Solr JMX changes and backwards (in)compatibility
Posted by Andrzej Białecki <an...@lucidworks.com>.
> On 18 Apr 2017, at 19:15, Walter Underwood <wu...@wunderwood.org> wrote:
>
> Pretty sure the back-compat did not work, because New Relic cannot find the MBeans in our 6.5.0 cluster.
>
I don’t use New Relic and nobody reported this until now, which is a pity… Could you please be more specific about what MBeans can’t be found? Does this affect all caches or just some of them?
>
>> On Apr 11, 2017, at 2:28 PM, Walter Underwood <wunder@wunderwood.org <ma...@wunderwood.org>> wrote:
>>
>> We are running 6.5.0 in prod and New Relic is not showing cache stats. I think this means it cannot find the MBeans.
>>
>> I gleaned that from the discussion here:
>>
>> https://discuss.newrelic.com/t/solr-data-not-appearing-in-apm-solr-tabs-caches-updates/37507/4 <https://discuss.newrelic.com/t/solr-data-not-appearing-in-apm-solr-tabs-caches-updates/37507/4>
>> https://docs.newrelic.com/docs/agents/java-agent/troubleshooting/solr-data-not-appearing-apm-solr-tab-java <https://docs.newrelic.com/docs/agents/java-agent/troubleshooting/solr-data-not-appearing-apm-solr-tab-java>
>>
>> wunder
>> Walter Underwood
>> wunder@wunderwood.org <ma...@wunderwood.org>
>> http://observer.wunderwood.org/ <http://observer.wunderwood.org/> (my blog)
>>
>>> On Mar 3, 2017, at 7:26 AM, Andrzej Białecki <andrzej.bialecki@lucidworks.com <ma...@lucidworks.com>> wrote:
>>>
>>>
>>>> On 2 Mar 2017, at 16:45, Otis Gospodnetić <otis.gospodnetic@gmail.com <ma...@gmail.com>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> While I love all the new metrics in Solr, I think metrics should be treated like code/features in terms of how backwards compatibility/deprecation is handled. Otherwise, on upgrade, people's monitoring breaks.... and monitoring is kind of important...
>>>> Note: Looks like recent Solr metrics changes broke/changed previously-existing MBeans...
>>>> Don't have the details about what was changed and how exactly, but I see people using Sematext SPM for monitoring Solr are reporting this with Solr 6.4.1.
>>>>
>>>
>>> Otis,
>>>
>>> Yes, we’ll be more careful, but we need proper feedback too. My understanding was that SOLR-10035 addressed this by adding back-combat registration under old names. Are you saying there are still some issues in 6.4.1? Can you please be more specific? 6.4.2 is almost out, but if it’s something serious then we should fix it.
>>>
>>>>
>>>> Otis
>>>> --
>>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/ <http://sematext.com/>
>>>>
>>>
>>
>
Re: Solr JMX changes and backwards (in)compatibility
Posted by Walter Underwood <wu...@wunderwood.org>.
Pretty sure the back-compat did not work, because New Relic cannot find the MBeans in our 6.5.0 cluster.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Apr 11, 2017, at 2:28 PM, Walter Underwood <wu...@wunderwood.org> wrote:
>
> We are running 6.5.0 in prod and New Relic is not showing cache stats. I think this means it cannot find the MBeans.
>
> I gleaned that from the discussion here:
>
> https://discuss.newrelic.com/t/solr-data-not-appearing-in-apm-solr-tabs-caches-updates/37507/4 <https://discuss.newrelic.com/t/solr-data-not-appearing-in-apm-solr-tabs-caches-updates/37507/4>
> https://docs.newrelic.com/docs/agents/java-agent/troubleshooting/solr-data-not-appearing-apm-solr-tab-java <https://docs.newrelic.com/docs/agents/java-agent/troubleshooting/solr-data-not-appearing-apm-solr-tab-java>
>
> wunder
> Walter Underwood
> wunder@wunderwood.org <ma...@wunderwood.org>
> http://observer.wunderwood.org/ (my blog)
>
>> On Mar 3, 2017, at 7:26 AM, Andrzej Białecki <andrzej.bialecki@lucidworks.com <ma...@lucidworks.com>> wrote:
>>
>>
>>> On 2 Mar 2017, at 16:45, Otis Gospodnetić <otis.gospodnetic@gmail.com <ma...@gmail.com>> wrote:
>>>
>>> Hi,
>>>
>>> While I love all the new metrics in Solr, I think metrics should be treated like code/features in terms of how backwards compatibility/deprecation is handled. Otherwise, on upgrade, people's monitoring breaks.... and monitoring is kind of important...
>>> Note: Looks like recent Solr metrics changes broke/changed previously-existing MBeans...
>>> Don't have the details about what was changed and how exactly, but I see people using Sematext SPM for monitoring Solr are reporting this with Solr 6.4.1.
>>>
>>
>> Otis,
>>
>> Yes, we’ll be more careful, but we need proper feedback too. My understanding was that SOLR-10035 addressed this by adding back-combat registration under old names. Are you saying there are still some issues in 6.4.1? Can you please be more specific? 6.4.2 is almost out, but if it’s something serious then we should fix it.
>>
>>>
>>> Otis
>>> --
>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/ <http://sematext.com/>
>>>
>>
>
Re: Solr JMX changes and backwards (in)compatibility
Posted by Walter Underwood <wu...@wunderwood.org>.
We are running 6.5.0 in prod and New Relic is not showing cache stats. I think this means it cannot find the MBeans.
I gleaned that from the discussion here:
https://discuss.newrelic.com/t/solr-data-not-appearing-in-apm-solr-tabs-caches-updates/37507/4 <https://discuss.newrelic.com/t/solr-data-not-appearing-in-apm-solr-tabs-caches-updates/37507/4>
https://docs.newrelic.com/docs/agents/java-agent/troubleshooting/solr-data-not-appearing-apm-solr-tab-java <https://docs.newrelic.com/docs/agents/java-agent/troubleshooting/solr-data-not-appearing-apm-solr-tab-java>
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Mar 3, 2017, at 7:26 AM, Andrzej Białecki <an...@lucidworks.com> wrote:
>
>
>> On 2 Mar 2017, at 16:45, Otis Gospodnetić <otis.gospodnetic@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hi,
>>
>> While I love all the new metrics in Solr, I think metrics should be treated like code/features in terms of how backwards compatibility/deprecation is handled. Otherwise, on upgrade, people's monitoring breaks.... and monitoring is kind of important...
>> Note: Looks like recent Solr metrics changes broke/changed previously-existing MBeans...
>> Don't have the details about what was changed and how exactly, but I see people using Sematext SPM for monitoring Solr are reporting this with Solr 6.4.1.
>>
>
> Otis,
>
> Yes, we’ll be more careful, but we need proper feedback too. My understanding was that SOLR-10035 addressed this by adding back-combat registration under old names. Are you saying there are still some issues in 6.4.1? Can you please be more specific? 6.4.2 is almost out, but if it’s something serious then we should fix it.
>
>>
>> Otis
>> --
>> Monitoring - Log Management - Alerting - Anomaly Detection
>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/ <http://sematext.com/>
>>
>
Re: Solr JMX changes and backwards (in)compatibility
Posted by Andrzej Białecki <an...@lucidworks.com>.
> On 2 Mar 2017, at 16:45, Otis Gospodnetić <ot...@gmail.com> wrote:
>
> Hi,
>
> While I love all the new metrics in Solr, I think metrics should be treated like code/features in terms of how backwards compatibility/deprecation is handled. Otherwise, on upgrade, people's monitoring breaks.... and monitoring is kind of important...
> Note: Looks like recent Solr metrics changes broke/changed previously-existing MBeans...
> Don't have the details about what was changed and how exactly, but I see people using Sematext SPM for monitoring Solr are reporting this with Solr 6.4.1.
>
Otis,
Yes, we’ll be more careful, but we need proper feedback too. My understanding was that SOLR-10035 addressed this by adding back-combat registration under old names. Are you saying there are still some issues in 6.4.1? Can you please be more specific? 6.4.2 is almost out, but if it’s something serious then we should fix it.
>
> Otis
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/ <http://sematext.com/>
>