You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by suresh pendap <su...@gmail.com> on 2018/01/08 22:02:28 UTC
regarding exposing merge metrics
Hi,
I am following the instructions from
https://lucene.apache.org/solr/guide/7_1/metrics-reporting.html
in order to expose the Index merge related metrics.
The document says that we have to add the below snippet in order to expose
the merge metrics
<config>
...
<indexConfig>
<metrics>
<majorMergeDocs>524288</majorMergeDocs>
<bool name="mergeDetails">true</bool>
</metrics>
...
</indexConfig>
...</config>
I would like to know why is this metrics not exposed by default just like
all the other metrics?
Is there any performance overhead that we should be concerned about it?
If there was no particular reason, then can we expose it by default?
Regards
Suresh
Re: regarding exposing merge metrics
Posted by suresh pendap <su...@gmail.com>.
Hi Shawn,
Thanks for replying to my questions.
So is it correct to assume that exposing merge metrics is not known to
cause any performance degradation?
-suresh
On Wed, Jan 10, 2018 at 5:40 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 1/10/2018 11:08 AM, S G wrote:
>
>> Last comment by Shawn on SOLR-10130 is:
>> Metrics was just a theory, sounds like that's not it.
>>
>> It would be very interesting to know what really caused the slowdown and
>> do
>> we really need the config or not.
>>
>
> That comment wasn't actually about SOLR-10130 itself.I commented on that
> issue because it was semi-related to what I was looking into, so I figured
> the developer who fixed it might have some insight to the question I needed
> answered.
>
> I think that SOLR-10130 was probably handled correctly and that they did
> indeed find/fix the problem.
>
> My comments were speculating about completely different performance issues
> -- someone was seeing reduced performance upgrading from 4.x to a version
> that included the fix for SOLR-10130. Because SOLR-10130 already indicated
> that metrics could cause performance issues, I was wondering whether the
> metrics that were added for keeping track of Jetty operation could possibly
> be the source of the user's problems. The response I got indicated that it
> was unlikely that the Jetty metrics were the cause.
>
> Performance regressions between 4.x and later versions have been noted by
> many users. I think some of them were caused by changes in Lucene, and
> it's possible that those who understand these issues have not yet found a
> solution to fix them.
>
> Thanks,
> Shawn
>
>
Re: regarding exposing merge metrics
Posted by Shawn Heisey <ap...@elyograg.org>.
On 1/10/2018 11:08 AM, S G wrote:
> Last comment by Shawn on SOLR-10130 is:
> Metrics was just a theory, sounds like that's not it.
>
> It would be very interesting to know what really caused the slowdown and do
> we really need the config or not.
That comment wasn't actually about SOLR-10130 itself.I commented on that
issue because it was semi-related to what I was looking into, so I
figured the developer who fixed it might have some insight to the
question I needed answered.
I think that SOLR-10130 was probably handled correctly and that they did
indeed find/fix the problem.
My comments were speculating about completely different performance
issues -- someone was seeing reduced performance upgrading from 4.x to a
version that included the fix for SOLR-10130. Because SOLR-10130
already indicated that metrics could cause performance issues, I was
wondering whether the metrics that were added for keeping track of Jetty
operation could possibly be the source of the user's problems. The
response I got indicated that it was unlikely that the Jetty metrics
were the cause.
Performance regressions between 4.x and later versions have been noted
by many users. I think some of them were caused by changes in Lucene,
and it's possible that those who understand these issues have not yet
found a solution to fix them.
Thanks,
Shawn
Re: regarding exposing merge metrics
Posted by S G <sg...@gmail.com>.
Last comment by Shawn on SOLR-10130 is:
Metrics was just a theory, sounds like that's not it.
It would be very interesting to know what really caused the slowdown and do
we really need the config or not.
Thanks
SG
On Tue, Jan 9, 2018 at 12:00 PM, suresh pendap <su...@gmail.com>
wrote:
> Thanks Shalin for sharing the link. However if I follow the thread then it
> seems like there was no conclusive evidence found that the performance
> degradation was due to the merge or index related metrics.
> If that is the case then can we just get rid of the config and publish
> these metrics by default?
>
>
> Regards
> suresh
>
>
>
> On Mon, Jan 8, 2018 at 10:25 PM, Shalin Shekhar Mangar <
> shalinmangar@gmail.com> wrote:
>
> > The merge metrics were enabled by default in 6.4 but they were turned
> > off in 6.4.2 because of large performance degradations. For more
> > details, see https://issues.apache.org/jira/browse/SOLR-10130
> >
> > On Tue, Jan 9, 2018 at 9:11 AM, S G <sg...@gmail.com> wrote:
> > > Yes, this is actually confusing and the documentation (
> > > https://lucene.apache.org/solr/guide/7_2/metrics-reporting.html) does
> > not
> > > help either:
> > >
> > > *Index Merge Metrics* : These metrics are collected in respective
> > > registries for each core (e.g., solr.core.collection1….), under the
> INDEX
> > > category.
> > > Basic metrics are always collected - collection of additional metrics
> can
> > > be turned on using boolean parameters in the
> /config/indexConfig/metrics.
> > >
> > > However, we do not see the merge-metrics being collected if the above
> > > config is absent. So what basic metrics are always collected for merge?
> > > And why do the merge metrics require an additional config while most of
> > the
> > > others are reported directly?
> > >
> > > Thanks
> > > SG
> > >
> > >
> > >
> > > On Mon, Jan 8, 2018 at 2:02 PM, suresh pendap <sureshforsolr@gmail.com
> >
> > > wrote:
> > >
> > >> Hi,
> > >> I am following the instructions from
> > >> https://lucene.apache.org/solr/guide/7_1/metrics-reporting.html
> > >> in order to expose the Index merge related metrics.
> > >>
> > >> The document says that we have to add the below snippet in order to
> > expose
> > >> the merge metrics
> > >>
> > >> <config>
> > >> ...
> > >> <indexConfig>
> > >> <metrics>
> > >> <majorMergeDocs>524288</majorMergeDocs>
> > >> <bool name="mergeDetails">true</bool>
> > >> </metrics>
> > >> ...
> > >> </indexConfig>
> > >> ...</config>
> > >>
> > >>
> > >>
> > >> I would like to know why is this metrics not exposed by default just
> > like
> > >> all the other metrics?
> > >>
> > >> Is there any performance overhead that we should be concerned about
> it?
> > >>
> > >> If there was no particular reason, then can we expose it by default?
> > >>
> > >>
> > >>
> > >> Regards
> > >> Suresh
> > >>
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
>
Re: regarding exposing merge metrics
Posted by suresh pendap <su...@gmail.com>.
Thanks Shalin for sharing the link. However if I follow the thread then it
seems like there was no conclusive evidence found that the performance
degradation was due to the merge or index related metrics.
If that is the case then can we just get rid of the config and publish
these metrics by default?
Regards
suresh
On Mon, Jan 8, 2018 at 10:25 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> The merge metrics were enabled by default in 6.4 but they were turned
> off in 6.4.2 because of large performance degradations. For more
> details, see https://issues.apache.org/jira/browse/SOLR-10130
>
> On Tue, Jan 9, 2018 at 9:11 AM, S G <sg...@gmail.com> wrote:
> > Yes, this is actually confusing and the documentation (
> > https://lucene.apache.org/solr/guide/7_2/metrics-reporting.html) does
> not
> > help either:
> >
> > *Index Merge Metrics* : These metrics are collected in respective
> > registries for each core (e.g., solr.core.collection1….), under the INDEX
> > category.
> > Basic metrics are always collected - collection of additional metrics can
> > be turned on using boolean parameters in the /config/indexConfig/metrics.
> >
> > However, we do not see the merge-metrics being collected if the above
> > config is absent. So what basic metrics are always collected for merge?
> > And why do the merge metrics require an additional config while most of
> the
> > others are reported directly?
> >
> > Thanks
> > SG
> >
> >
> >
> > On Mon, Jan 8, 2018 at 2:02 PM, suresh pendap <su...@gmail.com>
> > wrote:
> >
> >> Hi,
> >> I am following the instructions from
> >> https://lucene.apache.org/solr/guide/7_1/metrics-reporting.html
> >> in order to expose the Index merge related metrics.
> >>
> >> The document says that we have to add the below snippet in order to
> expose
> >> the merge metrics
> >>
> >> <config>
> >> ...
> >> <indexConfig>
> >> <metrics>
> >> <majorMergeDocs>524288</majorMergeDocs>
> >> <bool name="mergeDetails">true</bool>
> >> </metrics>
> >> ...
> >> </indexConfig>
> >> ...</config>
> >>
> >>
> >>
> >> I would like to know why is this metrics not exposed by default just
> like
> >> all the other metrics?
> >>
> >> Is there any performance overhead that we should be concerned about it?
> >>
> >> If there was no particular reason, then can we expose it by default?
> >>
> >>
> >>
> >> Regards
> >> Suresh
> >>
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
Re: regarding exposing merge metrics
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
The merge metrics were enabled by default in 6.4 but they were turned
off in 6.4.2 because of large performance degradations. For more
details, see https://issues.apache.org/jira/browse/SOLR-10130
On Tue, Jan 9, 2018 at 9:11 AM, S G <sg...@gmail.com> wrote:
> Yes, this is actually confusing and the documentation (
> https://lucene.apache.org/solr/guide/7_2/metrics-reporting.html) does not
> help either:
>
> *Index Merge Metrics* : These metrics are collected in respective
> registries for each core (e.g., solr.core.collection1….), under the INDEX
> category.
> Basic metrics are always collected - collection of additional metrics can
> be turned on using boolean parameters in the /config/indexConfig/metrics.
>
> However, we do not see the merge-metrics being collected if the above
> config is absent. So what basic metrics are always collected for merge?
> And why do the merge metrics require an additional config while most of the
> others are reported directly?
>
> Thanks
> SG
>
>
>
> On Mon, Jan 8, 2018 at 2:02 PM, suresh pendap <su...@gmail.com>
> wrote:
>
>> Hi,
>> I am following the instructions from
>> https://lucene.apache.org/solr/guide/7_1/metrics-reporting.html
>> in order to expose the Index merge related metrics.
>>
>> The document says that we have to add the below snippet in order to expose
>> the merge metrics
>>
>> <config>
>> ...
>> <indexConfig>
>> <metrics>
>> <majorMergeDocs>524288</majorMergeDocs>
>> <bool name="mergeDetails">true</bool>
>> </metrics>
>> ...
>> </indexConfig>
>> ...</config>
>>
>>
>>
>> I would like to know why is this metrics not exposed by default just like
>> all the other metrics?
>>
>> Is there any performance overhead that we should be concerned about it?
>>
>> If there was no particular reason, then can we expose it by default?
>>
>>
>>
>> Regards
>> Suresh
>>
--
Regards,
Shalin Shekhar Mangar.
Re: regarding exposing merge metrics
Posted by S G <sg...@gmail.com>.
Yes, this is actually confusing and the documentation (
https://lucene.apache.org/solr/guide/7_2/metrics-reporting.html) does not
help either:
*Index Merge Metrics* : These metrics are collected in respective
registries for each core (e.g., solr.core.collection1….), under the INDEX
category.
Basic metrics are always collected - collection of additional metrics can
be turned on using boolean parameters in the /config/indexConfig/metrics.
However, we do not see the merge-metrics being collected if the above
config is absent. So what basic metrics are always collected for merge?
And why do the merge metrics require an additional config while most of the
others are reported directly?
Thanks
SG
On Mon, Jan 8, 2018 at 2:02 PM, suresh pendap <su...@gmail.com>
wrote:
> Hi,
> I am following the instructions from
> https://lucene.apache.org/solr/guide/7_1/metrics-reporting.html
> in order to expose the Index merge related metrics.
>
> The document says that we have to add the below snippet in order to expose
> the merge metrics
>
> <config>
> ...
> <indexConfig>
> <metrics>
> <majorMergeDocs>524288</majorMergeDocs>
> <bool name="mergeDetails">true</bool>
> </metrics>
> ...
> </indexConfig>
> ...</config>
>
>
>
> I would like to know why is this metrics not exposed by default just like
> all the other metrics?
>
> Is there any performance overhead that we should be concerned about it?
>
> If there was no particular reason, then can we expose it by default?
>
>
>
> Regards
> Suresh
>