You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by dinesh naik <di...@gmail.com> on 2019/08/02 13:48:21 UTC

Solr 7.6.0: PingRequestHandler - Changing the default query (*:*)

Hi all,
I have few clusters with huge data set and whenever a node goes down its
not able to recover due to below reasons:

  1. ping request handler is taking more than 10-15 seconds to respond. The
ping requesthandler however, expects it will return in less than 1 second
and fails a requestrecovery if it is not responded to in this time.
Therefore recoveries never would start.

  2. soft commit is very low ie. 5 sec. This is a business requirement so
not much can be done here.

As the standard/default admin/ping request handler is using *:* queries ,
the response time is much higher, and i am looking for an option to change
the same so that the ping handler returns the results within few
miliseconds.

here is an example for standard query time:

----snip---
curl "
http://hostname:8983/solr/parts/select?indent=on&q=*:*&rows=0&wt=json&distrib=false&debug=timing
"
{
  "responseHeader":{
    "zkConnected":true,
    "status":0,
    "QTime":16620,
    "params":{
      "q":"*:*",
      "distrib":"false",
      "debug":"timing",
      "indent":"on",
      "rows":"0",
      "wt":"json"}},
  "response":{"numFound":1329638799,"start":0,"docs":[]
  },
  "debug":{
    "timing":{
      "time":16620.0,
      "prepare":{
        "time":0.0,
        "query":{
          "time":0.0},
        "facet":{
          "time":0.0},
        "facet_module":{
          "time":0.0},
        "mlt":{
          "time":0.0},
        "highlight":{
          "time":0.0},
        "stats":{
          "time":0.0},
        "expand":{
          "time":0.0},
        "terms":{
          "time":0.0},
        "block-expensive-queries":{
          "time":0.0},
        "slow-query-logger":{
          "time":0.0},
        "debug":{
          "time":0.0}},
      "process":{
        "time":16619.0,
        "query":{
          "time":16619.0},
        "facet":{
          "time":0.0},
        "facet_module":{
          "time":0.0},
        "mlt":{
          "time":0.0},
        "highlight":{
          "time":0.0},
        "stats":{
          "time":0.0},
        "expand":{
          "time":0.0},
        "terms":{
          "time":0.0},
        "block-expensive-queries":{
          "time":0.0},
        "slow-query-logger":{
          "time":0.0},
        "debug":{
          "time":0.0}}}}}


----snap----

can we use query: _root_:abc in the ping request handler ? Tried this query
and its returning the results within few miliseconds and also the nodes are
able to recover without any issue.

we want to use _root_ field for querying as this field is available in all
our clusters with below definition:
<field name="_root_" type="string" omitNorms="true" indexed="true"
termOffsets="false" stored="false" termPayloads="false" termPositions=
"false" docValues="false" termVectors="false"/>
Could you please let me know if using _root_ for querying in
pingRequestHandler will cause any problem?

<requestHandler name="/admin/ping" class="solr.PingRequestHandler"> <lst
name="invariants"> <str name="qt">/select</str><!-- handler to delegate to
--> <str name="q">_root_:abc</str> </lst> </requestHandler>


-- 
Best Regards,
Dinesh Naik

Re: Solr 7.6.0: PingRequestHandler - Changing the default query (*:*)

Posted by dinesh naik <di...@gmail.com>.
Hi Erick,
Each vm has 128GB of physical memory.


On Mon, Aug 5, 2019, 8:38 PM Erick Erickson <er...@gmail.com> wrote:

> How much total physical memory on your machine? Lucene holds a lot of the
> index in MMapDirectory space. My starting point is to allocate no more than
> 50% of my physical memory to the Java heap. You’re allocating 31G, if you
> don’t
> have at _least_ 64G on these machines you’re probably swapping.
>
> See:
> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
>
> Best,
> Erick
>
>
> > On Aug 5, 2019, at 10:58 AM, dinesh naik <di...@gmail.com>
> wrote:
> >
> > Hi Shawn,
> > yes i am running solr in cloud mode and Even after adding the params
> row=0
> > and distrib=false, the query response is more than 15 sec due to more
> than
> > a billion doc set.
> > Also the soft commit setting can not be changed to a higher no. due to
> > requirement from business team.
> >
> >
> http://hostname:8983/solr/parts/select?indent=on&q=*:*&rows=0&wt=json&distrib=false
> > takes more than 10 sec always.
> >
> > Here are the java heap and G1GC setting i have ,
> >
> > /usr/java/default/bin/java -server -Xmx31g -Xms31g -XX:+UseG1GC
> > -XX:MaxGCPauseMillis=250 -XX:ConcGCThreads=5
> > -XX:ParallelGCThreads=10 -XX:+UseLargePages -XX:+AggressiveOpts
> > -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled
> > -XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=18
> > -XX:MaxNewSize=6G -XX:PrintFLSStatistics=1
> > -XX:+PrintPromotionFailure -XX:+HeapDumpOnOutOfMemoryError
> > -XX:HeapDumpPath=/solr7/logs/heapdump
> > -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps
> > -XX:+PrintGCTimeStamps
> > -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime
> >
> > JVM  heap has never crossed 20GB in my setup , also Young G1GC timing is
> > well within milli seconds (in range of 25-200 ms).
> >
> > On Mon, Aug 5, 2019 at 6:37 PM Shawn Heisey <ap...@elyograg.org> wrote:
> >
> >> On 8/4/2019 10:15 PM, dinesh naik wrote:
> >>> My question is regarding the custom query being used. Here i am
> querying
> >>> for field _root_ which is available in all of my cluster and defined
> as a
> >>> string field. The result for _root_:abc might not get me any match as
> >>> well(i am ok with not finding any matches, the query should not be
> taking
> >>> 10-15 seconds for getting the response).
> >>
> >> Typically the *:* query is the fastest option.  It is special syntax
> >> that means "all documents" and it usually executes very quickly.  It
> >> will be faster than querying for a value in a specific field, which is
> >> what you have defined currently.
> >>
> >> I will typically add a "rows" parameter to the ping handler with a value
> >> of 1, so Solr will not be retrieving a large amount of data.  If you are
> >> running Solr in cloud mode, you should experiment with setting the
> >> distrib parameter to false, which will hopefully limit the query to the
> >> receiving node only.
> >>
> >> Erick has already mentioned GC pauses as a potential problem.  With a
> >> 10-15 second response time, I think that has high potential to be the
> >> underlying cause.
> >>
> >> The response you included at the beginning of the thread indicates there
> >> are 1.3 billion documents, which is going to require a fair amount of
> >> heap memory.  If seeing such long ping times with a *:* query is
> >> something that happens frequently, your heap may be too small, which
> >> will cause frequent full garbage collections.
> >>
> >> The very low autoSoftCommit time can contribute to system load.  I think
> >> it's very likely, especially with such a large index, that in many cases
> >> those automatic commits are taking far longer than 5 seconds to
> >> complete.  If that's the case, you're not achieving a 5 second
> >> visibility interval and you are putting a lot of load on Solr, so I
> >> would consider increasing it.
> >>
> >> Thanks,
> >> Shawn
> >>
> >
> >
> > --
> > Best Regards,
> > Dinesh Naik
>
>

Re: Solr 7.6.0: PingRequestHandler - Changing the default query (*:*)

Posted by Erick Erickson <er...@gmail.com>.
How much total physical memory on your machine? Lucene holds a lot of the
index in MMapDirectory space. My starting point is to allocate no more than
50% of my physical memory to the Java heap. You’re allocating 31G, if you don’t
have at _least_ 64G on these machines you’re probably swapping.

See: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html

Best,
Erick


> On Aug 5, 2019, at 10:58 AM, dinesh naik <di...@gmail.com> wrote:
> 
> Hi Shawn,
> yes i am running solr in cloud mode and Even after adding the params row=0
> and distrib=false, the query response is more than 15 sec due to more than
> a billion doc set.
> Also the soft commit setting can not be changed to a higher no. due to
> requirement from business team.
> 
> http://hostname:8983/solr/parts/select?indent=on&q=*:*&rows=0&wt=json&distrib=false
> takes more than 10 sec always.
> 
> Here are the java heap and G1GC setting i have ,
> 
> /usr/java/default/bin/java -server -Xmx31g -Xms31g -XX:+UseG1GC
> -XX:MaxGCPauseMillis=250 -XX:ConcGCThreads=5
> -XX:ParallelGCThreads=10 -XX:+UseLargePages -XX:+AggressiveOpts
> -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled
> -XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=18
> -XX:MaxNewSize=6G -XX:PrintFLSStatistics=1
> -XX:+PrintPromotionFailure -XX:+HeapDumpOnOutOfMemoryError
> -XX:HeapDumpPath=/solr7/logs/heapdump
> -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps
> -XX:+PrintGCTimeStamps
> -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime
> 
> JVM  heap has never crossed 20GB in my setup , also Young G1GC timing is
> well within milli seconds (in range of 25-200 ms).
> 
> On Mon, Aug 5, 2019 at 6:37 PM Shawn Heisey <ap...@elyograg.org> wrote:
> 
>> On 8/4/2019 10:15 PM, dinesh naik wrote:
>>> My question is regarding the custom query being used. Here i am querying
>>> for field _root_ which is available in all of my cluster and defined as a
>>> string field. The result for _root_:abc might not get me any match as
>>> well(i am ok with not finding any matches, the query should not be taking
>>> 10-15 seconds for getting the response).
>> 
>> Typically the *:* query is the fastest option.  It is special syntax
>> that means "all documents" and it usually executes very quickly.  It
>> will be faster than querying for a value in a specific field, which is
>> what you have defined currently.
>> 
>> I will typically add a "rows" parameter to the ping handler with a value
>> of 1, so Solr will not be retrieving a large amount of data.  If you are
>> running Solr in cloud mode, you should experiment with setting the
>> distrib parameter to false, which will hopefully limit the query to the
>> receiving node only.
>> 
>> Erick has already mentioned GC pauses as a potential problem.  With a
>> 10-15 second response time, I think that has high potential to be the
>> underlying cause.
>> 
>> The response you included at the beginning of the thread indicates there
>> are 1.3 billion documents, which is going to require a fair amount of
>> heap memory.  If seeing such long ping times with a *:* query is
>> something that happens frequently, your heap may be too small, which
>> will cause frequent full garbage collections.
>> 
>> The very low autoSoftCommit time can contribute to system load.  I think
>> it's very likely, especially with such a large index, that in many cases
>> those automatic commits are taking far longer than 5 seconds to
>> complete.  If that's the case, you're not achieving a 5 second
>> visibility interval and you are putting a lot of load on Solr, so I
>> would consider increasing it.
>> 
>> Thanks,
>> Shawn
>> 
> 
> 
> -- 
> Best Regards,
> Dinesh Naik


Re: Solr 7.6.0: PingRequestHandler - Changing the default query (*:*)

Posted by dinesh naik <di...@gmail.com>.
Hi Shawn,
yes i am running solr in cloud mode and Even after adding the params row=0
and distrib=false, the query response is more than 15 sec due to more than
a billion doc set.
Also the soft commit setting can not be changed to a higher no. due to
requirement from business team.

http://hostname:8983/solr/parts/select?indent=on&q=*:*&rows=0&wt=json&distrib=false
takes more than 10 sec always.

Here are the java heap and G1GC setting i have ,

/usr/java/default/bin/java -server -Xmx31g -Xms31g -XX:+UseG1GC
-XX:MaxGCPauseMillis=250 -XX:ConcGCThreads=5
-XX:ParallelGCThreads=10 -XX:+UseLargePages -XX:+AggressiveOpts
-XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled
-XX:InitiatingHeapOccupancyPercent=50 -XX:G1ReservePercent=18
-XX:MaxNewSize=6G -XX:PrintFLSStatistics=1
-XX:+PrintPromotionFailure -XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/solr7/logs/heapdump
-verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps
-XX:+PrintGCTimeStamps
-XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime

JVM  heap has never crossed 20GB in my setup , also Young G1GC timing is
well within milli seconds (in range of 25-200 ms).

On Mon, Aug 5, 2019 at 6:37 PM Shawn Heisey <ap...@elyograg.org> wrote:

> On 8/4/2019 10:15 PM, dinesh naik wrote:
> > My question is regarding the custom query being used. Here i am querying
> > for field _root_ which is available in all of my cluster and defined as a
> > string field. The result for _root_:abc might not get me any match as
> > well(i am ok with not finding any matches, the query should not be taking
> > 10-15 seconds for getting the response).
>
> Typically the *:* query is the fastest option.  It is special syntax
> that means "all documents" and it usually executes very quickly.  It
> will be faster than querying for a value in a specific field, which is
> what you have defined currently.
>
> I will typically add a "rows" parameter to the ping handler with a value
> of 1, so Solr will not be retrieving a large amount of data.  If you are
> running Solr in cloud mode, you should experiment with setting the
> distrib parameter to false, which will hopefully limit the query to the
> receiving node only.
>
> Erick has already mentioned GC pauses as a potential problem.  With a
> 10-15 second response time, I think that has high potential to be the
> underlying cause.
>
> The response you included at the beginning of the thread indicates there
> are 1.3 billion documents, which is going to require a fair amount of
> heap memory.  If seeing such long ping times with a *:* query is
> something that happens frequently, your heap may be too small, which
> will cause frequent full garbage collections.
>
> The very low autoSoftCommit time can contribute to system load.  I think
> it's very likely, especially with such a large index, that in many cases
> those automatic commits are taking far longer than 5 seconds to
> complete.  If that's the case, you're not achieving a 5 second
> visibility interval and you are putting a lot of load on Solr, so I
> would consider increasing it.
>
> Thanks,
> Shawn
>


-- 
Best Regards,
Dinesh Naik

Re: Solr 7.6.0: PingRequestHandler - Changing the default query (*:*)

Posted by Shawn Heisey <ap...@elyograg.org>.
On 8/4/2019 10:15 PM, dinesh naik wrote:
> My question is regarding the custom query being used. Here i am querying
> for field _root_ which is available in all of my cluster and defined as a
> string field. The result for _root_:abc might not get me any match as
> well(i am ok with not finding any matches, the query should not be taking
> 10-15 seconds for getting the response).

Typically the *:* query is the fastest option.  It is special syntax 
that means "all documents" and it usually executes very quickly.  It 
will be faster than querying for a value in a specific field, which is 
what you have defined currently.

I will typically add a "rows" parameter to the ping handler with a value 
of 1, so Solr will not be retrieving a large amount of data.  If you are 
running Solr in cloud mode, you should experiment with setting the 
distrib parameter to false, which will hopefully limit the query to the 
receiving node only.

Erick has already mentioned GC pauses as a potential problem.  With a 
10-15 second response time, I think that has high potential to be the 
underlying cause.

The response you included at the beginning of the thread indicates there 
are 1.3 billion documents, which is going to require a fair amount of 
heap memory.  If seeing such long ping times with a *:* query is 
something that happens frequently, your heap may be too small, which 
will cause frequent full garbage collections.

The very low autoSoftCommit time can contribute to system load.  I think 
it's very likely, especially with such a large index, that in many cases 
those automatic commits are taking far longer than 5 seconds to 
complete.  If that's the case, you're not achieving a 5 second 
visibility interval and you are putting a lot of load on Solr, so I 
would consider increasing it.

Thanks,
Shawn

Re: Solr 7.6.0: PingRequestHandler - Changing the default query (*:*)

Posted by dinesh naik <di...@gmail.com>.
Hi Nikolas,
The restart of node is not helping , the node keeps trying to recover and
always fails:

here is the log :
2019-07-31 06:10:08.049 INFO
 (coreZkRegister-1-thread-1-processing-n:replica_host:8983_solr
x:parts_shard30_replica_n2697 c:parts s:shard30 r:core_node2698)
x:parts_shard30_replica_n2697 o.a.s.c.ZkController Core needs to
recover:parts_shard30_replica_n2697

2019-07-31 06:10:08.050 INFO
 (updateExecutor-3-thread-1-processing-n:replica_host:8983_solr
x:parts_shard30_replica_n2697 c:parts s:shard30 r:core_node2698)
x:parts_shard30_replica_n2697 o.a.s.u.DefaultSolrCoreState Running recovery

2019-07-31 06:10:08.056 INFO
 (recoveryExecutor-4-thread-1-processing-n:replica_host:8983_solr
x:parts_shard30_replica_n2697 c:parts s:shard30 r:core_node2698)
x:parts_shard30_replica_n2697 o.a.s.c.RecoveryStrategy Starting recovery
process. recoveringAfterStartup=true

2019-07-31 06:10:08.261 INFO
 (recoveryExecutor-4-thread-1-processing-n:replica_host:8983_solr
x:parts_shard30_replica_n2697 c:parts s:shard30 r:core_node2698)
x:parts_shard30_replica_n2697 o.a.s.c.RecoveryStrategy startupVersions
size=49956 range=[1640550593276674048 to 1640542396328443904]

2019-07-31 06:10:08.328 INFO  (qtp689401025-58)  o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/info/key params={omitHeader=true&wt=json}
status=0 QTime=0

2019-07-31 06:10:09.276 INFO
 (recoveryExecutor-4-thread-1-processing-n:replica_host:8983_solr
x:parts_shard30_replica_n2697 c:parts s:shard30 r:core_node2698)
x:parts_shard30_replica_n2697 o.a.s.c.RecoveryStrategy Failed to connect
leader http://hostname:8983/solr on recovery, try again

The ping request query is being called from solr itself and not via some
script,so there is no way to stop it .

code where the time is hardcoded to 1 sec:

try (HttpSolrClient httpSolrClient = new
HttpSolrClient.Builder(leaderReplica.getCoreUrl())
          .withSocketTimeout(1000)
          .withConnectionTimeout(1000)

.withHttpClient(cc.getUpdateShardHandler().getRecoveryOnlyHttpClient())
          .build()) {
        SolrPingResponse resp = httpSolrClient.ping();
        return leaderReplica;
      } catch (IOException e) {
        log.info("Failed to connect leader {} on recovery, try again",
leaderReplica.getBaseUrl());
        Thread.sleep(500);
      } catch (Exception e) {
        if (e.getCause() instanceof IOException) {
          log.info("Failed to connect leader {} on recovery, try again",
leaderReplica.getBaseUrl());
          Thread.sleep(500);
        } else {
          return leaderReplica;
        }
      }



On Mon, Aug 5, 2019 at 1:19 PM Nicolas Franck <Ni...@ugent.be>
wrote:

> If the ping request handler is taking too long,
> and the server is not recovering automatically,
> there is not much you can do automatically on that server.
> You have to intervene manually, and restart Solr on that node.
>
> First of all: the ping is just an internal check. If it takes too long
> to respond, the requester (i.e. the script calling it), should stop
> the request, and mark that node as problematic. If there are
> for example memory problems every subsequent request will only enhance
> the problem, and Solr cannot recover from that.
>
> > On 5 Aug 2019, at 06:15, dinesh naik <di...@gmail.com> wrote:
> >
> > Thanks john,Erick and Furknan.
> >
> > I have already defined the ping request handler in solrconfig.xml as
> below:
> > <requestHandler name="/admin/ping" class="solr.PingRequestHandler"> <lst
> > name="invariants"> <str name="qt">/select</str><!-- handler to delegate
> to
> > --> <str name="q">_root_:abc</str> </lst> </requestHandler>
> >
> > My question is regarding the custom query being used. Here i am querying
> > for field _root_ which is available in all of my cluster and defined as a
> > string field. The result for _root_:abc might not get me any match as
> > well(i am ok with not finding any matches, the query should not be taking
> > 10-15 seconds for getting the response).
> >
> > If the response comes within 1 second , then the core recovery issue is
> > solved, hence need your suggestion if using _root_ field in custom query
> is
> > fine?
> >
> >
> > On Mon, Aug 5, 2019 at 2:49 AM Furkan KAMACI <fu...@gmail.com>
> wrote:
> >
> >> Hi,
> >>
> >> You can change invariants i.e. *qt* and *q* of a *PingRequestHandler*:
> >>
> >> <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
> >>   <lst name="invariants">
> >>     <str name="qt">/search</str><!-- handler to delegate to -->
> >>     <str name="q">some test query</str>
> >>   </lst>
> >> </requestHandler>
> >>
> >> Check documentation fore more info:
> >>
> >>
> https://lucene.apache.org/solr/7_6_0//solr-core/org/apache/solr/handler/PingRequestHandler.html
> >>
> >> Kind Regards,
> >> Furkan KAMACI
> >>
> >> On Sat, Aug 3, 2019 at 4:17 PM Erick Erickson <er...@gmail.com>
> >> wrote:
> >>
> >>> You can also (I think) explicitly define the ping request handler in
> >>> solrconfig.xml to do something else.
> >>>
> >>>> On Aug 2, 2019, at 9:50 AM, Jörn Franke <jo...@gmail.com> wrote:
> >>>>
> >>>> Not sure if this is possible, but why not create a query handler in
> >> Solr
> >>> with any custom query and you use that as ping replacement ?
> >>>>
> >>>>> Am 02.08.2019 um 15:48 schrieb dinesh naik <
> dineshkumarnaik@gmail.com
> >>> :
> >>>>>
> >>>>> Hi all,
> >>>>> I have few clusters with huge data set and whenever a node goes down
> >> its
> >>>>> not able to recover due to below reasons:
> >>>>>
> >>>>> 1. ping request handler is taking more than 10-15 seconds to respond.
> >>> The
> >>>>> ping requesthandler however, expects it will return in less than 1
> >>> second
> >>>>> and fails a requestrecovery if it is not responded to in this time.
> >>>>> Therefore recoveries never would start.
> >>>>>
> >>>>> 2. soft commit is very low ie. 5 sec. This is a business requirement
> >> so
> >>>>> not much can be done here.
> >>>>>
> >>>>> As the standard/default admin/ping request handler is using *:*
> >> queries
> >>> ,
> >>>>> the response time is much higher, and i am looking for an option to
> >>> change
> >>>>> the same so that the ping handler returns the results within few
> >>>>> miliseconds.
> >>>>>
> >>>>> here is an example for standard query time:
> >>>>>
> >>>>> ----snip---
> >>>>> curl "
> >>>>>
> >>>
> >>
> http://hostname:8983/solr/parts/select?indent=on&q=*:*&rows=0&wt=json&distrib=false&debug=timing
> >>>>> "
> >>>>> {
> >>>>> "responseHeader":{
> >>>>>  "zkConnected":true,
> >>>>>  "status":0,
> >>>>>  "QTime":16620,
> >>>>>  "params":{
> >>>>>    "q":"*:*",
> >>>>>    "distrib":"false",
> >>>>>    "debug":"timing",
> >>>>>    "indent":"on",
> >>>>>    "rows":"0",
> >>>>>    "wt":"json"}},
> >>>>> "response":{"numFound":1329638799,"start":0,"docs":[]
> >>>>> },
> >>>>> "debug":{
> >>>>>  "timing":{
> >>>>>    "time":16620.0,
> >>>>>    "prepare":{
> >>>>>      "time":0.0,
> >>>>>      "query":{
> >>>>>        "time":0.0},
> >>>>>      "facet":{
> >>>>>        "time":0.0},
> >>>>>      "facet_module":{
> >>>>>        "time":0.0},
> >>>>>      "mlt":{
> >>>>>        "time":0.0},
> >>>>>      "highlight":{
> >>>>>        "time":0.0},
> >>>>>      "stats":{
> >>>>>        "time":0.0},
> >>>>>      "expand":{
> >>>>>        "time":0.0},
> >>>>>      "terms":{
> >>>>>        "time":0.0},
> >>>>>      "block-expensive-queries":{
> >>>>>        "time":0.0},
> >>>>>      "slow-query-logger":{
> >>>>>        "time":0.0},
> >>>>>      "debug":{
> >>>>>        "time":0.0}},
> >>>>>    "process":{
> >>>>>      "time":16619.0,
> >>>>>      "query":{
> >>>>>        "time":16619.0},
> >>>>>      "facet":{
> >>>>>        "time":0.0},
> >>>>>      "facet_module":{
> >>>>>        "time":0.0},
> >>>>>      "mlt":{
> >>>>>        "time":0.0},
> >>>>>      "highlight":{
> >>>>>        "time":0.0},
> >>>>>      "stats":{
> >>>>>        "time":0.0},
> >>>>>      "expand":{
> >>>>>        "time":0.0},
> >>>>>      "terms":{
> >>>>>        "time":0.0},
> >>>>>      "block-expensive-queries":{
> >>>>>        "time":0.0},
> >>>>>      "slow-query-logger":{
> >>>>>        "time":0.0},
> >>>>>      "debug":{
> >>>>>        "time":0.0}}}}}
> >>>>>
> >>>>>
> >>>>> ----snap----
> >>>>>
> >>>>> can we use query: _root_:abc in the ping request handler ? Tried this
> >>> query
> >>>>> and its returning the results within few miliseconds and also the
> >> nodes
> >>> are
> >>>>> able to recover without any issue.
> >>>>>
> >>>>> we want to use _root_ field for querying as this field is available
> in
> >>> all
> >>>>> our clusters with below definition:
> >>>>> <field name="_root_" type="string" omitNorms="true" indexed="true"
> >>>>> termOffsets="false" stored="false" termPayloads="false"
> termPositions=
> >>>>> "false" docValues="false" termVectors="false"/>
> >>>>> Could you please let me know if using _root_ for querying in
> >>>>> pingRequestHandler will cause any problem?
> >>>>>
> >>>>> <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
> >> <lst
> >>>>> name="invariants"> <str name="qt">/select</str><!-- handler to
> >> delegate
> >>> to
> >>>>> --> <str name="q">_root_:abc</str> </lst> </requestHandler>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Best Regards,
> >>>>> Dinesh Naik
> >>>
> >>>
> >>
> >
> >
> > --
> > Best Regards,
> > Dinesh Naik
>
>

-- 
Best Regards,
Dinesh Naik

Re: Solr 7.6.0: PingRequestHandler - Changing the default query (*:*)

Posted by Nicolas Franck <Ni...@UGent.be>.
If the ping request handler is taking too long,
and the server is not recovering automatically,
there is not much you can do automatically on that server.
You have to intervene manually, and restart Solr on that node.

First of all: the ping is just an internal check. If it takes too long
to respond, the requester (i.e. the script calling it), should stop
the request, and mark that node as problematic. If there are
for example memory problems every subsequent request will only enhance
the problem, and Solr cannot recover from that.

> On 5 Aug 2019, at 06:15, dinesh naik <di...@gmail.com> wrote:
> 
> Thanks john,Erick and Furknan.
> 
> I have already defined the ping request handler in solrconfig.xml as below:
> <requestHandler name="/admin/ping" class="solr.PingRequestHandler"> <lst
> name="invariants"> <str name="qt">/select</str><!-- handler to delegate to
> --> <str name="q">_root_:abc</str> </lst> </requestHandler>
> 
> My question is regarding the custom query being used. Here i am querying
> for field _root_ which is available in all of my cluster and defined as a
> string field. The result for _root_:abc might not get me any match as
> well(i am ok with not finding any matches, the query should not be taking
> 10-15 seconds for getting the response).
> 
> If the response comes within 1 second , then the core recovery issue is
> solved, hence need your suggestion if using _root_ field in custom query is
> fine?
> 
> 
> On Mon, Aug 5, 2019 at 2:49 AM Furkan KAMACI <fu...@gmail.com> wrote:
> 
>> Hi,
>> 
>> You can change invariants i.e. *qt* and *q* of a *PingRequestHandler*:
>> 
>> <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
>>   <lst name="invariants">
>>     <str name="qt">/search</str><!-- handler to delegate to -->
>>     <str name="q">some test query</str>
>>   </lst>
>> </requestHandler>
>> 
>> Check documentation fore more info:
>> 
>> https://lucene.apache.org/solr/7_6_0//solr-core/org/apache/solr/handler/PingRequestHandler.html
>> 
>> Kind Regards,
>> Furkan KAMACI
>> 
>> On Sat, Aug 3, 2019 at 4:17 PM Erick Erickson <er...@gmail.com>
>> wrote:
>> 
>>> You can also (I think) explicitly define the ping request handler in
>>> solrconfig.xml to do something else.
>>> 
>>>> On Aug 2, 2019, at 9:50 AM, Jörn Franke <jo...@gmail.com> wrote:
>>>> 
>>>> Not sure if this is possible, but why not create a query handler in
>> Solr
>>> with any custom query and you use that as ping replacement ?
>>>> 
>>>>> Am 02.08.2019 um 15:48 schrieb dinesh naik <dineshkumarnaik@gmail.com
>>> :
>>>>> 
>>>>> Hi all,
>>>>> I have few clusters with huge data set and whenever a node goes down
>> its
>>>>> not able to recover due to below reasons:
>>>>> 
>>>>> 1. ping request handler is taking more than 10-15 seconds to respond.
>>> The
>>>>> ping requesthandler however, expects it will return in less than 1
>>> second
>>>>> and fails a requestrecovery if it is not responded to in this time.
>>>>> Therefore recoveries never would start.
>>>>> 
>>>>> 2. soft commit is very low ie. 5 sec. This is a business requirement
>> so
>>>>> not much can be done here.
>>>>> 
>>>>> As the standard/default admin/ping request handler is using *:*
>> queries
>>> ,
>>>>> the response time is much higher, and i am looking for an option to
>>> change
>>>>> the same so that the ping handler returns the results within few
>>>>> miliseconds.
>>>>> 
>>>>> here is an example for standard query time:
>>>>> 
>>>>> ----snip---
>>>>> curl "
>>>>> 
>>> 
>> http://hostname:8983/solr/parts/select?indent=on&q=*:*&rows=0&wt=json&distrib=false&debug=timing
>>>>> "
>>>>> {
>>>>> "responseHeader":{
>>>>>  "zkConnected":true,
>>>>>  "status":0,
>>>>>  "QTime":16620,
>>>>>  "params":{
>>>>>    "q":"*:*",
>>>>>    "distrib":"false",
>>>>>    "debug":"timing",
>>>>>    "indent":"on",
>>>>>    "rows":"0",
>>>>>    "wt":"json"}},
>>>>> "response":{"numFound":1329638799,"start":0,"docs":[]
>>>>> },
>>>>> "debug":{
>>>>>  "timing":{
>>>>>    "time":16620.0,
>>>>>    "prepare":{
>>>>>      "time":0.0,
>>>>>      "query":{
>>>>>        "time":0.0},
>>>>>      "facet":{
>>>>>        "time":0.0},
>>>>>      "facet_module":{
>>>>>        "time":0.0},
>>>>>      "mlt":{
>>>>>        "time":0.0},
>>>>>      "highlight":{
>>>>>        "time":0.0},
>>>>>      "stats":{
>>>>>        "time":0.0},
>>>>>      "expand":{
>>>>>        "time":0.0},
>>>>>      "terms":{
>>>>>        "time":0.0},
>>>>>      "block-expensive-queries":{
>>>>>        "time":0.0},
>>>>>      "slow-query-logger":{
>>>>>        "time":0.0},
>>>>>      "debug":{
>>>>>        "time":0.0}},
>>>>>    "process":{
>>>>>      "time":16619.0,
>>>>>      "query":{
>>>>>        "time":16619.0},
>>>>>      "facet":{
>>>>>        "time":0.0},
>>>>>      "facet_module":{
>>>>>        "time":0.0},
>>>>>      "mlt":{
>>>>>        "time":0.0},
>>>>>      "highlight":{
>>>>>        "time":0.0},
>>>>>      "stats":{
>>>>>        "time":0.0},
>>>>>      "expand":{
>>>>>        "time":0.0},
>>>>>      "terms":{
>>>>>        "time":0.0},
>>>>>      "block-expensive-queries":{
>>>>>        "time":0.0},
>>>>>      "slow-query-logger":{
>>>>>        "time":0.0},
>>>>>      "debug":{
>>>>>        "time":0.0}}}}}
>>>>> 
>>>>> 
>>>>> ----snap----
>>>>> 
>>>>> can we use query: _root_:abc in the ping request handler ? Tried this
>>> query
>>>>> and its returning the results within few miliseconds and also the
>> nodes
>>> are
>>>>> able to recover without any issue.
>>>>> 
>>>>> we want to use _root_ field for querying as this field is available in
>>> all
>>>>> our clusters with below definition:
>>>>> <field name="_root_" type="string" omitNorms="true" indexed="true"
>>>>> termOffsets="false" stored="false" termPayloads="false" termPositions=
>>>>> "false" docValues="false" termVectors="false"/>
>>>>> Could you please let me know if using _root_ for querying in
>>>>> pingRequestHandler will cause any problem?
>>>>> 
>>>>> <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
>> <lst
>>>>> name="invariants"> <str name="qt">/select</str><!-- handler to
>> delegate
>>> to
>>>>> --> <str name="q">_root_:abc</str> </lst> </requestHandler>
>>>>> 
>>>>> 
>>>>> --
>>>>> Best Regards,
>>>>> Dinesh Naik
>>> 
>>> 
>> 
> 
> 
> -- 
> Best Regards,
> Dinesh Naik


Re: Solr 7.6.0: PingRequestHandler - Changing the default query (*:*)

Posted by dinesh naik <di...@gmail.com>.
Thanks john,Erick and Furknan.

I have already defined the ping request handler in solrconfig.xml as below:
<requestHandler name="/admin/ping" class="solr.PingRequestHandler"> <lst
name="invariants"> <str name="qt">/select</str><!-- handler to delegate to
--> <str name="q">_root_:abc</str> </lst> </requestHandler>

My question is regarding the custom query being used. Here i am querying
for field _root_ which is available in all of my cluster and defined as a
string field. The result for _root_:abc might not get me any match as
well(i am ok with not finding any matches, the query should not be taking
10-15 seconds for getting the response).

If the response comes within 1 second , then the core recovery issue is
solved, hence need your suggestion if using _root_ field in custom query is
fine?


On Mon, Aug 5, 2019 at 2:49 AM Furkan KAMACI <fu...@gmail.com> wrote:

> Hi,
>
> You can change invariants i.e. *qt* and *q* of a *PingRequestHandler*:
>
> <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
>    <lst name="invariants">
>      <str name="qt">/search</str><!-- handler to delegate to -->
>      <str name="q">some test query</str>
>    </lst>
>  </requestHandler>
>
> Check documentation fore more info:
>
> https://lucene.apache.org/solr/7_6_0//solr-core/org/apache/solr/handler/PingRequestHandler.html
>
> Kind Regards,
> Furkan KAMACI
>
> On Sat, Aug 3, 2019 at 4:17 PM Erick Erickson <er...@gmail.com>
> wrote:
>
> > You can also (I think) explicitly define the ping request handler in
> > solrconfig.xml to do something else.
> >
> > > On Aug 2, 2019, at 9:50 AM, Jörn Franke <jo...@gmail.com> wrote:
> > >
> > > Not sure if this is possible, but why not create a query handler in
> Solr
> > with any custom query and you use that as ping replacement ?
> > >
> > >> Am 02.08.2019 um 15:48 schrieb dinesh naik <dineshkumarnaik@gmail.com
> >:
> > >>
> > >> Hi all,
> > >> I have few clusters with huge data set and whenever a node goes down
> its
> > >> not able to recover due to below reasons:
> > >>
> > >> 1. ping request handler is taking more than 10-15 seconds to respond.
> > The
> > >> ping requesthandler however, expects it will return in less than 1
> > second
> > >> and fails a requestrecovery if it is not responded to in this time.
> > >> Therefore recoveries never would start.
> > >>
> > >> 2. soft commit is very low ie. 5 sec. This is a business requirement
> so
> > >> not much can be done here.
> > >>
> > >> As the standard/default admin/ping request handler is using *:*
> queries
> > ,
> > >> the response time is much higher, and i am looking for an option to
> > change
> > >> the same so that the ping handler returns the results within few
> > >> miliseconds.
> > >>
> > >> here is an example for standard query time:
> > >>
> > >> ----snip---
> > >> curl "
> > >>
> >
> http://hostname:8983/solr/parts/select?indent=on&q=*:*&rows=0&wt=json&distrib=false&debug=timing
> > >> "
> > >> {
> > >> "responseHeader":{
> > >>   "zkConnected":true,
> > >>   "status":0,
> > >>   "QTime":16620,
> > >>   "params":{
> > >>     "q":"*:*",
> > >>     "distrib":"false",
> > >>     "debug":"timing",
> > >>     "indent":"on",
> > >>     "rows":"0",
> > >>     "wt":"json"}},
> > >> "response":{"numFound":1329638799,"start":0,"docs":[]
> > >> },
> > >> "debug":{
> > >>   "timing":{
> > >>     "time":16620.0,
> > >>     "prepare":{
> > >>       "time":0.0,
> > >>       "query":{
> > >>         "time":0.0},
> > >>       "facet":{
> > >>         "time":0.0},
> > >>       "facet_module":{
> > >>         "time":0.0},
> > >>       "mlt":{
> > >>         "time":0.0},
> > >>       "highlight":{
> > >>         "time":0.0},
> > >>       "stats":{
> > >>         "time":0.0},
> > >>       "expand":{
> > >>         "time":0.0},
> > >>       "terms":{
> > >>         "time":0.0},
> > >>       "block-expensive-queries":{
> > >>         "time":0.0},
> > >>       "slow-query-logger":{
> > >>         "time":0.0},
> > >>       "debug":{
> > >>         "time":0.0}},
> > >>     "process":{
> > >>       "time":16619.0,
> > >>       "query":{
> > >>         "time":16619.0},
> > >>       "facet":{
> > >>         "time":0.0},
> > >>       "facet_module":{
> > >>         "time":0.0},
> > >>       "mlt":{
> > >>         "time":0.0},
> > >>       "highlight":{
> > >>         "time":0.0},
> > >>       "stats":{
> > >>         "time":0.0},
> > >>       "expand":{
> > >>         "time":0.0},
> > >>       "terms":{
> > >>         "time":0.0},
> > >>       "block-expensive-queries":{
> > >>         "time":0.0},
> > >>       "slow-query-logger":{
> > >>         "time":0.0},
> > >>       "debug":{
> > >>         "time":0.0}}}}}
> > >>
> > >>
> > >> ----snap----
> > >>
> > >> can we use query: _root_:abc in the ping request handler ? Tried this
> > query
> > >> and its returning the results within few miliseconds and also the
> nodes
> > are
> > >> able to recover without any issue.
> > >>
> > >> we want to use _root_ field for querying as this field is available in
> > all
> > >> our clusters with below definition:
> > >> <field name="_root_" type="string" omitNorms="true" indexed="true"
> > >> termOffsets="false" stored="false" termPayloads="false" termPositions=
> > >> "false" docValues="false" termVectors="false"/>
> > >> Could you please let me know if using _root_ for querying in
> > >> pingRequestHandler will cause any problem?
> > >>
> > >> <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
> <lst
> > >> name="invariants"> <str name="qt">/select</str><!-- handler to
> delegate
> > to
> > >> --> <str name="q">_root_:abc</str> </lst> </requestHandler>
> > >>
> > >>
> > >> --
> > >> Best Regards,
> > >> Dinesh Naik
> >
> >
>


-- 
Best Regards,
Dinesh Naik

Re: Solr 7.6.0: PingRequestHandler - Changing the default query (*:*)

Posted by Furkan KAMACI <fu...@gmail.com>.
Hi,

You can change invariants i.e. *qt* and *q* of a *PingRequestHandler*:

<requestHandler name="/admin/ping" class="solr.PingRequestHandler">
   <lst name="invariants">
     <str name="qt">/search</str><!-- handler to delegate to -->
     <str name="q">some test query</str>
   </lst>
 </requestHandler>

Check documentation fore more info:
https://lucene.apache.org/solr/7_6_0//solr-core/org/apache/solr/handler/PingRequestHandler.html

Kind Regards,
Furkan KAMACI

On Sat, Aug 3, 2019 at 4:17 PM Erick Erickson <er...@gmail.com>
wrote:

> You can also (I think) explicitly define the ping request handler in
> solrconfig.xml to do something else.
>
> > On Aug 2, 2019, at 9:50 AM, Jörn Franke <jo...@gmail.com> wrote:
> >
> > Not sure if this is possible, but why not create a query handler in Solr
> with any custom query and you use that as ping replacement ?
> >
> >> Am 02.08.2019 um 15:48 schrieb dinesh naik <di...@gmail.com>:
> >>
> >> Hi all,
> >> I have few clusters with huge data set and whenever a node goes down its
> >> not able to recover due to below reasons:
> >>
> >> 1. ping request handler is taking more than 10-15 seconds to respond.
> The
> >> ping requesthandler however, expects it will return in less than 1
> second
> >> and fails a requestrecovery if it is not responded to in this time.
> >> Therefore recoveries never would start.
> >>
> >> 2. soft commit is very low ie. 5 sec. This is a business requirement so
> >> not much can be done here.
> >>
> >> As the standard/default admin/ping request handler is using *:* queries
> ,
> >> the response time is much higher, and i am looking for an option to
> change
> >> the same so that the ping handler returns the results within few
> >> miliseconds.
> >>
> >> here is an example for standard query time:
> >>
> >> ----snip---
> >> curl "
> >>
> http://hostname:8983/solr/parts/select?indent=on&q=*:*&rows=0&wt=json&distrib=false&debug=timing
> >> "
> >> {
> >> "responseHeader":{
> >>   "zkConnected":true,
> >>   "status":0,
> >>   "QTime":16620,
> >>   "params":{
> >>     "q":"*:*",
> >>     "distrib":"false",
> >>     "debug":"timing",
> >>     "indent":"on",
> >>     "rows":"0",
> >>     "wt":"json"}},
> >> "response":{"numFound":1329638799,"start":0,"docs":[]
> >> },
> >> "debug":{
> >>   "timing":{
> >>     "time":16620.0,
> >>     "prepare":{
> >>       "time":0.0,
> >>       "query":{
> >>         "time":0.0},
> >>       "facet":{
> >>         "time":0.0},
> >>       "facet_module":{
> >>         "time":0.0},
> >>       "mlt":{
> >>         "time":0.0},
> >>       "highlight":{
> >>         "time":0.0},
> >>       "stats":{
> >>         "time":0.0},
> >>       "expand":{
> >>         "time":0.0},
> >>       "terms":{
> >>         "time":0.0},
> >>       "block-expensive-queries":{
> >>         "time":0.0},
> >>       "slow-query-logger":{
> >>         "time":0.0},
> >>       "debug":{
> >>         "time":0.0}},
> >>     "process":{
> >>       "time":16619.0,
> >>       "query":{
> >>         "time":16619.0},
> >>       "facet":{
> >>         "time":0.0},
> >>       "facet_module":{
> >>         "time":0.0},
> >>       "mlt":{
> >>         "time":0.0},
> >>       "highlight":{
> >>         "time":0.0},
> >>       "stats":{
> >>         "time":0.0},
> >>       "expand":{
> >>         "time":0.0},
> >>       "terms":{
> >>         "time":0.0},
> >>       "block-expensive-queries":{
> >>         "time":0.0},
> >>       "slow-query-logger":{
> >>         "time":0.0},
> >>       "debug":{
> >>         "time":0.0}}}}}
> >>
> >>
> >> ----snap----
> >>
> >> can we use query: _root_:abc in the ping request handler ? Tried this
> query
> >> and its returning the results within few miliseconds and also the nodes
> are
> >> able to recover without any issue.
> >>
> >> we want to use _root_ field for querying as this field is available in
> all
> >> our clusters with below definition:
> >> <field name="_root_" type="string" omitNorms="true" indexed="true"
> >> termOffsets="false" stored="false" termPayloads="false" termPositions=
> >> "false" docValues="false" termVectors="false"/>
> >> Could you please let me know if using _root_ for querying in
> >> pingRequestHandler will cause any problem?
> >>
> >> <requestHandler name="/admin/ping" class="solr.PingRequestHandler"> <lst
> >> name="invariants"> <str name="qt">/select</str><!-- handler to delegate
> to
> >> --> <str name="q">_root_:abc</str> </lst> </requestHandler>
> >>
> >>
> >> --
> >> Best Regards,
> >> Dinesh Naik
>
>

Re: Solr 7.6.0: PingRequestHandler - Changing the default query (*:*)

Posted by Erick Erickson <er...@gmail.com>.
You can also (I think) explicitly define the ping request handler in solrconfig.xml to do something else.

> On Aug 2, 2019, at 9:50 AM, Jörn Franke <jo...@gmail.com> wrote:
> 
> Not sure if this is possible, but why not create a query handler in Solr with any custom query and you use that as ping replacement ?
> 
>> Am 02.08.2019 um 15:48 schrieb dinesh naik <di...@gmail.com>:
>> 
>> Hi all,
>> I have few clusters with huge data set and whenever a node goes down its
>> not able to recover due to below reasons:
>> 
>> 1. ping request handler is taking more than 10-15 seconds to respond. The
>> ping requesthandler however, expects it will return in less than 1 second
>> and fails a requestrecovery if it is not responded to in this time.
>> Therefore recoveries never would start.
>> 
>> 2. soft commit is very low ie. 5 sec. This is a business requirement so
>> not much can be done here.
>> 
>> As the standard/default admin/ping request handler is using *:* queries ,
>> the response time is much higher, and i am looking for an option to change
>> the same so that the ping handler returns the results within few
>> miliseconds.
>> 
>> here is an example for standard query time:
>> 
>> ----snip---
>> curl "
>> http://hostname:8983/solr/parts/select?indent=on&q=*:*&rows=0&wt=json&distrib=false&debug=timing
>> "
>> {
>> "responseHeader":{
>>   "zkConnected":true,
>>   "status":0,
>>   "QTime":16620,
>>   "params":{
>>     "q":"*:*",
>>     "distrib":"false",
>>     "debug":"timing",
>>     "indent":"on",
>>     "rows":"0",
>>     "wt":"json"}},
>> "response":{"numFound":1329638799,"start":0,"docs":[]
>> },
>> "debug":{
>>   "timing":{
>>     "time":16620.0,
>>     "prepare":{
>>       "time":0.0,
>>       "query":{
>>         "time":0.0},
>>       "facet":{
>>         "time":0.0},
>>       "facet_module":{
>>         "time":0.0},
>>       "mlt":{
>>         "time":0.0},
>>       "highlight":{
>>         "time":0.0},
>>       "stats":{
>>         "time":0.0},
>>       "expand":{
>>         "time":0.0},
>>       "terms":{
>>         "time":0.0},
>>       "block-expensive-queries":{
>>         "time":0.0},
>>       "slow-query-logger":{
>>         "time":0.0},
>>       "debug":{
>>         "time":0.0}},
>>     "process":{
>>       "time":16619.0,
>>       "query":{
>>         "time":16619.0},
>>       "facet":{
>>         "time":0.0},
>>       "facet_module":{
>>         "time":0.0},
>>       "mlt":{
>>         "time":0.0},
>>       "highlight":{
>>         "time":0.0},
>>       "stats":{
>>         "time":0.0},
>>       "expand":{
>>         "time":0.0},
>>       "terms":{
>>         "time":0.0},
>>       "block-expensive-queries":{
>>         "time":0.0},
>>       "slow-query-logger":{
>>         "time":0.0},
>>       "debug":{
>>         "time":0.0}}}}}
>> 
>> 
>> ----snap----
>> 
>> can we use query: _root_:abc in the ping request handler ? Tried this query
>> and its returning the results within few miliseconds and also the nodes are
>> able to recover without any issue.
>> 
>> we want to use _root_ field for querying as this field is available in all
>> our clusters with below definition:
>> <field name="_root_" type="string" omitNorms="true" indexed="true"
>> termOffsets="false" stored="false" termPayloads="false" termPositions=
>> "false" docValues="false" termVectors="false"/>
>> Could you please let me know if using _root_ for querying in
>> pingRequestHandler will cause any problem?
>> 
>> <requestHandler name="/admin/ping" class="solr.PingRequestHandler"> <lst
>> name="invariants"> <str name="qt">/select</str><!-- handler to delegate to
>> --> <str name="q">_root_:abc</str> </lst> </requestHandler>
>> 
>> 
>> -- 
>> Best Regards,
>> Dinesh Naik


Re: Solr 7.6.0: PingRequestHandler - Changing the default query (*:*)

Posted by Jörn Franke <jo...@gmail.com>.
 Not sure if this is possible, but why not create a query handler in Solr with any custom query and you use that as ping replacement ?

> Am 02.08.2019 um 15:48 schrieb dinesh naik <di...@gmail.com>:
> 
> Hi all,
> I have few clusters with huge data set and whenever a node goes down its
> not able to recover due to below reasons:
> 
>  1. ping request handler is taking more than 10-15 seconds to respond. The
> ping requesthandler however, expects it will return in less than 1 second
> and fails a requestrecovery if it is not responded to in this time.
> Therefore recoveries never would start.
> 
>  2. soft commit is very low ie. 5 sec. This is a business requirement so
> not much can be done here.
> 
> As the standard/default admin/ping request handler is using *:* queries ,
> the response time is much higher, and i am looking for an option to change
> the same so that the ping handler returns the results within few
> miliseconds.
> 
> here is an example for standard query time:
> 
> ----snip---
> curl "
> http://hostname:8983/solr/parts/select?indent=on&q=*:*&rows=0&wt=json&distrib=false&debug=timing
> "
> {
>  "responseHeader":{
>    "zkConnected":true,
>    "status":0,
>    "QTime":16620,
>    "params":{
>      "q":"*:*",
>      "distrib":"false",
>      "debug":"timing",
>      "indent":"on",
>      "rows":"0",
>      "wt":"json"}},
>  "response":{"numFound":1329638799,"start":0,"docs":[]
>  },
>  "debug":{
>    "timing":{
>      "time":16620.0,
>      "prepare":{
>        "time":0.0,
>        "query":{
>          "time":0.0},
>        "facet":{
>          "time":0.0},
>        "facet_module":{
>          "time":0.0},
>        "mlt":{
>          "time":0.0},
>        "highlight":{
>          "time":0.0},
>        "stats":{
>          "time":0.0},
>        "expand":{
>          "time":0.0},
>        "terms":{
>          "time":0.0},
>        "block-expensive-queries":{
>          "time":0.0},
>        "slow-query-logger":{
>          "time":0.0},
>        "debug":{
>          "time":0.0}},
>      "process":{
>        "time":16619.0,
>        "query":{
>          "time":16619.0},
>        "facet":{
>          "time":0.0},
>        "facet_module":{
>          "time":0.0},
>        "mlt":{
>          "time":0.0},
>        "highlight":{
>          "time":0.0},
>        "stats":{
>          "time":0.0},
>        "expand":{
>          "time":0.0},
>        "terms":{
>          "time":0.0},
>        "block-expensive-queries":{
>          "time":0.0},
>        "slow-query-logger":{
>          "time":0.0},
>        "debug":{
>          "time":0.0}}}}}
> 
> 
> ----snap----
> 
> can we use query: _root_:abc in the ping request handler ? Tried this query
> and its returning the results within few miliseconds and also the nodes are
> able to recover without any issue.
> 
> we want to use _root_ field for querying as this field is available in all
> our clusters with below definition:
> <field name="_root_" type="string" omitNorms="true" indexed="true"
> termOffsets="false" stored="false" termPayloads="false" termPositions=
> "false" docValues="false" termVectors="false"/>
> Could you please let me know if using _root_ for querying in
> pingRequestHandler will cause any problem?
> 
> <requestHandler name="/admin/ping" class="solr.PingRequestHandler"> <lst
> name="invariants"> <str name="qt">/select</str><!-- handler to delegate to
> --> <str name="q">_root_:abc</str> </lst> </requestHandler>
> 
> 
> -- 
> Best Regards,
> Dinesh Naik