Non Deterministic Results from /admin/luke

["Nussbaum, Ronen" <Ro...@verint.com>]

Hi All,

We are using the Luke API in order to get all dynamic field names from our collection:
/solr/collection/admin/luke?wt=csv&numTerms=0

This worked fine in 6.2.1 but it's non deterministic anymore (8.6.1) - looks like it queries a random single shard.

I've tried using /solr/collection/select?q=*:*&wt=csv&rows=0&facet but it behaves the same.

Can it be configured to query all shards?
Is there another way to achieve this?

Thanks in advance,
Ronen.




This electronic message may contain proprietary and confidential information of Verint Systems Inc., its affiliates and/or subsidiaries. The information is intended to be for the use of the individual(s) or entity(ies) named above. If you are not the intended recipient (or authorized to receive this e-mail for the intended recipient), you may not use, copy, disclose or distribute to anyone this message or any information contained in this message. If you have received this electronic message in error, please notify us by replying to this e-mail.

Re: Non Deterministic Results from /admin/luke

[Andrzej Białecki <ab...@getopt.org>]

You may want to check the COLSTATUS collection command added in 8.1 (https://lucene.apache.org/solr/guide/8_6/collection-management.html#colstatus <https://lucene.apache.org/solr/guide/8_6/collection-management.html#colstatus>).

This reports much of the information returned by /admin/luke but can also report this for all shard leaders in a collection.

> On 2 Oct 2020, at 01:06, Shawn Heisey <ap...@elyograg.org> wrote:
> 
> On 10/1/2020 4:24 AM, Nussbaum, Ronen wrote:
>> We are using the Luke API in order to get all dynamic field names from our collection:
>> /solr/collection/admin/luke?wt=csv&numTerms=0
>> This worked fine in 6.2.1 but it's non deterministic anymore (8.6.1) - looks like it queries a random single shard.
>> I've tried using /solr/collection/select?q=*:*&wt=csv&rows=0&facet but it behaves the same.
>> Can it be configured to query all shards?
>> Is there another way to achieve this?
> 
> The Luke handler (usually at /admin/luke) is not SolrCloud aware.  It is designed to operate on a single core.  So if you send the request to the collection and not a specific core, Solr must forward the request to a core in order for you to get ANY result.  The core selection will be random.
> 
> The software called Luke (which is where the Luke handler gets its name) operates on a Lucene index -- each Solr core is based around a Lucene index.  It would be a LOT of work to make the handler SolrCloud aware.
> 
> Depending on how your collection is set up, you may need to query the Luke handler on multiple cores in order to get a full picture of all fields present in the Lucene indexes.  I am not aware of any other way to do it.
> 
> Thanks,
> Shawn
> 

Re: Non Deterministic Results from /admin/luke

[Shawn Heisey <ap...@elyograg.org>]

On 10/1/2020 4:24 AM, Nussbaum, Ronen wrote:
> We are using the Luke API in order to get all dynamic field names from our collection:
> /solr/collection/admin/luke?wt=csv&numTerms=0
> 
> This worked fine in 6.2.1 but it's non deterministic anymore (8.6.1) - looks like it queries a random single shard.
> 
> I've tried using /solr/collection/select?q=*:*&wt=csv&rows=0&facet but it behaves the same.
> 
> Can it be configured to query all shards?
> Is there another way to achieve this?

The Luke handler (usually at /admin/luke) is not SolrCloud aware.  It is 
designed to operate on a single core.  So if you send the request to the 
collection and not a specific core, Solr must forward the request to a 
core in order for you to get ANY result.  The core selection will be random.

The software called Luke (which is where the Luke handler gets its name) 
operates on a Lucene index -- each Solr core is based around a Lucene 
index.  It would be a LOT of work to make the handler SolrCloud aware.

Depending on how your collection is set up, you may need to query the 
Luke handler on multiple cores in order to get a full picture of all 
fields present in the Lucene indexes.  I am not aware of any other way 
to do it.

Thanks,
Shawn