You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jean-Sebastien Vachon <je...@wantedanalytics.com> on 2013/05/23 16:10:31 UTC
Problem with document routing with Solr 4.2.1
Hi All,
I just started indexing data in my brand new Solr Cloud running on 4.2.1.
Since I am a big user of the grouping feature, I need to route my documents on the proper shard.
Following the instruction found here:
http://docs.lucidworks.com/display/solr/Shards+and+Indexing+Data+in+SolrCloud
I set my document id to something like this 'fieldA!id' where fieldA is the key I want to use to distribute my documents.
(All documents with the same value for fieldA will be sent to the same shard).
When I query my index, I can see that the number of documents increase but there are no fields at all in the index.
http://10.0.5.211:8201/solr/Current/select?q=*:*
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">11</int>
<lst name="params">
<str name="q">*:*</str>
</lst>
</lst>
<result name="response" numFound="26318" start="0" maxScore="1.0"/>
</response>
Specifying fields in the 'fl' parameter does nothing.
What am I doing wrong?
RE: Problem with document routing with Solr 4.2.1
Posted by Jean-Sebastien Vachon <je...@wantedanalytics.com>.
Hi All,
Evan Sayer from LucidWorks found the problem in our schema so this problem is not related at all to SolrCloud itself. (well it is but as least it is not a bug)
I don't why :( but at some point we changed the type of the id field from 'string' to 'text'.
Since we are doing custom hashing and that the id field was tokenized, Solr could not find back documents when collecting responses from each shards.
We changed back the id field to the 'string' type and it is now working
-----Original Message-----
From: Jean-Sebastien Vachon [mailto:jean-sebastien.vachon@wantedanalytics.com]
Sent: May-23-13 2:57 PM
To: solr-user@lucene.apache.org
Subject: RE: Problem with document routing with Solr 4.2.1
I must add the shard.keys=xxxx does not return anything on two on my nodes. But that is to be expected since I'm using a replication factor of 3 on a cloud of 5 servers
-----Original Message-----
From: Jean-Sebastien Vachon [mailto:jean-sebastien.vachon@wantedanalytics.com]
Sent: May-23-13 11:27 AM
To: solr-user@lucene.apache.org
Subject: RE: Problem with document routing with Solr 4.2.1
If that can help.. adding distrib=false or shard.keys=xxxx is giving back results.
-----Original Message-----
From: Jean-Sebastien Vachon [mailto:jean-sebastien.vachon@wantedanalytics.com]
Sent: May-23-13 10:39 AM
To: solr-user@lucene.apache.org
Subject: RE: Problem with document routing with Solr 4.2.1
I know. If a stop routing the documents and simply use a standard 'id' field then I am getting back my fields.
I forgot to tell you how the collection was created.
http://localhost:8201/solr/admin/collections?action=CREATE&name=Current&numShards=15&replicationFactor=3&maxShardsPerNode=9
Since I am using the numshards parameter then composite routing should be working... unless I misunderstood something
-----Original Message-----
From: Shalin Shekhar Mangar [mailto:shalinmangar@gmail.com]
Sent: May-23-13 10:27 AM
To: solr-user@lucene.apache.org
Subject: Re: Problem with document routing with Solr 4.2.1
That's strange. The default value of "rows" param is 10 so you should be getting 10 results back unless your StandardRequestHandler config in solrconfig has set "rows" to 0 or if none of your fields are stored.
On Thu, May 23, 2013 at 7:40 PM, Jean-Sebastien Vachon < jean-sebastien.vachon@wantedanalytics.com> wrote:
> Hi All,
>
> I just started indexing data in my brand new Solr Cloud running on 4.2.1.
> Since I am a big user of the grouping feature, I need to route my
> documents on the proper shard.
> Following the instruction found here:
>
> http://docs.lucidworks.com/display/solr/Shards+and+Indexing+Data+in+So
> lrCloud
>
> I set my document id to something like this 'fieldA!id' where fieldA
> is the key I want to use to distribute my documents.
> (All documents with the same value for fieldA will be sent to the same
> shard).
>
> When I query my index, I can see that the number of documents increase
> but there are no fields at all in the index.
>
> http://10.0.5.211:8201/solr/Current/select?q=*:*
>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">11</int>
> <lst name="params">
> <str name="q">*:*</str>
> </lst>
> </lst>
> <result name="response" numFound="26318" start="0" maxScore="1.0"/>
> </response>
>
> Specifying fields in the 'fl' parameter does nothing.
>
> What am I doing wrong?
>
--
Regards,
Shalin Shekhar Mangar.
-----
Aucun virus trouvé dans ce message.
Analyse effectuée par AVG - www.avg.fr
Version: 2013.0.3336 / Base de données virale: 3162/6319 - Date: 12/05/2013 La Base de données des virus a expiré.
-----
Aucun virus trouvé dans ce message.
Analyse effectuée par AVG - www.avg.fr
Version: 2013.0.3336 / Base de données virale: 3162/6319 - Date: 12/05/2013 La Base de données des virus a expiré.
-----
Aucun virus trouvé dans ce message.
Analyse effectuée par AVG - www.avg.fr
Version: 2013.0.3336 / Base de données virale: 3162/6319 - Date: 12/05/2013 La Base de données des virus a expiré.
-----
Aucun virus trouvé dans ce message.
Analyse effectuée par AVG - www.avg.fr
Version: 2013.0.3336 / Base de données virale: 3162/6319 - Date: 12/05/2013 La Base de données des virus a expiré.
RE: Problem with document routing with Solr 4.2.1
Posted by Jean-Sebastien Vachon <je...@wantedanalytics.com>.
I must add the shard.keys=xxxx does not return anything on two on my nodes. But that is to be expected since I'm using a replication factor of 3 on a cloud of 5 servers
-----Original Message-----
From: Jean-Sebastien Vachon [mailto:jean-sebastien.vachon@wantedanalytics.com]
Sent: May-23-13 11:27 AM
To: solr-user@lucene.apache.org
Subject: RE: Problem with document routing with Solr 4.2.1
If that can help.. adding distrib=false or shard.keys=xxxx is giving back results.
-----Original Message-----
From: Jean-Sebastien Vachon [mailto:jean-sebastien.vachon@wantedanalytics.com]
Sent: May-23-13 10:39 AM
To: solr-user@lucene.apache.org
Subject: RE: Problem with document routing with Solr 4.2.1
I know. If a stop routing the documents and simply use a standard 'id' field then I am getting back my fields.
I forgot to tell you how the collection was created.
http://localhost:8201/solr/admin/collections?action=CREATE&name=Current&numShards=15&replicationFactor=3&maxShardsPerNode=9
Since I am using the numshards parameter then composite routing should be working... unless I misunderstood something
-----Original Message-----
From: Shalin Shekhar Mangar [mailto:shalinmangar@gmail.com]
Sent: May-23-13 10:27 AM
To: solr-user@lucene.apache.org
Subject: Re: Problem with document routing with Solr 4.2.1
That's strange. The default value of "rows" param is 10 so you should be getting 10 results back unless your StandardRequestHandler config in solrconfig has set "rows" to 0 or if none of your fields are stored.
On Thu, May 23, 2013 at 7:40 PM, Jean-Sebastien Vachon < jean-sebastien.vachon@wantedanalytics.com> wrote:
> Hi All,
>
> I just started indexing data in my brand new Solr Cloud running on 4.2.1.
> Since I am a big user of the grouping feature, I need to route my
> documents on the proper shard.
> Following the instruction found here:
>
> http://docs.lucidworks.com/display/solr/Shards+and+Indexing+Data+in+So
> lrCloud
>
> I set my document id to something like this 'fieldA!id' where fieldA
> is the key I want to use to distribute my documents.
> (All documents with the same value for fieldA will be sent to the same
> shard).
>
> When I query my index, I can see that the number of documents increase
> but there are no fields at all in the index.
>
> http://10.0.5.211:8201/solr/Current/select?q=*:*
>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">11</int>
> <lst name="params">
> <str name="q">*:*</str>
> </lst>
> </lst>
> <result name="response" numFound="26318" start="0" maxScore="1.0"/>
> </response>
>
> Specifying fields in the 'fl' parameter does nothing.
>
> What am I doing wrong?
>
--
Regards,
Shalin Shekhar Mangar.
-----
Aucun virus trouvé dans ce message.
Analyse effectuée par AVG - www.avg.fr
Version: 2013.0.3336 / Base de données virale: 3162/6319 - Date: 12/05/2013 La Base de données des virus a expiré.
-----
Aucun virus trouvé dans ce message.
Analyse effectuée par AVG - www.avg.fr
Version: 2013.0.3336 / Base de données virale: 3162/6319 - Date: 12/05/2013 La Base de données des virus a expiré.
-----
Aucun virus trouvé dans ce message.
Analyse effectuée par AVG - www.avg.fr
Version: 2013.0.3336 / Base de données virale: 3162/6319 - Date: 12/05/2013 La Base de données des virus a expiré.
RE: Problem with document routing with Solr 4.2.1
Posted by Jean-Sebastien Vachon <je...@wantedanalytics.com>.
If that can help.. adding distrib=false or shard.keys=xxxx is giving back results.
-----Original Message-----
From: Jean-Sebastien Vachon [mailto:jean-sebastien.vachon@wantedanalytics.com]
Sent: May-23-13 10:39 AM
To: solr-user@lucene.apache.org
Subject: RE: Problem with document routing with Solr 4.2.1
I know. If a stop routing the documents and simply use a standard 'id' field then I am getting back my fields.
I forgot to tell you how the collection was created.
http://localhost:8201/solr/admin/collections?action=CREATE&name=Current&numShards=15&replicationFactor=3&maxShardsPerNode=9
Since I am using the numshards parameter then composite routing should be working... unless I misunderstood something
-----Original Message-----
From: Shalin Shekhar Mangar [mailto:shalinmangar@gmail.com]
Sent: May-23-13 10:27 AM
To: solr-user@lucene.apache.org
Subject: Re: Problem with document routing with Solr 4.2.1
That's strange. The default value of "rows" param is 10 so you should be getting 10 results back unless your StandardRequestHandler config in solrconfig has set "rows" to 0 or if none of your fields are stored.
On Thu, May 23, 2013 at 7:40 PM, Jean-Sebastien Vachon < jean-sebastien.vachon@wantedanalytics.com> wrote:
> Hi All,
>
> I just started indexing data in my brand new Solr Cloud running on 4.2.1.
> Since I am a big user of the grouping feature, I need to route my
> documents on the proper shard.
> Following the instruction found here:
>
> http://docs.lucidworks.com/display/solr/Shards+and+Indexing+Data+in+So
> lrCloud
>
> I set my document id to something like this 'fieldA!id' where fieldA
> is the key I want to use to distribute my documents.
> (All documents with the same value for fieldA will be sent to the same
> shard).
>
> When I query my index, I can see that the number of documents increase
> but there are no fields at all in the index.
>
> http://10.0.5.211:8201/solr/Current/select?q=*:*
>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">11</int>
> <lst name="params">
> <str name="q">*:*</str>
> </lst>
> </lst>
> <result name="response" numFound="26318" start="0" maxScore="1.0"/>
> </response>
>
> Specifying fields in the 'fl' parameter does nothing.
>
> What am I doing wrong?
>
--
Regards,
Shalin Shekhar Mangar.
-----
Aucun virus trouvé dans ce message.
Analyse effectuée par AVG - www.avg.fr
Version: 2013.0.3336 / Base de données virale: 3162/6319 - Date: 12/05/2013 La Base de données des virus a expiré.
-----
Aucun virus trouvé dans ce message.
Analyse effectuée par AVG - www.avg.fr
Version: 2013.0.3336 / Base de données virale: 3162/6319 - Date: 12/05/2013 La Base de données des virus a expiré.
RE: Problem with document routing with Solr 4.2.1
Posted by Jean-Sebastien Vachon <je...@wantedanalytics.com>.
I know. If a stop routing the documents and simply use a standard 'id' field then I am getting back my fields.
I forgot to tell you how the collection was created.
http://localhost:8201/solr/admin/collections?action=CREATE&name=Current&numShards=15&replicationFactor=3&maxShardsPerNode=9
Since I am using the numshards parameter then composite routing should be working... unless I misunderstood something
-----Original Message-----
From: Shalin Shekhar Mangar [mailto:shalinmangar@gmail.com]
Sent: May-23-13 10:27 AM
To: solr-user@lucene.apache.org
Subject: Re: Problem with document routing with Solr 4.2.1
That's strange. The default value of "rows" param is 10 so you should be getting 10 results back unless your StandardRequestHandler config in solrconfig has set "rows" to 0 or if none of your fields are stored.
On Thu, May 23, 2013 at 7:40 PM, Jean-Sebastien Vachon < jean-sebastien.vachon@wantedanalytics.com> wrote:
> Hi All,
>
> I just started indexing data in my brand new Solr Cloud running on 4.2.1.
> Since I am a big user of the grouping feature, I need to route my
> documents on the proper shard.
> Following the instruction found here:
>
> http://docs.lucidworks.com/display/solr/Shards+and+Indexing+Data+in+So
> lrCloud
>
> I set my document id to something like this 'fieldA!id' where fieldA
> is the key I want to use to distribute my documents.
> (All documents with the same value for fieldA will be sent to the same
> shard).
>
> When I query my index, I can see that the number of documents increase
> but there are no fields at all in the index.
>
> http://10.0.5.211:8201/solr/Current/select?q=*:*
>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">11</int>
> <lst name="params">
> <str name="q">*:*</str>
> </lst>
> </lst>
> <result name="response" numFound="26318" start="0" maxScore="1.0"/>
> </response>
>
> Specifying fields in the 'fl' parameter does nothing.
>
> What am I doing wrong?
>
--
Regards,
Shalin Shekhar Mangar.
-----
Aucun virus trouvé dans ce message.
Analyse effectuée par AVG - www.avg.fr
Version: 2013.0.3336 / Base de données virale: 3162/6319 - Date: 12/05/2013 La Base de données des virus a expiré.
Re: Problem with document routing with Solr 4.2.1
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
That's strange. The default value of "rows" param is 10 so you should be
getting 10 results back unless your StandardRequestHandler config in
solrconfig has set "rows" to 0 or if none of your fields are stored.
On Thu, May 23, 2013 at 7:40 PM, Jean-Sebastien Vachon <
jean-sebastien.vachon@wantedanalytics.com> wrote:
> Hi All,
>
> I just started indexing data in my brand new Solr Cloud running on 4.2.1.
> Since I am a big user of the grouping feature, I need to route my
> documents on the proper shard.
> Following the instruction found here:
>
> http://docs.lucidworks.com/display/solr/Shards+and+Indexing+Data+in+SolrCloud
>
> I set my document id to something like this 'fieldA!id' where fieldA is
> the key I want to use to distribute my documents.
> (All documents with the same value for fieldA will be sent to the same
> shard).
>
> When I query my index, I can see that the number of documents increase but
> there are no fields at all in the index.
>
> http://10.0.5.211:8201/solr/Current/select?q=*:*
>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">11</int>
> <lst name="params">
> <str name="q">*:*</str>
> </lst>
> </lst>
> <result name="response" numFound="26318" start="0" maxScore="1.0"/>
> </response>
>
> Specifying fields in the 'fl' parameter does nothing.
>
> What am I doing wrong?
>
--
Regards,
Shalin Shekhar Mangar.