You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Naresh Yadav <ny...@gmail.com> on 2015/01/13 13:33:37 UTC
Solr grouping problem - need help
*Schema :*
<field name="tenant_pool" type="text" stored="true"/>
*Code :*
SolrQuery q = new SolrQuery().setQuery("*:*");
q.set(GroupParams.GROUP, true);
q.set(GroupParams.GROUP_FIELD, "tenant_pool");
*Data :*
"tenant_pool" : "Baroda Farms"
"tenant_pool" : "Ketty Farms"
*Output coming :*
groupValue=Farms, docs=2
*Expected Output :*
groupValue=Baroda Farms, docs=1
groupValue=Ketty Farms, docs=1
Please guide me how i can tell solr not to tokenize stored field to decide
unique groups..
I want unique groups as exact value of field not the tokens which solr is
doing
currently.
Thanks
Naresh
Re: Solr grouping problem - need help
Posted by Naresh Yadav <ny...@gmail.com>.
as my problem is changed completely from first post so i had created new
thread for that.
On Wed, Jan 14, 2015 at 7:31 PM, Naresh Yadav <ny...@gmail.com> wrote:
> just wanted to share schema and results for same :
>
> solr version : 4.6.1
> Schema : http://www.imagesup.net/?di=10142124357616
> Code : http://www.imagesup.net/?di=10142124381116
> Response Group : http://www.imagesup.net/?di=1114212438351
> Response Terms : http://www.imagesup.net/?di=614212438580
>
> Please help me on this problem where no of groups are not matching with no
> of terms which is expected behaviour acc to me.
> Please give direction on this problem.
>
>
> On Wed, Jan 14, 2015 at 5:24 PM, Naresh Yadav <ny...@gmail.com>
> wrote:
>
>> I tried what you said also appended group.ngroups=true and got same
>> result not expected one....ngroups coming is 1.
>> i am on solr-4.6.1 single machine default setup.
>>
>>
>> On Wed, Jan 14, 2015 at 4:43 PM, Norgorn <ls...@mail.ru> wrote:
>>
>>> Can u get raw SOLR response?
>>>
>>> For me grouping works exactly the way u expect it to work.
>>>
>>> Try direct query in browser to be sure the problem is not in your code.
>>>
>>> http://192.168.0.1:8983/solr/collection1/select?q=*:*&group=true&group.field=tenant_pool
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://lucene.472066.n3.nabble.com/Solr-grouping-problem-need-help-tp4179149p4179464.html
>>> Sent from the Solr - User mailing list archive at Nabble.com.
>>>
>>
>>
>>
>>
>
>
>
>
Re: Solr grouping problem - need help
Posted by Naresh Yadav <ny...@gmail.com>.
just wanted to share schema and results for same :
solr version : 4.6.1
Schema : http://www.imagesup.net/?di=10142124357616
Code : http://www.imagesup.net/?di=10142124381116
Response Group : http://www.imagesup.net/?di=1114212438351
Response Terms : http://www.imagesup.net/?di=614212438580
Please help me on this problem where no of groups are not matching with no
of terms which is expected behaviour acc to me.
Please give direction on this problem.
On Wed, Jan 14, 2015 at 5:24 PM, Naresh Yadav <ny...@gmail.com> wrote:
> I tried what you said also appended group.ngroups=true and got same result
> not expected one....ngroups coming is 1.
> i am on solr-4.6.1 single machine default setup.
>
>
> On Wed, Jan 14, 2015 at 4:43 PM, Norgorn <ls...@mail.ru> wrote:
>
>> Can u get raw SOLR response?
>>
>> For me grouping works exactly the way u expect it to work.
>>
>> Try direct query in browser to be sure the problem is not in your code.
>>
>> http://192.168.0.1:8983/solr/collection1/select?q=*:*&group=true&group.field=tenant_pool
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Solr-grouping-problem-need-help-tp4179149p4179464.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>
>
>
>
Re: Solr grouping problem - need help
Posted by Naresh Yadav <ny...@gmail.com>.
I tried what you said also appended group.ngroups=true and got same result
not expected one....ngroups coming is 1.
i am on solr-4.6.1 single machine default setup.
On Wed, Jan 14, 2015 at 4:43 PM, Norgorn <ls...@mail.ru> wrote:
> Can u get raw SOLR response?
>
> For me grouping works exactly the way u expect it to work.
>
> Try direct query in browser to be sure the problem is not in your code.
>
> http://192.168.0.1:8983/solr/collection1/select?q=*:*&group=true&group.field=tenant_pool
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-grouping-problem-need-help-tp4179149p4179464.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Solr grouping problem - need help
Posted by Norgorn <ls...@mail.ru>.
Can u get raw SOLR response?
For me grouping works exactly the way u expect it to work.
Try direct query in browser to be sure the problem is not in your code.
http://192.168.0.1:8983/solr/collection1/select?q=*:*&group=true&group.field=tenant_pool
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-grouping-problem-need-help-tp4179149p4179464.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr grouping problem - need help
Posted by Naresh Yadav <ny...@gmail.com>.
Thanks much, now i got better idea on stored & indexed works internally in
solr.
>From this i tried and modified few things to understand grouping logic.
*Schema :*
<field name="tenant_pool" type="text" stored="true" *indexed="true"*/>
<field name="type" type="text" stored="true" indexed="true"/>
*Code :*
SolrQuery q = new SolrQuery().setQuery("type:t1");
q.set(GroupParams.GROUP, true);
q.set(GroupParams.GROUP_FIELD, "tenant_pool");
*Data/Docs :*
"tenant_pool" : "P1 L1", "type" : "t1"
"tenant_pool" : "P1 L1", "type" : "t2"
*Output coming :*
groupValue=L1, docs=2
*Expected Output :*
groupValue=P1, docs=2
groupValue=L1 Farms, docs=2
My understanding is field is indexed so it will tokenized by space and P1
and L1 will be tokens..Each token should be one group
when we call Group=true in query. Please help me understand this better.
Thanks
Naresh
On Wed, Jan 14, 2015 at 1:32 AM, Erick Erickson <er...@gmail.com>
wrote:
> bq: My question is for indexed=false, stored=true field..what is optimized
> way
> to get unique values in such field.
>
> There isn't any. To do this you'll have to read the doc from disk,
> it'll be decompressed
> along the way and then the field is read. Note that this happens
> automatically when
> you call doc.getFieldValue or similar.
>
> At the stored="true" level, you're always talking about complete documents.
> indexed="true" is about putting the field data into efficient-access
> structures.
> They're completely different beasts.
>
> your original question was:
> "Please guide me how i can tell solr not to tokenize stored field to decide
> unique groups.."
>
> Simply declare the field type you care about as a "string" type in
> schema.xml. The use a <copyFeld> directive to copy the data to the
> new type, and group on the new field.
>
> There are examples in the schema.xml of string types and copyFields that
> should help.
>
> Best,
> Erick
>
> On Tue, Jan 13, 2015 at 9:00 AM, Naresh Yadav <ny...@gmail.com>
> wrote:
> > Erick, my schema is same no change in that..
> > *Schema :*
> > <field name="tenant_pool" type="text" stored="true"/>
> > my guess is i had not mentioned indexed true or false....may be default
> > indexed is true....
> >
> > My question is for indexed=false, stored=true field..what is optimized
> way
> > to get unique values in such field..
> >
> > On Tue, Jan 13, 2015 at 10:07 PM, Erick Erickson <
> erickerickson@gmail.com>
> > wrote:
> >
> >> Something is very wrong here. Have you perhaps been changing your
> >> schema without re-indexing? And I recommend you completely remove
> >> your data directory (the one with "index" and "tlog" subdirectories)
> after
> >> you change your schema.xml file.
> >>
> >> Because you're trying to group on a field that is _not_ indexed, you
> >> should be getting an error returned, something like:
> >> "can not use FieldCache on a field which is neither indexed nor has
> >> doc values: ...."
> >>
> >> As far as the tokenization comment is, just start by making the field
> you
> >> want
> >> to group on be
> >> stored="false" indexed="true" type="string"
> >>
> >> Best,
> >> Erick
> >>
> >> On Tue, Jan 13, 2015 at 5:09 AM, Naresh Yadav <ny...@gmail.com>
> >> wrote:
> >> > Hi jack,
> >> >
> >> > Thanks for replying, i am new to solr please guide me on this. I have
> >> many
> >> > such columns in my schema
> >> > so copy field will create lot of duplicate fields beside i do not need
> >> any
> >> > search on original field.
> >> >
> >> > My usecase is i do not want any search on tenant_pool field thats why
> i
> >> > declared it as stored field not indexed.
> >> > I just need to get unique values in this field. Please show some
> >> direction.
> >> >
> >> >
> >> > On Tue, Jan 13, 2015 at 6:16 PM, Jack Krupansky <
> >> jack.krupansky@gmail.com>
> >> > wrote:
> >> >
> >> >> That's your job. The easiest way is to do a copyField to a "string"
> >> field.
> >> >>
> >> >> -- Jack Krupansky
> >> >>
> >> >> On Tue, Jan 13, 2015 at 7:33 AM, Naresh Yadav <ny...@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > *Schema :*
> >> >> > <field name="tenant_pool" type="text" stored="true"/>
> >> >> >
> >> >> > *Code :*
> >> >> > SolrQuery q = new SolrQuery().setQuery("*:*");
> >> >> > q.set(GroupParams.GROUP, true);
> >> >> > q.set(GroupParams.GROUP_FIELD, "tenant_pool");
> >> >> >
> >> >> > *Data :*
> >> >> > "tenant_pool" : "Baroda Farms"
> >> >> > "tenant_pool" : "Ketty Farms"
> >> >> >
> >> >> > *Output coming :*
> >> >> > groupValue=Farms, docs=2
> >> >> >
> >> >> > *Expected Output :*
> >> >> > groupValue=Baroda Farms, docs=1
> >> >> > groupValue=Ketty Farms, docs=1
> >> >> >
> >> >> > Please guide me how i can tell solr not to tokenize stored field to
> >> >> decide
> >> >> > unique groups..
> >> >> >
> >> >> > I want unique groups as exact value of field not the tokens which
> >> solr is
> >> >> > doing
> >> >> > currently.
> >> >> >
> >> >> > Thanks
> >> >> > Naresh
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >> >
> >>
>
Re: Solr grouping problem - need help
Posted by Erick Erickson <er...@gmail.com>.
bq: My question is for indexed=false, stored=true field..what is optimized way
to get unique values in such field.
There isn't any. To do this you'll have to read the doc from disk,
it'll be decompressed
along the way and then the field is read. Note that this happens
automatically when
you call doc.getFieldValue or similar.
At the stored="true" level, you're always talking about complete documents.
indexed="true" is about putting the field data into efficient-access structures.
They're completely different beasts.
your original question was:
"Please guide me how i can tell solr not to tokenize stored field to decide
unique groups.."
Simply declare the field type you care about as a "string" type in
schema.xml. The use a <copyFeld> directive to copy the data to the
new type, and group on the new field.
There are examples in the schema.xml of string types and copyFields that
should help.
Best,
Erick
On Tue, Jan 13, 2015 at 9:00 AM, Naresh Yadav <ny...@gmail.com> wrote:
> Erick, my schema is same no change in that..
> *Schema :*
> <field name="tenant_pool" type="text" stored="true"/>
> my guess is i had not mentioned indexed true or false....may be default
> indexed is true....
>
> My question is for indexed=false, stored=true field..what is optimized way
> to get unique values in such field..
>
> On Tue, Jan 13, 2015 at 10:07 PM, Erick Erickson <er...@gmail.com>
> wrote:
>
>> Something is very wrong here. Have you perhaps been changing your
>> schema without re-indexing? And I recommend you completely remove
>> your data directory (the one with "index" and "tlog" subdirectories) after
>> you change your schema.xml file.
>>
>> Because you're trying to group on a field that is _not_ indexed, you
>> should be getting an error returned, something like:
>> "can not use FieldCache on a field which is neither indexed nor has
>> doc values: ...."
>>
>> As far as the tokenization comment is, just start by making the field you
>> want
>> to group on be
>> stored="false" indexed="true" type="string"
>>
>> Best,
>> Erick
>>
>> On Tue, Jan 13, 2015 at 5:09 AM, Naresh Yadav <ny...@gmail.com>
>> wrote:
>> > Hi jack,
>> >
>> > Thanks for replying, i am new to solr please guide me on this. I have
>> many
>> > such columns in my schema
>> > so copy field will create lot of duplicate fields beside i do not need
>> any
>> > search on original field.
>> >
>> > My usecase is i do not want any search on tenant_pool field thats why i
>> > declared it as stored field not indexed.
>> > I just need to get unique values in this field. Please show some
>> direction.
>> >
>> >
>> > On Tue, Jan 13, 2015 at 6:16 PM, Jack Krupansky <
>> jack.krupansky@gmail.com>
>> > wrote:
>> >
>> >> That's your job. The easiest way is to do a copyField to a "string"
>> field.
>> >>
>> >> -- Jack Krupansky
>> >>
>> >> On Tue, Jan 13, 2015 at 7:33 AM, Naresh Yadav <ny...@gmail.com>
>> >> wrote:
>> >>
>> >> > *Schema :*
>> >> > <field name="tenant_pool" type="text" stored="true"/>
>> >> >
>> >> > *Code :*
>> >> > SolrQuery q = new SolrQuery().setQuery("*:*");
>> >> > q.set(GroupParams.GROUP, true);
>> >> > q.set(GroupParams.GROUP_FIELD, "tenant_pool");
>> >> >
>> >> > *Data :*
>> >> > "tenant_pool" : "Baroda Farms"
>> >> > "tenant_pool" : "Ketty Farms"
>> >> >
>> >> > *Output coming :*
>> >> > groupValue=Farms, docs=2
>> >> >
>> >> > *Expected Output :*
>> >> > groupValue=Baroda Farms, docs=1
>> >> > groupValue=Ketty Farms, docs=1
>> >> >
>> >> > Please guide me how i can tell solr not to tokenize stored field to
>> >> decide
>> >> > unique groups..
>> >> >
>> >> > I want unique groups as exact value of field not the tokens which
>> solr is
>> >> > doing
>> >> > currently.
>> >> >
>> >> > Thanks
>> >> > Naresh
>> >> >
>> >>
>> >
>> >
>> >
>> > --
>> > Cheers,
>> >
>> > Naresh Yadav
>> > +919960523401
>> > http://nareshyadav.blogspot.com/
>> > SSE, MetrixLine Inc.
>>
Re: Solr grouping problem - need help
Posted by Naresh Yadav <ny...@gmail.com>.
Erick, my schema is same no change in that..
*Schema :*
<field name="tenant_pool" type="text" stored="true"/>
my guess is i had not mentioned indexed true or false....may be default
indexed is true....
My question is for indexed=false, stored=true field..what is optimized way
to get unique values in such field..
On Tue, Jan 13, 2015 at 10:07 PM, Erick Erickson <er...@gmail.com>
wrote:
> Something is very wrong here. Have you perhaps been changing your
> schema without re-indexing? And I recommend you completely remove
> your data directory (the one with "index" and "tlog" subdirectories) after
> you change your schema.xml file.
>
> Because you're trying to group on a field that is _not_ indexed, you
> should be getting an error returned, something like:
> "can not use FieldCache on a field which is neither indexed nor has
> doc values: ...."
>
> As far as the tokenization comment is, just start by making the field you
> want
> to group on be
> stored="false" indexed="true" type="string"
>
> Best,
> Erick
>
> On Tue, Jan 13, 2015 at 5:09 AM, Naresh Yadav <ny...@gmail.com>
> wrote:
> > Hi jack,
> >
> > Thanks for replying, i am new to solr please guide me on this. I have
> many
> > such columns in my schema
> > so copy field will create lot of duplicate fields beside i do not need
> any
> > search on original field.
> >
> > My usecase is i do not want any search on tenant_pool field thats why i
> > declared it as stored field not indexed.
> > I just need to get unique values in this field. Please show some
> direction.
> >
> >
> > On Tue, Jan 13, 2015 at 6:16 PM, Jack Krupansky <
> jack.krupansky@gmail.com>
> > wrote:
> >
> >> That's your job. The easiest way is to do a copyField to a "string"
> field.
> >>
> >> -- Jack Krupansky
> >>
> >> On Tue, Jan 13, 2015 at 7:33 AM, Naresh Yadav <ny...@gmail.com>
> >> wrote:
> >>
> >> > *Schema :*
> >> > <field name="tenant_pool" type="text" stored="true"/>
> >> >
> >> > *Code :*
> >> > SolrQuery q = new SolrQuery().setQuery("*:*");
> >> > q.set(GroupParams.GROUP, true);
> >> > q.set(GroupParams.GROUP_FIELD, "tenant_pool");
> >> >
> >> > *Data :*
> >> > "tenant_pool" : "Baroda Farms"
> >> > "tenant_pool" : "Ketty Farms"
> >> >
> >> > *Output coming :*
> >> > groupValue=Farms, docs=2
> >> >
> >> > *Expected Output :*
> >> > groupValue=Baroda Farms, docs=1
> >> > groupValue=Ketty Farms, docs=1
> >> >
> >> > Please guide me how i can tell solr not to tokenize stored field to
> >> decide
> >> > unique groups..
> >> >
> >> > I want unique groups as exact value of field not the tokens which
> solr is
> >> > doing
> >> > currently.
> >> >
> >> > Thanks
> >> > Naresh
> >> >
> >>
> >
> >
> >
> > --
> > Cheers,
> >
> > Naresh Yadav
> > +919960523401
> > http://nareshyadav.blogspot.com/
> > SSE, MetrixLine Inc.
>
Re: Solr grouping problem - need help
Posted by Erick Erickson <er...@gmail.com>.
Something is very wrong here. Have you perhaps been changing your
schema without re-indexing? And I recommend you completely remove
your data directory (the one with "index" and "tlog" subdirectories) after
you change your schema.xml file.
Because you're trying to group on a field that is _not_ indexed, you
should be getting an error returned, something like:
"can not use FieldCache on a field which is neither indexed nor has
doc values: ...."
As far as the tokenization comment is, just start by making the field you want
to group on be
stored="false" indexed="true" type="string"
Best,
Erick
On Tue, Jan 13, 2015 at 5:09 AM, Naresh Yadav <ny...@gmail.com> wrote:
> Hi jack,
>
> Thanks for replying, i am new to solr please guide me on this. I have many
> such columns in my schema
> so copy field will create lot of duplicate fields beside i do not need any
> search on original field.
>
> My usecase is i do not want any search on tenant_pool field thats why i
> declared it as stored field not indexed.
> I just need to get unique values in this field. Please show some direction.
>
>
> On Tue, Jan 13, 2015 at 6:16 PM, Jack Krupansky <ja...@gmail.com>
> wrote:
>
>> That's your job. The easiest way is to do a copyField to a "string" field.
>>
>> -- Jack Krupansky
>>
>> On Tue, Jan 13, 2015 at 7:33 AM, Naresh Yadav <ny...@gmail.com>
>> wrote:
>>
>> > *Schema :*
>> > <field name="tenant_pool" type="text" stored="true"/>
>> >
>> > *Code :*
>> > SolrQuery q = new SolrQuery().setQuery("*:*");
>> > q.set(GroupParams.GROUP, true);
>> > q.set(GroupParams.GROUP_FIELD, "tenant_pool");
>> >
>> > *Data :*
>> > "tenant_pool" : "Baroda Farms"
>> > "tenant_pool" : "Ketty Farms"
>> >
>> > *Output coming :*
>> > groupValue=Farms, docs=2
>> >
>> > *Expected Output :*
>> > groupValue=Baroda Farms, docs=1
>> > groupValue=Ketty Farms, docs=1
>> >
>> > Please guide me how i can tell solr not to tokenize stored field to
>> decide
>> > unique groups..
>> >
>> > I want unique groups as exact value of field not the tokens which solr is
>> > doing
>> > currently.
>> >
>> > Thanks
>> > Naresh
>> >
>>
>
>
>
> --
> Cheers,
>
> Naresh Yadav
> +919960523401
> http://nareshyadav.blogspot.com/
> SSE, MetrixLine Inc.
Re: Solr grouping problem - need help
Posted by Naresh Yadav <ny...@gmail.com>.
Hi jack,
Thanks for replying, i am new to solr please guide me on this. I have many
such columns in my schema
so copy field will create lot of duplicate fields beside i do not need any
search on original field.
My usecase is i do not want any search on tenant_pool field thats why i
declared it as stored field not indexed.
I just need to get unique values in this field. Please show some direction.
On Tue, Jan 13, 2015 at 6:16 PM, Jack Krupansky <ja...@gmail.com>
wrote:
> That's your job. The easiest way is to do a copyField to a "string" field.
>
> -- Jack Krupansky
>
> On Tue, Jan 13, 2015 at 7:33 AM, Naresh Yadav <ny...@gmail.com>
> wrote:
>
> > *Schema :*
> > <field name="tenant_pool" type="text" stored="true"/>
> >
> > *Code :*
> > SolrQuery q = new SolrQuery().setQuery("*:*");
> > q.set(GroupParams.GROUP, true);
> > q.set(GroupParams.GROUP_FIELD, "tenant_pool");
> >
> > *Data :*
> > "tenant_pool" : "Baroda Farms"
> > "tenant_pool" : "Ketty Farms"
> >
> > *Output coming :*
> > groupValue=Farms, docs=2
> >
> > *Expected Output :*
> > groupValue=Baroda Farms, docs=1
> > groupValue=Ketty Farms, docs=1
> >
> > Please guide me how i can tell solr not to tokenize stored field to
> decide
> > unique groups..
> >
> > I want unique groups as exact value of field not the tokens which solr is
> > doing
> > currently.
> >
> > Thanks
> > Naresh
> >
>
--
Cheers,
Naresh Yadav
+919960523401
http://nareshyadav.blogspot.com/
SSE, MetrixLine Inc.
Re: Solr grouping problem - need help
Posted by Jack Krupansky <ja...@gmail.com>.
That's your job. The easiest way is to do a copyField to a "string" field.
-- Jack Krupansky
On Tue, Jan 13, 2015 at 7:33 AM, Naresh Yadav <ny...@gmail.com> wrote:
> *Schema :*
> <field name="tenant_pool" type="text" stored="true"/>
>
> *Code :*
> SolrQuery q = new SolrQuery().setQuery("*:*");
> q.set(GroupParams.GROUP, true);
> q.set(GroupParams.GROUP_FIELD, "tenant_pool");
>
> *Data :*
> "tenant_pool" : "Baroda Farms"
> "tenant_pool" : "Ketty Farms"
>
> *Output coming :*
> groupValue=Farms, docs=2
>
> *Expected Output :*
> groupValue=Baroda Farms, docs=1
> groupValue=Ketty Farms, docs=1
>
> Please guide me how i can tell solr not to tokenize stored field to decide
> unique groups..
>
> I want unique groups as exact value of field not the tokens which solr is
> doing
> currently.
>
> Thanks
> Naresh
>