You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Shamik Bandopadhyay <sh...@gmail.com> on 2014/01/13 22:21:49 UTC
Questionon CollapsingQParserPlugin
Hi,
I'm looking for some clarification on CollapsingQParserPlugin feature.
Here's what I tried. I downloaded 4.6, updated "solr.xml" under exampledocs
folder and added the following entry. I've added a new field "adskdedup"
on which I'm planning to test field collapsing. As you can see, out of four
documents, three have similar adskdedup values while the last one is
different.
<doc>
<field name="id">SOLR1000</field>
<field name="name">Solr, the Enterprise Search Server</field>
<field name="price">0</field>
<field name="popularity">10</field>
<field name="inStock">true</field>
<field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
<field name="adskdedup">ABCD-XYZ</field>
</doc>
<doc>
<field name="id">SOLR1001</field>
<field name="name">Solr, the Enterprise Search Server</field>
<field name="price">0</field>
<field name="popularity">10</field>
<field name="inStock">true</field>
<field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
<field name="adskdedup">ABCD-XYZ</field>
</doc>
<doc>
<field name="id">SOLR1002</field>
<field name="name">Solr, the Enterprise Search Server</field>
<field name="price">0</field>
<field name="popularity">10</field>
<field name="inStock">true</field>
<field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
<field name="adskdedup">ABCD-XYZ</field>
</doc>
<doc>
<field name="id">SOLR1003</field>
<field name="name">Solr, the Enterprise Search Server</field>
<field name="price">0</field>
<field name="popularity">10</field>
<field name="inStock">true</field>
<field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
<field name="adskdedup">MNOP-QRS</field>
</doc>
Here's my query :
http://localhost:8983/solr/collection1/select?q=solr&wt=xml&fq={!collapse%20field=adskdedup}
Based on my understanding of using group by, I was expecting couple of
results from the query. One with id=SOLR1000 and the second with
id=SOLR1003. Instead, its returning only 1 result based on the field
collapsing, i.e. id=SOLR1000.
Am I missing something here ?
Any pointer will be appreciated.
-Thanks
Re: Questionon CollapsingQParserPlugin
Posted by shamik <sh...@gmail.com>.
Thanks Joel, I found the issue. It had to do with the schema definition for
adskdedup field. I had defined it as a text_general which was analyzing it
based on "-". After I changed it to type string, it worked as expected.
Thanks for looking into this.
--
View this message in context: http://lucene.472066.n3.nabble.com/Re-Questionon-CollapsingQParserPlugin-tp4111357p4111376.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Questionon CollapsingQParserPlugin
Posted by Joel Bernstein <jo...@gmail.com>.
Just tried it with q=solr as well:
{
"responseHeader":{
"status":0,
"QTime":1,
"params":{
"indent":"true",
"q":"solr",
"wt":"json",
"fq":"{!collapse field=adskdedup_s}"}},
"response":{"numFound":2,"start":0,"docs":[
{
"id":"SOLR1000",
"name":"Solr, the Enterprise Search Server",
"price":0.0,
"price_c":"0,USD",
"popularity":10,
"inStock":true,
"incubationdate_dt":"2006-01-17T00:00:00Z",
"adskdedup_s":"ABCD-XYZ",
"_version_":1457264913719230464},
{
"id":"SOLR1003",
"name":"Solr, the Enterprise Search Server",
"price":0.0,
"price_c":"0,USD",
"popularity":10,
"inStock":true,
"incubationdate_dt":"2006-01-17T00:00:00Z",
"adskdedup_s":"MNOP-QRS",
"_version_":1457264913752784896}]
}}
Joel Bernstein
Search Engineer at Heliosearch
On Tue, Jan 14, 2014 at 10:25 PM, Joel Bernstein <jo...@gmail.com> wrote:
> I just did a quick test with the 4 docs and got the proper result.
> All I did was change the adskdedup field to adskdedup_s so it would load
> as a dynamic string field. You can see the output below.
>
> Can you provide more details on the exact steps you took?
>
> {
> "responseHeader":{
> "status":0,
> "QTime":24,
> "params":{
> "indent":"true",
> "q":"*:*",
> "wt":"json",
> "fq":"{!collapse field=adskdedup_s}"}},
> "response":{"numFound":2,"start":0,"docs":[
> {
> "id":"SOLR1000",
> "name":"Solr, the Enterprise Search Server",
> "price":0.0,
> "price_c":"0,USD",
> "popularity":10,
> "inStock":true,
> "incubationdate_dt":"2006-01-17T00:00:00Z",
> "adskdedup_s":"ABCD-XYZ",
> "_version_":1457264913719230464},
> {
> "id":"SOLR1003",
> "name":"Solr, the Enterprise Search Server",
> "price":0.0,
> "price_c":"0,USD",
> "popularity":10,
> "inStock":true,
> "incubationdate_dt":"2006-01-17T00:00:00Z",
> "adskdedup_s":"MNOP-QRS",
> "_version_":1457264913752784896}]
> }}
>
>
> Joel Bernstein
> Search Engineer at Heliosearch
>
>
> On Tue, Jan 14, 2014 at 10:10 PM, Joel Bernstein <jo...@gmail.com>wrote:
>
>> Something is off but I'm not sure what. A couple of questions.
>>
>> 1) You mention updating the solr.xml. Did you the schema.xml?
>> 2) Did you load only those 4 docs?
>>
>> Joel
>>
>> Joel Bernstein
>> Search Engineer at Heliosearch
>>
>>
>> On Mon, Jan 13, 2014 at 4:21 PM, Shamik Bandopadhyay <sh...@gmail.com>wrote:
>>
>>> Hi,
>>>
>>> I'm looking for some clarification on CollapsingQParserPlugin feature.
>>>
>>> Here's what I tried. I downloaded 4.6, updated "solr.xml" under
>>> exampledocs
>>> folder and added the following entry. I've added a new field "adskdedup"
>>> on which I'm planning to test field collapsing. As you can see, out of
>>> four
>>> documents, three have similar adskdedup values while the last one is
>>> different.
>>>
>>> <doc>
>>> <field name="id">SOLR1000</field>
>>> <field name="name">Solr, the Enterprise Search Server</field>
>>> <field name="price">0</field>
>>> <field name="popularity">10</field>
>>> <field name="inStock">true</field>
>>> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
>>> <field name="adskdedup">ABCD-XYZ</field>
>>> </doc>
>>> <doc>
>>> <field name="id">SOLR1001</field>
>>> <field name="name">Solr, the Enterprise Search Server</field>
>>> <field name="price">0</field>
>>> <field name="popularity">10</field>
>>> <field name="inStock">true</field>
>>> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
>>> <field name="adskdedup">ABCD-XYZ</field>
>>> </doc>
>>> <doc>
>>> <field name="id">SOLR1002</field>
>>> <field name="name">Solr, the Enterprise Search Server</field>
>>> <field name="price">0</field>
>>> <field name="popularity">10</field>
>>> <field name="inStock">true</field>
>>> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
>>> <field name="adskdedup">ABCD-XYZ</field>
>>> </doc>
>>> <doc>
>>> <field name="id">SOLR1003</field>
>>> <field name="name">Solr, the Enterprise Search Server</field>
>>> <field name="price">0</field>
>>> <field name="popularity">10</field>
>>> <field name="inStock">true</field>
>>> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
>>> <field name="adskdedup">MNOP-QRS</field>
>>> </doc>
>>>
>>> Here's my query :
>>>
>>>
>>> http://localhost:8983/solr/collection1/select?q=solr&wt=xml&fq={!collapse%20field=adskdedup}<http://localhost:8983/solr/collection1/select?q=solr&wt=xml&fq=%7B!collapse%20field=adskdedup%7D>
>>>
>>> Based on my understanding of using group by, I was expecting couple of
>>> results from the query. One with id=SOLR1000 and the second with
>>> id=SOLR1003. Instead, its returning only 1 result based on the field
>>> collapsing, i.e. id=SOLR1000.
>>>
>>> Am I missing something here ?
>>>
>>> Any pointer will be appreciated.
>>>
>>> -Thanks
>>>
>>
>>
>
Re: Questionon CollapsingQParserPlugin
Posted by Joel Bernstein <jo...@gmail.com>.
I just did a quick test with the 4 docs and got the proper result.
All I did was change the adskdedup field to adskdedup_s so it would load as
a dynamic string field. You can see the output below.
Can you provide more details on the exact steps you took?
{
"responseHeader":{
"status":0,
"QTime":24,
"params":{
"indent":"true",
"q":"*:*",
"wt":"json",
"fq":"{!collapse field=adskdedup_s}"}},
"response":{"numFound":2,"start":0,"docs":[
{
"id":"SOLR1000",
"name":"Solr, the Enterprise Search Server",
"price":0.0,
"price_c":"0,USD",
"popularity":10,
"inStock":true,
"incubationdate_dt":"2006-01-17T00:00:00Z",
"adskdedup_s":"ABCD-XYZ",
"_version_":1457264913719230464},
{
"id":"SOLR1003",
"name":"Solr, the Enterprise Search Server",
"price":0.0,
"price_c":"0,USD",
"popularity":10,
"inStock":true,
"incubationdate_dt":"2006-01-17T00:00:00Z",
"adskdedup_s":"MNOP-QRS",
"_version_":1457264913752784896}]
}}
Joel Bernstein
Search Engineer at Heliosearch
On Tue, Jan 14, 2014 at 10:10 PM, Joel Bernstein <jo...@gmail.com> wrote:
> Something is off but I'm not sure what. A couple of questions.
>
> 1) You mention updating the solr.xml. Did you the schema.xml?
> 2) Did you load only those 4 docs?
>
> Joel
>
> Joel Bernstein
> Search Engineer at Heliosearch
>
>
> On Mon, Jan 13, 2014 at 4:21 PM, Shamik Bandopadhyay <sh...@gmail.com>wrote:
>
>> Hi,
>>
>> I'm looking for some clarification on CollapsingQParserPlugin feature.
>>
>> Here's what I tried. I downloaded 4.6, updated "solr.xml" under
>> exampledocs
>> folder and added the following entry. I've added a new field "adskdedup"
>> on which I'm planning to test field collapsing. As you can see, out of
>> four
>> documents, three have similar adskdedup values while the last one is
>> different.
>>
>> <doc>
>> <field name="id">SOLR1000</field>
>> <field name="name">Solr, the Enterprise Search Server</field>
>> <field name="price">0</field>
>> <field name="popularity">10</field>
>> <field name="inStock">true</field>
>> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
>> <field name="adskdedup">ABCD-XYZ</field>
>> </doc>
>> <doc>
>> <field name="id">SOLR1001</field>
>> <field name="name">Solr, the Enterprise Search Server</field>
>> <field name="price">0</field>
>> <field name="popularity">10</field>
>> <field name="inStock">true</field>
>> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
>> <field name="adskdedup">ABCD-XYZ</field>
>> </doc>
>> <doc>
>> <field name="id">SOLR1002</field>
>> <field name="name">Solr, the Enterprise Search Server</field>
>> <field name="price">0</field>
>> <field name="popularity">10</field>
>> <field name="inStock">true</field>
>> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
>> <field name="adskdedup">ABCD-XYZ</field>
>> </doc>
>> <doc>
>> <field name="id">SOLR1003</field>
>> <field name="name">Solr, the Enterprise Search Server</field>
>> <field name="price">0</field>
>> <field name="popularity">10</field>
>> <field name="inStock">true</field>
>> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
>> <field name="adskdedup">MNOP-QRS</field>
>> </doc>
>>
>> Here's my query :
>>
>>
>> http://localhost:8983/solr/collection1/select?q=solr&wt=xml&fq={!collapse%20field=adskdedup}<http://localhost:8983/solr/collection1/select?q=solr&wt=xml&fq=%7B!collapse%20field=adskdedup%7D>
>>
>> Based on my understanding of using group by, I was expecting couple of
>> results from the query. One with id=SOLR1000 and the second with
>> id=SOLR1003. Instead, its returning only 1 result based on the field
>> collapsing, i.e. id=SOLR1000.
>>
>> Am I missing something here ?
>>
>> Any pointer will be appreciated.
>>
>> -Thanks
>>
>
>
Re: Questionon CollapsingQParserPlugin
Posted by Joel Bernstein <jo...@gmail.com>.
Something is off but I'm not sure what. A couple of questions.
1) You mention updating the solr.xml. Did you the schema.xml?
2) Did you load only those 4 docs?
Joel
Joel Bernstein
Search Engineer at Heliosearch
On Mon, Jan 13, 2014 at 4:21 PM, Shamik Bandopadhyay <sh...@gmail.com>wrote:
> Hi,
>
> I'm looking for some clarification on CollapsingQParserPlugin feature.
>
> Here's what I tried. I downloaded 4.6, updated "solr.xml" under exampledocs
> folder and added the following entry. I've added a new field "adskdedup"
> on which I'm planning to test field collapsing. As you can see, out of four
> documents, three have similar adskdedup values while the last one is
> different.
>
> <doc>
> <field name="id">SOLR1000</field>
> <field name="name">Solr, the Enterprise Search Server</field>
> <field name="price">0</field>
> <field name="popularity">10</field>
> <field name="inStock">true</field>
> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
> <field name="adskdedup">ABCD-XYZ</field>
> </doc>
> <doc>
> <field name="id">SOLR1001</field>
> <field name="name">Solr, the Enterprise Search Server</field>
> <field name="price">0</field>
> <field name="popularity">10</field>
> <field name="inStock">true</field>
> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
> <field name="adskdedup">ABCD-XYZ</field>
> </doc>
> <doc>
> <field name="id">SOLR1002</field>
> <field name="name">Solr, the Enterprise Search Server</field>
> <field name="price">0</field>
> <field name="popularity">10</field>
> <field name="inStock">true</field>
> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
> <field name="adskdedup">ABCD-XYZ</field>
> </doc>
> <doc>
> <field name="id">SOLR1003</field>
> <field name="name">Solr, the Enterprise Search Server</field>
> <field name="price">0</field>
> <field name="popularity">10</field>
> <field name="inStock">true</field>
> <field name="incubationdate_dt">2006-01-17T00:00:00.000Z</field>
> <field name="adskdedup">MNOP-QRS</field>
> </doc>
>
> Here's my query :
>
>
> http://localhost:8983/solr/collection1/select?q=solr&wt=xml&fq={!collapse%20field=adskdedup}
>
> Based on my understanding of using group by, I was expecting couple of
> results from the query. One with id=SOLR1000 and the second with
> id=SOLR1003. Instead, its returning only 1 result based on the field
> collapsing, i.e. id=SOLR1000.
>
> Am I missing something here ?
>
> Any pointer will be appreciated.
>
> -Thanks
>