You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Anders Dam <an...@gmail.com> on 2010/12/13 04:18:43 UTC
Search with facet.pivot
Hi,
I have a minor problem in getting the pivoting working correctly. The thing
is that two otherwise equal search queries behave differently, namely one is
returning the search result with the facet.pivot fields below and another is
returning the search result with an empty facet.pivot. This is a problem,
since I am particularly interested in displaying the pivots.
Perhaps anyone has an idea about what is going wrong in this case, For
clarity I paste the parameters used for searching:
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">41</int>
-
<lst name="params">
<str name="mm">
2<-1 5<-2 6<90%
</str>
<str name="facet">on</str>
<str name="facet.mincount">1</str>
<str name="tie">0.01</str>
<str name="qf">
category_search
</str>
<str name="f.category.hl.fragsize">0</str>
<str name="hl.simple.pre"><em></str>
<str name="q.alt">
*:*
</str>
<str name="hl.fl">category</str>
<str name="hl">true</str>
<str name="defType">dismax</str>
<str name="echoParams">all</str>
<str name="fl">
*,score
</str>
<str name="hl.useFastVectorHighlighter">true</str>
<str name="hl.snippets">1</str>
<str name="hl.simple.post"></em></str>
<str name="hl.usePhraseHighlighter">true</str>
<str name="fq">
shop_name:colorbob.dk
</str>
-
<arr name="facet.pivot">
<str>root_category_name,parent_category_name,category</str>
<str>root_category_id,parent_category_id,category_id</str>
</arr>
<str name="ps">100</str>
-
<arr name="facet.pivot">
<str>root_category_name,parent_category_name,category</str>
<str>root_category_id,parent_category_id,category_id</str>
</arr>
<str name="q">OKI</str>
<str name="rows">100</str>
</lst>
</lst>
I see no pattern in what queries is returning the pivot fields and which
ones are not
The field searched in is defined as:
<field name="category_search" type="edgytext" indexed="true" stored="false"
required="false" termVectors="on" termPositions="on" termOffsets="on" />
And the edgytext type is defined as
<fieldType name="edgytext" class="solr.TextField"
positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.WordDelimiterFilterFactory"
stemEnglishPossessive="0" splitOnNumerics="0" preserveOriginal="1"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="1" />
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
maxGramSize="25" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.WordDelimiterFilterFactory"
stemEnglishPossessive="0" splitOnNumerics="0" preserveOriginal="1"
generateWordParts="1" generateNumberParts="1" catenateWords="0"
catenateNumbers="0" catenateAll="0" />
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
I am using apache-solr-4.0-2010-11-26_08-36-06 release
Thanks in advance,
Anders Dam
Re: Search with facet.pivot
Posted by Erik Hatcher <er...@gmail.com>.
One oddity is the duplicated sections:
<arr name="facet.pivot">
<str>root_category_name,parent_category_name,category</str>
<str>root_category_id,parent_category_id,category_id</str>
</arr>
That's in your responseHeader twice. Perhaps something fishy caused from that? Is this hardcoded in your solrconfig.xml request handler mapping (I can't imagine how you could get that effect from request params)? Try removing this duplication and see if that helps.
Erik
On Dec 14, 2010, at 22:17 , Anders Dam wrote:
> I forgot to mention that the query is handlede by the Dismax Request Handler
>
> Grant, from the <lst name="params"> tag and down you see all the query
> parameters used. The only thing varying from query to query is the actual
> query (q), When searching on by example '1000' (q=1000) facet.pivot fields
> are correctly returned, while when searching on by example 'OKI' the
> facet.pivot fields are not returned.
>
> If it is of any help, what I am searching are products compatible with
> certain printers, the printer models are stored in a relational database,
> where each printer cartridge belongs to more categories, the categorized are
> in a often 2-3 level deep hierarchy which is flattened out at data import
> time, so that there is a column at import data (DataImportHandler) called
> category_name,parent_category_name and root_category_name, these fields are
> copied to the field category_search field also mentioned in my first mail.
>
> Here are the response header listing all the params used for querying on
> pastebin.com with a bit better formatting, hope that helps:
> http://pastebin.com/9FgijpJ6
>
> <http://pastebin.com/9FgijpJ6>If there is any information I can provide to
> help us solve this problem I will be happy to provide it.
>
> Thanks in advance,
>
> Anders
>
>
> On Tue, Dec 14, 2010 at 7:47 PM, Grant Ingersoll <gs...@apache.org>wrote:
>
>> The formatting of your message is a bit hard to read. Could you please
>> clarify which commands worked and which ones didn't? Since the pivot stuff
>> is relatively new, there could very well be a bug, so if you can give a
>> simple test case that shows what is going on that would also be helpful,
>> albeit not required.
>>
>> On Dec 12, 2010, at 10:18 PM, Anders Dam wrote:
>>
>>> Hi,
>>>
>>> I have a minor problem in getting the pivoting working correctly. The
>> thing
>>> is that two otherwise equal search queries behave differently, namely one
>> is
>>> returning the search result with the facet.pivot fields below and another
>> is
>>> returning the search result with an empty facet.pivot. This is a problem,
>>> since I am particularly interested in displaying the pivots.
>>>
>>> Perhaps anyone has an idea about what is going wrong in this case, For
>>> clarity I paste the parameters used for searching:
>>>
>>>
>>> <lst name="responseHeader">
>>> <int name="status">0</int>
>>> <int name="QTime">41</int>
>>> -
>>> <lst name="params">
>>> <str name="mm">
>>> 2<-1 5<-2 6<90%
>>> </str>
>>> <str name="facet">on</str>
>>> <str name="facet.mincount">1</str>
>>> <str name="tie">0.01</str>
>>> <str name="qf">
>>> category_search
>>> </str>
>>> <str name="f.category.hl.fragsize">0</str>
>>> <str name="hl.simple.pre"><em></str>
>>> <str name="q.alt">
>>> *:*
>>> </str>
>>> <str name="hl.fl">category</str>
>>> <str name="hl">true</str>
>>> <str name="defType">dismax</str>
>>> <str name="echoParams">all</str>
>>> <str name="fl">
>>> *,score
>>> </str>
>>> <str name="hl.useFastVectorHighlighter">true</str>
>>> <str name="hl.snippets">1</str>
>>> <str name="hl.simple.post"></em></str>
>>> <str name="hl.usePhraseHighlighter">true</str>
>>> <str name="fq">
>>> shop_name:colorbob.dk
>>> </str>
>>> -
>>> <arr name="facet.pivot">
>>> <str>root_category_name,parent_category_name,category</str>
>>> <str>root_category_id,parent_category_id,category_id</str>
>>> </arr>
>>> <str name="ps">100</str>
>>> -
>>> <arr name="facet.pivot">
>>> <str>root_category_name,parent_category_name,category</str>
>>> <str>root_category_id,parent_category_id,category_id</str>
>>> </arr>
>>> <str name="q">OKI</str>
>>> <str name="rows">100</str>
>>> </lst>
>>> </lst>
>>>
>>> I see no pattern in what queries is returning the pivot fields and which
>>> ones are not
>>>
>>>
>>> The field searched in is defined as:
>>>
>>> <field name="category_search" type="edgytext" indexed="true"
>> stored="false"
>>> required="false" termVectors="on" termPositions="on" termOffsets="on" />
>>>
>>> And the edgytext type is defined as
>>> <fieldType name="edgytext" class="solr.TextField"
>>> positionIncrementGap="100">
>>> <analyzer type="index">
>>> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>>> <filter class="solr.WordDelimiterFilterFactory"
>>> stemEnglishPossessive="0" splitOnNumerics="0" preserveOriginal="1"
>>> generateWordParts="1" generateNumberParts="1" catenateWords="1"
>>> catenateNumbers="1" catenateAll="1" />
>>> <filter class="solr.LowerCaseFilterFactory"/>
>>> <filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
>>> maxGramSize="25" />
>>> </analyzer>
>>> <analyzer type="query">
>>> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>>> <filter class="solr.WordDelimiterFilterFactory"
>>> stemEnglishPossessive="0" splitOnNumerics="0" preserveOriginal="1"
>>> generateWordParts="1" generateNumberParts="1" catenateWords="0"
>>> catenateNumbers="0" catenateAll="0" />
>>> <filter class="solr.LowerCaseFilterFactory"/>
>>> </analyzer>
>>> </fieldType>
>>>
>>> I am using apache-solr-4.0-2010-11-26_08-36-06 release
>>>
>>> Thanks in advance,
>>>
>>> Anders Dam
>>
>> --------------------------
>> Grant Ingersoll
>> http://www.lucidimagination.com
>>
>>
Re: Search with facet.pivot
Posted by Anders Dam <an...@gmail.com>.
I forgot to mention that the query is handlede by the Dismax Request Handler
Grant, from the <lst name="params"> tag and down you see all the query
parameters used. The only thing varying from query to query is the actual
query (q), When searching on by example '1000' (q=1000) facet.pivot fields
are correctly returned, while when searching on by example 'OKI' the
facet.pivot fields are not returned.
If it is of any help, what I am searching are products compatible with
certain printers, the printer models are stored in a relational database,
where each printer cartridge belongs to more categories, the categorized are
in a often 2-3 level deep hierarchy which is flattened out at data import
time, so that there is a column at import data (DataImportHandler) called
category_name,parent_category_name and root_category_name, these fields are
copied to the field category_search field also mentioned in my first mail.
Here are the response header listing all the params used for querying on
pastebin.com with a bit better formatting, hope that helps:
http://pastebin.com/9FgijpJ6
<http://pastebin.com/9FgijpJ6>If there is any information I can provide to
help us solve this problem I will be happy to provide it.
Thanks in advance,
Anders
On Tue, Dec 14, 2010 at 7:47 PM, Grant Ingersoll <gs...@apache.org>wrote:
> The formatting of your message is a bit hard to read. Could you please
> clarify which commands worked and which ones didn't? Since the pivot stuff
> is relatively new, there could very well be a bug, so if you can give a
> simple test case that shows what is going on that would also be helpful,
> albeit not required.
>
> On Dec 12, 2010, at 10:18 PM, Anders Dam wrote:
>
> > Hi,
> >
> > I have a minor problem in getting the pivoting working correctly. The
> thing
> > is that two otherwise equal search queries behave differently, namely one
> is
> > returning the search result with the facet.pivot fields below and another
> is
> > returning the search result with an empty facet.pivot. This is a problem,
> > since I am particularly interested in displaying the pivots.
> >
> > Perhaps anyone has an idea about what is going wrong in this case, For
> > clarity I paste the parameters used for searching:
> >
> >
> > <lst name="responseHeader">
> > <int name="status">0</int>
> > <int name="QTime">41</int>
> > -
> > <lst name="params">
> > <str name="mm">
> > 2<-1 5<-2 6<90%
> > </str>
> > <str name="facet">on</str>
> > <str name="facet.mincount">1</str>
> > <str name="tie">0.01</str>
> > <str name="qf">
> > category_search
> > </str>
> > <str name="f.category.hl.fragsize">0</str>
> > <str name="hl.simple.pre"><em></str>
> > <str name="q.alt">
> > *:*
> > </str>
> > <str name="hl.fl">category</str>
> > <str name="hl">true</str>
> > <str name="defType">dismax</str>
> > <str name="echoParams">all</str>
> > <str name="fl">
> > *,score
> > </str>
> > <str name="hl.useFastVectorHighlighter">true</str>
> > <str name="hl.snippets">1</str>
> > <str name="hl.simple.post"></em></str>
> > <str name="hl.usePhraseHighlighter">true</str>
> > <str name="fq">
> > shop_name:colorbob.dk
> > </str>
> > -
> > <arr name="facet.pivot">
> > <str>root_category_name,parent_category_name,category</str>
> > <str>root_category_id,parent_category_id,category_id</str>
> > </arr>
> > <str name="ps">100</str>
> > -
> > <arr name="facet.pivot">
> > <str>root_category_name,parent_category_name,category</str>
> > <str>root_category_id,parent_category_id,category_id</str>
> > </arr>
> > <str name="q">OKI</str>
> > <str name="rows">100</str>
> > </lst>
> > </lst>
> >
> > I see no pattern in what queries is returning the pivot fields and which
> > ones are not
> >
> >
> > The field searched in is defined as:
> >
> > <field name="category_search" type="edgytext" indexed="true"
> stored="false"
> > required="false" termVectors="on" termPositions="on" termOffsets="on" />
> >
> > And the edgytext type is defined as
> > <fieldType name="edgytext" class="solr.TextField"
> > positionIncrementGap="100">
> > <analyzer type="index">
> > <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> > <filter class="solr.WordDelimiterFilterFactory"
> > stemEnglishPossessive="0" splitOnNumerics="0" preserveOriginal="1"
> > generateWordParts="1" generateNumberParts="1" catenateWords="1"
> > catenateNumbers="1" catenateAll="1" />
> > <filter class="solr.LowerCaseFilterFactory"/>
> > <filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
> > maxGramSize="25" />
> > </analyzer>
> > <analyzer type="query">
> > <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> > <filter class="solr.WordDelimiterFilterFactory"
> > stemEnglishPossessive="0" splitOnNumerics="0" preserveOriginal="1"
> > generateWordParts="1" generateNumberParts="1" catenateWords="0"
> > catenateNumbers="0" catenateAll="0" />
> > <filter class="solr.LowerCaseFilterFactory"/>
> > </analyzer>
> > </fieldType>
> >
> > I am using apache-solr-4.0-2010-11-26_08-36-06 release
> >
> > Thanks in advance,
> >
> > Anders Dam
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com
>
>
Re: Search with facet.pivot
Posted by Grant Ingersoll <gs...@apache.org>.
The formatting of your message is a bit hard to read. Could you please clarify which commands worked and which ones didn't? Since the pivot stuff is relatively new, there could very well be a bug, so if you can give a simple test case that shows what is going on that would also be helpful, albeit not required.
On Dec 12, 2010, at 10:18 PM, Anders Dam wrote:
> Hi,
>
> I have a minor problem in getting the pivoting working correctly. The thing
> is that two otherwise equal search queries behave differently, namely one is
> returning the search result with the facet.pivot fields below and another is
> returning the search result with an empty facet.pivot. This is a problem,
> since I am particularly interested in displaying the pivots.
>
> Perhaps anyone has an idea about what is going wrong in this case, For
> clarity I paste the parameters used for searching:
>
>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">41</int>
> -
> <lst name="params">
> <str name="mm">
> 2<-1 5<-2 6<90%
> </str>
> <str name="facet">on</str>
> <str name="facet.mincount">1</str>
> <str name="tie">0.01</str>
> <str name="qf">
> category_search
> </str>
> <str name="f.category.hl.fragsize">0</str>
> <str name="hl.simple.pre"><em></str>
> <str name="q.alt">
> *:*
> </str>
> <str name="hl.fl">category</str>
> <str name="hl">true</str>
> <str name="defType">dismax</str>
> <str name="echoParams">all</str>
> <str name="fl">
> *,score
> </str>
> <str name="hl.useFastVectorHighlighter">true</str>
> <str name="hl.snippets">1</str>
> <str name="hl.simple.post"></em></str>
> <str name="hl.usePhraseHighlighter">true</str>
> <str name="fq">
> shop_name:colorbob.dk
> </str>
> -
> <arr name="facet.pivot">
> <str>root_category_name,parent_category_name,category</str>
> <str>root_category_id,parent_category_id,category_id</str>
> </arr>
> <str name="ps">100</str>
> -
> <arr name="facet.pivot">
> <str>root_category_name,parent_category_name,category</str>
> <str>root_category_id,parent_category_id,category_id</str>
> </arr>
> <str name="q">OKI</str>
> <str name="rows">100</str>
> </lst>
> </lst>
>
> I see no pattern in what queries is returning the pivot fields and which
> ones are not
>
>
> The field searched in is defined as:
>
> <field name="category_search" type="edgytext" indexed="true" stored="false"
> required="false" termVectors="on" termPositions="on" termOffsets="on" />
>
> And the edgytext type is defined as
> <fieldType name="edgytext" class="solr.TextField"
> positionIncrementGap="100">
> <analyzer type="index">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.WordDelimiterFilterFactory"
> stemEnglishPossessive="0" splitOnNumerics="0" preserveOriginal="1"
> generateWordParts="1" generateNumberParts="1" catenateWords="1"
> catenateNumbers="1" catenateAll="1" />
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.EdgeNGramFilterFactory" minGramSize="1"
> maxGramSize="25" />
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.WordDelimiterFilterFactory"
> stemEnglishPossessive="0" splitOnNumerics="0" preserveOriginal="1"
> generateWordParts="1" generateNumberParts="1" catenateWords="0"
> catenateNumbers="0" catenateAll="0" />
> <filter class="solr.LowerCaseFilterFactory"/>
> </analyzer>
> </fieldType>
>
> I am using apache-solr-4.0-2010-11-26_08-36-06 release
>
> Thanks in advance,
>
> Anders Dam
--------------------------
Grant Ingersoll
http://www.lucidimagination.com