You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Justin <th...@gmail.com> on 2009/05/19 11:50:07 UTC
small facets not working
I have a solr index which contains research data from the human genome
project.
Each document contains about 60 facets, including one general composite
field that contains all the facet data. the general facet is anywhere from
100KB to 7MB.
One facet is called Gene.Symbol and, you guessed it, it contains only the
gene symbol. There is only one Symbol per gene (for smarty pantses out
there, the aliases are contained in another facet).
When I do a search for anything in the big general facet, I find what i'm
looking for. But if I do a search in the Gene.Symbol facet, it does not
find anything.
I realize it's probably finding the string repeated elsewhere in the
document, but how do I get it to find it in the Gene.Symbol facet?
so a search for
http://localhost:8983/solr/core0/select?indent=on&version=2.2&q=Gene.Symbol:abc
returns nothing, but a search for
http://localhost:8983/solr/core0/select?indent=on&version=2.2&q=abc
returns
ABCC2
ABCC8
ABCD1
ABCG1
ABCA1
...
CABC1
...
ABCD3
ABCC5
ABCC9
ABCG2
ABCB11
ABCC3
ABCF1
ABCC1
ABCF2
ABCB9
Schema.xml:
<fieldType name="symbol" class="solr.TextField" positionIncrementGap="0">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.TrimFilterFactory" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.TrimFilterFactory" />
</analyzer>
</fieldType>
...
<!-- yes, taken directly from the example -->
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="0"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
ignoreCase="true" expand="true"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="0"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
...
<field name="Gene.Symbol" type="symbol" indexed="true"
stored="true" required="true" multiValued="false" omitNorms="false"/>
<field name="BFDText" type="text" indexed="true"
stored="false" multiValued="true" omitNorms="true"/>
...
<defaultSearchField>BFDText</defaultSearchField>
<solrQueryParser defaultOperator="AND"/>
<copyField source="*" dest="BFDText"/>
Re: small facets not working
Posted by Grant Ingersoll <gs...@apache.org>.
On May 19, 2009, at 5:50 AM, Justin wrote:
> I have a solr index which contains research data from the human genome
> project.
>
> Each document contains about 60 facets, including one general
> composite
> field that contains all the facet data. the general facet is
> anywhere from
> 100KB to 7MB.
>
> One facet is called Gene.Symbol and, you guessed it, it contains
> only the
> gene symbol. There is only one Symbol per gene (for smarty pantses out
> there, the aliases are contained in another facet).
>
> When I do a search for anything in the big general facet, I find
> what i'm
> looking for. But if I do a search in the Gene.Symbol facet, it does
> not
> find anything.
>
> I realize it's probably finding the string repeated elsewhere in the
> document, but how do I get it to find it in the Gene.Symbol facet?
I'd look at the analysis tool in Solr admin and compare putting in
various gene names. It seems a bit odd that you are applying Porter
stemming to gene names.
You are likely getting matches due to the WordDelimiterFilter and
other manipulations in the BFDText. In the Symbol field you aren't
doing nearly as much to the tokens, so I doubt there is an "abc" gene
in there.
You could try doing a prefix query. You could also try creating n-
grams during indexing or other mechanisms for allowing matches within
a string.
>
>
> so a search for
>
> http://localhost:8983/solr/core0/select?indent=on&version=2.2&q=Gene.Symbol:abc
>
> returns nothing, but a search for
>
> http://localhost:8983/solr/core0/select?indent=on&version=2.2&q=abc
>
> returns
> ABCC2
> ABCC8
> ABCD1
> ABCG1
> ABCA1
> ...
> CABC1
> ...
> ABCD3
> ABCC5
> ABCC9
> ABCG2
> ABCB11
> ABCC3
> ABCF1
> ABCC1
> ABCF2
> ABCB9
>
>
>
> Schema.xml:
>
> <fieldType name="symbol" class="solr.TextField"
> positionIncrementGap="0">
> <analyzer type="index">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.LowerCaseFilterFactory" />
> <filter class="solr.TrimFilterFactory" />
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.LowerCaseFilterFactory" />
> <filter class="solr.TrimFilterFactory" />
> </analyzer>
>
> </fieldType>
> ...
> <!-- yes, taken directly from the example -->
>
> <fieldType name="text" class="solr.TextField"
> positionIncrementGap="100">
> <analyzer type="index">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="stopwords.txt"/>
> <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="1" generateNumberParts="1" catenateWords="1"
> catenateNumbers="1" catenateAll="0"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.EnglishPorterFilterFactory"
> protected="protwords.txt"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.SynonymFilterFactory"
> synonyms="synonyms.txt"
> ignoreCase="true" expand="true"/>
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="stopwords.txt"/>
> <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="1" generateNumberParts="1" catenateWords="1"
> catenateNumbers="1" catenateAll="0"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.EnglishPorterFilterFactory"
> protected="protwords.txt"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> </fieldType>
>
> ...
> <field name="Gene.Symbol" type="symbol" indexed="true"
> stored="true" required="true" multiValued="false" omitNorms="false"/>
> <field name="BFDText" type="text"
> indexed="true"
> stored="false" multiValued="true" omitNorms="true"/>
> ...
> <defaultSearchField>BFDText</defaultSearchField>
> <solrQueryParser defaultOperator="AND"/>
> <copyField source="*" dest="BFDText"/>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search