You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Jerry Quinn (JIRA)" <ji...@apache.org> on 2008/08/12 21:37:44 UTC
[jira] Created: (SOLR-693) IntFieldSource incompatible with sint
field type
IntFieldSource incompatible with sint field type
------------------------------------------------
Key: SOLR-693
URL: https://issues.apache.org/jira/browse/SOLR-693
Project: Solr
Issue Type: Bug
Components: search
Affects Versions: 1.3
Environment: RHEL 5, java6, builtin jetty container
Reporter: Jerry Quinn
I'm trying to create a custom scoring query in Solr to implement a date bias. I have a custom query parser that I'm using, that does nothing but wrap a BoostedQuery around the original query, which works in general.
I'm indexing and storing the day number in an sint field. To implement my query, I extract the contents using org.apache.solr.search.function.IntFieldSource. Unfortunately, this throws an exception when it executes:
HTTP ERROR: 500
For input string: "€?"
java.lang.NumberFormatException: For input string: "€?"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:61)
at java.lang.Integer.parseInt(Integer.java:460)
at java.lang.Integer.parseInt(Integer.java:510)
at org.apache.lucene.search.FieldCacheImpl$3.parseInt(FieldCacheImpl.java:148)
at org.apache.lucene.search.FieldCacheImpl$7.createValue(FieldCacheImpl.java:262)
at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:245)
at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:239)
at org.apache.solr.search.function.IntFieldSource.getValues(IntFieldSource.java:50)
at org.apache.solr.search.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:103)
at org.apache.solr.search.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:81)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:132)
at org.apache.lucene.search.Searcher.search(Searcher.java:126)
at org.apache.lucene.search.Searcher.search(Searcher.java:105)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:966)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:175)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1151)
I ran into exactly the same problem when I tried to use the CustomScoreQuery and IntFieldSource classes from Lucene.
I've tracked the problem down to the fact that IntFieldSource expects the contents of the field to actually be an integer as returned by FieldCache.getInts(). However, Solr converts a sortable int using NumberUtils.int2sortablestr().
If I change my code to create a custom FieldCache.IntParser that applies NumberUtils.SortableStr2int before returning the value, my query works as expected. For example:
class MyIntParser implements FieldCache.IntParser {
public int parseInt(String val) { return NumberUtils.SortedStr2int(val, 0, val.length()); }
}
Query q = new BoostedQuery(qry, new IntFieldSource("myfield", new MyIntParser()));
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
RE: List of available facet fields returned with the query results
Posted by Barry Harding <ba...@systemaxdev.com>.
Hi Shalin,
As there is certainly the potential for several thousand different
attribute types across all of our category's I guess I will have to
manage them myself (was hoping for a short-cut or that I was missing a
trick) but no problem. Solr still seems to outperform the commercial
package we are using
Thanks
barry
-----Original Message-----
From: Shalin Shekhar Mangar [mailto:shalinmangar@gmail.com]
Sent: 14 August 2008 16:06
To: solr-user@lucene.apache.org
Subject: Re: List of available facet fields returned with the query
results
Hi Barry,
If each category has an exclusive set of fields on which you want to
facet
on, then you can simply facet on all facet-able fields (across all
categories). The ones which are not present for the selected category
will
show up with zero facets which your front-end can suppress. However if
the
total number of such fields are very large, you may be better off
managing
the mappings yourself for performance reasons. But, as always, first
measure
then optimize :)
On Thu, Aug 14, 2008 at 7:12 PM, Barry Harding
<ba...@systemaxdev.com>wrote:
> Hi,
> I have solr setup to index technical data for a number of different
> types of products, and this means that different product have
different
> facet fields available.
>
> For example here would be a small example of the sort of data we are
> indexing, in reality there are between 10 and 20 facet fields per
> product dependent upon its category, but a user could perform a search
> across more than one category.
>
> A typical search would be:
>
> Stage 1
> 1) User types a keyword
> 2) The matched category's from that search are displayed
> 3) User chooses a category
> 4) All results that match that category and keyword are displayed and
> it's at this point I would like to display the available facets and
> values.
>
>
>
>
> Name: Microsoft Optical Mouse for PC
> DPI: 1200
> Type: Laser
> Category: PC
> Price: 45.00
>
> Name: Eee PC
> Storage Type: Flash
> Screen Size: 17
> Category: Netbook
> Price: 200.00
>
> And so on.
>
> So if I search for "PC" and then category "Netbook" I would like solr
to
> be able to tell me what facet fields are available without me
resorting
> to a database to store which facets fields are available to search for
> which products, is there any way to get SOLR to return as part of the
> results a list of the facets available for the current search results
or
> even better could I get it to automatically do a facet query for each
of
> those fields to allow drill down querys.
>
> The current commercial tool we use that we are hoping solr can replace
> is called "FactFinder" and does exactly this, but I do have to have
> drilled down a number of times before this occurs, to stop the search
> attempting to return facets for every item in the index.
>
> I suspect am I missing a trick here or making this more complicated
than
> needed, any help or ideas much appreciated.
>
>
> Thanks
>
> Barry H
>
>
________________________________________________________________________
> Misco is a division of Systemax Europe Ltd. Registered in Scotland
Number
> 114143. Registered Office: Caledonian Exchange, 19a Canning Street,
> Edinburgh EH3 8EG. Telephone +44 (0)1933 686000.
>
--
Regards,
Shalin Shekhar Mangar.
________________________________________________________________________
Misco is a division of Systemax Europe Ltd. Registered in Scotland Number 114143. Registered Office: Caledonian Exchange, 19a Canning Street, Edinburgh EH3 8EG. Telephone +44 (0)1933 686000.
Re: List of available facet fields returned with the query results
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Hi Barry,
If each category has an exclusive set of fields on which you want to facet
on, then you can simply facet on all facet-able fields (across all
categories). The ones which are not present for the selected category will
show up with zero facets which your front-end can suppress. However if the
total number of such fields are very large, you may be better off managing
the mappings yourself for performance reasons. But, as always, first measure
then optimize :)
On Thu, Aug 14, 2008 at 7:12 PM, Barry Harding <ba...@systemaxdev.com>wrote:
> Hi,
> I have solr setup to index technical data for a number of different
> types of products, and this means that different product have different
> facet fields available.
>
> For example here would be a small example of the sort of data we are
> indexing, in reality there are between 10 and 20 facet fields per
> product dependent upon its category, but a user could perform a search
> across more than one category.
>
> A typical search would be:
>
> Stage 1
> 1) User types a keyword
> 2) The matched category's from that search are displayed
> 3) User chooses a category
> 4) All results that match that category and keyword are displayed and
> it's at this point I would like to display the available facets and
> values.
>
>
>
>
> Name: Microsoft Optical Mouse for PC
> DPI: 1200
> Type: Laser
> Category: PC
> Price: 45.00
>
> Name: Eee PC
> Storage Type: Flash
> Screen Size: 17
> Category: Netbook
> Price: 200.00
>
> And so on.
>
> So if I search for "PC" and then category "Netbook" I would like solr to
> be able to tell me what facet fields are available without me resorting
> to a database to store which facets fields are available to search for
> which products, is there any way to get SOLR to return as part of the
> results a list of the facets available for the current search results or
> even better could I get it to automatically do a facet query for each of
> those fields to allow drill down querys.
>
> The current commercial tool we use that we are hoping solr can replace
> is called "FactFinder" and does exactly this, but I do have to have
> drilled down a number of times before this occurs, to stop the search
> attempting to return facets for every item in the index.
>
> I suspect am I missing a trick here or making this more complicated than
> needed, any help or ideas much appreciated.
>
>
> Thanks
>
> Barry H
>
> ________________________________________________________________________
> Misco is a division of Systemax Europe Ltd. Registered in Scotland Number
> 114143. Registered Office: Caledonian Exchange, 19a Canning Street,
> Edinburgh EH3 8EG. Telephone +44 (0)1933 686000.
>
--
Regards,
Shalin Shekhar Mangar.
List of available facet fields returned with the query results
Posted by Barry Harding <ba...@systemaxdev.com>.
Hi,
I have solr setup to index technical data for a number of different
types of products, and this means that different product have different
facet fields available.
For example here would be a small example of the sort of data we are
indexing, in reality there are between 10 and 20 facet fields per
product dependent upon its category, but a user could perform a search
across more than one category.
A typical search would be:
Stage 1
1) User types a keyword
2) The matched category's from that search are displayed
3) User chooses a category
4) All results that match that category and keyword are displayed and
it's at this point I would like to display the available facets and
values.
Name: Microsoft Optical Mouse for PC
DPI: 1200
Type: Laser
Category: PC
Price: 45.00
Name: Eee PC
Storage Type: Flash
Screen Size: 17
Category: Netbook
Price: 200.00
And so on.
So if I search for "PC" and then category "Netbook" I would like solr to
be able to tell me what facet fields are available without me resorting
to a database to store which facets fields are available to search for
which products, is there any way to get SOLR to return as part of the
results a list of the facets available for the current search results or
even better could I get it to automatically do a facet query for each of
those fields to allow drill down querys.
The current commercial tool we use that we are hoping solr can replace
is called "FactFinder" and does exactly this, but I do have to have
drilled down a number of times before this occurs, to stop the search
attempting to return facets for every item in the index.
I suspect am I missing a trick here or making this more complicated than
needed, any help or ideas much appreciated.
Thanks
Barry H
________________________________________________________________________
Misco is a division of Systemax Europe Ltd. Registered in Scotland Number 114143. Registered Office: Caledonian Exchange, 19a Canning Street, Edinburgh EH3 8EG. Telephone +44 (0)1933 686000.
Re: [jira] Commented: (SOLR-693) IntFieldSource incompatible with sint field type
Posted by Yonik Seeley <yo...@apache.org>.
Switching to solr-user.
Jerry, what type of function are you trying to do that Solr won't do
for you out of the box?
-Yonik
On Tue, Aug 12, 2008 at 5:13 PM, Jerry Quinn (JIRA) <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/SOLR-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12621980#action_12621980 ]
>
> Jerry Quinn commented on SOLR-693:
> ----------------------------------
>
> I found SortableIntFieldSource but it's not publicly accessible. It's also in org.apache.solr.schema instead of org.apache.solr.search.function like all the other ValueSource objects.
>
>> IntFieldSource incompatible with sint field type
>> ------------------------------------------------
>>
>> Key: SOLR-693
>> URL: https://issues.apache.org/jira/browse/SOLR-693
>> Project: Solr
>> Issue Type: Bug
>> Components: search
>> Affects Versions: 1.3
>> Environment: RHEL 5, java6, builtin jetty container
>> Reporter: Jerry Quinn
>>
>> I'm trying to create a custom scoring query in Solr to implement a date bias. I have a custom query parser that I'm using, that does nothing but wrap a BoostedQuery around the original query, which works in general.
>> I'm indexing and storing the day number in an sint field. To implement my query, I extract the contents using org.apache.solr.search.function.IntFieldSource. Unfortunately, this throws an exception when it executes:
>> HTTP ERROR: 500
>> For input string: "€?"
>> java.lang.NumberFormatException: For input string: "€?"
>> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:61)
>> at java.lang.Integer.parseInt(Integer.java:460)
>> at java.lang.Integer.parseInt(Integer.java:510)
>> at org.apache.lucene.search.FieldCacheImpl$3.parseInt(FieldCacheImpl.java:148)
>> at org.apache.lucene.search.FieldCacheImpl$7.createValue(FieldCacheImpl.java:262)
>> at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
>> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:245)
>> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:239)
>> at org.apache.solr.search.function.IntFieldSource.getValues(IntFieldSource.java:50)
>> at org.apache.solr.search.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:103)
>> at org.apache.solr.search.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:81)
>> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:132)
>> at org.apache.lucene.search.Searcher.search(Searcher.java:126)
>> at org.apache.lucene.search.Searcher.search(Searcher.java:105)
>> at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:966)
>> at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838)
>> at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269)
>> at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)
>> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:175)
>> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1151)
>> I ran into exactly the same problem when I tried to use the CustomScoreQuery and IntFieldSource classes from Lucene.
>> I've tracked the problem down to the fact that IntFieldSource expects the contents of the field to actually be an integer as returned by FieldCache.getInts(). However, Solr converts a sortable int using NumberUtils.int2sortablestr().
>> If I change my code to create a custom FieldCache.IntParser that applies NumberUtils.SortableStr2int before returning the value, my query works as expected. For example:
>> class MyIntParser implements FieldCache.IntParser {
>> public int parseInt(String val) { return NumberUtils.SortedStr2int(val, 0, val.length()); }
>> }
>> Query q = new BoostedQuery(qry, new IntFieldSource("myfield", new MyIntParser()));
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
[jira] Commented: (SOLR-693) IntFieldSource incompatible with sint
field type
Posted by "Jerry Quinn (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12621980#action_12621980 ]
Jerry Quinn commented on SOLR-693:
----------------------------------
I found SortableIntFieldSource but it's not publicly accessible. It's also in org.apache.solr.schema instead of org.apache.solr.search.function like all the other ValueSource objects.
> IntFieldSource incompatible with sint field type
> ------------------------------------------------
>
> Key: SOLR-693
> URL: https://issues.apache.org/jira/browse/SOLR-693
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.3
> Environment: RHEL 5, java6, builtin jetty container
> Reporter: Jerry Quinn
>
> I'm trying to create a custom scoring query in Solr to implement a date bias. I have a custom query parser that I'm using, that does nothing but wrap a BoostedQuery around the original query, which works in general.
> I'm indexing and storing the day number in an sint field. To implement my query, I extract the contents using org.apache.solr.search.function.IntFieldSource. Unfortunately, this throws an exception when it executes:
> HTTP ERROR: 500
> For input string: "€?"
> java.lang.NumberFormatException: For input string: "€?"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:61)
> at java.lang.Integer.parseInt(Integer.java:460)
> at java.lang.Integer.parseInt(Integer.java:510)
> at org.apache.lucene.search.FieldCacheImpl$3.parseInt(FieldCacheImpl.java:148)
> at org.apache.lucene.search.FieldCacheImpl$7.createValue(FieldCacheImpl.java:262)
> at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:245)
> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:239)
> at org.apache.solr.search.function.IntFieldSource.getValues(IntFieldSource.java:50)
> at org.apache.solr.search.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:103)
> at org.apache.solr.search.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:81)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:132)
> at org.apache.lucene.search.Searcher.search(Searcher.java:126)
> at org.apache.lucene.search.Searcher.search(Searcher.java:105)
> at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:966)
> at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838)
> at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269)
> at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:175)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1151)
> I ran into exactly the same problem when I tried to use the CustomScoreQuery and IntFieldSource classes from Lucene.
> I've tracked the problem down to the fact that IntFieldSource expects the contents of the field to actually be an integer as returned by FieldCache.getInts(). However, Solr converts a sortable int using NumberUtils.int2sortablestr().
> If I change my code to create a custom FieldCache.IntParser that applies NumberUtils.SortableStr2int before returning the value, my query works as expected. For example:
> class MyIntParser implements FieldCache.IntParser {
> public int parseInt(String val) { return NumberUtils.SortedStr2int(val, 0, val.length()); }
> }
> Query q = new BoostedQuery(qry, new IntFieldSource("myfield", new MyIntParser()));
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (SOLR-693) IntFieldSource incompatible with sint
field type
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley closed SOLR-693.
-----------------------------
Resolution: Invalid
IntFieldSource is for IntFieldType, SortableIntFieldSorce is probably what you are looking for.
You also shouldn't need a custom plugin just to do a BoostedQuery... stock solr should be fine.
Followup questions on this should go to solr-user.
> IntFieldSource incompatible with sint field type
> ------------------------------------------------
>
> Key: SOLR-693
> URL: https://issues.apache.org/jira/browse/SOLR-693
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.3
> Environment: RHEL 5, java6, builtin jetty container
> Reporter: Jerry Quinn
>
> I'm trying to create a custom scoring query in Solr to implement a date bias. I have a custom query parser that I'm using, that does nothing but wrap a BoostedQuery around the original query, which works in general.
> I'm indexing and storing the day number in an sint field. To implement my query, I extract the contents using org.apache.solr.search.function.IntFieldSource. Unfortunately, this throws an exception when it executes:
> HTTP ERROR: 500
> For input string: "€?"
> java.lang.NumberFormatException: For input string: "€?"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:61)
> at java.lang.Integer.parseInt(Integer.java:460)
> at java.lang.Integer.parseInt(Integer.java:510)
> at org.apache.lucene.search.FieldCacheImpl$3.parseInt(FieldCacheImpl.java:148)
> at org.apache.lucene.search.FieldCacheImpl$7.createValue(FieldCacheImpl.java:262)
> at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:245)
> at org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:239)
> at org.apache.solr.search.function.IntFieldSource.getValues(IntFieldSource.java:50)
> at org.apache.solr.search.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:103)
> at org.apache.solr.search.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:81)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:132)
> at org.apache.lucene.search.Searcher.search(Searcher.java:126)
> at org.apache.lucene.search.Searcher.search(Searcher.java:105)
> at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:966)
> at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:838)
> at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:269)
> at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:160)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:175)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1151)
> I ran into exactly the same problem when I tried to use the CustomScoreQuery and IntFieldSource classes from Lucene.
> I've tracked the problem down to the fact that IntFieldSource expects the contents of the field to actually be an integer as returned by FieldCache.getInts(). However, Solr converts a sortable int using NumberUtils.int2sortablestr().
> If I change my code to create a custom FieldCache.IntParser that applies NumberUtils.SortableStr2int before returning the value, my query works as expected. For example:
> class MyIntParser implements FieldCache.IntParser {
> public int parseInt(String val) { return NumberUtils.SortedStr2int(val, 0, val.length()); }
> }
> Query q = new BoostedQuery(qry, new IntFieldSource("myfield", new MyIntParser()));
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.