Sorting on a computed value

[tedsolr <ts...@sciquest.com>]

I'll bet some super user has figured this out. How can I perform a sort on a
single computed field? I have a QParserPlugin that is collapsing docs based
on data from multiple fields. I am summing the values from one numerical
field 'X'. I was going to use a DocTransformer to inject that summed value
into the search results as a new field. But I have now realized that I have
to be able to sort on this summed field.

Without retrieving all results (which could be 1M+) in my app and sorting
manually, is there any way to sort on my computed field within Solr?
(using Solr 4.9)



--
View this message in context: http://lucene.472066.n3.nabble.com/Sorting-on-a-computed-value-tp4181875.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Sorting on a computed value

[Mikhail Khludnev <mk...@griddynamics.com>]

I'm sorry for spoiling, but it's a fabulous FakeScorer pattern in Lucene.
e.g. look at
https://github.com/apache/lucene-solr/blob/trunk/lucene/grouping/src/java/org/apache/lucene/search/grouping/BlockGroupingCollector.java#L355
when your delegating collector is provided by scorer via setScorer(), it
shouldn't just pass it to the delegate as is, but pass a FakeScorer
instance instead (copy-paste your own and make it private, absolutely!),
which your collector can set docNum and application calculated score into
before delegating collect() notification. here is.

On Tue, Jan 27, 2015 at 12:30 AM, tedsolr <ts...@sciquest.com> wrote:

> That's an interesting link Shawn. Especially since it mentions the
> possibility of sorting on pseudo-fields.
>
> My delegating collector computes the customs stats and stores them in the
> request context. I have a doc transformer that then grabs the stats for
> each
> doc and inserts the data in the output. Here's a sample return doc:
>
>      {
>         "ITEM_DESCRIPTION": "FREIGHT PAY AMT FOR ITEM - 30934014",
>         "SUPPLIER_NAME": "JESUS ACOSTA MORENO",
>         "GL_ACCOUNT_NAME": "-",
>         "PART_NUMBER": "-",
>         "MCC_CODE": "SDBHAULER.NA",
>         "[AggregationStats]": {
>           "count": 1,
>           "spend": 8402.39
>         }
>       },
>
> My stats are in the [AggregationStats] "field". I don't know if this
> qualifies as a pseudo-field. Sorting happens before my doc transformer is
> called, so I don't think this data is available for sort. Like you said,
> I'm
> not using a function query to create this data, so maybe this idea won't
> work.
>
> I'm going to try to use doc scoring. If I can make the score match my
> pseudo
> fields then it might work.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Sorting-on-a-computed-value-tp4181875p4182060.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
<mk...@griddynamics.com>

Re: Sorting on a computed value

[tedsolr <ts...@sciquest.com>]

That's an interesting link Shawn. Especially since it mentions the
possibility of sorting on pseudo-fields.

My delegating collector computes the customs stats and stores them in the
request context. I have a doc transformer that then grabs the stats for each
doc and inserts the data in the output. Here's a sample return doc:

     {
        "ITEM_DESCRIPTION": "FREIGHT PAY AMT FOR ITEM - 30934014",
        "SUPPLIER_NAME": "JESUS ACOSTA MORENO",
        "GL_ACCOUNT_NAME": "-",
        "PART_NUMBER": "-",
        "MCC_CODE": "SDBHAULER.NA",
        "[AggregationStats]": {
          "count": 1,
          "spend": 8402.39
        }
      },

My stats are in the [AggregationStats] "field". I don't know if this
qualifies as a pseudo-field. Sorting happens before my doc transformer is
called, so I don't think this data is available for sort. Like you said, I'm
not using a function query to create this data, so maybe this idea won't
work.

I'm going to try to use doc scoring. If I can make the score match my pseudo
fields then it might work.



--
View this message in context: http://lucene.472066.n3.nabble.com/Sorting-on-a-computed-value-tp4181875p4182060.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Sorting on a computed value

[Shawn Heisey <ap...@elyograg.org>]

On 1/25/2015 4:13 PM, tedsolr wrote:
> I'll bet some super user has figured this out. How can I perform a sort on a
> single computed field? I have a QParserPlugin that is collapsing docs based
> on data from multiple fields. I am summing the values from one numerical
> field 'X'. I was going to use a DocTransformer to inject that summed value
> into the search results as a new field. But I have now realized that I have
> to be able to sort on this summed field.
> 
> Without retrieving all results (which could be 1M+) in my app and sorting
> manually, is there any way to sort on my computed field within Solr?
> (using Solr 4.9)

Sorting by a function query:

http://wiki.apache.org/solr/FunctionQuery#Sort_By_Function
https://cwiki.apache.org/confluence/display/solr/Function+Queries#FunctionQueries-SortByFunction

The second URL also shows how to put the results of a function into the
search results as a pseudo-field.

You do mention that you are talking about summing the values from one
field ... if that's not being done with a function query, then this
probably does not apply, and I don't know what you'll need.

Thanks,
Shawn