You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Mukundaraman valakumaresan <mu...@8kmiles.com> on 2013/12/09 15:54:15 UTC
Getting Solr Document Attributes from a Custom Function
Hi All,
I have a written a custom solr function and I would like to read a property
of the document inside my custom function. Is it possible to get that using
Solr?
For eg. inside the floatVal method, I would like to get the value of the
attribute "name"
public class CustomValueSource extends ValueSource {
@Override
public FunctionValues getValues(Map context,
AtomicReaderContext readerContext) throws IOException {
return new FloatDocValues(this) { @Override public float floatVal(int doc)
{
/*******
getDocument(doc).getAttribute("name")
********/ }}}
Thanks & Regards
Mukund
Re: Getting Solr Document Attributes from a Custom Function
Posted by Mukundaraman valakumaresan <mu...@8kmiles.com>.
Hi
Thanks a lot , that helps
Regards
Mukund
On Thu, Dec 12, 2013 at 1:18 AM, Kydryavtsev Andrey <we...@yandex.ru>wrote:
> As I know (not 100% sure actually), function queries don't work with
> multivalued fields. Why do you need multivalued fields here? Your "price"
> and "numberOfCities" don't look like multivalued. At least you can try to
> use, you know, some tricky format like
> "50;40;20" to index multivalued field as single-valued and then parse this
> into values list in function.
>
> 11.12.2013, 11:13, "Mukundaraman valakumaresan" <mu...@8kmiles.com>:
> > Hi Kydryavtsev
> >
> > Thanks a lot it works, but how do i pass a multivalued field values to a
> > function query?
> >
> > Can it be passed as a String array?
> >
> > Thanks & Regards
> > Mukund
> >
> > On Tue, Dec 10, 2013 at 12:05 PM, Kydryavtsev Andrey <werder06@yandex.ru
> >wrote:
> >
> >> You can implement it in this way:
> >> Index number of cities as new int field (like <field
> >> name="numberOfCities">2</field>) and implement user function like
> >>
> >> "customFunction(price, numberOfCities, 10000, 2000, 5)"
> >>
> >> Custom parser should parse this into value sources list. From first two
> >> field sources we can get per doc value for this particular fields,
> another
> >> three will be ConstValueSource instances - just constants, so we can
> access
> >> all 5 values and implement custom formula per doc id. Find examples in
> >> ValueSourceParser and solr functions like DefFunction or
> MinFloatFunction
> >>
> >> 10.12.2013, 09:31, "Mukundaraman valakumaresan" <mu...@8kmiles.com>:
> >>> Hi Hoss,
> >>>
> >>> Thanks a lot for your response. The actual problem is,
> >>>
> >>> For every record that I query, I have to execute a formula and sort
> the
> >>> records based on the value of the formula.
> >>> The formula has elements from the record.
> >>>
> >>> For eg. for the following document ,I need to apply the formula
> >> (maxprice -
> >>> solrprice)/ (maxprice - minprice) + count(cities)/totalcities.
> >>> where maxprice, maxprice and total cities will be available at run
> time.
> >>>
> >>> So for the following record, it has to execute as (10000 -
> >>> *5000*)/(10000-2000)
> >>> + *2*/5 (where 5000 and 2, which are in bold are from the document)
> >>>
> >>> <doc>
> >>> <field name="id">apartment_1</field>
> >>> <field name="name">Casa Grande</field>
> >>> <field name="locality">chennai</field>
> >>> <field name="locality">bangalore</field>
> >>> <field name="price">5000</field>
> >>> </doc>
> >>>
> >>> Thanks & Regards
> >>> Mukund
> >>>
> >>> On Tue, Dec 10, 2013 at 12:22 AM, Chris Hostetter
> >>> <ho...@fucit.org>wrote:
> >>>> Smells like an XY problem ...
> >>>>
> >>>> Can you please describe what your end goal is in writing a custom
> >>>> function, and what you would do with things like the "name" field
> >> inside
> >>>> your funciton?
> >>>>
> >>>> In general, accessing stored field values for indexed documents ca
> be
> >>>> prohibitively expensive, it rather defeats the entire point of the
> >>>> inverted index data structure. If you help us understand what your
> >> goal
> >>>> is, people may be able to offer performant suggestions.
> >>>>
> >>>> https://people.apache.org/~hossman/#xyproblem
> >>>> XY Problem
> >>>>
> >>>> Your question appears to be an "XY Problem" ... that is: you are
> >> dealing
> >>>> with "X", you are assuming "Y" will help you, and you are asking
> about
> >> "Y"
> >>>> without giving more details about the "X" so that we can understand
> the
> >>>> full issue. Perhaps the best solution doesn't involve "Y" at all?
> >>>> See Also: http://www.perlmonks.org/index.pl?node_id=542341
> >>>>
> >>>> : Date: Mon, 9 Dec 2013 20:24:15 +0530
> >>>> : From: Mukundaraman valakumaresan <mu...@8kmiles.com>
> >>>> : Reply-To: solr-user@lucene.apache.org
> >>>> : To: solr-user@lucene.apache.org
> >>>> : Subject: Getting Solr Document Attributes from a Custom Function
> >>>> :
> >>>> : Hi All,
> >>>> :
> >>>> : I have a written a custom solr function and I would like to read a
> >>>> property
> >>>> : of the document inside my custom function. Is it possible to get
> that
> >>>> using
> >>>> : Solr?
> >>>> :
> >>>> : For eg. inside the floatVal method, I would like to get the value
> of
> >> the
> >>>> : attribute "name"
> >>>> :
> >>>> : public class CustomValueSource extends ValueSource {
> >>>> :
> >>>> : @Override
> >>>> : public FunctionValues getValues(Map context,
> >>>> : AtomicReaderContext readerContext) throws IOException {
> >>>> : return new FloatDocValues(this) { @Override public float
> >> floatVal(int
> >>>> doc)
> >>>> : {
> >>>> : /*******
> >>>> : getDocument(doc).getAttribute("name")
> >>>> :
> >>>> : ********/ }}}
> >>>> :
> >>>> : Thanks & Regards
> >>>> : Mukund
> >>>> :
> >>>>
> >>>> -Hoss
> >>>> http://www.lucidworks.com/
>
Re: Getting Solr Document Attributes from a Custom Function
Posted by Kydryavtsev Andrey <we...@yandex.ru>.
As I know (not 100% sure actually), function queries don't work with multivalued fields. Why do you need multivalued fields here? Your "price" and "numberOfCities" don't look like multivalued. At least you can try to use, you know, some tricky format like
"50;40;20" to index multivalued field as single-valued and then parse this into values list in function.
11.12.2013, 11:13, "Mukundaraman valakumaresan" <mu...@8kmiles.com>:
> Hi Kydryavtsev
>
> Thanks a lot it works, but how do i pass a multivalued field values to a
> function query?
>
> Can it be passed as a String array?
>
> Thanks & Regards
> Mukund
>
> On Tue, Dec 10, 2013 at 12:05 PM, Kydryavtsev Andrey <we...@yandex.ru>wrote:
>
>> You can implement it in this way:
>> Index number of cities as new int field (like <field
>> name="numberOfCities">2</field>) and implement user function like
>>
>> "customFunction(price, numberOfCities, 10000, 2000, 5)"
>>
>> Custom parser should parse this into value sources list. From first two
>> field sources we can get per doc value for this particular fields, another
>> three will be ConstValueSource instances - just constants, so we can access
>> all 5 values and implement custom formula per doc id. Find examples in
>> ValueSourceParser and solr functions like DefFunction or MinFloatFunction
>>
>> 10.12.2013, 09:31, "Mukundaraman valakumaresan" <mu...@8kmiles.com>:
>>> Hi Hoss,
>>>
>>> Thanks a lot for your response. The actual problem is,
>>>
>>> For every record that I query, I have to execute a formula and sort the
>>> records based on the value of the formula.
>>> The formula has elements from the record.
>>>
>>> For eg. for the following document ,I need to apply the formula
>> (maxprice -
>>> solrprice)/ (maxprice - minprice) + count(cities)/totalcities.
>>> where maxprice, maxprice and total cities will be available at run time.
>>>
>>> So for the following record, it has to execute as (10000 -
>>> *5000*)/(10000-2000)
>>> + *2*/5 (where 5000 and 2, which are in bold are from the document)
>>>
>>> <doc>
>>> <field name="id">apartment_1</field>
>>> <field name="name">Casa Grande</field>
>>> <field name="locality">chennai</field>
>>> <field name="locality">bangalore</field>
>>> <field name="price">5000</field>
>>> </doc>
>>>
>>> Thanks & Regards
>>> Mukund
>>>
>>> On Tue, Dec 10, 2013 at 12:22 AM, Chris Hostetter
>>> <ho...@fucit.org>wrote:
>>>> Smells like an XY problem ...
>>>>
>>>> Can you please describe what your end goal is in writing a custom
>>>> function, and what you would do with things like the "name" field
>> inside
>>>> your funciton?
>>>>
>>>> In general, accessing stored field values for indexed documents ca be
>>>> prohibitively expensive, it rather defeats the entire point of the
>>>> inverted index data structure. If you help us understand what your
>> goal
>>>> is, people may be able to offer performant suggestions.
>>>>
>>>> https://people.apache.org/~hossman/#xyproblem
>>>> XY Problem
>>>>
>>>> Your question appears to be an "XY Problem" ... that is: you are
>> dealing
>>>> with "X", you are assuming "Y" will help you, and you are asking about
>> "Y"
>>>> without giving more details about the "X" so that we can understand the
>>>> full issue. Perhaps the best solution doesn't involve "Y" at all?
>>>> See Also: http://www.perlmonks.org/index.pl?node_id=542341
>>>>
>>>> : Date: Mon, 9 Dec 2013 20:24:15 +0530
>>>> : From: Mukundaraman valakumaresan <mu...@8kmiles.com>
>>>> : Reply-To: solr-user@lucene.apache.org
>>>> : To: solr-user@lucene.apache.org
>>>> : Subject: Getting Solr Document Attributes from a Custom Function
>>>> :
>>>> : Hi All,
>>>> :
>>>> : I have a written a custom solr function and I would like to read a
>>>> property
>>>> : of the document inside my custom function. Is it possible to get that
>>>> using
>>>> : Solr?
>>>> :
>>>> : For eg. inside the floatVal method, I would like to get the value of
>> the
>>>> : attribute "name"
>>>> :
>>>> : public class CustomValueSource extends ValueSource {
>>>> :
>>>> : @Override
>>>> : public FunctionValues getValues(Map context,
>>>> : AtomicReaderContext readerContext) throws IOException {
>>>> : return new FloatDocValues(this) { @Override public float
>> floatVal(int
>>>> doc)
>>>> : {
>>>> : /*******
>>>> : getDocument(doc).getAttribute("name")
>>>> :
>>>> : ********/ }}}
>>>> :
>>>> : Thanks & Regards
>>>> : Mukund
>>>> :
>>>>
>>>> -Hoss
>>>> http://www.lucidworks.com/
Re: Getting Solr Document Attributes from a Custom Function
Posted by Mukundaraman valakumaresan <mu...@8kmiles.com>.
Hi Kydryavtsev
Thanks a lot it works, but how do i pass a multivalued field values to a
function query?
Can it be passed as a String array?
Thanks & Regards
Mukund
On Tue, Dec 10, 2013 at 12:05 PM, Kydryavtsev Andrey <we...@yandex.ru>wrote:
> You can implement it in this way:
> Index number of cities as new int field (like <field
> name="numberOfCities">2</field>) and implement user function like
>
> "customFunction(price, numberOfCities, 10000, 2000, 5)"
>
> Custom parser should parse this into value sources list. From first two
> field sources we can get per doc value for this particular fields, another
> three will be ConstValueSource instances - just constants, so we can access
> all 5 values and implement custom formula per doc id. Find examples in
> ValueSourceParser and solr functions like DefFunction or MinFloatFunction
>
> 10.12.2013, 09:31, "Mukundaraman valakumaresan" <mu...@8kmiles.com>:
> > Hi Hoss,
> >
> > Thanks a lot for your response. The actual problem is,
> >
> > For every record that I query, I have to execute a formula and sort the
> > records based on the value of the formula.
> > The formula has elements from the record.
> >
> > For eg. for the following document ,I need to apply the formula
> (maxprice -
> > solrprice)/ (maxprice - minprice) + count(cities)/totalcities.
> > where maxprice, maxprice and total cities will be available at run time.
> >
> > So for the following record, it has to execute as (10000 -
> > *5000*)/(10000-2000)
> > + *2*/5 (where 5000 and 2, which are in bold are from the document)
> >
> > <doc>
> > <field name="id">apartment_1</field>
> > <field name="name">Casa Grande</field>
> > <field name="locality">chennai</field>
> > <field name="locality">bangalore</field>
> > <field name="price">5000</field>
> > </doc>
> >
> > Thanks & Regards
> > Mukund
> >
> > On Tue, Dec 10, 2013 at 12:22 AM, Chris Hostetter
> > <ho...@fucit.org>wrote:
> >
> >> Smells like an XY problem ...
> >>
> >> Can you please describe what your end goal is in writing a custom
> >> function, and what you would do with things like the "name" field
> inside
> >> your funciton?
> >>
> >> In general, accessing stored field values for indexed documents ca be
> >> prohibitively expensive, it rather defeats the entire point of the
> >> inverted index data structure. If you help us understand what your
> goal
> >> is, people may be able to offer performant suggestions.
> >>
> >> https://people.apache.org/~hossman/#xyproblem
> >> XY Problem
> >>
> >> Your question appears to be an "XY Problem" ... that is: you are
> dealing
> >> with "X", you are assuming "Y" will help you, and you are asking about
> "Y"
> >> without giving more details about the "X" so that we can understand the
> >> full issue. Perhaps the best solution doesn't involve "Y" at all?
> >> See Also: http://www.perlmonks.org/index.pl?node_id=542341
> >>
> >> : Date: Mon, 9 Dec 2013 20:24:15 +0530
> >> : From: Mukundaraman valakumaresan <mu...@8kmiles.com>
> >> : Reply-To: solr-user@lucene.apache.org
> >> : To: solr-user@lucene.apache.org
> >> : Subject: Getting Solr Document Attributes from a Custom Function
> >> :
> >> : Hi All,
> >> :
> >> : I have a written a custom solr function and I would like to read a
> >> property
> >> : of the document inside my custom function. Is it possible to get that
> >> using
> >> : Solr?
> >> :
> >> : For eg. inside the floatVal method, I would like to get the value of
> the
> >> : attribute "name"
> >> :
> >> : public class CustomValueSource extends ValueSource {
> >> :
> >> : @Override
> >> : public FunctionValues getValues(Map context,
> >> : AtomicReaderContext readerContext) throws IOException {
> >> : return new FloatDocValues(this) { @Override public float
> floatVal(int
> >> doc)
> >> : {
> >> : /*******
> >> : getDocument(doc).getAttribute("name")
> >> :
> >> : ********/ }}}
> >> :
> >> : Thanks & Regards
> >> : Mukund
> >> :
> >>
> >> -Hoss
> >> http://www.lucidworks.com/
>
Re: Getting Solr Document Attributes from a Custom Function
Posted by Kydryavtsev Andrey <we...@yandex.ru>.
You can implement it in this way:
Index number of cities as new int field (like <field name="numberOfCities">2</field>) and implement user function like
"customFunction(price, numberOfCities, 10000, 2000, 5)"
Custom parser should parse this into value sources list. From first two field sources we can get per doc value for this particular fields, another three will be ConstValueSource instances - just constants, so we can access all 5 values and implement custom formula per doc id. Find examples in ValueSourceParser and solr functions like DefFunction or MinFloatFunction
10.12.2013, 09:31, "Mukundaraman valakumaresan" <mu...@8kmiles.com>:
> Hi Hoss,
>
> Thanks a lot for your response. The actual problem is,
>
> For every record that I query, I have to execute a formula and sort the
> records based on the value of the formula.
> The formula has elements from the record.
>
> For eg. for the following document ,I need to apply the formula (maxprice -
> solrprice)/ (maxprice - minprice) + count(cities)/totalcities.
> where maxprice, maxprice and total cities will be available at run time.
>
> So for the following record, it has to execute as (10000 -
> *5000*)/(10000-2000)
> + *2*/5 (where 5000 and 2, which are in bold are from the document)
>
> <doc>
> <field name="id">apartment_1</field>
> <field name="name">Casa Grande</field>
> <field name="locality">chennai</field>
> <field name="locality">bangalore</field>
> <field name="price">5000</field>
> </doc>
>
> Thanks & Regards
> Mukund
>
> On Tue, Dec 10, 2013 at 12:22 AM, Chris Hostetter
> <ho...@fucit.org>wrote:
>
>> Smells like an XY problem ...
>>
>> Can you please describe what your end goal is in writing a custom
>> function, and what you would do with things like the "name" field inside
>> your funciton?
>>
>> In general, accessing stored field values for indexed documents ca be
>> prohibitively expensive, it rather defeats the entire point of the
>> inverted index data structure. If you help us understand what your goal
>> is, people may be able to offer performant suggestions.
>>
>> https://people.apache.org/~hossman/#xyproblem
>> XY Problem
>>
>> Your question appears to be an "XY Problem" ... that is: you are dealing
>> with "X", you are assuming "Y" will help you, and you are asking about "Y"
>> without giving more details about the "X" so that we can understand the
>> full issue. Perhaps the best solution doesn't involve "Y" at all?
>> See Also: http://www.perlmonks.org/index.pl?node_id=542341
>>
>> : Date: Mon, 9 Dec 2013 20:24:15 +0530
>> : From: Mukundaraman valakumaresan <mu...@8kmiles.com>
>> : Reply-To: solr-user@lucene.apache.org
>> : To: solr-user@lucene.apache.org
>> : Subject: Getting Solr Document Attributes from a Custom Function
>> :
>> : Hi All,
>> :
>> : I have a written a custom solr function and I would like to read a
>> property
>> : of the document inside my custom function. Is it possible to get that
>> using
>> : Solr?
>> :
>> : For eg. inside the floatVal method, I would like to get the value of the
>> : attribute "name"
>> :
>> : public class CustomValueSource extends ValueSource {
>> :
>> : @Override
>> : public FunctionValues getValues(Map context,
>> : AtomicReaderContext readerContext) throws IOException {
>> : return new FloatDocValues(this) { @Override public float floatVal(int
>> doc)
>> : {
>> : /*******
>> : getDocument(doc).getAttribute("name")
>> :
>> : ********/ }}}
>> :
>> : Thanks & Regards
>> : Mukund
>> :
>>
>> -Hoss
>> http://www.lucidworks.com/
Re: Getting Solr Document Attributes from a Custom Function
Posted by Mukundaraman valakumaresan <mu...@8kmiles.com>.
Hi Hoss,
Thanks a lot for your response. The actual problem is,
For every record that I query, I have to execute a formula and sort the
records based on the value of the formula.
The formula has elements from the record.
For eg. for the following document ,I need to apply the formula (maxprice -
solrprice)/ (maxprice - minprice) + count(cities)/totalcities.
where maxprice, maxprice and total cities will be available at run time.
So for the following record, it has to execute as (10000 -
*5000*)/(10000-2000)
+ *2*/5 (where 5000 and 2, which are in bold are from the document)
<doc>
<field name="id">apartment_1</field>
<field name="name">Casa Grande</field>
<field name="locality">chennai</field>
<field name="locality">bangalore</field>
<field name="price">5000</field>
</doc>
Thanks & Regards
Mukund
On Tue, Dec 10, 2013 at 12:22 AM, Chris Hostetter
<ho...@fucit.org>wrote:
>
> Smells like an XY problem ...
>
> Can you please describe what your end goal is in writing a custom
> function, and what you would do with things like the "name" field inside
> your funciton?
>
> In general, accessing stored field values for indexed documents ca be
> prohibitively expensive, it rather defeats the entire point of the
> inverted index data structure. If you help us understand what your goal
> is, people may be able to offer performant suggestions.
>
>
>
> https://people.apache.org/~hossman/#xyproblem
> XY Problem
>
> Your question appears to be an "XY Problem" ... that is: you are dealing
> with "X", you are assuming "Y" will help you, and you are asking about "Y"
> without giving more details about the "X" so that we can understand the
> full issue. Perhaps the best solution doesn't involve "Y" at all?
> See Also: http://www.perlmonks.org/index.pl?node_id=542341
>
>
>
>
> : Date: Mon, 9 Dec 2013 20:24:15 +0530
> : From: Mukundaraman valakumaresan <mu...@8kmiles.com>
> : Reply-To: solr-user@lucene.apache.org
> : To: solr-user@lucene.apache.org
> : Subject: Getting Solr Document Attributes from a Custom Function
> :
> : Hi All,
> :
> : I have a written a custom solr function and I would like to read a
> property
> : of the document inside my custom function. Is it possible to get that
> using
> : Solr?
> :
> : For eg. inside the floatVal method, I would like to get the value of the
> : attribute "name"
> :
> : public class CustomValueSource extends ValueSource {
> :
> : @Override
> : public FunctionValues getValues(Map context,
> : AtomicReaderContext readerContext) throws IOException {
> : return new FloatDocValues(this) { @Override public float floatVal(int
> doc)
> : {
> : /*******
> : getDocument(doc).getAttribute("name")
> :
> : ********/ }}}
> :
> : Thanks & Regards
> : Mukund
> :
>
> -Hoss
> http://www.lucidworks.com/
>
Re: Getting Solr Document Attributes from a Custom Function
Posted by Chris Hostetter <ho...@fucit.org>.
Smells like an XY problem ...
Can you please describe what your end goal is in writing a custom
function, and what you would do with things like the "name" field inside
your funciton?
In general, accessing stored field values for indexed documents ca be
prohibitively expensive, it rather defeats the entire point of the
inverted index data structure. If you help us understand what your goal
is, people may be able to offer performant suggestions.
https://people.apache.org/~hossman/#xyproblem
XY Problem
Your question appears to be an "XY Problem" ... that is: you are dealing
with "X", you are assuming "Y" will help you, and you are asking about "Y"
without giving more details about the "X" so that we can understand the
full issue. Perhaps the best solution doesn't involve "Y" at all?
See Also: http://www.perlmonks.org/index.pl?node_id=542341
: Date: Mon, 9 Dec 2013 20:24:15 +0530
: From: Mukundaraman valakumaresan <mu...@8kmiles.com>
: Reply-To: solr-user@lucene.apache.org
: To: solr-user@lucene.apache.org
: Subject: Getting Solr Document Attributes from a Custom Function
:
: Hi All,
:
: I have a written a custom solr function and I would like to read a property
: of the document inside my custom function. Is it possible to get that using
: Solr?
:
: For eg. inside the floatVal method, I would like to get the value of the
: attribute "name"
:
: public class CustomValueSource extends ValueSource {
:
: @Override
: public FunctionValues getValues(Map context,
: AtomicReaderContext readerContext) throws IOException {
: return new FloatDocValues(this) { @Override public float floatVal(int doc)
: {
: /*******
: getDocument(doc).getAttribute("name")
:
: ********/ }}}
:
: Thanks & Regards
: Mukund
:
-Hoss
http://www.lucidworks.com/