You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Anshul jain <an...@gmail.com> on 2008/10/23 15:07:53 UTC
Lucene Payload
Hi all,
Has anyone used the payload functionality in Lucene? I would really
appreciate if someone can provide an explain using a code or something.
Thanks,
Anshul
Re: Lucene Payload
Posted by Anshul jain <an...@gmail.com>.
Thanks Grant the presentation, it was very useful.
Can payload work for queries other than Term queries and Span queries? Or is
there any function to convert Query into span query?
Thanks
On Thu, Oct 23, 2008 at 4:08 PM, Grant Ingersoll <gs...@apache.org>wrote:
> You can search the archives for some background info. Also, Michael Busch
> has a nice presentation from ApacheCon at
> http://people.apache.org/~buschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt<http://people.apache.org/%7Ebuschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt>
>
> Basically, the payload allows you to associate an arbitrary byte array with
> 1 or more terms.
>
>
> On Oct 23, 2008, at 9:07 AM, Anshul jain wrote:
>
> Hi all,
>>
>> Has anyone used the payload functionality in Lucene? I would really
>> appreciate if someone can provide an explain using a code or something.
>>
>> Thanks,
>> Anshul
>>
>
> --------------------------
> Grant Ingersoll
> Lucene Boot Camp Training Nov. 3-4, 2008, ApacheCon US New Orleans.
> http://www.lucenebootcamp.com
>
>
> Lucene Helpful Hints:
> http://wiki.apache.org/lucene-java/BasicsOfPerformance
> http://wiki.apache.org/lucene-java/LuceneFAQ
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
Anshul Jain
Re: Lucene Payload
Posted by Grant Ingersoll <gs...@apache.org>.
On Oct 30, 2008, at 7:28 PM, Anshul jain wrote:
> I want to give more weight to some terms in the document. Like title
> of the
> book should be given more weight than the contents. And we are
> testing over
> a wide varieties of lucene queries, with quotes, w/o quotes, phrase,
> span
> etc.
If the whole title is important, just boost the Title field.
>
>
> As our system will be expecting more number of queries that contain
> nouns
> and documents in which that term in heading or title or in bold
> should be
> given higher score.
>
> But this payload thing is not supporting us fully. As it is only
> working for
> single term queries or boolean term queries. It would also be good
> if it
> supports phrase queries also, as using closeness of terms for
> scoring is
> quite important for us.
>
> Do you've any suggestions?
Could you get by using just field boosts?
As for phrase payload queries, I agree they would be nice to have.
Only way they are getting in is if someone donates a patch. I can
review it if you can produce it, but don't have time at the moment to
produce it.
-Grant
>
>
>
> On Thu, Oct 30, 2008 at 7:08 PM, Grant Ingersoll
> <gs...@apache.org>wrote:
>
>> Not directly, I don't think. Mark Miller contributed some
>> highlighting
>> code that converts phrase queries to SpanNearQueries, I believe,
>> but this
>> isn't general purpose. We probably need a QueryParser that
>> produces
>> SpanQueries instead of regular Queries, I suppose, but they aren't
>> always
>> 1-1. Ideally, there would be no perf. penalty for SpanQueries and
>> everything would be a span query, but until someone figures out
>> that magic,
>> I don't know.
>>
>> What is it that you are trying to do?
>>
>> -Grant
>>
>> On Oct 30, 2008, at 12:42 PM, Anshul jain wrote:
>>
>> Thanks Grant the presentation, it was very useful.
>>>
>>> Can payload work for queries other than Term queries and Span
>>> queries? Or
>>> is
>>> there any function to convert Query into span query?
>>>
>>> Thanks
>>>
>>>
>>>
>>>
>>> On Thu, Oct 23, 2008 at 4:08 PM, Grant Ingersoll
>>> <gsingers@apache.org
>>>> wrote:
>>>
>>> You can search the archives for some background info. Also, Michael
>>>> Busch
>>>> has a nice presentation from ApacheCon at
>>>>
>>>> http://people.apache.org/~buschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt
>>>> <http://people.apache.org/%7Ebuschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt
>>>> >
>>>> <
>>>> http://people.apache.org/%7Ebuschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt
>>>>>
>>>>
>>>>
>>>> Basically, the payload allows you to associate an arbitrary byte
>>>> array
>>>> with
>>>> 1 or more terms.
>>>>
>>>>
>>>> On Oct 23, 2008, at 9:07 AM, Anshul jain wrote:
>>>>
>>>> Hi all,
>>>>
>>>>>
>>>>> Has anyone used the payload functionality in Lucene? I would
>>>>> really
>>>>> appreciate if someone can provide an explain using a code or
>>>>> something.
>>>>>
>>>>> Thanks,
>>>>> Anshul
>>>>>
>>>>>
>>>> --------------------------
>>>> Grant Ingersoll
>>>> Lucene Boot Camp Training Nov. 3-4, 2008, ApacheCon US New Orleans.
>>>> http://www.lucenebootcamp.com
>>>>
>>>>
>>>> Lucene Helpful Hints:
>>>> http://wiki.apache.org/lucene-java/BasicsOfPerformance
>>>> http://wiki.apache.org/lucene-java/LuceneFAQ
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>>
>>>
>>> --
>>> Anshul Jain
>>>
>>
>> --------------------------
>> Grant Ingersoll
>> Lucene Boot Camp Training Nov. 3-4, 2008, ApacheCon US New Orleans.
>> http://www.lucenebootcamp.com
>>
>>
>> Lucene Helpful Hints:
>> http://wiki.apache.org/lucene-java/BasicsOfPerformance
>> http://wiki.apache.org/lucene-java/LuceneFAQ
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
> --
> Anshul Jain
--------------------------
Grant Ingersoll
Lucene Boot Camp Training Nov. 3-4, 2008, ApacheCon US New Orleans.
http://www.lucenebootcamp.com
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Payload
Posted by Anshul jain <an...@gmail.com>.
I want to give more weight to some terms in the document. Like title of the
book should be given more weight than the contents. And we are testing over
a wide varieties of lucene queries, with quotes, w/o quotes, phrase, span
etc.
As our system will be expecting more number of queries that contain nouns
and documents in which that term in heading or title or in bold should be
given higher score.
But this payload thing is not supporting us fully. As it is only working for
single term queries or boolean term queries. It would also be good if it
supports phrase queries also, as using closeness of terms for scoring is
quite important for us.
Do you've any suggestions?
On Thu, Oct 30, 2008 at 7:08 PM, Grant Ingersoll <gs...@apache.org>wrote:
> Not directly, I don't think. Mark Miller contributed some highlighting
> code that converts phrase queries to SpanNearQueries, I believe, but this
> isn't general purpose. We probably need a QueryParser that produces
> SpanQueries instead of regular Queries, I suppose, but they aren't always
> 1-1. Ideally, there would be no perf. penalty for SpanQueries and
> everything would be a span query, but until someone figures out that magic,
> I don't know.
>
> What is it that you are trying to do?
>
> -Grant
>
> On Oct 30, 2008, at 12:42 PM, Anshul jain wrote:
>
> Thanks Grant the presentation, it was very useful.
>>
>> Can payload work for queries other than Term queries and Span queries? Or
>> is
>> there any function to convert Query into span query?
>>
>> Thanks
>>
>>
>>
>>
>> On Thu, Oct 23, 2008 at 4:08 PM, Grant Ingersoll <gsingers@apache.org
>> >wrote:
>>
>> You can search the archives for some background info. Also, Michael
>>> Busch
>>> has a nice presentation from ApacheCon at
>>>
>>> http://people.apache.org/~buschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt<http://people.apache.org/%7Ebuschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt>
>>> <
>>> http://people.apache.org/%7Ebuschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt
>>> >
>>>
>>>
>>> Basically, the payload allows you to associate an arbitrary byte array
>>> with
>>> 1 or more terms.
>>>
>>>
>>> On Oct 23, 2008, at 9:07 AM, Anshul jain wrote:
>>>
>>> Hi all,
>>>
>>>>
>>>> Has anyone used the payload functionality in Lucene? I would really
>>>> appreciate if someone can provide an explain using a code or something.
>>>>
>>>> Thanks,
>>>> Anshul
>>>>
>>>>
>>> --------------------------
>>> Grant Ingersoll
>>> Lucene Boot Camp Training Nov. 3-4, 2008, ApacheCon US New Orleans.
>>> http://www.lucenebootcamp.com
>>>
>>>
>>> Lucene Helpful Hints:
>>> http://wiki.apache.org/lucene-java/BasicsOfPerformance
>>> http://wiki.apache.org/lucene-java/LuceneFAQ
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>>
>> --
>> Anshul Jain
>>
>
> --------------------------
> Grant Ingersoll
> Lucene Boot Camp Training Nov. 3-4, 2008, ApacheCon US New Orleans.
> http://www.lucenebootcamp.com
>
>
> Lucene Helpful Hints:
> http://wiki.apache.org/lucene-java/BasicsOfPerformance
> http://wiki.apache.org/lucene-java/LuceneFAQ
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
Anshul Jain
Re: Lucene Payload
Posted by Grant Ingersoll <gs...@apache.org>.
Not directly, I don't think. Mark Miller contributed some
highlighting code that converts phrase queries to SpanNearQueries, I
believe, but this isn't general purpose. We probably need a
QueryParser that produces SpanQueries instead of regular Queries, I
suppose, but they aren't always 1-1. Ideally, there would be no perf.
penalty for SpanQueries and everything would be a span query, but
until someone figures out that magic, I don't know.
What is it that you are trying to do?
-Grant
On Oct 30, 2008, at 12:42 PM, Anshul jain wrote:
> Thanks Grant the presentation, it was very useful.
>
> Can payload work for queries other than Term queries and Span
> queries? Or is
> there any function to convert Query into span query?
>
> Thanks
>
>
>
>
> On Thu, Oct 23, 2008 at 4:08 PM, Grant Ingersoll
> <gs...@apache.org>wrote:
>
>> You can search the archives for some background info. Also,
>> Michael Busch
>> has a nice presentation from ApacheCon at
>> http://people.apache.org/~buschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt
>> <http://people.apache.org/%7Ebuschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt
>> >
>>
>> Basically, the payload allows you to associate an arbitrary byte
>> array with
>> 1 or more terms.
>>
>>
>> On Oct 23, 2008, at 9:07 AM, Anshul jain wrote:
>>
>> Hi all,
>>>
>>> Has anyone used the payload functionality in Lucene? I would really
>>> appreciate if someone can provide an explain using a code or
>>> something.
>>>
>>> Thanks,
>>> Anshul
>>>
>>
>> --------------------------
>> Grant Ingersoll
>> Lucene Boot Camp Training Nov. 3-4, 2008, ApacheCon US New Orleans.
>> http://www.lucenebootcamp.com
>>
>>
>> Lucene Helpful Hints:
>> http://wiki.apache.org/lucene-java/BasicsOfPerformance
>> http://wiki.apache.org/lucene-java/LuceneFAQ
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
> --
> Anshul Jain
--------------------------
Grant Ingersoll
Lucene Boot Camp Training Nov. 3-4, 2008, ApacheCon US New Orleans.
http://www.lucenebootcamp.com
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Payload
Posted by Anshul jain <an...@epfl.ch>.
Thanks Grant the presentation, it was very useful.
Can payload work for queries other than Term queries and Span queries? Or is
there any function to convert Query into span query?
Thanks
On Thu, Oct 23, 2008 at 4:08 PM, Grant Ingersoll <gs...@apache.org>wrote:
> You can search the archives for some background info. Also, Michael Busch
> has a nice presentation from ApacheCon at
> http://people.apache.org/~buschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt<http://people.apache.org/%7Ebuschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt>
>
> Basically, the payload allows you to associate an arbitrary byte array with
> 1 or more terms.
>
>
> On Oct 23, 2008, at 9:07 AM, Anshul jain wrote:
>
> Hi all,
>>
>> Has anyone used the payload functionality in Lucene? I would really
>> appreciate if someone can provide an explain using a code or something.
>>
>> Thanks,
>> Anshul
>>
>
> --------------------------
> Grant Ingersoll
> Lucene Boot Camp Training Nov. 3-4, 2008, ApacheCon US New Orleans.
> http://www.lucenebootcamp.com
>
>
> Lucene Helpful Hints:
> http://wiki.apache.org/lucene-java/BasicsOfPerformance
> http://wiki.apache.org/lucene-java/LuceneFAQ
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
Anshul Jain
Re: Lucene Payload
Posted by Grant Ingersoll <gs...@apache.org>.
You can search the archives for some background info. Also, Michael
Busch has a nice presentation from ApacheCon at http://people.apache.org/~buschmi/apachecon/AdvancedIndexingLuceneAtlanta07.ppt
Basically, the payload allows you to associate an arbitrary byte array
with 1 or more terms.
On Oct 23, 2008, at 9:07 AM, Anshul jain wrote:
> Hi all,
>
> Has anyone used the payload functionality in Lucene? I would really
> appreciate if someone can provide an explain using a code or
> something.
>
> Thanks,
> Anshul
--------------------------
Grant Ingersoll
Lucene Boot Camp Training Nov. 3-4, 2008, ApacheCon US New Orleans.
http://www.lucenebootcamp.com
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org