Re: Custom Payload Analyzer and Query

[Ian Lea <ia...@gmail.com>]

Not sure if you got an answer to this or not.  Don't recall seeing one
and gmail threading says not.

> Is the use of payloads I've described appropriate?

Sounds OK to me, although I'm not sure why you can't store the
metadata as a Document Field.

> Can I exclude/filter the matching terms based on the payload within a query itself ?

I think not.  Could if the metadata was an indexed Field.



--
Ian.


On Mon, Jan 30, 2012 at 10:24 PM,  <kt...@mmm.com> wrote:
> I'm working on providing advanced searching for annotated Medical
> Documents (using UIMA).  In the context of an annotated document, I
> identify relevant medical terms, as well as the negation of certain terms.
>  Following what I've read and seen in Lucene examples, I've been able to
> provide a search that takes into account the metadata contained in the
> payload.  Although very primitive, I've implemented a search which returns
> the payloads (using PayloadSpanUtil), and then excludes those terms where
> the payload doesn't meet the criteria.
>
> Is the use of payloads I've described appropriate?  Can I exclude/filter
> the matching terms based on the payload within a query itself ?   Are
> there any examples that do this?
>
> Cheers,
> Kyley

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Custom Payload Analyzer and Query

[Ian Lea <ia...@gmail.com>]

How does searching with PayloadSpanUtil/PayloadTermQuery/etc work to
exclude/filter the matching terms based on the payload within a query
itself, the original question?

The javadocs for PayloadSpanUtil say that the IndexReader should only
contain doc of interest so not much use for a general query on a
normal index.  PayloadTermQuery and PayloadNearQuery factor in the
value of the payloads using Similarity.scorePayload(...).  Can you
return 0 from that to exclude docs?


--
Ian.


On Tue, Feb 7, 2012 at 9:11 AM, Tommaso Teofili
<to...@gmail.com> wrote:
> 2012/2/6 Ian Lea <ia...@gmail.com>
>
>> Not sure if you got an answer to this or not.  Don't recall seeing one
>> and gmail threading says not.
>>
>> > Is the use of payloads I've described appropriate?
>>
>> Sounds OK to me, although I'm not sure why you can't store the
>> metadata as a Document Field.
>>
>> > Can I exclude/filter the matching terms based on the payload within a
>> query itself ?
>>
>> I think not.  Could if the metadata was an indexed Field.
>>
>
> What you may do is initially put your metadata inside the token type, then
> use the TypeTokenFilter to filter out some of them then "copy“ them inside
> the payloads using TypeAsPayloadTokenFilter and search with
> PayloadSpanUtil/PayloadTermQuery/etc.
>
> HTH,
> Tommaso
>
>
>>
>>
>>
>> --
>> Ian.
>>
>>
>> On Mon, Jan 30, 2012 at 10:24 PM,  <kt...@mmm.com> wrote:
>> > I'm working on providing advanced searching for annotated Medical
>> > Documents (using UIMA).  In the context of an annotated document, I
>> > identify relevant medical terms, as well as the negation of certain
>> terms.
>> >  Following what I've read and seen in Lucene examples, I've been able to
>> > provide a search that takes into account the metadata contained in the
>> > payload.  Although very primitive, I've implemented a search which
>> returns
>> > the payloads (using PayloadSpanUtil), and then excludes those terms where
>> > the payload doesn't meet the criteria.
>> >
>> > Is the use of payloads I've described appropriate?  Can I exclude/filter
>> > the matching terms based on the payload within a query itself ?   Are
>> > there any examples that do this?
>> >
>> > Cheers,
>> > Kyley
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Custom Payload Analyzer and Query

[Tommaso Teofili <to...@gmail.com>]

2012/2/6 Ian Lea <ia...@gmail.com>

> Not sure if you got an answer to this or not.  Don't recall seeing one
> and gmail threading says not.
>
> > Is the use of payloads I've described appropriate?
>
> Sounds OK to me, although I'm not sure why you can't store the
> metadata as a Document Field.
>
> > Can I exclude/filter the matching terms based on the payload within a
> query itself ?
>
> I think not.  Could if the metadata was an indexed Field.
>

What you may do is initially put your metadata inside the token type, then
use the TypeTokenFilter to filter out some of them then "copy“ them inside
the payloads using TypeAsPayloadTokenFilter and search with
PayloadSpanUtil/PayloadTermQuery/etc.

HTH,
Tommaso


>
>
>
> --
> Ian.
>
>
> On Mon, Jan 30, 2012 at 10:24 PM,  <kt...@mmm.com> wrote:
> > I'm working on providing advanced searching for annotated Medical
> > Documents (using UIMA).  In the context of an annotated document, I
> > identify relevant medical terms, as well as the negation of certain
> terms.
> >  Following what I've read and seen in Lucene examples, I've been able to
> > provide a search that takes into account the metadata contained in the
> > payload.  Although very primitive, I've implemented a search which
> returns
> > the payloads (using PayloadSpanUtil), and then excludes those terms where
> > the payload doesn't meet the criteria.
> >
> > Is the use of payloads I've described appropriate?  Can I exclude/filter
> > the matching terms based on the payload within a query itself ?   Are
> > there any examples that do this?
> >
> > Cheers,
> > Kyley
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>