You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by sp...@gmx.eu on 2008/01/28 15:27:44 UTC
TermVector
Hi,
how do I get the TermVector from a document which I have gotten from an
IndexSearcher via IndexSearcher#search(Query q).
Luke can do it, but I do not know how...
Thank you.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: TermVector
Posted by Grant Ingersoll <gs...@apache.org>.
Have a look at the SpanQuery, specifically the SpanNearQuery. The
getSpans() method will return a Spans object, which you can use to
access the positions.
-Grant
On Jan 29, 2008, at 7:17 AM, <sp...@gmx.eu> <sp...@gmx.eu> wrote:
>>> And how can I find the offsets of something like "foo bar"?
>> I think
>>> this
>>> will get tokenized into 2 terms and thus I have no chance to find
>>> it, right?
>>
>> I wouldn't say no chance... TermVectorMapper would be good
>> for this,
>> as you can watch the terms as they are being loaded. Just
>> keep track
>> of your last term and see if it is "foo", when you hit "bar"
>>
>> What kind of special term are you looking for? There may be other
>> ways of solving your problem...
>
> Well. I do not only want to find documents with a certain phrase but
> the
> positions of these phrases (e.g. "foo bar") in the document too.
> It must be possible I think because a highlighter can do it?
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: TermVector
Posted by sp...@gmx.eu.
> > And how can I find the offsets of something like "foo bar"?
> I think
> > this
> > will get tokenized into 2 terms and thus I have no chance to find
> > it, right?
>
> I wouldn't say no chance... TermVectorMapper would be good
> for this,
> as you can watch the terms as they are being loaded. Just
> keep track
> of your last term and see if it is "foo", when you hit "bar"
>
> What kind of special term are you looking for? There may be other
> ways of solving your problem...
Well. I do not only want to find documents with a certain phrase but the
positions of these phrases (e.g. "foo bar") in the document too.
It must be possible I think because a highlighter can do it?
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: TermVector
Posted by Grant Ingersoll <gs...@apache.org>.
On Jan 28, 2008, at 4:04 PM, <sp...@gmx.eu> <sp...@gmx.eu> wrote:
>> Also, search the archives for Term Vector, as you will find
>> discussion
>> of it there.
>
> Ah I see, I need to cast it to TermPositionVector. OK.
yep
>
>
>> You may also, eventually, be interested in the new
>> TermVectorMapper capabilities in 2.3 which should help speed up the
>> processing of term vectors by providing a callback mechanism
>> to allow
>> you to load them into data structures that make sense for your
>> application.
>
> Hm.
>
> What I need is are startOffsets of a special term.
>
> I use
> TermPositionVector#getOffsets(TermPositionVector.indexOf("foo")).
>
> Can TermVectorMapper speed this up?
>
>
> And how can I find the offsets of something like "foo bar"? I think
> this
> will get tokenized into 2 terms and thus I have no chance to find
> it, right?
I wouldn't say no chance... TermVectorMapper would be good for this,
as you can watch the terms as they are being loaded. Just keep track
of your last term and see if it is "foo", when you hit "bar"
What kind of special term are you looking for? There may be other
ways of solving your problem...
-Grant
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: TermVector
Posted by sp...@gmx.eu.
> Also, search the archives for Term Vector, as you will find
> discussion
> of it there.
Ah I see, I need to cast it to TermPositionVector. OK.
> You may also, eventually, be interested in the new
> TermVectorMapper capabilities in 2.3 which should help speed up the
> processing of term vectors by providing a callback mechanism
> to allow
> you to load them into data structures that make sense for your
> application.
Hm.
What I need is are startOffsets of a special term.
I use TermPositionVector#getOffsets(TermPositionVector.indexOf("foo")).
Can TermVectorMapper speed this up?
And how can I find the offsets of something like "foo bar"? I think this
will get tokenized into 2 terms and thus I have no chance to find it, right?
Thank you.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: TermVector
Posted by Grant Ingersoll <gs...@apache.org>.
See IndexReader.getTermFreqVector() and various versions of that.
Also, search the archives for Term Vector, as you will find discussion
of it there. You may also, eventually, be interested in the new
TermVectorMapper capabilities in 2.3 which should help speed up the
processing of term vectors by providing a callback mechanism to allow
you to load them into data structures that make sense for your
application.
-Grant
On Jan 28, 2008, at 10:26 AM, <sp...@gmx.eu> <sp...@gmx.eu> wrote:
> Sorry, this was a bit nonsense ;)
>
> I store a document with a content field like this:
>
> Document#add(new Field("content", someReader,
> TermVector.WITH_OFFSETS));
>
> Later I search this document with an IndexSearcher and want the
> TermPositions from this single document.
>
> There is a IndexReader#termPositions(Term t) - but this returns the
> positions for the whole index, not a single document.
>
>> -----Original Message-----
>> From: spring@gmx.eu [mailto:spring@gmx.eu]
>> Sent: Montag, 28. Januar 2008 15:28
>> To: java-user@lucene.apache.org
>> Subject: TermVector
>>
>> Hi,
>>
>> how do I get the TermVector from a document which I have
>> gotten from an
>> IndexSearcher via IndexSearcher#search(Query q).
>>
>> Luke can do it, but I do not know how...
>>
>> Thank you.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
--------------------------
Grant Ingersoll
http://lucene.grantingersoll.com
http://www.lucenebootcamp.com
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: TermVector
Posted by sp...@gmx.eu.
Sorry, this was a bit nonsense ;)
I store a document with a content field like this:
Document#add(new Field("content", someReader, TermVector.WITH_OFFSETS));
Later I search this document with an IndexSearcher and want the
TermPositions from this single document.
There is a IndexReader#termPositions(Term t) - but this returns the
positions for the whole index, not a single document.
> -----Original Message-----
> From: spring@gmx.eu [mailto:spring@gmx.eu]
> Sent: Montag, 28. Januar 2008 15:28
> To: java-user@lucene.apache.org
> Subject: TermVector
>
> Hi,
>
> how do I get the TermVector from a document which I have
> gotten from an
> IndexSearcher via IndexSearcher#search(Query q).
>
> Luke can do it, but I do not know how...
>
> Thank you.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org