You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Günter Ladwig <gl...@aifb.uni-karlsruhe.de> on 2009/07/14 17:12:30 UTC
Stream field values
Hi,
I have a situation, where stored, un-indexed fields can contain
potentially large amounts of data. Is it possibly to read the contents
of a field incrementally? That is, do not load the complete contents
from disk, but read X bytes at a time. Does the Reader returned by
Field.readerValue() work that way? Or is this only possible with
tokenized fields?
Thanks!
Regards,
Günter
--
Dipl.-Inform. Günter Ladwig
Institute AIFB, University of Karlsruhe, D-76128 Karlsruhe
Phone +49 (0)721 608 7946 Building 11.40, Room 250
gla@aifb.uni-karlsruhe.de www.aifb.uni-karlsruhe.de
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Stream field values
Posted by Günter Ladwig <gl...@aifb.uni-karlsruhe.de>.
Hi,
thanks for your answer. I know about lazy loading fields, but my
question is whether fields are always loaded as a whole or if it is
possible in some way to stream a field's contents.
Regards,
Günter
--
Dipl.-Inform. Günter Ladwig
Institute AIFB, University of Karlsruhe, D-76128 Karlsruhe
Phone +49 (0)721 608 7946 Building 11.40, Room 250
gla@aifb.uni-karlsruhe.de www.aifb.uni-karlsruhe.de
On 14.07.2009, at 19:21, Grant Ingersoll wrote:
> Have a look at the FieldSelector and the Lazy load capability. See http://www.lucidimagination.com/search/?q=FieldSelector
> for some pointers.
>
> -Grant
> On Jul 14, 2009, at 11:12 AM, Günter Ladwig wrote:
>
>> Hi,
>>
>> I have a situation, where stored, un-indexed fields can contain
>> potentially large amounts of data. Is it possibly to read the
>> contents of a field incrementally? That is, do not load the
>> complete contents from disk, but read X bytes at a time. Does the
>> Reader returned by Field.readerValue() work that way? Or is this
>> only possible with tokenized fields?
>>
>> Thanks!
>>
>> Regards,
>> Günter
>> --
>> Dipl.-Inform. Günter Ladwig
>> Institute AIFB, University of Karlsruhe, D-76128 Karlsruhe
>> Phone +49 (0)721 608 7946 Building 11.40, Room 250
>> gla@aifb.uni-karlsruhe.de www.aifb.uni-karlsruhe.de
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
> using Solr/Lucene:
> http://www.lucidimagination.com/search
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Stream field values
Posted by Günter Ladwig <gl...@gmail.com>.
Hi,
thanks for your answer. I know about lazy loading fields, but my
question is whether fields are always loaded as a whole or if it is
possible in some way to stream a field's contents.
Regards,
Günter
--
Dipl.-Inform. Günter Ladwig
Institute AIFB, University of Karlsruhe, D-76128 Karlsruhe
Phone +49 (0)721 608 7946 Building 11.40, Room 250
gla@aifb.uni-karlsruhe.de www.aifb.uni-karlsruhe.de
On 14.07.2009, at 19:21, Grant Ingersoll wrote:
> Have a look at the FieldSelector and the Lazy load capability. See http://www.lucidimagination.com/search/?q=FieldSelector
> for some pointers.
>
> -Grant
> On Jul 14, 2009, at 11:12 AM, Günter Ladwig wrote:
>
>> Hi,
>>
>> I have a situation, where stored, un-indexed fields can contain
>> potentially large amounts of data. Is it possibly to read the
>> contents of a field incrementally? That is, do not load the
>> complete contents from disk, but read X bytes at a time. Does the
>> Reader returned by Field.readerValue() work that way? Or is this
>> only possible with tokenized fields?
>>
>> Thanks!
>>
>> Regards,
>> Günter
>> --
>> Dipl.-Inform. Günter Ladwig
>> Institute AIFB, University of Karlsruhe, D-76128 Karlsruhe
>> Phone +49 (0)721 608 7946 Building 11.40, Room 250
>> gla@aifb.uni-karlsruhe.de www.aifb.uni-karlsruhe.de
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
> using Solr/Lucene:
> http://www.lucidimagination.com/search
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Stream field values
Posted by Grant Ingersoll <gs...@apache.org>.
Have a look at the FieldSelector and the Lazy load capability. See http://www.lucidimagination.com/search/?q=FieldSelector
for some pointers.
-Grant
On Jul 14, 2009, at 11:12 AM, Günter Ladwig wrote:
> Hi,
>
> I have a situation, where stored, un-indexed fields can contain
> potentially large amounts of data. Is it possibly to read the
> contents of a field incrementally? That is, do not load the complete
> contents from disk, but read X bytes at a time. Does the Reader
> returned by Field.readerValue() work that way? Or is this only
> possible with tokenized fields?
>
> Thanks!
>
> Regards,
> Günter
> --
> Dipl.-Inform. Günter Ladwig
> Institute AIFB, University of Karlsruhe, D-76128 Karlsruhe
> Phone +49 (0)721 608 7946 Building 11.40, Room 250
> gla@aifb.uni-karlsruhe.de www.aifb.uni-karlsruhe.de
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org