StoredFields directly from a file

[John Wang <jo...@gmail.com>]

Hi folks:

A question on indexing StoredFields (using the compressioncodec):

I have a flat file storing in sequential order contents I intend to add to
stored fields. I keep a list of lengths in memory:

(byte,byte),(byte,byte,byte)...  <--- file
[2,3,...] <-- memory

where each element correspond to the storedField for a doc.
e.g.
doc1: 2 bytes
doc2: 3 bytes

Currently, I would need to read them into a BytesRef allocating either
bunch of small byte[] or load this entire file into a rather large byte[].

Do you think it makes sense to change the StoredField api to read from a
more generic version of BytesRef, that it can point either to file or
memory?

Thanks

-John