Clarification on Multiple calls to Off heap Memory & on combining storedFields

[aravinth thangasami <ar...@gmail.com>]

Hi all,

Our search System built on top of Lucene 4.10
we are having multiple indexes on a single machine with each index have
millions of documents each having about 500 to 1000 fields

we are using Lucene40StoredFieldsFormat to avoid compression of
storedFields it seems to give us better performance

We are using StoredFields to update the index.for every update call, we
will read the old document from the index, update the changed fields and
add the updated document to the index and delete the old

Inside Lucene40StoredFieldsreader's, we are calling readField method for
every field inside the document during update.
 So we thought of combining all the storedFields like Lucene Stored inside
disk and storing it as single stored field so that we can avoid the cost of
multiple readField

Inside readField, we are reading bytes from Non-heap DirectByteBuffers. So
I think this will reduce the number of calls to the Off-heap memory.
Correct me if I'm wrong.

Please clarify following questions

Does access off-heap memory is costly?
Is there any overhead in using readField multiple times?




Thanks
Aravinth