[GitHub] [lucene] jpountz edited a comment on pull request #184: LUCENE-9996: Reduce RAM usage of DWPT for a single document.

[GitBox <gi...@apache.org>]

jpountz edited a comment on pull request #184:
URL: https://github.com/apache/lucene/pull/184#issuecomment-861287012


   On this simple test, memory usage for a single doc in the DWPT goes from 9.6MB to 1.4MB.
   
   ```java
   import org.apache.lucene.document.Document;
   import org.apache.lucene.document.Field.Store;
   import org.apache.lucene.document.IntPoint;
   import org.apache.lucene.document.SortedNumericDocValuesField;
   import org.apache.lucene.document.SortedSetDocValuesField;
   import org.apache.lucene.document.StoredField;
   import org.apache.lucene.document.StringField;
   import org.apache.lucene.index.IndexWriter;
   import org.apache.lucene.index.IndexWriterConfig;
   import org.apache.lucene.store.ByteBuffersDirectory;
   import org.apache.lucene.util.BytesRef;
   
   public class IWBuffer {
   
     public static void main(String[] args) throws Exception {
       try (ByteBuffersDirectory dir = new ByteBuffersDirectory();
           IndexWriter w = new IndexWriter(dir, new IndexWriterConfig().setRAMBufferSizeMB(1000))) {
         Document doc = new Document();
         for (int i = 0; i < 100; ++i) {
           String keywordFieldName = "keyword" + i;
           doc.add(new StringField(keywordFieldName, "Lucene", Store.YES));
           doc.add(new SortedSetDocValuesField(keywordFieldName, new BytesRef("Lucene")));
   
           String numericFieldName = "numeric" + i;
           doc.add(new IntPoint(numericFieldName, 4));
           doc.add(new SortedNumericDocValuesField(numericFieldName, 4));
           doc.add(new StoredField(numericFieldName, 4));
         }
         w.addDocument(doc);
         System.out.println(w.ramBytesUsed() / 1024. / 1024.);
       }
     }
   
   }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org