[GitHub] [lucene] jpountz commented on issue #11592: Relax the maximum dirtiness for stored fields and term vectors? [LUCENE-10556]

[GitBox <gi...@apache.org>]

jpountz commented on issue #11592:
URL: https://github.com/apache/lucene/issues/11592#issuecomment-1259677572

   Updated numbers now that TMP no longer runs quadratic merges, and BEST_COMPRESSION:
   
   |Dirtiness|Indexing time (msec)|
   |-|-|
   |0%|27219|
   |1%|26897|
   |5%|27196|
   |20%|27043|
   |33%|26188|
   |50%|24729|
   |75%|20993|
   |100%|9971|
   
   So the tolerable dirtiness doesn't affect indexing time too much unless unlimited (100%). My understanding is that because we merge 10 segments at once, an optimized merge typically increases dirtiness by 10x, so you need to bump the tolerable dirtiness very significantly to start seeing benefits. This benchmark suggests we wouldn't get very significant benefits from raising the maximum tolerable dirtiness.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org