You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "dnhatn (via GitHub)" <gi...@apache.org> on 2024/07/19 05:49:34 UTC
[PR] Avoid wrap readers without soft-deletes (#13588) [lucene]
dnhatn opened a new pull request, #13590:
URL: https://github.com/apache/lucene/pull/13590
I analyzed a heap dump of Elasticsearch where FixedBitSet uses more than 1GB of memory. Most of these FixedBitSets are used by soft-deletes reader wrappers, even though these segments have no deletes at all. I believe these segments previously had soft-deletes, but these deletes were pruned by merges. The reason we wrap soft-deletes is that the soft-deletes field exists. Since these segments had soft-deletes previously, we carried the field-infos into the new segment. Ideally, we should have ways to check whether the returned docValues iterator is empty or not so that we can avoid allocating FixedBitSet completely, or we should prune fields without values after merges.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Avoid wrap readers without soft-deletes (#13588) [lucene]
Posted by "dnhatn (via GitHub)" <gi...@apache.org>.
dnhatn merged PR #13590:
URL: https://github.com/apache/lucene/pull/13590
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org