You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "s1monw (via GitHub)" <gi...@apache.org> on 2023/09/26 15:18:14 UTC
[GitHub] [lucene] s1monw opened a new pull request, #12595: Make IndexWriter#flushNextBuffer also apply deletes if necessary
s1monw opened a new pull request, #12595:
URL: https://github.com/apache/lucene/pull/12595
`IndexWriter#flushNextBuffer()` is a convenient way to control indexing buffer sizes across multiple index writers. This change also flushes deletes if necessary when `#flushNextBuffer()` is called even if there is no buffer to be flushed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] jpountz commented on pull request #12595: Make IndexWriter#flushNextBuffer also apply deletes if necessary
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #12595:
URL: https://github.com/apache/lucene/pull/12595#issuecomment-1736121993
Thinking out loud: it looks like your change always flushes both the largest pending writer and deletes. I wonder if we should try to make it more granular and e.g. check whichever of deletes or the largest pending DWPT is bigger, or maybe round robin between deletes and the largest pending DWPT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Make IndexWriter#flushNextBuffer also apply deletes if necessary [lucene]
Posted by "s1monw (via GitHub)" <gi...@apache.org>.
s1monw closed pull request #12595: Make IndexWriter#flushNextBuffer also apply deletes if necessary
URL: https://github.com/apache/lucene/pull/12595
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Make IndexWriter#flushNextBuffer also apply deletes if necessary [lucene]
Posted by "s1monw (via GitHub)" <gi...@apache.org>.
s1monw commented on PR #12595:
URL: https://github.com/apache/lucene/pull/12595#issuecomment-1749460562
see https://github.com/apache/lucene/issues/12572#issuecomment-1749458532
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] s1monw commented on pull request #12595: Make IndexWriter#flushNextBuffer also apply deletes if necessary
Posted by "s1monw (via GitHub)" <gi...@apache.org>.
s1monw commented on PR #12595:
URL: https://github.com/apache/lucene/pull/12595#issuecomment-1737068633
@jpountz I added a simplified version of your idea that applies deletes if they consume more RAM than the DWPT we are flushing. The applyAllDeletes() only has an impact if the `FlushControl` has the flag set. Which means it's not always applied. Now, I think if we do round robin or whatever is using more ram we need to rename the method. Which I am ok with but freeing more ram as a side effect we could do too. Maybe add an overload with a boolean? not sure it's necessary...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org