You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "jpountz (via GitHub)" <gi...@apache.org> on 2025/02/01 17:16:56 UTC
[PR] Bump floor segment size to 16MB. [lucene]
jpountz opened a new pull request, #14189:
URL: https://github.com/apache/lucene/pull/14189
This bumps the floor segment size from 2MB (`TieredMergePolicy`) / 1.6MB (`LogByteSizeMergePolicy`) to 16MB in Lucene 11.
My motivation is that such small segment sizes don't make index structures actually helpful vs. linear scans, so we should avoid them. Furthermore, there has been progress on merging rules for segments below the floor size, in particular merge policies no longer perform quadratic merging (#900) so this change will not make indexing/merging absurdly slow if an application flushes tiny segments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Bump floor segment size to 16MB. [lucene]
Posted by "jainankitk (via GitHub)" <gi...@apache.org>.
jainankitk commented on PR #14189:
URL: https://github.com/apache/lucene/pull/14189#issuecomment-2686186026
Any reason this PR is not merged and got marked as stale?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Bump floor segment size to 16MB. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #14189:
URL: https://github.com/apache/lucene/pull/14189#issuecomment-2686190876
None other than me being a bit anxious about side-effects, e.g. this floor segment size also affects the behavior of merge-on-full-flush. But it shouldn't be a big deal. I'll merge shortly!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Bump floor segment size to 16MB. [lucene]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #14189:
URL: https://github.com/apache/lucene/pull/14189#issuecomment-2664271800
This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Bump floor segment size to 16MB. [lucene]
Posted by "jainankitk (via GitHub)" <gi...@apache.org>.
jainankitk commented on PR #14189:
URL: https://github.com/apache/lucene/pull/14189#issuecomment-2686184062
> For reference, this is roughly a 10x increase of the floor segment size, so given that `TieredMergePolicy` defaults to 10 segments per tier, indexes should have about 10 fewer segments after this change.
That's a good point @jpountz. Reading it initially, I thought reduction will be much more, but this does make sense. Gets rid of the 2MB segments in the bottom most tier!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Bump floor segment size to 16MB. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz merged PR #14189:
URL: https://github.com/apache/lucene/pull/14189
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Bump floor segment size to 16MB. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #14189:
URL: https://github.com/apache/lucene/pull/14189#issuecomment-2631514489
Thanks for the feedback, I was hesitating. Let's pull this in 10.2 then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Bump floor segment size to 16MB. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #14189:
URL: https://github.com/apache/lucene/pull/14189#issuecomment-2631518128
For reference, this is roughly a 10x increase of the floor segment size, so given that `TieredMergePolicy` defaults to 10 segments per tier, indexes should have about 10 fewer segments after this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org