You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "tyronecai (via GitHub)" <gi...@apache.org> on 2026/02/28 00:58:20 UTC
[PR] Improve BytesRefHash.sort performance by get byte directly from the pool. [lucene]
tyronecai opened a new pull request, #15775:
URL: https://github.com/apache/lucene/pull/15775
### Description
<!--
If this is your first contribution to Lucene, please make sure you have reviewed the contribution guide.
https://github.com/apache/lucene/blob/main/CONTRIBUTING.md
-->
RadixSort involves a large number of statistical histogram calculations, there are numerous byteAt calls.
Currently, byteAt calls `get` to retrieve the BytesRef corresponding to position i from the pool,
and then uses `cmp.ByteAt` to get the value of BytesRef at position k.
Because the byteAt calls are so frequent, they cause an observable performance penalty.
Therefore, we can directly retrieve the byte values corresponding to start and i from the pool.
Use save environment as https://github.com/apache/lucene/pull/15772
## without #15772 + without pool.byteAt
```
sort 33554432 unique terms in 4543.19 ms
```
## without #15772 + with pool.byteAt
```
sort 33554432 unique terms in 3866.08 ms. (4543.19 - 3866.08) / 4543.19 = 0.149
```
## with #15772 + without pool.byteAt
```
sort 33554432 unique terms in 3385.94 ms
```
## with #15772 + with pool.byteAt
```
sort 33554432 unique terms in 2937.54 ms. (3385.94 - 2937.54) / 3385.94 = 0.132
```
However, I'm not sure if this change is appropriate from a code structure perspective,
although it does improve performance.
@dweiss @mikemccand please take a look and give some advice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Improve BytesRefHash.sort performance by retrieve byte directly from the pool. [lucene]
Posted by "tyronecai (via GitHub)" <gi...@apache.org>.
tyronecai commented on PR #15775:
URL: https://github.com/apache/lucene/pull/15775#issuecomment-3982201957
@dweiss
Could you please take a look and see if there are any issues with this change ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Improve BytesRefHash.sort performance by retrieve byte directly from the pool. [lucene]
Posted by "tyronecai (via GitHub)" <gi...@apache.org>.
tyronecai closed pull request #15775: Improve BytesRefHash.sort performance by retrieve byte directly from the pool.
URL: https://github.com/apache/lucene/pull/15775
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Improve BytesRefHash.sort performance by retrieve byte directly from the pool. [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on PR #15775:
URL: https://github.com/apache/lucene/pull/15775#issuecomment-3985743675
Checks don't pass. Also, refining this class in microbenchmarks is not the same as running it in the wild - there is significant complexity if you add higher-tier code. I'd say that this may not be worth the increased code complexity. You'd need to try macro-benchmarks (luceneutil) and see if this shows any improvement, I don't think you'll see much there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Improve BytesRefHash.sort performance by retrieve byte directly from the pool. [lucene]
Posted by "tyronecai (via GitHub)" <gi...@apache.org>.
tyronecai commented on PR #15775:
URL: https://github.com/apache/lucene/pull/15775#issuecomment-3987587612
> Checks don't pass. Also, refining this class in microbenchmarks is not the same as running it in the wild - there is significant complexity if you add higher-tier code. I'd say that this may not be worth the increased code complexity. You'd need to try macro-benchmarks (luceneutil) and see if this shows any improvement, I don't think you'll see much there.检查未通过。此外，在微基准测试中优化这个类与在真实环境中运行是不同的——如果添加了更高层次的代码，复杂性会显著增加。我认为这种增加的代码复杂性并不值得。您需要尝试宏观基准测试（luceneutil），看看是否能看到任何改进，不过我觉得效果不会太明显。
Okay, I also think this change is a bit strange.
> Checks don't pass. Also, refining this class in microbenchmarks is not the same as running it in the wild - there is significant complexity if you add higher-tier code. I'd say that this may not be worth the increased code complexity. You'd need to try macro-benchmarks (luceneutil) and see if this shows any improvement, I don't think you'll see much there.
Okay, I also think this change is a bit strange.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org