You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by s1monw <gi...@git.apache.org> on 2018/12/10 16:06:21 UTC
[GitHub] lucene-solr pull request #522: LUCENE-8599: Use sparse bitset to store docs ...
GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/522
LUCENE-8599: Use sparse bitset to store docs in SingleValueDocValuesFieldUpdates
Using a sparse bitset in SingleValueDocValuesFieldUdpates allows storing
which documents have an update much more efficient and prevents the need
to sort the docs array altogether that showed to be a significant bottleneck
in LUCENE-8598. Using the spares bitset yields another 10x performance improvement
in applying updates versus the changes proposed in LUCENE-8598.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/s1monw/lucene-solr use_sparse_bitset
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/lucene-solr/pull/522.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #522
----
commit 71dce85ccbd074b2afb580bd9e066e986daa989a
Author: Simon Willnauer <si...@...>
Date: 2018-12-10T14:31:43Z
LUCENE-8599: Use sparse bitset to store docs in SingleValueDocValuesFieldUpdates
Using a sparse bitset in SingleValueDocValuesFieldUdpates allows storing
which documents have an update much more efficient and prevents the need
to sort the docs array altogether that showed to be a significant bottleneck
in LUCENE-8598. Using the spares bitset yields another 10x performance improvement
in applying updates versus the changes proposed in LUCENE-8598.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[GitHub] lucene-solr issue #522: LUCENE-8599: Use sparse bitset to store docs in Sing...
Posted by s1monw <gi...@git.apache.org>.
Github user s1monw commented on the issue:
https://github.com/apache/lucene-solr/pull/522
@jpountz I will get this in as is and then explore what we can do as a followup with DocIdSetBuilder. We do need additional stats to make efficient use of it IMO and this already yields a significant improvement.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[GitHub] lucene-solr issue #522: LUCENE-8599: Use sparse bitset to store docs in Sing...
Posted by jpountz <gi...@git.apache.org>.
Github user jpountz commented on the issue:
https://github.com/apache/lucene-solr/pull/522
For the record, DocIdSetBuilder needs to sort as well if the number of collected docs is low, but it does it with a pretty fast radix sort.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[GitHub] lucene-solr pull request #522: LUCENE-8599: Use sparse bitset to store docs ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/lucene-solr/pull/522
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[GitHub] lucene-solr issue #522: LUCENE-8599: Use sparse bitset to store docs in Sing...
Posted by jpountz <gi...@git.apache.org>.
Github user jpountz commented on the issue:
https://github.com/apache/lucene-solr/pull/522
Sure!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org