You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Uwe Schindler <us...@apache.org> on 2017/03/10 18:18:38 UTC
ByteBuffer performance issue in Java 9?
Hi,
we just noticed this issue: https://bugs.openjdk.java.net/browse/JDK-8176513
As Apache Lucene relies heavily on performance of ByteBuffers (especially MappedByteBuffer), this would be a desaster if it would get even slower than Java 8. We were so happy that there was much work going on to improve the performance of ByteBuffers so they were at some point almost as fast as byte[]. Because of that we are currently working on making Lucene work fine with Java 9, because once it is out, we would inform all users and recommend to use Java 9 because of these optimizations. The tests of Lucene are already running perfectly, no problems like data corrumption (remember Java 7 GA or 7u40). Also Apache Solr people try to fix the remaining Hadoop/Kerberos-Auth issues because of Jigsaw. We also have unmapping working using Unsafe::invokeCleaner, so we are prepared...
I am not yet sure if this bug affects us, we have to run perf tests first. Nevertheless, this is our usage pattern:
- we only read from ByteBuffers (MappedByteBuffer), never store anything (we use mmap to execute Lucene searches on the mapped index with often 10th sometimes also 100th of gigabytes of data files that were mmapped).
- we do sequential reads (position() and then getByte(), getXxX)
- we also do positional reads like this test (for the so called DocValues in Lucene, which is a column based store). This is mainly like sorting search results based on a compare function that accesses a ByteBuffer as random-access source for comparison values.
- we don't use IntBuffer, we work directly on ByteBuffer
I asked Mike McCandless to run the performance tests with a recent Java 9 version to see if it affects us. Is there any information, when this bug was introduced into Java 9? Nevertheless, please fix it for Java 9 GA, if it affects us it would be a major loss of search performance for us!
It would be good to get some information about your plans :-) Thanks!
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: ByteBuffer performance issue in Java 9?
Posted by Andrew Haley <ap...@redhat.com>.
On 10/03/17 18:34, Uwe Schindler wrote:
> Do you know in which build these problems were introduced, so we can have a comparison:
I have no idea.
Andrew.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: ByteBuffer performance issue in Java 9?
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Andrew,
> > It would be good to get some information about your plans :-) Thanks!
>
> I believe you should not worry too much. The biggest performance drop
> seems to be heap-based ByteBuffers rather than off-heap ByteBuffers,
> which is what you use. The only problem you'll see is that they don't
> get vectorized, which you can probably live with. On the other hand,
> heap ByteBuffers are very badly affected.
>
> Sorry to cause a panic; I should have explained better.
No problem, thanks for the explanation. We will of course do some performance tests, this is why I put Mike McCandless on the list of people on this mail. We will report back if we see any significant problems. It depends if vectorization affects us. We have vectorization at other places, but I don't know about our use of ByteBuffers (seems unlikely).
Do you know in which build these problems were introduced, so we can have a comparison:
- Java 8u121
- Java 9 build ??? - before the problem
- Java 9 b159
Uwe
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: ByteBuffer performance issue in Java 9?
Posted by Andrew Haley <ap...@redhat.com>.
On 10/03/17 18:18, Uwe Schindler wrote:
> It would be good to get some information about your plans :-) Thanks!
I believe you should not worry too much. The biggest performance drop
seems to be heap-based ByteBuffers rather than off-heap ByteBuffers,
which is what you use. The only problem you'll see is that they don't
get vectorized, which you can probably live with. On the other hand,
heap ByteBuffers are very badly affected.
Sorry to cause a panic; I should have explained better.
Andrew.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: ByteBuffer performance issue in Java 9?
Posted by Andrew Haley <ap...@redhat.com>.
On 14/03/17 09:25, Uwe Schindler wrote:
> * As there is a larger slowdown if Lucene uses the MMapDirectory
> implementation on Java 9 in comparison to Java 8, we should really
> look into the issues! So Andrew Haley is right, there is some
> slowdown not only affecting HeapByteBuffers! Maybe it is caused by
> missing vectorization or because the code calling the ByteBuffers is
> much more complicated than Andrew\u2019s tests, Hotspot is failing to
> properly optimize the access to DirectByteBuffer, too.
I'm happy to look at this, but you'll have to provide me with some
sort of test case. "Read the Lucene manual. Install this server."
probably isn't such a test case.
Andrew.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: ByteBuffer performance issue in Java 9?
Posted by Uwe Schindler <us...@apache.org>.
Hallo Mike,
thank you for the update! The comparison with NIOFSDirectory is helpful, as we can now conclude:
* There is a minor slowdown implied by Java 9 in comparison to Java 8, which can have to do with other changed due to Jigsaw module system, and is not related to the ByteBuffer issue. As NIOFSDirectory does not really use ByteBuffers in a performance critical way (it is just used to read buffers of 8 or 16 KiB size from FileChannel into a HeapByteBuffer wrapped around a byte[] that is used for BufferedIndexInput), Andrew Haley’s investigation of byte buffer slowdowns cannot have any impact here. The ByteBuffer API is never used to access the byte[] contents, ByteBuffer is just used to pass the Lucene byte[] between FileChannel and our code!
* NIOFSDirectory is still much slower for accessing the Lucene index than mmap (as expected, especially for sorting where positional reads are really needed), so the current Lucene default of using MMapDirectory of 64 bit platforms still outweights any other Lucene directory implementation.
* As there is a larger slowdown if Lucene uses the MMapDirectory implementation on Java 9 in comparison to Java 8, we should really look into the issues! So Andrew Haley is right, there is some slowdown not only affecting HeapByteBuffers! Maybe it is caused by missing vectorization or because the code calling the ByteBuffers is much more complicated than Andrew’s tests, Hotspot is failing to properly optimize the access to DirectByteBuffer, too.
Andrew & others, if you have a patch ready, we can try to benchmark it!
Uwe
From: Michael McCandless [mailto:lucene@mikemccandless.com]
Sent: Monday, March 13, 2017 4:44 PM
To: Uwe Schindler <us...@apache.org>
Cc: Andrew Haley <ap...@redhat.com>; Lucene/Solr dev <de...@lucene.apache.org>; jdk9-dev@openjdk.java.net; hotspot-dev@openjdk.java.net
Subject: Re: ByteBuffer performance issue in Java 9?
I reran the same test as before, this time using Lucene's NIOFSDirectory (java.nio's FileChannel for positional reads).
There is still some slowdown, though a bit less than with MMapDirectory:
Task QPS base StdDev QPS comp StdDev Pct diff
LowSloppyPhrase 9.12 (5.2%) 7.89 (3.6%) -13.5% ( -21% - -4%)
HighSloppyPhrase 6.25 (4.1%) 5.57 (3.3%) -10.8% ( -17% - -3%)
MedSloppyPhrase 4.29 (3.8%) 3.85 (3.0%) -10.1% ( -16% - -3%)
BrowseDayOfYearSSDVFacets 4.72 (8.8%) 4.32 (5.4%) -8.4% ( -20% - 6%)
BrowseMonthTaxoFacets 1.31 (2.9%) 1.20 (4.7%) -8.3% ( -15% - 0%)
BrowseDayOfYearTaxoFacets 1.17 (4.0%) 1.08 (4.1%) -7.7% ( -15% - 0%)
BrowseDateTaxoFacets 1.18 (4.0%) 1.09 (4.1%) -7.6% ( -15% - 0%)
BrowseMonthSSDVFacets 5.39 (4.3%) 4.98 (10.7%) -7.6% ( -21% - 7%)
HighTerm 29.47 (5.3%) 27.45 (4.2%) -6.9% ( -15% - 2%)
HighTermDayOfYearSort 14.24 (4.6%) 13.35 (5.5%) -6.3% ( -15% - 3%)
MedTerm 44.47 (4.0%) 42.02 (3.4%) -5.5% ( -12% - 1%)
OrHighNotLow 33.13 (5.0%) 31.39 (4.3%) -5.2% ( -13% - 4%)
OrHighLow 26.84 (4.2%) 25.49 (4.0%) -5.0% ( -12% - 3%)
HighPhrase 7.51 (5.4%) 7.17 (4.0%) -4.5% ( -13% - 5%)
Fuzzy2 51.32 (0.9%) 49.03 (1.2%) -4.5% ( -6% - -2%)
OrHighNotHigh 13.18 (3.5%) 12.64 (3.7%) -4.2% ( -10% - 3%)
IntNRQ 6.28 (6.3%) 6.03 (9.9%) -4.0% ( -18% - 12%)
OrHighNotMed 27.69 (2.9%) 26.65 (3.3%) -3.8% ( -9% - 2%)
Fuzzy1 42.32 (0.8%) 40.88 (1.1%) -3.4% ( -5% - -1%)
MedSpanNear 27.44 (2.4%) 26.57 (2.8%) -3.2% ( -8% - 2%)
OrNotHighHigh 17.58 (2.8%) 17.04 (3.3%) -3.1% ( -8% - 3%)
HighSpanNear 26.83 (2.2%) 26.03 (2.5%) -3.0% ( -7% - 1%)
Respell 49.07 (1.2%) 47.62 (0.8%) -3.0% ( -4% - 0%)
LowPhrase 31.19 (1.5%) 30.34 (1.2%) -2.7% ( -5% - 0%)
Wildcard 51.30 (4.8%) 49.93 (4.3%) -2.7% ( -11% - 6%)
LowSpanNear 31.05 (1.2%) 30.40 (1.5%) -2.1% ( -4% - 0%)
LowTerm 105.83 (1.1%) 103.78 (1.4%) -1.9% ( -4% - 0%)
OrNotHighMed 49.08 (1.8%) 48.16 (2.1%) -1.9% ( -5% - 2%)
OrHighMed 18.44 (5.8%) 18.09 (5.1%) -1.9% ( -12% - 9%)
AndHighHigh 21.22 (1.0%) 20.82 (1.3%) -1.9% ( -4% - 0%)
HighTermMonthSort 32.01 (3.9%) 31.43 (5.3%) -1.8% ( -10% - 7%)
OrHighHigh 12.89 (7.0%) 12.66 (6.0%) -1.8% ( -13% - 12%)
MedPhrase 24.09 (2.3%) 23.80 (2.2%) -1.2% ( -5% - 3%)
AndHighLow 447.74 (1.5%) 443.85 (1.7%) -0.9% ( -3% - 2%)
Prefix3 18.41 (6.8%) 18.28 (5.6%) -0.7% ( -12% - 12%)
OrNotHighLow 254.77 (1.4%) 254.59 (1.2%) -0.1% ( -2% - 2%)
AndHighMed 63.15 (1.5%) 63.47 (0.9%) 0.5% ( -1% - 3%)
PKLookup 347.80 (2.3%) 349.93 (2.1%) 0.6% ( -3% - 5%)
Mike McCandless
<http://blog.mikemccandless.com> http://blog.mikemccandless.com
On Mon, Mar 13, 2017 at 10:16 AM, Michael McCandless < <ma...@mikemccandless.com> lucene@mikemccandless.com> wrote:
Hi Uwe,
OK, I'll test with NIOFSDirectory as well ... that's a good idea.
I do remember testing earlier Java 9 builds long ago, but I can't remember what the outcome was.
Mike McCandless
<http://blog.mikemccandless.com> http://blog.mikemccandless.com
On Mon, Mar 13, 2017 at 6:35 AM, Uwe Schindler < <ma...@apache.org> uschindler@apache.org> wrote:
Hi Andrew,
yes that was my impression, too.
Just for cross-checking: Mike, is it possible to also add a perf comparison between Java 8 and Java 9 when using SimpleFSDirectory or NIOFSDirectory (which are both FileChannel based since Java 7, the name is just backwards-compatibility)? If we see a slowdown there (maybe even larger) than it is not related to ByteBuffer positional/byte-wise reads and there is a general performance issue somewhere else.
> Right, but ByteBuffers were significantly rewritten for a significant
> performance *increase*. Any slowdown shows that something has gone
> very wrong indeed.
That would be a pity, because of that we should check the above case with non-mmap based, conventional index access. As far as I remember: at the time when you announced the bytebuffer improvements we did some performance measurements and were impressed by the speedup. I think Robert Muir did something. Mike, do you know?
Maybe we should check with a Java 9 preview build from that time. I know that you can download older builds by changing the build number in the download URL.
Uwe
Re: ByteBuffer performance issue in Java 9?
Posted by Michael McCandless <lu...@mikemccandless.com>.
I reran the same test as before, this time using Lucene's NIOFSDirectory
(java.nio's FileChannel for positional reads).
There is still some slowdown, though a bit less than with MMapDirectory:
Task QPS base StdDev QPS comp
StdDev Pct diff
LowSloppyPhrase 9.12 (5.2%) 7.89
(3.6%) -13.5% ( -21% - -4%)
HighSloppyPhrase 6.25 (4.1%) 5.57
(3.3%) -10.8% ( -17% - -3%)
MedSloppyPhrase 4.29 (3.8%) 3.85
(3.0%) -10.1% ( -16% - -3%)
BrowseDayOfYearSSDVFacets 4.72 (8.8%) 4.32
(5.4%) -8.4% ( -20% - 6%)
BrowseMonthTaxoFacets 1.31 (2.9%) 1.20
(4.7%) -8.3% ( -15% - 0%)
BrowseDayOfYearTaxoFacets 1.17 (4.0%) 1.08
(4.1%) -7.7% ( -15% - 0%)
BrowseDateTaxoFacets 1.18 (4.0%) 1.09
(4.1%) -7.6% ( -15% - 0%)
BrowseMonthSSDVFacets 5.39 (4.3%) 4.98
(10.7%) -7.6% ( -21% - 7%)
HighTerm 29.47 (5.3%) 27.45
(4.2%) -6.9% ( -15% - 2%)
HighTermDayOfYearSort 14.24 (4.6%) 13.35
(5.5%) -6.3% ( -15% - 3%)
MedTerm 44.47 (4.0%) 42.02
(3.4%) -5.5% ( -12% - 1%)
OrHighNotLow 33.13 (5.0%) 31.39
(4.3%) -5.2% ( -13% - 4%)
OrHighLow 26.84 (4.2%) 25.49
(4.0%) -5.0% ( -12% - 3%)
HighPhrase 7.51 (5.4%) 7.17
(4.0%) -4.5% ( -13% - 5%)
Fuzzy2 51.32 (0.9%) 49.03
(1.2%) -4.5% ( -6% - -2%)
OrHighNotHigh 13.18 (3.5%) 12.64
(3.7%) -4.2% ( -10% - 3%)
IntNRQ 6.28 (6.3%) 6.03
(9.9%) -4.0% ( -18% - 12%)
OrHighNotMed 27.69 (2.9%) 26.65
(3.3%) -3.8% ( -9% - 2%)
Fuzzy1 42.32 (0.8%) 40.88
(1.1%) -3.4% ( -5% - -1%)
MedSpanNear 27.44 (2.4%) 26.57
(2.8%) -3.2% ( -8% - 2%)
OrNotHighHigh 17.58 (2.8%) 17.04
(3.3%) -3.1% ( -8% - 3%)
HighSpanNear 26.83 (2.2%) 26.03
(2.5%) -3.0% ( -7% - 1%)
Respell 49.07 (1.2%) 47.62
(0.8%) -3.0% ( -4% - 0%)
LowPhrase 31.19 (1.5%) 30.34
(1.2%) -2.7% ( -5% - 0%)
Wildcard 51.30 (4.8%) 49.93
(4.3%) -2.7% ( -11% - 6%)
LowSpanNear 31.05 (1.2%) 30.40
(1.5%) -2.1% ( -4% - 0%)
LowTerm 105.83 (1.1%) 103.78
(1.4%) -1.9% ( -4% - 0%)
OrNotHighMed 49.08 (1.8%) 48.16
(2.1%) -1.9% ( -5% - 2%)
OrHighMed 18.44 (5.8%) 18.09
(5.1%) -1.9% ( -12% - 9%)
AndHighHigh 21.22 (1.0%) 20.82
(1.3%) -1.9% ( -4% - 0%)
HighTermMonthSort 32.01 (3.9%) 31.43
(5.3%) -1.8% ( -10% - 7%)
OrHighHigh 12.89 (7.0%) 12.66
(6.0%) -1.8% ( -13% - 12%)
MedPhrase 24.09 (2.3%) 23.80
(2.2%) -1.2% ( -5% - 3%)
AndHighLow 447.74 (1.5%) 443.85
(1.7%) -0.9% ( -3% - 2%)
Prefix3 18.41 (6.8%) 18.28
(5.6%) -0.7% ( -12% - 12%)
OrNotHighLow 254.77 (1.4%) 254.59
(1.2%) -0.1% ( -2% - 2%)
AndHighMed 63.15 (1.5%) 63.47
(0.9%) 0.5% ( -1% - 3%)
PKLookup 347.80 (2.3%) 349.93
(2.1%) 0.6% ( -3% - 5%)
Mike McCandless
http://blog.mikemccandless.com
On Mon, Mar 13, 2017 at 10:16 AM, Michael McCandless <
lucene@mikemccandless.com> wrote:
> Hi Uwe,
>
> OK, I'll test with NIOFSDirectory as well ... that's a good idea.
>
> I do remember testing earlier Java 9 builds long ago, but I can't remember
> what the outcome was.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Mon, Mar 13, 2017 at 6:35 AM, Uwe Schindler <us...@apache.org>
> wrote:
>
>> Hi Andrew,
>>
>> yes that was my impression, too.
>>
>> Just for cross-checking: Mike, is it possible to also add a perf
>> comparison between Java 8 and Java 9 when using SimpleFSDirectory or
>> NIOFSDirectory (which are both FileChannel based since Java 7, the name is
>> just backwards-compatibility)? If we see a slowdown there (maybe even
>> larger) than it is not related to ByteBuffer positional/byte-wise reads and
>> there is a general performance issue somewhere else.
>>
>> > Right, but ByteBuffers were significantly rewritten for a significant
>> > performance *increase*. Any slowdown shows that something has gone
>> > very wrong indeed.
>>
>> That would be a pity, because of that we should check the above case with
>> non-mmap based, conventional index access. As far as I remember: at the
>> time when you announced the bytebuffer improvements we did some performance
>> measurements and were impressed by the speedup. I think Robert Muir did
>> something. Mike, do you know?
>>
>> Maybe we should check with a Java 9 preview build from that time. I know
>> that you can download older builds by changing the build number in the
>> download URL.
>>
>> Uwe
>>
>>
>
Re: ByteBuffer performance issue in Java 9?
Posted by Michael McCandless <lu...@mikemccandless.com>.
Hi Uwe,
OK, I'll test with NIOFSDirectory as well ... that's a good idea.
I do remember testing earlier Java 9 builds long ago, but I can't remember
what the outcome was.
Mike McCandless
http://blog.mikemccandless.com
On Mon, Mar 13, 2017 at 6:35 AM, Uwe Schindler <us...@apache.org>
wrote:
> Hi Andrew,
>
> yes that was my impression, too.
>
> Just for cross-checking: Mike, is it possible to also add a perf
> comparison between Java 8 and Java 9 when using SimpleFSDirectory or
> NIOFSDirectory (which are both FileChannel based since Java 7, the name is
> just backwards-compatibility)? If we see a slowdown there (maybe even
> larger) than it is not related to ByteBuffer positional/byte-wise reads and
> there is a general performance issue somewhere else.
>
> > Right, but ByteBuffers were significantly rewritten for a significant
> > performance *increase*. Any slowdown shows that something has gone
> > very wrong indeed.
>
> That would be a pity, because of that we should check the above case with
> non-mmap based, conventional index access. As far as I remember: at the
> time when you announced the bytebuffer improvements we did some performance
> measurements and were impressed by the speedup. I think Robert Muir did
> something. Mike, do you know?
>
> Maybe we should check with a Java 9 preview build from that time. I know
> that you can download older builds by changing the build number in the
> download URL.
>
> Uwe
>
>
RE: ByteBuffer performance issue in Java 9?
Posted by Uwe Schindler <us...@apache.org>.
Hi Andrew,
yes that was my impression, too.
Just for cross-checking: Mike, is it possible to also add a perf comparison between Java 8 and Java 9 when using SimpleFSDirectory or NIOFSDirectory (which are both FileChannel based since Java 7, the name is just backwards-compatibility)? If we see a slowdown there (maybe even larger) than it is not related to ByteBuffer positional/byte-wise reads and there is a general performance issue somewhere else.
> Right, but ByteBuffers were significantly rewritten for a significant
> performance *increase*. Any slowdown shows that something has gone
> very wrong indeed.
That would be a pity, because of that we should check the above case with non-mmap based, conventional index access. As far as I remember: at the time when you announced the bytebuffer improvements we did some performance measurements and were impressed by the speedup. I think Robert Muir did something. Mike, do you know?
Maybe we should check with a Java 9 preview build from that time. I know that you can download older builds by changing the build number in the download URL.
Uwe
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: ByteBuffer performance issue in Java 9?
Posted by Andrew Haley <ap...@redhat.com>.
On 13/03/17 08:37, Andrew Haley wrote:
> On 12/03/17 21:15, Michael McCandless wrote:
>> I compared Java 1.8.0_121 (base, below) and Java 9 ea 160 (comp), indexing
>> and searching all Wikipedia English content, using MMapDirectory, and
>> net/net there is some smallish slowdown, but it's not catastrophic.
>
> Right, but ByteBuffers were significantly rewritten for a significant
> performance *increase*.
Just for clarification: heap-based ByteBuffers are most significantly
affected. Mapped ByteBuffers see a much smaller, if any, effect, as
I said in my earlier post.
Andrew.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: ByteBuffer performance issue in Java 9?
Posted by Andrew Haley <ap...@redhat.com>.
On 12/03/17 21:15, Michael McCandless wrote:
> I compared Java 1.8.0_121 (base, below) and Java 9 ea 160 (comp), indexing
> and searching all Wikipedia English content, using MMapDirectory, and
> net/net there is some smallish slowdown, but it's not catastrophic.
Right, but ByteBuffers were significantly rewritten for a significant
performance *increase*. Any slowdown shows that something has gone
very wrong indeed.
Andrew.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: ByteBuffer performance issue in Java 9?
Posted by Michael McCandless <lu...@mikemccandless.com>.
Hi Uwe,
I ran the standard luceneutil Wikipedia benchmark (same as what's run
nightly at https://home.apache.org/~mikemccand/lucenebench).
I compared Java 1.8.0_121 (base, below) and Java 9 ea 160 (comp), indexing
and searching all Wikipedia English content, using MMapDirectory, and
net/net there is some smallish slowdown, but it's not catastrophic.
I'm not sure why sloppy phrase queries are particularly hit. This was
after 20 JVM iterations:
Task QPS base StdDev QPS comp
StdDev Pct diff
LowSloppyPhrase 11.02 (6.0%) 9.17
(3.1%) -16.8% ( -24% - -8%)
HighSloppyPhrase 7.40 (4.5%) 6.36
(2.9%) -14.1% ( -20% - -6%)
MedSloppyPhrase 5.13 (4.3%) 4.48
(2.4%) -12.7% ( -18% - -6%)
HighTerm 44.54 (5.6%) 39.01
(3.6%) -12.4% ( -20% - -3%)
OrHighLow 39.53 (7.1%) 34.70
(4.9%) -12.2% ( -22% - 0%)
MedTerm 86.34 (5.2%) 75.88
(3.3%) -12.1% ( -19% - -3%)
OrHighNotLow 52.00 (7.1%) 46.10
(6.0%) -11.3% ( -22% - 1%)
BrowseDayOfYearSSDVFacets 5.33 (3.3%) 4.76
(5.8%) -10.6% ( -19% - -1%)
HighTermDayOfYearSort 20.49 (5.5%) 18.36
(5.8%) -10.4% ( -20% - 1%)
BrowseMonthTaxoFacets 1.47 (2.2%) 1.35
(6.8%) -8.6% ( -17% - 0%)
OrHighNotMed 41.62 (3.5%) 38.18
(4.7%) -8.3% ( -15% - 0%)
OrHighNotHigh 16.08 (3.3%) 14.84
(4.4%) -7.7% ( -14% - 0%)
BrowseDayOfYearTaxoFacets 1.31 (2.1%) 1.21
(6.4%) -7.1% ( -15% - 1%)
BrowseDateTaxoFacets 1.31 (2.1%) 1.22
(6.5%) -7.0% ( -15% - 1%)
Prefix3 20.13 (7.2%) 18.74
(5.9%) -6.9% ( -18% - 6%)
OrNotHighHigh 23.22 (3.1%) 21.63
(3.9%) -6.9% ( -13% - 0%)
IntNRQ 7.33 (9.3%) 6.83
(11.2%) -6.8% ( -24% - 15%)
Fuzzy2 105.61 (1.7%) 98.65
(1.3%) -6.6% ( -9% - -3%)
Wildcard 56.59 (5.4%) 52.86
(4.3%) -6.6% ( -15% - 3%)
MedSpanNear 36.06 (4.0%) 33.88
(3.6%) -6.0% ( -13% - 1%)
HighSpanNear 35.40 (4.0%) 33.39
(3.8%) -5.7% ( -12% - 2%)
Fuzzy1 89.36 (1.6%) 84.28
(1.5%) -5.7% ( -8% - -2%)
OrHighMed 28.17 (9.8%) 26.66
(3.6%) -5.4% ( -17% - 8%)
OrHighHigh 17.94 (10.5%) 17.01
(3.7%) -5.2% ( -17% - 10%)
LowTerm 397.33 (2.4%) 377.12
(1.4%) -5.1% ( -8% - -1%)
OrNotHighMed 86.69 (2.1%) 82.75
(3.1%) -4.5% ( -9% - 0%)
HighPhrase 8.58 (7.2%) 8.19
(6.6%) -4.5% ( -17% - 10%)
LowSpanNear 42.81 (1.5%) 41.17
(1.6%) -3.8% ( -6% - 0%)
Respell 71.63 (1.3%) 69.17
(1.4%) -3.4% ( -6% - 0%)
BrowseMonthSSDVFacets 5.74 (11.1%) 5.55
(10.8%) -3.4% ( -22% - 20%)
AndHighHigh 33.22 (1.6%) 32.15
(1.5%) -3.2% ( -6% - 0%)
LowPhrase 43.46 (1.6%) 42.39
(1.4%) -2.5% ( -5% - 0%)
PKLookup 360.20 (2.1%) 352.10
(1.3%) -2.3% ( -5% - 1%)
MedPhrase 33.01 (3.9%) 32.56
(3.7%) -1.4% ( -8% - 6%)
AndHighLow 706.78 (2.4%) 706.29
(1.5%) -0.1% ( -3% - 3%)
HighTermMonthSort 40.02 (4.4%) 40.36
(4.9%) 0.9% ( -8% - 10%)
OrNotHighLow 585.09 (2.8%) 593.40
(1.8%) 1.4% ( -3% - 6%)
AndHighMed 111.84 (2.6%) 113.65
(1.5%) 1.6% ( -2% - 5%)
Mike McCandless
http://blog.mikemccandless.com
On Fri, Mar 10, 2017 at 1:18 PM, Uwe Schindler <us...@apache.org>
wrote:
> Hi,
>
> we just noticed this issue: https://bugs.openjdk.java.net/
> browse/JDK-8176513
>
> As Apache Lucene relies heavily on performance of ByteBuffers (especially
> MappedByteBuffer), this would be a desaster if it would get even slower
> than Java 8. We were so happy that there was much work going on to improve
> the performance of ByteBuffers so they were at some point almost as fast as
> byte[]. Because of that we are currently working on making Lucene work fine
> with Java 9, because once it is out, we would inform all users and
> recommend to use Java 9 because of these optimizations. The tests of Lucene
> are already running perfectly, no problems like data corrumption (remember
> Java 7 GA or 7u40). Also Apache Solr people try to fix the remaining
> Hadoop/Kerberos-Auth issues because of Jigsaw. We also have unmapping
> working using Unsafe::invokeCleaner, so we are prepared...
>
> I am not yet sure if this bug affects us, we have to run perf tests first.
> Nevertheless, this is our usage pattern:
> - we only read from ByteBuffers (MappedByteBuffer), never store anything
> (we use mmap to execute Lucene searches on the mapped index with often 10th
> sometimes also 100th of gigabytes of data files that were mmapped).
> - we do sequential reads (position() and then getByte(), getXxX)
> - we also do positional reads like this test (for the so called DocValues
> in Lucene, which is a column based store). This is mainly like sorting
> search results based on a compare function that accesses a ByteBuffer as
> random-access source for comparison values.
> - we don't use IntBuffer, we work directly on ByteBuffer
>
> I asked Mike McCandless to run the performance tests with a recent Java 9
> version to see if it affects us. Is there any information, when this bug
> was introduced into Java 9? Nevertheless, please fix it for Java 9 GA, if
> it affects us it would be a major loss of search performance for us!
>
> It would be good to get some information about your plans :-) Thanks!
>
> Uwe
>
> -----
> Uwe Schindler
> uschindler@apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>