You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2022/01/24 07:47:34 UTC
[GitHub] [lucene] gf2121 edited a comment on pull request #592: LUCENE-10366: Override #readVInt and #readVLong for ByteBufferDataInput to avoid the abstraction confusion of #readByte.
gf2121 edited a comment on pull request #592:
URL: https://github.com/apache/lucene/pull/592#issuecomment-1019803986
I find that in a higher level, we are using `MultiLevelSkipListReader#SkipBuffer` for top skipdata and `ByteBufferIndexInput` for low skipdata, and this is the point making JVM confused. I tried to remove the `MultiLevelSkipListReader#SkipBuffer`, getting similar speed up (baseline is `main` and my_modified_version is `main + remove skipbuffer`):
```
TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value
IntNRQ 32.23 (24.8%) 30.20 (28.2%) -6.3% ( -47% - 62%) 0.452
HighTermMonthSort 62.08 (24.3%) 60.47 (15.8%) -2.6% ( -34% - 49%) 0.690
HighSpanNear 2.67 (6.1%) 2.62 (4.6%) -1.7% ( -11% - 9%) 0.320
PKLookup 200.97 (3.8%) 198.22 (5.9%) -1.4% ( -10% - 8%) 0.382
Fuzzy1 68.17 (2.1%) 67.41 (2.1%) -1.1% ( -5% - 3%) 0.089
OrHighMedDayTaxoFacets 0.76 (5.6%) 0.75 (8.4%) -1.0% ( -14% - 13%) 0.658
TermDTSort 47.59 (15.1%) 47.14 (17.2%) -0.9% ( -28% - 37%) 0.854
Respell 45.57 (2.1%) 45.17 (2.5%) -0.9% ( -5% - 3%) 0.222
Fuzzy2 47.98 (2.0%) 47.57 (1.9%) -0.8% ( -4% - 3%) 0.179
MedTermDayTaxoFacets 15.33 (4.2%) 15.24 (5.1%) -0.6% ( -9% - 9%) 0.673
MedSpanNear 4.55 (4.0%) 4.55 (2.9%) 0.1% ( -6% - 7%) 0.956
LowIntervalsOrdered 4.51 (7.4%) 4.52 (7.8%) 0.1% ( -13% - 16%) 0.952
BrowseMonthSSDVFacets 4.62 (5.1%) 4.63 (5.7%) 0.2% ( -10% - 11%) 0.918
HighIntervalsOrdered 8.12 (7.9%) 8.15 (8.2%) 0.4% ( -14% - 17%) 0.887
MedSloppyPhrase 3.66 (5.7%) 3.67 (4.9%) 0.5% ( -9% - 11%) 0.777
BrowseRandomLabelSSDVFacets 3.15 (5.4%) 3.17 (6.5%) 0.6% ( -10% - 13%) 0.735
BrowseDayOfYearSSDVFacets 4.27 (6.4%) 4.31 (9.4%) 0.9% ( -13% - 17%) 0.727
MedIntervalsOrdered 7.56 (6.0%) 7.62 (6.1%) 0.9% ( -10% - 13%) 0.642
AndHighHighDayTaxoFacets 3.12 (3.5%) 3.16 (5.3%) 1.2% ( -7% - 10%) 0.391
OrHighHigh 17.23 (4.4%) 17.44 (3.9%) 1.2% ( -6% - 9%) 0.347
LowSpanNear 11.17 (3.5%) 11.32 (2.7%) 1.4% ( -4% - 7%) 0.159
LowTerm 1569.56 (2.7%) 1593.46 (3.3%) 1.5% ( -4% - 7%) 0.108
BrowseRandomLabelTaxoFacets 4.40 (11.5%) 4.47 (12.7%) 1.6% ( -20% - 29%) 0.672
Prefix3 270.42 (18.3%) 275.42 (16.9%) 1.8% ( -28% - 45%) 0.740
HighTermDayOfYearSort 63.17 (19.4%) 64.52 (18.0%) 2.1% ( -29% - 49%) 0.719
HighTermTitleBDVSort 37.21 (17.0%) 38.01 (17.5%) 2.1% ( -27% - 44%) 0.695
BrowseDateTaxoFacets 5.04 (14.6%) 5.17 (16.2%) 2.6% ( -24% - 39%) 0.598
BrowseDayOfYearTaxoFacets 5.10 (14.9%) 5.23 (16.3%) 2.6% ( -24% - 39%) 0.596
HighSloppyPhrase 5.80 (4.4%) 5.96 (4.9%) 2.8% ( -6% - 12%) 0.059
OrHighMed 48.70 (4.7%) 50.10 (3.4%) 2.9% ( -5% - 11%) 0.028
Wildcard 46.19 (9.7%) 47.52 (5.4%) 2.9% ( -11% - 19%) 0.247
LowSloppyPhrase 9.06 (3.0%) 9.33 (3.8%) 2.9% ( -3% - 9%) 0.006
AndHighMedDayTaxoFacets 15.86 (2.3%) 16.44 (3.1%) 3.6% ( -1% - 9%) 0.000
MedPhrase 15.16 (2.3%) 15.72 (2.4%) 3.6% ( -1% - 8%) 0.000
AndHighMed 21.94 (3.9%) 22.76 (4.3%) 3.7% ( -4% - 12%) 0.004
AndHighHigh 48.32 (4.3%) 50.40 (4.4%) 4.3% ( -4% - 13%) 0.002
LowPhrase 55.55 (2.6%) 58.03 (2.8%) 4.5% ( 0% - 10%) 0.000
OrHighNotMed 1192.57 (4.5%) 1251.05 (4.4%) 4.9% ( -3% - 14%) 0.001
HighPhrase 189.54 (3.2%) 199.56 (3.3%) 5.3% ( -1% - 12%) 0.000
OrHighNotLow 962.35 (4.3%) 1013.28 (4.6%) 5.3% ( -3% - 14%) 0.000
BrowseMonthTaxoFacets 5.38 (13.8%) 5.68 (14.7%) 5.5% ( -20% - 39%) 0.219
OrNotHighHigh 871.79 (3.9%) 920.18 (3.7%) 5.5% ( -1% - 13%) 0.000
OrHighNotHigh 982.57 (4.2%) 1038.77 (4.2%) 5.7% ( -2% - 14%) 0.000
OrNotHighMed 945.32 (3.2%) 1001.55 (3.5%) 5.9% ( 0% - 13%) 0.000
HighTerm 1238.69 (4.8%) 1317.72 (4.8%) 6.4% ( -3% - 16%) 0.000
MedTerm 1911.70 (4.7%) 2044.18 (4.2%) 6.9% ( -1% - 16%) 0.000
OrHighLow 574.17 (4.5%) 623.52 (3.1%) 8.6% ( 0% - 16%) 0.000
OrNotHighLow 784.78 (5.0%) 875.15 (4.2%) 11.5% ( 2% - 21%) 0.000
AndHighLow 463.10 (5.8%) 521.43 (4.5%) 12.6% ( 2% - 24%) 0.000
```
I'm not very sure if we should move on in this approach to avoid potential confusion in a more complex environment other than benchmark, but IMO it should be reasonable to remove `MultiLevelSkipListReader#SkipBuffer` as it seems like an unnecessay complexity?
I raised https://issues.apache.org/jira/browse/LUCENE-10388 (https://github.com/apache/lucene/pull/620)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org