You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "slow-J (via GitHub)" <gi...@apache.org> on 2023/11/23 14:20:18 UTC
Re: [I] Explore a single scoring implementation in DrillSidewaysScorer [LUCENE-10037] [lucene]
slow-J commented on issue #11076:
URL: https://github.com/apache/lucene/issues/11076#issuecomment-1824516300
Hi @gsmiller, I found a drill sideways tasks file in luceneutil ([ds.tasks](https://github.com/mikemccand/luceneutil/blob/5fae60800edd84c70492ac8765824ca2e4a6a991/tasks/ds.tasks)) I tried locally creating a wikimediumds (`WIKI_MEDIUM_DS = Data('wikimediumds', constants.WIKI_MEDIUM_DOCS_LINE_FILE, constants.WIKI_MEDIUM_DOCS_COUNT, constants.WIKI_DS)`) and ran `python3 src/python/localrun.py -source wikimediumds`
I commented out calls to `doUnionScoring` and `doDrillDownAdvanceScoring` and left only `doQueryFirstScoring` https://github.com/slow-J/lucene/commit/7423ddda30a2ca5e3fd90ee9743b3c40ad6b94e0 , it passes all unit tests.
```
TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value
HighTermEasyDD2 2718.56 (6.0%) 2656.82 (4.8%) -2.3% ( -12% - 9%) 0.189
MedTermEasyOrDD1 2454.91 (5.6%) 2417.95 (4.4%) -1.5% ( -10% - 8%) 0.343
MedTermMixedDD2 3184.16 (5.5%) 3137.70 (7.0%) -1.5% ( -13% - 11%) 0.465
LowTermEasyDD2 2602.71 (6.0%) 2570.49 (5.6%) -1.2% ( -12% - 11%) 0.503
LowTermEasyDD1 2583.23 (4.7%) 2553.44 (5.2%) -1.2% ( -10% - 9%) 0.465
HighTermMixedDD2 2673.50 (4.3%) 2643.56 (6.8%) -1.1% ( -11% - 10%) 0.536
LowTermHardDD1 2668.88 (5.9%) 2639.66 (4.0%) -1.1% ( -10% - 9%) 0.496
MedTermHardDD1 2320.13 (5.1%) 2298.27 (5.7%) -0.9% ( -11% - 10%) 0.580
HighTermHardOrDD2 2477.61 (4.8%) 2454.33 (3.6%) -0.9% ( -8% - 7%) 0.481
PKLookup 110.75 (12.0%) 109.98 (11.3%) -0.7% ( -21% - 25%) 0.849
HighTermEasyDD1 2777.91 (6.0%) 2762.76 (6.7%) -0.5% ( -12% - 12%) 0.787
MedTermHardOrDD1 2998.49 (5.2%) 2983.15 (5.8%) -0.5% ( -10% - 11%) 0.769
HighTermEasyOrDD1 2513.54 (4.9%) 2503.33 (4.4%) -0.4% ( -9% - 9%) 0.782
LowTermHardOrDD2 2349.94 (6.2%) 2346.39 (4.9%) -0.2% ( -10% - 11%) 0.932
LowTermHardDD2 2566.37 (4.5%) 2570.74 (5.5%) 0.2% ( -9% - 10%) 0.914
MedTermEasyDD2 3231.73 (6.8%) 3239.67 (6.4%) 0.2% ( -12% - 14%) 0.906
MedTermEasyDD1 3323.66 (7.4%) 3344.85 (5.4%) 0.6% ( -11% - 14%) 0.755
LowTermEasyOrDD1 2385.39 (5.0%) 2401.56 (5.6%) 0.7% ( -9% - 11%) 0.686
LowTermEasyOrDD2 2430.31 (4.7%) 2454.95 (5.6%) 1.0% ( -8% - 11%) 0.534
HighTermEasyOrDD2 2394.51 (5.0%) 2424.64 (6.2%) 1.3% ( -9% - 13%) 0.478
MedTermEasyOrDD2 2856.22 (7.0%) 2892.76 (5.1%) 1.3% ( -10% - 14%) 0.508
LowTermMixedDD2 2672.52 (5.9%) 2716.97 (5.9%) 1.7% ( -9% - 14%) 0.372
MedTermHardDD2 2319.03 (6.4%) 2361.41 (4.6%) 1.8% ( -8% - 13%) 0.299
HighTermHardOrDD1 2550.89 (5.4%) 2598.09 (2.7%) 1.9% ( -5% - 10%) 0.172
LowTermHardOrDD1 2429.07 (7.3%) 2484.87 (4.6%) 2.3% ( -8% - 15%) 0.234
HighTermHardDD2 2627.08 (5.8%) 2697.49 (5.7%) 2.7% ( -8% - 14%) 0.139
MedTermHardOrDD2 2846.21 (6.0%) 2956.36 (6.2%) 3.9% ( -7% - 17%) 0.045
HighTermHardDD1 2684.90 (4.4%) 2800.76 (3.3%) 4.3% ( -3% - 12%) 0.000
```
I think from results we can see that there is no performance loss from removing doUnionScoring` and `doDrillDownAdvanceScoring`.
I am wondering is the setup correct enough to trust the results? Does ([ds.tasks](https://github.com/mikemccand/luceneutil/blob/5fae60800edd84c70492ac8765824ca2e4a6a991/tasks/ds.tasks)) contain enough tasks?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org