You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/07/08 11:17:50 UTC
[GitHub] [pinot] gortiz commented on pull request #8979: optimize `order by sorted ASC, unsorted` case
gortiz commented on PR #8979:
URL: https://github.com/apache/pinot/pull/8979#issuecomment-1178869657
I've added some changes to also optimize the `order by sorted_col DESC (, more columns)*` case. The results are even better in this case because the priority queue is quite less effective in the desc case, as it has each of the N insertions has to move the whole queue.
The benchmarks now are:
sortedAsc:
```
SELECT SORTED_COL
FROM MyTable
ORDER BY SORTED_COL ASC
LIMIT 1052
option(partial-order-by= + _partialOrderBy + )
```
sortedAscPartially
```
SELECT SORTED_COL
FROM MyTable
ORDER BY SORTED_COL ASC, LOW_CARDINALITY_STRING_COL
LIMIT 1052
option(partial-order-by= + _partialOrderBy + )
```
sortedDesc
```
SELECT SORTED_COL
FROM MyTable
ORDER BY SORTED_COL DESC
LIMIT 1052
option(partial-order-by= + _partialOrderBy + )
```
sortedDescPartially
```
SELECT SORTED_COL
FROM MyTable
ORDER BY SORTED_COL DESC, LOW_CARDINALITY_STRING_COL
LIMIT 1052
option(partial-order-by= + _partialOrderBy + )
```
```
Benchmark (_numRows) (_partialOrderBy) (_primaryRepetitions) (_scenario) Mode Cnt Score Error Units
BenchmarkOrderByQueries.sortedAsc 1500000 true 1 EXP(0.5) avgt 5 0.982 ± 0.041 ms/op
BenchmarkOrderByQueries.sortedAsc 1500000 true 1000 EXP(0.5) avgt 5 0.330 ± 0.032 ms/op
BenchmarkOrderByQueries.sortedAsc 1500000 false 1 EXP(0.5) avgt 5 0.993 ± 0.040 ms/op
BenchmarkOrderByQueries.sortedAsc 1500000 false 1000 EXP(0.5) avgt 5 0.329 ± 0.011 ms/op
BenchmarkOrderByQueries.sortedAscPartially 1500000 true 1 EXP(0.5) avgt 5 1.108 ± 0.047 ms/op
BenchmarkOrderByQueries.sortedAscPartially 1500000 true 1000 EXP(0.5) avgt 5 2.280 ± 0.156 ms/op
BenchmarkOrderByQueries.sortedAscPartially 1500000 false 1 EXP(0.5) avgt 5 82.985 ± 2.531 ms/op
BenchmarkOrderByQueries.sortedAscPartially 1500000 false 1000 EXP(0.5) avgt 5 86.798 ± 14.747 ms/op
BenchmarkOrderByQueries.sortedDesc 1500000 true 1 EXP(0.5) avgt 5 1.646 ± 0.213 ms/op
BenchmarkOrderByQueries.sortedDesc 1500000 true 1000 EXP(0.5) avgt 5 1.740 ± 0.162 ms/op
BenchmarkOrderByQueries.sortedDesc 1500000 false 1 EXP(0.5) avgt 5 314.980 ± 14.875 ms/op
BenchmarkOrderByQueries.sortedDesc 1500000 false 1000 EXP(0.5) avgt 5 253.157 ± 10.557 ms/op
BenchmarkOrderByQueries.sortedDescPartially 1500000 true 1 EXP(0.5) avgt 5 2.025 ± 0.113 ms/op
BenchmarkOrderByQueries.sortedDescPartially 1500000 true 1000 EXP(0.5) avgt 5 5.113 ± 0.717 ms/op
BenchmarkOrderByQueries.sortedDescPartially 1500000 false 1 EXP(0.5) avgt 5 410.110 ± 28.135 ms/op
BenchmarkOrderByQueries.sortedDescPartially 1500000 false 1000 EXP(0.5) avgt 5 583.025 ± 23.282 ms/op
```
I still need to add tests to verify the correctness, which is in what I'm going to be focused early next week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org