You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "xiaokang (via GitHub)" <gi...@apache.org> on 2024/02/07 07:29:57 UTC
[PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
xiaokang opened a new pull request, #30950:
URL: https://github.com/apache/doris/pull/30950
## Proposed changes
It's time to replace BITMAP INDEX with INVERTED INDEX, since INVERTED INDEX is production ready in 2.0 version.
For compatibility,
1. already created BITMAP INDEX can still be used BUT users can not create new BITMAP INDEX any more.
2. If the user use the `USING BITMAP INDEX` to create index, a INVERTED INDEX is created actually, to avoid error and get a similar query acceleration.
For the further version, `BITMAP INDEX` will be completely removed.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang commented on code in PR #30950:
URL: https://github.com/apache/doris/pull/30950#discussion_r1481023201
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -2527,6 +2527,10 @@ public IndexDefinition visitIndexDef(IndexDefContext ctx) {
Map<String, String> properties = visitPropertyItemList(ctx.properties);
String indexType = ctx.indexType != null ? ctx.indexType.getText().toUpperCase() : null;
String comment = ctx.comment != null ? ctx.comment.getText() : "";
+ // change BITMAP index to INVERTED index
+ if (indexType.equals("BITMAP")) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1933470647
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit ea1803e5f6f666d62585c1af22cc5bd7c7ee6114 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.5 seconds inserted 10000000 Rows, about 740K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1933601736
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #30950:
URL: https://github.com/apache/doris/pull/30950
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1931482872
<details>
<summary>TPC-DS: <b>Total hot run time: 174010 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7fa8984fbb22769043697ff03e2e4492d5e40ba4, data reload: false
query1 933 342 337 337
query2 6532 1906 1904 1904
query3 6697 209 204 204
query4 31916 22146 22000 22000
query5 4225 360 403 360
query6 256 163 160 160
query7 4606 291 281 281
query8 268 173 176 173
query9 9106 2263 2238 2238
query10 415 213 230 213
query11 18787 15387 15323 15323
query12 129 76 75 75
query13 1617 422 408 408
query14 9200 7175 6684 6684
query15 237 184 184 184
query16 8165 261 248 248
query17 2007 537 499 499
query18 2101 274 263 263
query19 358 139 135 135
query20 79 77 81 77
query21 195 123 129 123
query22 4953 4632 4594 4594
query23 31046 30104 30013 30013
query24 10970 2795 2729 2729
query25 586 342 338 338
query26 1366 152 155 152
query27 2864 309 306 306
query28 7432 1833 1830 1830
query29 869 625 606 606
query30 290 129 144 129
query31 931 709 706 706
query32 87 57 51 51
query33 726 219 225 219
query34 1065 458 457 457
query35 878 757 741 741
query36 1082 932 938 932
query37 115 58 56 56
query38 3203 3083 3074 3074
query39 1309 1256 1233 1233
query40 188 91 94 91
query41 36 36 34 34
query42 96 86 90 86
query43 502 505 527 505
query44 1102 678 686 678
query45 191 182 175 175
query46 1056 642 642 642
query47 1628 1422 1509 1422
query48 415 351 347 347
query49 1094 295 289 289
query50 774 372 383 372
query51 5244 5169 5168 5168
query52 99 92 86 86
query53 338 272 279 272
query54 274 215 215 215
query55 83 77 75 75
query56 216 200 210 200
query57 935 894 887 887
query58 199 177 179 177
query59 2520 2328 2379 2328
query60 232 213 219 213
query61 82 81 86 81
query62 668 371 354 354
query63 294 273 268 268
query64 5235 3689 3401 3401
query65 3236 3224 3242 3224
query66 954 305 305 305
query67 14206 14286 14208 14208
query68 4405 537 561 537
query69 453 328 343 328
query70 1267 1251 1251 1251
query71 337 255 251 251
query72 5946 2840 2688 2688
query73 711 334 323 323
query74 6664 6155 6167 6155
query75 3021 2344 2364 2344
query76 2559 885 876 876
query77 445 234 225 225
query78 9223 8796 8463 8463
query79 2619 506 489 489
query80 2213 346 332 332
query81 543 197 202 197
query82 908 83 79 79
query83 255 125 120 120
query84 282 90 74 74
query85 2319 328 327 327
query86 498 301 295 295
query87 3417 3277 3204 3204
query88 4191 2378 2384 2378
query89 420 364 367 364
query90 2164 160 161 160
query91 168 120 140 120
query92 58 43 44 43
query93 4140 498 487 487
query94 1401 172 175 172
query95 479 356 345 345
query96 610 272 275 272
query97 4241 4080 4161 4080
query98 238 202 189 189
query99 1086 684 700 684
Total cold run time: 286779 ms
Total hot run time: 174010 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1931444258
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1931495299
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 7fa8984fbb22769043697ff03e2e4492d5e40ba4 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.5 seconds inserted 10000000 Rows, about 740K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1933428130
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1933451564
<details>
<summary>TPC-H: <b>Total hot run time: 41828 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ea1803e5f6f666d62585c1af22cc5bd7c7ee6114, data reload: false
------ Round 1 ----------------------------------
q1 17604 5010 4958 4958
q2 2054 152 137 137
q3 10778 1079 1046 1046
q4 4988 980 985 980
q5 7734 3208 3295 3208
q6 199 140 136 136
q7 1281 780 765 765
q8 9271 2106 2089 2089
q9 7728 6719 6743 6719
q10 8337 2651 2647 2647
q11 410 206 202 202
q12 731 333 354 333
q13 18011 3710 3684 3684
q14 300 256 261 256
q15 636 510 505 505
q16 484 429 416 416
q17 942 870 859 859
q18 7365 6765 6746 6746
q19 1563 1503 1515 1503
q20 627 334 360 334
q21 7065 4000 3964 3964
q22 876 341 356 341
Total cold run time: 108984 ms
Total hot run time: 41828 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4962 4951 4912 4912
q2 303 190 196 190
q3 3629 3632 3646 3632
q4 2580 2571 2573 2571
q5 5791 5735 5758 5735
q6 214 129 128 128
q7 2257 1695 1702 1695
q8 3046 3111 3159 3111
q9 8755 8761 8746 8746
q10 6913 4259 4241 4241
q11 553 421 418 418
q12 800 545 561 545
q13 4225 3419 3415 3415
q14 280 245 236 236
q15 633 508 503 503
q16 492 448 464 448
q17 1718 1650 1644 1644
q18 8382 7691 7749 7691
q19 1652 1653 1651 1651
q20 2138 1847 1860 1847
q21 6806 6356 6341 6341
q22 590 526 505 505
Total cold run time: 66719 ms
Total hot run time: 60205 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #30950:
URL: https://github.com/apache/doris/pull/30950#discussion_r1481016534
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -2527,6 +2527,10 @@ public IndexDefinition visitIndexDef(IndexDefContext ctx) {
Map<String, String> properties = visitPropertyItemList(ctx.properties);
String indexType = ctx.indexType != null ? ctx.indexType.getText().toUpperCase() : null;
String comment = ctx.comment != null ? ctx.comment.getText() : "";
+ // change BITMAP index to INVERTED index
+ if (indexType.equals("BITMAP")) {
Review Comment:
equalIgnoreCase?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1931462696
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1933634001
<details>
<summary>TPC-H: <b>Total hot run time: 41978 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f94bc9f352245d22235b3545a0870cdcba199743, data reload: false
------ Round 1 ----------------------------------
q1 17696 5056 4943 4943
q2 2052 143 134 134
q3 10657 1041 1068 1041
q4 4706 1006 1002 1002
q5 7696 3360 3314 3314
q6 202 141 130 130
q7 1284 817 778 778
q8 9347 2107 2119 2107
q9 7662 6775 6712 6712
q10 8345 2652 2661 2652
q11 422 212 225 212
q12 734 326 333 326
q13 18014 3682 3903 3682
q14 289 268 269 268
q15 632 522 546 522
q16 532 407 425 407
q17 978 876 835 835
q18 7671 6695 6727 6695
q19 1561 1525 1546 1525
q20 634 342 363 342
q21 6658 4008 4024 4008
q22 877 352 343 343
Total cold run time: 108649 ms
Total hot run time: 41978 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4944 4899 4877 4877
q2 297 184 194 184
q3 3615 3607 3586 3586
q4 2538 2512 2530 2512
q5 5824 5837 5825 5825
q6 216 123 123 123
q7 2268 1659 1698 1659
q8 3040 3134 3132 3132
q9 8746 8770 8737 8737
q10 6788 4255 4267 4255
q11 541 370 390 370
q12 766 544 552 544
q13 4306 3415 3431 3415
q14 257 245 239 239
q15 596 549 544 544
q16 483 440 437 437
q17 1715 1629 1619 1619
q18 8411 7755 7785 7755
q19 1676 1672 1683 1672
q20 2111 1815 1822 1815
q21 6604 6181 6153 6153
q22 566 513 495 495
Total cold run time: 66308 ms
Total hot run time: 59948 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1933664242
<details>
<summary>ClickBench: <b>Total hot run time: 31 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f94bc9f352245d22235b3545a0870cdcba199743, data reload: false
query1 0.03 0.03 0.03
query2 0.06 0.03 0.03
query3 0.23 0.07 0.07
query4 1.65 0.08 0.09
query5 0.50 0.49 0.47
query6 1.39 0.61 0.60
query7 0.02 0.02 0.01
query8 0.04 0.02 0.02
query9 0.52 0.46 0.43
query10 0.50 0.49 0.47
query11 0.12 0.10 0.10
query12 0.12 0.10 0.10
query13 0.59 0.60 0.58
query14 0.76 0.78 0.80
query15 0.82 0.79 0.81
query16 0.32 0.34 0.31
query17 0.91 0.91 0.91
query18 0.18 0.15 0.19
query19 1.79 1.70 1.66
query20 0.02 0.01 0.01
query21 15.44 0.66 0.58
query22 3.21 4.28 2.52
query23 17.49 1.16 1.02
query24 2.07 0.56 0.21
query25 0.58 0.07 0.06
query26 0.17 0.14 0.15
query27 0.06 0.04 0.04
query28 12.14 0.84 0.84
query29 12.49 3.42 3.32
query30 0.57 0.46 0.47
query31 2.78 0.36 0.37
query32 3.35 0.48 0.48
query33 3.12 3.13 3.18
query34 15.36 4.52 4.55
query35 4.52 4.51 4.51
query36 1.07 0.94 0.96
query37 0.07 0.05 0.06
query38 0.04 0.03 0.03
query39 0.02 0.01 0.02
query40 0.17 0.14 0.14
query41 0.07 0.02 0.01
query42 0.02 0.01 0.02
query43 0.03 0.02 0.02
Total cold run time: 105.41 s
Total hot run time: 31 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1933669020
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit f94bc9f352245d22235b3545a0870cdcba199743 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.4 seconds inserted 10000000 Rows, about 746K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1933650923
<details>
<summary>TPC-DS: <b>Total hot run time: 180563 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f94bc9f352245d22235b3545a0870cdcba199743, data reload: false
query1 927 363 348 348
query2 6523 1908 1757 1757
query3 6703 210 204 204
query4 24563 22474 22399 22399
query5 4236 462 380 380
query6 257 193 180 180
query7 4601 293 304 293
query8 243 202 212 202
query9 8470 2817 2811 2811
query10 416 229 231 229
query11 16027 15550 15468 15468
query12 145 84 87 84
query13 1715 421 410 410
query14 9080 7495 7507 7495
query15 210 177 189 177
query16 7444 267 251 251
query17 1398 555 533 533
query18 1963 296 267 267
query19 188 144 146 144
query20 85 81 88 81
query21 187 114 128 114
query22 5042 4777 4842 4777
query23 32620 31788 31604 31604
query24 12620 3428 3449 3428
query25 620 368 362 362
query26 1707 160 152 152
query27 3055 313 314 313
query28 6614 1846 1840 1840
query29 1119 623 610 610
query30 278 137 149 137
query31 939 768 784 768
query32 104 57 55 55
query33 717 241 255 241
query34 1049 497 488 488
query35 927 833 820 820
query36 964 936 898 898
query37 139 63 59 59
query38 3409 3340 3327 3327
query39 1406 1297 1318 1297
query40 277 103 111 103
query41 38 36 37 36
query42 102 99 96 96
query43 480 453 456 453
query44 1080 689 701 689
query45 192 204 184 184
query46 1046 794 764 764
query47 1645 1615 1617 1615
query48 416 343 347 343
query49 1164 303 304 303
query50 790 388 365 365
query51 5291 5131 5221 5131
query52 105 93 89 89
query53 392 301 302 301
query54 299 219 220 219
query55 83 76 84 76
query56 228 203 195 195
query57 1056 988 1002 988
query58 211 201 197 197
query59 2430 2245 2196 2196
query60 237 229 222 222
query61 93 84 86 84
query62 595 398 384 384
query63 323 290 290 290
query64 6395 3086 3131 3086
query65 3298 3287 3230 3230
query66 1345 321 322 321
query67 14630 14484 14149 14149
query68 5060 540 526 526
query69 529 356 353 353
query70 1205 1273 1221 1221
query71 381 269 253 253
query72 6346 2806 2598 2598
query73 692 308 318 308
query74 7080 6686 6730 6686
query75 3206 2611 2543 2543
query76 3226 1146 1198 1146
query77 354 237 237 237
query78 9462 8797 8790 8790
query79 960 489 497 489
query80 511 351 342 342
query81 438 206 208 206
query82 163 84 84 84
query83 139 121 121 121
query84 224 77 79 77
query85 1044 358 336 336
query86 293 301 302 301
query87 3649 3391 3453 3391
query88 2641 2281 2282 2281
query89 447 349 354 349
query90 1991 161 162 161
query91 158 133 140 133
query92 55 48 50 48
query93 973 479 462 462
query94 1141 181 181 181
query95 8804 366 367 366
query96 586 262 261 261
query97 4407 4301 4278 4278
query98 221 214 195 195
query99 1096 728 742 728
Total cold run time: 280433 ms
Total hot run time: 180563 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1931438463
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1931468628
<details>
<summary>TPC-H: <b>Total hot run time: 37220 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7fa8984fbb22769043697ff03e2e4492d5e40ba4, data reload: false
------ Round 1 ----------------------------------
q1 17928 4641 4569 4569
q2 2586 147 138 138
q3 10912 935 960 935
q4 6153 843 738 738
q5 8048 2803 2842 2803
q6 183 120 118 118
q7 1144 723 736 723
q8 9303 2015 2025 2015
q9 7313 6397 6394 6394
q10 8092 2431 2439 2431
q11 425 206 203 203
q12 805 280 288 280
q13 18037 3317 3305 3305
q14 312 275 242 242
q15 530 505 484 484
q16 474 411 409 409
q17 930 483 525 483
q18 6853 6000 5954 5954
q19 1576 1354 1348 1348
q20 584 352 329 329
q21 6563 3143 3020 3020
q22 806 299 300 299
Total cold run time: 109557 ms
Total hot run time: 37220 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4499 4327 4478 4327
q2 320 225 236 225
q3 2995 2873 2821 2821
q4 1903 1676 1670 1670
q5 5228 5231 5244 5231
q6 190 117 115 115
q7 2182 1777 1752 1752
q8 3140 3199 3238 3199
q9 8312 8285 8257 8257
q10 5815 3591 3616 3591
q11 535 460 465 460
q12 735 598 609 598
q13 11903 3067 3074 3067
q14 281 259 240 240
q15 535 492 488 488
q16 509 471 474 471
q17 1865 1693 1746 1693
q18 8076 7766 7524 7524
q19 6682 1516 1570 1516
q20 2128 1914 1894 1894
q21 4783 4524 4581 4524
q22 552 487 442 442
Total cold run time: 73168 ms
Total hot run time: 54105 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1933462434
<details>
<summary>TPC-DS: <b>Total hot run time: 180647 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ea1803e5f6f666d62585c1af22cc5bd7c7ee6114, data reload: false
query1 939 346 356 346
query2 6506 1665 1800 1665
query3 6717 206 201 201
query4 24448 22398 22495 22398
query5 4310 379 453 379
query6 257 168 163 163
query7 4612 307 290 290
query8 243 196 220 196
query9 8424 2891 2889 2889
query10 420 228 234 228
query11 16093 15643 15559 15559
query12 140 82 83 82
query13 1679 423 417 417
query14 9153 7742 8048 7742
query15 222 191 189 189
query16 7521 257 246 246
query17 1400 553 532 532
query18 1975 281 268 268
query19 191 148 146 146
query20 88 87 85 85
query21 186 120 114 114
query22 5130 4860 4902 4860
query23 32531 31593 31552 31552
query24 12468 3417 3432 3417
query25 644 374 362 362
query26 1882 155 156 155
query27 3050 326 312 312
query28 6741 1830 1844 1830
query29 1125 613 612 612
query30 281 137 145 137
query31 941 743 774 743
query32 96 59 56 56
query33 716 240 239 239
query34 1058 483 486 483
query35 945 820 807 807
query36 988 895 866 866
query37 164 61 65 61
query38 3393 3346 3328 3328
query39 1371 1329 1307 1307
query40 282 107 106 106
query41 37 34 34 34
query42 104 102 98 98
query43 477 464 442 442
query44 1076 690 705 690
query45 198 179 176 176
query46 1029 766 744 744
query47 1675 1608 1581 1581
query48 431 345 342 342
query49 1207 303 301 301
query50 788 367 379 367
query51 5319 5214 5182 5182
query52 108 91 92 91
query53 393 305 297 297
query54 295 221 227 221
query55 84 83 80 80
query56 210 195 199 195
query57 1038 988 977 977
query58 216 194 191 191
query59 2182 2179 2137 2137
query60 257 220 213 213
query61 81 87 82 82
query62 597 389 384 384
query63 328 280 282 280
query64 6299 3073 3143 3073
query65 3287 3254 3255 3254
query66 1321 329 326 326
query67 14648 14262 14568 14262
query68 5148 546 561 546
query69 522 354 355 354
query70 1258 1257 1261 1257
query71 447 251 250 250
query72 6426 2770 2625 2625
query73 704 307 306 306
query74 7229 6606 6685 6606
query75 3218 2557 2545 2545
query76 3257 1140 1210 1140
query77 344 233 232 232
query78 9341 8801 8698 8698
query79 973 499 506 499
query80 508 353 334 334
query81 439 205 208 205
query82 158 83 89 83
query83 145 125 118 118
query84 224 78 78 78
query85 1013 368 329 329
query86 300 316 292 292
query87 3597 3419 3392 3392
query88 2734 2278 2270 2270
query89 439 357 353 353
query90 1896 165 163 163
query91 154 128 124 124
query92 57 49 51 49
query93 1010 475 463 463
query94 1096 183 176 176
query95 8661 368 367 367
query96 570 263 266 263
query97 4477 4240 4337 4240
query98 221 200 196 196
query99 1072 706 711 706
Total cold run time: 280653 ms
Total hot run time: 180647 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1933467466
<details>
<summary>ClickBench: <b>Total hot run time: 30.95 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ea1803e5f6f666d62585c1af22cc5bd7c7ee6114, data reload: false
query1 0.03 0.03 0.02
query2 0.07 0.03 0.03
query3 0.22 0.07 0.08
query4 1.64 0.08 0.08
query5 0.49 0.49 0.49
query6 1.40 0.60 0.60
query7 0.02 0.01 0.02
query8 0.05 0.03 0.03
query9 0.54 0.46 0.47
query10 0.50 0.50 0.50
query11 0.13 0.10 0.10
query12 0.12 0.10 0.11
query13 0.58 0.58 0.57
query14 0.75 0.77 0.80
query15 0.82 0.80 0.79
query16 0.32 0.32 0.34
query17 0.93 0.90 0.88
query18 0.19 0.17 0.18
query19 1.81 1.68 1.69
query20 0.02 0.01 0.01
query21 15.40 0.68 0.59
query22 3.08 3.12 2.29
query23 17.45 1.05 0.97
query24 2.05 0.58 0.50
query25 0.62 0.07 0.06
query26 0.16 0.14 0.14
query27 0.05 0.05 0.06
query28 11.79 0.85 0.79
query29 12.60 3.28 3.38
query30 0.53 0.49 0.48
query31 2.77 0.37 0.36
query32 3.34 0.47 0.48
query33 3.16 3.12 3.16
query34 15.36 4.49 4.50
query35 4.52 4.52 4.49
query36 1.08 0.93 0.96
query37 0.07 0.05 0.06
query38 0.04 0.02 0.03
query39 0.02 0.02 0.01
query40 0.18 0.15 0.14
query41 0.07 0.02 0.02
query42 0.02 0.02 0.01
query43 0.02 0.02 0.01
Total cold run time: 105.01 s
Total hot run time: 30.95 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1934246416
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](index) Replace BITMAP INDEX with INVERTED INDEX [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30950:
URL: https://github.com/apache/doris/pull/30950#issuecomment-1931491140
<details>
<summary>ClickBench: <b>Total hot run time: 30.75 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7fa8984fbb22769043697ff03e2e4492d5e40ba4, data reload: false
query1 0.04 0.03 0.04
query2 0.06 0.02 0.02
query3 0.24 0.06 0.07
query4 1.67 0.10 0.10
query5 0.52 0.52 0.52
query6 1.18 0.66 0.68
query7 0.02 0.01 0.01
query8 0.04 0.02 0.03
query9 0.55 0.51 0.51
query10 0.56 0.54 0.56
query11 0.12 0.08 0.08
query12 0.11 0.09 0.10
query13 0.60 0.61 0.60
query14 0.78 0.80 0.80
query15 0.81 0.78 0.77
query16 0.40 0.37 0.38
query17 1.04 1.05 1.03
query18 0.22 0.25 0.27
query19 1.82 1.74 1.80
query20 0.01 0.01 0.01
query21 15.40 0.56 0.56
query22 2.42 2.45 1.70
query23 17.34 0.78 0.75
query24 2.40 0.86 1.44
query25 0.38 0.10 0.06
query26 0.60 0.13 0.15
query27 0.05 0.05 0.06
query28 11.26 0.85 0.87
query29 12.57 3.15 3.31
query30 0.65 0.56 0.55
query31 2.81 0.35 0.37
query32 3.34 0.48 0.48
query33 3.23 3.25 3.19
query34 15.99 4.32 4.30
query35 4.27 4.26 4.25
query36 1.12 1.08 1.05
query37 0.06 0.05 0.05
query38 0.03 0.03 0.03
query39 0.02 0.02 0.02
query40 0.17 0.13 0.12
query41 0.07 0.02 0.01
query42 0.02 0.01 0.02
query43 0.02 0.02 0.02
Total cold run time: 105.01 s
Total hot run time: 30.75 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org