You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/02/02 03:57:51 UTC
[PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
w41ter opened a new pull request, #30740:
URL: https://github.com/apache/doris/pull/30740
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30740:
URL: https://github.com/apache/doris/pull/30740#issuecomment-1923507965
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 99e5abc6822a864abc29d20aa6ef24d0772ff787 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 14.1 seconds inserted 10000000 Rows, about 709K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30740:
URL: https://github.com/apache/doris/pull/30740#issuecomment-1923457246
TeamCity be ut coverage result:
Function Coverage: 36.16% (8627/23856)
Line Coverage: 28.23% (70543/249928)
Region Coverage: 27.25% (36415/133643)
Branch Coverage: 24.03% (18648/77600)
Coverage Report: http://coverage.selectdb-in.cc/coverage/99e5abc6822a864abc29d20aa6ef24d0772ff787_99e5abc6822a864abc29d20aa6ef24d0772ff787/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30740:
URL: https://github.com/apache/doris/pull/30740#issuecomment-1922767395
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30740:
URL: https://github.com/apache/doris/pull/30740#issuecomment-1923375305
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30740:
URL: https://github.com/apache/doris/pull/30740#issuecomment-1923586479
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30740:
URL: https://github.com/apache/doris/pull/30740#issuecomment-1923472752
<details>
<summary>TPC-H: <b>Total hot run time: 37072 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 99e5abc6822a864abc29d20aa6ef24d0772ff787, data reload: false
------ Round 1 ----------------------------------
q1 17650 4657 4373 4373
q2 2056 144 138 138
q3 10591 920 919 919
q4 4676 789 704 704
q5 7682 2805 2875 2805
q6 180 119 120 119
q7 1171 743 717 717
q8 9290 2018 2033 2018
q9 7264 6353 6368 6353
q10 8103 2461 2446 2446
q11 410 226 213 213
q12 769 279 287 279
q13 18009 3297 3292 3292
q14 274 241 250 241
q15 527 491 491 491
q16 476 392 403 392
q17 932 589 499 499
q18 6761 5995 5938 5938
q19 1548 1346 1426 1346
q20 583 365 356 356
q21 7197 3137 3115 3115
q22 820 318 322 318
Total cold run time: 106969 ms
Total hot run time: 37072 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4674 4564 4473 4473
q2 335 239 228 228
q3 3013 2857 2860 2857
q4 1883 1745 1669 1669
q5 5231 5298 5340 5298
q6 191 114 116 114
q7 2219 1752 1792 1752
q8 3118 3221 3349 3221
q9 8368 8445 8410 8410
q10 5805 3562 3608 3562
q11 545 466 473 466
q12 741 569 591 569
q13 14687 3287 3355 3287
q14 291 258 292 258
q15 534 492 505 492
q16 573 508 529 508
q17 1865 1699 1694 1694
q18 9328 9061 7818 7818
q19 10172 1554 1487 1487
q20 2158 1944 1948 1944
q21 5660 4560 4696 4560
q22 570 492 466 466
Total cold run time: 81961 ms
Total hot run time: 55133 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30740:
URL: https://github.com/apache/doris/pull/30740#issuecomment-1923492355
<details>
<summary>TPC-DS: <b>Total hot run time: 174438 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 99e5abc6822a864abc29d20aa6ef24d0772ff787, data reload: false
query1 935 345 333 333
query2 6542 1941 2072 1941
query3 6702 212 201 201
query4 31634 22072 22033 22033
query5 4240 354 348 348
query6 253 177 163 163
query7 4603 279 279 279
query8 241 167 173 167
query9 9153 2328 2336 2328
query10 416 211 224 211
query11 18605 15340 15381 15340
query12 136 75 74 74
query13 1643 407 408 407
query14 9517 6747 7114 6747
query15 311 177 187 177
query16 8138 254 244 244
query17 2043 542 489 489
query18 2119 260 267 260
query19 360 141 139 139
query20 80 83 78 78
query21 191 122 125 122
query22 4773 4866 4751 4751
query23 30859 30007 30049 30007
query24 10523 2779 2702 2702
query25 575 342 338 338
query26 1193 147 136 136
query27 2496 301 295 295
query28 7212 1868 1842 1842
query29 858 598 599 598
query30 290 135 139 135
query31 934 701 711 701
query32 91 59 49 49
query33 727 236 217 217
query34 1101 466 469 466
query35 872 745 766 745
query36 1093 949 846 846
query37 113 58 58 58
query38 3295 3062 3110 3062
query39 1323 1257 1251 1251
query40 182 94 90 90
query41 37 36 37 36
query42 98 93 90 90
query43 527 512 489 489
query44 1054 689 705 689
query45 193 181 173 173
query46 1044 658 655 655
query47 1561 1506 1456 1456
query48 420 353 354 353
query49 1076 287 283 283
query50 757 370 382 370
query51 5277 5191 5174 5174
query52 98 87 91 87
query53 333 266 264 264
query54 279 219 222 219
query55 78 72 77 72
query56 208 201 197 197
query57 984 901 916 901
query58 194 175 177 175
query59 2587 2479 2410 2410
query60 226 212 210 210
query61 83 89 84 84
query62 642 328 368 328
query63 293 266 275 266
query64 5208 3699 3397 3397
query65 3260 3253 3228 3228
query66 821 311 296 296
query67 14876 14498 14311 14311
query68 5100 512 521 512
query69 473 316 324 316
query70 1192 1264 1241 1241
query71 320 239 274 239
query72 6071 2867 2648 2648
query73 703 315 318 315
query74 6756 6274 6292 6274
query75 3117 2301 2303 2301
query76 3268 986 948 948
query77 488 232 219 219
query78 9123 8767 8524 8524
query79 2902 485 494 485
query80 2157 351 334 334
query81 551 199 196 196
query82 863 81 76 76
query83 257 122 120 120
query84 288 84 81 81
query85 2338 339 328 328
query86 489 306 307 306
query87 3410 3260 3196 3196
query88 4079 2329 2296 2296
query89 442 362 368 362
query90 2150 162 157 157
query91 151 121 123 121
query92 52 41 40 40
query93 3745 499 451 451
query94 1408 175 173 173
query95 492 349 350 349
query96 588 274 276 274
query97 4248 4086 4151 4086
query98 221 201 194 194
query99 1180 693 706 693
Total cold run time: 287208 ms
Total hot run time: 174438 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30740:
URL: https://github.com/apache/doris/pull/30740#issuecomment-1923501286
<details>
<summary>ClickBench: <b>Total hot run time: 30.97 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 99e5abc6822a864abc29d20aa6ef24d0772ff787, data reload: false
query1 0.03 0.04 0.03
query2 0.05 0.02 0.02
query3 0.23 0.06 0.06
query4 1.67 0.09 0.09
query5 0.53 0.51 0.52
query6 1.18 0.65 0.63
query7 0.02 0.01 0.01
query8 0.04 0.03 0.03
query9 0.54 0.51 0.49
query10 0.57 0.57 0.56
query11 0.12 0.08 0.08
query12 0.12 0.09 0.09
query13 0.62 0.61 0.60
query14 0.80 0.80 0.79
query15 0.79 0.77 0.78
query16 0.38 0.37 0.41
query17 1.04 1.01 1.04
query18 0.20 0.28 0.25
query19 1.86 1.79 1.78
query20 0.01 0.01 0.01
query21 15.40 0.57 0.55
query22 2.53 2.17 1.99
query23 17.15 0.72 0.79
query24 3.10 0.78 0.87
query25 0.28 0.20 0.15
query26 0.60 0.14 0.14
query27 0.06 0.05 0.05
query28 12.23 0.86 0.81
query29 12.60 3.18 3.22
query30 0.62 0.54 0.53
query31 2.79 0.35 0.35
query32 3.36 0.48 0.48
query33 3.22 3.22 3.23
query34 15.82 4.30 4.27
query35 4.27 4.24 4.32
query36 1.09 1.07 1.06
query37 0.07 0.05 0.05
query38 0.03 0.03 0.03
query39 0.02 0.02 0.02
query40 0.16 0.13 0.13
query41 0.07 0.02 0.01
query42 0.03 0.01 0.02
query43 0.02 0.02 0.02
Total cold run time: 106.32 s
Total hot run time: 30.97 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30740:
URL: https://github.com/apache/doris/pull/30740#issuecomment-1922777409
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #30740:
URL: https://github.com/apache/doris/pull/30740#issuecomment-1923408179
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Init calc delete bitmap executor for cloud storage engine [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #30740:
URL: https://github.com/apache/doris/pull/30740
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org