You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "luwei16 (via GitHub)" <gi...@apache.org> on 2024/07/22 13:41:13 UTC
[PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
luwei16 opened a new pull request, #38224:
URL: https://github.com/apache/doris/pull/38224
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38224:
URL: https://github.com/apache/doris/pull/38224#issuecomment-2243057049
<details>
<summary>TPC-H: <b>Total hot run time: 39964 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d28230d9992e85f5a35bbe4fb41f64965faf421d, data reload: false
------ Round 1 ----------------------------------
q1 17964 4620 4437 4437
q2 2560 196 204 196
q3 11496 1188 1165 1165
q4 10776 799 798 798
q5 7611 2740 2684 2684
q6 226 138 139 138
q7 971 612 604 604
q8 9277 2089 2084 2084
q9 8820 6513 6580 6513
q10 8746 3790 3755 3755
q11 481 238 239 238
q12 394 223 224 223
q13 18742 2996 2990 2990
q14 294 238 234 234
q15 539 486 482 482
q16 499 373 383 373
q17 954 638 673 638
q18 7890 7545 7329 7329
q19 6426 1409 1342 1342
q20 702 316 336 316
q21 4872 3186 3142 3142
q22 344 287 283 283
Total cold run time: 120584 ms
Total hot run time: 39964 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4405 4297 4296 4296
q2 381 276 274 274
q3 2982 2717 2717 2717
q4 1917 1612 1581 1581
q5 5281 5292 5303 5292
q6 218 131 130 130
q7 2098 1678 1728 1678
q8 3218 3355 3309 3309
q9 8471 8418 8435 8418
q10 3882 3747 3699 3699
q11 593 528 503 503
q12 765 616 609 609
q13 17465 2977 3016 2977
q14 306 262 271 262
q15 523 484 467 467
q16 494 419 419 419
q17 1801 1486 1478 1478
q18 7656 7445 7428 7428
q19 1660 1544 1567 1544
q20 1970 1758 1815 1758
q21 4922 4768 4741 4741
q22 579 488 497 488
Total cold run time: 71587 ms
Total hot run time: 54068 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38224:
URL: https://github.com/apache/doris/pull/38224#issuecomment-2243082253
<details>
<summary>TPC-DS: <b>Total hot run time: 171772 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d28230d9992e85f5a35bbe4fb41f64965faf421d, data reload: false
query1 909 370 360 360
query2 6473 1969 1857 1857
query3 6750 204 210 204
query4 28329 17500 17205 17205
query5 4204 485 484 484
query6 294 168 154 154
query7 4590 292 282 282
query8 244 204 197 197
query9 8439 2430 2397 2397
query10 454 289 267 267
query11 11192 9927 9980 9927
query12 135 86 82 82
query13 1640 368 371 368
query14 10271 7019 8084 7019
query15 215 173 168 168
query16 7515 488 422 422
query17 1593 567 544 544
query18 1358 276 284 276
query19 209 165 158 158
query20 94 85 82 82
query21 212 126 128 126
query22 4480 4185 3943 3943
query23 33799 33127 33118 33118
query24 11918 2884 2855 2855
query25 686 386 409 386
query26 1773 149 156 149
query27 2765 271 269 269
query28 7260 1996 1973 1973
query29 1070 627 617 617
query30 288 151 151 151
query31 970 741 766 741
query32 102 55 56 55
query33 786 412 335 335
query34 933 465 483 465
query35 858 731 764 731
query36 1122 910 929 910
query37 280 78 76 76
query38 2880 2735 2738 2735
query39 871 797 814 797
query40 284 116 116 116
query41 50 46 43 43
query42 114 103 107 103
query43 502 473 461 461
query44 1208 735 728 728
query45 191 162 158 158
query46 1098 713 728 713
query47 1859 1760 1792 1760
query48 361 286 289 286
query49 1178 415 417 415
query50 789 385 386 385
query51 6717 6674 6626 6626
query52 108 88 97 88
query53 358 292 293 292
query54 1106 448 441 441
query55 78 74 91 74
query56 280 269 274 269
query57 1136 1065 1084 1065
query58 259 253 246 246
query59 2906 2568 2674 2568
query60 312 273 283 273
query61 94 92 120 92
query62 836 652 649 649
query63 319 291 288 288
query64 10497 2225 1689 1689
query65 3146 3164 3109 3109
query66 1266 333 327 327
query67 15470 14882 15024 14882
query68 5148 547 557 547
query69 688 430 358 358
query70 1228 1126 1173 1126
query71 448 279 283 279
query72 8444 5797 5470 5470
query73 760 324 317 317
query74 6104 5639 5598 5598
query75 3517 2638 2738 2638
query76 3458 915 934 915
query77 703 311 319 311
query78 9817 9007 9114 9007
query79 3924 523 536 523
query80 1899 523 484 484
query81 590 226 218 218
query82 1174 139 142 139
query83 327 177 171 171
query84 276 85 96 85
query85 1558 333 321 321
query86 464 322 302 302
query87 3277 3084 3056 3056
query88 4929 2367 2372 2367
query89 513 378 383 378
query90 2008 196 194 194
query91 135 100 102 100
query92 70 51 50 50
query93 4965 504 502 502
query94 1372 282 298 282
query95 408 328 324 324
query96 603 269 265 265
query97 3292 2971 3039 2971
query98 213 206 200 200
query99 1567 1258 1243 1243
Total cold run time: 294362 ms
Total hot run time: 171772 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on PR #38224:
URL: https://github.com/apache/doris/pull/38224#issuecomment-2243044454
we'd better add a case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on PR #38224:
URL: https://github.com/apache/doris/pull/38224#issuecomment-2243009044
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38224:
URL: https://github.com/apache/doris/pull/38224#issuecomment-2243008107
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38224:
URL: https://github.com/apache/doris/pull/38224#issuecomment-2243094468
<details>
<summary>ClickBench: <b>Total hot run time: 30.67 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d28230d9992e85f5a35bbe4fb41f64965faf421d, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.03
query3 0.23 0.06 0.06
query4 1.65 0.09 0.08
query5 0.49 0.47 0.50
query6 1.12 0.73 0.72
query7 0.02 0.01 0.02
query8 0.06 0.05 0.04
query9 0.54 0.49 0.48
query10 0.54 0.54 0.54
query11 0.16 0.12 0.11
query12 0.15 0.12 0.12
query13 0.60 0.58 0.57
query14 0.74 0.79 0.78
query15 0.86 0.81 0.81
query16 0.35 0.36 0.36
query17 1.02 0.96 0.98
query18 0.22 0.21 0.23
query19 1.90 1.81 1.74
query20 0.01 0.01 0.02
query21 15.55 0.76 0.66
query22 4.50 7.24 2.09
query23 18.28 1.36 1.21
query24 2.13 0.24 0.23
query25 0.14 0.08 0.09
query26 0.30 0.21 0.22
query27 0.45 0.23 0.23
query28 13.19 1.01 0.99
query29 12.55 3.26 3.25
query30 0.25 0.06 0.06
query31 2.86 0.39 0.38
query32 3.31 0.48 0.47
query33 2.91 2.92 2.88
query34 17.02 4.37 4.34
query35 4.39 4.41 4.45
query36 0.65 0.46 0.49
query37 0.20 0.16 0.16
query38 0.16 0.14 0.16
query39 0.04 0.03 0.04
query40 0.15 0.13 0.13
query41 0.10 0.05 0.05
query42 0.05 0.04 0.04
query43 0.05 0.04 0.04
Total cold run time: 110.01 s
Total hot run time: 30.67 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #38224:
URL: https://github.com/apache/doris/pull/38224
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38224:
URL: https://github.com/apache/doris/pull/38224#issuecomment-2242993643
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38224:
URL: https://github.com/apache/doris/pull/38224#issuecomment-2243008258
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38224:
URL: https://github.com/apache/doris/pull/38224#issuecomment-2243008814
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on PR #38224:
URL: https://github.com/apache/doris/pull/38224#issuecomment-2244133184
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on PR #38224:
URL: https://github.com/apache/doris/pull/38224#issuecomment-2244133321
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org