You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "luwei16 (via GitHub)" <gi...@apache.org> on 2024/03/27 08:40:27 UTC
[PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
luwei16 opened a new pull request, #32896:
URL: https://github.com/apache/doris/pull/32896
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2025348690
<details>
<summary>TPC-H: <b>Total hot run time: 37481 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1b20cb752b6bf8d3bb857fe2848fd392b5cad528, data reload: false
------ Round 1 ----------------------------------
q1 17640 4155 4059 4059
q2 2106 160 155 155
q3 10577 1178 1146 1146
q4 10242 774 739 739
q5 7444 2887 2863 2863
q6 199 124 126 124
q7 1019 596 559 559
q8 9330 2007 1946 1946
q9 6853 6322 6307 6307
q10 8425 3481 3574 3481
q11 433 233 224 224
q12 390 197 192 192
q13 17786 2840 2857 2840
q14 226 204 203 203
q15 505 486 466 466
q16 517 380 377 377
q17 943 616 541 541
q18 7083 6464 6407 6407
q19 4569 1418 1411 1411
q20 547 258 251 251
q21 3571 2900 3024 2900
q22 343 290 302 290
Total cold run time: 110748 ms
Total hot run time: 37481 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4133 4096 4047 4047
q2 331 231 231 231
q3 2931 2848 2844 2844
q4 1890 1617 1584 1584
q5 5216 5263 5243 5243
q6 198 118 121 118
q7 2182 1832 1805 1805
q8 3182 3273 3288 3273
q9 8443 8418 8491 8418
q10 3781 3910 3962 3910
q11 571 477 454 454
q12 780 609 583 583
q13 17190 3064 3059 3059
q14 341 280 282 280
q15 540 477 502 477
q16 509 451 456 451
q17 1777 1529 1504 1504
q18 8035 7649 7562 7562
q19 5063 1564 1515 1515
q20 2050 1859 1730 1730
q21 5028 4768 5022 4768
q22 539 475 459 459
Total cold run time: 74710 ms
Total hot run time: 54315 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2027342345
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2031432956
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2027076143
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit cd2644da25e12345d282159d1a5a95e8d546b4c4 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 13.6 seconds inserted 10000000 Rows, about 735K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #32896:
URL: https://github.com/apache/doris/pull/32896
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2031732983
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2022280105
<details>
<summary>TPC-DS: <b>Total hot run time: 180842 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bec112d33d9a96c37be3b61fd76b3e7839d1642e, data reload: false
query1 927 389 362 362
query2 6541 2051 1920 1920
query3 6713 210 205 205
query4 31879 21512 21242 21242
query5 4338 403 381 381
query6 274 171 172 171
query7 4641 291 285 285
query8 231 174 177 174
query9 9221 2309 2287 2287
query10 567 242 254 242
query11 14732 14212 14220 14212
query12 137 93 85 85
query13 1641 422 409 409
query14 10113 7255 7453 7255
query15 247 197 196 196
query16 8047 268 263 263
query17 1946 573 561 561
query18 2039 292 279 279
query19 221 163 163 163
query20 94 91 88 88
query21 211 132 126 126
query22 5052 4938 4853 4853
query23 33454 32724 32843 32724
query24 11974 2832 2846 2832
query25 658 393 397 393
query26 1830 167 160 160
query27 3041 343 350 343
query28 7875 1879 1876 1876
query29 1020 623 597 597
query30 304 146 150 146
query31 969 713 718 713
query32 95 58 55 55
query33 763 249 244 244
query34 1068 488 483 483
query35 834 603 595 595
query36 1008 890 898 890
query37 141 69 62 62
query38 3586 3433 3393 3393
query39 1496 1429 1437 1429
query40 287 113 111 111
query41 50 47 44 44
query42 113 93 97 93
query43 480 460 446 446
query44 1258 728 742 728
query45 270 260 253 253
query46 1109 717 693 693
query47 1920 1877 1872 1872
query48 440 351 354 351
query49 1214 332 335 332
query50 747 373 372 372
query51 6668 6630 6499 6499
query52 107 87 91 87
query53 339 272 271 271
query54 296 237 233 233
query55 86 79 77 77
query56 236 209 224 209
query57 1227 1178 1158 1158
query58 219 199 210 199
query59 2776 2579 2495 2495
query60 269 234 248 234
query61 100 100 99 99
query62 643 441 433 433
query63 297 271 274 271
query64 6821 4086 3982 3982
query65 3038 3025 3009 3009
query66 1210 378 376 376
query67 15412 14880 15178 14880
query68 5413 508 516 508
query69 577 378 388 378
query70 1224 1132 1203 1132
query71 422 266 277 266
query72 6361 2863 2676 2676
query73 709 322 318 318
query74 7900 6339 6540 6339
query75 2967 2199 2230 2199
query76 3517 847 814 814
query77 451 255 250 250
query78 10941 10257 10117 10117
query79 8599 527 535 527
query80 2096 366 369 366
query81 541 219 212 212
query82 1404 87 83 83
query83 306 141 147 141
query84 286 80 76 76
query85 2194 323 316 316
query86 497 285 286 285
query87 3745 3576 3536 3536
query88 5058 2293 2263 2263
query89 542 360 362 360
query90 2016 178 174 174
query91 165 136 140 136
query92 65 47 47 47
query93 6336 501 497 497
query94 1248 186 176 176
query95 446 326 323 323
query96 619 264 266 264
query97 2658 2465 2493 2465
query98 221 210 209 209
query99 1195 936 888 888
Total cold run time: 308279 ms
Total hot run time: 180842 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2027066523
<details>
<summary>TPC-DS: <b>Total hot run time: 181134 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cd2644da25e12345d282159d1a5a95e8d546b4c4, data reload: false
query1 901 380 351 351
query2 7011 1933 1856 1856
query3 6675 219 220 219
query4 37713 21621 21322 21322
query5 4244 403 385 385
query6 232 185 178 178
query7 3987 300 299 299
query8 220 170 185 170
query9 5669 2359 2330 2330
query10 485 230 243 230
query11 14756 14330 14209 14209
query12 136 97 96 96
query13 968 375 378 375
query14 8885 6800 7015 6800
query15 201 177 174 174
query16 6748 271 271 271
query17 1635 580 562 562
query18 1382 304 288 288
query19 208 159 157 157
query20 93 93 87 87
query21 203 124 129 124
query22 5019 4880 4838 4838
query23 33427 32694 32869 32694
query24 12193 2875 2842 2842
query25 615 390 388 388
query26 1836 151 158 151
query27 3362 318 333 318
query28 7228 1898 1885 1885
query29 898 636 618 618
query30 296 153 153 153
query31 915 726 736 726
query32 64 54 56 54
query33 564 260 262 260
query34 881 478 484 478
query35 864 702 689 689
query36 1009 877 917 877
query37 216 75 76 75
query38 3504 3450 3429 3429
query39 1562 1574 1537 1537
query40 273 131 128 128
query41 51 48 46 46
query42 110 99 97 97
query43 470 453 456 453
query44 1148 737 732 732
query45 294 256 280 256
query46 1076 695 689 689
query47 1951 1867 1844 1844
query48 384 308 310 308
query49 1148 373 380 373
query50 775 396 391 391
query51 6622 6523 6584 6523
query52 112 94 95 94
query53 352 289 283 283
query54 273 246 247 246
query55 86 85 81 81
query56 246 227 238 227
query57 1247 1138 1129 1129
query58 228 212 212 212
query59 2720 2589 2583 2583
query60 268 251 254 251
query61 120 123 118 118
query62 634 460 445 445
query63 311 278 284 278
query64 5802 3983 4135 3983
query65 3109 3053 3087 3053
query66 1277 353 342 342
query67 15666 14887 14809 14809
query68 10656 542 544 542
query69 632 317 311 311
query70 1226 1200 1151 1151
query71 589 274 270 270
query72 7072 2646 2446 2446
query73 1641 331 335 331
query74 7406 6439 6405 6405
query75 3984 2360 2288 2288
query76 6596 914 1019 914
query77 682 257 257 257
query78 11729 10707 10182 10182
query79 11770 526 525 525
query80 1362 429 417 417
query81 524 217 217 217
query82 247 92 91 91
query83 223 172 170 170
query84 269 86 82 82
query85 805 261 265 261
query86 352 309 312 309
query87 3764 3581 3579 3579
query88 5597 2418 2424 2418
query89 495 381 393 381
query90 2418 179 180 179
query91 130 99 100 99
query92 60 47 48 47
query93 5876 525 504 504
query94 1559 181 179 179
query95 414 310 307 307
query96 609 267 272 267
query97 2628 2530 2486 2486
query98 228 221 215 215
query99 1068 851 835 835
Total cold run time: 316169 ms
Total hot run time: 181134 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2022259162
<details>
<summary>TPC-H: <b>Total hot run time: 38045 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bec112d33d9a96c37be3b61fd76b3e7839d1642e, data reload: false
------ Round 1 ----------------------------------
q1 17615 4774 4069 4069
q2 2112 161 151 151
q3 10591 1152 1206 1152
q4 10240 764 768 764
q5 7460 3015 2994 2994
q6 204 128 122 122
q7 1050 592 575 575
q8 9349 2040 2000 2000
q9 7171 6615 6571 6571
q10 8443 3466 3588 3466
q11 433 227 222 222
q12 381 202 194 194
q13 17789 2842 2886 2842
q14 243 204 207 204
q15 510 462 463 462
q16 491 377 369 369
q17 944 596 584 584
q18 7118 6557 6456 6456
q19 1532 1452 1503 1452
q20 551 261 244 244
q21 3601 2964 2861 2861
q22 352 297 291 291
Total cold run time: 108180 ms
Total hot run time: 38045 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4098 4071 4042 4042
q2 323 223 235 223
q3 3008 2832 2850 2832
q4 1889 1576 1544 1544
q5 5306 5323 5332 5323
q6 195 114 115 114
q7 2269 1879 1860 1860
q8 3134 3267 3276 3267
q9 8622 8717 8702 8702
q10 3837 3807 3762 3762
q11 528 440 443 440
q12 713 564 546 546
q13 16915 2834 2860 2834
q14 287 258 257 257
q15 492 459 451 451
q16 489 427 432 427
q17 1719 1505 1466 1466
q18 7344 7139 7157 7139
q19 1584 1558 1505 1505
q20 1888 1719 1697 1697
q21 4961 4528 4572 4528
q22 508 444 444 444
Total cold run time: 70109 ms
Total hot run time: 53403 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2031723854
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2031767866
TeamCity be ut coverage result:
Function Coverage: 35.65% (8880/24911)
Line Coverage: 27.37% (72900/266308)
Region Coverage: 26.56% (37706/141988)
Branch Coverage: 23.34% (19214/82314)
Coverage Report: http://coverage.selectdb-in.cc/coverage/95d9056f2a4286542a8df04187c5433b3852b2ec_95d9056f2a4286542a8df04187c5433b3852b2ec/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2027025133
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2022294982
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit bec112d33d9a96c37be3b61fd76b3e7839d1642e with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 13.8 seconds inserted 10000000 Rows, about 724K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2025259058
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on code in PR #32896:
URL: https://github.com/apache/doris/pull/32896#discussion_r1543027384
##########
be/src/olap/merger.cpp:
##########
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2022269648
TeamCity be ut coverage result:
Function Coverage: 35.25% (8741/24796)
Line Coverage: 27.03% (71548/264661)
Region Coverage: 26.28% (37131/141281)
Branch Coverage: 23.17% (18980/81912)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bec112d33d9a96c37be3b61fd76b3e7839d1642e_bec112d33d9a96c37be3b61fd76b3e7839d1642e/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2025311398
TeamCity be ut coverage result:
Function Coverage: 35.61% (8835/24808)
Line Coverage: 27.32% (72381/264957)
Region Coverage: 26.55% (37545/141412)
Branch Coverage: 23.36% (19148/81984)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1b20cb752b6bf8d3bb857fe2848fd392b5cad528_1b20cb752b6bf8d3bb857fe2848fd392b5cad528/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2027341645
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2031715582
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2022221588
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2027072079
<details>
<summary>ClickBench: <b>Total hot run time: 29.77 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cd2644da25e12345d282159d1a5a95e8d546b4c4, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.24 0.06 0.05
query4 1.66 0.07 0.08
query5 0.50 0.50 0.50
query6 1.14 0.67 0.66
query7 0.02 0.01 0.01
query8 0.04 0.04 0.04
query9 0.55 0.51 0.51
query10 0.56 0.56 0.55
query11 0.16 0.10 0.10
query12 0.14 0.11 0.11
query13 0.61 0.59 0.59
query14 0.78 0.77 0.76
query15 0.85 0.82 0.83
query16 0.38 0.35 0.36
query17 0.95 1.01 0.96
query18 0.21 0.24 0.22
query19 1.80 1.70 1.70
query20 0.02 0.01 0.01
query21 15.54 0.65 0.64
query22 2.41 2.72 1.72
query23 17.09 0.96 0.88
query24 1.14 0.25 0.22
query25 0.09 0.08 0.08
query26 0.23 0.17 0.17
query27 0.08 0.08 0.08
query28 14.03 0.96 0.94
query29 12.58 3.25 3.20
query30 0.29 0.08 0.08
query31 2.80 0.40 0.39
query32 3.26 0.47 0.46
query33 2.98 2.83 2.88
query34 16.66 4.46 4.45
query35 4.55 4.55 4.48
query36 0.61 0.48 0.48
query37 0.18 0.16 0.15
query38 0.15 0.14 0.14
query39 0.05 0.03 0.03
query40 0.17 0.15 0.15
query41 0.09 0.06 0.05
query42 0.05 0.04 0.04
query43 0.04 0.03 0.04
Total cold run time: 105.8 s
Total hot run time: 29.77 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2025285306
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2031562898
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2031819778
<details>
<summary>TPC-H: <b>Total hot run time: 38361 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 95d9056f2a4286542a8df04187c5433b3852b2ec, data reload: false
------ Round 1 ----------------------------------
q1 17609 4076 4039 4039
q2 2015 202 181 181
q3 10469 1255 1366 1255
q4 10207 847 998 847
q5 7456 2936 2885 2885
q6 215 131 135 131
q7 1085 613 598 598
q8 9390 1987 2009 1987
q9 6713 6147 6098 6098
q10 8458 3514 3488 3488
q11 416 247 237 237
q12 381 216 219 216
q13 17763 2896 2901 2896
q14 277 243 235 235
q15 524 482 492 482
q16 518 383 375 375
q17 950 912 890 890
q18 7087 6363 6374 6363
q19 1639 1525 1526 1525
q20 590 319 305 305
q21 3484 3076 3034 3034
q22 352 294 306 294
Total cold run time: 107598 ms
Total hot run time: 38361 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4051 4035 4046 4035
q2 326 215 227 215
q3 2956 2923 2939 2923
q4 1832 1839 1843 1839
q5 5209 5191 5182 5182
q6 207 124 123 123
q7 2213 1825 1777 1777
q8 3193 3273 3278 3273
q9 8404 8385 8408 8385
q10 3728 3870 3987 3870
q11 555 451 459 451
q12 775 571 597 571
q13 16823 3057 3145 3057
q14 306 266 269 266
q15 531 487 486 486
q16 503 433 439 433
q17 1757 1747 1730 1730
q18 8375 7756 7610 7610
q19 1680 1676 1701 1676
q20 2011 1821 1831 1821
q21 5117 4991 4916 4916
q22 507 419 442 419
Total cold run time: 71059 ms
Total hot run time: 55058 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2031865063
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 95d9056f2a4286542a8df04187c5433b3852b2ec with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 16.8 seconds inserted 10000000 Rows, about 595K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2022230098
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2022235088
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2022225646
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2031850151
<details>
<summary>TPC-DS: <b>Total hot run time: 181799 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 95d9056f2a4286542a8df04187c5433b3852b2ec, data reload: false
query1 876 1121 1108 1108
query2 6281 2182 1930 1930
query3 6665 214 212 212
query4 23690 21500 21603 21500
query5 4145 396 411 396
query6 278 209 191 191
query7 4604 309 297 297
query8 234 184 171 171
query9 8477 2313 2317 2313
query10 455 258 271 258
query11 15150 14490 14454 14454
query12 148 94 100 94
query13 1617 385 386 385
query14 8520 6896 6917 6896
query15 225 171 173 171
query16 6811 275 271 271
query17 991 579 560 560
query18 1846 277 276 276
query19 203 157 157 157
query20 93 91 93 91
query21 199 133 134 133
query22 4945 4814 4687 4687
query23 33036 32431 32687 32431
query24 12816 3219 3156 3156
query25 720 439 450 439
query26 1889 175 166 166
query27 3231 383 376 376
query28 6988 1945 1920 1920
query29 1375 642 618 618
query30 326 170 171 170
query31 1021 753 746 746
query32 97 65 61 61
query33 707 250 252 250
query34 1128 531 517 517
query35 871 746 730 730
query36 1024 870 879 870
query37 266 82 80 80
query38 3661 3506 3598 3506
query39 1660 1612 1609 1609
query40 253 153 144 144
query41 50 48 49 48
query42 123 109 109 109
query43 458 411 423 411
query44 1138 726 731 726
query45 289 266 279 266
query46 1112 840 831 831
query47 2055 1883 1898 1883
query48 417 329 313 313
query49 1031 362 375 362
query50 825 417 407 407
query51 6827 6724 6713 6713
query52 108 109 99 99
query53 377 310 300 300
query54 321 263 249 249
query55 90 89 81 81
query56 258 242 252 242
query57 1274 1188 1213 1188
query58 264 247 259 247
query59 2686 2587 2373 2373
query60 255 237 238 237
query61 92 88 88 88
query62 646 433 449 433
query63 303 278 283 278
query64 5757 3064 3053 3053
query65 3020 3016 2993 2993
query66 1305 316 317 316
query67 15304 14877 14817 14817
query68 9029 590 591 590
query69 547 330 316 316
query70 1234 1044 1116 1044
query71 513 267 270 267
query72 6314 2580 2423 2423
query73 804 334 331 331
query74 6672 6240 6234 6234
query75 3553 2332 2305 2305
query76 5492 1073 1183 1073
query77 567 251 243 243
query78 10756 10205 10120 10120
query79 11350 534 515 515
query80 1636 424 423 423
query81 528 242 238 238
query82 341 103 98 98
query83 206 164 170 164
query84 270 89 89 89
query85 971 294 281 281
query86 352 286 297 286
query87 3684 3481 3490 3481
query88 4247 2341 2361 2341
query89 543 372 377 372
query90 2039 176 187 176
query91 129 104 104 104
query92 58 51 53 51
query93 6985 534 525 525
query94 1198 203 196 196
query95 448 327 335 327
query96 610 274 272 272
query97 2677 2489 2490 2489
query98 229 216 213 213
query99 1195 825 809 809
Total cold run time: 299084 ms
Total hot run time: 181799 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2025418116
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 1b20cb752b6bf8d3bb857fe2848fd392b5cad528 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 13.8 seconds inserted 10000000 Rows, about 724K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on code in PR #32896:
URL: https://github.com/apache/doris/pull/32896#discussion_r1542290438
##########
be/src/olap/merger.cpp:
##########
@@ -207,16 +207,25 @@ void Merger::vertical_split_columns(const TabletSchema& tablet_schema,
column_groups->emplace_back(std::move(key_columns));
}
auto&& cluster_key_idxes = tablet_schema.cluster_key_idxes();
+
+ std::vector<uint32_t> value_columns;
for (uint32_t i = num_key_cols; i < total_cols; ++i) {
if (i == sequence_col_idx || i == delete_sign_idx ||
cluster_key_idxes.end() !=
std::find(cluster_key_idxes.begin(), cluster_key_idxes.end(), i)) {
continue;
}
- if ((i - num_key_cols) % config::vertical_compaction_num_columns_per_group == 0) {
- column_groups->emplace_back();
+
+ if (!value_columns.empty() &&
Review Comment:
For the unqiue key table,if there are only key columns in the schema, why does the code run here ? After all, besides the key column, there are only delete sign and sequence columns, which are skipped at line 213.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2025252234
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2031859551
<details>
<summary>ClickBench: <b>Total hot run time: 30.98 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 95d9056f2a4286542a8df04187c5433b3852b2ec, data reload: false
query1 0.04 0.03 0.04
query2 0.08 0.04 0.05
query3 0.23 0.05 0.05
query4 1.67 0.07 0.07
query5 0.48 0.48 0.50
query6 1.12 0.65 0.65
query7 0.02 0.01 0.01
query8 0.06 0.05 0.05
query9 0.56 0.52 0.50
query10 0.57 0.57 0.56
query11 0.15 0.12 0.11
query12 0.14 0.12 0.12
query13 0.62 0.59 0.60
query14 0.77 0.79 0.80
query15 0.87 0.84 0.83
query16 0.35 0.36 0.34
query17 0.96 0.98 0.97
query18 0.24 0.25 0.25
query19 1.79 1.71 1.70
query20 0.02 0.01 0.01
query21 15.42 0.77 0.72
query22 2.78 5.43 2.48
query23 17.75 1.19 1.18
query24 1.39 0.22 0.41
query25 0.13 0.10 0.08
query26 0.30 0.17 0.19
query27 0.08 0.09 0.09
query28 13.74 0.98 0.93
query29 12.85 3.72 3.43
query30 0.25 0.06 0.05
query31 2.86 0.37 0.38
query32 3.30 0.47 0.47
query33 2.85 2.92 2.86
query34 15.51 4.29 4.34
query35 4.35 4.35 4.33
query36 0.68 0.48 0.49
query37 0.19 0.18 0.17
query38 0.17 0.17 0.16
query39 0.06 0.05 0.05
query40 0.18 0.17 0.16
query41 0.09 0.05 0.05
query42 0.06 0.05 0.05
query43 0.05 0.04 0.04
Total cold run time: 105.78 s
Total hot run time: 30.98 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2027050763
<details>
<summary>TPC-H: <b>Total hot run time: 38068 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cd2644da25e12345d282159d1a5a95e8d546b4c4, data reload: false
------ Round 1 ----------------------------------
q1 17618 4382 4209 4209
q2 2088 195 186 186
q3 10456 1056 1157 1056
q4 10190 730 758 730
q5 7449 2796 2629 2629
q6 217 134 130 130
q7 968 537 542 537
q8 9218 2073 2041 2041
q9 7507 6630 6562 6562
q10 8468 3490 3497 3490
q11 450 229 232 229
q12 487 212 210 210
q13 17802 2982 2939 2939
q14 271 233 230 230
q15 506 471 465 465
q16 539 381 386 381
q17 963 584 562 562
q18 7289 6836 6756 6756
q19 1562 1428 1351 1351
q20 686 304 309 304
q21 3456 2753 2816 2753
q22 370 319 318 318
Total cold run time: 108560 ms
Total hot run time: 38068 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4272 4148 4185 4148
q2 377 277 267 267
q3 3033 2687 2782 2687
q4 1831 1550 1509 1509
q5 5253 5207 5237 5207
q6 211 123 128 123
q7 1774 1501 1422 1422
q8 3214 3297 3297 3297
q9 8568 8570 8551 8551
q10 3901 3622 3635 3622
q11 586 493 479 479
q12 744 582 636 582
q13 17733 2946 2941 2941
q14 299 269 277 269
q15 499 464 466 464
q16 483 429 444 429
q17 1752 1475 1456 1456
q18 7662 7480 7377 7377
q19 1618 1487 1545 1487
q20 1974 1748 1734 1734
q21 4913 4696 4702 4696
q22 514 473 444 444
Total cold run time: 71211 ms
Total hot run time: 53191 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2027046459
TeamCity be ut coverage result:
Function Coverage: 35.56% (8840/24861)
Line Coverage: 27.30% (72492/265565)
Region Coverage: 26.50% (37515/141541)
Branch Coverage: 23.32% (19131/82050)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cd2644da25e12345d282159d1a5a95e8d546b4c4_cd2644da25e12345d282159d1a5a95e8d546b4c4/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2027146466
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #32896:
URL: https://github.com/apache/doris/pull/32896#discussion_r1540691370
##########
be/src/olap/merger.cpp:
##########
Review Comment:
should we also add a regression test for it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on code in PR #32896:
URL: https://github.com/apache/doris/pull/32896#discussion_r1540702719
##########
be/src/olap/merger.cpp:
##########
Review Comment:
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2022216410
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) fix incorrect grouping of vertical compaction columns in tables only with key columns [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32896:
URL: https://github.com/apache/doris/pull/32896#issuecomment-2025394046
<details>
<summary>TPC-DS: <b>Total hot run time: 184088 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1b20cb752b6bf8d3bb857fe2848fd392b5cad528, data reload: false
query1 1123 359 359 359
query2 7939 2171 1962 1962
query3 6695 204 202 202
query4 31818 22235 22092 22092
query5 4698 472 434 434
query6 1382 203 189 189
query7 5737 297 309 297
query8 511 183 177 177
query9 10565 2339 2332 2332
query10 1351 263 258 258
query11 16851 14300 14247 14247
query12 132 87 88 87
query13 1618 421 415 415
query14 9991 7572 7533 7533
query15 257 175 186 175
query16 8260 264 266 264
query17 2003 605 553 553
query18 2107 295 296 295
query19 365 159 157 157
query20 96 94 91 91
query21 201 126 128 126
query22 4975 4838 4799 4799
query23 33855 33042 33062 33042
query24 10688 2966 2989 2966
query25 600 381 385 381
query26 1112 159 162 159
query27 3058 358 377 358
query28 7681 1904 1910 1904
query29 907 656 641 641
query30 302 154 152 152
query31 994 812 801 801
query32 99 57 55 55
query33 754 256 250 250
query34 1253 522 516 516
query35 919 730 719 719
query36 1083 935 929 929
query37 111 61 64 61
query38 3802 3695 3660 3660
query39 1796 1592 1611 1592
query40 173 113 113 113
query41 53 47 45 45
query42 113 101 101 101
query43 515 451 462 451
query44 1242 731 753 731
query45 316 273 255 255
query46 1135 714 749 714
query47 2047 1935 1919 1919
query48 455 355 378 355
query49 901 346 371 346
query50 796 389 397 389
query51 6894 6956 6848 6848
query52 107 93 99 93
query53 363 286 284 284
query54 301 251 252 251
query55 139 80 82 80
query56 271 278 243 243
query57 1316 1185 1175 1175
query58 247 230 221 221
query59 3061 2732 2693 2693
query60 282 267 268 267
query61 115 110 110 110
query62 625 439 443 439
query63 311 287 285 285
query64 5431 4027 4056 4027
query65 3075 3038 3045 3038
query66 945 345 349 345
query67 15687 14868 15192 14868
query68 8685 544 527 527
query69 632 397 372 372
query70 1289 1197 1107 1107
query71 541 268 271 268
query72 6310 2550 2347 2347
query73 745 309 322 309
query74 8383 6360 6488 6360
query75 3993 2193 2203 2193
query76 5249 904 863 863
query77 809 259 262 259
query78 11024 10238 10222 10222
query79 10815 519 520 519
query80 1417 390 375 375
query81 503 216 219 216
query82 829 89 84 84
query83 203 144 142 142
query84 288 82 72 72
query85 1390 311 303 303
query86 407 297 298 297
query87 3834 3580 3557 3557
query88 4993 2300 2307 2300
query89 505 375 364 364
query90 2036 176 175 175
query91 168 135 135 135
query92 59 46 48 46
query93 7132 481 486 481
query94 1122 194 180 180
query95 404 314 311 311
query96 601 271 275 271
query97 2626 2555 2465 2465
query98 252 212 221 212
query99 1214 845 840 840
Total cold run time: 323063 ms
Total hot run time: 184088 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org