You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "starocean999 (via GitHub)" <gi...@apache.org> on 2024/01/15 08:31:45 UTC
[PR] [fix](nereids)need validate auto partition columns in DDL [doris]
starocean999 opened a new pull request, #29985:
URL: https://github.com/apache/doris/pull/29985
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](nereids)need validate auto partition columns in DDL [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29985:
URL: https://github.com/apache/doris/pull/29985#issuecomment-1891644092
<details>
<summary>TPC-DS: <b>Total hot run time: 179461 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e6c02a3263efbd9696e914917caf8565ac23e658, data reload: false
query1 945 337 329 329
query2 6726 1879 1864 1864
query3 6706 211 203 203
query4 33674 22138 22310 22138
query5 6961 587 538 538
query6 258 189 204 189
query7 4627 282 284 282
query8 231 207 193 193
query9 9119 2736 2871 2736
query10 455 247 239 239
query11 16854 15450 15327 15327
query12 120 72 72 72
query13 1693 388 397 388
query14 11340 7446 7323 7323
query15 225 181 184 181
query16 4633 253 246 246
query17 1005 483 482 482
query18 1803 263 260 260
query19 200 151 152 151
query20 79 80 79 79
query21 190 139 130 130
query22 4933 4788 4766 4766
query23 32106 31046 31105 31046
query24 11462 2807 2863 2807
query25 556 319 327 319
query26 1091 151 155 151
query27 3180 282 281 281
query28 6788 1879 1878 1878
query29 1125 670 669 669
query30 282 137 144 137
query31 939 752 774 752
query32 79 56 61 56
query33 714 244 239 239
query34 1158 461 455 455
query35 885 832 785 785
query36 1450 1292 1327 1292
query37 104 73 73 73
query38 3391 3254 3255 3254
query39 1322 1283 1259 1259
query40 208 103 99 99
query41 39 35 35 35
query42 100 94 92 92
query43 552 509 538 509
query44 1087 713 728 713
query45 202 190 179 179
query46 1060 655 659 655
query47 1701 1606 1620 1606
query48 404 320 322 320
query49 1160 305 309 305
query50 696 312 311 311
query51 5275 5306 5216 5216
query52 100 93 95 93
query53 375 279 302 279
query54 899 453 459 453
query55 99 82 90 82
query56 200 196 204 196
query57 1021 954 972 954
query58 204 197 198 197
query59 3027 2831 2809 2809
query60 234 216 229 216
query61 86 84 87 84
query62 696 408 405 405
query63 329 282 288 282
query64 4952 1721 1719 1719
query65 3353 3286 3281 3281
query66 1268 350 367 350
query67 15743 15220 15014 15014
query68 14180 512 507 507
query69 656 333 344 333
query70 2354 1523 1593 1523
query71 568 227 247 227
query72 5196 2877 2849 2849
query73 3657 330 312 312
query74 7198 6449 6495 6449
query75 5380 2351 2345 2345
query76 6301 1040 1025 1025
query77 779 290 289 289
query78 9593 8632 8723 8632
query79 2745 518 509 509
query80 586 335 348 335
query81 469 206 206 206
query82 215 89 91 89
query83 165 128 125 125
query84 285 78 73 73
query85 1074 347 332 332
query86 386 432 432 432
query87 3537 3395 3383 3383
query88 3308 2218 2223 2218
query89 443 381 391 381
query90 2215 206 199 199
query91 153 126 129 126
query92 59 57 52 52
query93 1453 409 426 409
query94 1307 187 179 179
query95 526 482 474 474
query96 621 320 322 320
query97 4274 4198 4189 4189
query98 208 189 187 187
query99 1058 729 741 729
Total cold run time: 305428 ms
Total hot run time: 179461 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](nereids)need validate auto partition columns in DDL [doris]
Posted by "starocean999 (via GitHub)" <gi...@apache.org>.
starocean999 commented on PR #29985:
URL: https://github.com/apache/doris/pull/29985#issuecomment-1891558585
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](nereids)need validate auto partition columns in DDL [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29985:
URL: https://github.com/apache/doris/pull/29985
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](nereids)need validate auto partition columns in DDL [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29985:
URL: https://github.com/apache/doris/pull/29985#issuecomment-1891569028
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](nereids)need validate auto partition columns in DDL [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29985:
URL: https://github.com/apache/doris/pull/29985#issuecomment-1891626158
<details>
<summary>TPC-H: <b>Total hot run time: 39561 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e6c02a3263efbd9696e914917caf8565ac23e658, data reload: false
------ Round 1 ----------------------------------
q1 18005 6014 5663 5663
q2 2042 161 139 139
q3 10869 1134 1149 1134
q4 10533 800 871 800
q5 7848 3018 3130 3018
q6 208 126 126 126
q7 903 526 489 489
q8 9287 2000 2020 2000
q9 7343 6425 6402 6402
q10 8206 3071 3069 3069
q11 413 218 216 216
q12 369 197 202 197
q13 18118 3448 3404 3404
q14 249 223 228 223
q15 560 519 516 516
q16 431 391 383 383
q17 951 450 589 450
q18 7779 6945 6874 6874
q19 1633 1406 1385 1385
q20 515 318 299 299
q21 2858 2446 2574 2446
q22 384 328 332 328
Total cold run time: 109504 ms
Total hot run time: 39561 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5931 5766 5871 5766
q2 351 237 230 230
q3 3445 3144 3255 3144
q4 2271 2184 2136 2136
q5 5949 6070 5969 5969
q6 215 126 126 126
q7 2422 2003 1981 1981
q8 3683 3762 3808 3762
q9 9650 25425 9780 9780
q10 8854 4080 4052 4052
q11 724 513 520 513
q12 1261 751 764 751
q13 18377 3602 3617 3602
q14 806 331 346 331
q15 978 527 519 519
q16 676 508 541 508
q17 11140 2105 2128 2105
q18 9568 11163 12721 11163
q19 12990 1627 1613 1613
q20 4149 1963 1949 1949
q21 13536 6013 5708 5708
q22 1055 603 580 580
Total cold run time: 118031 ms
Total hot run time: 66288 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](nereids)need validate auto partition columns in DDL [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29985:
URL: https://github.com/apache/doris/pull/29985#issuecomment-1891630845
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.87 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 21.9 seconds inserted 10000000 Rows, about 456K ops/s
storage size: 17183916159 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](nereids)need validate auto partition columns in DDL [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29985:
URL: https://github.com/apache/doris/pull/29985#issuecomment-1891568975
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](nereids)need validate auto partition columns in DDL [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29985:
URL: https://github.com/apache/doris/pull/29985#issuecomment-1891661170
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit e6c02a3263efbd9696e914917caf8565ac23e658 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 14.3 seconds inserted 10000000 Rows, about 699K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](nereids)need validate auto partition columns in DDL [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29985:
URL: https://github.com/apache/doris/pull/29985#issuecomment-1891652532
<details>
<summary>ClickBench: <b>Total hot run time: 31.67 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e6c02a3263efbd9696e914917caf8565ac23e658, data reload: false
query1 0.06 0.05 0.06
query2 0.07 0.03 0.02
query3 0.26 0.12 0.12
query4 1.73 0.14 0.11
query5 0.53 0.53 0.53
query6 1.30 0.64 0.63
query7 0.02 0.01 0.01
query8 0.04 0.03 0.02
query9 0.56 0.50 0.49
query10 0.57 0.55 0.56
query11 0.12 0.09 0.09
query12 0.10 0.09 0.09
query13 0.60 0.60 0.61
query14 0.78 0.78 0.78
query15 0.79 0.78 0.77
query16 0.39 0.38 0.38
query17 1.02 1.03 1.00
query18 0.26 0.23 0.27
query19 1.86 1.71 1.76
query20 0.01 0.01 0.01
query21 15.44 0.59 0.55
query22 2.30 2.31 2.38
query23 17.11 0.83 0.76
query24 2.51 1.21 1.18
query25 0.40 0.28 0.15
query26 0.49 0.14 0.13
query27 0.07 0.07 0.06
query28 11.06 0.82 0.81
query29 12.59 3.19 3.26
query30 0.61 0.57 0.50
query31 2.78 0.34 0.35
query32 3.38 0.49 0.48
query33 3.20 3.22 3.21
query34 15.78 4.28 4.22
query35 4.23 4.24 4.24
query36 1.12 1.08 1.09
query37 0.07 0.05 0.05
query38 0.04 0.03 0.03
query39 0.02 0.01 0.02
query40 0.15 0.13 0.13
query41 0.07 0.02 0.01
query42 0.03 0.02 0.02
query43 0.02 0.02 0.02
Total cold run time: 104.54 s
Total hot run time: 31.67 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org