You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zclllyybb (via GitHub)" <gi...@apache.org> on 2024/01/09 15:04:58 UTC
[PR] [enhance](auto-partition) forbid null column for auto partition [doris]
zclllyybb opened a new pull request, #29749:
URL: https://github.com/apache/doris/pull/29749
## Proposed changes
Issue Number: close #xxx
1.
```sql
mysql> create table test_null1( k0 datetime(6) null ) auto partition by range date_trunc(k0, 'hour') ( ) DISTRIBUTED BY HASH(`k0`) BUCKETS 2 properties("replication_num" = "1");
ERROR 1105 (HY000): errCode = 2, detailMessage = The auto partition column must be NOT NULL
mysql> create table test_null2( k0 int null ) auto partition by list (k0) ( ) DISTRIBUTED BY HASH(`k0`) BUCKETS 2 properties("replication_num" = "1");
ERROR 1105 (HY000): errCode = 2, detailMessage = The auto partition column must be NOT NULL
```
2. add docs of `allowPartitionColumnNullable`
need pick 2.1, no 2.0
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29749:
URL: https://github.com/apache/doris/pull/29749
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883350170
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1884192543
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883385323
<details>
<summary>TPC-H: <b>Total hot run time: 39639 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b6efb90ccd0eb2a7920dfa97cd94452f07d15a05, data reload: false
------ Round 1 ----------------------------------
q1 18230 5559 5555 5555
q2 2026 155 147 147
q3 11189 1148 1204 1148
q4 10598 849 833 833
q5 8175 2942 2917 2917
q6 214 133 134 133
q7 920 526 512 512
q8 9481 2046 2042 2042
q9 6852 6537 6390 6390
q10 8229 3066 3042 3042
q11 441 217 222 217
q12 381 213 213 213
q13 18253 3466 3446 3446
q14 245 220 205 205
q15 551 519 533 519
q16 439 425 428 425
q17 941 729 655 655
q18 7417 6766 6702 6702
q19 1577 1521 1531 1521
q20 572 352 317 317
q21 2758 2375 2423 2375
q22 385 325 327 325
Total cold run time: 109874 ms
Total hot run time: 39639 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5537 5489 5520 5489
q2 341 261 252 252
q3 3365 3328 3318 3318
q4 2104 2044 2072 2044
q5 5859 5813 5825 5813
q6 202 121 123 121
q7 2313 1917 1910 1910
q8 3476 3508 3540 3508
q9 8939 8866 8951 8866
q10 3830 3857 3842 3842
q11 611 496 499 496
q12 789 630 632 630
q13 3929 3258 3262 3258
q14 289 273 271 271
q15 551 513 507 507
q16 548 503 506 503
q17 2394 2385 2394 2385
q18 8742 8523 8608 8523
q19 1583 1628 1620 1620
q20 2196 1962 1954 1954
q21 5675 5308 5310 5308
q22 581 528 531 528
Total cold run time: 63854 ms
Total hot run time: 61146 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1884192559
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883545566
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 75f1de3ab80600872023c32041fd59488ad06905 with default session variables
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 17.2 seconds inserted 10000000 Rows, about 581K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883456134
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.73 seconds
stream load tsv: 578 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.6 seconds inserted 10000000 Rows, about 462K ops/s
storage size: 17183893331 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883541090
<details>
<summary>ClickBench: <b>Total hot run time: 31.76 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 75f1de3ab80600872023c32041fd59488ad06905, data reload: false
query1 0.07 0.07 0.06
query2 0.06 0.02 0.02
query3 0.31 0.21 0.23
query4 1.72 0.20 0.19
query5 0.52 0.52 0.53
query6 1.55 0.68 0.68
query7 0.02 0.01 0.01
query8 0.04 0.03 0.03
query9 0.60 0.57 0.56
query10 0.62 0.62 0.62
query11 0.16 0.12 0.12
query12 0.16 0.12 0.11
query13 0.61 0.62 0.60
query14 0.77 0.81 0.76
query15 0.82 0.79 0.79
query16 0.36 0.35 0.34
query17 1.00 0.99 1.00
query18 0.27 0.25 0.26
query19 1.86 1.79 1.81
query20 0.01 0.01 0.02
query21 15.40 0.57 0.55
query22 2.37 4.85 1.40
query23 17.71 1.00 1.04
query24 8.25 0.32 0.81
query25 1.96 0.18 0.17
query26 0.18 0.17 0.15
query27 0.17 0.18 0.18
query28 6.03 0.89 0.85
query29 13.07 3.55 3.58
query30 0.52 0.52 0.51
query31 2.91 0.37 0.37
query32 3.32 0.49 0.49
query33 3.19 3.25 3.22
query34 16.34 4.49 4.61
query35 4.49 4.70 4.49
query36 1.16 1.10 1.09
query37 0.06 0.04 0.05
query38 0.03 0.02 0.02
query39 0.02 0.02 0.02
query40 0.15 0.15 0.14
query41 0.07 0.02 0.02
query42 0.02 0.02 0.01
query43 0.02 0.02 0.02
Total cold run time: 108.97 s
Total hot run time: 31.76 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1884157951
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.69 seconds
stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.8 seconds inserted 10000000 Rows, about 458K ops/s
storage size: 17183815972 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883515726
<details>
<summary>TPC-H: <b>Total hot run time: 39144 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 75f1de3ab80600872023c32041fd59488ad06905, data reload: false
------ Round 1 ----------------------------------
q1 18371 5557 5505 5505
q2 2040 156 142 142
q3 10632 1128 1175 1128
q4 10275 817 879 817
q5 7842 2931 2831 2831
q6 204 130 131 130
q7 920 508 483 483
q8 9284 1978 2016 1978
q9 6462 6368 6306 6306
q10 8193 3076 3041 3041
q11 443 231 208 208
q12 375 221 218 218
q13 18275 3460 3435 3435
q14 233 220 204 204
q15 549 516 505 505
q16 453 414 435 414
q17 930 742 641 641
q18 7321 6691 6662 6662
q19 1544 1496 1495 1495
q20 736 322 310 310
q21 2753 2355 2401 2355
q22 368 347 336 336
Total cold run time: 108203 ms
Total hot run time: 39144 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5527 5509 5501 5501
q2 358 253 258 253
q3 3356 3332 3329 3329
q4 2122 2092 2059 2059
q5 5883 5815 5822 5815
q6 210 124 123 123
q7 2298 1939 1922 1922
q8 3400 3472 3542 3472
q9 8906 8868 8915 8868
q10 3830 3875 3844 3844
q11 603 513 504 504
q12 799 633 607 607
q13 6947 3270 3263 3263
q14 293 272 258 258
q15 551 504 512 504
q16 564 507 488 488
q17 2357 2374 2346 2346
q18 8702 8395 8387 8387
q19 1568 1617 1601 1601
q20 2206 1987 1955 1955
q21 5635 5335 5440 5335
q22 579 523 508 508
Total cold run time: 66694 ms
Total hot run time: 60942 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1884133828
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883416283
<details>
<summary>TPC-H: <b>Total hot run time: 39740 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0c1c0bb2953b55eda273ab3e2ccc23ef55787fc5, data reload: false
------ Round 1 ----------------------------------
q1 14880 5597 5519 5519
q2 414 161 148 148
q3 1399 1121 1186 1121
q4 1045 831 835 831
q5 3114 3082 2910 2910
q6 212 136 132 132
q7 917 509 498 498
q8 1931 2037 2012 2012
q9 6417 6437 6422 6422
q10 3044 3065 3043 3043
q11 333 223 226 223
q12 369 218 217 217
q13 4241 3487 3488 3487
q14 244 220 212 212
q15 546 521 510 510
q16 465 412 423 412
q17 920 763 707 707
q18 7255 6767 6749 6749
q19 1514 1533 1529 1529
q20 494 326 339 326
q21 2700 2404 2415 2404
q22 370 328 347 328
Total cold run time: 52824 ms
Total hot run time: 39740 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5542 5533 5527 5527
q2 350 256 282 256
q3 3353 3328 3239 3239
q4 2113 2062 2069 2062
q5 5888 5858 5872 5858
q6 204 122 127 122
q7 2319 1905 1909 1905
q8 3396 3571 3564 3564
q9 8922 8924 8943 8924
q10 3846 3872 3809 3809
q11 596 500 500 500
q12 801 645 634 634
q13 3947 3278 3253 3253
q14 309 265 260 260
q15 548 507 514 507
q16 553 481 493 481
q17 2403 2397 2390 2390
q18 8711 8666 8450 8450
q19 1607 1626 1636 1626
q20 2195 1953 1952 1952
q21 5664 5169 5406 5169
q22 571 531 520 520
Total cold run time: 63838 ms
Total hot run time: 61008 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883470729
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883532587
<details>
<summary>TPC-DS: <b>Total hot run time: 186904 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 75f1de3ab80600872023c32041fd59488ad06905, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 951 358 335 335
query2 6853 2008 1981 1981
query3 6734 241 234 234
query4 27262 22601 22450 22450
query5 5614 570 570 570
query6 333 214 220 214
query7 4636 314 300 300
query8 265 245 232 232
query9 9953 3299 3266 3266
query10 606 355 336 336
query11 16286 15575 15662 15575
query12 173 128 121 121
query13 1682 344 343 343
query14 14087 7457 7681 7457
query15 258 247 255 247
query16 6469 361 351 351
query17 1702 531 537 531
query18 1962 315 314 314
query19 234 165 164 164
query20 127 115 121 115
query21 183 96 100 96
query22 4826 4563 4487 4487
query23 32651 31375 31380 31375
query24 12168 3016 2988 2988
query25 611 371 380 371
query26 1855 176 180 176
query27 3071 312 314 312
query28 7047 2151 2127 2127
query29 2022 428 425 425
query30 299 146 151 146
query31 1071 855 865 855
query32 131 103 101 101
query33 832 367 360 360
query34 879 491 483 483
query35 985 836 879 836
query36 1507 1360 1351 1351
query37 229 122 112 112
query38 3465 3424 3367 3367
query39 1367 1303 1327 1303
query40 302 94 91 91
query41 39 38 36 36
query42 123 116 119 116
query43 606 576 575 575
query44 1267 834 828 828
query45 221 195 209 195
query46 1065 735 708 708
query47 1725 1622 1693 1622
query48 376 294 289 289
query49 1219 354 334 334
query50 737 358 368 358
query51 5469 5397 5395 5395
query52 139 104 113 104
query53 238 194 189 189
query54 1449 711 685 685
query55 117 105 105 105
query56 293 267 278 267
query57 1111 1053 1064 1053
query58 345 337 332 332
query59 3351 3059 3053 3053
query60 392 322 300 300
query61 149 142 156 142
query62 567 519 493 493
query63 201 201 200 200
query64 6079 1741 1762 1741
query65 3469 3375 3405 3375
query66 1413 421 423 421
query67 15512 15257 15303 15257
query68 12135 565 554 554
query69 627 381 385 381
query70 1849 1785 1733 1733
query71 602 300 293 293
query72 5897 3454 3505 3454
query73 2440 365 372 365
query74 7441 7056 7023 7023
query75 4993 2298 2254 2254
query76 6537 1029 1194 1029
query77 799 340 368 340
query78 9039 8889 8828 8828
query79 1069 557 533 533
query80 571 352 362 352
query81 475 214 225 214
query82 245 124 128 124
query83 169 145 149 145
query84 262 57 58 57
query85 960 275 268 268
query86 429 462 473 462
query87 3686 3504 3499 3499
query88 3321 2637 2624 2624
query89 397 333 303 303
query90 1967 284 273 273
query91 119 94 98 94
query92 127 85 87 85
query93 1726 445 496 445
query94 839 264 247 247
query95 573 537 527 527
query96 659 355 353 353
query97 4399 4271 4269 4269
query98 249 213 210 210
query99 1180 895 903 895
Total cold run time: 305141 ms
Total hot run time: 186904 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883539296
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.1 seconds
stream load tsv: 573 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.6 seconds inserted 10000000 Rows, about 462K ops/s
storage size: 17184130616 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1884161911
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 1ca4bcb57b7f117c110feddf63910cd556e5003a with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 13.6 seconds inserted 10000000 Rows, about 735K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883451588
<details>
<summary>ClickBench: <b>Total hot run time: 31.63 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0c1c0bb2953b55eda273ab3e2ccc23ef55787fc5, data reload: false
query1 0.07 0.07 0.06
query2 0.06 0.03 0.03
query3 0.30 0.23 0.23
query4 1.73 0.19 0.20
query5 0.52 0.52 0.52
query6 1.56 0.68 0.68
query7 0.02 0.01 0.02
query8 0.04 0.03 0.03
query9 0.61 0.55 0.55
query10 0.59 0.61 0.60
query11 0.17 0.13 0.11
query12 0.16 0.13 0.12
query13 0.60 0.60 0.60
query14 0.78 0.78 0.77
query15 0.81 0.80 0.80
query16 0.34 0.34 0.36
query17 0.97 0.98 0.98
query18 0.25 0.25 0.25
query19 1.85 1.81 1.80
query20 0.01 0.01 0.01
query21 15.40 0.55 0.54
query22 2.54 2.99 1.49
query23 17.49 1.33 0.95
query24 10.04 0.34 0.33
query25 1.51 0.17 0.16
query26 0.17 0.18 0.15
query27 0.18 0.18 0.18
query28 6.31 0.91 0.86
query29 12.94 3.49 3.37
query30 0.52 0.51 0.52
query31 2.91 0.37 0.37
query32 3.32 0.48 0.48
query33 3.20 3.18 3.21
query34 16.60 4.54 4.52
query35 4.48 4.53 4.55
query36 1.16 1.08 1.10
query37 0.07 0.05 0.04
query38 0.04 0.03 0.02
query39 0.02 0.02 0.01
query40 0.15 0.13 0.13
query41 0.08 0.02 0.02
query42 0.02 0.01 0.02
query43 0.03 0.02 0.02
Total cold run time: 110.62 s
Total hot run time: 31.63 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883456825
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 0c1c0bb2953b55eda273ab3e2ccc23ef55787fc5 with default session variables
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 17.4 seconds inserted 10000000 Rows, about 574K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1884150032
<details>
<summary>TPC-H: <b>Total hot run time: 38777 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1ca4bcb57b7f117c110feddf63910cd556e5003a, data reload: false
------ Round 1 ----------------------------------
q1 18005 5048 5018 5018
q2 2035 149 133 133
q3 10668 1161 1181 1161
q4 10539 788 835 788
q5 7858 3100 3180 3100
q6 202 125 127 125
q7 865 520 495 495
q8 9273 2016 1982 1982
q9 6582 6455 6457 6455
q10 8272 3076 3086 3076
q11 423 205 199 199
q12 366 197 197 197
q13 18116 3423 3406 3406
q14 252 211 206 206
q15 571 516 504 504
q16 437 390 389 389
q17 949 518 558 518
q18 7357 6744 6691 6691
q19 1563 1359 1347 1347
q20 532 318 301 301
q21 2789 2417 2364 2364
q22 385 323 322 322
Total cold run time: 108039 ms
Total hot run time: 38777 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4975 4970 4954 4954
q2 316 206 205 205
q3 3358 3304 3313 3304
q4 2237 2254 2226 2226
q5 5799 5781 5790 5781
q6 193 120 119 119
q7 2322 1871 1855 1855
q8 3444 3558 3562 3558
q9 8892 8843 8843 8843
q10 3748 3829 3842 3829
q11 554 439 427 427
q12 780 600 612 600
q13 5920 3273 3215 3215
q14 287 266 266 266
q15 571 520 510 510
q16 518 458 467 458
q17 2056 2077 2020 2020
q18 8724 8354 8235 8235
q19 1603 1583 1600 1583
q20 2186 1943 1922 1922
q21 6131 5748 5731 5731
q22 614 506 516 506
Total cold run time: 65228 ms
Total hot run time: 60147 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883221404
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883439995
<details>
<summary>TPC-DS: <b>Total hot run time: 187017 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0c1c0bb2953b55eda273ab3e2ccc23ef55787fc5, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 942 349 336 336
query2 6870 2118 1993 1993
query3 6745 234 238 234
query4 26685 22636 22595 22595
query5 6112 627 621 621
query6 353 223 234 223
query7 4634 305 322 305
query8 254 254 248 248
query9 9784 3366 3301 3301
query10 560 387 351 351
query11 16515 15786 15786 15786
query12 185 137 119 119
query13 1696 347 357 347
query14 14177 7693 7474 7474
query15 253 256 251 251
query16 6362 365 351 351
query17 1686 544 529 529
query18 1956 331 323 323
query19 236 170 182 170
query20 134 127 134 127
query21 190 100 107 100
query22 4900 4670 4605 4605
query23 32405 31333 31467 31333
query24 11939 3024 3031 3024
query25 612 383 383 383
query26 1771 197 191 191
query27 2764 322 322 322
query28 7047 2153 2147 2147
query29 2123 448 423 423
query30 290 148 152 148
query31 1058 850 864 850
query32 137 103 101 101
query33 850 334 358 334
query34 899 503 517 503
query35 1034 887 870 870
query36 1489 1378 1306 1306
query37 213 118 115 115
query38 3501 3351 3423 3351
query39 1358 1327 1336 1327
query40 311 98 96 96
query41 49 39 37 37
query42 124 115 114 114
query43 628 584 566 566
query44 1259 826 839 826
query45 232 212 211 211
query46 1083 735 730 730
query47 1774 1745 1724 1724
query48 395 303 307 303
query49 1212 328 330 328
query50 795 345 342 342
query51 5509 5360 5430 5360
query52 136 105 112 105
query53 249 179 183 179
query54 1461 722 740 722
query55 120 108 107 107
query56 310 310 288 288
query57 1151 1017 1028 1017
query58 374 348 343 343
query59 3319 3078 3035 3035
query60 363 308 326 308
query61 154 129 144 129
query62 604 535 472 472
query63 214 199 215 199
query64 6074 1818 1735 1735
query65 3507 3390 3417 3390
query66 1335 502 429 429
query67 16313 15571 15123 15123
query68 16605 568 556 556
query69 615 369 378 369
query70 2532 1787 1829 1787
query71 632 296 320 296
query72 5281 3456 3474 3456
query73 4541 363 360 360
query74 7015 6495 6569 6495
query75 5512 2300 2286 2286
query76 6421 1246 1286 1246
query77 1009 371 377 371
query78 9150 8673 8587 8587
query79 2485 564 532 532
query80 618 367 387 367
query81 476 219 236 219
query82 239 135 126 126
query83 316 155 151 151
query84 264 59 56 56
query85 995 282 264 264
query86 448 463 462 462
query87 3670 3470 3511 3470
query88 3241 2626 2562 2562
query89 396 325 304 304
query90 2181 297 274 274
query91 165 96 101 96
query92 113 94 100 94
query93 1339 435 498 435
query94 957 268 261 261
query95 560 520 523 520
query96 662 353 368 353
query97 4360 4268 4229 4229
query98 244 227 206 206
query99 1162 892 916 892
Total cold run time: 313978 ms
Total hot run time: 187017 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1884156393
<details>
<summary>TPC-DS: <b>Total hot run time: 179972 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1ca4bcb57b7f117c110feddf63910cd556e5003a, data reload: false
query1 943 342 338 338
query2 6695 1912 1880 1880
query3 6715 220 210 210
query4 26228 22258 22147 22147
query5 5749 520 529 520
query6 287 189 195 189
query7 4635 290 280 280
query8 254 216 213 213
query9 9083 2742 2833 2742
query10 490 263 246 246
query11 16143 15368 15478 15368
query12 137 84 82 82
query13 1630 356 350 350
query14 12036 7308 7218 7218
query15 266 195 200 195
query16 6405 284 271 271
query17 1891 514 491 491
query18 1948 285 275 275
query19 302 147 140 140
query20 83 87 82 82
query21 188 98 90 90
query22 5092 4835 4770 4770
query23 32263 31266 31215 31215
query24 12005 2813 2863 2813
query25 570 346 355 346
query26 1892 163 154 154
query27 2849 291 280 280
query28 6865 1895 1891 1891
query29 2068 415 396 396
query30 289 147 145 145
query31 976 769 799 769
query32 93 63 62 62
query33 762 271 264 264
query34 915 447 430 430
query35 898 776 777 776
query36 1402 1262 1248 1248
query37 105 68 72 68
query38 3380 3289 3345 3289
query39 1348 1283 1302 1283
query40 310 101 100 100
query41 39 35 35 35
query42 102 96 93 93
query43 557 542 503 503
query44 1044 700 711 700
query45 211 194 190 190
query46 1068 660 668 660
query47 1652 1619 1616 1616
query48 397 318 324 318
query49 1264 325 324 324
query50 774 376 326 326
query51 5351 5309 5317 5309
query52 105 85 95 85
query53 224 150 160 150
query54 1434 595 596 595
query55 102 93 91 91
query56 232 208 203 203
query57 1050 956 958 956
query58 251 223 220 220
query59 2711 2527 2521 2521
query60 267 243 253 243
query61 85 87 90 87
query62 655 471 467 467
query63 180 160 154 154
query64 5806 1703 1683 1683
query65 3348 3279 3279 3279
query66 1330 364 321 321
query67 15708 15679 15629 15629
query68 11031 552 552 552
query69 585 283 316 283
query70 1832 1594 1525 1525
query71 501 224 238 224
query72 5027 2906 2879 2879
query73 2104 334 332 332
query74 7025 6475 6476 6475
query75 4862 2296 2275 2275
query76 6321 1109 1114 1109
query77 695 266 272 266
query78 9783 8817 8595 8595
query79 1028 522 503 503
query80 561 369 372 369
query81 459 217 215 215
query82 207 90 95 90
query83 177 149 139 139
query84 275 62 56 56
query85 954 302 302 302
query86 398 418 392 392
query87 3536 3428 3401 3401
query88 2906 2333 2326 2326
query89 335 275 269 269
query90 1861 216 205 205
query91 178 135 133 133
query92 67 55 56 55
query93 1040 489 443 443
query94 766 195 195 195
query95 487 440 444 440
query96 639 331 340 331
query97 4281 4173 4166 4166
query98 222 199 186 186
query99 1098 857 863 857
Total cold run time: 293378 ms
Total hot run time: 179972 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1884160135
<details>
<summary>ClickBench: <b>Total hot run time: 31.62 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1ca4bcb57b7f117c110feddf63910cd556e5003a, data reload: false
query1 0.07 0.05 0.05
query2 0.06 0.02 0.03
query3 0.25 0.12 0.10
query4 1.77 0.12 0.11
query5 0.53 0.52 0.52
query6 1.40 0.65 0.64
query7 0.02 0.01 0.02
query8 0.04 0.03 0.03
query9 0.56 0.49 0.50
query10 0.55 0.54 0.54
query11 0.13 0.08 0.09
query12 0.11 0.10 0.10
query13 0.61 0.60 0.61
query14 0.79 0.80 0.80
query15 0.82 0.81 0.81
query16 0.35 0.35 0.36
query17 0.97 1.00 0.99
query18 0.25 0.23 0.25
query19 1.80 1.75 1.73
query20 0.02 0.01 0.01
query21 15.44 0.59 0.58
query22 2.71 2.20 2.53
query23 17.27 0.94 0.83
query24 2.27 5.66 1.08
query25 2.12 0.16 0.15
query26 0.15 0.14 0.13
query27 0.14 0.16 0.14
query28 8.86 0.83 0.83
query29 12.61 3.33 3.21
query30 0.58 0.51 0.50
query31 2.78 0.37 0.36
query32 3.38 0.48 0.49
query33 3.20 3.27 3.25
query34 15.90 4.20 4.19
query35 4.19 4.20 4.17
query36 1.10 1.06 1.05
query37 0.07 0.06 0.05
query38 0.04 0.03 0.02
query39 0.02 0.02 0.01
query40 0.15 0.13 0.14
query41 0.06 0.02 0.02
query42 0.02 0.01 0.02
query43 0.02 0.02 0.02
Total cold run time: 104.18 s
Total hot run time: 31.62 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](auto-partition) forbid null column for auto partition [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #29749:
URL: https://github.com/apache/doris/pull/29749#issuecomment-1883384368
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org