You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "HHoflittlefish777 (via GitHub)" <gi...@apache.org> on 2024/01/11 09:55:41 UTC
[PR] [improve](multi-table-load) pause job when can not find table [doris]
HHoflittlefish777 opened a new pull request, #29870:
URL: https://github.com/apache/doris/pull/29870
## Proposed changes
If there is no table that can be found, the task will cycle forever and no data will be loaded. To avoid invalid scheduled tasks, It is better to pause the job rather than run it.
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1887339293
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit c3a8312c1a739d26844add96c06ab574e0ee88ab with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 12.5 seconds inserted 10000000 Rows, about 800K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "HHoflittlefish777 (via GitHub)" <gi...@apache.org>.
HHoflittlefish777 commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1886752574
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1887218215
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.61 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 21.4 seconds inserted 10000000 Rows, about 467K ops/s
storage size: 17183905518 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1886991899
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1887299806
<details>
<summary>TPC-H: <b>Total hot run time: 39143 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c3a8312c1a739d26844add96c06ab574e0ee88ab, data reload: false
------ Round 1 ----------------------------------
q1 17887 4991 5015 4991
q2 2042 145 138 138
q3 11620 1171 1212 1171
q4 10535 803 817 803
q5 8320 3266 3237 3237
q6 213 128 133 128
q7 906 544 516 516
q8 9594 2044 2057 2044
q9 7800 6539 6462 6462
q10 8253 3070 3081 3070
q11 427 223 210 210
q12 366 200 191 191
q13 18121 3432 3481 3432
q14 240 209 214 209
q15 565 512 504 504
q16 435 378 390 378
q17 962 567 533 533
q18 7430 6804 6707 6707
q19 1577 1421 1390 1390
q20 581 302 324 302
q21 2787 2425 2436 2425
q22 362 302 304 302
Total cold run time: 111023 ms
Total hot run time: 39143 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5003 4963 4984 4963
q2 312 204 209 204
q3 3336 3326 3317 3317
q4 2253 2256 2269 2256
q5 5818 5817 5832 5817
q6 198 119 120 119
q7 2326 1856 1894 1856
q8 3455 3552 3603 3552
q9 8918 8874 8933 8874
q10 3767 3844 3829 3829
q11 561 437 416 416
q12 790 616 604 604
q13 3981 3192 3213 3192
q14 295 247 265 247
q15 586 516 508 508
q16 519 462 448 448
q17 2076 2051 2016 2016
q18 8751 8382 8335 8335
q19 1631 1639 1624 1624
q20 2188 1935 1931 1931
q21 6033 5709 5771 5709
q22 590 471 477 471
Total cold run time: 63387 ms
Total hot run time: 60288 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888568573
<details>
<summary>ClickBench: <b>Total hot run time: 30.42 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 18f2f7ffc643f4de59bdf7f301947ba907da1155, data reload: false
query1 0.06 0.06 0.06
query2 0.07 0.03 0.02
query3 0.26 0.12 0.12
query4 1.76 0.14 0.12
query5 0.53 0.51 0.52
query6 1.38 0.66 0.64
query7 0.02 0.02 0.01
query8 0.04 0.03 0.02
query9 0.57 0.51 0.51
query10 0.57 0.55 0.57
query11 0.14 0.10 0.10
query12 0.12 0.10 0.09
query13 0.61 0.59 0.59
query14 0.77 0.81 0.79
query15 0.82 0.80 0.80
query16 0.35 0.34 0.36
query17 0.98 0.98 0.99
query18 0.26 0.25 0.25
query19 1.86 1.73 1.78
query20 0.01 0.01 0.02
query21 15.40 0.55 0.58
query22 2.42 2.55 1.80
query23 17.28 0.83 0.73
query24 16.01 0.59 0.58
query25 2.17 0.16 0.15
query26 0.13 0.14 0.13
query27 0.14 0.16 0.15
query28 6.24 0.84 0.85
query29 12.58 3.18 3.13
query30 0.56 0.53 0.46
query31 2.79 0.35 0.36
query32 3.40 0.48 0.48
query33 3.23 3.25 3.22
query34 15.83 4.15 4.17
query35 4.15 4.19 4.16
query36 1.11 1.04 1.06
query37 0.06 0.05 0.05
query38 0.04 0.03 0.03
query39 0.03 0.01 0.01
query40 0.15 0.13 0.13
query41 0.07 0.02 0.01
query42 0.02 0.02 0.01
query43 0.02 0.02 0.02
Total cold run time: 115.01 s
Total hot run time: 30.42 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1887323359
<details>
<summary>TPC-DS: <b>Total hot run time: 180268 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c3a8312c1a739d26844add96c06ab574e0ee88ab, data reload: false
query1 952 342 336 336
query2 6702 1897 1826 1826
query3 6740 207 202 202
query4 26224 22344 22174 22174
query5 5487 513 497 497
query6 270 187 200 187
query7 4624 293 273 273
query8 230 210 209 209
query9 9138 2802 2858 2802
query10 469 264 247 247
query11 16218 15437 15363 15363
query12 138 77 76 76
query13 1708 392 402 392
query14 11814 7389 7302 7302
query15 251 196 199 196
query16 6431 258 246 246
query17 1908 497 481 481
query18 1939 267 273 267
query19 294 153 149 149
query20 90 85 79 79
query21 204 133 122 122
query22 4987 4810 4667 4667
query23 32037 30956 30965 30956
query24 12107 2870 2882 2870
query25 555 349 340 340
query26 1754 155 156 155
query27 2833 281 284 281
query28 6976 1850 1861 1850
query29 2087 704 691 691
query30 284 143 145 143
query31 980 760 795 760
query32 91 64 65 64
query33 722 242 241 241
query34 1028 478 473 473
query35 943 814 773 773
query36 1408 1246 1235 1235
query37 185 74 73 73
query38 3402 3306 3316 3306
query39 1339 1278 1310 1278
query40 345 101 101 101
query41 38 36 40 36
query42 100 102 98 98
query43 540 505 479 479
query44 1090 704 714 704
query45 216 188 188 188
query46 1068 673 691 673
query47 1714 1572 1473 1473
query48 405 333 330 330
query49 1280 326 318 318
query50 711 325 332 325
query51 5349 5223 5193 5193
query52 106 90 102 90
query53 376 302 294 294
query54 2064 1600 1625 1600
query55 99 90 91 90
query56 216 202 201 201
query57 1048 1001 993 993
query58 231 206 204 204
query59 2762 2452 2509 2452
query60 246 225 243 225
query61 86 85 85 85
query62 591 419 415 415
query63 325 293 295 293
query64 5779 1717 1696 1696
query65 3394 3284 3279 3279
query66 1344 345 316 316
query67 15479 15332 15109 15109
query68 12640 509 515 509
query69 670 391 390 390
query70 1728 1614 1581 1581
query71 493 265 250 250
query72 4920 2866 2875 2866
query73 2759 328 312 312
query74 6862 6430 6486 6430
query75 5108 2362 2296 2296
query76 6411 1095 1071 1071
query77 670 286 277 277
query78 9767 8753 8535 8535
query79 1044 497 515 497
query80 535 349 351 349
query81 447 206 208 206
query82 205 93 91 91
query83 141 121 122 121
query84 285 77 78 77
query85 1026 375 334 334
query86 408 401 386 386
query87 3545 3423 3405 3405
query88 3112 2175 2195 2175
query89 469 413 386 386
query90 1816 207 199 199
query91 161 127 136 127
query92 63 55 54 54
query93 1381 445 432 432
query94 1238 185 182 182
query95 539 492 498 492
query96 612 314 313 313
query97 4304 4169 4181 4169
query98 215 198 190 190
query99 1093 724 759 724
Total cold run time: 297218 ms
Total hot run time: 180268 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888503033
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 36001b717c3a102aec17f772c4958a776d52b161 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 12.7 seconds inserted 10000000 Rows, about 787K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888662797
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1886796668
<details>
<summary>TPC-H: <b>Total hot run time: 38862 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8f01202752fb66ffc6aee7135a9ac0ba4db21ed1, data reload: false
------ Round 1 ----------------------------------
q1 18021 5045 5062 5045
q2 2050 142 132 132
q3 10686 1090 1151 1090
q4 10369 806 812 806
q5 7851 3196 3206 3196
q6 202 150 134 134
q7 899 512 519 512
q8 9298 2027 2019 2019
q9 7381 6506 6467 6467
q10 8256 3128 3071 3071
q11 412 203 212 203
q12 367 206 201 201
q13 18124 3457 3396 3396
q14 242 213 218 213
q15 558 507 505 505
q16 437 386 388 386
q17 950 495 526 495
q18 7425 6755 6701 6701
q19 1611 1454 1332 1332
q20 534 290 313 290
q21 2853 2374 2430 2374
q22 362 315 294 294
Total cold run time: 108888 ms
Total hot run time: 38862 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4971 4998 4975 4975
q2 313 200 198 198
q3 3345 3310 3320 3310
q4 2269 2279 2253 2253
q5 5848 5809 5832 5809
q6 193 123 123 123
q7 2317 1857 1868 1857
q8 3501 3580 3598 3580
q9 8916 8813 8837 8813
q10 3763 3841 3833 3833
q11 557 439 439 439
q12 795 622 612 612
q13 3934 3209 3198 3198
q14 292 259 272 259
q15 582 516 510 510
q16 515 466 458 458
q17 2060 2046 2059 2046
q18 8812 8390 8383 8383
q19 1621 1636 1637 1636
q20 2187 1927 1927 1927
q21 6200 5766 5763 5763
q22 571 460 479 460
Total cold run time: 63562 ms
Total hot run time: 60442 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1887075698
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888571394
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 18f2f7ffc643f4de59bdf7f301947ba907da1155 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 12.6 seconds inserted 10000000 Rows, about 793K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888500755
<details>
<summary>ClickBench: <b>Total hot run time: 30.52 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 36001b717c3a102aec17f772c4958a776d52b161, data reload: false
query1 0.06 0.06 0.06
query2 0.06 0.02 0.02
query3 0.25 0.12 0.10
query4 1.77 0.11 0.12
query5 0.52 0.51 0.52
query6 1.34 0.65 0.63
query7 0.02 0.01 0.01
query8 0.03 0.02 0.02
query9 0.55 0.48 0.48
query10 0.55 0.54 0.54
query11 0.13 0.08 0.09
query12 0.11 0.10 0.09
query13 0.61 0.59 0.61
query14 0.76 0.80 0.79
query15 0.82 0.80 0.79
query16 0.34 0.36 0.35
query17 0.95 0.94 0.97
query18 0.25 0.25 0.24
query19 1.81 1.77 1.81
query20 0.01 0.01 0.01
query21 15.42 0.59 0.55
query22 2.57 2.63 2.01
query23 17.34 0.82 0.77
query24 16.00 0.58 0.56
query25 2.17 0.15 0.14
query26 0.13 0.13 0.13
query27 0.14 0.14 0.14
query28 6.02 0.80 0.83
query29 12.60 3.20 3.26
query30 0.58 0.49 0.44
query31 2.78 0.34 0.36
query32 3.41 0.49 0.48
query33 3.21 3.22 3.24
query34 15.83 4.21 4.17
query35 4.18 4.17 4.22
query36 1.10 1.03 1.03
query37 0.06 0.05 0.05
query38 0.04 0.02 0.02
query39 0.02 0.02 0.02
query40 0.16 0.12 0.13
query41 0.07 0.02 0.01
query42 0.02 0.02 0.01
query43 0.03 0.01 0.02
Total cold run time: 114.82 s
Total hot run time: 30.52 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1886829776
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 8f01202752fb66ffc6aee7135a9ac0ba4db21ed1 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 12.4 seconds inserted 10000000 Rows, about 806K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888562824
<details>
<summary>TPC-DS: <b>Total hot run time: 178516 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 18f2f7ffc643f4de59bdf7f301947ba907da1155, data reload: false
query1 917 361 327 327
query2 6696 1938 1907 1907
query3 6704 212 207 207
query4 27076 22213 22249 22213
query5 8543 549 577 549
query6 267 181 184 181
query7 4624 286 273 273
query8 223 203 196 196
query9 9156 2749 2733 2733
query10 650 236 238 236
query11 16128 15408 15424 15408
query12 138 73 74 73
query13 1715 379 410 379
query14 12123 7288 7191 7191
query15 237 188 191 188
query16 5852 255 242 242
query17 1932 494 480 480
query18 1965 287 255 255
query19 298 146 146 146
query20 80 79 71 71
query21 205 131 120 120
query22 5063 4728 4752 4728
query23 32027 30939 30928 30928
query24 12068 2870 2803 2803
query25 545 339 320 320
query26 1744 151 156 151
query27 2871 272 265 265
query28 7085 1858 1859 1858
query29 2101 702 649 649
query30 279 144 138 138
query31 1006 778 751 751
query32 87 61 59 59
query33 710 255 257 255
query34 1031 475 448 448
query35 877 756 722 722
query36 1341 1138 1171 1138
query37 182 71 68 68
query38 3475 3307 3309 3307
query39 1350 1297 1283 1283
query40 333 95 92 92
query41 38 34 35 34
query42 111 95 103 95
query43 541 510 512 510
query44 1043 699 710 699
query45 211 182 191 182
query46 1068 658 666 658
query47 1670 1596 1554 1554
query48 395 313 337 313
query49 1276 325 321 321
query50 709 315 313 313
query51 5299 5235 5223 5223
query52 103 95 94 94
query53 365 301 285 285
query54 904 451 466 451
query55 100 89 96 89
query56 209 203 194 194
query57 995 973 935 935
query58 212 189 196 189
query59 2783 2595 2591 2591
query60 236 218 220 218
query61 85 81 81 81
query62 637 392 434 392
query63 308 286 274 274
query64 5775 1691 1647 1647
query65 3336 3273 3282 3273
query66 1277 327 325 325
query67 15766 15633 15146 15146
query68 12353 503 511 503
query69 706 377 380 377
query70 1649 1527 1543 1527
query71 562 239 244 239
query72 4839 2824 2846 2824
query73 2620 317 308 308
query74 7094 6461 6445 6445
query75 5034 2369 2356 2356
query76 6308 901 1084 901
query77 665 277 273 273
query78 9850 8723 8533 8533
query79 1052 499 493 493
query80 535 342 339 339
query81 453 211 212 211
query82 203 92 90 90
query83 143 116 129 116
query84 276 76 75 75
query85 1001 336 336 336
query86 419 353 407 353
query87 3596 3480 3402 3402
query88 3125 2197 2193 2193
query89 438 398 400 398
query90 1894 207 197 197
query91 169 127 127 127
query92 70 53 56 53
query93 1070 425 403 403
query94 1221 185 178 178
query95 540 488 480 480
query96 619 314 304 304
query97 4310 4202 4177 4177
query98 208 196 184 184
query99 1027 723 697 697
Total cold run time: 299205 ms
Total hot run time: 178516 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888561633
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.08 seconds
stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.7 seconds inserted 10000000 Rows, about 460K ops/s
storage size: 17183829804 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888550173
<details>
<summary>TPC-H: <b>Total hot run time: 39124 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 18f2f7ffc643f4de59bdf7f301947ba907da1155, data reload: false
------ Round 1 ----------------------------------
q1 17891 5141 5065 5065
q2 2043 142 131 131
q3 11003 1131 1200 1131
q4 10883 773 861 773
q5 8155 3301 3245 3245
q6 215 129 133 129
q7 915 526 517 517
q8 9630 2067 2050 2050
q9 7721 6506 6496 6496
q10 8276 3092 3075 3075
q11 421 204 200 200
q12 371 196 201 196
q13 18107 3446 3457 3446
q14 272 218 213 213
q15 552 519 515 515
q16 441 381 384 381
q17 958 537 535 535
q18 7389 6783 6703 6703
q19 1600 1390 1344 1344
q20 579 302 314 302
q21 2836 2375 2452 2375
q22 364 315 302 302
Total cold run time: 110622 ms
Total hot run time: 39124 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5003 4984 4952 4952
q2 319 202 198 198
q3 3345 3341 3325 3325
q4 2287 2280 2267 2267
q5 5824 5819 5815 5815
q6 201 119 122 119
q7 2333 1867 1841 1841
q8 3496 3580 3598 3580
q9 8959 8879 8848 8848
q10 3763 3840 3869 3840
q11 549 427 429 427
q12 801 627 618 618
q13 4118 3227 3265 3227
q14 287 261 268 261
q15 580 527 517 517
q16 517 451 480 451
q17 2070 2052 2048 2048
q18 8829 8383 8354 8354
q19 1610 1602 1614 1602
q20 2174 1975 1938 1938
q21 6047 5790 5800 5790
q22 568 450 467 450
Total cold run time: 63680 ms
Total hot run time: 60468 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1886825111
<details>
<summary>ClickBench: <b>Total hot run time: 30.61 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8f01202752fb66ffc6aee7135a9ac0ba4db21ed1, data reload: false
query1 0.06 0.06 0.05
query2 0.06 0.03 0.03
query3 0.26 0.12 0.11
query4 1.75 0.13 0.11
query5 0.52 0.52 0.52
query6 1.33 0.64 0.62
query7 0.01 0.01 0.01
query8 0.03 0.02 0.02
query9 0.57 0.50 0.49
query10 0.56 0.53 0.55
query11 0.13 0.09 0.09
query12 0.12 0.09 0.09
query13 0.60 0.60 0.60
query14 0.76 0.81 0.77
query15 0.84 0.82 0.79
query16 0.35 0.35 0.35
query17 0.96 0.98 1.00
query18 0.26 0.25 0.26
query19 1.86 1.77 1.73
query20 0.01 0.01 0.01
query21 15.43 0.59 0.57
query22 2.77 2.08 2.09
query23 17.14 0.77 0.94
query24 16.18 0.61 0.58
query25 2.14 0.15 0.14
query26 0.12 0.12 0.12
query27 0.15 0.14 0.14
query28 6.75 0.89 0.84
query29 12.56 3.20 3.18
query30 0.57 0.51 0.49
query31 2.78 0.34 0.36
query32 3.35 0.48 0.49
query33 3.17 3.21 3.21
query34 15.89 4.18 4.11
query35 4.18 4.19 4.12
query36 1.10 1.04 1.05
query37 0.07 0.05 0.05
query38 0.04 0.03 0.03
query39 0.02 0.01 0.02
query40 0.16 0.13 0.13
query41 0.07 0.01 0.01
query42 0.02 0.01 0.02
query43 0.02 0.02 0.01
Total cold run time: 115.72 s
Total hot run time: 30.61 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1886832351
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.55 seconds
stream load tsv: 571 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.6 seconds inserted 10000000 Rows, about 462K ops/s
storage size: 17184159977 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1887197657
<details>
<summary>TPC-H: <b>Total hot run time: 38628 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1a46d948e22fd221910b8aa97067b10cafa36d4d, data reload: false
------ Round 1 ----------------------------------
q1 17991 5053 4978 4978
q2 2042 145 128 128
q3 10696 1097 1197 1097
q4 10253 759 848 759
q5 7811 3213 3147 3147
q6 204 132 126 126
q7 890 508 485 485
q8 9302 2033 2051 2033
q9 7353 6515 6474 6474
q10 8237 3065 3061 3061
q11 425 212 214 212
q12 364 192 196 192
q13 18106 3440 3419 3419
q14 233 213 213 213
q15 557 516 506 506
q16 435 380 376 376
q17 946 502 490 490
q18 7340 6711 6633 6633
q19 1605 1377 1403 1377
q20 539 277 299 277
q21 2790 2343 2433 2343
q22 360 324 302 302
Total cold run time: 108479 ms
Total hot run time: 38628 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4959 4920 4928 4920
q2 314 202 193 193
q3 3325 3299 3294 3294
q4 2229 2258 2252 2252
q5 5798 5805 5808 5805
q6 194 116 118 116
q7 2327 1841 1865 1841
q8 3444 3564 3588 3564
q9 8914 8901 8863 8863
q10 3748 3815 3829 3815
q11 570 423 438 423
q12 783 596 602 596
q13 4839 3210 3196 3196
q14 295 254 257 254
q15 576 513 512 512
q16 514 453 454 453
q17 2065 2032 2037 2032
q18 8845 8371 8295 8295
q19 1623 1602 1610 1602
q20 2161 1916 1925 1916
q21 6016 5777 5804 5777
q22 554 456 494 456
Total cold run time: 64093 ms
Total hot run time: 60175 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1887217123
<details>
<summary>TPC-DS: <b>Total hot run time: 179077 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1a46d948e22fd221910b8aa97067b10cafa36d4d, data reload: false
query1 926 338 326 326
query2 6720 1888 1865 1865
query3 6716 205 205 205
query4 27561 22154 22229 22154
query5 4510 465 472 465
query6 274 173 176 173
query7 4629 272 269 269
query8 239 198 200 198
query9 9211 2710 2759 2710
query10 549 244 237 237
query11 16152 15415 15260 15260
query12 129 70 74 70
query13 1706 378 386 378
query14 11886 7269 7078 7078
query15 254 183 198 183
query16 6520 248 248 248
query17 1934 488 478 478
query18 1956 267 259 259
query19 302 152 147 147
query20 82 75 69 69
query21 199 134 126 126
query22 4902 4811 4840 4811
query23 31617 30844 30825 30825
query24 12177 2821 2755 2755
query25 556 324 320 320
query26 1790 148 149 148
query27 2942 274 288 274
query28 7097 1864 1850 1850
query29 2135 692 663 663
query30 286 138 141 138
query31 1031 747 774 747
query32 93 60 58 58
query33 718 244 243 243
query34 1122 462 471 462
query35 867 760 764 760
query36 1404 1298 1316 1298
query37 184 68 71 68
query38 3386 3297 3262 3262
query39 1345 1281 1268 1268
query40 333 95 94 94
query41 39 35 35 35
query42 98 99 93 93
query43 509 519 522 519
query44 1064 696 697 696
query45 202 187 185 185
query46 1042 652 659 652
query47 1620 1548 1588 1548
query48 397 323 314 314
query49 1266 319 300 300
query50 701 309 322 309
query51 5342 5137 5178 5137
query52 104 93 104 93
query53 354 283 284 283
query54 2056 1608 1609 1608
query55 91 86 90 86
query56 220 209 196 196
query57 1013 950 965 950
query58 211 199 194 194
query59 2726 2575 2527 2527
query60 258 219 230 219
query61 84 84 90 84
query62 585 372 398 372
query63 311 278 288 278
query64 5840 1643 1669 1643
query65 3340 3257 3229 3229
query66 1425 328 320 320
query67 15568 15463 14976 14976
query68 12277 504 511 504
query69 658 375 383 375
query70 1795 1595 1553 1553
query71 511 243 230 230
query72 4840 2853 2817 2817
query73 2464 320 310 310
query74 6911 6415 6408 6408
query75 4959 2341 2330 2330
query76 6334 1089 1071 1071
query77 686 272 259 259
query78 9807 8696 8665 8665
query79 1018 494 506 494
query80 532 339 345 339
query81 458 204 202 202
query82 208 86 89 86
query83 141 118 122 118
query84 281 75 76 75
query85 999 334 336 334
query86 405 395 375 375
query87 3563 3372 3373 3372
query88 3194 2196 2188 2188
query89 456 370 405 370
query90 1949 203 192 192
query91 160 132 127 127
query92 66 53 53 53
query93 1157 402 406 402
query94 1294 180 180 180
query95 530 463 477 463
query96 616 308 313 308
query97 4319 4180 4175 4175
query98 212 193 183 183
query99 1004 742 731 731
Total cold run time: 296710 ms
Total hot run time: 179077 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1887333874
<details>
<summary>ClickBench: <b>Total hot run time: 30.5 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c3a8312c1a739d26844add96c06ab574e0ee88ab, data reload: false
query1 0.06 0.05 0.06
query2 0.06 0.02 0.02
query3 0.25 0.12 0.12
query4 1.75 0.12 0.12
query5 0.51 0.52 0.52
query6 1.38 0.64 0.62
query7 0.01 0.01 0.01
query8 0.04 0.02 0.02
query9 0.56 0.50 0.50
query10 0.56 0.55 0.55
query11 0.14 0.11 0.09
query12 0.12 0.10 0.10
query13 0.60 0.60 0.60
query14 0.77 0.79 0.80
query15 0.82 0.80 0.80
query16 0.34 0.36 0.35
query17 1.00 0.97 0.97
query18 0.27 0.25 0.25
query19 1.85 1.76 1.77
query20 0.01 0.01 0.01
query21 15.39 0.57 0.57
query22 2.41 2.27 1.68
query23 17.08 0.85 0.77
query24 16.01 0.60 0.58
query25 2.16 0.16 0.14
query26 0.14 0.14 0.13
query27 0.14 0.15 0.16
query28 5.66 0.85 0.87
query29 12.57 3.23 3.14
query30 0.56 0.49 0.48
query31 2.78 0.34 0.36
query32 3.40 0.49 0.49
query33 3.23 3.27 3.24
query34 15.82 4.22 4.19
query35 4.15 4.21 4.19
query36 1.11 1.05 1.05
query37 0.07 0.05 0.05
query38 0.04 0.03 0.03
query39 0.02 0.02 0.02
query40 0.16 0.13 0.13
query41 0.07 0.01 0.02
query42 0.02 0.01 0.02
query43 0.02 0.02 0.01
Total cold run time: 114.11 s
Total hot run time: 30.5 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1886815697
<details>
<summary>TPC-DS: <b>Total hot run time: 179446 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8f01202752fb66ffc6aee7135a9ac0ba4db21ed1, data reload: false
query1 919 337 334 334
query2 6722 1871 1846 1846
query3 6723 212 208 208
query4 26119 22211 22155 22155
query5 5365 515 484 484
query6 267 201 186 186
query7 4640 289 266 266
query8 229 210 196 196
query9 9191 2853 2798 2798
query10 532 261 255 255
query11 16302 15374 15300 15300
query12 135 76 70 70
query13 1706 377 394 377
query14 11916 7153 7293 7153
query15 239 185 187 185
query16 6440 255 246 246
query17 1896 492 482 482
query18 1925 267 269 267
query19 292 146 143 143
query20 83 74 71 71
query21 196 121 131 121
query22 4963 4694 4700 4694
query23 31926 31057 31006 31006
query24 12431 2868 2814 2814
query25 562 330 320 320
query26 1724 153 161 153
query27 2885 274 278 274
query28 7102 1866 1858 1858
query29 2109 703 650 650
query30 291 141 139 139
query31 970 780 772 772
query32 94 65 58 58
query33 727 233 245 233
query34 1107 475 471 471
query35 878 786 723 723
query36 1402 1124 1181 1124
query37 187 74 70 70
query38 3445 3290 3245 3245
query39 1333 1309 1288 1288
query40 343 99 95 95
query41 41 35 36 35
query42 104 94 92 92
query43 561 510 528 510
query44 1110 719 728 719
query45 216 184 186 184
query46 1050 654 661 654
query47 1647 1565 1579 1565
query48 401 330 325 325
query49 1278 320 332 320
query50 705 329 325 325
query51 5316 5231 5216 5216
query52 106 96 93 93
query53 354 295 302 295
query54 2065 1596 1638 1596
query55 94 84 88 84
query56 216 209 194 194
query57 1025 964 954 954
query58 219 203 194 194
query59 2774 2603 2623 2603
query60 217 234 207 207
query61 86 84 85 84
query62 627 412 393 393
query63 305 296 283 283
query64 5784 1712 1646 1646
query65 3357 3268 3272 3268
query66 1343 335 336 335
query67 15768 15003 15391 15003
query68 12630 539 532 532
query69 664 379 387 379
query70 1727 1561 1542 1542
query71 497 239 247 239
query72 4789 2877 2848 2848
query73 3006 313 321 313
query74 6991 6476 6397 6397
query75 5137 2312 2322 2312
query76 6409 894 1073 894
query77 666 284 292 284
query78 9889 8493 8631 8493
query79 1046 497 496 496
query80 540 359 344 344
query81 465 207 209 207
query82 211 89 95 89
query83 147 122 120 120
query84 282 76 72 72
query85 1069 346 330 330
query86 400 375 386 375
query87 3563 3412 3415 3412
query88 3161 2226 2213 2213
query89 488 386 407 386
query90 1890 199 198 198
query91 156 129 135 129
query92 64 63 53 53
query93 1344 451 448 448
query94 1214 182 182 182
query95 509 476 461 461
query96 622 335 312 312
query97 4343 4160 4216 4160
query98 217 192 180 180
query99 1044 739 753 739
Total cold run time: 298265 ms
Total hot run time: 179446 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "HHoflittlefish777 (via GitHub)" <gi...@apache.org>.
HHoflittlefish777 commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888463374
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888500154
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.46 seconds
stream load tsv: 560 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.0 seconds inserted 10000000 Rows, about 476K ops/s
storage size: 17183478341 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "HHoflittlefish777 (via GitHub)" <gi...@apache.org>.
HHoflittlefish777 commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888522458
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29870:
URL: https://github.com/apache/doris/pull/29870
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "HHoflittlefish777 (via GitHub)" <gi...@apache.org>.
HHoflittlefish777 commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1887224471
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888487085
<details>
<summary>TPC-H: <b>Total hot run time: 38532 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 36001b717c3a102aec17f772c4958a776d52b161, data reload: false
------ Round 1 ----------------------------------
q1 17958 5035 4961 4961
q2 2036 134 128 128
q3 10901 1101 1214 1101
q4 10461 774 804 774
q5 7805 3177 3170 3170
q6 200 128 128 128
q7 866 493 485 485
q8 9313 1978 2011 1978
q9 7350 6489 6388 6388
q10 8252 3047 3051 3047
q11 416 199 206 199
q12 366 193 193 193
q13 18117 3437 3438 3437
q14 242 213 208 208
q15 546 516 510 510
q16 422 368 376 368
q17 936 489 518 489
q18 7296 6688 6639 6639
q19 1584 1367 1364 1364
q20 534 305 285 285
q21 2806 2395 2389 2389
q22 347 291 297 291
Total cold run time: 108754 ms
Total hot run time: 38532 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4964 4966 4983 4966
q2 308 194 195 194
q3 3307 3328 3295 3295
q4 2243 2245 2245 2245
q5 5811 5786 5769 5769
q6 191 117 117 117
q7 2310 1870 1863 1863
q8 3440 3553 3589 3553
q9 8784 8791 8784 8784
q10 3760 3814 3819 3814
q11 537 431 406 406
q12 775 618 601 601
q13 4982 3211 3205 3205
q14 284 258 270 258
q15 567 512 514 512
q16 503 452 453 452
q17 2042 2052 2041 2041
q18 8704 8325 8364 8325
q19 1613 1609 1586 1586
q20 2154 1911 1900 1900
q21 6150 5602 5814 5602
q22 556 445 454 445
Total cold run time: 63985 ms
Total hot run time: 59933 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "HHoflittlefish777 (via GitHub)" <gi...@apache.org>.
HHoflittlefish777 commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1886758256
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1887317185
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.43 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.5 seconds inserted 10000000 Rows, about 465K ops/s
storage size: 17183838599 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1887075265
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](multi-table-load) pause job when can not find table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29870:
URL: https://github.com/apache/doris/pull/29870#issuecomment-1888496517
<details>
<summary>TPC-DS: <b>Total hot run time: 178802 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 36001b717c3a102aec17f772c4958a776d52b161, data reload: false
query1 924 331 328 328
query2 6729 1836 1901 1836
query3 6703 207 208 207
query4 26112 22146 21995 21995
query5 5045 504 515 504
query6 269 190 190 190
query7 4620 276 270 270
query8 230 198 192 192
query9 9169 2674 2842 2674
query10 470 235 231 231
query11 16219 15228 15228 15228
query12 120 73 73 73
query13 1721 386 380 380
query14 11687 7219 7214 7214
query15 300 187 192 187
query16 6492 259 258 258
query17 1889 498 483 483
query18 1928 264 258 258
query19 287 144 145 144
query20 83 76 70 70
query21 197 123 121 121
query22 5028 4838 4840 4838
query23 31573 30690 30816 30690
query24 12117 2841 2840 2840
query25 560 332 330 330
query26 1780 153 151 151
query27 2873 279 278 278
query28 7102 1864 1858 1858
query29 2092 703 680 680
query30 279 137 137 137
query31 985 775 768 768
query32 97 64 60 60
query33 720 236 266 236
query34 977 470 455 455
query35 961 759 750 750
query36 1402 1282 1244 1244
query37 105 68 71 68
query38 3378 3340 3268 3268
query39 1331 1295 1278 1278
query40 334 93 90 90
query41 37 34 37 34
query42 99 98 93 93
query43 549 485 492 485
query44 1106 705 716 705
query45 207 173 180 173
query46 1045 648 654 648
query47 1619 1492 1541 1492
query48 401 325 314 314
query49 1283 310 314 310
query50 695 315 323 315
query51 5251 5192 5151 5151
query52 106 86 100 86
query53 362 288 280 280
query54 2058 1617 1600 1600
query55 94 80 93 80
query56 211 204 203 203
query57 1037 963 940 940
query58 214 188 194 188
query59 2689 2498 2468 2468
query60 252 212 208 208
query61 86 94 86 86
query62 595 384 370 370
query63 309 283 276 276
query64 5894 1698 1677 1677
query65 3337 3263 3247 3247
query66 1371 343 316 316
query67 15699 15384 15246 15246
query68 12036 498 532 498
query69 661 375 366 366
query70 1624 1531 1487 1487
query71 506 225 224 224
query72 4914 2840 2937 2840
query73 2416 313 318 313
query74 7007 6422 6437 6422
query75 4926 2295 2305 2295
query76 6314 880 1170 880
query77 668 259 279 259
query78 9704 8514 9011 8514
query79 1055 522 524 522
query80 543 348 385 348
query81 464 212 211 211
query82 209 92 85 85
query83 157 129 130 129
query84 285 80 76 76
query85 1034 354 327 327
query86 439 390 385 385
query87 3588 3462 3381 3381
query88 3165 2207 2214 2207
query89 467 392 381 381
query90 1850 202 193 193
query91 161 127 150 127
query92 63 53 57 53
query93 1323 407 403 403
query94 1303 180 185 180
query95 520 465 466 465
query96 624 323 320 320
query97 4278 4162 4173 4162
query98 203 183 181 181
query99 1030 733 715 715
Total cold run time: 295031 ms
Total hot run time: 178802 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org