You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/03/13 08:43:29 UTC
[PR] [feature](merge-cloud) Add cloud copy load job meta definition [doris]
w41ter opened a new pull request, #32182:
URL: https://github.com/apache/doris/pull/32182
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add cloud copy load job meta definition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32182:
URL: https://github.com/apache/doris/pull/32182#issuecomment-1993900674
<details>
<summary>ClickBench: <b>Total hot run time: 30.42 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2f1ae806336c637266eb219573807e1953b6b447, data reload: false
query1 0.04 0.04 0.04
query2 0.06 0.03 0.03
query3 0.22 0.06 0.06
query4 1.67 0.09 0.09
query5 0.52 0.48 0.49
query6 1.21 0.64 0.62
query7 0.02 0.02 0.02
query8 0.04 0.03 0.03
query9 0.57 0.48 0.50
query10 0.55 0.55 0.54
query11 0.12 0.09 0.10
query12 0.11 0.10 0.09
query13 0.57 0.56 0.56
query14 0.75 0.77 0.77
query15 0.81 0.79 0.78
query16 0.38 0.37 0.38
query17 0.98 1.03 0.96
query18 0.18 0.28 0.22
query19 1.78 1.76 1.67
query20 0.01 0.01 0.01
query21 15.41 0.55 0.55
query22 2.72 1.80 1.77
query23 16.96 0.72 0.71
query24 3.24 2.09 1.19
query25 0.32 0.11 0.06
query26 0.72 0.12 0.12
query27 0.04 0.03 0.04
query28 9.73 0.86 0.88
query29 12.55 3.23 3.20
query30 0.69 0.60 0.59
query31 2.79 0.32 0.32
query32 3.42 0.44 0.44
query33 2.77 2.82 2.77
query34 16.06 4.31 4.30
query35 4.32 4.35 4.31
query36 1.07 1.01 1.01
query37 0.07 0.06 0.05
query38 0.04 0.03 0.02
query39 0.03 0.02 0.02
query40 0.17 0.13 0.14
query41 0.07 0.02 0.02
query42 0.03 0.02 0.02
query43 0.03 0.02 0.03
Total cold run time: 103.84 s
Total hot run time: 30.42 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add cloud copy load job meta definition [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32182:
URL: https://github.com/apache/doris/pull/32182#issuecomment-1993839532
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add cloud copy load job meta definition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32182:
URL: https://github.com/apache/doris/pull/32182#issuecomment-1993891143
<details>
<summary>TPC-DS: <b>Total hot run time: 187448 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2f1ae806336c637266eb219573807e1953b6b447, data reload: false
query1 921 358 334 334
query2 7373 2083 1857 1857
query3 6707 219 213 213
query4 31336 20747 20670 20670
query5 4349 415 528 415
query6 257 176 179 176
query7 4628 293 289 289
query8 225 165 174 165
query9 9889 2268 2251 2251
query10 450 244 240 240
query11 17073 14029 14118 14029
query12 129 85 84 84
query13 1638 424 421 421
query14 12257 10955 10491 10491
query15 260 212 188 188
query16 8067 262 259 259
query17 1940 571 544 544
query18 2086 286 277 277
query19 219 164 159 159
query20 105 85 82 82
query21 194 131 128 128
query22 4679 4476 4460 4460
query23 31798 31287 31117 31117
query24 10797 2862 2861 2861
query25 626 383 377 377
query26 1442 154 162 154
query27 3125 348 357 348
query28 8042 1901 1889 1889
query29 892 634 626 626
query30 306 145 161 145
query31 941 741 739 739
query32 103 59 54 54
query33 785 282 245 245
query34 1099 470 476 470
query35 833 616 601 601
query36 983 884 894 884
query37 128 75 77 75
query38 3334 3220 3205 3205
query39 1537 1380 1382 1380
query40 208 112 113 112
query41 56 46 46 46
query42 103 95 94 94
query43 468 446 445 445
query44 1098 726 706 706
query45 267 255 254 254
query46 1078 697 684 684
query47 1704 1626 1619 1619
query48 461 373 360 360
query49 1127 351 348 348
query50 763 374 376 374
query51 6894 6788 6790 6788
query52 107 88 91 88
query53 345 275 280 275
query54 339 252 244 244
query55 87 81 82 81
query56 259 230 234 230
query57 1106 1011 1059 1011
query58 229 208 207 207
query59 2794 2715 2597 2597
query60 271 246 247 246
query61 108 109 108 108
query62 617 397 412 397
query63 304 281 278 278
query64 6897 3912 3809 3809
query65 3100 3019 3003 3003
query66 1403 322 311 311
query67 15145 14414 14425 14414
query68 7918 543 544 543
query69 619 382 388 382
query70 1273 1187 1179 1179
query71 515 271 291 271
query72 6580 2648 2506 2506
query73 732 319 328 319
query74 6974 6598 6513 6513
query75 9682 8165 8178 8165
query76 5003 920 941 920
query77 657 268 271 268
query78 10459 9735 9536 9536
query79 10222 514 520 514
query80 1556 368 441 368
query81 516 209 205 205
query82 861 193 197 193
query83 211 157 140 140
query84 282 78 74 74
query85 1524 331 314 314
query86 466 294 291 291
query87 3471 3264 3278 3264
query88 5028 2299 2291 2291
query89 504 363 351 351
query90 1996 194 172 172
query91 158 128 132 128
query92 63 48 45 45
query93 7004 515 490 490
query94 1216 170 177 170
query95 430 329 326 326
query96 609 272 270 270
query97 4435 4270 4293 4270
query98 230 204 196 196
query99 1261 754 743 743
Total cold run time: 321346 ms
Total hot run time: 187448 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add cloud copy load job meta definition [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on PR #32182:
URL: https://github.com/apache/doris/pull/32182#issuecomment-1993838531
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add cloud copy load job meta definition [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32182:
URL: https://github.com/apache/doris/pull/32182#issuecomment-1993876366
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add cloud copy load job meta definition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32182:
URL: https://github.com/apache/doris/pull/32182#issuecomment-1993834165
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add cloud copy load job meta definition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32182:
URL: https://github.com/apache/doris/pull/32182#issuecomment-1993869338
<details>
<summary>TPC-H: <b>Total hot run time: 36591 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2f1ae806336c637266eb219573807e1953b6b447, data reload: false
------ Round 1 ----------------------------------
q1 17971 4184 4114 4114
q2 2032 149 144 144
q3 10587 916 912 912
q4 4648 700 675 675
q5 7576 2720 2798 2720
q6 196 123 122 122
q7 1190 813 798 798
q8 9494 2044 2054 2044
q9 7388 6486 6426 6426
q10 8224 2491 2479 2479
q11 420 224 210 210
q12 794 303 284 284
q13 17981 3018 3022 3018
q14 278 264 248 248
q15 503 455 454 454
q16 477 391 408 391
q17 951 510 539 510
q18 6781 5951 5854 5854
q19 1550 1411 1464 1411
q20 537 278 282 278
q21 7372 3218 3221 3218
q22 793 281 297 281
Total cold run time: 107743 ms
Total hot run time: 36591 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4090 4138 4045 4045
q2 330 234 225 225
q3 2998 2811 2840 2811
q4 1845 1584 1629 1584
q5 5205 5253 5250 5250
q6 207 114 118 114
q7 2278 1891 1828 1828
q8 3163 3291 3282 3282
q9 8536 8495 8535 8495
q10 6155 3687 3704 3687
q11 540 471 432 432
q12 707 539 575 539
q13 12268 2891 2850 2850
q14 273 252 251 251
q15 480 460 456 456
q16 461 422 406 406
q17 1713 1492 1481 1481
q18 7563 7253 7106 7106
q19 6452 1514 1551 1514
q20 1913 1723 1724 1723
q21 4818 4641 4540 4540
q22 519 450 453 450
Total cold run time: 72514 ms
Total hot run time: 53069 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add cloud copy load job meta definition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32182:
URL: https://github.com/apache/doris/pull/32182#issuecomment-1993906759
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 2f1ae806336c637266eb219573807e1953b6b447 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.6 seconds inserted 10000000 Rows, about 735K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add cloud copy load job meta definition [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #32182:
URL: https://github.com/apache/doris/pull/32182
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org