You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Jibing-Li (via GitHub)" <gi...@apache.org> on 2024/03/07 14:31:10 UTC
[PR] [fix](statistics)Add synchronize for modify analysisTaskInfoMap and analysisJobInfoMap. #31940 [doris]
Jibing-Li opened a new pull request, #31967:
URL: https://github.com/apache/doris/pull/31967
backport https://github.com/apache/doris/pull/31940
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Add synchronize for modify analysisTaskInfoMap and analysisJobInfoMap. #31940 [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li merged PR #31967:
URL: https://github.com/apache/doris/pull/31967
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Add synchronize for modify analysisTaskInfoMap and analysisJobInfoMap. #31940 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31967:
URL: https://github.com/apache/doris/pull/31967#issuecomment-1983682686
<details>
<summary>TPC-H: <b>Total hot run time: 50015 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a3e1f441e79b9089f4f553c92dea902e32a3b739, data reload: false
------ Round 1 ----------------------------------
q1 17929 4386 4361 4361
q2 2047 154 138 138
q3 10505 1901 1926 1901
q4 10289 1274 1332 1274
q5 8525 3951 3945 3945
q6 234 123 124 123
q7 2023 1589 1605 1589
q8 9263 2686 2687 2686
q9 11007 10595 10416 10416
q10 8622 3508 3529 3508
q11 404 239 242 239
q12 466 294 302 294
q13 18368 4032 4107 4032
q14 356 325 333 325
q15 507 460 463 460
q16 722 624 622 622
q17 1136 993 997 993
q18 7291 6922 6878 6878
q19 1677 1567 1472 1472
q20 539 296 285 285
q21 4453 4080 4107 4080
q22 515 398 394 394
Total cold run time: 116878 ms
Total hot run time: 50015 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4377 4311 4297 4297
q2 320 219 225 219
q3 4144 4120 4112 4112
q4 2745 2748 2724 2724
q5 7245 7185 7141 7141
q6 239 119 120 119
q7 3235 2804 2824 2804
q8 4337 4450 4443 4443
q9 17081 16993 17061 16993
q10 4240 4270 4256 4256
q11 750 692 701 692
q12 1005 861 847 847
q13 6737 3750 3757 3750
q14 442 421 428 421
q15 498 465 451 451
q16 744 706 697 697
q17 3863 3895 3801 3801
q18 8679 8803 8768 8768
q19 1689 1692 1671 1671
q20 2408 2147 2088 2088
q21 8464 8393 8525 8393
q22 1017 962 920 920
Total cold run time: 84259 ms
Total hot run time: 79607 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Add synchronize for modify analysisTaskInfoMap and analysisJobInfoMap. #31940 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31967:
URL: https://github.com/apache/doris/pull/31967#issuecomment-1983716217
<details>
<summary>TPC-DS: <b>Total hot run time: 201954 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a3e1f441e79b9089f4f553c92dea902e32a3b739, data reload: false
query1 912 389 379 379
query2 6520 2156 2178 2156
query3 6917 192 195 192
query4 20238 17853 18550 17853
query5 19706 6549 6493 6493
query6 292 214 223 214
query7 4169 292 307 292
query8 232 221 231 221
query9 3073 2655 2556 2556
query10 404 292 296 292
query11 11408 10781 10633 10633
query12 120 70 72 70
query13 5572 630 637 630
query14 17909 13437 13792 13437
query15 364 241 242 241
query16 6474 265 258 258
query17 1772 1449 895 895
query18 2291 402 406 402
query19 219 149 150 149
query20 75 76 75 75
query21 188 92 95 92
query22 5215 5240 5161 5161
query23 32795 32014 32564 32014
query24 7002 6573 6658 6573
query25 564 466 448 448
query26 1292 179 168 168
query27 2618 316 304 304
query28 7268 2206 2187 2187
query29 2881 2781 2633 2633
query30 254 164 167 164
query31 1209 726 703 703
query32 66 58 58 58
query33 426 246 254 246
query34 850 461 471 461
query35 1090 927 917 917
query36 1511 1256 1333 1256
query37 87 60 60 60
query38 3077 2893 2920 2893
query39 1373 1333 1294 1294
query40 203 89 94 89
query41 34 32 31 31
query42 85 79 82 79
query43 608 564 502 502
query44 1105 710 712 710
query45 242 227 229 227
query46 1225 980 957 957
query47 1804 1692 1716 1692
query48 970 667 667 667
query49 646 374 350 350
query50 858 630 615 615
query51 4686 4655 4704 4655
query52 90 78 81 78
query53 450 316 325 316
query54 2659 2469 2441 2441
query55 82 77 78 77
query56 206 206 190 190
query57 1294 1106 1097 1097
query58 206 201 186 186
query59 3415 3341 3167 3167
query60 207 195 202 195
query61 85 82 86 82
query62 778 552 574 552
query63 480 335 331 331
query64 2414 1496 1314 1314
query65 3613 3509 3548 3509
query66 813 374 371 371
query67 15586 15885 15876 15876
query68 10534 648 669 648
query69 573 345 333 333
query70 2480 1704 1616 1616
query71 426 305 309 305
query72 6424 3438 3436 3436
query73 728 321 319 319
query74 6367 5874 5792 5792
query75 5446 3748 3653 3653
query76 6666 1154 1199 1154
query77 1178 259 250 250
query78 12705 11866 11399 11399
query79 11287 651 627 627
query80 1097 385 378 378
query81 488 229 233 229
query82 698 94 97 94
query83 170 131 129 129
query84 257 69 66 66
query85 899 276 276 276
query86 393 364 357 357
query87 3246 3002 3063 3002
query88 5167 2288 2291 2288
query89 432 328 271 271
query90 2085 211 191 191
query91 157 113 112 112
query92 57 50 50 50
query93 6414 544 557 544
query94 763 203 206 203
query95 1095 1061 1047 1047
query96 636 317 320 317
query97 6460 6390 6377 6377
query98 189 179 168 168
query99 3140 897 942 897
Total cold run time: 321934 ms
Total hot run time: 201954 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Add synchronize for modify analysisTaskInfoMap and analysisJobInfoMap. #31940 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31967:
URL: https://github.com/apache/doris/pull/31967#issuecomment-1983628327
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Add synchronize for modify analysisTaskInfoMap and analysisJobInfoMap. #31940 [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #31967:
URL: https://github.com/apache/doris/pull/31967#issuecomment-1983628791
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Add synchronize for modify analysisTaskInfoMap and analysisJobInfoMap. #31940 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31967:
URL: https://github.com/apache/doris/pull/31967#issuecomment-1983728382
<details>
<summary>ClickBench: <b>Total hot run time: 30.25 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a3e1f441e79b9089f4f553c92dea902e32a3b739, data reload: false
query1 0.02 0.02 0.02
query2 0.07 0.02 0.02
query3 0.25 0.05 0.04
query4 1.80 0.06 0.06
query5 0.53 0.53 0.51
query6 1.25 0.62 0.67
query7 0.01 0.01 0.01
query8 0.03 0.02 0.02
query9 0.52 0.48 0.48
query10 0.54 0.53 0.53
query11 0.12 0.08 0.09
query12 0.12 0.09 0.09
query13 0.61 0.62 0.62
query14 0.78 0.79 0.78
query15 0.78 0.75 0.75
query16 0.36 0.39 0.36
query17 1.03 1.03 1.03
query18 0.21 0.28 0.21
query19 1.91 1.84 1.80
query20 0.01 0.01 0.01
query21 15.48 0.61 0.55
query22 2.02 2.28 1.33
query23 17.08 0.95 0.96
query24 6.65 1.36 1.04
query25 0.42 0.09 0.05
query26 0.75 0.16 0.14
query27 0.05 0.05 0.03
query28 6.21 0.84 0.71
query29 12.63 2.12 2.36
query30 0.61 0.50 0.52
query31 2.81 0.39 0.38
query32 3.40 0.49 0.49
query33 3.13 3.03 3.09
query34 15.26 4.78 4.78
query35 4.87 4.84 4.85
query36 1.04 1.01 1.01
query37 0.06 0.05 0.04
query38 0.04 0.02 0.02
query39 0.02 0.02 0.01
query40 0.15 0.14 0.14
query41 0.07 0.02 0.02
query42 0.02 0.01 0.01
query43 0.03 0.02 0.02
Total cold run time: 103.75 s
Total hot run time: 30.25 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Add synchronize for modify analysisTaskInfoMap and analysisJobInfoMap. #31940 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31967:
URL: https://github.com/apache/doris/pull/31967#issuecomment-1983735633
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit a3e1f441e79b9089f4f553c92dea902e32a3b739 with default session variables
Stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 20.9 seconds inserted 10000000 Rows, about 478K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org