You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "TangSiyang2001 (via GitHub)" <gi...@apache.org> on 2024/10/12 09:25:41 UTC
[PR] [fix](delete) Fix potential delete job stuck util timeout if exception happend in FE DeleteJob execution (#41672) [doris]
TangSiyang2001 opened a new pull request, #41763:
URL: https://github.com/apache/doris/pull/41763
## Proposed changes
pick: #41672
Fail task should also count down for the count down latch to prevent job stuck.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](delete) Fix potential delete job stuck util timeout if exception happend in FE DeleteJob execution (#41672) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41763:
URL: https://github.com/apache/doris/pull/41763#issuecomment-2408509879
<details>
<summary>ClickBench: <b>Total hot run time: 31.12 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0bf9647047ae8bb8962ae8470c964492a04aab3f, data reload: false
query1 0.02 0.03 0.02
query2 0.07 0.03 0.03
query3 0.25 0.04 0.04
query4 1.80 0.07 0.07
query5 0.54 0.53 0.52
query6 1.28 0.61 0.63
query7 0.01 0.01 0.01
query8 0.04 0.03 0.02
query9 0.51 0.49 0.47
query10 0.55 0.52 0.52
query11 0.12 0.09 0.09
query12 0.12 0.09 0.08
query13 0.62 0.60 0.62
query14 0.80 0.77 0.81
query15 0.77 0.76 0.75
query16 0.37 0.39 0.36
query17 1.03 1.02 1.03
query18 0.23 0.26 0.24
query19 1.91 1.77 1.84
query20 0.02 0.01 0.01
query21 15.48 0.56 0.56
query22 2.07 2.41 1.86
query23 17.33 0.98 0.97
query24 4.56 1.64 1.14
query25 0.39 0.09 0.04
query26 0.53 0.16 0.15
query27 0.04 0.04 0.03
query28 8.19 0.74 0.71
query29 12.76 2.31 2.31
query30 0.62 0.50 0.53
query31 2.83 0.39 0.37
query32 3.38 0.50 0.49
query33 3.11 3.08 3.09
query34 15.25 4.82 4.80
query35 4.86 4.85 4.84
query36 1.04 1.02 1.02
query37 0.06 0.04 0.05
query38 0.03 0.02 0.02
query39 0.02 0.01 0.02
query40 0.15 0.15 0.14
query41 0.06 0.02 0.01
query42 0.02 0.01 0.01
query43 0.03 0.02 0.02
Total cold run time: 103.87 s
Total hot run time: 31.12 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](delete) Fix potential delete job stuck util timeout if exception happend in FE DeleteJob execution (#41672) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41763:
URL: https://github.com/apache/doris/pull/41763#issuecomment-2408504455
<details>
<summary>TPC-H: <b>Total hot run time: 48870 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0bf9647047ae8bb8962ae8470c964492a04aab3f, data reload: false
------ Round 1 ----------------------------------
q1 18130 4352 4373 4352
q2 2080 153 148 148
q3 10367 1911 1932 1911
q4 10317 1229 1300 1229
q5 8471 3896 3906 3896
q6 231 121 122 121
q7 2056 1623 1576 1576
q8 9273 2726 2703 2703
q9 10270 9754 9762 9754
q10 8645 3507 3472 3472
q11 424 243 253 243
q12 459 295 300 295
q13 18367 3961 4020 3961
q14 348 329 322 322
q15 510 475 456 456
q16 551 461 450 450
q17 1128 977 959 959
q18 7206 6787 6873 6787
q19 1688 1589 1543 1543
q20 539 318 286 286
q21 4368 4071 4010 4010
q22 505 396 406 396
Total cold run time: 115933 ms
Total hot run time: 48870 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4333 4288 4259 4259
q2 327 223 219 219
q3 4191 4166 4136 4136
q4 2759 2735 2742 2735
q5 7168 7089 7106 7089
q6 238 118 116 116
q7 3228 2892 2838 2838
q8 4335 4440 4461 4440
q9 13689 13581 13565 13565
q10 4328 4292 4232 4232
q11 739 709 686 686
q12 1026 864 865 864
q13 7041 3719 3750 3719
q14 448 421 419 419
q15 488 459 466 459
q16 606 583 599 583
q17 3806 3907 3825 3825
q18 8818 8710 8839 8710
q19 1721 1690 1639 1639
q20 2396 2114 2101 2101
q21 8492 8439 8428 8428
q22 1060 927 913 913
Total cold run time: 81237 ms
Total hot run time: 75975 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](delete) Fix potential delete job stuck util timeout if exception happend in FE DeleteJob execution (#41672) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41763:
URL: https://github.com/apache/doris/pull/41763#issuecomment-2408508330
<details>
<summary>TPC-DS: <b>Total hot run time: 211695 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0bf9647047ae8bb8962ae8470c964492a04aab3f, data reload: false
query1 929 392 409 392
query2 6552 2253 2261 2253
query3 6922 207 203 203
query4 23121 21256 21151 21151
query5 19993 6517 6573 6517
query6 282 221 239 221
query7 4327 303 312 303
query8 249 234 239 234
query9 3113 2704 2613 2613
query10 458 323 336 323
query11 16192 15111 15434 15111
query12 131 75 73 73
query13 1034 446 464 446
query14 18428 13427 13513 13427
query15 378 235 255 235
query16 6707 295 267 267
query17 1935 976 958 958
query18 1467 354 352 352
query19 513 170 179 170
query20 120 115 120 115
query21 1131 125 145 125
query22 5759 5102 4831 4831
query23 34469 33486 33429 33429
query24 6904 6369 6325 6325
query25 504 412 424 412
query26 1087 159 160 159
query27 2261 291 288 288
query28 6086 2285 2234 2234
query29 2843 2675 2690 2675
query30 243 167 168 167
query31 948 735 751 735
query32 73 59 56 56
query33 452 257 258 257
query34 855 485 472 472
query35 1157 901 924 901
query36 1497 1169 1297 1169
query37 94 58 58 58
query38 3056 2886 2850 2850
query39 1373 1318 1330 1318
query40 310 96 94 94
query41 39 38 37 37
query42 85 90 82 82
query43 560 534 634 534
query44 1179 733 727 727
query45 242 227 229 227
query46 1225 939 973 939
query47 1926 1915 1872 1872
query48 495 426 412 412
query49 636 366 381 366
query50 855 624 576 576
query51 4743 4667 4682 4667
query52 88 75 81 75
query53 238 185 187 185
query54 2675 2444 2452 2444
query55 93 86 89 86
query56 233 194 200 194
query57 1261 1155 1095 1095
query58 218 212 205 205
query59 3518 3471 3204 3204
query60 215 210 215 210
query61 97 94 104 94
query62 815 451 471 451
query63 194 172 173 172
query64 3355 1576 1467 1467
query65 3590 3528 3540 3528
query66 812 414 436 414
query67 15396 16313 15022 15022
query68 9496 653 675 653
query69 497 272 259 259
query70 1764 1691 1324 1324
query71 419 312 309 309
query72 6813 4778 4802 4778
query73 763 332 320 320
query74 6333 5783 5795 5783
query75 5016 3685 3690 3685
query76 5117 1118 1147 1118
query77 845 251 248 248
query78 12509 11483 11717 11483
query79 8092 625 638 625
query80 1797 385 377 377
query81 489 242 240 240
query82 1659 92 96 92
query83 169 136 134 134
query84 258 70 69 69
query85 910 323 314 314
query86 335 292 296 292
query87 3203 3004 3034 3004
query88 4786 2312 2314 2312
query89 512 295 306 295
query90 2092 207 213 207
query91 155 127 131 127
query92 55 49 51 49
query93 6895 540 541 540
query94 779 211 212 211
query95 1985 1967 1942 1942
query96 644 328 321 321
query97 6381 6335 6321 6321
query98 228 205 204 204
query99 3028 855 887 855
Total cold run time: 322760 ms
Total hot run time: 211695 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](delete) Fix potential delete job stuck util timeout if exception happend in FE DeleteJob execution (#41672) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41763:
URL: https://github.com/apache/doris/pull/41763#issuecomment-2408510729
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 0bf9647047ae8bb8962ae8470c964492a04aab3f with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 57 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.4 seconds inserted 10000000 Rows, about 467K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](delete) Fix potential delete job stuck util timeout if exception happend in FE DeleteJob execution (#41672) [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #41763:
URL: https://github.com/apache/doris/pull/41763
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](delete) Fix potential delete job stuck util timeout if exception happend in FE DeleteJob execution (#41672) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41763:
URL: https://github.com/apache/doris/pull/41763#issuecomment-2408473851
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](delete) Fix potential delete job stuck util timeout if exception happend in FE DeleteJob execution (#41672) [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #41763:
URL: https://github.com/apache/doris/pull/41763#issuecomment-2408473863
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org