You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yiguolei (via GitHub)" <gi...@apache.org> on 2024/03/24 04:06:10 UTC
[PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
yiguolei opened a new pull request, #32730:
URL: https://github.com/apache/doris/pull/32730
## Proposed changes
If task ctx lock failed, it means the runtime state or fragment context is deconstruted.
And it also means the operator and exec node is deconstruted, then if we want to set set finish dependency, it will core because of use after free.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32730:
URL: https://github.com/apache/doris/pull/32730#issuecomment-2016801593
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32730:
URL: https://github.com/apache/doris/pull/32730#issuecomment-2016693348
<details>
<summary>TPC-DS: <b>Total hot run time: 186023 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit add9dfcb4a472570d64a48e67ff90090c590bfed, data reload: false
query1 936 372 345 345
query2 7372 1994 2012 1994
query3 6713 204 207 204
query4 31533 21259 21560 21259
query5 4268 410 404 404
query6 280 177 180 177
query7 4628 289 285 285
query8 226 177 170 170
query9 9189 2270 2248 2248
query10 559 244 251 244
query11 17259 14367 14455 14367
query12 138 93 85 85
query13 1623 398 413 398
query14 14580 11407 10895 10895
query15 276 201 187 187
query16 8210 264 259 259
query17 1976 571 549 549
query18 2060 295 282 282
query19 359 151 153 151
query20 98 86 87 86
query21 206 132 124 124
query22 5027 4760 4790 4760
query23 33581 32659 32726 32659
query24 11060 2808 2827 2808
query25 624 385 388 385
query26 1169 155 160 155
query27 2563 351 357 351
query28 7216 1879 1838 1838
query29 888 639 632 632
query30 307 153 153 153
query31 973 747 739 739
query32 91 60 56 56
query33 780 256 250 250
query34 986 481 502 481
query35 841 614 601 601
query36 1012 895 898 895
query37 119 81 79 79
query38 3569 3446 3416 3416
query39 1451 1447 1408 1408
query40 218 119 112 112
query41 52 49 49 49
query42 105 100 94 94
query43 464 450 454 450
query44 1170 716 716 716
query45 278 265 262 262
query46 1111 712 710 710
query47 1907 1834 1881 1834
query48 437 351 360 351
query49 1111 351 342 342
query50 755 371 365 365
query51 6746 6578 6651 6578
query52 107 93 86 86
query53 340 275 276 275
query54 315 248 260 248
query55 90 76 80 76
query56 251 236 230 230
query57 1204 1163 1163 1163
query58 222 206 198 198
query59 2699 2575 2508 2508
query60 258 243 248 243
query61 93 92 93 92
query62 659 449 450 449
query63 318 278 272 272
query64 5218 4105 3936 3936
query65 3030 2995 3030 2995
query66 869 372 358 358
query67 15547 14797 14736 14736
query68 5463 514 504 504
query69 567 376 353 353
query70 1226 1128 1182 1128
query71 438 275 276 275
query72 6406 2736 2527 2527
query73 711 308 308 308
query74 7279 6702 6698 6698
query75 3723 2979 2927 2927
query76 3568 865 848 848
query77 390 250 262 250
query78 10961 10044 10238 10044
query79 7686 520 517 517
query80 1942 389 414 389
query81 572 225 221 221
query82 1535 219 204 204
query83 327 152 154 152
query84 292 88 86 86
query85 1717 369 363 363
query86 485 311 311 311
query87 3754 3532 3557 3532
query88 5265 2294 2295 2294
query89 478 371 373 371
query90 2021 182 177 177
query91 185 147 150 147
query92 62 50 57 50
query93 5884 490 472 472
query94 1212 177 175 175
query95 425 327 330 327
query96 610 276 266 266
query97 3116 2914 2880 2880
query98 244 210 201 201
query99 1228 938 919 919
Total cold run time: 310092 ms
Total hot run time: 186023 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32730:
URL: https://github.com/apache/doris/pull/32730#issuecomment-2016687032
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32730:
URL: https://github.com/apache/doris/pull/32730#issuecomment-2016689057
TeamCity be ut coverage result:
Function Coverage: 35.26% (8733/24766)
Line Coverage: 27.06% (71515/264294)
Region Coverage: 26.30% (37108/141097)
Branch Coverage: 23.20% (18979/81816)
Coverage Report: http://coverage.selectdb-in.cc/coverage/add9dfcb4a472570d64a48e67ff90090c590bfed_add9dfcb4a472570d64a48e67ff90090c590bfed/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #32730:
URL: https://github.com/apache/doris/pull/32730#issuecomment-2016686328
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32730:
URL: https://github.com/apache/doris/pull/32730#issuecomment-2016691578
<details>
<summary>TPC-H: <b>Total hot run time: 37795 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit add9dfcb4a472570d64a48e67ff90090c590bfed, data reload: false
------ Round 1 ----------------------------------
q1 17636 4301 4100 4100
q2 2109 160 169 160
q3 10569 1100 1197 1100
q4 10234 750 750 750
q5 7458 2997 2967 2967
q6 206 123 119 119
q7 1030 578 556 556
q8 9338 2001 1982 1982
q9 7253 6535 6537 6535
q10 8457 3450 3575 3450
q11 428 219 213 213
q12 432 200 187 187
q13 17791 2879 2856 2856
q14 246 199 204 199
q15 506 470 455 455
q16 491 377 368 368
q17 937 520 600 520
q18 7109 6476 6586 6476
q19 2030 1421 1476 1421
q20 534 261 249 249
q21 3570 2926 2836 2836
q22 343 296 303 296
Total cold run time: 108707 ms
Total hot run time: 37795 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4123 4063 4048 4048
q2 322 229 226 226
q3 2948 2857 2796 2796
q4 1839 1565 1557 1557
q5 5316 5327 5355 5327
q6 195 118 116 116
q7 2247 1885 1853 1853
q8 3185 3301 3261 3261
q9 8696 8695 8691 8691
q10 3788 3779 3760 3760
q11 549 446 442 442
q12 721 540 572 540
q13 16915 2848 2854 2848
q14 279 254 258 254
q15 494 459 460 459
q16 478 423 432 423
q17 1721 1486 1461 1461
q18 7436 7242 7133 7133
q19 1607 1364 1533 1364
q20 1911 1731 1736 1731
q21 4808 4671 4785 4671
q22 531 452 459 452
Total cold run time: 70109 ms
Total hot run time: 53413 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32730:
URL: https://github.com/apache/doris/pull/32730#issuecomment-2016801599
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32730:
URL: https://github.com/apache/doris/pull/32730#issuecomment-2016693756
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit add9dfcb4a472570d64a48e67ff90090c590bfed with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 20.8 seconds inserted 10000000 Rows, about 480K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #32730:
URL: https://github.com/apache/doris/pull/32730
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32730:
URL: https://github.com/apache/doris/pull/32730#issuecomment-2016685995
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](use after free) should not set finish depdency any more if task ctx lock failed [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #32730:
URL: https://github.com/apache/doris/pull/32730#issuecomment-2016686092
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org