You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "eldenmoon (via GitHub)" <gi...@apache.org> on 2024/07/17 03:03:49 UTC
[PR] [Fix](Variant) fix potential heap use after free when concurrently fl… [doris]
eldenmoon opened a new pull request, #37976:
URL: https://github.com/apache/doris/pull/37976
…ush segments on one tablet
TabletSchema is shallow copied, should do deep copy for _flush_schema
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Variant) fix potential heap use after free when concurrently flush segments on one tablet [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37976:
URL: https://github.com/apache/doris/pull/37976#issuecomment-2232310356
<details>
<summary>TPC-H: <b>Total hot run time: 39693 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dbb14491234d3b21dff97cb52ce15ff1838ffc42, data reload: false
------ Round 1 ----------------------------------
q1 17619 4305 4297 4297
q2 2011 185 182 182
q3 10461 1173 1098 1098
q4 10191 761 809 761
q5 7639 2694 2777 2694
q6 220 134 133 133
q7 945 601 595 595
q8 9227 2026 2058 2026
q9 8890 6521 6523 6521
q10 8884 3766 3790 3766
q11 436 231 233 231
q12 417 226 224 224
q13 17759 2944 2955 2944
q14 282 242 245 242
q15 520 483 488 483
q16 499 376 383 376
q17 943 671 609 609
q18 8075 7458 7459 7458
q19 4687 1412 1354 1354
q20 682 320 328 320
q21 4838 3111 3143 3111
q22 330 268 282 268
Total cold run time: 115555 ms
Total hot run time: 39693 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4393 4235 4204 4204
q2 373 275 260 260
q3 2986 2741 2884 2741
q4 1970 1705 1750 1705
q5 5659 5556 5449 5449
q6 230 130 131 130
q7 2188 1844 1856 1844
q8 3259 3380 3379 3379
q9 8713 8786 8874 8786
q10 4078 3929 3699 3699
q11 579 494 511 494
q12 827 652 609 609
q13 17169 3202 3188 3188
q14 307 289 285 285
q15 527 481 496 481
q16 500 437 435 435
q17 1806 1554 1504 1504
q18 8134 7965 7812 7812
q19 1757 1508 1418 1418
q20 2130 1876 1832 1832
q21 4946 4987 4657 4657
q22 588 504 497 497
Total cold run time: 73119 ms
Total hot run time: 55409 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Variant) fix potential heap use after free when concurrently flush segments on one tablet [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37976:
URL: https://github.com/apache/doris/pull/37976#issuecomment-2232337374
<details>
<summary>ClickBench: <b>Total hot run time: 30.95 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dbb14491234d3b21dff97cb52ce15ff1838ffc42, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.03
query3 0.22 0.05 0.05
query4 1.68 0.08 0.09
query5 0.49 0.51 0.48
query6 1.15 0.73 0.73
query7 0.02 0.01 0.01
query8 0.05 0.04 0.05
query9 0.55 0.49 0.50
query10 0.55 0.56 0.55
query11 0.16 0.11 0.11
query12 0.15 0.12 0.12
query13 0.60 0.58 0.58
query14 0.75 0.78 0.78
query15 0.85 0.82 0.80
query16 0.36 0.35 0.36
query17 0.94 0.96 0.96
query18 0.23 0.21 0.22
query19 1.86 1.67 1.74
query20 0.01 0.01 0.01
query21 15.39 0.73 0.65
query22 3.96 6.74 2.20
query23 18.22 1.44 1.34
query24 2.12 0.23 0.23
query25 0.16 0.08 0.08
query26 0.30 0.21 0.21
query27 0.45 0.23 0.23
query28 13.34 1.00 1.00
query29 12.64 3.37 3.36
query30 0.27 0.07 0.06
query31 2.86 0.41 0.39
query32 3.26 0.48 0.47
query33 2.87 2.88 2.90
query34 17.16 4.35 4.31
query35 4.47 4.43 4.42
query36 0.66 0.49 0.47
query37 0.19 0.16 0.16
query38 0.16 0.14 0.14
query39 0.04 0.04 0.04
query40 0.14 0.11 0.13
query41 0.10 0.04 0.05
query42 0.06 0.04 0.05
query43 0.05 0.04 0.04
Total cold run time: 109.61 s
Total hot run time: 30.95 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Variant) fix potential heap use after free when concurrently fl… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37976:
URL: https://github.com/apache/doris/pull/37976#issuecomment-2232277080
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Variant) fix potential heap use after free when concurrently flush segments on one tablet [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37976:
URL: https://github.com/apache/doris/pull/37976#issuecomment-2232321058
<details>
<summary>TPC-DS: <b>Total hot run time: 173205 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dbb14491234d3b21dff97cb52ce15ff1838ffc42, data reload: false
query1 891 366 360 360
query2 6434 1902 1740 1740
query3 6637 209 220 209
query4 25684 17461 17442 17442
query5 3660 476 479 476
query6 262 178 161 161
query7 4588 291 287 287
query8 245 194 186 186
query9 8636 2401 2411 2401
query10 433 282 270 270
query11 10904 10011 10082 10011
query12 117 86 79 79
query13 1647 359 365 359
query14 10257 7807 7546 7546
query15 218 167 166 166
query16 7685 317 305 305
query17 1826 558 518 518
query18 1909 278 270 270
query19 213 143 160 143
query20 91 83 81 81
query21 214 125 122 122
query22 4287 4025 4002 4002
query23 34438 33753 33634 33634
query24 10636 2913 2906 2906
query25 578 383 376 376
query26 701 153 149 149
query27 2248 279 275 275
query28 6118 2069 2070 2069
query29 891 641 611 611
query30 261 161 160 160
query31 977 769 777 769
query32 96 54 54 54
query33 651 301 298 298
query34 912 486 498 486
query35 693 581 579 579
query36 1124 983 987 983
query37 142 85 82 82
query38 2968 2916 2815 2815
query39 918 869 821 821
query40 201 114 117 114
query41 45 43 44 43
query42 118 98 100 98
query43 498 467 469 467
query44 1091 761 722 722
query45 193 162 161 161
query46 1182 715 750 715
query47 1857 1757 1771 1757
query48 356 284 296 284
query49 838 402 409 402
query50 770 390 383 383
query51 6960 6804 6791 6791
query52 102 93 89 89
query53 362 283 279 279
query54 860 447 436 436
query55 74 71 73 71
query56 280 261 266 261
query57 1136 1054 1037 1037
query58 242 248 273 248
query59 2784 2580 2548 2548
query60 324 276 273 273
query61 95 125 98 98
query62 770 671 616 616
query63 322 288 286 286
query64 9109 2184 7419 2184
query65 3127 3057 3097 3057
query66 744 318 325 318
query67 15461 14952 14773 14773
query68 4480 541 540 540
query69 487 343 317 317
query70 1210 1076 1078 1076
query71 373 281 272 272
query72 7124 5662 5139 5139
query73 738 325 320 320
query74 6029 5759 5714 5714
query75 3391 2709 2688 2688
query76 2136 928 953 928
query77 417 297 301 297
query78 9882 9046 9030 9030
query79 2706 529 523 523
query80 2348 473 478 473
query81 617 224 217 217
query82 886 138 130 130
query83 287 173 172 172
query84 275 89 87 87
query85 1936 333 306 306
query86 490 306 311 306
query87 3338 3129 3150 3129
query88 4432 2363 2391 2363
query89 491 386 388 386
query90 1798 202 196 196
query91 206 99 100 99
query92 59 50 50 50
query93 4021 518 505 505
query94 1172 212 221 212
query95 407 323 323 323
query96 609 282 269 269
query97 3186 3013 2991 2991
query98 213 203 201 201
query99 1565 1282 1234 1234
Total cold run time: 278400 ms
Total hot run time: 173205 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Variant) fix potential heap use after free when concurrently flush segments on one tablet [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37976:
URL: https://github.com/apache/doris/pull/37976#issuecomment-2233054079
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Variant) fix potential heap use after free when concurrently flush segments on one tablet [doris]
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon merged PR #37976:
URL: https://github.com/apache/doris/pull/37976
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Variant) fix potential heap use after free when concurrently flush segments on one tablet [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37976:
URL: https://github.com/apache/doris/pull/37976#issuecomment-2232282562
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Variant) fix potential heap use after free when concurrently flush segments on one tablet [doris]
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon commented on PR #37976:
URL: https://github.com/apache/doris/pull/37976#issuecomment-2232277858
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Variant) fix potential heap use after free when concurrently flush segments on one tablet [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37976:
URL: https://github.com/apache/doris/pull/37976#issuecomment-2233054136
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org