You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org> on 2024/06/14 09:10:49 UTC
[PR] [fix](memory allocate) Fix reinitialization of BlockReader [doris]
zhiqiang-hhhh opened a new pull request, #36314:
URL: https://github.com/apache/doris/pull/36314
`ASSIGN_STATUS_IF_CATCH_EXCEPTION` will convert error code from `MEM_ALLOC_FAILED` to `MEM_LIMIT_EXCEEDED`, this will lead to problem like below:
```text
F20240614 16:07:18.529981 3400047 delete_handler.cpp:388] Check failed: !_is_inited reinitialize delete handler.
*** Check failure stack trace: ***
@ 0x561cdc597926 google::LogMessage::SendToLog()
@ 0x561cdc594370 google::LogMessage::Flush()
@ 0x561cdc598169 google::LogMessageFatal::~LogMessageFatal()
@ 0x561cacf6ba70 doris::DeleteHandler::init()
@ 0x561caf6e9298 doris::TabletReader::_init_delete_condition()
@ 0x561caf6e257c doris::TabletReader::_init_params()
@ 0x561caf6e1a09 doris::TabletReader::init()
@ 0x561cd847917f doris::vectorized::BlockReader::init()
@ 0x561cdbc466a4 doris::vectorized::NewOlapScanner::open()
@ 0x561cc3a1b623 doris::vectorized::ScannerScheduler::_scanner_scan()
@ 0x561cc3a1fae1 _ZZZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS0_14ScannerContextEES2_INS0_8ScanTaskEEENK3$_1clEvENKUlvE_clEvENKUlvE_clEv
@ 0x561cc3a1f554 _ZZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS0_14ScannerContextEES2_INS0_8ScanTaskEEENK3$_1clEvENKUlvE_clEv
@ 0x561cc3a1f3c5 _ZSt13__invoke_implIvRZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS1_14ScannerContextEES3_INS1_8ScanTaskEEENK3$_1clEvEUlvE_JEET_St14__invoke_otherOT0_DpOT1_
@ 0x561cc3a1f365 _ZSt10__invoke_rIvRZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS1_14ScannerContextEES3_INS1_8ScanTaskEEENK3$_1clEvEUlvE_JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESC_E4typeEOSD_DpOSE_
@ 0x561cc3a1f18d _ZNSt17_Function_handlerIFvvEZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS2_14ScannerContextEES4_INS2_8ScanTaskEEENK3$_1clEvEUlvE_E9_M_invokeERKSt9_Any_data
@ 0x561cac9f8023 std::function<>::operator()()
@ 0x561cc3a333b5 _ZZN5doris10vectorized23SimplifiedScanScheduler16submit_scan_taskENS0_18SimplifiedScanTaskEENKUlvE_clEv
@ 0x561cc3a33395 _ZSt13__invoke_implIvRZN5doris10vectorized23SimplifiedScanScheduler16submit_scan_taskENS1_18SimplifiedScanTaskEEUlvE_JEET_St14__invoke_otherOT0_DpOT1_
@ 0x561cc3a33335 _ZSt10__invoke_rIvRZN5doris10vectorized23SimplifiedScanScheduler16submit_scan_taskENS1_18SimplifiedScanTaskEEUlvE_JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EES7_E4typeEOS8_DpOS9_
@ 0x561cc3a3312d _ZNSt17_Function_handlerIFvvEZN5doris10vectorized23SimplifiedScanScheduler16submit_scan_taskENS2_18SimplifiedScanTaskEEUlvE_E9_M_invokeERKSt9_Any_data
@ 0x561cac9f8023 std::function<>::operator()()
@ 0x561cb095d4e9 doris::FunctionRunnable::run()
@ 0x561cb094913e doris::ThreadPool::dispatch_thread()
@ 0x561cb0970564 std::__invoke_impl<>()
@ 0x561cb097043d std::__invoke<>()
@ 0x561cb09703c5 _ZNSt5_BindIFMN5doris10ThreadPoolEFvvEPS1_EE6__callIvJEJLm0EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x561cb097026e std::_Bind<>::operator()<>()
@ 0x561cb0970185 std::__invoke_impl<>()
@ 0x561cb0970125 _ZSt10__invoke_rIvRSt5_BindIFMN5doris10ThreadPoolEFvvEPS2_EEJEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESA_E4typeEOSB_DpOSC_
@ 0x561cb096fdcd std::_Function_handler<>::_M_invoke()
@ 0x561cac9f8023 std::function<>::operator()()
@ 0x561cb091666c doris::Thread::supervise_thread()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory allocate) Fix reinitialization of TabletReader [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36314:
URL: https://github.com/apache/doris/pull/36314#issuecomment-2167927355
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory allocate) Fix reinitialization of TabletReader [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #36314:
URL: https://github.com/apache/doris/pull/36314#issuecomment-2167603726
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory allocate) Fix reinitialization of TabletReader [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #36314:
URL: https://github.com/apache/doris/pull/36314
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory allocate) Fix reinitialization of TabletReader [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36314:
URL: https://github.com/apache/doris/pull/36314#issuecomment-2167688265
<details>
<summary>TPC-DS: <b>Total hot run time: 173785 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1d078db7ff5656e784d89d2f429c4af5d444e749, data reload: false
query1 921 382 385 382
query2 6458 2359 2417 2359
query3 6645 205 204 204
query4 19787 17274 17263 17263
query5 3595 458 466 458
query6 243 152 151 151
query7 4573 299 287 287
query8 320 298 293 293
query9 8481 2403 2387 2387
query10 582 292 293 292
query11 10701 10101 9973 9973
query12 117 85 82 82
query13 1631 357 360 357
query14 9998 7674 7648 7648
query15 243 186 189 186
query16 7846 258 255 255
query17 1948 530 517 517
query18 1994 267 288 267
query19 199 155 148 148
query20 92 82 81 81
query21 211 132 126 126
query22 4530 4214 4185 4185
query23 33705 33502 33619 33502
query24 11222 3055 2944 2944
query25 611 365 377 365
query26 925 155 158 155
query27 2385 332 323 323
query28 6714 2119 2121 2119
query29 896 635 632 632
query30 226 152 151 151
query31 946 789 760 760
query32 93 58 58 58
query33 749 285 274 274
query34 1091 485 477 477
query35 724 638 645 638
query36 1120 949 964 949
query37 146 74 72 72
query38 2919 2824 2809 2809
query39 924 844 847 844
query40 220 129 131 129
query41 59 60 58 58
query42 115 106 103 103
query43 592 583 531 531
query44 1260 729 733 729
query45 208 162 173 162
query46 1086 731 730 730
query47 1868 1782 1790 1782
query48 371 299 306 299
query49 868 415 421 415
query50 765 394 399 394
query51 6788 6662 6645 6645
query52 111 90 95 90
query53 355 299 292 292
query54 911 442 447 442
query55 75 77 71 71
query56 288 267 280 267
query57 1102 1062 1047 1047
query58 255 253 248 248
query59 3255 3288 2982 2982
query60 309 283 306 283
query61 112 110 105 105
query62 615 444 446 444
query63 321 292 287 287
query64 8932 2356 1809 1809
query65 3193 3093 3119 3093
query66 783 335 327 327
query67 15489 14893 14747 14747
query68 7811 537 531 531
query69 687 466 388 388
query70 1169 1119 1025 1025
query71 523 261 267 261
query72 8923 5275 5419 5275
query73 814 322 319 319
query74 5871 5412 5432 5412
query75 5014 2655 2659 2655
query76 4303 943 885 885
query77 765 293 286 286
query78 10459 10050 9712 9712
query79 4038 514 519 514
query80 1016 464 464 464
query81 557 219 224 219
query82 782 105 105 105
query83 380 170 170 170
query84 276 88 95 88
query85 1258 269 269 269
query86 439 323 303 303
query87 3239 3037 3094 3037
query88 4643 2356 2330 2330
query89 497 383 384 383
query90 1913 196 189 189
query91 128 99 99 99
query92 65 49 49 49
query93 4852 501 500 500
query94 1184 185 193 185
query95 406 309 315 309
query96 606 271 261 261
query97 3165 3085 3050 3050
query98 223 198 194 194
query99 1205 834 851 834
Total cold run time: 284028 ms
Total hot run time: 173785 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory allocate) Fix reinitialization of TabletReader [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36314:
URL: https://github.com/apache/doris/pull/36314#issuecomment-2167697510
<details>
<summary>ClickBench: <b>Total hot run time: 29.88 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1d078db7ff5656e784d89d2f429c4af5d444e749, data reload: false
query1 0.04 0.03 0.04
query2 0.08 0.04 0.03
query3 0.24 0.05 0.04
query4 1.68 0.06 0.07
query5 0.51 0.47 0.48
query6 1.15 0.74 0.72
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.54 0.50 0.50
query10 0.53 0.55 0.54
query11 0.15 0.11 0.11
query12 0.15 0.12 0.12
query13 0.60 0.60 0.61
query14 0.78 0.78 0.81
query15 0.85 0.81 0.81
query16 0.37 0.34 0.36
query17 0.99 1.01 1.03
query18 0.21 0.24 0.27
query19 1.77 1.67 1.69
query20 0.01 0.01 0.01
query21 15.41 0.66 0.65
query22 4.67 7.98 1.31
query23 18.28 1.41 1.34
query24 2.18 0.21 0.23
query25 0.16 0.07 0.08
query26 0.25 0.18 0.17
query27 0.08 0.08 0.08
query28 13.24 1.00 0.98
query29 12.63 3.26 3.26
query30 0.25 0.07 0.06
query31 2.86 0.39 0.38
query32 3.30 0.46 0.46
query33 2.83 2.90 2.94
query34 17.19 4.39 4.40
query35 4.48 4.45 4.49
query36 0.63 0.48 0.46
query37 0.18 0.15 0.15
query38 0.15 0.15 0.14
query39 0.05 0.04 0.04
query40 0.18 0.13 0.14
query41 0.10 0.04 0.05
query42 0.05 0.05 0.04
query43 0.04 0.04 0.04
Total cold run time: 109.91 s
Total hot run time: 29.88 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory allocate) Fix reinitialization of BlockReader [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36314:
URL: https://github.com/apache/doris/pull/36314#issuecomment-2167598483
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory allocate) Fix reinitialization of TabletReader [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36314:
URL: https://github.com/apache/doris/pull/36314#issuecomment-2167669425
<details>
<summary>TPC-H: <b>Total hot run time: 39503 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1d078db7ff5656e784d89d2f429c4af5d444e749, data reload: false
------ Round 1 ----------------------------------
q1 17641 4337 4289 4289
q2 2029 194 189 189
q3 10445 1102 1189 1102
q4 10190 786 749 749
q5 7567 2619 2639 2619
q6 217 135 134 134
q7 986 595 592 592
q8 9213 2041 2069 2041
q9 8978 6452 6482 6452
q10 8944 3796 3706 3706
q11 448 242 242 242
q12 459 233 220 220
q13 17761 2990 2976 2976
q14 258 220 217 217
q15 524 477 494 477
q16 523 370 381 370
q17 958 697 673 673
q18 7989 7298 7388 7298
q19 4702 1567 1456 1456
q20 654 321 322 321
q21 4910 3106 3051 3051
q22 383 329 330 329
Total cold run time: 115779 ms
Total hot run time: 39503 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4402 4218 4206 4206
q2 377 260 257 257
q3 2979 2719 2969 2719
q4 1999 1723 1746 1723
q5 5594 5489 5517 5489
q6 230 127 129 127
q7 2165 1801 1856 1801
q8 3260 3428 3419 3419
q9 8674 8623 8782 8623
q10 4097 3924 3764 3764
q11 609 475 467 467
q12 795 635 642 635
q13 17040 3194 3144 3144
q14 301 269 285 269
q15 545 472 493 472
q16 495 420 436 420
q17 1804 1517 1480 1480
q18 7935 8070 7715 7715
q19 1809 1665 1566 1566
q20 2061 1904 1881 1881
q21 5098 4938 4760 4760
q22 625 549 515 515
Total cold run time: 72894 ms
Total hot run time: 55452 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory allocate) Fix reinitialization of TabletReader [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36314:
URL: https://github.com/apache/doris/pull/36314#issuecomment-2167927253
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory allocate) Fix reinitialization of TabletReader [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36314:
URL: https://github.com/apache/doris/pull/36314#issuecomment-2167649335
TeamCity be ut coverage result:
Function Coverage: 36.45% (8988/24661)
Line Coverage: 28.02% (73677/262936)
Region Coverage: 27.49% (38276/139213)
Branch Coverage: 24.19% (19515/80660)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1d078db7ff5656e784d89d2f429c4af5d444e749_1d078db7ff5656e784d89d2f429c4af5d444e749/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory allocate) Fix reinitialization of TabletReader [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36314:
URL: https://github.com/apache/doris/pull/36314#issuecomment-2167608491
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org