You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Mryange (via GitHub)" <gi...@apache.org> on 2024/07/05 10:02:36 UTC
[PR] [fix](function) fix core when input not null array in foreach function [doris]
Mryange opened a new pull request, #37349:
URL: https://github.com/apache/doris/pull/37349
## Proposed changes
error code
```C++
return creator_without_type::create<AggregateFunctionForEach>(transform_arguments, true,
nested_function);
```
"transform_arguments is an internal type of array. All internal types of the array are null, so an array that is not null was mistakenly treated as a null array."
core
```
INFO: java_cmd /mnt/disk2/yanxuecheng/java17/bin/java
INFO: jdk_version 17
start BE in local mode
F20240705 15:45:48.466758 3587508 assert_cast.h:45] Bad cast from type:doris::vectorized::ColumnArray* to doris::vectorized::ColumnNullable const*
*** Check failure stack trace: ***
@ 0x5612b1cf86a6 google::LogMessage::SendToLog()
@ 0x5612b1cf50f0 google::LogMessage::Flush()
@ 0x5612b1cf8ee9 google::LogMessageFatal::~LogMessageFatal()
@ 0x5612a3373e74 assert_cast<>()
@ 0x5612a778e60d doris::vectorized::AggregateFunctionNullUnaryInline<>::add_batch_single_place()
@ 0x5612b0d0dba1 doris::vectorized::AggFnEvaluator::execute_single_add()
@ 0x5612b0c62b52 doris::pipeline::AggSinkLocalState::_execute_without_key()
@ 0x5612b0d0a90c doris::pipeline::AggSinkLocalState::Executor<>::execute()
@ 0x5612b0c653df doris::pipeline::AggSinkOperatorX::sink()
@ 0x5612b1bf27c8 doris::pipeline::PipelineTask::execute()::$_1::operator()()
@ 0x5612b1bf15c6 doris::pipeline::PipelineTask::execute()
@ 0x5612b1c0e8e7 doris::pipeline::TaskScheduler::_do_work()
@ 0x5612b1c0f72c doris::pipeline::TaskScheduler::start()::$_0::operator()()
@ 0x5612b1c0f705 std::__invoke_impl<>()
@ 0x5612b1c0f6b5 _ZSt10__invoke_rIvRZN5doris8pipeline13TaskScheduler5startEvE3$_0JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EES6_E4typeEOS7_DpOS8_
@ 0x5612b1c0f5bd std::_Function_handler<>::_M_invoke()
@ 0x5612a1400905 std::function<>::operator()()
@ 0x5612a2ad6319 doris::FunctionRunnable::run()
@ 0x5612a2acf9b6 doris::ThreadPool::dispatch_thread()
@ 0x5612a2addf09 std::__invoke_impl<>()
@ 0x5612a2adde4d std::__invoke<>()
@ 0x5612a2adde1d _ZNSt5_BindIFMN5doris10ThreadPoolEFvvEPS1_EE6__callIvJEJLm0EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x5612a2adddd6 std::_Bind<>::operator()<>()
@ 0x5612a2addda5 std::__invoke_impl<>()
@ 0x5612a2addd65 _ZSt10__invoke_rIvRSt5_BindIFMN5doris10ThreadPoolEFvvEPS2_EEJEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESA_E4typeEOSB_DpOSC_
@ 0x5612a2addbbd std::_Function_handler<>::_M_invoke()
@ 0x5612a1400905 std::function<>::operator()()
@ 0x5612a2abecff doris::Thread::supervise_thread()
@ 0x7f6d1c7841ca start_thread
@ 0x7f6d1d173e73 __GI___clone
@ (nil) (unknown)
```
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function) fix core when input not null array in foreach function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37349:
URL: https://github.com/apache/doris/pull/37349#issuecomment-2210587146
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function) fix core when input not null array in foreach function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37349:
URL: https://github.com/apache/doris/pull/37349#issuecomment-2210637114
<details>
<summary>TPC-H: <b>Total hot run time: 40690 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 083604df5168f586dd6a5f37c7df475d9c03fba7, data reload: false
------ Round 1 ----------------------------------
q1 17633 4523 4310 4310
q2 2017 195 196 195
q3 10467 1191 1102 1102
q4 10186 737 833 737
q5 7469 2652 2699 2652
q6 221 138 136 136
q7 954 601 597 597
q8 9431 2061 2087 2061
q9 8801 6662 6631 6631
q10 9038 3751 3725 3725
q11 449 236 242 236
q12 491 225 223 223
q13 19313 3011 3018 3011
q14 264 231 236 231
q15 556 503 485 485
q16 511 398 375 375
q17 977 646 678 646
q18 8186 7603 7556 7556
q19 6137 1497 1343 1343
q20 653 334 314 314
q21 4971 3792 3900 3792
q22 382 340 332 332
Total cold run time: 119107 ms
Total hot run time: 40690 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4440 4382 4374 4374
q2 374 298 284 284
q3 3290 3119 3083 3083
q4 2055 1765 1775 1765
q5 5709 5770 5629 5629
q6 242 137 134 134
q7 2444 1929 1834 1834
q8 3333 3508 3473 3473
q9 8780 8879 8761 8761
q10 4119 3839 3849 3839
q11 583 496 504 496
q12 826 629 616 616
q13 15986 3150 3202 3150
q14 321 303 259 259
q15 560 486 507 486
q16 524 454 420 420
q17 1835 1539 1522 1522
q18 8349 7966 7741 7741
q19 1784 1658 1547 1547
q20 3238 1908 1868 1868
q21 10945 4842 4816 4816
q22 638 558 563 558
Total cold run time: 80375 ms
Total hot run time: 56655 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function) fix core when input not null array in foreach function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37349:
URL: https://github.com/apache/doris/pull/37349#issuecomment-2210652342
<details>
<summary>TPC-DS: <b>Total hot run time: 173309 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 083604df5168f586dd6a5f37c7df475d9c03fba7, data reload: false
query1 917 374 390 374
query2 6465 2544 2344 2344
query3 6636 207 219 207
query4 22833 17540 17314 17314
query5 3579 480 524 480
query6 258 169 166 166
query7 4591 309 293 293
query8 290 302 284 284
query9 8718 2467 2415 2415
query10 573 296 280 280
query11 10656 10076 10130 10076
query12 114 84 81 81
query13 1639 362 364 362
query14 10152 6854 7501 6854
query15 244 184 185 184
query16 7746 294 300 294
query17 1574 552 551 551
query18 1949 283 274 274
query19 182 151 157 151
query20 90 81 79 79
query21 213 132 125 125
query22 4410 4196 3997 3997
query23 33837 33844 33527 33527
query24 10732 2852 2867 2852
query25 573 394 379 379
query26 714 161 167 161
query27 2210 318 330 318
query28 5965 2169 2146 2146
query29 872 663 633 633
query30 258 157 159 157
query31 997 766 807 766
query32 93 52 53 52
query33 658 290 296 290
query34 892 503 500 500
query35 728 628 646 628
query36 1129 988 997 988
query37 140 80 85 80
query38 3003 2861 2785 2785
query39 930 843 860 843
query40 205 130 133 130
query41 54 50 52 50
query42 120 100 107 100
query43 612 582 547 547
query44 1126 753 734 734
query45 191 165 163 163
query46 1077 749 712 712
query47 1880 1772 1798 1772
query48 385 295 306 295
query49 861 431 437 431
query50 773 387 396 387
query51 6843 6833 6719 6719
query52 105 93 97 93
query53 362 298 291 291
query54 886 469 452 452
query55 75 75 75 75
query56 316 291 298 291
query57 1112 1075 1092 1075
query58 259 256 266 256
query59 3434 3266 3267 3266
query60 314 294 298 294
query61 116 115 114 114
query62 589 457 459 457
query63 330 308 296 296
query64 9261 2327 1626 1626
query65 3160 3108 3097 3097
query66 734 326 337 326
query67 15867 14995 15120 14995
query68 5736 546 540 540
query69 707 472 373 373
query70 1206 1130 1157 1130
query71 475 283 290 283
query72 9187 5181 5412 5181
query73 787 323 319 319
query74 5997 5472 5513 5472
query75 4212 2637 2678 2637
query76 3381 1045 951 951
query77 709 322 321 321
query78 13156 9467 8963 8963
query79 6927 541 546 541
query80 1297 491 486 486
query81 595 230 226 226
query82 719 115 106 106
query83 343 173 171 171
query84 275 85 93 85
query85 708 310 303 303
query86 455 308 296 296
query87 3336 3100 3126 3100
query88 5088 2390 2355 2355
query89 506 403 402 402
query90 1964 191 192 191
query91 136 101 107 101
query92 59 49 50 49
query93 4815 524 522 522
query94 1316 210 209 209
query95 419 314 325 314
query96 621 274 266 266
query97 3215 3057 3067 3057
query98 226 204 192 192
query99 1129 833 866 833
Total cold run time: 288309 ms
Total hot run time: 173309 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function) fix core when input not null array in foreach function [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #37349:
URL: https://github.com/apache/doris/pull/37349#issuecomment-2210587763
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function) fix core when input not null array in foreach function [doris]
Posted by "zhangstar333 (via GitHub)" <gi...@apache.org>.
zhangstar333 merged PR #37349:
URL: https://github.com/apache/doris/pull/37349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function) fix core when input not null array in foreach function [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37349:
URL: https://github.com/apache/doris/pull/37349#issuecomment-2210596624
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function) fix core when input not null array in foreach function [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37349:
URL: https://github.com/apache/doris/pull/37349#issuecomment-2212856722
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function) fix core when input not null array in foreach function [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37349:
URL: https://github.com/apache/doris/pull/37349#issuecomment-2210659810
<details>
<summary>ClickBench: <b>Total hot run time: 31.08 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 083604df5168f586dd6a5f37c7df475d9c03fba7, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.22 0.05 0.06
query4 1.66 0.09 0.09
query5 0.51 0.51 0.50
query6 1.12 0.72 0.72
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.55 0.50 0.50
query10 0.54 0.55 0.56
query11 0.16 0.12 0.11
query12 0.14 0.12 0.12
query13 0.61 0.59 0.59
query14 0.78 0.77 0.78
query15 0.85 0.81 0.82
query16 0.36 0.37 0.37
query17 0.99 1.03 1.00
query18 0.22 0.27 0.25
query19 1.80 1.66 1.76
query20 0.01 0.00 0.01
query21 15.39 0.76 0.67
query22 4.62 6.38 2.26
query23 18.28 1.42 1.27
query24 2.15 0.23 0.22
query25 0.16 0.08 0.08
query26 0.30 0.22 0.21
query27 0.46 0.23 0.24
query28 13.67 1.01 1.02
query29 12.64 3.31 3.33
query30 0.26 0.07 0.05
query31 2.86 0.38 0.40
query32 3.30 0.48 0.47
query33 2.84 2.91 2.89
query34 17.15 4.35 4.36
query35 4.44 4.43 4.46
query36 0.64 0.48 0.48
query37 0.18 0.15 0.16
query38 0.16 0.14 0.14
query39 0.05 0.03 0.03
query40 0.15 0.12 0.11
query41 0.09 0.06 0.05
query42 0.05 0.04 0.05
query43 0.04 0.04 0.04
Total cold run time: 110.59 s
Total hot run time: 31.08 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function) fix core when input not null array in foreach function [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37349:
URL: https://github.com/apache/doris/pull/37349#issuecomment-2212856758
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org