You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "HappenLee (via GitHub)" <gi...@apache.org> on 2023/05/11 09:40:08 UTC
[GitHub] [doris] HappenLee opened a new pull request, #19545: [pipeline](bug) DCHECK may failed in pip sender queue
HappenLee opened a new pull request, #19545:
URL: https://github.com/apache/doris/pull/19545
# Proposed changes
```
F0505 22:46:41.875746 497986 vdata_stream_recvr.cpp:87] Check failed: _num_remaining_senders == 0 (26 vs. 0)
Check failure stack trace: ***
@ 0x55c8198dc95d google::LogMessage::Fail()
@ 0x55c8198dee99 google::LogMessage::SendToLog()
@ 0x55c8198dc4c6 google::LogMessage::Flush()
@ 0x55c8198df509 google::LogMessageFatal::~LogMessageFatal()
@ 0x55c81814cccb doris::vectorized::VDataStreamRecvr::SenderQueue::_inner_get_batch()
@ 0x55c81815fa74 doris::vectorized::VDataStreamRecvr::PipSenderQueue::get_batch()
@ 0x55c818155072 doris::vectorized::VDataStreamRecvr::get_next()
@ 0x55c808360620 doris::vectorized::VExchangeNode::get_next()
@ 0x55c80071a114 doris::ExecNode::pull()
@ 0x55c818e68e8d std::__invoke_impl<>()
@ 0x55c818e68c30 std::__invoke<>()
@ 0x55c818e68b67 ZNSt5_BindIFMN5doris8ExecNodeEFNS0_6StatusEPNS0_12RuntimeStateEPNS0_10vectorized5BlockEPbEPNS5_13VExchangeNodeESt12_PlaceholderILi1EESD_ILi2EESD_ILi3EEEE6_callIS2_JOS4_OS7_OS8_EJLm0ELm1ELm2ELm3EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x55c818e688f8 std::_Bind<>::operator()<>()
@ 0x55c818e687a8 std::__invoke_impl<>()
@ 0x55c818e68718 ZSt10invoke_rIN5doris6StatusERSt5_BindIFMNS0_8ExecNodeEFS1_PNS0_12RuntimeStateEPNS0_10vectorized5BlockEPbEPNS6_13VExchangeNodeESt12_PlaceholderILi1EESE_ILi2EESE_ILi3EEEEJS5_S8_S9_EENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESM_E4typeEOSN_DpOSO
@ 0x55c818e68368 std::_Function_handler<>::_M_invoke()
@ 0x55c800719fff std::function<>::operator()()
@ 0x55c80070e18a doris::ExecNode::get_next_after_projects()
@ 0x55c818e655f3 doris::pipeline::SourceOperator<>::get_block()
@ 0x55c818e05c3f doris::pipeline::PipelineTask::execute()
@ 0x55c818e2429a doris::pipeline::TaskScheduler::_do_work()
```
## Problem summary
Describe your changes.
## Checklist(Required)
* [ ] Does it affect the original behavior
* [ ] Has unit tests been added
* [ ] Has document been added or modified
* [ ] Does it need to update dependencies
* [ ] Is this PR support rollback (If NO, please explain WHY)
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19545:
URL: https://github.com/apache/doris/pull/19545#issuecomment-1543802474
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19545:
URL: https://github.com/apache/doris/pull/19545#issuecomment-1544976008
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] HappenLee commented on pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #19545:
URL: https://github.com/apache/doris/pull/19545#issuecomment-1543807138
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] HappenLee commented on pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #19545:
URL: https://github.com/apache/doris/pull/19545#issuecomment-1545233928
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19545:
URL: https://github.com/apache/doris/pull/19545#issuecomment-1545555593
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] BiteTheDDDDt merged pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt merged PR #19545:
URL: https://github.com/apache/doris/pull/19545
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] HappenLee commented on pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #19545:
URL: https://github.com/apache/doris/pull/19545#issuecomment-1544984070
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19545:
URL: https://github.com/apache/doris/pull/19545#issuecomment-1543810522
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #19545:
URL: https://github.com/apache/doris/pull/19545#issuecomment-1543719679
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.16 seconds
stream load tsv: 420 seconds loaded 74807831229 Bytes, about 169 MB/s
stream load json: 23 seconds loaded 2358488459 Bytes, about 97 MB/s
stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230511101128_clickbench_pr_142299.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19545:
URL: https://github.com/apache/doris/pull/19545#issuecomment-1545236054
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19545:
URL: https://github.com/apache/doris/pull/19545#issuecomment-1543678957
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] HappenLee commented on pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #19545:
URL: https://github.com/apache/doris/pull/19545#issuecomment-1543682412
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] zclllyybb commented on a diff in pull request #19545: [pipeline](bug) DCHECK may failed in pip sender queue
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on code in PR #19545:
URL: https://github.com/apache/doris/pull/19545#discussion_r1190982213
##########
be/src/vec/runtime/vdata_stream_recvr.h:
##########
@@ -219,23 +218,21 @@ class VDataStreamRecvr::PipSenderQueue : public SenderQueue {
PipSenderQueue(VDataStreamRecvr* parent_recvr, int num_senders, RuntimeProfile* profile)
: SenderQueue(parent_recvr, num_senders, profile) {}
- bool should_wait() override {
- return !_is_cancelled && _block_queue_empty && _num_remaining_senders > 0;
- }
-
- void _update_block_queue_empty() override { _block_queue_empty = _block_queue.empty(); }
-
Status get_batch(Block* block, bool* eos) override {
+ std::lock_guard<std::mutex> l(_lock); // protect _block_queue
Review Comment:
this will make deadlock since we would lock `_lock` in `should_wait()`. maybe rewrite `should_wait` to `should_wait_without_lock`?
##########
be/src/vec/runtime/vdata_stream_recvr.h:
##########
@@ -219,23 +218,21 @@ class VDataStreamRecvr::PipSenderQueue : public SenderQueue {
PipSenderQueue(VDataStreamRecvr* parent_recvr, int num_senders, RuntimeProfile* profile)
: SenderQueue(parent_recvr, num_senders, profile) {}
- bool should_wait() override {
- return !_is_cancelled && _block_queue_empty && _num_remaining_senders > 0;
- }
-
- void _update_block_queue_empty() override { _block_queue_empty = _block_queue.empty(); }
-
Status get_batch(Block* block, bool* eos) override {
+ std::lock_guard<std::mutex> l(_lock); // protect _block_queue
CHECK(!should_wait()) << " _is_cancelled: " << _is_cancelled
<< ", _block_queue_empty: " << _block_queue_empty
Review Comment:
should change all usage of `_block_queue_empty` to `_block_queue.empty()` since won't update it any more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org