You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yiguolei (via GitHub)" <gi...@apache.org> on 2023/12/20 04:09:38 UTC
[PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
yiguolei opened a new pull request, #28698:
URL: https://github.com/apache/doris/pull/28698
## Proposed changes
1. Add a new class broadcastbufferholderqueue to manage holders
2. Using shared ptr to manage holders, not use ref and unref, it is too difficult to maintain.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei closed pull request #28698: [refactor](broadcastbuffer) using a queue to remove ref and unref codes
URL: https://github.com/apache/doris/pull/28698
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864387072
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864057971
run compile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432410931
##########
be/src/pipeline/exec/exchange_sink_buffer.h:
##########
@@ -71,25 +72,53 @@ struct AtomicWrapper {
// We use BroadcastPBlockHolder to hold a broadcasted PBlock. For broadcast shuffle, one PBlock
// will be shared between different channel, so we have to use a ref count to mark if this
// PBlock is available for next serialization.
+class BroadcastPBlockHolderQueue;
class BroadcastPBlockHolder {
+ ENABLE_FACTORY_CREATOR(BroadcastPBlockHolder);
+
+public:
+ BroadcastPBlockHolder() { _pblock = std::make_unique<PBlock>(); }
+ BroadcastPBlockHolder(std::unique_ptr<PBlock>&& pblock) { _pblock = std::move(pblock); }
+ ~BroadcastPBlockHolder();
+
+ PBlock* get_block() { return _pblock.get(); }
+
+private:
+ friend class BroadcastPBlockHolderQueue;
+ std::unique_ptr<PBlock> _pblock;
+ std::weak_ptr<BroadcastPBlockHolderQueue> _parent_creator;
+ void set_parent_creator(std::shared_ptr<BroadcastPBlockHolderQueue> parent_creator) {
+ _parent_creator = parent_creator;
+ }
+};
+
+// Use a stack inside to ensure that the PBlock is in cpu cache
+class BroadcastPBlockHolderQueue : public std::enable_shared_from_this<BroadcastPBlockHolderQueue> {
+ ENABLE_FACTORY_CREATOR(BroadcastPBlockHolderQueue);
+
public:
- BroadcastPBlockHolder() : _ref_count(0), _dep(nullptr) {}
- BroadcastPBlockHolder(pipeline::BroadcastDependency* dep) : _ref_count(0), _dep(dep) {}
- ~BroadcastPBlockHolder() noexcept = default;
+ BroadcastPBlockHolderQueue() = default;
- void ref(int delta) noexcept { _ref_count._value.fetch_add(delta); }
- void unref() noexcept;
- void ref() noexcept { ref(1); }
+ BroadcastPBlockHolderQueue(
+ std::shared_ptr<pipeline::BroadcastDependency>& broadcast_dependency) {
+ _broadcast_dependency = broadcast_dependency;
+ }
- bool available() { return _ref_count._value == 0; }
+ void push(std::shared_ptr<BroadcastPBlockHolder> holder);
- PBlock* get_block() { return &pblock; }
+ bool empty() {
Review Comment:
warning: method 'empty' can be made static [readability-convert-member-functions-to-static]
```suggestion
static bool empty() {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864349464
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.26 seconds
stream load tsv: 584 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17184180654 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864276836
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1863968600
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864058219
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864046917
run compile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432356524
##########
be/src/pipeline/exec/exchange_sink_buffer.cpp:
##########
@@ -45,16 +45,40 @@
#include "vec/sink/vdata_stream_sender.h"
namespace doris {
+
namespace vectorized {
+BroadcastPBlockHolder::~BroadcastPBlockHolder() {
+ // lock the parent queue, if the queue could lock success, then return the block
+ // to the queue, to reuse the block
+ std::shared_ptr<BroadcastPBlockHolderQueue> tmp_queue = _parent_creator.lock();
+ if (tmp_queue != nullptr) {
+ tmp_queue->push(BroadcastPBlockHolder::create_shared(std::move(_pblock)));
+ }
+ // If the queue already deconstruted, then release pblock automatically since it
+ // is a unique ptr.
+}
-void BroadcastPBlockHolder::unref() noexcept {
- DCHECK_GT(_ref_count._value, 0);
- auto old_value = _ref_count._value.fetch_sub(1);
- if (_dep && old_value == 1) {
- _dep->return_available_block();
+void BroadcastPBlockHolderQueue::push(std::shared_ptr<BroadcastPBlockHolder> holder) {
Review Comment:
warning: method 'push' can be made static [readability-convert-member-functions-to-static]
be/src/pipeline/exec/exchange_sink_buffer.h:107:
```diff
- void push(std::shared_ptr<BroadcastPBlockHolder> holder);
+ static void push(std::shared_ptr<BroadcastPBlockHolder> holder);
```
##########
be/src/pipeline/exec/exchange_sink_buffer.cpp:
##########
@@ -45,16 +45,40 @@
#include "vec/sink/vdata_stream_sender.h"
namespace doris {
+
namespace vectorized {
+BroadcastPBlockHolder::~BroadcastPBlockHolder() {
+ // lock the parent queue, if the queue could lock success, then return the block
+ // to the queue, to reuse the block
+ std::shared_ptr<BroadcastPBlockHolderQueue> tmp_queue = _parent_creator.lock();
+ if (tmp_queue != nullptr) {
+ tmp_queue->push(BroadcastPBlockHolder::create_shared(std::move(_pblock)));
+ }
+ // If the queue already deconstruted, then release pblock automatically since it
+ // is a unique ptr.
+}
-void BroadcastPBlockHolder::unref() noexcept {
- DCHECK_GT(_ref_count._value, 0);
- auto old_value = _ref_count._value.fetch_sub(1);
- if (_dep && old_value == 1) {
- _dep->return_available_block();
+void BroadcastPBlockHolderQueue::push(std::shared_ptr<BroadcastPBlockHolder> holder) {
+ std::unique_lock(_holders_lock);
+ holder->set_parent_creator(shared_from_this());
+ _holders.push(holder);
+ if (_broadcast_dependency) {
+ _broadcast_dependency->set_ready();
}
}
+std::shared_ptr<BroadcastPBlockHolder> BroadcastPBlockHolderQueue::pop() {
Review Comment:
warning: method 'pop' can be made static [readability-convert-member-functions-to-static]
be/src/pipeline/exec/exchange_sink_buffer.h:111:
```diff
- std::shared_ptr<BroadcastPBlockHolder> pop();
+ static std::shared_ptr<BroadcastPBlockHolder> pop();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1863841270
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1863977326
run compile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432528268
##########
be/src/pipeline/exec/exchange_sink_operator.h:
##########
@@ -76,45 +76,16 @@ class BroadcastDependency final : public Dependency {
public:
ENABLE_FACTORY_CREATOR(BroadcastDependency);
BroadcastDependency(int id, int node_id, QueryContext* query_ctx)
- : Dependency(id, node_id, "BroadcastDependency", true, query_ctx),
- _available_block(0) {}
+ : Dependency(id, node_id, "BroadcastDependency", true, query_ctx) {}
~BroadcastDependency() override = default;
std::string debug_string(int indentation_level = 0) override {
Review Comment:
delete this function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864111435
TeamCity be ut coverage result:
Function Coverage: 36.48% (8534/23393)
Line Coverage: 28.61% (69382/242519)
Region Coverage: 27.63% (35898/129927)
Branch Coverage: 24.37% (18347/75290)
Coverage Report: http://coverage.selectdb-in.cc/coverage/776f61933c8c65d96bd9860b2dd3bc1d12257a4a_776f61933c8c65d96bd9860b2dd3bc1d12257a4a/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1863901575
TeamCity be ut coverage result:
Function Coverage: 36.50% (8535/23382)
Line Coverage: 28.63% (69372/242332)
Region Coverage: 27.65% (35905/129843)
Branch Coverage: 24.38% (18339/75234)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4fdd8f5313ec1bad3bec41e6a5f27fb675b1fbae_4fdd8f5313ec1bad3bec41e6a5f27fb675b1fbae/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #28698:
URL: https://github.com/apache/doris/pull/28698
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432228834
##########
be/src/vec/sink/vdata_stream_sender.cpp:
##########
@@ -777,18 +775,14 @@ void VDataStreamSender::_roll_pb_block() {
_cur_pb_block = (_cur_pb_block == &_pb_block1 ? &_pb_block2 : &_pb_block1);
}
-Status VDataStreamSender::_get_next_available_buffer(BroadcastPBlockHolder** holder) {
- if (_broadcast_pb_block_idx >= _broadcast_pb_blocks.size()) {
- return Status::InternalError(
- "get_next_available_buffer meet invalid index, index={}, size={}",
- _broadcast_pb_block_idx, _broadcast_pb_blocks.size());
- }
- if (!_broadcast_pb_blocks[_broadcast_pb_block_idx].available()) {
- return Status::InternalError("broadcast_pb_blocks not available");
+Status VDataStreamSender::_get_next_available_buffer(
Review Comment:
warning: method '_get_next_available_buffer' can be made static [readability-convert-member-functions-to-static]
be/src/vec/sink/vdata_stream_sender.h:160:
```diff
- Status _get_next_available_buffer(std::shared_ptr<BroadcastPBlockHolder>* holder);
+ static Status _get_next_available_buffer(std::shared_ptr<BroadcastPBlockHolder>* holder);
```
##########
be/src/pipeline/exec/exchange_sink_buffer.h:
##########
@@ -71,25 +72,51 @@ struct AtomicWrapper {
// We use BroadcastPBlockHolder to hold a broadcasted PBlock. For broadcast shuffle, one PBlock
// will be shared between different channel, so we have to use a ref count to mark if this
// PBlock is available for next serialization.
+class BroadcastPBlockHolderQueue;
class BroadcastPBlockHolder {
+ ENABLE_FACTORY_CREATOR(BroadcastPBlockHolder);
+
public:
- BroadcastPBlockHolder() : _ref_count(0), _dep(nullptr) {}
- BroadcastPBlockHolder(pipeline::BroadcastDependency* dep) : _ref_count(0), _dep(dep) {}
- ~BroadcastPBlockHolder() noexcept = default;
+ BroadcastPBlockHolder() { _pblock = std::make_unique<PBlock>(); }
Review Comment:
warning: use '= default' to define a trivial default constructor [modernize-use-equals-default]
```cpp
BroadcastPBlockHolder() { _pblock = std::make_unique<PBlock>(); }
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432531540
##########
be/src/pipeline/exec/exchange_sink_buffer.cpp:
##########
@@ -45,16 +45,40 @@
#include "vec/sink/vdata_stream_sender.h"
namespace doris {
+
namespace vectorized {
+BroadcastPBlockHolder::~BroadcastPBlockHolder() {
+ // lock the parent queue, if the queue could lock success, then return the block
+ // to the queue, to reuse the block
+ std::shared_ptr<BroadcastPBlockHolderQueue> tmp_queue = _parent_creator.lock();
+ if (tmp_queue != nullptr) {
+ tmp_queue->push(BroadcastPBlockHolder::create_shared(std::move(_pblock)));
+ }
+ // If the queue already deconstruted, then release pblock automatically since it
+ // is a unique ptr.
+}
-void BroadcastPBlockHolder::unref() noexcept {
- DCHECK_GT(_ref_count._value, 0);
- auto old_value = _ref_count._value.fetch_sub(1);
- if (_dep && old_value == 1) {
- _dep->return_available_block();
+void BroadcastPBlockHolderQueue::push(std::shared_ptr<BroadcastPBlockHolder> holder) {
+ std::unique_lock l(_holders_lock);
+ holder->set_parent_creator(shared_from_this());
+ _holders.push(holder);
+ if (_broadcast_dependency) {
+ _broadcast_dependency->set_ready();
}
}
+std::shared_ptr<BroadcastPBlockHolder> BroadcastPBlockHolderQueue::pop() {
+ std::unique_lock l(_holders_lock);
+ if (_holders.empty()) {
+ return {};
+ }
+ std::shared_ptr<BroadcastPBlockHolder> res = _holders.top();
+ _holders.pop();
+ if (_holders.empty() && _broadcast_dependency != nullptr) {
Review Comment:
C ould not add this decheck ,because pipeline also use this code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432246978
##########
be/src/pipeline/exec/exchange_sink_buffer.cpp:
##########
@@ -45,16 +45,38 @@
#include "vec/sink/vdata_stream_sender.h"
namespace doris {
+
namespace vectorized {
+BroadcastPBlockHolder::~BroadcastPBlockHolder() {
+ // lock the parent queue, if the queue could lock success, then return the block
+ // to the queue, to reuse the block
+ std::shared_ptr<BroadcastPBlockHolderQueue> tmp_queue = _parent_creator.lock();
+ if (tmp_queue != nullptr) {
+ tmp_queue->push(BroadcastPBlockHolder::create_shared(std::move(_pblock)));
+ }
+ // If the queue already deconstruted, then release pblock automatically since it
+ // is a unique ptr.
+}
-void BroadcastPBlockHolder::unref() noexcept {
- DCHECK_GT(_ref_count._value, 0);
- auto old_value = _ref_count._value.fetch_sub(1);
- if (_dep && old_value == 1) {
- _dep->return_available_block();
+void BroadcastPBlockHolderQueue::push(std::shared_ptr<BroadcastPBlockHolder> holder) {
+ holder->set_parent_creator(shared_from_this());
+ _holders.push(holder);
+ if (_broadcast_dependency) {
+ _broadcast_dependency->set_ready();
}
}
+std::shared_ptr<BroadcastPBlockHolder> BroadcastPBlockHolderQueue::pop() {
Review Comment:
warning: method 'pop' can be made static [readability-convert-member-functions-to-static]
be/src/pipeline/exec/exchange_sink_buffer.h:113:
```diff
- std::shared_ptr<BroadcastPBlockHolder> pop();
+ static std::shared_ptr<BroadcastPBlockHolder> pop();
```
##########
be/src/pipeline/exec/exchange_sink_buffer.cpp:
##########
@@ -45,16 +45,38 @@
#include "vec/sink/vdata_stream_sender.h"
namespace doris {
+
namespace vectorized {
+BroadcastPBlockHolder::~BroadcastPBlockHolder() {
+ // lock the parent queue, if the queue could lock success, then return the block
+ // to the queue, to reuse the block
+ std::shared_ptr<BroadcastPBlockHolderQueue> tmp_queue = _parent_creator.lock();
+ if (tmp_queue != nullptr) {
+ tmp_queue->push(BroadcastPBlockHolder::create_shared(std::move(_pblock)));
+ }
+ // If the queue already deconstruted, then release pblock automatically since it
+ // is a unique ptr.
+}
-void BroadcastPBlockHolder::unref() noexcept {
- DCHECK_GT(_ref_count._value, 0);
- auto old_value = _ref_count._value.fetch_sub(1);
- if (_dep && old_value == 1) {
- _dep->return_available_block();
+void BroadcastPBlockHolderQueue::push(std::shared_ptr<BroadcastPBlockHolder> holder) {
Review Comment:
warning: method 'push' can be made static [readability-convert-member-functions-to-static]
be/src/pipeline/exec/exchange_sink_buffer.h:109:
```diff
- void push(std::shared_ptr<BroadcastPBlockHolder> holder);
+ static void push(std::shared_ptr<BroadcastPBlockHolder> holder);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864346188
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Tpch sf100 test result on commit beb05158a9b98babaaa508efd56612fea55c12db, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4673 4423 4424 4423
q2 357 155 160 155
q3 1453 1306 1234 1234
q4 1112 926 917 917
q5 3160 3137 3155 3137
q6 246 128 130 128
q7 987 493 484 484
q8 2193 2229 2164 2164
q9 6678 6667 6681 6667
q10 3210 3277 3263 3263
q11 307 191 181 181
q12 353 206 199 199
q13 4579 3797 3801 3797
q14 248 216 212 212
q15 560 525 517 517
q16 440 384 390 384
q17 1005 594 555 555
q18 7163 6896 6876 6876
q19 1505 1382 1410 1382
q20 546 297 300 297
q21 3085 2622 2682 2622
q22 349 280 277 277
Total cold run time: 44209 ms
Total hot run time: 39871 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4329 4394 4345 4345
q2 268 164 182 164
q3 3551 3525 3523 3523
q4 2388 2388 2371 2371
q5 5737 5740 5742 5740
q6 242 120 122 120
q7 2388 1876 1850 1850
q8 3522 3533 3521 3521
q9 9003 8962 8954 8954
q10 3930 4022 4006 4006
q11 501 368 375 368
q12 767 597 587 587
q13 4295 3574 3545 3545
q14 287 264 253 253
q15 571 519 532 519
q16 508 487 485 485
q17 1872 1862 1850 1850
q18 8675 8205 8279 8205
q19 1708 1721 1782 1721
q20 2259 1957 1940 1940
q21 6553 6206 6166 6166
q22 516 432 425 425
Total cold run time: 63870 ms
Total hot run time: 60658 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864154759
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Tpch sf100 test result on commit 776f61933c8c65d96bd9860b2dd3bc1d12257a4a, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4666 4411 4444 4411
q2 374 140 160 140
q3 1459 1245 1238 1238
q4 1132 912 918 912
q5 3148 3166 3158 3158
q6 247 128 126 126
q7 1008 492 474 474
q8 2208 2238 2160 2160
q9 6698 6633 6651 6633
q10 3212 3288 3258 3258
q11 313 193 178 178
q12 346 205 213 205
q13 4558 3828 3806 3806
q14 235 216 218 216
q15 569 526 519 519
q16 442 385 385 385
q17 1006 577 565 565
q18 7170 6961 6875 6875
q19 1520 1356 1362 1356
q20 502 301 303 301
q21 3088 2622 2727 2622
q22 344 280 274 274
Total cold run time: 44245 ms
Total hot run time: 39812 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4341 4370 4357 4357
q2 267 163 167 163
q3 3529 3525 3505 3505
q4 2388 2378 2374 2374
q5 5742 5755 5833 5755
q6 240 124 120 120
q7 2385 1855 1870 1855
q8 3518 3520 3531 3520
q9 9034 9040 9002 9002
q10 3933 3984 4023 3984
q11 479 364 367 364
q12 767 622 626 622
q13 4275 3541 3550 3541
q14 291 256 253 253
q15 571 519 526 519
q16 521 468 464 464
q17 1867 1831 1852 1831
q18 8584 8253 8357 8253
q19 1721 1751 1747 1747
q20 2251 1967 1926 1926
q21 6532 6184 6160 6160
q22 518 433 423 423
Total cold run time: 63754 ms
Total hot run time: 60738 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864318269
TeamCity be ut coverage result:
Function Coverage: 36.48% (8533/23391)
Line Coverage: 28.61% (69372/242516)
Region Coverage: 27.63% (35896/129927)
Branch Coverage: 24.36% (18344/75290)
Coverage Report: http://coverage.selectdb-in.cc/coverage/beb05158a9b98babaaa508efd56612fea55c12db_beb05158a9b98babaaa508efd56612fea55c12db/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864136841
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.83 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17183897235 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432547851
##########
be/src/pipeline/exec/exchange_sink_buffer.cpp:
##########
@@ -45,16 +45,40 @@
#include "vec/sink/vdata_stream_sender.h"
namespace doris {
+
namespace vectorized {
+BroadcastPBlockHolder::~BroadcastPBlockHolder() {
+ // lock the parent queue, if the queue could lock success, then return the block
+ // to the queue, to reuse the block
+ std::shared_ptr<BroadcastPBlockHolderQueue> tmp_queue = _parent_creator.lock();
+ if (tmp_queue != nullptr) {
+ tmp_queue->push(BroadcastPBlockHolder::create_shared(std::move(_pblock)));
+ }
+ // If the queue already deconstruted, then release pblock automatically since it
+ // is a unique ptr.
+}
-void BroadcastPBlockHolder::unref() noexcept {
- DCHECK_GT(_ref_count._value, 0);
- auto old_value = _ref_count._value.fetch_sub(1);
- if (_dep && old_value == 1) {
- _dep->return_available_block();
+void BroadcastPBlockHolderQueue::push(std::shared_ptr<BroadcastPBlockHolder> holder) {
Review Comment:
warning: method 'push' can be made static [readability-convert-member-functions-to-static]
be/src/pipeline/exec/exchange_sink_buffer.h:104:
```diff
- void push(std::shared_ptr<BroadcastPBlockHolder> holder);
+ static void push(std::shared_ptr<BroadcastPBlockHolder> holder);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432530620
##########
be/src/pipeline/exec/exchange_sink_operator.cpp:
##########
@@ -456,21 +449,16 @@ void ExchangeSinkLocalState::register_channels(
}
Status ExchangeSinkLocalState::get_next_available_buffer(
- vectorized::BroadcastPBlockHolder** holder) {
+ std::shared_ptr<vectorized::BroadcastPBlockHolder>* holder) {
Review Comment:
We'd better keep the pattern. If the parameter is the output parameter, then use pointer not reference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432407126
##########
be/src/pipeline/exec/exchange_sink_buffer.cpp:
##########
@@ -45,16 +45,40 @@
#include "vec/sink/vdata_stream_sender.h"
namespace doris {
+
namespace vectorized {
+BroadcastPBlockHolder::~BroadcastPBlockHolder() {
+ // lock the parent queue, if the queue could lock success, then return the block
+ // to the queue, to reuse the block
+ std::shared_ptr<BroadcastPBlockHolderQueue> tmp_queue = _parent_creator.lock();
+ if (tmp_queue != nullptr) {
+ tmp_queue->push(BroadcastPBlockHolder::create_shared(std::move(_pblock)));
+ }
+ // If the queue already deconstruted, then release pblock automatically since it
+ // is a unique ptr.
+}
-void BroadcastPBlockHolder::unref() noexcept {
- DCHECK_GT(_ref_count._value, 0);
- auto old_value = _ref_count._value.fetch_sub(1);
- if (_dep && old_value == 1) {
- _dep->return_available_block();
+void BroadcastPBlockHolderQueue::push(std::shared_ptr<BroadcastPBlockHolder> holder) {
+ std::unique_lock l(_holders_lock);
+ holder->set_parent_creator(shared_from_this());
+ _holders.push(holder);
+ if (_broadcast_dependency) {
+ _broadcast_dependency->set_ready();
}
}
+std::shared_ptr<BroadcastPBlockHolder> BroadcastPBlockHolderQueue::pop() {
Review Comment:
warning: method 'pop' can be made static [readability-convert-member-functions-to-static]
be/src/pipeline/exec/exchange_sink_buffer.h:110:
```diff
- std::shared_ptr<BroadcastPBlockHolder> pop();
+ static std::shared_ptr<BroadcastPBlockHolder> pop();
```
##########
be/src/pipeline/exec/exchange_sink_buffer.cpp:
##########
@@ -45,16 +45,40 @@
#include "vec/sink/vdata_stream_sender.h"
namespace doris {
+
namespace vectorized {
+BroadcastPBlockHolder::~BroadcastPBlockHolder() {
+ // lock the parent queue, if the queue could lock success, then return the block
+ // to the queue, to reuse the block
+ std::shared_ptr<BroadcastPBlockHolderQueue> tmp_queue = _parent_creator.lock();
+ if (tmp_queue != nullptr) {
+ tmp_queue->push(BroadcastPBlockHolder::create_shared(std::move(_pblock)));
+ }
+ // If the queue already deconstruted, then release pblock automatically since it
+ // is a unique ptr.
+}
-void BroadcastPBlockHolder::unref() noexcept {
- DCHECK_GT(_ref_count._value, 0);
- auto old_value = _ref_count._value.fetch_sub(1);
- if (_dep && old_value == 1) {
- _dep->return_available_block();
+void BroadcastPBlockHolderQueue::push(std::shared_ptr<BroadcastPBlockHolder> holder) {
Review Comment:
warning: method 'push' can be made static [readability-convert-member-functions-to-static]
be/src/pipeline/exec/exchange_sink_buffer.h:106:
```diff
- void push(std::shared_ptr<BroadcastPBlockHolder> holder);
+ static void push(std::shared_ptr<BroadcastPBlockHolder> holder);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1863898487
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.1 seconds
stream load tsv: 568 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17188157932 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432518792
##########
be/src/pipeline/exec/exchange_sink_operator.cpp:
##########
@@ -456,21 +449,16 @@ void ExchangeSinkLocalState::register_channels(
}
Status ExchangeSinkLocalState::get_next_available_buffer(
- vectorized::BroadcastPBlockHolder** holder) {
+ std::shared_ptr<vectorized::BroadcastPBlockHolder>* holder) {
Review Comment:
```suggestion
std::shared_ptr<vectorized::BroadcastPBlockHolder>& holder) {
```
##########
be/src/pipeline/exec/exchange_sink_buffer.cpp:
##########
@@ -45,16 +45,40 @@
#include "vec/sink/vdata_stream_sender.h"
namespace doris {
+
namespace vectorized {
+BroadcastPBlockHolder::~BroadcastPBlockHolder() {
+ // lock the parent queue, if the queue could lock success, then return the block
+ // to the queue, to reuse the block
+ std::shared_ptr<BroadcastPBlockHolderQueue> tmp_queue = _parent_creator.lock();
+ if (tmp_queue != nullptr) {
+ tmp_queue->push(BroadcastPBlockHolder::create_shared(std::move(_pblock)));
+ }
+ // If the queue already deconstruted, then release pblock automatically since it
+ // is a unique ptr.
+}
-void BroadcastPBlockHolder::unref() noexcept {
- DCHECK_GT(_ref_count._value, 0);
- auto old_value = _ref_count._value.fetch_sub(1);
- if (_dep && old_value == 1) {
- _dep->return_available_block();
+void BroadcastPBlockHolderQueue::push(std::shared_ptr<BroadcastPBlockHolder> holder) {
+ std::unique_lock l(_holders_lock);
+ holder->set_parent_creator(shared_from_this());
+ _holders.push(holder);
+ if (_broadcast_dependency) {
+ _broadcast_dependency->set_ready();
}
}
+std::shared_ptr<BroadcastPBlockHolder> BroadcastPBlockHolderQueue::pop() {
+ std::unique_lock l(_holders_lock);
+ if (_holders.empty()) {
+ return {};
+ }
+ std::shared_ptr<BroadcastPBlockHolder> res = _holders.top();
+ _holders.pop();
+ if (_holders.empty() && _broadcast_dependency != nullptr) {
Review Comment:
DCHECK(_broadcast_dependency != nullptr);
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1863890065
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Tpch sf100 test result on commit 4fdd8f5313ec1bad3bec41e6a5f27fb675b1fbae, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4665 4471 4429 4429
q2 363 150 158 150
q3 1463 1225 1239 1225
q4 1110 906 915 906
q5 3125 3168 3179 3168
q6 248 126 128 126
q7 985 491 487 487
q8 2204 2215 2181 2181
q9 6683 6667 6674 6667
q10 3238 3267 3291 3267
q11 312 184 186 184
q12 353 208 206 206
q13 4547 3818 3801 3801
q14 243 213 210 210
q15 561 525 531 525
q16 444 388 386 386
q17 1008 579 535 535
q18 7141 6868 6833 6833
q19 1519 1395 1413 1395
q20 553 316 299 299
q21 3055 2622 2607 2607
q22 354 281 276 276
Total cold run time: 44174 ms
Total hot run time: 39863 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4348 4369 4344 4344
q2 268 165 178 165
q3 3541 3530 3524 3524
q4 2387 2376 2384 2376
q5 5736 5730 5733 5730
q6 241 119 121 119
q7 2368 1888 1873 1873
q8 3527 3539 3518 3518
q9 9008 8981 8969 8969
q10 3940 4011 4011 4011
q11 485 372 383 372
q12 767 593 607 593
q13 4268 3577 3583 3577
q14 282 249 248 248
q15 565 518 522 518
q16 515 476 451 451
q17 1870 1853 1838 1838
q18 8549 8225 8113 8113
q19 1767 1773 1760 1760
q20 2274 1957 1951 1951
q21 6486 6153 6151 6151
q22 498 426 431 426
Total cold run time: 63690 ms
Total hot run time: 60627 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864387033
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864062232
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1863933352
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org