You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "TangSiyang2001 (via GitHub)" <gi...@apache.org> on 2023/11/07 13:08:10 UTC
[PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
TangSiyang2001 opened a new pull request, #26544:
URL: https://github.com/apache/doris/pull/26544
## Proposed changes
added metrics for flush thread pool and flush tasks num
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1798488740
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on code in PR #26544:
URL: https://github.com/apache/doris/pull/26544#discussion_r1386157604
##########
be/src/olap/memtable_memory_limiter.cpp:
##########
@@ -27,6 +27,7 @@ namespace doris {
DEFINE_GAUGE_METRIC_PROTOTYPE_5ARG(memtable_memory_limiter_mem_consumption, MetricUnit::BYTES, "",
memtable_memory_limiter_mem_consumption,
Labels({{"type", "load"}}));
+bvar::Adder<int64_t> g_flush_async_num("memtable_flush_async_num");
Review Comment:
Redundant, just for testing, I'll rm it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1801395458
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1798653297
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.4 seconds
stream load tsv: 585 seconds loaded 74807831229 Bytes, about 121 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162426087 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on code in PR #26544:
URL: https://github.com/apache/doris/pull/26544#discussion_r1386159321
##########
be/src/olap/memtable_flush_executor.cpp:
##########
@@ -122,7 +131,8 @@ Status FlushToken::_do_flush_memtable(MemTable* memtable, int32_t segment_id, in
return Status::OK();
}
-void FlushToken::_flush_memtable(MemTable* memtable, int32_t segment_id, int64_t submit_task_time) {
+void FlushToken::_flush_memtable(MemTable* mem_table, int32_t segment_id,
Review Comment:
Just unify the param name and make the clang-tidy happy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "freemandealer (via GitHub)" <gi...@apache.org>.
freemandealer commented on code in PR #26544:
URL: https://github.com/apache/doris/pull/26544#discussion_r1384928386
##########
be/src/olap/memtable_memory_limiter.cpp:
##########
@@ -27,6 +27,7 @@ namespace doris {
DEFINE_GAUGE_METRIC_PROTOTYPE_5ARG(memtable_memory_limiter_mem_consumption, MetricUnit::BYTES, "",
memtable_memory_limiter_mem_consumption,
Labels({{"type", "load"}}));
+bvar::Adder<int64_t> g_flush_async_num("memtable_flush_async_num");
Review Comment:
what knowledge can we get from this bvar?
##########
be/src/olap/memtable_flush_executor.cpp:
##########
@@ -122,7 +131,8 @@ Status FlushToken::_do_flush_memtable(MemTable* memtable, int32_t segment_id, in
return Status::OK();
}
-void FlushToken::_flush_memtable(MemTable* memtable, int32_t segment_id, int64_t submit_task_time) {
+void FlushToken::_flush_memtable(MemTable* mem_table, int32_t segment_id,
Review Comment:
renaming to mem_table does not make any sense
##########
be/src/olap/memtable_flush_executor.cpp:
##########
@@ -189,26 +200,38 @@ Status MemTableFlushExecutor::create_flush_token(std::unique_ptr<FlushToken>& fl
if (!is_high_priority) {
if (rowset_writer->type() == BETA_ROWSET && !should_serial) {
// beta rowset can be flush in CONCURRENT, because each memtable using a new segment writer.
- flush_token.reset(
- new FlushToken(_flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT)));
+ flush_token = std::make_unique<FlushToken>(
+ _flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT));
} else {
// alpha rowset do not support flush in CONCURRENT.
- flush_token.reset(
- new FlushToken(_flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL)));
+ flush_token = std::make_unique<FlushToken>(
+ _flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL));
}
} else {
if (rowset_writer->type() == BETA_ROWSET && !should_serial) {
// beta rowset can be flush in CONCURRENT, because each memtable using a new segment writer.
- flush_token.reset(new FlushToken(
- _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT)));
+ flush_token = std::make_unique<FlushToken>(
+ _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT));
} else {
// alpha rowset do not support flush in CONCURRENT.
- flush_token.reset(new FlushToken(
- _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL)));
+ flush_token = std::make_unique<FlushToken>(
+ _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL));
}
}
flush_token->set_rowset_writer(rowset_writer);
return Status::OK();
}
+void MemTableFlushExecutor::_register_metrics() {
+ REGISTER_HOOK_METRIC(flush_thread_pool_queue_size,
+ [this]() { return _flush_pool->get_queue_size(); });
Review Comment:
we prefer an unsafe & inaccurate queue size to accurate one with heavy lock overhead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "freemandealer (via GitHub)" <gi...@apache.org>.
freemandealer commented on code in PR #26544:
URL: https://github.com/apache/doris/pull/26544#discussion_r1388888120
##########
be/src/olap/memtable_flush_executor.cpp:
##########
@@ -122,7 +131,8 @@ Status FlushToken::_do_flush_memtable(MemTable* memtable, int32_t segment_id, in
return Status::OK();
}
-void FlushToken::_flush_memtable(MemTable* memtable, int32_t segment_id, int64_t submit_task_time) {
+void FlushToken::_flush_memtable(MemTable* mem_table, int32_t segment_id,
Review Comment:
> Just unify the param name with declaration and make the clang-tidy happy.
we all know that clang-tidy's happiness is super important, so is memtable's. Maybe renaming the declaration is a win-win solution :P
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on code in PR #26544:
URL: https://github.com/apache/doris/pull/26544#discussion_r1386160550
##########
be/src/olap/memtable_flush_executor.cpp:
##########
@@ -189,26 +200,38 @@ Status MemTableFlushExecutor::create_flush_token(std::unique_ptr<FlushToken>& fl
if (!is_high_priority) {
if (rowset_writer->type() == BETA_ROWSET && !should_serial) {
// beta rowset can be flush in CONCURRENT, because each memtable using a new segment writer.
- flush_token.reset(
- new FlushToken(_flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT)));
+ flush_token = std::make_unique<FlushToken>(
+ _flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT));
} else {
// alpha rowset do not support flush in CONCURRENT.
- flush_token.reset(
- new FlushToken(_flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL)));
+ flush_token = std::make_unique<FlushToken>(
+ _flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL));
}
} else {
if (rowset_writer->type() == BETA_ROWSET && !should_serial) {
// beta rowset can be flush in CONCURRENT, because each memtable using a new segment writer.
- flush_token.reset(new FlushToken(
- _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT)));
+ flush_token = std::make_unique<FlushToken>(
+ _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT));
} else {
// alpha rowset do not support flush in CONCURRENT.
- flush_token.reset(new FlushToken(
- _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL)));
+ flush_token = std::make_unique<FlushToken>(
+ _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL));
}
}
flush_token->set_rowset_writer(rowset_writer);
return Status::OK();
}
+void MemTableFlushExecutor::_register_metrics() {
+ REGISTER_HOOK_METRIC(flush_thread_pool_queue_size,
+ [this]() { return _flush_pool->get_queue_size(); });
Review Comment:
Yes it is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool and scanner scheduler thread pools [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1801383942
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on code in PR #26544:
URL: https://github.com/apache/doris/pull/26544#discussion_r1386159321
##########
be/src/olap/memtable_flush_executor.cpp:
##########
@@ -122,7 +131,8 @@ Status FlushToken::_do_flush_memtable(MemTable* memtable, int32_t segment_id, in
return Status::OK();
}
-void FlushToken::_flush_memtable(MemTable* memtable, int32_t segment_id, int64_t submit_task_time) {
+void FlushToken::_flush_memtable(MemTable* mem_table, int32_t segment_id,
Review Comment:
Just unified the param name and make the clang-tidy happy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1801470299
TeamCity be ut coverage result:
Function Coverage: 36.71% (8391/22860)
Line Coverage: 29.22% (68051/232897)
Region Coverage: 27.88% (35203/126273)
Branch Coverage: 24.70% (17994/72850)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c8e07183ddea4b3b574768580ac5027dffb42029_c8e07183ddea4b3b574768580ac5027dffb42029/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool and scanner scheduler thread pools [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1801380271
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1801568370
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.05 seconds
stream load tsv: 560 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162529463 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "freemandealer (via GitHub)" <gi...@apache.org>.
freemandealer commented on code in PR #26544:
URL: https://github.com/apache/doris/pull/26544#discussion_r1384954351
##########
be/src/olap/memtable_flush_executor.cpp:
##########
@@ -189,26 +200,38 @@ Status MemTableFlushExecutor::create_flush_token(std::unique_ptr<FlushToken>& fl
if (!is_high_priority) {
if (rowset_writer->type() == BETA_ROWSET && !should_serial) {
// beta rowset can be flush in CONCURRENT, because each memtable using a new segment writer.
- flush_token.reset(
- new FlushToken(_flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT)));
+ flush_token = std::make_unique<FlushToken>(
+ _flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT));
} else {
// alpha rowset do not support flush in CONCURRENT.
- flush_token.reset(
- new FlushToken(_flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL)));
+ flush_token = std::make_unique<FlushToken>(
+ _flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL));
}
} else {
if (rowset_writer->type() == BETA_ROWSET && !should_serial) {
// beta rowset can be flush in CONCURRENT, because each memtable using a new segment writer.
- flush_token.reset(new FlushToken(
- _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT)));
+ flush_token = std::make_unique<FlushToken>(
+ _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::CONCURRENT));
} else {
// alpha rowset do not support flush in CONCURRENT.
- flush_token.reset(new FlushToken(
- _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL)));
+ flush_token = std::make_unique<FlushToken>(
+ _high_prio_flush_pool->new_token(ThreadPool::ExecutionMode::SERIAL));
}
}
flush_token->set_rowset_writer(rowset_writer);
return Status::OK();
}
+void MemTableFlushExecutor::_register_metrics() {
+ REGISTER_HOOK_METRIC(flush_thread_pool_queue_size,
+ [this]() { return _flush_pool->get_queue_size(); });
Review Comment:
since metrics are updated every 15 sec I think it is affordable now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1798575689
TeamCity be ut coverage result:
Function Coverage: 37.02% (8393/22670)
Line Coverage: 29.44% (67935/230763)
Region Coverage: 28.13% (35158/124990)
Branch Coverage: 24.90% (17958/72112)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d967d9406a73f00bbde7972d6305c7508eac6b75_d967d9406a73f00bbde7972d6305c7508eac6b75/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #26544:
URL: https://github.com/apache/doris/pull/26544
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on code in PR #26544:
URL: https://github.com/apache/doris/pull/26544#discussion_r1386159321
##########
be/src/olap/memtable_flush_executor.cpp:
##########
@@ -122,7 +131,8 @@ Status FlushToken::_do_flush_memtable(MemTable* memtable, int32_t segment_id, in
return Status::OK();
}
-void FlushToken::_flush_memtable(MemTable* memtable, int32_t segment_id, int64_t submit_task_time) {
+void FlushToken::_flush_memtable(MemTable* mem_table, int32_t segment_id,
Review Comment:
Just unify the param name with declaration and make the clang-tidy happy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1798472090
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1801390452
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1801470318
TeamCity be ut coverage result:
Function Coverage: 36.71% (8391/22860)
Line Coverage: 29.22% (68056/232897)
Region Coverage: 27.88% (35204/126273)
Branch Coverage: 24.70% (17996/72850)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0081240d4149e9131595ba3300b3921b2617db22_0081240d4149e9131595ba3300b3921b2617db22/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1805068250
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1801375522
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1801387671
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1806793557
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1798474953
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance visibility of flush thread pool [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26544:
URL: https://github.com/apache/doris/pull/26544#issuecomment-1798484988
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](metrics) enhance observability of flush thread pool and scanner scheduler thread pools [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #26544:
URL: https://github.com/apache/doris/pull/26544#discussion_r1388888818
##########
be/src/olap/memtable_flush_executor.cpp:
##########
@@ -29,22 +29,30 @@
#include "olap/memtable.h"
#include "olap/rowset/rowset_writer.h"
#include "util/doris_metrics.h"
+#include "util/metrics.h"
#include "util/stopwatch.hpp"
#include "util/time.h"
namespace doris {
using namespace ErrorCode;
+DEFINE_GAUGE_METRIC_PROTOTYPE_2ARG(flush_thread_pool_queue_size, MetricUnit::NOUNIT);
+DEFINE_GAUGE_METRIC_PROTOTYPE_2ARG(flush_thread_pool_thread_num, MetricUnit::NOUNIT);
+
+bvar::Adder<int64_t> g_flush_task_num("memtable_flush_task_num");
Review Comment:
Consider latency recorder
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org