You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/04/23 04:06:55 UTC
[GitHub] [incubator-doris] HappenLee opened a new pull request #3383: Make running profile clearer and more intuitive to improve usability (#3365)
HappenLee opened a new pull request #3383:
URL: https://github.com/apache/incubator-doris/pull/3383
This CL mainly made the following modifications:
1. Delete Invalid method in Running Profile Class.
2. Move Memlimit Counter from blockmgr to frgment and add PeakMemUsage Counter
3. Fix the bug of buffer pool memlimit counter
4. Call compute_time_in_profile() before pretty_print() to show the _local_time_percent without child running profile
5. Add TransferThread ThreadToken count in AveThreadToken Counter
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] HappenLee commented on a change in pull request #3383: Make running profile clearer and more intuitive to improve usability (#3365)
Posted by GitBox <gi...@apache.org>.
HappenLee commented on a change in pull request #3383:
URL: https://github.com/apache/incubator-doris/pull/3383#discussion_r414258601
##########
File path: be/src/runtime/runtime_state.cpp
##########
@@ -226,8 +226,12 @@ Status RuntimeState::init_mem_trackers(const TUniqueId& query_id) {
// _query_mem_tracker = MemTracker::get_query_mem_tracker(
// query_id, bytes_limit, _exec_env->process_mem_tracker());
+
+ auto mem_tracker_counter = ADD_COUNTER(&_profile, "MemoryLimit", TUnit::BYTES);
+ mem_tracker_counter->set(bytes_limit);
+
_query_mem_tracker.reset(
- new MemTracker(bytes_limit, runtime_profile()->name(), _exec_env->process_mem_tracker()));
+ new MemTracker(&_profile, bytes_limit, runtime_profile()->name(), _exec_env->process_mem_tracker()));
Review comment:
yes，I check code。the ```_instance_mem_tracker``` is more appropriate
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] imay commented on a change in pull request #3383: Make running profile clearer and more intuitive to improve usability (#3365)
Posted by GitBox <gi...@apache.org>.
imay commented on a change in pull request #3383:
URL: https://github.com/apache/incubator-doris/pull/3383#discussion_r413926676
##########
File path: be/src/exec/olap_scan_node.cpp
##########
@@ -1038,6 +1038,7 @@ Status OlapScanNode::normalize_binary_predicate(SlotDescriptor* slot, ColumnValu
void OlapScanNode::transfer_thread(RuntimeState* state) {
// scanner open pushdown to scanThread
+ state->resource_pool()->acquire_thread_token();
Review comment:
Can you tell the side effect of this operation? Will this thread hang if there is not enough thread resource?
##########
File path: be/src/runtime/runtime_state.cpp
##########
@@ -226,8 +226,12 @@ Status RuntimeState::init_mem_trackers(const TUniqueId& query_id) {
// _query_mem_tracker = MemTracker::get_query_mem_tracker(
// query_id, bytes_limit, _exec_env->process_mem_tracker());
+
+ auto mem_tracker_counter = ADD_COUNTER(&_profile, "MemoryLimit", TUnit::BYTES);
+ mem_tracker_counter->set(bytes_limit);
+
_query_mem_tracker.reset(
- new MemTracker(bytes_limit, runtime_profile()->name(), _exec_env->process_mem_tracker()));
+ new MemTracker(&_profile, bytes_limit, runtime_profile()->name(), _exec_env->process_mem_tracker()));
Review comment:
If you add profile for this memory tracker, why not `_instance_mem_tracker`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] HappenLee commented on a change in pull request #3383: Make running profile clearer and more intuitive to improve usability (#3365)
Posted by GitBox <gi...@apache.org>.
HappenLee commented on a change in pull request #3383:
URL: https://github.com/apache/incubator-doris/pull/3383#discussion_r414257609
##########
File path: be/src/exec/olap_scan_node.cpp
##########
@@ -1038,6 +1038,7 @@ Status OlapScanNode::normalize_binary_predicate(SlotDescriptor* slot, ColumnValu
void OlapScanNode::transfer_thread(RuntimeState* state) {
// scanner open pushdown to scanThread
+ state->resource_pool()->acquire_thread_token();
Review comment:
Add this operation just to Make AverageThreadTokens more accurate
- AverageThreadTokens: 1.00
Do not warry thread hang on this operation。
acquire_thread_token() just do a lightweight add operation without doing the really thread assignment
inline void ThreadResourceMgr::ResourcePool::acquire_thread_token() {
__sync_fetch_and_add(&_num_threads, 1);
}
@imay
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org