You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/07/01 04:47:04 UTC
[GitHub] [incubator-doris] HappenLee opened a new pull request #3991: Fix the problem of mem exec, when analytic eval node need to spill to disk with a low mem limit
HappenLee opened a new pull request #3991:
URL: https://github.com/apache/incubator-doris/pull/3991
[Bug] Fix the problem of mem exec, when analytic eval node need to spill to disk with a low mem limit. And clear_reservations
of Analytic node reservation of block manager.
[Running Profile] Add Spilled flag in Running Profile, when Analytic eval node and sort node spill to Disk.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #3991: Fix the problem of mem exec, when analytic eval node need to spill to disk with a low mem limit
Posted by GitBox <gi...@apache.org>.
morningman merged pull request #3991:
URL: https://github.com/apache/incubator-doris/pull/3991
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] HappenLee commented on a change in pull request #3991: Fix the problem of mem exec, when analytic eval node need to spill to disk with a low mem limit
Posted by GitBox <gi...@apache.org>.
HappenLee commented on a change in pull request #3991:
URL: https://github.com/apache/incubator-doris/pull/3991#discussion_r448740284
##########
File path: be/src/exec/analytic_eval_node.cpp
##########
@@ -710,7 +709,13 @@ Status AnalyticEvalNode::process_child_batch(RuntimeState* state) {
// Transfer resources to _prev_tuple_pool when enough resources have accumulated
// and the _prev_tuple_pool has already been transfered to an output batch.
- if (_curr_tuple_pool->total_allocated_bytes() > MAX_TUPLE_POOL_SIZE &&
+
+ // The memory limit of _curr_tuple_pool is set by the fixed value MAX_TUPLE_POOL_SIZE
Review comment:
ok，i will change the comment
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] wutiangan commented on a change in pull request #3991: Fix the problem of mem exec, when analytic eval node need to spill to disk with a low mem limit
Posted by GitBox <gi...@apache.org>.
wutiangan commented on a change in pull request #3991:
URL: https://github.com/apache/incubator-doris/pull/3991#discussion_r448239716
##########
File path: be/src/runtime/spill_sorter.cc
##########
@@ -1109,6 +1110,7 @@ Status SpillSorter::add_batch(RowBatch* batch) {
// The current run is full. Sort it and begin the next one.
RETURN_IF_ERROR(sort_run());
RETURN_IF_ERROR(_sorted_runs.back()->unpin_all_blocks());
+ _spilled = true;
Review comment:
Align spaces
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] wutiangan commented on a change in pull request #3991: Fix the problem of mem exec, when analytic eval node need to spill to disk with a low mem limit
Posted by GitBox <gi...@apache.org>.
wutiangan commented on a change in pull request #3991:
URL: https://github.com/apache/incubator-doris/pull/3991#discussion_r448240802
##########
File path: be/src/exec/analytic_eval_node.cpp
##########
@@ -710,7 +709,13 @@ Status AnalyticEvalNode::process_child_batch(RuntimeState* state) {
// Transfer resources to _prev_tuple_pool when enough resources have accumulated
// and the _prev_tuple_pool has already been transfered to an output batch.
- if (_curr_tuple_pool->total_allocated_bytes() > MAX_TUPLE_POOL_SIZE &&
+
+ // The memory limit of _curr_tuple_pool is set by the fixed value MAX_TUPLE_POOL_SIZE
Review comment:
there is no MAX_TUPLE_POOL_SIZE， you deleted it before
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org