You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "mrhhsg (via GitHub)" <gi...@apache.org> on 2026/03/31 14:46:15 UTC
[PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
mrhhsg opened a new pull request, #61973:
URL: https://github.com/apache/doris/pull/61973
### What problem does this PR solve?
When a partition's level reaches or exceeds the max repartition depth, it cannot be repartitioned further. In this case, `revocable_mem_size()` should return 0 because the memory is not actually revocable — attempting to spill it would lead to futile repartition attempts.
This fix adds the depth check to both `PartitionedAggSourceOperatorX` and `PartitionedHashJoinProbeOperatorX`, along with corresponding unit tests that verify:
- revocable_mem_size returns 0 at max depth
- revocable_mem_size returns >0 at (max depth - 1)
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #61973:
URL: https://github.com/apache/doris/pull/61973#discussion_r3018897665
##########
be/src/exec/operator/partitioned_hash_join_probe_operator.cpp:
##########
@@ -875,6 +875,11 @@ size_t PartitionedHashJoinProbeOperatorX::revocable_mem_size(RuntimeState* state
// Or if current partition has finished build hash table.
return 0;
}
+ // If the current partition has reached the max repartition depth, it cannot be
+ // repartitioned further, so its data is not revocable.
+ if (local_state._current_partition.level >= _repartition_max_depth) {
Review Comment:
This has the same off-by-one as the aggregation path. here goes through , which errors once . That means is already non-revocable, but this guard still reports the recovered build block as revocable and can drive the query into . Please align the cutoff with the actual repartition failure boundary, and update the new UT to exercise as the zero case.
##########
be/src/exec/operator/partitioned_aggregation_source_operator.cpp:
##########
@@ -191,6 +191,11 @@ size_t PartitionedAggSourceOperatorX::revocable_mem_size(RuntimeState* state) co
if (!local_state._shared_state->_is_spilled || !local_state._current_partition.spill_file) {
return 0;
}
+ // If the current partition has reached the max repartition depth, it cannot be
+ // repartitioned further, so its data is not revocable.
+ if (local_state._current_partition.level >= _repartition_max_depth) {
Review Comment:
This guard is one level too late. eventually calls , which computes and fails once . So a partition at is already non-revocable: reporting it here lets the scheduler try to revoke it and hit instead of finishing the partition in memory. The cutoff needs to match that failure boundary, and the new UT should assert on rather than .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4166188639
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4163433811
<details>
<summary>TPC-H: <b>Total hot run time: 26498 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6ca13dfc46d77d35a9f851b2ddbdceaf0bb59e88, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17598 4533 4351 4351
q2 q3 10647 778 523 523
q4 4676 343 255 255
q5 7559 1218 1032 1032
q6 178 178 148 148
q7 778 856 666 666
q8 9281 1494 1326 1326
q9 4849 4738 4712 4712
q10 6263 1897 1635 1635
q11 463 259 271 259
q12 718 584 466 466
q13 18040 2705 1934 1934
q14 227 231 215 215
q15 q16 720 732 677 677
q17 715 875 411 411
q18 5926 5404 5247 5247
q19 1122 977 600 600
q20 541 484 383 383
q21 4394 1840 1420 1420
q22 338 291 238 238
Total cold run time: 95033 ms
Total hot run time: 26498 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 5022 4750 4650 4650
q2 q3 3911 4342 3804 3804
q4 861 1211 779 779
q5 4115 4378 4363 4363
q6 190 186 148 148
q7 1866 1637 1549 1549
q8 2497 2724 2621 2621
q9 7652 7523 7397 7397
q10 3784 4005 3719 3719
q11 504 444 411 411
q12 493 613 459 459
q13 2469 2925 2142 2142
q14 289 308 283 283
q15 q16 734 770 722 722
q17 1175 1332 1347 1332
q18 7132 6699 6819 6699
q19 902 885 900 885
q20 2063 2151 2012 2012
q21 3953 3535 3356 3356
q22 466 443 381 381
Total cold run time: 50078 ms
Total hot run time: 47712 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4166771490
@copilot apply changes based on the comments in [this thread](https://github.com/apache/doris/pull/61973#pullrequestreview-4040898260)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #61973:
URL: https://github.com/apache/doris/pull/61973#discussion_r3019033254
##########
be/test/exec/operator/partitioned_aggregation_source_operator_test.cpp:
##########
@@ -737,6 +737,44 @@ TEST_F(PartitionedAggregationSourceOperatorTest, RevocableMemSizeWithAggContaine
EXPECT_EQ(source_operator->revocable_mem_size(_helper.runtime_state.get()), container_bytes);
}
+// Condition: partition level >= _repartition_max_depth → 0 (not revocable).
+TEST_F(PartitionedAggregationSourceOperatorTest, RevocableMemSizeAtMaxDepthReturnsZero) {
+ auto [source_operator, sink_operator] = _helper.create_operators();
+ const auto tnode = _helper.create_test_plan_node();
+ ASSERT_TRUE(source_operator->init(tnode, _helper.runtime_state.get()).ok());
+ ASSERT_TRUE(source_operator->prepare(_helper.runtime_state.get()).ok());
+
+ std::shared_ptr<MockPartitionedAggSharedState> shared_state;
+ auto* local_state = _helper.create_source_local_state(_helper.runtime_state.get(),
+ source_operator.get(), shared_state);
+
+ // Add a large block so that without the depth check, revocable_mem_size would be > 0.
+ std::vector<int32_t> large_data(300000);
+ std::iota(large_data.begin(), large_data.end(), 0);
+ auto large_block = ColumnHelper::create_block<DataTypeInt32>(large_data);
+ ASSERT_GT(large_block.allocated_bytes(), 1UL << 20);
+ local_state->_blocks.push_back(std::move(large_block));
+
+ SpillFileSPtr spill_file;
+ ASSERT_TRUE(ExecEnv::GetInstance()
+ ->spill_file_mgr()
+ ->create_spill_file("ut/revocable_max_depth", spill_file)
+ .ok());
+ local_state->_current_partition.spill_file = spill_file;
+
+ ASSERT_GE(source_operator->_repartition_max_depth, 2);
+
+ // Set partition level to max depth → not revocable.
+ local_state->_current_partition.level =
+ static_cast<int>(source_operator->_repartition_max_depth) - 1;
+ EXPECT_EQ(source_operator->revocable_mem_size(_helper.runtime_state.get()), 0UL);
+
+ // Also verify level == max_depth - 1 IS still revocable.
Review Comment:
This comment says `level == max_depth - 1` is still revocable, but the code below sets the level to `max_depth - 2` for the revocable case. Please correct the comment so it matches the asserted behavior.
```suggestion
// Also verify level == max_depth - 2 (one below max depth) IS still revocable.
```
##########
be/test/exec/operator/partitioned_hash_join_probe_operator_test.cpp:
##########
@@ -1027,6 +1027,51 @@ TEST_F(PartitionedHashJoinProbeOperatorTest, revocable_mem_size) {
ASSERT_EQ(probe_operator->revocable_mem_size(_helper.runtime_state.get()), 0);
}
+// Partition (level + 1) >= _repartition_max_depth → revocable_mem_size returns 0.
+TEST_F(PartitionedHashJoinProbeOperatorTest, revocable_mem_size_at_max_depth) {
+ auto [probe_operator, sink_operator] = _helper.create_operators();
+
+ std::shared_ptr<MockPartitionedHashJoinSharedState> shared_state;
+ auto local_state = _helper.create_probe_local_state(_helper.runtime_state.get(),
+ probe_operator.get(), shared_state);
+
+ local_state->_shared_state->_is_spilled = true;
+ local_state->_child_eos = true;
+
+ // Set up a valid current partition with build not finished (the path that
+ // checks level vs max depth).
+ SpillFileSPtr build_file;
+ ASSERT_TRUE(ExecEnv::GetInstance()
+ ->spill_file_mgr()
+ ->create_spill_file("ut/revocable_join_max_depth_build", build_file)
+ .ok());
+ SpillFileSPtr probe_file;
+ ASSERT_TRUE(ExecEnv::GetInstance()
+ ->spill_file_mgr()
+ ->create_spill_file("ut/revocable_join_max_depth_probe", probe_file)
+ .ok());
Review Comment:
These spill file paths are hard-coded. In this test file, most spill files use a query_id/next_id-based relative path to avoid collisions across runs or parallel test execution. Consider generating unique paths here as well to prevent flaky failures if the same path already exists.
##########
be/test/exec/operator/partitioned_aggregation_source_operator_test.cpp:
##########
@@ -737,6 +737,44 @@ TEST_F(PartitionedAggregationSourceOperatorTest, RevocableMemSizeWithAggContaine
EXPECT_EQ(source_operator->revocable_mem_size(_helper.runtime_state.get()), container_bytes);
}
+// Condition: partition level >= _repartition_max_depth → 0 (not revocable).
+TEST_F(PartitionedAggregationSourceOperatorTest, RevocableMemSizeAtMaxDepthReturnsZero) {
Review Comment:
The test comment says `partition level >= _repartition_max_depth` implies not revocable, but the implementation checks `(level + 1) >= _repartition_max_depth` (i.e., `level >= max_depth - 1`). Please update the comment to match the actual boundary condition to avoid off-by-one confusion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4163201527
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4181286601
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #61973:
URL: https://github.com/apache/doris/pull/61973#discussion_r3018903682
##########
be/test/exec/operator/partitioned_hash_join_probe_operator_test.cpp:
##########
@@ -1027,6 +1027,49 @@ TEST_F(PartitionedHashJoinProbeOperatorTest, revocable_mem_size) {
ASSERT_EQ(probe_operator->revocable_mem_size(_helper.runtime_state.get()), 0);
}
+// Partition level >= _repartition_max_depth → revocable_mem_size returns 0.
+TEST_F(PartitionedHashJoinProbeOperatorTest, revocable_mem_size_at_max_depth) {
+ auto [probe_operator, sink_operator] = _helper.create_operators();
+
+ std::shared_ptr<MockPartitionedHashJoinSharedState> shared_state;
+ auto local_state = _helper.create_probe_local_state(_helper.runtime_state.get(),
+ probe_operator.get(), shared_state);
+
+ local_state->_shared_state->_is_spilled = true;
+ local_state->_child_eos = true;
+
+ // Set up a valid current partition with build not finished (the path that
+ // checks level vs max depth).
+ SpillFileSPtr build_file;
+ ASSERT_TRUE(ExecEnv::GetInstance()
+ ->spill_file_mgr()
+ ->create_spill_file("ut/revocable_join_max_depth_build", build_file)
+ .ok());
+ SpillFileSPtr probe_file;
+ ASSERT_TRUE(ExecEnv::GetInstance()
+ ->spill_file_mgr()
+ ->create_spill_file("ut/revocable_join_max_depth_probe", probe_file)
+ .ok());
+ local_state->_current_partition = JoinSpillPartitionInfo(
+ build_file, probe_file, static_cast<int>(probe_operator->_repartition_max_depth));
+ local_state->_current_partition.build_finished = false;
+
+ // Add a recovered build block large enough to exceed the threshold.
+ std::vector<int32_t> large_data(256 * 1024); // 1MB of int32
+ std::iota(large_data.begin(), large_data.end(), 0);
+ Block large_block = ColumnHelper::create_block<DataTypeInt32>(large_data);
+ ASSERT_GE(large_block.allocated_bytes(), SpillFile::MIN_SPILL_WRITE_BATCH_MEM);
+ local_state->_recovered_build_block = MutableBlock::create_unique(std::move(large_block));
+
+ // At max depth → not revocable.
+ ASSERT_EQ(probe_operator->revocable_mem_size(_helper.runtime_state.get()), 0);
+
+ // One level below max depth → revocable.
+ local_state->_current_partition.level =
+ static_cast<int>(probe_operator->_repartition_max_depth) - 1;
+ ASSERT_GT(probe_operator->revocable_mem_size(_helper.runtime_state.get()), 0);
Review Comment:
This test sets the partition `level` to exactly `_repartition_max_depth` and expects revocable=0, but that level appears unreachable in production because repartitioning rejects `new_level >= _repartition_max_depth` (so the maximum attainable level is `_repartition_max_depth - 1`). To validate real behavior (and catch the off-by-one), adjust the assertions to target `level == _repartition_max_depth - 1` as non-revocable and `level == _repartition_max_depth - 2` as still revocable (or explicitly set the runtime max depth to a small value and test boundary conditions).
##########
be/src/exec/operator/partitioned_hash_join_probe_operator.cpp:
##########
@@ -875,6 +875,11 @@ size_t PartitionedHashJoinProbeOperatorX::revocable_mem_size(RuntimeState* state
// Or if current partition has finished build hash table.
return 0;
}
+ // If the current partition has reached the max repartition depth, it cannot be
+ // repartitioned further, so its data is not revocable.
+ if (local_state._current_partition.level >= _repartition_max_depth) {
Review Comment:
The depth guard is likely off-by-one relative to the actual repartition limit. The repartition path rejects `new_level = level + 1` when `new_level >= _repartition_max_depth` (see `PartitionedHashJoinProbeLocalState::repartition_current_partition`), which means a partition at `level == _repartition_max_depth - 1` already cannot be repartitioned and should be treated as non-revocable. Consider aligning this check with the repartition predicate (i.e., return 0 when `level + 1 >= _repartition_max_depth` / `level >= _repartition_max_depth - 1`) to avoid `revoke_memory()` attempting a repartition that will immediately error.
```suggestion
// If repartitioning the current partition would exceed the max repartition depth
// (i.e., level + 1 >= _repartition_max_depth), it cannot be repartitioned further,
// so its data is not revocable.
if (local_state._current_partition.level + 1 >= _repartition_max_depth) {
```
##########
be/src/exec/operator/partitioned_aggregation_source_operator.cpp:
##########
@@ -191,6 +191,11 @@ size_t PartitionedAggSourceOperatorX::revocable_mem_size(RuntimeState* state) co
if (!local_state._shared_state->_is_spilled || !local_state._current_partition.spill_file) {
return 0;
}
+ // If the current partition has reached the max repartition depth, it cannot be
+ // repartitioned further, so its data is not revocable.
+ if (local_state._current_partition.level >= _repartition_max_depth) {
Review Comment:
Same off-by-one issue as Hash Join: `_flush_and_repartition` errors when `new_level = level + 1 >= _repartition_max_depth`, so a partition at `level == _repartition_max_depth - 1` cannot be repartitioned and its memory is not revocable. The current `level >= _repartition_max_depth` guard won’t prevent futile revoke attempts at the last allowed level. Update the condition to match the repartition limit predicate.
```suggestion
// If repartitioning this partition would exceed the max repartition depth, it cannot be
// repartitioned further, so its data is not revocable.
if (local_state._current_partition.level + 1 >= _repartition_max_depth) {
```
##########
be/test/exec/operator/partitioned_aggregation_source_operator_test.cpp:
##########
@@ -737,6 +737,42 @@ TEST_F(PartitionedAggregationSourceOperatorTest, RevocableMemSizeWithAggContaine
EXPECT_EQ(source_operator->revocable_mem_size(_helper.runtime_state.get()), container_bytes);
}
+// Condition: partition level >= _repartition_max_depth → 0 (not revocable).
+TEST_F(PartitionedAggregationSourceOperatorTest, RevocableMemSizeAtMaxDepthReturnsZero) {
+ auto [source_operator, sink_operator] = _helper.create_operators();
+ const auto tnode = _helper.create_test_plan_node();
+ ASSERT_TRUE(source_operator->init(tnode, _helper.runtime_state.get()).ok());
+ ASSERT_TRUE(source_operator->prepare(_helper.runtime_state.get()).ok());
+
+ std::shared_ptr<MockPartitionedAggSharedState> shared_state;
+ auto* local_state = _helper.create_source_local_state(_helper.runtime_state.get(),
+ source_operator.get(), shared_state);
+
+ // Add a large block so that without the depth check, revocable_mem_size would be > 0.
+ std::vector<int32_t> large_data(300000);
+ std::iota(large_data.begin(), large_data.end(), 0);
+ auto large_block = ColumnHelper::create_block<DataTypeInt32>(large_data);
+ ASSERT_GT(large_block.allocated_bytes(), 1UL << 20);
+ local_state->_blocks.push_back(std::move(large_block));
+
+ SpillFileSPtr spill_file;
+ ASSERT_TRUE(ExecEnv::GetInstance()
+ ->spill_file_mgr()
+ ->create_spill_file("ut/revocable_max_depth", spill_file)
+ .ok());
+ local_state->_current_partition.spill_file = spill_file;
+
+ // Set partition level to max depth → not revocable.
+ local_state->_current_partition.level =
+ static_cast<int>(source_operator->_repartition_max_depth);
+ EXPECT_EQ(source_operator->revocable_mem_size(_helper.runtime_state.get()), 0UL);
+
+ // Also verify level == max_depth - 1 IS still revocable.
+ local_state->_current_partition.level =
+ static_cast<int>(source_operator->_repartition_max_depth) - 1;
+ EXPECT_GT(source_operator->revocable_mem_size(_helper.runtime_state.get()), 0UL);
+}
Review Comment:
The boundary levels asserted here don’t match the operator’s repartition limit: `_flush_and_repartition` fails when `new_level = level + 1 >= _repartition_max_depth`, so `level == _repartition_max_depth - 1` is already at the maximum depth and should be treated as non-revocable. Setting `level == _repartition_max_depth` is likely an unreachable state, so the test should instead cover `max_depth - 1` (expect 0) vs `max_depth - 2` (expect >0), or configure the runtime max depth to exercise the edge case deterministically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #61973:
URL: https://github.com/apache/doris/pull/61973#discussion_r3018899476
##########
be/src/exec/operator/partitioned_hash_join_probe_operator.cpp:
##########
@@ -875,6 +875,11 @@ size_t PartitionedHashJoinProbeOperatorX::revocable_mem_size(RuntimeState* state
// Or if current partition has finished build hash table.
return 0;
}
+ // If the current partition has reached the max repartition depth, it cannot be
+ // repartitioned further, so its data is not revocable.
+ if (local_state._current_partition.level >= _repartition_max_depth) {
Review Comment:
This has the same off-by-one as the aggregation path. `revoke_memory()` here goes through `repartition_current_partition()`, which errors once `partition.level + 1 >= _repartition_max_depth`. That means `level == _repartition_max_depth - 1` is already non-revocable, but this guard still reports the recovered build block as revocable and can drive the query into `InternalError`. Please align the cutoff with the actual repartition failure boundary, and update the new UT to exercise `max_depth - 1` as the zero case.
```suggestion
if (local_state._current_partition.level >= _repartition_max_depth - 1) {
return 0;
}
```
##########
be/src/exec/operator/partitioned_aggregation_source_operator.cpp:
##########
@@ -191,6 +191,11 @@ size_t PartitionedAggSourceOperatorX::revocable_mem_size(RuntimeState* state) co
if (!local_state._shared_state->_is_spilled || !local_state._current_partition.spill_file) {
return 0;
}
+ // If the current partition has reached the max repartition depth, it cannot be
+ // repartitioned further, so its data is not revocable.
+ if (local_state._current_partition.level >= _repartition_max_depth) {
Review Comment:
This guard is one level too late. `revoke_memory()` eventually calls `_flush_and_repartition()`, which computes `new_level = current_level + 1` and fails once `new_level >= _repartition_max_depth`. So a partition at `level == _repartition_max_depth - 1` is already non-revocable: reporting it here lets the scheduler try to revoke it and hit `InternalError` instead of finishing the partition in memory. The cutoff needs to match that failure boundary, and the new UT should assert on `max_depth - 1` rather than `max_depth`.
```suggestion
if (local_state._current_partition.level >= _repartition_max_depth - 1) {
return 0;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4166331081
<details>
<summary>TPC-H: <b>Total hot run time: 26709 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 10762a61257fedb8ff2899f11dd247ebedb57d1e, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17595 4532 4335 4335
q2 q3 10649 784 529 529
q4 4678 347 249 249
q5 7553 1203 1050 1050
q6 176 174 151 151
q7 767 851 681 681
q8 9415 1466 1344 1344
q9 4917 4719 4720 4719
q10 6339 1912 1638 1638
q11 502 254 259 254
q12 756 576 462 462
q13 18051 2708 1924 1924
q14 223 233 212 212
q15 q16 755 750 679 679
q17 743 849 433 433
q18 6168 5487 5338 5338
q19 1106 976 611 611
q20 537 484 389 389
q21 4544 1847 1413 1413
q22 417 448 298 298
Total cold run time: 95891 ms
Total hot run time: 26709 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4767 4616 4612 4612
q2 q3 3896 4369 3886 3886
q4 884 1227 797 797
q5 4168 4448 4347 4347
q6 181 175 144 144
q7 1749 1678 1581 1581
q8 2560 2738 2589 2589
q9 7543 7502 7517 7502
q10 3745 4004 3606 3606
q11 515 435 415 415
q12 506 589 459 459
q13 2468 3058 2029 2029
q14 280 296 277 277
q15 q16 709 771 737 737
q17 1168 1297 1325 1297
q18 7165 6794 6979 6794
q19 946 946 938 938
q20 2078 2225 2040 2040
q21 3984 3485 3337 3337
q22 453 418 400 400
Total cold run time: 49765 ms
Total hot run time: 47787 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4167095358
<details>
<summary>TPC-DS: <b>Total hot run time: 169638 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e15e7bcefdf1c0bb12476945aa8b499c47000817, data reload: false
query5 4351 614 506 506
query6 326 219 201 201
query7 4231 474 261 261
query8 339 236 223 223
query9 8718 2760 2720 2720
query10 510 412 348 348
query11 6966 5116 4886 4886
query12 184 129 133 129
query13 1276 459 344 344
query14 5710 3760 3451 3451
query14_1 2870 2856 2842 2842
query15 204 199 176 176
query16 961 483 447 447
query17 1114 741 628 628
query18 2453 462 366 366
query19 238 210 171 171
query20 129 124 123 123
query21 212 131 106 106
query22 13248 14542 14425 14425
query23 16723 16574 16025 16025
query23_1 16238 16080 16062 16062
query24 7217 1589 1201 1201
query24_1 1220 1224 1229 1224
query25 552 454 411 411
query26 1240 261 141 141
query27 2786 496 297 297
query28 4424 1829 1864 1829
query29 820 558 473 473
query30 300 228 190 190
query31 1010 930 867 867
query32 84 73 71 71
query33 521 344 302 302
query34 888 877 540 540
query35 647 691 624 624
query36 1097 1163 994 994
query37 133 98 84 84
query38 2979 2937 2863 2863
query39 874 842 806 806
query39_1 809 799 799 799
query40 240 157 143 143
query41 71 65 66 65
query42 260 259 258 258
query43 245 244 218 218
query44
query45 217 198 185 185
query46 898 987 620 620
query47 2164 2656 2047 2047
query48 323 327 253 253
query49 646 467 399 399
query50 694 319 222 222
query51 4093 4052 4042 4042
query52 264 271 256 256
query53 297 345 287 287
query54 313 282 280 280
query55 97 86 86 86
query56 323 341 325 325
query57 1935 1809 1669 1669
query58 294 288 277 277
query59 2829 2977 2738 2738
query60 350 353 330 330
query61 178 180 212 180
query62 626 573 528 528
query63 307 278 277 277
query64 5055 1256 1009 1009
query65
query66 1463 454 366 366
query67 24228 24228 24133 24133
query68
query69 420 325 291 291
query70 973 970 966 966
query71 335 310 296 296
query72 2814 2629 2459 2459
query73 556 546 318 318
query74 9631 9594 9414 9414
query75 2875 2752 2446 2446
query76 2282 1021 682 682
query77 364 389 306 306
query78 10951 11150 10483 10483
query79 1501 764 578 578
query80 1350 618 541 541
query81 551 267 222 222
query82 1005 155 119 119
query83 327 267 237 237
query84 252 119 106 106
query85 911 497 451 451
query86 409 315 304 304
query87 3152 3154 2969 2969
query88 3576 2691 2647 2647
query89 422 366 341 341
query90 2032 185 178 178
query91 174 163 139 139
query92 78 76 74 74
query93 982 873 510 510
query94 629 327 285 285
query95 592 336 380 336
query96 646 514 227 227
query97 2477 2546 2396 2396
query98 237 228 222 222
query99 999 1010 903 903
Total cold run time: 250834 ms
Total hot run time: 169638 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4163503999
<details>
<summary>TPC-DS: <b>Total hot run time: 168739 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6ca13dfc46d77d35a9f851b2ddbdceaf0bb59e88, data reload: false
query5 4332 635 511 511
query6 332 226 202 202
query7 4223 461 266 266
query8 352 244 221 221
query9 8683 2723 2722 2722
query10 496 387 332 332
query11 6943 5129 4888 4888
query12 195 130 122 122
query13 1283 462 341 341
query14 5760 3713 3489 3489
query14_1 2884 2800 2806 2800
query15 205 193 179 179
query16 1013 470 463 463
query17 1115 731 623 623
query18 2463 471 340 340
query19 221 199 179 179
query20 133 123 124 123
query21 209 132 109 109
query22 13298 13306 13142 13142
query23 16243 15742 16294 15742
query23_1 16056 16364 16330 16330
query24 7874 1678 1277 1277
query24_1 1239 1260 1262 1260
query25 564 471 397 397
query26 1232 262 149 149
query27 2794 486 300 300
query28 4476 1831 1861 1831
query29 834 565 500 500
query30 298 226 191 191
query31 995 962 868 868
query32 87 71 71 71
query33 500 332 288 288
query34 895 861 512 512
query35 640 680 614 614
query36 1111 1143 1001 1001
query37 131 94 85 85
query38 2898 2935 2914 2914
query39 847 839 818 818
query39_1 800 798 793 793
query40 227 146 132 132
query41 62 61 66 61
query42 255 254 257 254
query43 236 247 213 213
query44
query45 198 191 181 181
query46 885 984 607 607
query47 2108 2546 2064 2064
query48 335 314 247 247
query49 633 452 375 375
query50 708 284 225 225
query51 4086 4096 3996 3996
query52 257 267 247 247
query53 289 350 286 286
query54 293 269 259 259
query55 85 90 83 83
query56 309 331 314 314
query57 1820 1856 1712 1712
query58 286 277 272 272
query59 2797 2987 2752 2752
query60 340 339 327 327
query61 160 154 156 154
query62 640 604 506 506
query63 304 276 272 272
query64 5078 1295 1038 1038
query65
query66 1457 455 362 362
query67 24349 24295 24295 24295
query68
query69 401 323 310 310
query70 945 920 1001 920
query71 340 287 298 287
query72 2806 2964 2567 2567
query73 548 545 315 315
query74 9614 9520 9416 9416
query75 2876 2817 2504 2504
query76 2286 1028 683 683
query77 367 389 317 317
query78 10956 11098 10485 10485
query79 1140 786 582 582
query80 1303 685 535 535
query81 527 273 230 230
query82 1351 152 116 116
query83 362 259 236 236
query84 253 109 92 92
query85 983 506 458 458
query86 407 305 299 299
query87 3151 3167 3012 3012
query88 3527 2647 2663 2647
query89 420 379 345 345
query90 1828 174 174 174
query91 168 168 143 143
query92 79 69 71 69
query93 900 878 496 496
query94 554 324 289 289
query95 594 390 317 317
query96 652 516 231 231
query97 2459 2479 2396 2396
query98 233 235 226 226
query99 1017 982 911 911
Total cold run time: 250344 ms
Total hot run time: 168739 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4166188191
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4163202092
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg merged PR #61973:
URL: https://github.com/apache/doris/pull/61973
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4168183468
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61973_e15e7bcefdf1c0bb12476945aa8b499c47000817_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61973_e15e7bcefdf1c0bb12476945aa8b499c47000817_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.58% (27219/36992) |
| Line Coverage | 57.15% (293017/512677) |
| Region Coverage | 54.27% (243425/448535) |
| Branch Coverage | 56.09% (105848/188697) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4167414370
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61973_e15e7bcefdf1c0bb12476945aa8b499c47000817_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61973_e15e7bcefdf1c0bb12476945aa8b499c47000817_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.58% (27219/36992) |
| Line Coverage | 57.15% (293017/512677) |
| Region Coverage | 54.27% (243425/448535) |
| Branch Coverage | 56.09% (105848/188697) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4166954156
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4181286501
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg closed pull request #61973: [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth
URL: https://github.com/apache/doris/pull/61973
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4167203078
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/e15e7bcefdf1c0bb12476945aa8b499c47000817_e15e7bcefdf1c0bb12476945aa8b499c47000817/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/e15e7bcefdf1c0bb12476945aa8b499c47000817_e15e7bcefdf1c0bb12476945aa8b499c47000817/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.93% (19992/37773) |
| Line Coverage | 36.49% (187622/514221) |
| Region Coverage | 32.71% (145369/444417) |
| Branch Coverage | 33.90% (63780/188131) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4163820464
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/6ca13dfc46d77d35a9f851b2ddbdceaf0bb59e88_6ca13dfc46d77d35a9f851b2ddbdceaf0bb59e88/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/6ca13dfc46d77d35a9f851b2ddbdceaf0bb59e88_6ca13dfc46d77d35a9f851b2ddbdceaf0bb59e88/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.97% (20004/37768) |
| Line Coverage | 36.48% (187694/514442) |
| Region Coverage | 32.71% (145398/444497) |
| Branch Coverage | 33.92% (63844/188223) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4166384113
<details>
<summary>TPC-DS: <b>Total hot run time: 169559 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 10762a61257fedb8ff2899f11dd247ebedb57d1e, data reload: false
query5 4333 653 524 524
query6 332 228 226 226
query7 4219 494 264 264
query8 347 243 229 229
query9 8722 2761 2758 2758
query10 503 393 333 333
query11 6889 5099 4870 4870
query12 185 126 123 123
query13 1280 469 352 352
query14 5577 3707 3473 3473
query14_1 2926 2872 2857 2857
query15 213 196 177 177
query16 978 509 439 439
query17 857 693 604 604
query18 2447 439 335 335
query19 218 211 177 177
query20 127 124 125 124
query21 211 137 109 109
query22 13197 14118 14659 14118
query23 16759 16638 16086 16086
query23_1 16151 16065 15730 15730
query24 7310 1628 1205 1205
query24_1 1233 1272 1239 1239
query25 544 510 399 399
query26 1237 259 146 146
query27 2786 485 296 296
query28 4505 1857 1833 1833
query29 861 575 488 488
query30 293 226 190 190
query31 1015 952 878 878
query32 83 73 72 72
query33 507 334 293 293
query34 889 866 539 539
query35 635 680 608 608
query36 1062 1100 978 978
query37 127 93 88 88
query38 2906 2921 2934 2921
query39 851 841 802 802
query39_1 802 789 811 789
query40 232 154 134 134
query41 62 59 61 59
query42 265 262 260 260
query43 254 263 225 225
query44
query45 198 185 183 183
query46 891 980 604 604
query47 2128 2152 2085 2085
query48 315 316 231 231
query49 635 462 389 389
query50 700 286 209 209
query51 4115 4089 4056 4056
query52 264 267 253 253
query53 293 349 296 296
query54 309 269 280 269
query55 94 91 86 86
query56 320 323 317 317
query57 1921 1831 1739 1739
query58 281 276 270 270
query59 2809 2931 2770 2770
query60 353 355 341 341
query61 184 187 181 181
query62 652 595 544 544
query63 314 292 282 282
query64 5259 1393 1129 1129
query65
query66 1484 468 376 376
query67 24220 24270 24216 24216
query68
query69 424 327 299 299
query70 983 964 1000 964
query71 341 318 304 304
query72 3045 2694 2451 2451
query73 539 545 338 338
query74 9581 9591 9448 9448
query75 2874 2753 2459 2459
query76 2295 1044 690 690
query77 353 374 310 310
query78 10867 11015 10449 10449
query79 3060 802 571 571
query80 1806 633 547 547
query81 583 256 223 223
query82 997 161 121 121
query83 341 289 241 241
query84 304 121 102 102
query85 911 504 459 459
query86 494 298 293 293
query87 3168 3083 2992 2992
query88 3605 2686 2670 2670
query89 425 372 340 340
query90 2153 183 178 178
query91 172 165 141 141
query92 85 74 69 69
query93 1808 853 497 497
query94 646 326 297 297
query95 583 408 325 325
query96 640 513 230 230
query97 2450 2471 2428 2428
query98 236 221 221 221
query99 1004 968 903 903
Total cold run time: 253972 ms
Total hot run time: 169559 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](spill) Fix revocable_mem_size to return 0 when partition reaches max repartition depth [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61973:
URL: https://github.com/apache/doris/pull/61973#issuecomment-4167062618
<details>
<summary>TPC-H: <b>Total hot run time: 26332 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e15e7bcefdf1c0bb12476945aa8b499c47000817, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17656 4402 4254 4254
q2 q3 10673 775 510 510
q4 4677 370 254 254
q5 7556 1220 1021 1021
q6 174 177 148 148
q7 797 843 669 669
q8 9295 1475 1314 1314
q9 4757 4740 4710 4710
q10 6301 1940 1640 1640
q11 455 262 237 237
q12 735 591 454 454
q13 18033 2674 1924 1924
q14 233 225 206 206
q15 q16 738 739 661 661
q17 736 843 420 420
q18 5833 5447 5293 5293
q19 1099 988 604 604
q20 532 506 373 373
q21 4517 1810 1395 1395
q22 343 282 245 245
Total cold run time: 95140 ms
Total hot run time: 26332 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4435 4336 4349 4336
q2 q3 3832 4270 3797 3797
q4 857 1164 750 750
q5 4014 4303 4309 4303
q6 175 180 140 140
q7 1762 1623 1517 1517
q8 2444 2650 2519 2519
q9 7559 7609 7563 7563
q10 3773 4004 3624 3624
q11 493 436 412 412
q12 480 635 491 491
q13 2549 2944 2116 2116
q14 298 295 277 277
q15 q16 743 769 738 738
q17 1155 1459 1408 1408
q18 7182 6682 6582 6582
q19 898 861 926 861
q20 2112 2160 2051 2051
q21 3943 3531 3329 3329
q22 502 534 400 400
Total cold run time: 49206 ms
Total hot run time: 47214 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org