You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "rui-mo (via GitHub)" <gi...@apache.org> on 2025/12/04 06:35:08 UTC
[PR] [GLUTEN-9456][VL] Release Velox memory manager when the reserved bytes becomes empty [incubator-gluten]
rui-mo opened a new pull request, #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249
<!--
Thank you for submitting a pull request! Here are some tips:
1. For first-time contributors, please read our contributing guide:
https://github.com/apache/incubator-gluten/blob/main/CONTRIBUTING.md
2. If necessary, create a GitHub issue for discussion beforehand to avoid duplicate work.
3. If the PR is specific to a single backend, include [VL] or [CH] in the PR title to indicate the
Velox or ClickHouse backend, respectively.
4. If the PR is not ready for review, please mark it as a draft.
-->
## What changes are proposed in this pull request?
<!--
Provide a clear and concise description of the changes introduced in this PR.
Ensure the PR description aligns with the code changes, especially after updates.
If applicable, include "Fixes #<GitHub_Issue_ID>" to automatically close the corresponding issue
when the PR is merged.
-->
## How was this patch tested?
<!--
Describe how the changes were tested, if applicable.
Include new tests to validate the functionality, if necessary.
For UI-related changes, attach screenshots to demonstrate the updates.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#discussion_r2588561901
##########
cpp/velox/memory/VeloxMemoryManager.cc:
##########
@@ -445,27 +445,11 @@ bool VeloxMemoryManager::tryDestructSafe() {
}
VeloxMemoryManager::~VeloxMemoryManager() {
- static const uint32_t kWaitTimeoutMs = FLAGS_gluten_velox_async_timeout_on_task_stopping; // 30s by default
- uint32_t accumulatedWaitMs = 0UL;
bool destructed = false;
- for (int32_t tryCount = 0; accumulatedWaitMs < kWaitTimeoutMs; tryCount++) {
Review Comment:
> This appears to be a different solution from the ones we previously discussed.
Just to clarify a bit, `Task::requestCancel` is not a different solution but something we already called and expected it could get reliable gradually in Velox:
https://github.com/apache/incubator-gluten/blob/48efe26923f8e8148d9e3812b29343c6f6d398b2/cpp/velox/compute/WholeStageResultIterator.h#L51
We don't need this wait mechanism in memory manager if `Task::requestCancel` works smoothly. Memory management is a sub-optimal place to determine whether the task is completely ended or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on code in PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#discussion_r2589391954
##########
cpp/velox/memory/VeloxMemoryManager.cc:
##########
@@ -445,27 +445,11 @@ bool VeloxMemoryManager::tryDestructSafe() {
}
VeloxMemoryManager::~VeloxMemoryManager() {
- static const uint32_t kWaitTimeoutMs = FLAGS_gluten_velox_async_timeout_on_task_stopping; // 30s by default
- uint32_t accumulatedWaitMs = 0UL;
bool destructed = false;
- for (int32_t tryCount = 0; accumulatedWaitMs < kWaitTimeoutMs; tryCount++) {
Review Comment:
request cancel should be used for different case. In this case, the task finishes normally, not is cancelled.
8025's close() is the most elegant way to close the pending I/O request but it changes too much velox code and even hard to get merged. In theory velox has a close() for each operator which should free all resources the operator holds.
14722 relys on the class descrution to close the resource. It's easier for us to pick.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW merged PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#discussion_r2588578548
##########
cpp/velox/memory/VeloxMemoryManager.cc:
##########
@@ -445,27 +445,11 @@ bool VeloxMemoryManager::tryDestructSafe() {
}
VeloxMemoryManager::~VeloxMemoryManager() {
- static const uint32_t kWaitTimeoutMs = FLAGS_gluten_velox_async_timeout_on_task_stopping; // 30s by default
- uint32_t accumulatedWaitMs = 0UL;
bool destructed = false;
- for (int32_t tryCount = 0; accumulatedWaitMs < kWaitTimeoutMs; tryCount++) {
Review Comment:
> does it cancel the async load
It should have them all closed as long as they are associated with the current Velox task. But apparently that's not happening so we are encountering the leak / segfault issue and now have to optimize around the memory manager destructor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo commented on code in PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#discussion_r2588535497
##########
cpp/velox/memory/VeloxMemoryManager.cc:
##########
@@ -445,27 +445,11 @@ bool VeloxMemoryManager::tryDestructSafe() {
}
VeloxMemoryManager::~VeloxMemoryManager() {
- static const uint32_t kWaitTimeoutMs = FLAGS_gluten_velox_async_timeout_on_task_stopping; // 30s by default
- uint32_t accumulatedWaitMs = 0UL;
bool destructed = false;
- for (int32_t tryCount = 0; accumulatedWaitMs < kWaitTimeoutMs; tryCount++) {
Review Comment:
> The final goal will be removing this wait mechanism and rely on `Task::requestCancel` instead.
This appears to be a different solution from the ones we previously discussed. Could you provide a bit more detail? Specifically, does it cancel the async load, and should we invoke it from Gluten, or are we relying on Velox’s task cancellation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reserved bytes becomes empty [incubator-gluten]
Posted by "xinghuayu007 (via GitHub)" <gi...@apache.org>.
xinghuayu007 commented on PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#issuecomment-3610711553
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#issuecomment-3611229018
Thank you, @rui-mo . It's a good summary of the memory leak issue we analyzed so far. We noted the issue in this https://github.com/facebookincubator/velox/pull/8205 but it's not merged and we work around it in Gluten by adding the wait time during memory manager close.
With current upstream velox, there are two issues:
1. The risk of segfault leading by 4) in Rui's comment.
2. Then pending I/O is executed any way even we doesn't need the data at all.
What we should do and PR14722 does is to wait the current I/O request to finish and close the pending I/O request. But it may cause Prestoc++ deadlock. So PR14722 can hardly be merged into upstream. Instead we will add a factory to create customized input in Gluten which use PR14722.
We temporarily picked PR14722 into gluten/velox now. With this PR we should solve all the I/O thread memleak issue totally. Let's know if you noted more failure.
In theory with PR14722 we needn't the wait in the PR, we still use it for other potential memleaks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo commented on PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#issuecomment-3611425503
> We temporarily picked PR14722 into gluten/velox now.
@FelixYBW It seems the base branch does not contain this PR: https://github.com/IBM/velox/tree/oss-2025_12_03.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#discussion_r2588578548
##########
cpp/velox/memory/VeloxMemoryManager.cc:
##########
@@ -445,27 +445,11 @@ bool VeloxMemoryManager::tryDestructSafe() {
}
VeloxMemoryManager::~VeloxMemoryManager() {
- static const uint32_t kWaitTimeoutMs = FLAGS_gluten_velox_async_timeout_on_task_stopping; // 30s by default
- uint32_t accumulatedWaitMs = 0UL;
bool destructed = false;
- for (int32_t tryCount = 0; accumulatedWaitMs < kWaitTimeoutMs; tryCount++) {
Review Comment:
> does it cancel the async load
It should have them all closed as long as they are associated with the current Velox task. But apparently that's not happening so we are encountering the leak / segfault issue now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reserved bytes becomes empty [incubator-gluten]
Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo commented on PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#issuecomment-3610750697
cc: @j7nhai @liujiayi771
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on code in PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#discussion_r2588409378
##########
cpp/velox/memory/VeloxMemoryManager.cc:
##########
@@ -445,27 +445,11 @@ bool VeloxMemoryManager::tryDestructSafe() {
}
VeloxMemoryManager::~VeloxMemoryManager() {
- static const uint32_t kWaitTimeoutMs = FLAGS_gluten_velox_async_timeout_on_task_stopping; // 30s by default
- uint32_t accumulatedWaitMs = 0UL;
bool destructed = false;
- for (int32_t tryCount = 0; accumulatedWaitMs < kWaitTimeoutMs; tryCount++) {
Review Comment:
@rui-mo we still need the wait time. Customer can config it and use spark.unsafe.exceptionOnMemoryLeak to detect memleaks. Without the config, the task will wait here always on real memleaks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo commented on code in PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#discussion_r2589042375
##########
cpp/velox/memory/VeloxMemoryManager.cc:
##########
@@ -445,27 +445,11 @@ bool VeloxMemoryManager::tryDestructSafe() {
}
VeloxMemoryManager::~VeloxMemoryManager() {
- static const uint32_t kWaitTimeoutMs = FLAGS_gluten_velox_async_timeout_on_task_stopping; // 30s by default
- uint32_t accumulatedWaitMs = 0UL;
bool destructed = false;
- for (int32_t tryCount = 0; accumulatedWaitMs < kWaitTimeoutMs; tryCount++) {
Review Comment:
@zhztheplayer Is this patch (https://github.com/facebookincubator/velox/commit/059ffcbeac2523a5eade65007adf0319829a7c4e) intended for the purpose you mentioned? I noticed the calling of cancel would do nothing right now. If we proceed this way, it would differ a bit from importing https://github.com/facebookincubator/velox/pull/14722 or customizing buffered input in Gluten. However, the fix would become more like https://github.com/facebookincubator/velox/pull/8205, which releases the load through table scan's close.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo commented on PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#issuecomment-3610768684
cc: @FelixYBW @zhztheplayer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#issuecomment-3611305314
spark.unsafe.exceptionOnMemoryLeak is a spark config which can cause spark job failure on memleak.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#discussion_r2588500876
##########
cpp/velox/memory/VeloxMemoryManager.cc:
##########
@@ -445,27 +445,11 @@ bool VeloxMemoryManager::tryDestructSafe() {
}
VeloxMemoryManager::~VeloxMemoryManager() {
- static const uint32_t kWaitTimeoutMs = FLAGS_gluten_velox_async_timeout_on_task_stopping; // 30s by default
- uint32_t accumulatedWaitMs = 0UL;
bool destructed = false;
- for (int32_t tryCount = 0; accumulatedWaitMs < kWaitTimeoutMs; tryCount++) {
Review Comment:
Same comment here. We'd maintain a timeout so we won't hang forever by a remaining allocation in Velox.
The final goal will be removing this wait mechanism and rely on `Task::requestCancel` instead. So feel free to add a code comment marking this is a workaround against an Issue from Velox if the lines might confuse you already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#issuecomment-3612642926
> @FelixYBW It seems the base branch hasn't included this PR: [IBM/velox@`oss-2025_12_03`](https://github.com/IBM/velox/tree/oss-2025_12_03).
Oh, we pick into this branch:https://github.com/IBM/velox/commits/ibm-2025_12_03.
Let's move to OAP
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#discussion_r2591074396
##########
cpp/velox/memory/VeloxMemoryManager.cc:
##########
@@ -445,27 +445,11 @@ bool VeloxMemoryManager::tryDestructSafe() {
}
VeloxMemoryManager::~VeloxMemoryManager() {
- static const uint32_t kWaitTimeoutMs = FLAGS_gluten_velox_async_timeout_on_task_stopping; // 30s by default
- uint32_t accumulatedWaitMs = 0UL;
bool destructed = false;
- for (int32_t tryCount = 0; accumulatedWaitMs < kWaitTimeoutMs; tryCount++) {
Review Comment:
By close / cancel (Velox PR 8205 / 14722 included) I think we are talking about the same concept - to end the computation and release the resources before the Velox task can be destructed. You can consider `requestCancel` a way to ensure all the Velox `close` APIs are called. So I guess we are already on the same page.
It was just worth noting this wait logic in `~VeloxMemoryManager` is only a temporary solution for keeping the pools alive in case the operators were halted too early without completing all the async operations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-9456][VL] Release Velox memory manager when the reservedBytes become empty [incubator-gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on code in PR #11249:
URL: https://github.com/apache/incubator-gluten/pull/11249#discussion_r2596746719
##########
cpp/velox/memory/VeloxMemoryManager.cc:
##########
@@ -445,27 +445,11 @@ bool VeloxMemoryManager::tryDestructSafe() {
}
VeloxMemoryManager::~VeloxMemoryManager() {
- static const uint32_t kWaitTimeoutMs = FLAGS_gluten_velox_async_timeout_on_task_stopping; // 30s by default
- uint32_t accumulatedWaitMs = 0UL;
bool destructed = false;
- for (int32_t tryCount = 0; accumulatedWaitMs < kWaitTimeoutMs; tryCount++) {
Review Comment:
The problem is that not all of resource related class has the `close` API, which should be designed initially. PR8205 is trying to solve the table scan chain but it can be hard to merged.
Waiting in `~VeloxMemoryManager` here is still useful because we only solve the table scan's async IO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org