You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "akupchinskiy (via GitHub)" <gi...@apache.org> on 2025/06/26 06:20:00 UTC
[PR] [SPARK-50686][SQL] Hash to sort aggregation fallback - memory usage optimization [spark]
akupchinskiy opened a new pull request, #51290:
URL: https://github.com/apache/spark/pull/51290
What changes were proposed in this pull request?
Added deallocation of internal array which is redundant in certain hash to sort aggregation fallback scenarios
Why are the changes needed?
Under memory pressure, allocation of new internal structures is oom-prone. So, it is better to get rid of all the redundant structures at that point to reduce the risk.
Does this PR introduce any user-facing change?
No
How was this patch tested?
UnsafeKVExternalSuite spilling tests have been adjusted to the reduced spilled memory introduced by the change.
Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50686][SQL] Hash to sort aggregation fallback - memory usage optimization [spark]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #51290:
URL: https://github.com/apache/spark/pull/51290#issuecomment-3368630063
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50686][SQL] Hash to sort aggregation fallback - memory usage optimization [spark]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #51290: [SPARK-50686][SQL] Hash to sort aggregation fallback - memory usage optimization
URL: https://github.com/apache/spark/pull/51290
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org