You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "FelixYBW (via GitHub)" <gi...@apache.org> on 2026/07/30 06:32:12 UTC
[I] Gluten consumes much higher memory and created much larger bloomfilter subquery datasize than Spark [gluten]
FelixYBW opened a new issue, #12659:
URL: https://github.com/apache/gluten/issues/12659
### Backend
VL (Velox)
### Bug description
Noted as a known issue here. The issue happens when you set **large creationSideThreshold, maxNumBits, and maxNumItems with CBO enabled**.
When CBO (Cost-Based Optimizer) is disabled, Spark creates a Bloom filter subquery based on creationSideThreshold, falling back to default values for estimatedNumItems and numBits. The query successfully runs with partial_bloom_filter_agg(expr, estimatedNumItems, numBits, 0, 0).
However, when CBO is enabled, Spark estimates estimatedNumItems and numBits dynamically. If these CBO-estimated values become exceptionally large, partial_velox_bloom_filter_agg will consume significantly more memory and generate a much larger Bloom filter compared to native Spark under the exact same configuration.
This discrepancy leads to the following cascading failures:
Gluten throws a spark.rpc.message.maxSize error due to the massive payload size.
Even if spark.rpc.message.maxSize is significantly increased, the subsequent velox_bloom_filter_agg operator eventually throws an OOM (Out of Memory) error.
e.g. if we test TPCDS SF1T, the ```partial_bloom_filter_agg(xxhash64(wr_order_number#65L, 42), 72002305, 739372223, 0, 0)``` can pass on Spark but OOM on gluten.
The shuffle data size on Spark is only **69.4M**, while on Gluten it's **9.3 GiB**!
cc @jinchengchenghh
### Gluten version
_No response_
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Gluten consumes much higher memory and created much larger bloomfilter subquery datasize than Spark [gluten]
Posted by "ReemaAlzaid (via GitHub)" <gi...@apache.org>.
ReemaAlzaid commented on issue #12659:
URL: https://github.com/apache/gluten/issues/12659#issuecomment-5415821846
Hey @FelixYBW could I take a look into this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Gluten consumes much higher memory and created much larger bloomfilter subquery datasize than Spark [gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on issue #12659:
URL: https://github.com/apache/gluten/issues/12659#issuecomment-5416189040
Thank you, @ReemaAlzaid. It's low priority. Currently we configure creationSideThreshold as 200M which does consume more memory than Spark but still acceptable.
Meanwhile Velox introduced new BF algorithm, @zhztheplayer is working on that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org