You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "zhouyuan (via GitHub)" <gi...@apache.org> on 2024/08/15 01:08:15 UTC

Re: [I] [VL] TPC-DS 30TB regression on 1.2-rc1 vs 1.1 on hash operations [incubator-gluten]

zhouyuan commented on issue #6789:
URL: https://github.com/apache/incubator-gluten/issues/6789#issuecomment-2290201093

   @estoianovici 
   The hash build allocates more memory(783G vs 1157G) although the number of input rows are the same. We have run into such issue due to huge page based allocator, but it should be fixed in the recent release. Can you please check if below config is enabled?
   
   `spark.gluten.sql.columnar.backend.velox.memoryUseHugePages`
   https://github.com/apache/incubator-gluten/pull/3483
   
   thanks, -yuan


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org