[GitHub] [incubator-uniffle] jerqi commented on pull request #294: [Improvement] Skip blocks when read from memory

[GitBox <gi...@apache.org>]

jerqi commented on PR #294:
URL: https://github.com/apache/incubator-uniffle/pull/294#issuecomment-1327001018

   > > I think in most cases, we only need to read from one replica, so i think we can give priority to AQE.
   > > > 100w tasks will occupy 125k memory.
   > > 
   > > 
   > > And i think this is acceptable and `taskIdBitmap` is more precision for multiple replicas. @jerqi @zuston
   > 
   > It's OK for me. I think we could disable this taskIdBitmap filter in no-AQE optimization. And especially for AQE, the taskIds size for one reader should not be large.
   > 
   > What do u think so? @jerqi @xianjingfeng
   
   We should only pass the taskBitmap when the reduce task has skew data.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org