You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by "mingmwang (via GitHub)" <gi...@apache.org> on 2023/02/08 07:39:29 UTC

[GitHub] [arrow-ballista] mingmwang commented on issue #619: Prune unneccessary data from task definition

mingmwang commented on issue #619:
URL: https://github.com/apache/arrow-ballista/issues/619#issuecomment-1422165090

   I think there is prons and cons. The good part is that each partition/task will have a relatively small plan to deserialize, especially if the SQL include lots of UNION branches. The bad part is that, each partition/task will have different plan bytes.  the Scheduler will have to serialize the different plan multiple times and can not be shared in the Stage level.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org