You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "viirya (via GitHub)" <gi...@apache.org> on 2024/04/05 05:36:46 UTC

[I] Support SubqueryBroadcastExec in Comet [arrow-datafusion-comet]

viirya opened a new issue, #242:
URL: https://github.com/apache/arrow-datafusion-comet/issues/242

   ### What is the problem the feature request solves?
   
   Currently we support `BroadcastExchange` if it is under `BroadcastHashJoin` in Comet.
   
   Besides broadcast join, `BroadcastExchange ` can also occur under `SubqueryBroadcastExec` for dynamic pruning expression. Because Spark `BroadcastExchange` assumes it is row-based so we cannot simply transform `BroadcastExchange` under `SubqueryBroadcastExec` into Comet's broadcast operator.
   
   We need to come out with a Comet `SubqueryBroadcastExec` which supports Comet's `BroadcastExchange`.
   
   ### Describe the potential solution
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Support SubqueryBroadcastExec in Comet [datafusion-comet]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich closed issue #242: Support SubqueryBroadcastExec in Comet
URL: https://github.com/apache/datafusion-comet/issues/242


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] Support SubqueryBroadcastExec in Comet [datafusion-comet]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich closed issue #242: Support SubqueryBroadcastExec in Comet
URL: https://github.com/apache/datafusion-comet/issues/242


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] Support SubqueryBroadcastExec in Comet [datafusion-comet]

Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #242:
URL: https://github.com/apache/datafusion-comet/issues/242#issuecomment-4361219221

   we still have some Spark SQL tests ignored and linked to this issue so we should update those tests before closing. I will do this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] Support SubqueryBroadcastExec in Comet [datafusion-comet]

Posted by "ganeshkumar269 (via GitHub)" <gi...@apache.org>.
ganeshkumar269 commented on issue #242:
URL: https://github.com/apache/datafusion-comet/issues/242#issuecomment-2108280414

   Hi @viirya, I would like to work on this. Seems like this might require some knowledge of spark codebase aswell, nevertheless will give it a shot, your guidance will be helpful ๐Ÿ™๐Ÿพ 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] Support SubqueryBroadcastExec in Comet [datafusion-comet]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on issue #242:
URL: https://github.com/apache/datafusion-comet/issues/242#issuecomment-4361430626

   > we still have some Spark SQL tests ignored and linked to this issue so we should update those tests before closing. I will do this.
   
   My bad. I thought I got all of the subquery/DPP filter related Spark SQL tests. I can take a look on Monday if you donโ€™t beat me to it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] Support SubqueryBroadcastExec in Comet [datafusion-comet]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on issue #242:
URL: https://github.com/apache/datafusion-comet/issues/242#issuecomment-4682998908

   Closed by #4178. I don't see any references to this issue in the codebase anymore and I think we're fully exercising this now in Spark SQL tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org