You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "viirya (via GitHub)" <gi...@apache.org> on 2024/03/10 21:13:45 UTC

[I] ShuffleQueryStage and ReusedExchange are not processed by CometExecRule [arrow-datafusion-comet]

viirya opened a new issue, #185:
URL: https://github.com/apache/arrow-datafusion-comet/issues/185

   ### Describe the bug
   
   I found that Sort in some TPCDS queries are not transformed to CometSort, e.g.,
   
   ```
   +- * SortMergeJoin Inner (19)
      :- * Sort (14)
      :  +- * ColumnarToRow (13)
      :     +- ShuffleQueryStage (12), Statistics(sizeInBytes=16.1 GiB, rowCount=7.20E+8)
      :        +- CometColumnarExchange (11)
      :          +- CometFilter (10)
      :            +- CometBatchScan iceberg.1000g_parquet_zstd_512mb_block_partitioned_iceberg.web_sales (9)
      : - * Sort (18)
         +- * ColumnarToRow (17)
           +- ShuffleQueryStage (16), Statistics(sizeInBytes=16.1 GiB, rowCount=7.20E+8)
             +- ReusedExchange (15)
   ```
   
   It is blocked by `ShuffleQueryStage` and/or `ReusedExchange` which `CometExecRule` doesn't handle currently.
   
   We should handle these operators in `CometExecRule`.
   
   ### Steps to reproduce
   
   _No response_
   
   ### Expected behavior
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] ShuffleQueryStage and ReusedExchange are not processed by CometExecRule [arrow-datafusion-comet]

Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya closed issue #185: ShuffleQueryStage and ReusedExchange are not processed by CometExecRule
URL: https://github.com/apache/arrow-datafusion-comet/issues/185


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org