[I] Exception occurred while broadcasting an empty result [arrow-datafusion-comet]

["wForget (via GitHub)" <gi...@apache.org>]

wForget opened a new issue, #91:
URL: https://github.com/apache/arrow-datafusion-comet/issues/91

   ### Describe the bug
   
   Since the empty result will also write extra bytes in the compressed stream during encode, an exception occurs during decode.
   
   error detail:
   
   ```
   Caused by: java.io.IOException: Unexpected end of input. Missing schema.
   	at org.apache.arrow.vector.ipc.ArrowStreamReader.readSchema(ArrowStreamReader.java:205)
   	at org.apache.arrow.vector.ipc.ArrowReader.initialize(ArrowReader.java:185)
   	at org.apache.arrow.vector.ipc.ArrowReader.ensureInitialized(ArrowReader.java:176)
   	at org.apache.arrow.vector.ipc.ArrowReader.getVectorSchemaRoot(ArrowReader.java:73)
   	at org.apache.comet.vector.StreamReader.<init>(StreamReader.scala:39)
   	at org.apache.spark.sql.comet.execution.shuffle.ArrowReaderIterator.<init>(ArrowReaderIterator.scala:36)
   	at org.apache.spark.sql.comet.CometExec$.decodeBatches(operators.scala:135)
   	at org.apache.spark.sql.comet.CometBatchRDD.$anonfun$compute$1(CometBroadcastExchangeExec.scala:249)
   	at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
   	at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
   	at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
   	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
   	at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
   	at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
   	at scala.collection.mutable.ArrayOps$ofRef.flatMap(ArrayOps.scala:198)
   	at org.apache.spark.sql.comet.CometBatchRDD.compute(CometBroadcastExchangeExec.scala:249)
   	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:364)
   ```
   
   ### Steps to reproduce
   
   broadcast `tbl_a` in CometBroadcastExchangeExec test case
   
   ```
   SELECT /*+ BROADCAST(tbl_a) */ tbl_a._1, tbl_b._2 FROM tbl_a JOIN tbl_b " +
                   "WHERE tbl_a._1 > tbl_a._2 LIMIT 2
   ```
   
   ### Expected behavior
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Exception occurred while broadcasting an empty result [arrow-datafusion-comet]

["sunchao (via GitHub)" <gi...@apache.org>]

sunchao closed issue #91: Exception occurred while broadcasting an empty result
URL: https://github.com/apache/arrow-datafusion-comet/issues/91


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org