You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "wForget (via GitHub)" <gi...@apache.org> on 2024/10/14 09:59:09 UTC

[I] [VL][UNIFFLE] NPE occurs after uniffle shuffle writer fallback to row based [incubator-gluten]

wForget opened a new issue, #7524:
URL: https://github.com/apache/incubator-gluten/issues/7524

   ### Backend
   
   VL (Velox)
   
   ### Bug description
   
   NPE occurs after uniffle shuffle writer fallback to row based.
   
   sql:
   ```
   select count(1) from (select /*+ repartition(100) */ * from web_page) t
   ```
   
   fallback message:
   
   ![image](https://github.com/user-attachments/assets/0bd074bd-2292-4d36-ae73-a362d9810926)
   
   error:
   
   ```
   24/10/14 17:18:38 ERROR TaskResources: Task 0 failed by error: 
   java.lang.NullPointerException
   	at org.apache.spark.shuffle.writer.WriteBufferManager.addRecord(WriteBufferManager.java:300)
   	at org.apache.spark.shuffle.writer.RssShuffleWriter.writeImpl(RssShuffleWriter.java:357)
   	at org.apache.spark.shuffle.writer.RssShuffleWriter.write(RssShuffleWriter.java:280)
   	at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
   	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:104)
   	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:54)
   	at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
   	at org.apache.spark.scheduler.Task.run(Task.scala:141)
   	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
   	at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
   	at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
   	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
   	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:750)
   ```
   
   We should reset `rss.row.based` after shuffle writer fallback.
   
   ### Spark version
   
   None
   
   ### Spark configurations
   
   _No response_
   
   ### System information
   
   _No response_
   
   ### Relevant logs
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [I] [VL][UNIFFLE] NPE occurs after uniffle shuffle writer fallback to row based [incubator-gluten]

Posted by "jackylee-ch (via GitHub)" <gi...@apache.org>.
jackylee-ch closed issue #7524: [VL][UNIFFLE] NPE occurs after uniffle shuffle writer fallback to row based
URL: https://github.com/apache/incubator-gluten/issues/7524


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org