You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/02/10 02:28:31 UTC

[GitHub] [spark] cloud-fan commented on pull request #35425: [SPARK-38129][SQL] Adaptively enable timeout for BroadcastQueryStageExec in AQE

cloud-fan commented on pull request #35425:
URL: https://github.com/apache/spark/pull/35425#issuecomment-1034427438


   This reverts [SPARK-36414](https://issues.apache.org/jira/browse/SPARK-36414), right? If a query has many broadcast stages (not converted from shuffle), and the broadcast may be waiting to be scheduled for a long time, having a timeout will break valid queries.
   
   Unfortunately, we don't have a good solution for it today. One idea is to make the broadcast itself dynamic: it should cancel the job if it has already collected much data at the driver side.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org