[GitHub] [spark] cloud-fan commented on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

[GitBox <gi...@apache.org>]

cloud-fan commented on pull request #29413:
URL: https://github.com/apache/spark/pull/29413#issuecomment-675331719


   I think we can't 100% guarantee no events drop, whatever we do is just a best effort. That said, the consumer side of the event queue should not be sensitive to event drop. If your use case is to check if a Spark application is still running jobs, can we query the spark driver status via REST API directly?
   
   I get the intention of this PR to make the queue size more dynamic, as the peek number of events is unpredictable. But I don't see how this patch solves it, as users still need to predicate it and set the new config.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org