[GitHub] [spark] adamkennedy commented on pull request #28376: [SPARK-31582] [Yarn] Being able to not populate Hadoop classpath

[GitBox <gi...@apache.org>]

adamkennedy commented on pull request #28376:
URL: https://github.com/apache/spark/pull/28376#issuecomment-620914526


   @viirya The embedded Hadoop could easily be incompatible with the cluster. But then the Spark version could be incompatible with the cluster's Hadoop as well. If the Spark chooses to bundle Hadoop libraries, then at least we know that works, and then the onus is on the Spark distribution to provide a different-but-compatible version, and the failure is only a failure for that one job. But at least the Spark provider gets an opportunity to provide something working, as opposed to the situation where different parts of the two different distributions end up cross-loading each other unexpectedly.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org