[GitHub] [spark] vanzin commented on issue #24785: [SPARK-27937][CORE] Revert partial logic for auto namespace discovery

[GitBox <gi...@apache.org>]

vanzin commented on issue #24785: [SPARK-27937][CORE] Revert partial logic for auto namespace discovery
URL: https://github.com/apache/spark/pull/24785#issuecomment-505213002
 
 
   > Technically a federation config doesn’t even belong in a client config and as such should never be used by a client.
   
   Well, technically you have similar configuration on the client so that the viewfs mount points can reference the appropriate namespaces... but that's nitpicking.
   
   But in general I agree that it feels weird for the client to be looking at random HDFS configurations (in fact I made a similar comment on the original review and requested to special-case viewfs if the config scraping code was going to be added).
   
   Given that I'm ok with removing the config scraping code. Viewfs users (which in my view should be all users accessing a federated HDFS) shouldn't see any difference. Those using the Spark config should not see any difference either. Only those relying on a weird HDFS config that had multiple configured namespaces but not viewfs should see a difference... but well, it's the price to pay if this causes problems for people.
   
   (I haven't looked at the code yet.)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org