[GitHub] [incubator-hudi] umehrot2 edited a comment on issue #869: Hudi Spark error when spark bundle jar is added to spark's classpath

[GitBox <gi...@apache.org>]

umehrot2 edited a comment on issue #869: Hudi Spark error when spark bundle jar is added to spark's classpath
URL: https://github.com/apache/incubator-hudi/issues/869#issuecomment-530599588
 
 
   @vinothchandar The PR #873 did not fix this specific issue. 
   
   I do not really see like a recommended way of using a data source. But in EMR, if it is something we want to support by default we configure the driver/executor class-paths to have the libraries available at runtime by configuring them in `spark.driver.extraClassPath` and `spark.executor.extraClassPath`, and configuring hudi-spark-bundle library path here results in the above error.
   
   Yes, we cannot configure `spark.jars` by default because this value will get overridden if customers use `--jars` option while running. This leaves us with option of not configuring anything by default, and having customer always having to specify bundle jar using `--jars` or add jar using code everytime they want to use Hudi.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services