[I] [Improvement]: The Spark optimizer can still function properly even in the presence of class conflicts [amoro]

["zhoujinsong (via GitHub)" <gi...@apache.org>]

zhoujinsong opened a new issue, #3012:
URL: https://github.com/apache/amoro/issues/3012

   ### Search before asking
   
   - [X] I have searched in the [issues](https://github.com/apache/amoro/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What would you like to be improved?
   
   The Spark optimizer job includes all the dependencies required for task execution. However, when there is a conflict between the existing libraries in Spark Jars and the dependencies of the task, the Spark optimizer cannot function properly.
   
   We should fix this issue because Spark Jars may typically contain additional dependencies, such as the commonly used iceberg-spark-runtime package.
   
   ### How should we improve?
   
   Add `spark.executor.userClassPathFirst` and `spark.driver.userClassPathFirst` parameter support for spark optimizer.
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Subtasks
   
   _No response_
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Improvement]: The Spark optimizer can still function properly even in the presence of class conflicts [amoro]

["zhoujinsong (via GitHub)" <gi...@apache.org>]

zhoujinsong closed issue #3012: [Improvement]: The Spark optimizer can still function properly even in the presence of class conflicts
URL: https://github.com/apache/amoro/issues/3012


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org