You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/03/19 12:40:39 UTC

[GitHub] [hudi] nsivabalan removed a comment on pull request #2625: [1568] Fixing spark3 bundles

nsivabalan removed a comment on pull request #2625:
URL: https://github.com/apache/hudi/pull/2625#issuecomment-802782822


   @vinothchandar : wanted to see if we need to fix the artifact for hudi-spark-datasource/hudi-spark.
   If we run w/ spark2 and scala12, we generate 
   hudi-spark_2.12-0.8.0-SNAPSHOT.jar
   If we run w/ spark2 and scala11, we generate
   hudi-spark_2.11-0.8.0-SNAPSHOT.jar
   if we run w/ spark3 and scala12, we generate
   hudi-spark_2.12-0.8.0-SNAPSHOT.jar
   
   Wondering, if we can introduce the sparkbundle.version here as well similar to the bundle. 
   So, for spark2, we will still be backwards compatible. and for spark3, we will generate something like 
   hudi-spark3_2.12-0.8.0-SNAPSHOT.jar
   
   Artifacts generated for hudi-spark-datasource/hudi-spark2 and hudi-spark-datasource/hudi-spark3 already has spark version embedded. so we don't have any issues. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org