[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #1005: [HUDI-91][HUDI-12]Migrate to spark 2.4.4, migrate to spark-avro library instead of databricks-avro, add support for Decimal/Date types

[GitBox <gi...@apache.org>]

vinothchandar commented on a change in pull request #1005: [HUDI-91][HUDI-12]Migrate to spark 2.4.4, migrate to spark-avro library instead of databricks-avro, add support for Decimal/Date types
URL: https://github.com/apache/incubator-hudi/pull/1005#discussion_r344968699
 
 

 ##########
 File path: hudi-spark/pom.xml
 ##########
 @@ -213,9 +213,9 @@
 
     <!-- Spark (Packages) -->
     <dependency>
-      <groupId>com.databricks</groupId>
+      <groupId>org.apache.spark</groupId>
       <artifactId>spark-avro_2.11</artifactId>
-      <version>4.0.0</version>
+      <scope>provided</scope>
 
 Review comment:
    I understand we can't bundle this since its tied to a spark version now.  `spark-avro` is still a package and the user must explicitly include using `--packages` ? So, if I upgrade hudi in the next release, then as a user I need to change something? Should/how do we document this? 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services