You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2019/05/16 03:10:24 UTC
[GitHub] [incubator-hudi] bvaradar commented on issue #681: spark-bundle
packaging shades HiveDriver
bvaradar commented on issue #681: spark-bundle packaging shades HiveDriver
URL: https://github.com/apache/incubator-hudi/issues/681#issuecomment-492899527
@arw357 : The shading process is supposed to regex-replace org.apache.hive.jdbc to com.uber.hoodie.org.apache.hive.jdbc in all call sites. Also , HoodieHiveClient resides in hoodie-hive which does not depend on the bundle directly. It is the other way round.
To confirm, you can do this
jar tf $HUDI_SPARK_BUNDLE | grep HiveDriver
com/uber/hoodie/org/apache/hive/jdbc/HiveDriver.class
Now, unzip the jar (mkdir tmp; cd tmp; unzip $HUDI_SPARK_BUNDLE '*.class' ). If you run the disassembler, you will see
javap -c com/uber/hoodie/hive/HoodieHiveClient.class | grep HiveDriver
0: ldc_w #819 // class com/uber/hoodie/org/apache/hive/jdbc/HiveDriver
This should tell that HiveDriver is indeed shaded.
The reason why you are seeing org/apache/hive/jdbc/HiveDriver is because you are likely including some hoodie jar other than hoodie-spark-bundle.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services