You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/11/10 16:15:09 UTC
[GitHub] [flink-table-store] LadyForest opened a new pull request, #373: [FLINK-29848] Fix Table Store with Hive3 lacking hive-standalone-metastore dependency
LadyForest opened a new pull request, #373:
URL: https://github.com/apache/flink-table-store/pull/373
For Hive3, `org.apache.hadoop.hive.metastore.api` is moved to `org.apache.hive:hive-standalone-metastore`. We should shade this package as well to avoid `ClassNotFoundException`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] JingsongLi commented on a diff in pull request #373: [FLINK-29983] Fix Table Store with Hive3 lacking hive-standalone-metastore dependency
Posted by GitBox <gi...@apache.org>.
JingsongLi commented on code in PR #373:
URL: https://github.com/apache/flink-table-store/pull/373#discussion_r1020216429
##########
flink-table-store-hive/pom.xml:
##########
@@ -83,7 +83,50 @@ under the License.
<hive.main.version>3</hive.main.version>
<hive.version>3.1.2</hive.version>
</properties>
+ <dependencies>
+ <!-- For Hive 3.1.2, package org.apache.hadoop.hive.metastore.api relies on this dependency-->
+ <dependency>
+ <groupId>org.apache.hive</groupId>
+ <artifactId>hive-standalone-metastore</artifactId>
+ <version>${hive.version}</version>
+ <exclusions>
+ <exclusion>
+ <groupId>log4j</groupId>
+ <artifactId>log4j</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
Review Comment:
I don't think we should shade hive in parent module.
We should shade in `flink-table-store-hive-catalog`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] LadyForest commented on pull request #373: [FLINK-29983] Fix Table Store with Hive3 lacking hive-standalone-metastore dependency
Posted by GitBox <gi...@apache.org>.
LadyForest commented on PR #373:
URL: https://github.com/apache/flink-table-store/pull/373#issuecomment-1311180702
The reason why E2E does not reveal this issue is due to the hive docker image is based on Hive2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] JingsongLi commented on pull request #373: [FLINK-29983] Fix Table Store with Hive3 lacking hive-standalone-metastore dependency
Posted by GitBox <gi...@apache.org>.
JingsongLi commented on PR #373:
URL: https://github.com/apache/flink-table-store/pull/373#issuecomment-1311172988
Where is the problem? In Flink? Or in Hive?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] LadyForest commented on pull request #373: [FLINK-29983] Fix Table Store with Hive3 lacking hive-standalone-metastore dependency
Posted by GitBox <gi...@apache.org>.
LadyForest commented on PR #373:
URL: https://github.com/apache/flink-table-store/pull/373#issuecomment-1311178286
> Where is the problem? In Flink? Or in Hive?
For all use case where `HiveCatalog` engaged
1. `HiveCatalog` depends on `org.apache.hadoop.hive.metastore.api`
2. During the shading phase， all the package pattern `org.apache.hadoop.hive` is shaded as `org.apache.flink.table.store.shaded.org.apache.hadoop.hive`
Well for `org.apache.hadoop.hive.metastore.api`, they exists in `org.apache.hive:hive-metastore` for Hive2, but is moved to `org.apache.hive:hive-standalone-mteastore` for Hive3.
This will cause `NoClassDefFoundError` for `org.apache.flink.table.store.shaded.org.apache.hadoop.hive.metastore.api.XXX`
Attach the stacktrace
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] JingsongLi merged pull request #373: [FLINK-29983] Fix Table Store with Hive3 lacking hive-standalone-metastore dependency
Posted by GitBox <gi...@apache.org>.
JingsongLi merged PR #373:
URL: https://github.com/apache/flink-table-store/pull/373
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] LadyForest commented on pull request #373: [FLINK-29983] Fix Table Store with Hive3 lacking hive-standalone-metastore dependency
Posted by GitBox <gi...@apache.org>.
LadyForest commented on PR #373:
URL: https://github.com/apache/flink-table-store/pull/373#issuecomment-1311180158
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] JingsongLi commented on pull request #373: [FLINK-29983] Fix Table Store with Hive3 lacking hive-standalone-metastore dependency
Posted by GitBox <gi...@apache.org>.
JingsongLi commented on PR #373:
URL: https://github.com/apache/flink-table-store/pull/373#issuecomment-1311691417
Currently, we are using Hive2.3 Catalog (metastore client) for Hive 3.1, you can see it in `hive.md`:
| |Jar|
|---|---|
|Hive 2.3+|[flink-table-store-hive-catalog-{{< version >}}_2.3.jar]|
|Hive 2.2|[flink-table-store-hive-catalog-{{< version >}}_2.2.jar]|
|Hive 2.1|[flink-table-store-hive-catalog-{{< version >}}_2.1.jar]|
If you want to support hive3 by Hive3 catalog, you should modify documentation too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org