You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "xjt1995 (via GitHub)" <gi...@apache.org> on 2024/04/03 01:32:48 UTC
[I] [Bug] When the version 0.7 is updated, hive cannot be connected [paimon]
xjt1995 opened a new issue, #3153:
URL: https://github.com/apache/paimon/issues/3153
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
paimon 0.7.0
### Compute Engine
Flink 1.17.2
### Minimal reproduce step
<FLINK_HOME>/bin/flink run \
-Dpipeline.name=test_paimon \
-Dexecution.checkpointing.interval=10000 \
lib/paimon-flink-action-0.7.0-incubating.jar \
mysql_sync_database\
--warehouse hdfs://cdh1:8020/user/hive/warehouse/paimon \
--database test3 \
--mode combined\
--mysql-conf hostname=cdh1 \
--mysql-conf username=root \
--mysql-conf password=123456 \
--mysql-conf database-name=test \
--catalog-conf metastore=hive \
--catalog-conf uri=thrift://cdh1:9083 \
--table-conf bucket=4 \
--table-conf changelog-producer=input \
--table-conf sink.parallelism=4 \
--including_tables '.*'
### What doesn't meet your expectations?
The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: java.lang.reflect.InvocationTargetException
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:105)
at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:851)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:245)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1095)
at org.apache.flink.client.cli.CliFrontend.lambda$mainInternal$9(CliFrontend.java:1189)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.mainInternal(CliFrontend.java:1189)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1157)
Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.paimon.hive.RetryingMetaStoreClientFactory.createClient(RetryingMetaStoreClientFactory.java:146)
at org.apache.paimon.hive.HiveCatalog.createClient(HiveCatalog.java:600)
at org.apache.paimon.hive.HiveCatalog.<init>(HiveCatalog.java:147)
at org.apache.paimon.hive.HiveCatalog.createHiveCatalog(HiveCatalog.java:692)
at org.apache.paimon.hive.HiveCatalogFactory.create(HiveCatalogFactory.java:53)
at org.apache.paimon.catalog.CatalogFactory.createCatalog(CatalogFactory.java:76)
at org.apache.paimon.catalog.CatalogFactory.createCatalog(CatalogFactory.java:66)
at org.apache.paimon.flink.FlinkCatalogFactory.createPaimonCatalog(FlinkCatalogFactory.java:80)
at org.apache.paimon.flink.action.ActionBase.<init>(ActionBase.java:59)
at org.apache.paimon.flink.action.cdc.SynchronizationActionBase.<init>(SynchronizationActionBase.java:71)
at org.apache.paimon.flink.action.cdc.SyncDatabaseActionBase.<init>(SyncDatabaseActionBase.java:61)
at org.apache.paimon.flink.action.cdc.mysql.MySqlSyncDatabaseAction.<init>(MySqlSyncDatabaseAction.java:107)
at org.apache.paimon.flink.action.cdc.mysql.MySqlSyncDatabaseActionFactory.create(MySqlSyncDatabaseActionFactory.java:61)
at org.apache.paimon.flink.action.ActionFactory.createAction(ActionFactory.java:82)
at org.apache.paimon.flink.action.FlinkActions.main(FlinkActions.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
... 12 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.paimon.hive.RetryingMetaStoreClientFactory.lambda$static$7(RetryingMetaStoreClientFactory.java:111)
at org.apache.paimon.hive.RetryingMetaStoreClientFactory.createClient(RetryingMetaStoreClientFactory.java:144)
... 31 more
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1775)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:86)
... 37 more
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.getDeclaredConstructor(Class.java:2178)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771)
... 40 more
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] When the version 0.7 is updated, hive cannot be connected [paimon]
Posted by "xjt1995 (via GitHub)" <gi...@apache.org>.
xjt1995 commented on issue #3153:
URL: https://github.com/apache/paimon/issues/3153#issuecomment-2033384502
My hive environment is cdh6.3.2-2.1.1,I also placed the paimon-hive jar package into Flink
![Uploading WX20240403-093811@2x.png…]()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] When the version 0.7 is updated, hive cannot be connected [paimon]
Posted by "xjt1995 (via GitHub)" <gi...@apache.org>.
xjt1995 commented on issue #3153:
URL: https://github.com/apache/paimon/issues/3153#issuecomment-2041919045
> Can you remove flink-hive and `Presto Bundle` hive jar? https://paimon.apache.org/docs/master/flink/sql-ddl/#creating-hive-catalog
@JingsongLi I also tried to remove the Flinker-connector-hive_2.12-1.17.2.jar, but restarting the flink cluster and running the script again returned the same error。“Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(org.apache.hadoop.hive.conf.HiveConf, java.lang.Boolean)”
<img width="429" alt="image" src="https://github.com/apache/paimon/assets/30589894/cc35af25-d2fc-48ed-b48b-5f57faf631d7">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] When the version 0.7 is updated, hive cannot be connected [paimon]
Posted by "Guanpx (via GitHub)" <gi...@apache.org>.
Guanpx commented on issue #3153:
URL: https://github.com/apache/paimon/issues/3153#issuecomment-2048794051
I have encountered the same issue as well when I use **paimon-flink-1.15-0.7-incubating**
But when I use **paimon-flink-1.14-0.7-incubating**, this issue has been resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] When the version 0.7 is updated, hive cannot be connected [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on issue #3153:
URL: https://github.com/apache/paimon/issues/3153#issuecomment-2041866365
Can you remove flink-hive and `Presto Bundle` hive jar? https://paimon.apache.org/docs/master/flink/sql-ddl/#creating-hive-catalog
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] When the version 0.7 is updated, hive cannot be connected [paimon]
Posted by "xjt1995 (via GitHub)" <gi...@apache.org>.
xjt1995 commented on issue #3153:
URL: https://github.com/apache/paimon/issues/3153#issuecomment-2033386595
My hive environment is cdh6.3.2-2.1.1,I also placed the paimon-hive jar package into Flink
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] When the version 0.7 is updated, hive cannot be connected [paimon]
Posted by "BoYiZhang (via GitHub)" <gi...@apache.org>.
BoYiZhang commented on issue #3153:
URL: https://github.com/apache/paimon/issues/3153#issuecomment-2044698673
When I was a part of the group, I wrote a new version.
https://github.com/apache/paimon/issues/3184
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org