You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@inlong.apache.org by "zhubangpei (via GitHub)" <gi...@apache.org> on 2023/02/10 09:10:38 UTC
[GitHub] [inlong] zhubangpei opened a new issue, #7352: [Bug]Create hudi sink fail
zhubangpei opened a new issue, #7352:
URL: https://github.com/apache/inlong/issues/7352
### What happened
<img width="1182" alt="企业微信截图_6533fb46-c779-4677-bae0-8b12dfc54171" src="https://user-images.githubusercontent.com/17564903/218050004-2e78d7b2-aa78-42aa-97bd-8d9712c9edfd.png">
### What you expected to happen
<img width="1570" alt="企业微信截图_a5380983-b827-42b3-a236-56666c3b46ef" src="https://user-images.githubusercontent.com/17564903/218050343-8f64e9bb-0fbb-4fb6-a13c-fbea466334f2.png">
<img width="1197" alt="企业微信截图_15c719f1-7441-41d4-9b30-55a175521b3d" src="https://user-images.githubusercontent.com/17564903/218050369-27b8e11c-c875-4f73-b900-0863ffc0a280.png">
fix the bug
### How to reproduce
sink hudi
### Environment
MacBook Pro 12.2.1
Apple M1 Pro
### InLong version
1.5.0
### InLong Component
InLong DataProxy
### Are you willing to submit PR?
- [X] Yes, I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@inlong.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] dockerzhang closed issue #7352: [Bug]Create hudi sink fail
Posted by "dockerzhang (via GitHub)" <gi...@apache.org>.
dockerzhang closed issue #7352: [Bug]Create hudi sink fail
URL: https://github.com/apache/inlong/issues/7352
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] featzhang commented on issue #7352: [Bug]Create hudi sink fail
Posted by "featzhang (via GitHub)" <gi...@apache.org>.
featzhang commented on issue #7352:
URL: https://github.com/apache/inlong/issues/7352#issuecomment-1485055392
When hive metastore can not connected, the error is as follows in InLong 1.7.0 :
```java
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86) ~[hive-exec-3.1.3.jar:3.1.3]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:95) ~[hive-exec-3.1.3.jar:3.1.3]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148) ~[hive-exec-3.1.3.jar:3.1.3]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119) ~[hive-exec-3.1.3.jar:3.1.3]
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:4306) ~[hive-exec-3.1.3.jar:3.1.3]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] featzhang commented on issue #7352: [Bug]Create hudi sink fail
Posted by "featzhang (via GitHub)" <gi...@apache.org>.
featzhang commented on issue #7352:
URL: https://github.com/apache/inlong/issues/7352#issuecomment-1484495810
@zhubangpei Please add a description of how to reproduce the problem, such as deployment method, component version, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] featzhang commented on issue #7352: [Bug]Create hudi sink fail
Posted by "featzhang (via GitHub)" <gi...@apache.org>.
featzhang commented on issue #7352:
URL: https://github.com/apache/inlong/issues/7352#issuecomment-1485060269
This bug was fixed in InLong 1.7.0 by https://github.com/apache/inlong/pull/7657, welcome to use InLong 1.7.0 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] featzhang commented on issue #7352: [Bug]Create hudi sink fail
Posted by "featzhang (via GitHub)" <gi...@apache.org>.
featzhang commented on issue #7352:
URL: https://github.com/apache/inlong/issues/7352#issuecomment-1484992036
```java
m.s.r.s.h.HudiResourceOperator:154 - create Hudi table failed: org/apache/hudi/exception/HoodieCatalogException
java.lang.NoClassDefFoundError: org/apache/hudi/exception/HoodieCatalogException
at org.apache.inlong.manager.service.resource.sink.hudi.HudiResourceOperator.createTableIfAbsent(HudiResourceOperator.java:128) ~[manager-service-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.service.resource.sink.hudi.HudiResourceOperator.createSinkResource(HudiResourceOperator.java:88) ~[manager-service-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.service.listener.sink.StreamSinkResourceListener.listen(StreamSinkResourceListener.java:94) ~[manager-service-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.workflow.event.LogableEventListener.executeListenerWithLog(LogableEventListener.java:78) ~[manager-workflow-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.workflow.event.LogableEventListener.listen(LogableEventListener.java:59) ~[manager-workflow-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.workflow.event.task.TaskEventNotifier.notify(TaskEventNotifier.java:56) ~[manager-workflow-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.workflow.processor.ServiceTaskProcessor.complete(ServiceTaskProcessor.java:114) ~[manager-workflow-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.workflow.core.impl.ProcessorExecutorImpl.executeComplete(ProcessorExecutorImpl.java:97) ~[manager-workflow-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.workflow.core.impl.ProcessorExecutorImpl.executeStart(ProcessorExecutorImpl.java:90) ~[manager-workflow-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.workflow.core.impl.ProcessorExecutorImpl.executeComplete(ProcessorExecutorImpl.java:102) ~[manager-workflow-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.workflow.core.impl.ProcessorExecutorImpl.executeStart(ProcessorExecutorImpl.java:90) ~[manager-workflow-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.workflow.core.impl.ProcessorExecutorImpl.executeComplete(ProcessorExecutorImpl.java:102) ~[manager-workflow-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.workflow.core.impl.ProcessorExecutorImpl.executeStart(ProcessorExecutorImpl.java:90) ~[manager-workflow-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.workflow.core.impl.ProcessServiceImpl.start(ProcessServiceImpl.java:63) ~[manager-workflow-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.service.workflow.WorkflowServiceImpl.start(WorkflowServiceImpl.java:90) ~[manager-service-1.6.0.jar:1.6.0]
at org.apache.inlong.manager.service.listener.queue.QueueResourceListener.lambda$createQueueForStreams$0(QueueResourceListener.java:156) ~[manager-service-1.6.0.jar:1.6.0]
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) ~[?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_292]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_292]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292]
Caused by: java.lang.ClassNotFoundException: org.apache.hudi.exception.HoodieCatalogException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_292]
at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_292]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) ~[?:1.8.0_292]
at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_292]
... 20 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] dockerzhang commented on issue #7352: [Bug]Create hudi sink fail
Posted by "dockerzhang (via GitHub)" <gi...@apache.org>.
dockerzhang commented on issue #7352:
URL: https://github.com/apache/inlong/issues/7352#issuecomment-1427259466
@featzhang PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] dockerzhang commented on issue #7352: [Bug]Create hudi sink fail
Posted by "dockerzhang (via GitHub)" <gi...@apache.org>.
dockerzhang commented on issue #7352:
URL: https://github.com/apache/inlong/issues/7352#issuecomment-1485103175
you could add `hudi-flink1.13-bundle` jar to the lib or pick #7657 and build the latest package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] featzhang commented on issue #7352: [Bug]Create hudi sink fail
Posted by "featzhang (via GitHub)" <gi...@apache.org>.
featzhang commented on issue #7352:
URL: https://github.com/apache/inlong/issues/7352#issuecomment-1483099176
> @featzhang PTAL
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org