You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/07/21 16:10:19 UTC
[GitHub] [hudi] Mulavar opened a new issue, #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
Mulavar opened a new issue, #6169:
URL: https://github.com/apache/hudi/issues/6169
**_Tips before filing an issue_**
- Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)?
- Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.
- If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.
**Describe the problem you faced**
I am a beginner in hudi, and when I tried to run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java, I got the java.lang.NoClassDefFoundError exception like following:
I knew this is caused by the wrong maven scope, and there was a lot `provided` dependencies preventing me from running the examples properly.
**To Reproduce**
Steps to reproduce the behavior:
Run the example org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java.
**Expected behavior**
Maybe we can use `compile` dependency for the example? It is very troublesome to run the example normally now.
**Environment Description**
* Hudi version :
* Spark version :
* Hive version :
* Hadoop version :
* Storage (HDFS/S3/GCS..) :
* Running on Docker? (yes/no) :
**Additional context**
Add any other context about the problem here.
**Stacktrace**
```Add the stacktrace of the error.```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
Posted by GitBox <gi...@apache.org>.
xicm commented on issue #6169:
URL: https://github.com/apache/hudi/issues/6169#issuecomment-1206189741
You don't neet to create the directory, If the directory exists, HUDi will assume that the table exists but cannot find the.hoodie directory and throw an exception.
Maybe we can improve the example.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Mulavar commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
Posted by GitBox <gi...@apache.org>.
Mulavar commented on issue #6169:
URL: https://github.com/apache/hudi/issues/6169#issuecomment-1206198899
>
So could I create a hoodie directory to support the example running properly? If yes, how do I create a hoodir directory.
I totally agree with you about improving the example cause it requires much prior knowledge……
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Mulavar commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
Posted by GitBox <gi...@apache.org>.
Mulavar commented on issue #6169:
URL: https://github.com/apache/hudi/issues/6169#issuecomment-1206138784
Thanks, I tried to add dependencies with "provided" scope to classpath and solved the problem, and then ran into another problem:
```bash
Exception in thread "main" org.apache.hudi.exception.TableNotFoundException: Hoodie table not found in path /Users/xxx/workspace/github.com/apache/hudi/data0/.hoodie
at org.apache.hudi.exception.TableNotFoundException.checkTableValidity(TableNotFoundException.java:57)
at org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:128)
at org.apache.hudi.common.table.HoodieTableMetaClient.newMetaClient(HoodieTableMetaClient.java:641)
at org.apache.hudi.common.table.HoodieTableMetaClient.access$000(HoodieTableMetaClient.java:80)
at org.apache.hudi.common.table.HoodieTableMetaClient$Builder.build(HoodieTableMetaClient.java:710)
at org.apache.hudi.client.BaseHoodieClient.createMetaClient(BaseHoodieClient.java:139)
at org.apache.hudi.client.BaseHoodieWriteClient.startCommit(BaseHoodieWriteClient.java:933)
at org.apache.hudi.examples.java.HoodieJavaWriteClientExample.main(HoodieJavaWriteClientExample.java:93)
Caused by: java.io.FileNotFoundException: File file:/Users/xxx/workspace/github.com/apache/hudi/data0/.hoodie does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:634)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:860)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:624)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:446)
at org.apache.hudi.common.fs.HoodieWrapperFileSystem.lambda$getFileStatus$17(HoodieWrapperFileSystem.java:402)
at org.apache.hudi.common.fs.HoodieWrapperFileSystem.executeFuncWithTimeMetrics(HoodieWrapperFileSystem.java:106)
at org.apache.hudi.common.fs.HoodieWrapperFileSystem.getFileStatus(HoodieWrapperFileSystem.java:396)
at org.apache.hudi.exception.TableNotFoundException.checkTableValidity(TableNotFoundException.java:51)
... 7 more
```
I created a directory named 'data0' in the project root path, and ran the example with arg0 '/Users/djh/workspace/github.com/apache/hudi/data0' and arg1 'hoodie_rt'.
What are the pre-actions I should do before running this demo?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
Posted by GitBox <gi...@apache.org>.
xushiyan commented on issue #6169:
URL: https://github.com/apache/hudi/issues/6169#issuecomment-1195738741
looks like you're missing dependencies. make sure run mvn clean install so your local m2 repo has the needed jars.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Zouxxyy commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
Posted by GitBox <gi...@apache.org>.
Zouxxyy commented on issue #6169:
URL: https://github.com/apache/hudi/issues/6169#issuecomment-1205321914
<img width="1007" alt="image" src="https://user-images.githubusercontent.com/37108074/182869692-518b19e4-63d8-4fa0-bf01-6c46c99bd364.png">
try this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
Posted by GitBox <gi...@apache.org>.
xicm commented on issue #6169:
URL: https://github.com/apache/hudi/issues/6169#issuecomment-1203548015
If you are using idea, try Add dependencies with "provided" scope to classpath.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Mulavar closed issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
Posted by GitBox <gi...@apache.org>.
Mulavar closed issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
URL: https://github.com/apache/hudi/issues/6169
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
Posted by GitBox <gi...@apache.org>.
xicm commented on issue #6169:
URL: https://github.com/apache/hudi/issues/6169#issuecomment-1206215365
You can create the parent path of the base path, or just pass a directory string.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Mulavar commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
Posted by GitBox <gi...@apache.org>.
Mulavar commented on issue #6169:
URL: https://github.com/apache/hudi/issues/6169#issuecomment-1206253584
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on issue #6169: [QUESTION]How can I run the example at org/apache/hudi/examples/java/HoodieJavaWriteClientExample.java properly?
Posted by GitBox <gi...@apache.org>.
xicm commented on issue #6169:
URL: https://github.com/apache/hudi/issues/6169#issuecomment-1203546595
Master branch works well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org