You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/07/17 20:04:01 UTC
[GitHub] [hudi] sambhav13 opened a new issue, #6126: [SUPPORT] Hudi Table(MOR) not getting created from Flink Sql Client Shell
sambhav13 opened a new issue, #6126:
URL: https://github.com/apache/hudi/issues/6126
**_Tips before filing an issue_**
- Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)?
- Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.
- If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.
**Describe the problem you faced**
When trying to run create table command with from Flink Sql Client Shell, it throws an error saying Caused by: java.net.UnknownHostException: hudiflink2.localhost: nodename nor servname provided, or not known\
NOTE: hudiflink2 is the bucket name for s3 path where the table is being stored using the path
A clear and concise description of the problem.
Need to create a hudi table from flink sql client shell and insert some data in it. But when trying to create a table using the flink sql client shell with a create sql with create sql an error is thrown saying **UnknownHost Exception : hudiflink2.localhost: nodename nor servname provided, or not known.**
**To Reproduce**
Steps to reproduce the behavior:
1. Download and extract flink-1.14.5
2. ./bin/start-cluster.sh
3. ./bin/sql-client.sh embedded -j hudi-flink-bundle_2.12-0.10.1.jar -j hadoop-aws-2.9.0.jar -j aws-java-sdk-s3-1.11.0.jar
4. run
create table t2_minio(id int PRIMARY KEY, name varchar(50)) with ('connector' = 'hudi', 'table.type' = 'MERGE_ON_READ', 'path' = 's3a://hudiflink2/t2', 'hadoop.fs.s3a.endpoint' = 'http://127.0.0.1:9000' , 'hadoop.fs.s3a.access.key' = 'minio' , 'hadoop.fs.s3a.secret.key' = 'minio123', 'hadoop.fs.s3a.connection.timeout' = '600000', 'hadoop.fs.s3a.impl' = 'org.apache.hadoop.fs.s3a.S3AFileSystem', 'hadoop.fs.s3a.path.style.access' = 'true');
on client shell
**Expected behavior**
Ideally the Create command should run successfully and a table should be created inside the bucket hudiflink2 with name t2.
**Environment Description**
Local machine MacOs
* Hudi version :
0.10.1
* Spark version :
Flink version 1.14.5
* Hive version :
* Hadoop version :
2.9.0
* Storage (HDFS/S3/GCS..) :
S3(MinIO) running locally and on port 9000
* Running on Docker? (yes/no) :
running local cluster for flink
**Additional context**
Add any other context about the problem here.
Attached snapshots
<img width="1898" alt="FlinkSqlClient_invokeCommand" src="https://user-images.githubusercontent.com/4696334/179422879-2b96cb5d-3bc3-475c-a9d2-9eef6ec0a0a3.png">
<img width="1776" alt="flink_HudiTable_createCommand" src="https://user-images.githubusercontent.com/4696334/179422881-5300eac1-b45a-4da3-8371-a326982c008a.png">
**Stacktrace**
```Add the stacktrace of the error.```
[ErrorStackTrace (3).log](https://github.com/apache/hudi/files/9128466/ErrorStackTrace.3.log)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on issue #6126: [SUPPORT] Hudi Table(MOR) not getting created from Flink Sql Client Shell
Posted by GitBox <gi...@apache.org>.
xushiyan commented on issue #6126:
URL: https://github.com/apache/hudi/issues/6126#issuecomment-1205406483
hey @yuzhaojing would you kindly help here? you may just follow the docker setup provided and reproduce in local environment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #6126: [SUPPORT] Hudi Table(MOR) not getting created from Flink Sql Client Shell
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #6126:
URL: https://github.com/apache/hudi/issues/6126#issuecomment-1229345724
@yuzhaojing : any updates here please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yuzhaojing commented on issue #6126: [SUPPORT] Hudi Table(MOR) not getting created from Flink Sql Client Shell
Posted by GitBox <gi...@apache.org>.
yuzhaojing commented on issue #6126:
URL: https://github.com/apache/hudi/issues/6126#issuecomment-1206006030
Sure, I will try to reproduce it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on issue #6126: [SUPPORT] Hudi Table(MOR) not getting created from Flink Sql Client Shell
Posted by GitBox <gi...@apache.org>.
danny0405 commented on issue #6126:
URL: https://github.com/apache/hudi/issues/6126#issuecomment-1306538262
Close because of prolonged inactivity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] sambhav13 commented on issue #6126: [SUPPORT] Hudi Table(MOR) not getting created from Flink Sql Client Shell
Posted by GitBox <gi...@apache.org>.
sambhav13 commented on issue #6126:
URL: https://github.com/apache/hudi/issues/6126#issuecomment-1205161654
@xushiyan any update on this issue?
Attaching a reference to the docker-compose.yml and dockerfile to setup the environment for flink and minio
and commands to run in a zip file
ref: https://hub.docker.com/layers/266300429/sambhav13/flink-sql/2.0/images/sha256-782647055e3f62de3df0ce4056bfd04c515738f364bda8809fdb6184ebdecd05?context=repo
[Archive.zip](https://github.com/apache/hudi/files/9259442/Archive.zip)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 closed issue #6126: [SUPPORT] Hudi Table(MOR) not getting created from Flink Sql Client Shell
Posted by GitBox <gi...@apache.org>.
danny0405 closed issue #6126: [SUPPORT] Hudi Table(MOR) not getting created from Flink Sql Client Shell
URL: https://github.com/apache/hudi/issues/6126
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on issue #6126: [SUPPORT] Hudi Table(MOR) not getting created from Flink Sql Client Shell
Posted by GitBox <gi...@apache.org>.
danny0405 commented on issue #6126:
URL: https://github.com/apache/hudi/issues/6126#issuecomment-1205192949
What is the error stacktrace thrown in the JobManager log, can you paste that ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] sambhav13 commented on issue #6126: [SUPPORT] Hudi Table(MOR) not getting created from Flink Sql Client Shell
Posted by GitBox <gi...@apache.org>.
sambhav13 commented on issue #6126:
URL: https://github.com/apache/hudi/issues/6126#issuecomment-1205422365
@danny0405 updated the comments above for the zip and the commands to follow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org