You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "jkl0898 (via GitHub)" <gi...@apache.org> on 2023/05/04 08:43:02 UTC

[GitHub] [hudi] jkl0898 opened a new issue, #8630: Flink + Hudi quickstart demo can create table but not read or insert.

jkl0898 opened a new issue, #8630:
URL: https://github.com/apache/hudi/issues/8630

   **Describe the problem you faced**
   
   Hi, we tested flink + hudi example , referenced by : [Flink-Hudi QuickStart](https://hudi.apache.org/docs/flink-quick-start-guide)
   
   After we created table t1, it successed. But when we select, it returns:
   [ERROR] Could not execute SQL statement. Reason:
   java.io.FileNotFoundException: File file:/tmp/hudi_test/.hoodie does not exist
   
    
   
   **To Reproduce**
   
   Steps to reproduce the behavior:
   
   1. DownLoad flink tar (version 13.1) and untar it.
   2. Configured HADOOP_CLASSPATH.
   3. Start flink cluster, start-cluster.sh
   4. Start Flink-client 
   5. Create table t1
   ```
   CREATE TABLE t1(
     uuid VARCHAR(20) PRIMARY KEY NOT ENFORCED,
     name VARCHAR(10),
     age INT,
     ts TIMESTAMP(3),
     `partition` VARCHAR(20)
   )
   PARTITIONED BY (`partition`)
   WITH (
     'connector' = 'hudi',
     'path' = 'file:///tmp/hudi_test',
     'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is COPY_ON_WRITE
   );
   ```
   6. select * from t1;
   7. Flink sql-client return errors.
   ```
   [ERROR] Could not execute SQL statement. Reason:
   java.io.FileNotFoundException: File file:/tmp/hudi_test/.hoodie does not exist
   ```
   
   **Expected behavior**
   
   Can Select/Insert data to hudi used by flink 
   
   **Environment Description**
   
   * Hudi version : 0.13
   
   *Flink version : 1.13
   
   * Storage (HDFS) : 3.1.1
   
   * Running on Docker? (yes/no) : no
   
   
   **Additional context**
   -
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] danny0405 commented on issue #8630: Flink + Hudi quickstart demo can create table but not select.

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on issue #8630:
URL: https://github.com/apache/hudi/issues/8630#issuecomment-1534331340

   Hi, did you enable the checkpoint yet?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] jkl0898 commented on issue #8630: Flink + Hudi quickstart demo can create table but not select.

Posted by "jkl0898 (via GitHub)" <gi...@apache.org>.
jkl0898 commented on issue #8630:
URL: https://github.com/apache/hudi/issues/8630#issuecomment-1535633427

   > Hi, did you enable the checkpoint yet?
   
   Hi, thanks for your replied. we solved the problem now. the way is:
   1. Create table -> Insert Data -> Select Data
   
   After Insert Data , it generated the .hoodie file ,  but if Select Data before Insert Data, the .hoodie file does not generate.
   
   I am not sure whether it is a bug or a design .
   
   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] danny0405 commented on issue #8630: Flink + Hudi quickstart demo can create table but not select.

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on issue #8630:
URL: https://github.com/apache/hudi/issues/8630#issuecomment-1535637928

   For batch query, it is by design, because there is not even a table there at all, for streaming query, it is okay because the query itself is long running.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] danny0405 closed issue #8630: Flink + Hudi quickstart demo can create table but not select.

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 closed issue #8630: Flink + Hudi quickstart demo can create table but not select.
URL: https://github.com/apache/hudi/issues/8630


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] jkl0898 commented on issue #8630: Flink + Hudi quickstart demo can create table but not select.

Posted by "jkl0898 (via GitHub)" <gi...@apache.org>.
jkl0898 commented on issue #8630:
URL: https://github.com/apache/hudi/issues/8630#issuecomment-1534370645

   > Hi, did you enable the checkpoint yet?
   
   Hi, we added these checkpoints's config to flink-conf.yaml then restart flink-cluster, but get the same errors 
   ```
   # flink checkpoints
   execution.checkpointing.interval: 5000
   execution.checkpointing.mode: EXACTLY_ONCE
   state.backend: filesystem
   state.checkpoints.dir: hdfs:///flink/checkpoints
   state.savepoints.dir: hdfs:///flink/savepoints
   execution.checkpointing.timeout: 600000
   execution.checkpointing.min-pause: 500
   execution.checkpointing.max-concurrent-checkpoints: 1
   state.checkpoints.num-retained: 3
   execution.checkpointing.externalized-checkpoint-retention: RETAIN_ON_CANCELLATION
   ```
   
   Create table command:
   ```
   CREATE TABLE t1(
     uuid VARCHAR(20) PRIMARY KEY NOT ENFORCED,
     name VARCHAR(10),
     age INT,
     ts TIMESTAMP(3),
     `partition` VARCHAR(20)
   )
   PARTITIONED BY (`partition`)
   WITH (
     'connector' = 'hudi',
     'path' = 'hdfs:///flink/hudi_test',
     'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is COPY_ON_WRITE
   );
   
   ```
   
   When select from 1t, sql-client return errors:
   ```
   [ERROR] Could not execute SQL statement. Reason:
   java.io.FileNotFoundException: File does not exist: hdfs:/flink/hudi_test/.hoodie
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org