You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/06/29 07:53:28 UTC

[GitHub] [hudi] JoshuaZhuCN opened a new issue, #6001: [SUPPORT] Cannot create again after deleting the Hudi external table using Spark SQL

JoshuaZhuCN opened a new issue, #6001:
URL: https://github.com/apache/hudi/issues/6001

     Cannot create again after deleting the Hudi external table using Spark SQL
   
   **To Reproduce**
   
   Steps to reproduce the behavior:
   
   1. 
   ```
   CREATE TABLE IF NOT EXISTS `default`.`spark_hudi_test_ddl` (
   `id` int, `vname` string, `price` decimal(14, 2), `sync_time` timestamp
   ) USING HUDI
   OPTIONS(`hoodie.query.as.ro.table` = 'false')
   TBLPROPERTIES (
      type='mor', 
      primaryKey='id', 
      preCombineField='sync_time'
   )
   LOCATION 'hdfs://localhost:9000/hudi/test/spark_hudi_test_ddl';
   ```
   2.
   ```
   DROP TABLE IF EXISTS `default`.`spark_hudi_test_ddl`;
   ```
   3.
   repeat step 1
   4.
   it will get an error message like 
   ```
   Specified schema in create table statement is not equal to the table schema.You should not specify the schema for an exist table: `default`.`spark_hudi_test_ddl` 
   ```
   5、If we delete the .hoodie directory, and repeat step 1, then it works
   
   
   **Environment Description**
   
   * Hudi version : 0.10.1
   
   * Spark version : 3.1.3
   
   * Hive version : 3.1.0
   
   * Hadoop version : 3.1.1
   
   * Storage (HDFS/S3/GCS..) : hdfs
   
   * Running on Docker? (yes/no) : not
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on issue #6001: [SUPPORT] Cannot create again after deleting the Hudi external table using Spark SQL

Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #6001:
URL: https://github.com/apache/hudi/issues/6001#issuecomment-1229350210

   @JoshuaZhuCN : hope that answers. if you don't have any more questions, feel free to close out the issue. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] xushiyan closed issue #6001: [SUPPORT] Cannot create again after deleting the Hudi external table using Spark SQL

Posted by GitBox <gi...@apache.org>.
xushiyan closed issue #6001: [SUPPORT] Cannot create again after deleting the Hudi external table using Spark SQL
URL: https://github.com/apache/hudi/issues/6001


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] xushiyan commented on issue #6001: [SUPPORT] Cannot create again after deleting the Hudi external table using Spark SQL

Posted by GitBox <gi...@apache.org>.
xushiyan commented on issue #6001:
URL: https://github.com/apache/hudi/issues/6001#issuecomment-1296297443

   @JoshuaZhuCN let me clarify: when it comes to delete the whole table, we support 3 syntaxes
   
   - TRUNCATE TABLE: delete all records via file system; table retained in metastore
   - DROP TABLE: delete no record; table removed from metastore
   - DROP TABLE PURGE: delete all records via file system; table removed from metastore
   
   So in your case where you expect data deleted from the storage, you should use 
   
   ```sql
   DROP TABLE IF EXISTS `default`.`spark_hudi_test_ddl` PURGE;
   ```
   
   If you use `DROP TABLE` (without purge) and you want to recreate the table again, your CREATE TABLE statement should just be
   
   ```sql
   CREATE TABLE IF NOT EXISTS `default`.`spark_hudi_test_ddl` USING HUDI
   LOCATION 'hdfs://localhost:9000/hudi/test/spark_hudi_test_ddl';
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] yihua commented on issue #6001: [SUPPORT] Cannot create again after deleting the Hudi external table using Spark SQL

Posted by GitBox <gi...@apache.org>.
yihua commented on issue #6001:
URL: https://github.com/apache/hudi/issues/6001#issuecomment-1170263634

   @XuQianJin-Stars @YannByron could you folks chime in?  Is it because the `DROP TABLE` does not delete the files in the table?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] KnightChess commented on issue #6001: [SUPPORT] Cannot create again after deleting the Hudi external table using Spark SQL

Posted by GitBox <gi...@apache.org>.
KnightChess commented on issue #6001:
URL: https://github.com/apache/hudi/issues/6001#issuecomment-1172115069

   try `Create Table for an existing Hudi Table` if you not use `purge` param in drop sql
   <img width="1296" alt="image" src="https://user-images.githubusercontent.com/20125927/176862535-cdc8c32a-3c70-41f1-b75b-6a8ee035c1db.png">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] YannByron commented on issue #6001: [SUPPORT] Cannot create again after deleting the Hudi external table using Spark SQL

Posted by GitBox <gi...@apache.org>.
YannByron commented on issue #6001:
URL: https://github.com/apache/hudi/issues/6001#issuecomment-1185257883

   The table created with `location` will be defined as an external table. Dropping this table will just remove the metadata from metastore, not delete any files in the location.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org