You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/06/29 09:52:48 UTC
[GitHub] [hudi] JoshuaZhuCN opened a new issue, #6004: [SUPPORT] Creating a Hudi external table will modify the value of "hoodie.table.name" in the "hoodie.properties" file
JoshuaZhuCN opened a new issue, #6004:
URL: https://github.com/apache/hudi/issues/6004
Creating a Hudi external table will modify the value of "hoodie.table.name" in the "hoodie.properties" file.
This problem does not occur when using auto sync to hive.
Then how can i create _rt table manually by using spark sql?
**To Reproduce**
Steps to reproduce the behavior:
1.execute the sql to create an external table
```
create table `default`.`spark_hudi_test_ddl` (`id` int, `vname` string, `price` decimal(14, 2), `sync_time` timestamp) using hudi
options(`hoodie.query.as.ro.table` = 'true')
TBLPROPERTIES (type='mor', primaryKey='id', preCombineField='sync_time')
location 'hdfs://localhost:9000/hoodie/test/oms/spark_hudi_ddl';
```
2. see the hoodie.properties file，the value of hoodie.table.name is "spark_hudi_test_ddl"
3. try to execute the sql to create an _rt table
```
create table `default`.`spark_hudi_test_ddl_rt`
using hudi
options(`hoodie.query.as.ro.table` = 'false')
TBLPROPERTIES (type='mor', primaryKey='id', preCombineField='sync_time')
location 'hdfs://localhost:9000/hoodie/test/oms/spark_hudi_ddl';
```
4. see the hoodie.properties file，the value of hoodie.table.name is "spark_hudi_test_ddl_rt"
**Expected behavior**
A clear and concise description of what you expected to happen.
**Environment Description**
* Hudi version : 0.10.1
* Spark version : 3.1.3
* Hive version : 3.1.0
* Hadoop version : 3.1.1
* Storage (HDFS/S3/GCS..) : hdfs
* Running on Docker? (yes/no) : no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] YannByron commented on issue #6004: [SUPPORT] Creating a Hudi external table will modify the value of "hoodie.table.name" in the "hoodie.properties" file
Posted by GitBox <gi...@apache.org>.
YannByron commented on issue #6004:
URL: https://github.com/apache/hudi/issues/6004#issuecomment-1174531092
@yihua copy that.
1. there should be a judgement to make it pass or throw an exception about the different properties;
2. if you want to create a rt table that can be queried in hive or other non-spark engine. you maybe execute the sql in hive , or in spark. @JoshuaZhuCN
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on issue #6004: [SUPPORT] Creating a Hudi external table will modify the value of "hoodie.table.name" in the "hoodie.properties" file
Posted by GitBox <gi...@apache.org>.
yihua commented on issue #6004:
URL: https://github.com/apache/hudi/issues/6004#issuecomment-1170265977
@XuQianJin-Stars @YannByron could you folks help here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] YannByron commented on issue #6004: [SUPPORT] Creating a Hudi external table will modify the value of "hoodie.table.name" in the "hoodie.properties" file
Posted by GitBox <gi...@apache.org>.
YannByron commented on issue #6004:
URL: https://github.com/apache/hudi/issues/6004#issuecomment-1185255635
if create a mor table in spark, the rt table and ro table will be created automatically at the same time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] YannByron commented on issue #6004: [SUPPORT] Creating a Hudi external table will modify the value of "hoodie.table.name" in the "hoodie.properties" file
Posted by GitBox <gi...@apache.org>.
YannByron commented on issue #6004:
URL: https://github.com/apache/hudi/issues/6004#issuecomment-1196426361
@yihua @JoshuaZhuCN if no more questions, can close this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] YannByron commented on issue #6004: [SUPPORT] Creating a Hudi external table will modify the value of "hoodie.table.name" in the "hoodie.properties" file
Posted by GitBox <gi...@apache.org>.
YannByron commented on issue #6004:
URL: https://github.com/apache/hudi/issues/6004#issuecomment-1196424296
create a ticket: https://issues.apache.org/jira/browse/HUDI-4486 to trace the question 1 mentioned above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan closed issue #6004: [SUPPORT] Creating a Hudi external table will modify the value of "hoodie.table.name" in the "hoodie.properties" file
Posted by GitBox <gi...@apache.org>.
nsivabalan closed issue #6004: [SUPPORT] Creating a Hudi external table will modify the value of "hoodie.table.name" in the "hoodie.properties" file
URL: https://github.com/apache/hudi/issues/6004
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] JoshuaZhuCN commented on issue #6004: [SUPPORT] Creating a Hudi external table will modify the value of "hoodie.table.name" in the "hoodie.properties" file
Posted by GitBox <gi...@apache.org>.
JoshuaZhuCN commented on issue #6004:
URL: https://github.com/apache/hudi/issues/6004#issuecomment-1174739155
> @yihua copy that.
>
> 1. there should be a judgement to make it pass or throw an exception about the different properties;
> 2. if you want to create a rt table that can be queried in hive or other non-spark engine. you maybe execute the sql in hive , or in spark. @JoshuaZhuCN
@YannByron
2、These two DDLS are executed in spark SQL. Do you mean that the other DDL needs to be created in other engines？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] YannByron commented on issue #6004: [SUPPORT] Creating a Hudi external table will modify the value of "hoodie.table.name" in the "hoodie.properties" file
Posted by GitBox <gi...@apache.org>.
YannByron commented on issue #6004:
URL: https://github.com/apache/hudi/issues/6004#issuecomment-1196783678
another ticket: https://issues.apache.org/jira/browse/HUDI-4487 to support that create ro/rt table by spark-sql.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org