You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "bithw1 (via GitHub)" <gi...@apache.org> on 2024/12/05 12:44:20 UTC
[I] [SUPPORT]hive_sync.skip_ro_suffix looks not working as expected [hudi]
bithw1 opened a new issue, #12011:
URL: https://github.com/apache/hudi/issues/12011
Hi,
I am using Hudi 0.15.0, I find that the option`hoodie.datasource.hive_sync.skip_ro_suffix` looks not working as expected.
I first created an mor table using spark-sql cli(version: spark-3.3.2), with `hoodie.datasource.hive_sync.skip_ro_suffix=true`, then I look into the create table definition from the hive cli
```
CREATE TABLE IF NOT EXISTS hudi_mor_17 (
a INT,
b INT,
c INT
)
USING hudi
tblproperties(
type='mor',
primaryKey='a',
preCombineField='c',
hoodie.datasource.hive_sync.skip_ro_suffix='true'
)
insert into hudi_mor_17 select 1,1,1;
insert into hudi_mor_17 select 2,2,2;
insert into hudi_mor_17 select 3,3,3;
insert into hudi_mor_17 select 4,4,4;
insert into hudi_mor_17 select 5,5,5;
update hudi_mor_17 set b = 11 where a = 1;
update hudi_mor_17 set b = 22 where a = 2;
```
when the table is created and some data insert/update in, I looked into the table definition.
```
CREATE TABLE `hudi_mor_17`(
`_hoodie_commit_time` string,
`_hoodie_commit_seqno` string,
`_hoodie_record_key` string,
`_hoodie_partition_path` string,
`_hoodie_file_name` string,
`a` int,
`b` int,
`c` int)
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
WITH SERDEPROPERTIES (
'hoodie.query.as.ro.table'='false',
'path'='hdfs://hadoop.master:9000/user/hive/warehouse/hudi_mor_17')
STORED AS INPUTFORMAT
'org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
'hdfs://hadoop.master:9000/user/hive/warehouse/hudi_mor_17'
TBLPROPERTIES (
'hoodie.datasource.hive_sync.skip_ro_suffix'='true',
'last_commit_completion_time_sync'='20240926132024516',
'last_commit_time_sync'='20240926132023403',
'preCombineField'='c',
'primaryKey'='a',
'provider'='hudi',
'spark.sql.create.version'='3.3.2',
'spark.sql.sources.provider'='hudi',
'spark.sql.sources.schema.numParts'='1',
'spark.sql.sources.schema.part.0'='{"type":"struct","fields":[{"name":"_hoodie_commit_time","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_commit_seqno","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_record_key","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_partition_path","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_file_name","type":"string","nullable":true,"metadata":{}},{"name":"a","type":"integer","nullable":true,"metadata":{}},{"name":"b","type":"integer","nullable":true,"metadata":{}},{"name":"c","type":"integer","nullable":true,"metadata":{}}]}',
'transient_lastDdlTime'='1727327985',
'type'='mor')
```
and
```
CREATE EXTERNAL TABLE `hudi_mor_17_rt`(
`_hoodie_commit_time` string COMMENT '',
`_hoodie_commit_seqno` string COMMENT '',
`_hoodie_record_key` string COMMENT '',
`_hoodie_partition_path` string COMMENT '',
`_hoodie_file_name` string COMMENT '',
`a` int COMMENT '',
`b` int COMMENT '',
`c` int COMMENT '')
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
WITH SERDEPROPERTIES (
'hoodie.query.as.ro.table'='false',
'path'='hdfs://hadoop.master:9000/user/hive/warehouse/hudi_mor_17')
STORED AS INPUTFORMAT
'org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
'hdfs://hadoop.master:9000/user/hive/warehouse/hudi_mor_17'
TBLPROPERTIES (
'last_commit_completion_time_sync'='20240926132024516',
'last_commit_time_sync'='20240926132023403',
'spark.sql.create.version'='3.3.2',
'spark.sql.sources.provider'='hudi',
'spark.sql.sources.schema.numParts'='1',
'spark.sql.sources.schema.part.0'='{"type":"struct","fields":[{"name":"_hoodie_commit_time","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_commit_seqno","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_record_key","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_partition_path","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_file_name","type":"string","nullable":true,"metadata":{}},{"name":"a","type":"integer","nullable":true,"metadata":{}},{"name":"b","type":"integer","nullable":true,"metadata":{}},{"name":"c","type":"integer","nullable":true,"metadata":{}}]}',
'transient_lastDdlTime'='1727328010')
```
It is expected that `hudi_mor_17_ro` is not created, but the two created tables have same `'hoodie.query.as.ro.table'='false'` and
`STORED AS INPUTFORMAT
'org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat'`,
I query the two tables and am able to query out the updated record,so neither of these two tables are ro tables, I would ask what option`hoodie.datasource.hive_sync.skip_ro_suffix` does, it looks that ro table has bee lost...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [SUPPORT]hive_sync.skip_ro_suffix looks not working as expected [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on issue #12011:
URL: https://github.com/apache/hudi/issues/12011#issuecomment-3461348502
**Hudi 0.14:**
```sh
spark-shell \
--jars $HOME/hudi_jars/hudi-spark3.4-bundle_2.12-0.14.0.jar \
--conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer' \
--conf 'spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog' \
--conf 'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension' \
--conf 'spark.kryo.registrator=org.apache.spark.HoodieSparkKryoRegistrar' \
--conf "spark.sql.catalogImplementation=hive" \
--conf "spark.hadoop.hive.metastore.uris=thrift://localhost:9083"
```
```scala
val databaseName = "default"
val tableName = "my_trips_table_014_mor_skip_ro_suffix_true"
val basePath = f"file:///tmp/hudi/$tableName"
spark.sql(s"SHOW TABLES LIKE '$tableName*'").show(false)
val columns = Seq("ts","uuid","rider","driver","fare","city")
val data =
Seq((1695159649087L,"334e26e9-8355-45cc-97c6-c31daf0df330","rider-A","driver-K",19.10,"san_francisco"),
(1695091554788L,"e96c4396-3fad-413a-a942-4cb36106d721","rider-C","driver-M",27.70 ,"san_francisco"),
(1695046462179L,"9909a8b1-2d15-4d3d-8ec9-efc48c536a00","rider-D","driver-L",33.90 ,"san_francisco"),
(1695516137016L,"e3cf430c-889d-4015-bc98-59bdce1e530c","rider-F","driver-P",34.15,"sao_paulo" ),
(1695115999911L,"c8abbe79-8d89-47ea-b4ce-4d224bae5bfa","rider-J","driver-T",17.85,"chennai"));
var inserts = spark.createDataFrame(data).toDF(columns:_*)
inserts.write.format("hudi").
option("hoodie.table.name", tableName).
option("hoodie.database.name", databaseName).
option("hoodie.datasource.write.table.name", tableName).
option("hoodie.datasource.write.recordkey.field", "uuid").
option("hoodie.datasource.write.precombine.field", "ts").
option("hoodie.datasource.write.partitionpath.field", "city").
option("hoodie.datasource.write.table.type", "MERGE_ON_READ").
option("hoodie.datasource.hive_sync.enable", "true").
option("hoodie.datasource.hive_sync.mode", "hms").
option("hoodie.datasource.hive_sync.database", databaseName).
option("hoodie.datasource.hive_sync.table", tableName).
option("hoodie.datasource.hive_sync.partition_fields", "city").
option("hoodie.datasource.hive_sync.skip_ro.suffix", "true").
mode("overwrite").
save(basePath)
spark.sql(s"SHOW TABLES LIKE '$tableName*'").show(false)
scala> spark.sql(s"SHOW TABLES LIKE '$tableName*'").show(false)
+---------+---------------------------------------------+-----------+
|namespace|tableName |isTemporary|
+---------+---------------------------------------------+-----------+
|default |my_trips_table_014_mor_skip_ro_suffix_true_ro|false |
|default |my_trips_table_014_mor_skip_ro_suffix_true_rt|false |
+---------+---------------------------------------------+-----------+
val databaseName = "default"
val tableName = "my_trips_table_014_mor_skip_ro_suffix_false"
val basePath = f"file:///tmp/hudi/$tableName"
spark.sql(s"SHOW TABLES LIKE '$tableName*'").show(false)
val columns = Seq("ts","uuid","rider","driver","fare","city")
val data =
Seq((1695159649087L,"334e26e9-8355-45cc-97c6-c31daf0df330","rider-A","driver-K",19.10,"san_francisco"),
(1695091554788L,"e96c4396-3fad-413a-a942-4cb36106d721","rider-C","driver-M",27.70 ,"san_francisco"),
(1695046462179L,"9909a8b1-2d15-4d3d-8ec9-efc48c536a00","rider-D","driver-L",33.90 ,"san_francisco"),
(1695516137016L,"e3cf430c-889d-4015-bc98-59bdce1e530c","rider-F","driver-P",34.15,"sao_paulo" ),
(1695115999911L,"c8abbe79-8d89-47ea-b4ce-4d224bae5bfa","rider-J","driver-T",17.85,"chennai"));
var inserts = spark.createDataFrame(data).toDF(columns:_*)
inserts.write.format("hudi").
option("hoodie.table.name", tableName).
option("hoodie.database.name", databaseName).
option("hoodie.datasource.write.table.name", tableName).
option("hoodie.datasource.write.recordkey.field", "uuid").
option("hoodie.datasource.write.precombine.field", "ts").
option("hoodie.datasource.write.partitionpath.field", "city").
option("hoodie.datasource.write.table.type", "MERGE_ON_READ").
option("hoodie.datasource.hive_sync.enable", "true").
option("hoodie.datasource.hive_sync.mode", "hms").
option("hoodie.datasource.hive_sync.database", databaseName).
option("hoodie.datasource.hive_sync.table", tableName).
option("hoodie.datasource.hive_sync.partition_fields", "city").
option("hoodie.datasource.hive_sync.skip_ro.suffix", "false").
mode("overwrite").
save(basePath)
spark.sql(s"SHOW TABLES LIKE '$tableName*'").show(false)
scala> spark.sql(s"SHOW TABLES LIKE '$tableName*'").show(false)
+---------+----------------------------------------------+-----------+
|namespace|tableName |isTemporary|
+---------+----------------------------------------------+-----------+
|default |my_trips_table_014_mor_skip_ro_suffix_false_ro|false |
|default |my_trips_table_014_mor_skip_ro_suffix_false_rt|false |
+---------+----------------------------------------------+-----------+
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [SUPPORT]hive_sync.skip_ro_suffix looks not working as expected [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy closed issue #12011: [SUPPORT]hive_sync.skip_ro_suffix looks not working as expected
URL: https://github.com/apache/hudi/issues/12011
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [SUPPORT]hive_sync.skip_ro_suffix looks not working as expected [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on issue #12011:
URL: https://github.com/apache/hudi/issues/12011#issuecomment-2400976344
> The input format of a ro table should be HoodieParquetInputFormat.
@xicm Looks like a bug, could you fire a fix for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [SUPPORT]hive_sync.skip_ro_suffix looks not working as expected [hudi]
Posted by "ad1happy2go (via GitHub)" <gi...@apache.org>.
ad1happy2go commented on issue #12011:
URL: https://github.com/apache/hudi/issues/12011#issuecomment-2376082279
Thanks for raising this @bithw1 . I confirmed this is the regression for release 0.15.0. With 0.14.X versions it works fine. i will created jIRA for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [SUPPORT]hive_sync.skip_ro_suffix looks not working as expected [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on issue #12011:
URL: https://github.com/apache/hudi/issues/12011#issuecomment-3491178233
In the latest version using hoodie.datasource.hive_sync.skip_ro.suffix parameter are not useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [SUPPORT]hive_sync.skip_ro_suffix looks not working as expected [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on issue #12011:
URL: https://github.com/apache/hudi/issues/12011#issuecomment-3461280230
Hi Team,
When testing with the parameter `hoodie.datasource.hive_sync.skip_ro.suffix` set to true, Hudi unexpectedly still created the read-optimized (RO) suffix table.
```sh
spark-shell \
--jars $HOME/hudi_jars/hudi-spark3.5-bundle_2.12-1.0.2.jar \
--conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer' \
--conf 'spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog' \
--conf 'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension' \
--conf 'spark.kryo.registrator=org.apache.spark.HoodieSparkKryoRegistrar' \
--conf "spark.sql.catalogImplementation=hive" \
--conf "spark.hadoop.hive.metastore.uris=thrift://localhost:9083"
```
```scala
val databaseName = "default"
val tableName = "my_trips_table_mor_skip_ro_suffix_true"
val basePath = f"file:///tmp/hudi/$tableName"
spark.sql(f"show tables like '$tableName'").show(false)
val columns = Seq("ts","uuid","rider","driver","fare","city")
val data =
Seq((1695159649087L,"334e26e9-8355-45cc-97c6-c31daf0df330","rider-A","driver-K",19.10,"san_francisco"),
(1695091554788L,"e96c4396-3fad-413a-a942-4cb36106d721","rider-C","driver-M",27.70 ,"san_francisco"),
(1695046462179L,"9909a8b1-2d15-4d3d-8ec9-efc48c536a00","rider-D","driver-L",33.90 ,"san_francisco"),
(1695516137016L,"e3cf430c-889d-4015-bc98-59bdce1e530c","rider-F","driver-P",34.15,"sao_paulo" ),
(1695115999911L,"c8abbe79-8d89-47ea-b4ce-4d224bae5bfa","rider-J","driver-T",17.85,"chennai"));
var inserts = spark.createDataFrame(data).toDF(columns:_*)
inserts.write.format("hudi").
option("hoodie.table.name", tableName).
option("hoodie.database.name", databaseName).
option("hoodie.datasource.write.table.name", tableName).
option("hoodie.datasource.write.recordkey.field", "uuid").
option("hoodie.datasource.write.precombine.field", "ts").
option("hoodie.datasource.write.partitionpath.field", "city").
option("hoodie.datasource.write.table.type", "MERGE_ON_READ").
option("hoodie.datasource.hive_sync.enable", "true").
option("hoodie.datasource.hive_sync.mode", "hms").
option("hoodie.datasource.hive_sync.database", databaseName).
option("hoodie.datasource.hive_sync.table", tableName).
option("hoodie.datasource.hive_sync.partition_fields", "city").
option("hoodie.datasource.hive_sync.skip_ro.suffix", "true").
mode("overwrite").
save(basePath)
scala> spark.sql(s"SHOW TABLES LIKE '$tableName*'").show(false)
+---------+-----------------------------------------+-----------+
|namespace|tableName |isTemporary|
+---------+-----------------------------------------+-----------+
|default |my_trips_table_mor_skip_ro_suffix_true |false |
|default |my_trips_table_mor_skip_ro_suffix_true_ro|false |
|default |my_trips_table_mor_skip_ro_suffix_true_rt|false |
+---------+-----------------------------------------+-----------+
val databaseName = "default"
val tableName = "my_trips_table_mor_skip_ro_suffix_false"
val basePath = f"file:///tmp/hudi/$tableName"
spark.sql(s"SHOW TABLES LIKE '$tableName*'").show(false)
val columns = Seq("ts","uuid","rider","driver","fare","city")
val data =
Seq((1695159649087L,"334e26e9-8355-45cc-97c6-c31daf0df330","rider-A","driver-K",19.10,"san_francisco"),
(1695091554788L,"e96c4396-3fad-413a-a942-4cb36106d721","rider-C","driver-M",27.70 ,"san_francisco"),
(1695046462179L,"9909a8b1-2d15-4d3d-8ec9-efc48c536a00","rider-D","driver-L",33.90 ,"san_francisco"),
(1695516137016L,"e3cf430c-889d-4015-bc98-59bdce1e530c","rider-F","driver-P",34.15,"sao_paulo" ),
(1695115999911L,"c8abbe79-8d89-47ea-b4ce-4d224bae5bfa","rider-J","driver-T",17.85,"chennai"));
var inserts = spark.createDataFrame(data).toDF(columns:_*)
inserts.write.format("hudi").
option("hoodie.table.name", tableName).
option("hoodie.database.name", databaseName).
option("hoodie.datasource.write.table.name", tableName).
option("hoodie.datasource.write.recordkey.field", "uuid").
option("hoodie.datasource.write.precombine.field", "ts").
option("hoodie.datasource.write.partitionpath.field", "city").
option("hoodie.datasource.write.table.type", "MERGE_ON_READ").
option("hoodie.datasource.hive_sync.enable", "true").
option("hoodie.datasource.hive_sync.mode", "hms").
option("hoodie.datasource.hive_sync.database", databaseName).
option("hoodie.datasource.hive_sync.table", tableName).
option("hoodie.datasource.hive_sync.partition_fields", "city").
option("hoodie.datasource.hive_sync.skip_ro.suffix", "false").
mode("overwrite").
save(basePath)
spark.sql(s"SHOW TABLES LIKE '$tableName*'").show(false)
spark.sql(s"SHOW TABLES LIKE '$tableName*'").show(false)
+---------+------------------------------------------+-----------+
|namespace|tableName |isTemporary|
+---------+------------------------------------------+-----------+
|default |my_trips_table_mor_skip_ro_suffix_false |false |
|default |my_trips_table_mor_skip_ro_suffix_false_ro|false |
|default |my_trips_table_mor_skip_ro_suffix_false_rt|false |
+---------+------------------------------------------+-----------+
scala> spark.sql("show create table my_trips_table_mor_skip_ro_suffix_false_rt").show(100, false)
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|createtab_stmt |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|CREATE TABLE spark_catalog.default.my_trips_table_mor_skip_ro_suffix_false_rt (\n _hoodie_commit_time STRING,\n _hoodie_commit_seqno STRING,\n _hoodie_record_key STRING,\n _hoodie_partition_path STRING,\n _hoodie_file_name STRING,\n ts BIGINT NOT NULL,\n uuid STRING,\n rider STRING,\n driver STRING,\n fare DOUBLE NOT NULL,\n city STRING)\nUSING hudi\nOPTIONS (\n 'hoodie.query.as.ro.table' = 'false')\nPARTITIONED BY (city)\nLOCATION 'file:/tmp/hudi/my_trips_table_mor_skip_ro_suffix_false'\nTBLPROPERTIES (\n 'last_commit_completion_time_sync' = '20251029174433737',\n 'last_commit_time_sync' = '20251029174429189')\n|
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
scala> spark.sql("show create table my_trips_table_mor_skip_ro_suffix_false_ro").show(100, false)
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|createtab_stmt |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|CREATE TABLE spark_catalog.default.my_trips_table_mor_skip_ro_suffix_false_ro (\n _hoodie_commit_time STRING,\n _hoodie_commit_seqno STRING,\n _hoodie_record_key STRING,\n _hoodie_partition_path STRING,\n _hoodie_file_name STRING,\n ts BIGINT NOT NULL,\n uuid STRING,\n rider STRING,\n driver STRING,\n fare DOUBLE NOT NULL,\n city STRING)\nUSING hudi\nOPTIONS (\n 'hoodie.query.as.ro.table' = 'true')\nPARTITIONED BY (city)\nLOCATION 'file:/tmp/hudi/my_trips_table_mor_skip_ro_suffix_false'\nTBLPROPERTIES (\n 'last_commit_completion_time_sync' = '20251029174433737',\n 'last_commit_time_sync' = '20251029174429189')\n|
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
scala> spark.sql("show create table my_trips_table_mor_skip_ro_suffix_false").show(100, false)
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|createtab_stmt |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|CREATE TABLE spark_catalog.default.my_trips_table_mor_skip_ro_suffix_false (\n _hoodie_commit_time STRING,\n _hoodie_commit_seqno STRING,\n _hoodie_record_key STRING,\n _hoodie_partition_path STRING,\n _hoodie_file_name STRING,\n ts BIGINT NOT NULL,\n uuid STRING,\n rider STRING,\n driver STRING,\n fare DOUBLE NOT NULL,\n city STRING)\nUSING hudi\nOPTIONS (\n 'hoodie.query.as.ro.table' = 'false')\nPARTITIONED BY (city)\nLOCATION 'file:/tmp/hudi/my_trips_table_mor_skip_ro_suffix_false'\nTBLPROPERTIES (\n 'last_commit_completion_time_sync' = '20251029174433737',\n 'last_commit_time_sync' = '20251029174429189')\n|
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [SUPPORT]hive_sync.skip_ro_suffix looks not working as expected [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on issue #12011:
URL: https://github.com/apache/hudi/issues/12011#issuecomment-3466960875
Hi @bithw1
In the latest code, when we sync the Hudi MOR table to Hive, three tables are created: table_name, table_name_ro, and table_name_rt. Please let me know if you require any further assistance on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [SUPPORT]hive_sync.skip_ro_suffix looks not working as expected [hudi]
Posted by "ad1happy2go (via GitHub)" <gi...@apache.org>.
ad1happy2go commented on issue #12011:
URL: https://github.com/apache/hudi/issues/12011#issuecomment-2376324631
JIRA - https://issues.apache.org/jira/browse/HUDI-8264
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org