You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/11/13 10:54:40 UTC
[GitHub] [iceberg] get1boat opened a new issue, #6180: can't upsert v2 table by add hint /*+ OPTIONS('upsert-enabled'='true') */
get1boat opened a new issue, #6180:
URL: https://github.com/apache/iceberg/issues/6180
### Apache Iceberg version
1.0.0 (latest release)
### Query engine
Flink
### Please describe the bug 🐞
table DDL:
`
CREATE TABLE hive_catalog.ods_ice.tmp_lkj_kafka2iceberg (
id varchar NOT NULL,
create_time timestamp(6),
user_id bigint
)
WITH (
format = 'PARQUET',
format_version = 2,
location = 'hdfs://eva:8020/user/hive/warehouse/ods_ice.db/tmp_lkj_kafka2iceberg'
)
;
`
with the hint,insert 2 rows which having the same key by trino:
`
insert into hive_catalog.ods_ice.tmp_lkj_kafka2iceberg /*+ OPTIONS('upsert-enabled'='true') */
select '1'as id ,localtimestamp as create_time,111 as user_id
;
insert into hive_catalog.ods_ice.tmp_lkj_kafka2iceberg /*+ OPTIONS('upsert-enabled'='true') */
select '1'as id ,localtimestamp as create_time,222 as user_id
;
`
Then,we select the table and we found it has two rows with the same key.Upsert hint did not work:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] commented on issue #6180: can't upsert v2 table by add hint /*+ OPTIONS('upsert-enabled'='true') */
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6180:
URL: https://github.com/apache/iceberg/issues/6180#issuecomment-1546773775
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] closed issue #6180: can't upsert v2 table by add hint /*+ OPTIONS('upsert-enabled'='true') */
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #6180: can't upsert v2 table by add hint /*+ OPTIONS('upsert-enabled'='true') */
URL: https://github.com/apache/iceberg/issues/6180
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] get1boat commented on issue #6180: can't upsert v2 table by add hint /*+ OPTIONS('upsert-enabled'='true') */
Posted by GitBox <gi...@apache.org>.
get1boat commented on issue #6180:
URL: https://github.com/apache/iceberg/issues/6180#issuecomment-1312945114
>
I created the table in flinksql definitely with the primary key.This DDL was gained by `show create table ` in the Trino.In flink client,this table has a primary key.
The original DDL is blow,and my question remain:
`
create table if not exists hive_catalog.ods_ice.tmp_lkj_kafka2iceberg
(
`id` STRING
,`create_time` timestamp
,`user_id` BIGINT
,primary key(`id`) not enforced
) comment 'kafka2iceberg test table'
with (
'format-version'='2'
)
;
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] krvikash commented on issue #6180: can't upsert v2 table by add hint /*+ OPTIONS('upsert-enabled'='true') */
Posted by GitBox <gi...@apache.org>.
krvikash commented on issue #6180:
URL: https://github.com/apache/iceberg/issues/6180#issuecomment-1312779669
Hi @get1boat,
You are missing the primary key while creating the table.
> Enabling UPSERT mode using upsert-enabled in the [write options](#Write options) provides more flexibility than a table level config. Note that you still need to use v2 table format and specify the primary key when creating the table.
https://iceberg.apache.org/docs/latest/flink/#upsert
Please try adding a `PRIMARY KEY` while creating the table like below.
`CREATE TABLE hive_catalog.ods_ice.tmp_lkj_kafka2iceberg( id varchar NOT NULL, create_time timestamp(6), user_id bigint, PRIMARY KEY(id) NOT ENFORCED ) WITH ( format = 'PARQUET', format_version = 2, location = 'hdfs://eva:8020/user/hive/warehouse/ods_ice.db/tmp_lkj_kafka2iceberg' ) ;`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] commented on issue #6180: can't upsert v2 table by add hint /*+ OPTIONS('upsert-enabled'='true') */
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6180:
URL: https://github.com/apache/iceberg/issues/6180#issuecomment-1565753132
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org