You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "xuCoCo (via GitHub)" <gi...@apache.org> on 2023/06/12 09:42:03 UTC
[GitHub] [hudi] xuCoCo commented on issue #8916: [SUPPORT] Spark SQL creates a mor table, and writing data will not deduplicate
xuCoCo commented on issue #8916:
URL: https://github.com/apache/hudi/issues/8916#issuecomment-1586972612
> This is a known behavior, instead of using `INSERT INTO`, tries `MERGE INTO` instead.
Thank you for your reply, I tried MERGE INTO
```
merge into spark_mor_no_pre_t10 as target
using (
select 1 as id, 'a' as name
union select 1 as id,'b' as name
) source
on target.id = source.id
when matched then update set *
when not matched then insert *
```
Still the same problem id=1 data duplication
My problem is that when the table has data, I also insert a batch of data with duplicate ids, and the result is deduplication, such as id=2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org