You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Ashok Kumar (Jira)" <ji...@apache.org> on 2020/08/27 02:26:00 UTC

[jira] [Created] (HUDI-1231) Duplicate record while querying

Ashok Kumar created HUDI-1231:
---------------------------------

             Summary: Duplicate record while querying
                 Key: HUDI-1231
                 URL: https://issues.apache.org/jira/browse/HUDI-1231
             Project: Apache Hudi
          Issue Type: Bug
            Reporter: Ashok Kumar


I am writting in upsert mode with precombine flag enabled. Still when i query i see same record available 3 times in same parquet file

 

spark.sql("select _hoodie_commit_time,_hoodie_commit_seqno,_hoodie_record_key,_hoodie_partition_path,_hoodie_file_name from hudi5_mor_ro where id1=1086187 and timestamp=1598461500 and _hoodie_record_key='timestamp:1598461500,id1:1086187,id2:1872725,flowId:23'").show(10,false)
+-------------------+---------------------------+----------------------------------------------------------------+----------------------+------------------------------------------------------------------------------+
|_hoodie_commit_time|_hoodie_commit_seqno |_hoodie_record_key |_hoodie_partition_path|_hoodie_file_name |
+-------------------+---------------------------+----------------------------------------------------------------+----------------------+------------------------------------------------------------------------------+
|20200826171813 |20200826171813_13856_855766|timestamp:1598461500,id1:1086187,id2:1872725,flowId:23|1086187/2020082617 |5ecb020f-29be-4eed-b130-8c02ae819603-0_13856-104-296775_20200826171813.parquet|
|20200826171813 |20200826171813_13856_855766|timestamp:1598461500,id1:1086187,id2:1872725,flowId:23|1086187/2020082617 |5ecb020f-29be-4eed-b130-8c02ae819603-0_13856-104-296775_20200826171813.parquet|
|20200826171813 |20200826171813_13856_855766|timestamp:1598461500,id1:1086187,id2:1872725,flowId:23|1086187/2020082617 |5ecb020f-29be-4eed-b130-8c02ae819603-0_13856-104-296775_20200826171813.parquet|
+-------------------+---------------------------+----------------------------------------------------------------+----------------------+------------------------------------------------------------------------------+

 

This issue i am getting with both kind of table i.e COW and MOR. 

I have tried it 0.6.3 version but i had tried 0.5.3 and in that also this bug was coming.

This issue is not coming with small data set. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)