[GitHub] [iceberg] rdblue commented on pull request #4415: Flink - Demonstrate (position-based) bug with multiple upserts to same row

[GitBox <gi...@apache.org>]

rdblue commented on PR #4415:
URL: https://github.com/apache/iceberg/pull/4415#issuecomment-1173192373

   I just spent some time debugging this and I think the problem is with Flink, not with Iceberg.
   
   When I run the case that fails, the issue is that the upserted rows are out of order. In both the final insert data file and the records that I see in the debugger, the column with `bool=true` is passed in last. Because this is an upsert, the last row is the version that you end up with. I'm not sure why Flink is passing the rows in the wrong order, but it could be that the parser doesn't keep track of the order of rows in a `VALUES` clause.
   
   I also did a little exploration into what is different with the "succeeds" case. The significant change between the two in my testing was the use of `TO_DATE('2022-03-01')` instead of `DATE '2022-03-01'`. `TO_DATE` appears to trigger the rows getting out of order.
   
   I'm going to close this issue. Feel free to reopen if you think it is a problem with Iceberg somehow reordering the rows, but that seems unlikely given that these rows are coming from Flink and are identical when they come through the Iceberg API (the `TO_DATE` / `DATE` distinction is internal to Flink).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org