[I] [Bug] lookup compaction may lead to wrong results. [paimon]

["zhoulii (via GitHub)" <gi...@apache.org>]

zhoulii opened a new issue, #5631:
URL: https://github.com/apache/paimon/issues/5631

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
   
   
   ### Paimon version
   
   1.0
   
   ### Compute Engine
   
   flink 1.18
   
   ### Minimal reproduce step
   
   `LookupChangelogMergeFunctionWrapper function =
                   new LookupChangelogMergeFunctionWrapper(
                           LookupMergeFunction.wrap(DeduplicateMergeFunction.factory()),
                           highLevel::get,
                           null,
                           LookupStrategy.from(false, true, false, false),
                           null,
                           null);
   
           function.reset();
           function.add(new KeyValue().replace(row(1), 1, INSERT, row(2)).setLevel(1));
           function.add(new KeyValue().replace(row(1), 1, INSERT, row(1)).setLevel(2));
           ChangelogResult result = function.getResult();
           assertThat(result).isNotNull();
           assertThat(result.changelogs()).isEmpty();
           KeyValue kv = result.result();
           assertThat(kv).isNotNull();
           assertThat(kv.value().getInt(0)).isEqualTo(2);`
   
   ### What doesn't meet your expectations?
   
   
   <img width="1422" alt="Image" src="https://github.com/user-attachments/assets/aab721fe-4323-4c4a-83e3-9e083daeb4ea" />
   
   A key may exist in multi high-levels and have same sequence number，when merge the same key in multi high-levels with lookup compaction, we should keep the record with lowest level number. The current implement cannot guarantee this.
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] lookup compaction may lead to wrong results. [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi closed issue #5631: [Bug] lookup compaction may lead to wrong results.
URL: https://github.com/apache/paimon/issues/5631


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org