[I] InMemorySetState.retract NPEs on an absent secondary-index key in MEMORY-cache lookup join [paimon]

["LuciferYang (via GitHub)" <gi...@apache.org>]

LuciferYang opened a new issue, #10279:
URL: https://github.com/apache/paimon/issues/10279

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found no similar issues.
   
   ### Paimon version
   
   master (1.5-SNAPSHOT)
   
   ### Compute Engine
   
   Flink (secondary-index lookup join with `lookup.cache = MEMORY`).
   
   ### Minimal reproduce step
   
   1. Run a lookup join whose join key is not the table's primary key (a secondary-index lookup), with `lookup.cache = MEMORY` and a lookup filter (predicate).
   2. Feed a changelog where a row that does not pass the predicate is later deleted (a `-D` / `-U` for that key).
   
   ### What doesn't meet your expectations?
   
   The lookup refresh crashes with a `NullPointerException`. `SecondaryIndexLookupTable.refreshRow` only adds a row to the index state when the predicate passes (on `+I` / `+U`), but retracts unconditionally on `-D` / `-U`. So a row that was filtered out was never added, and its later retract reaches `InMemorySetState.retract`, which does `values.get(secKey).remove(...)` with no null check and throws. The disk-backed sibling `LocalKvSetState.retract` already treats an absent key as a no-op, so the same join crashes only in `MEMORY` cache mode.
   
   ### Anything else?
   
   This is a behavior inconsistency between the two `SetState` implementations; `MEMORY` mode should tolerate the absent-key retract like the disk-backed one.
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] InMemorySetState.retract NPEs on an absent secondary-index key in MEMORY-cache lookup join [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi closed issue #10279: InMemorySetState.retract NPEs on an absent secondary-index key in MEMORY-cache lookup join
URL: https://github.com/apache/paimon/issues/10279


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org