Re: [I] Compound RLI does not escape the char that used as separator [hudi]

["rangareddy (via GitHub)" <gi...@apache.org>]

rangareddy commented on issue #17168:
URL: https://github.com/apache/hudi/issues/17168#issuecomment-5366103879

   This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-9738).
   
   **Findings: confirmed, and the parsing code is itself the strongest evidence.**
   
   **Write side does not escape.** `hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/BuiltinKeyGenerator.java:224` and `:275` append `DEFAULT_RECORD_KEY_PARTS_SEPARATOR` directly around raw field values, so a value containing the separator goes in verbatim - which is how you get `customer_name:Bob, Johnson`.
   
   **Read side compensates with a heuristic.** `KeyGenUtils.extractRecordKeysByFields` (`hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java:139`) starts with:
   
   ```java
   // if there is no ',' and ':', then it's a key value
   if (!recordKey.contains(DEFAULT_RECORD_KEY_PARTS_SEPARATOR) || !recordKey.contains(DEFAULT_COLUMN_VALUE_SEPARATOR)) {
     return new String[] {recordKey};
   }
   ```
   
   and then does index arithmetic with repeated `lastIndexOf(DEFAULT_RECORD_KEY_PARTS_SEPARATOR, keyValueSep2)` calls at `:166`, `:181`, `:204` and `:208` - scanning backwards to guess where one field ends and the next begins.
   
   That backwards-scanning logic exists *only* because the separator is unescaped in the value. It is a heuristic, which means whether a key round-trips correctly depends on the data rather than on the format. Your caution is well placed: absence of an observed correctness bug is not the same as correctness here, and a value containing `:` as well as `,` is the case most likely to break it.
   
   Worth noting that changing the encoding is an on-disk compatibility change for existing record-index entries, so it likely needs to ride an index-version bump rather than being fixed in place.
   
   Keeping this open.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org