[I] [Bug][Rust] Fixed-schema Arrow scans return NULL for values written before a column rename [fluss]

["slfan1989 (via GitHub)" <gi...@apache.org>]

slfan1989 opened a new issue, #3768:
URL: https://github.com/apache/fluss/issues/3768

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
   
   
   ### Fluss version
   
   0.9.0 (latest release)
   
   ### Please describe the bug 🐞
   
   ### Description
   
   Fixed-schema Arrow log scans align batches written with older schemas to the
   schema captured when the scanner is created.
   
   The current alignment matches columns by Arrow field name. After a column is
   renamed, an old batch still contains the old field name, so the renamed target
   column is treated as missing and filled with NULL values, even though the old
   and new columns have the same stable Fluss column ID.
   
   This causes silent data loss in scan results.
   
   ### Steps to reproduce
   
   1. Create a log table with the following schema:
   
   ```text
      id INT,
      old_name STRING
   ```
   
   2. Append and flush records containing `non-null` values in `old_name`.
   
   3. Rename the column using `ALTER TABLE` or the Rust Admin API:```
   old_name -> new_name
   ```
   
   4. Obtain the updated table handle.
   5. Create a fixed-schema record-batch log scanner and subscribe from the
   earliest offset.
   6. Read the records written before the rename.
   
   ### Solution
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug][Rust] Fixed-schema Arrow scans return NULL for values written before a column rename [fluss]

["slfan1989 (via GitHub)" <gi...@apache.org>]

slfan1989 closed issue #3768: [Bug][Rust] Fixed-schema Arrow scans return NULL for values written before a column rename
URL: https://github.com/apache/fluss/issues/3768


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org