[I] Paimon V2 native scan does not handle metadata columns correctly [auron]

["lyne7-sc (via GitHub)" <gi...@apache.org>]

lyne7-sc opened a new issue, #2366:
URL: https://github.com/apache/auron/issues/2366

   **Describe the bug**
   <!--
   A clear and concise description of what the bug is.
   -->
   
   Paimon exposes metadata columns such as `__paimon_file_path` and `__paimon_bucket`. These columns are produced by Paimon's scan layer and are not physical columns in the data files.
   
   The Paimon V2 native scan currently classifies requested metadata columns as file columns, so the native Parquet/ORC reader tries to read them from the data file payload. This can produce incorrect values, for example `null` for `__paimon_file_path`.
   
   **To Reproduce**
   <!--
   Steps to reproduce the behavior:
   1. Go to '...'
   2. Click on '....'
   3. Scroll down to '....'
   4. See error
   -->
   Run the following query with native Paimon V2 scan enabled:
   
     ```sql
   create table paimon.db.t_metadata (id int, v string) using paimon;
   insert into paimon.db.t_metadata values (1, 'a');
   select id, __paimon_file_path from paimon.db.t_metadata;
   ```
   
   The query returns null for `__paimon_file_path` on the native scan path.
   
   **Expected behavior**
   <!--
   A clear and concise description of what you expected to happen.
   -->
   The native scan should return the same metadata values as Spark/Paimon's scan path.
   
   **Screenshots**
   <!--
   If applicable, add screenshots to help explain your problem.
   -->
   
   **Additional context**
   <!--
   Add any other context about the problem here.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Paimon V2 native scan does not handle metadata columns correctly [auron]

["SteNicholas (via GitHub)" <gi...@apache.org>]

SteNicholas closed issue #2366: Paimon V2 native scan does not handle metadata columns correctly
URL: https://github.com/apache/auron/issues/2366


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org