[I] [Bug] Read metadata info error for pk table [paimon]

["askwang (via GitHub)" <gi...@apache.org>]

askwang opened a new issue, #5467:
URL: https://github.com/apache/paimon/issues/5467

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
   
   
   ### Paimon version
   
   1.0
   
   ### Compute Engine
   
   spark
   
   ### Minimal reproduce step
   
   When we write twice and read metadata info throws exception.
   ```sql
     test("xxx-pk table") {
       sql("""
             |CREATE TABLE test_tb (
             |    id BIGINT,
             |    name STRING
             |) USING paimon
             |TBLPROPERTIES (
             |    'primary-key' = 'id',
             |    'file.format' = 'parquet',
             |    'file.compression' = 'snappy',
             |    'bucket' = '1'
             |);
             |""".stripMargin)
   
       spark.sql("insert into test_tb values(1,'a');")
       spark.sql("select __paimon_file_path from test_tb;").show(false)
   
       spark.sql("insert into test_tb values(3,'c');")
       // throw exception
       spark.sql("select __paimon_file_path from test_tb;").show(false)
     }
   ```
   
   exceptions:
   ```text
   java.lang.RuntimeException: There need be FileRecordIterator when metadata columns are required.
   	at org.apache.paimon.spark.PaimonRecordReaderIterator.validateMetadataColumns(PaimonRecordReaderIterator.scala:53)
   	at org.apache.paimon.spark.PaimonRecordReaderIterator.advanceIfNeeded(PaimonRecordReaderIterator.scala:114)
   	at org.apache.paimon.spark.PaimonRecordReaderIterator.hasNext(PaimonRecordReaderIterator.scala:62)
   	at org.apache.paimon.spark.PaimonPartitionReader.advanceIfNeeded(PaimonPartitionReader.scala:70)
   	at org.apache.paimon.spark.PaimonPartitionReader.next(PaimonPartitionReader.scala:51)
   	at org.apache.spark.sql.execution.datasources.v2.PartitionIterator.hasNext(DataSourceRDD.scala:119)
   ```
   
   ### What doesn't meet your expectations?
   
   exception
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Read metadata info error for pk table [paimon]

["askwang (via GitHub)" <gi...@apache.org>]

askwang commented on issue #5467:
URL: https://github.com/apache/paimon/issues/5467#issuecomment-2823766328

   Only append table or deletion vector table support querying metadata columns


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Read metadata info error for pk table [paimon]

["askwang (via GitHub)" <gi...@apache.org>]

askwang closed issue #5467: [Bug] Read metadata info error for pk table
URL: https://github.com/apache/paimon/issues/5467


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org