You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "vbabenkoru (via GitHub)" <gi...@apache.org> on 2026/07/31 23:03:14 UTC

[I] [Bug][iceberg] REST commit fails on format-version 3: Cannot parse missing long: next-row-id [paimon]

vbabenkoru opened a new issue, #8972:
URL: https://github.com/apache/paimon/issues/8972

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
   
   
   ### Paimon version
   
   2.0.0-rc4
   
   ### Compute Engine
   
   Flink 2.2.0
   
   ### Minimal reproduce step
   
   ```
   CREATE TABLE t (
     project_id BIGINT,
     email STRING,
     ...
     PRIMARY KEY (project_id, email) NOT ENFORCED
   ) PARTITIONED BY (project_id) WITH (
     'bucket' = '16',
     'changelog-producer' = 'lookup',
     'metadata.iceberg.storage' = 'rest-catalog',
     'metadata.iceberg.format-version' = '3',
     -- or enable deletion-vectors (docs: DV requires format-version 3)
     'metadata.iceberg.rest.uri' = 'https://glue.<region>.amazonaws.com/iceberg',
     'metadata.iceberg.rest.warehouse' = '<account-id>',
     'metadata.iceberg.rest.io-impl' = 'org.apache.iceberg.aws.s3.S3FileIO',
     'metadata.iceberg.rest.rest.sigv4-enabled' = 'true',
     'metadata.iceberg.rest.rest.signing-name' = 'glue',
     'metadata.iceberg.rest.rest.region' = '<region>',
     'metadata.iceberg.rest.client.credentials-provider' =
       'software.amazon.awssdk.auth.credentials.WebIdentityTokenFileCredentialsProvider'
   );
   
   -- insert / Flink sink until a Paimon snapshot commits
   ```
   Inspect generated metadata (example): format-version is 3, but top-level next-row-id is absent. Snapshots may carry first-row-id / added-rows, but table-level next-row-id is missing — Iceberg 1.11 requires it for v3.
   
   ### What doesn't meet your expectations?
   
   The Flink job fails on a commit with error `Cannot parse missing long: next-row-id`
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug][iceberg] REST commit fails on format-version 3: Cannot parse missing long: next-row-id [paimon]

Posted by "vbabenkoru (via GitHub)" <gi...@apache.org>.
vbabenkoru commented on issue #8972:
URL: https://github.com/apache/paimon/issues/8972#issuecomment-5173437919

   After patching next-row-id in a fork, I'm still getting `SQL Compilation error: Invalid first_row_id 'null' for entry in manifest list file for V3 snapshot.` when trying to read the Paimon-generated Iceberg tables. So looks like Paimon's implementation of Iceberg v3 does not work.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug][iceberg] REST commit fails on format-version 3: Cannot parse missing long: next-row-id [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi closed issue #8972: [Bug][iceberg] REST commit fails on format-version 3: Cannot parse missing long: next-row-id
URL: https://github.com/apache/paimon/issues/8972


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug][iceberg] REST commit fails on format-version 3: Cannot parse missing long: next-row-id [paimon]

Posted by "vbabenkoru (via GitHub)" <gi...@apache.org>.
vbabenkoru commented on issue #8972:
URL: https://github.com/apache/paimon/issues/8972#issuecomment-5148186874

   Iceberg v3 row lineage requires table metadata next-row-id (see Iceberg TableMetadataParser). Paimon’s IcebergMetadata.toJson() emits v3 without that field, so TableMetadataParser.fromJson fails before any REST createTable/updateTable.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org