[I] `TableCreation::format_version` is ignored when creating a table using the Iceberg REST catalog interface [iceberg-rust]

["JosephLenton (via GitHub)" <gi...@apache.org>]

JosephLenton opened a new issue, #2923:
URL: https://github.com/apache/iceberg-rust/issues/2923

   ### Apache Iceberg Rust version
   
   None
   
   ### Describe the bug
   
   When using the Rest catalog, the `TableCreation::format_version` is silently ignored and never used. The result is you can't set a table to be V3 when using `format_version` and a REST catalog.
   
   # Steps to reproduce
   
    * Spin up a Rest catalog (say Polaris), and use that.
    * Make a new table.
    * Set it to use `format_version(FormatVersion::V3)`
    * Then add a V3 column, and try to use the table.
    * Outcome: the table is in V2 format (not v3).
   
   The table building code would be something like this:
   ```rust
   let table = TableCreation::builder()
       .name(table_name)
       .schema(table_schema)
       .format_version(FormatVersion::V3) // <-- this call here is ignored
       .build();
   ```
   
   # Context
   
   In the Rest `catalog.rs` here the tables `.properties` are passed, and `format_version` is never used: https://github.com/apache/iceberg-rust/blob/main/crates/catalog/rest/src/catalog.rs#L794-L815 
   
   A fix could be to either add it to the properties here, or add it to the properties upstream.
   
   # Workaround
   
   Set the table properties to use V3 by hand like so:
   ```rust
   let table = TableCreation::builder()
       .name(table_name)
       .schema(table_schema)
       .partition_spec(table_partition_spec)
       .properties(HashMap::from([
           // Format v3 is needed as we are using nanosecond times.
           (
               TableProperties::PROPERTY_FORMAT_VERSION.to_string(),
               "3".to_string(),
           ),
       ]))
       .build();
   ```
   
   
   ### To Reproduce
   
   _No response_
   
   ### Expected behavior
   
    * The format version set by `format_version` should be used, when it is set on it's own.
   
   I'm not 100% sure what should happen when setting the version by both `format_version` and `properties`, or if we even care about having precise behaviour there.
   
   
   ### Willingness to contribute
   
   I can contribute a fix for this bug independently


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

Re: [I] `TableCreation::format_version` is ignored when creating a table using the Iceberg REST catalog interface [iceberg-rust]

["TwinklerG (via GitHub)" <gi...@apache.org>]

TwinklerG commented on issue #2923:
URL: https://github.com/apache/iceberg-rust/issues/2923#issuecomment-5163292937

   However, there is a problem that if the user defines `format-version` both in `TableCreation::properties` and `TableCreation::format_version`, which one comes first.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org