[GitHub] [incubator-iceberg] prodeezy opened a new issue #505: Timestamp based day transformed partitions fails on read

[GitBox <gi...@apache.org>]

prodeezy opened a new issue #505: Timestamp based day transformed partitions fails on read
URL: https://github.com/apache/incubator-iceberg/issues/505
 
 
   Changes made in https://github.com/apache/incubator-iceberg/pull/424 is causing partition interpretation to fail during append/read. Here's an example ..
   
   ```
   
   scala> import org.apache.iceberg.types.Types.NestedField._
   import org.apache.iceberg.types.Types.NestedField._
   
   scala> import org.apache.iceberg.types._
   import org.apache.iceberg.types._
   
   scala> import org.apache.iceberg._
   import org.apache.iceberg._
   
   
   scala> val timestampSchema = new Schema(optional(1, "timestamp", Types.TimestampType.withoutZone()))
   timestampSchema: org.apache.iceberg.Schema =
   table {
     1: timestamp: optional timestamp
   }
   
   scala> val partitionByDate = PartitionSpec.builderFor(timestampSchema).day("timestamp", "date").build
   partitionByDate: org.apache.iceberg.PartitionSpec =
   [
     date: day(1)
   ]
   
   scala> DataFiles.builder(partitionByDate).withPath("/path/to/data-1.parquet").withPartitionPath("date=2019-09-01").build()
   java.lang.UnsupportedOperationException: Unsupported type for fromPartitionString: date
     at org.apache.iceberg.types.Conversions.fromPartitionString(Conversions.java:70)
     at org.apache.iceberg.DataFiles.fillFromPath(DataFiles.java:83)
     at org.apache.iceberg.DataFiles.access$200(DataFiles.java:36)
     at org.apache.iceberg.DataFiles$Builder.withPartitionPath(DataFiles.java:292)
     ... 55 elided
   
   ```
   
   With this change, Timestamp `day()` transform returns  Date type instead of Integer ... and `Conversions` class doesn't support `Date` type. The impact this has is when one tries to append files to a partition like that, fails with this error. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org