[GitHub] [incubator-iceberg] rdblue commented on pull request #983: Convert date and timestamp values in generics

[GitBox <gi...@apache.org>]

rdblue commented on pull request #983:
URL: https://github.com/apache/incubator-iceberg/pull/983#issuecomment-632331000


   @chenjunjiedada, I think what you have here works, but right now we have two sets of accessors (Spark and internal) and we would need more for other data models.
   
   I think what we need to do instead is to separate the concerns. Accessors should understand structure and rows should return the correct value types. That way we can eventually move to use the same accessors and we will just need rows that correctly translate for StructLike.
   
   How about using a wrapper class like this one when passing records? Would that work?
   
   ```java
   class InternalRecordWrapper implements StructLike {
     private final Function<Object, Object>[] transforms;
     private StructLike wrapped = null;
   
     @SuppressWarnings("unchecked")
     InternalRecordWrapper(Types.StructType struct) {
       this.transforms = struct.fields().stream()
           .map(field -> converter(field.type()))
           .toArray(length -> (Function<Object, Object>[]) Array.newInstance(Function.class, length));
     }
   
     private static Function<Object, Object> converter(Type type) {
       switch (type.typeId()) {
         case DATE:
           return date -> DateTimeUtil.daysFromDate((LocalDate) date);
         case TIME:
           return time -> DateTimeUtil.microsFromTime((LocalTime) time);
         case TIMESTAMP:
           if (((Types.TimestampType) type).shouldAdjustToUTC()) {
             return timestamp -> DateTimeUtil.microsFromTimestamptz((OffsetDateTime) timestamp);
           } else {
             return timestamp -> DateTimeUtil.microsFromTimestamp((LocalDateTime) timestamp);
           }
         case FIXED:
           return bytes -> ByteBuffer.wrap((byte[]) bytes);
         case STRUCT:
           InternalRecordWrapper wrapper = new InternalRecordWrapper(type.asStructType());
           return struct -> wrapper.wrap((StructLike) struct);
         default:
       }
       return null;
     }
   
     public InternalRecordWrapper wrap(StructLike record) {
       this.wrapped = record;
       return this;
     }
   
     @Override
     public int size() {
       return wrapped.size();
     }
   
     @Override
     public <T> T get(int pos, Class<T> javaClass) {
       if (transforms[pos] != null) {
         return javaClass.cast(transforms[pos].apply(wrapped.get(pos, Object.class)));
       }
       return wrapped.get(pos, javaClass);
     }
   
     @Override
     public <T> void set(int pos, T value) {
       throw new UnsupportedOperationException("Cannot update InternalRecordWrapper");
     }
   }
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org