[GitHub] [arrow-rs] jorgecarleitao commented on issue #597: Can not create a Timestamp array that has a specified timezone

[GitBox <gi...@apache.org>]

jorgecarleitao commented on issue #597:
URL: https://github.com/apache/arrow-rs/issues/597#issuecomment-886022683


   I do not see what is the concern with `String`. It represent exactly what the spec expects; an owned utf8.
   
   The root cause here seems to be the trait of the generic of the `PrimitiveArray`, that expects a type with a compiled-defined data type, which only works for small, finite-sized datatypes. Even the decimal array could have been represented as a `PrimitiveArray`.
   
   Arrow2 solves this by decoupling the generic on the PrimitiveArray from the (logical) `DataType`, exactly to natively support timestamps with timezones.
   
   I would recommend bridging this gap by refactoring the code to not have the `DATA_TYPE` as the constant of the trait.
   
   AFAI understand `&'static` forbids people from declaring an offset based on some transformation. E.g. a field composed by something like `"some_garbage_{offset}"` can no longer be parsed into an offset without matching for all cases, since the "_offset" will not be `'static`.
   
   Having a generic for every possible timezone significantly bloats the binary, as it requires all generics to be re-compiled for every declared timezone variation, and there are a lot of them. The typical problem will be in `match`ing the `DataType` and having to have one branch per Timezone, which requires a significant number of cases.
   
   Generics are used to support different physical in-memory representations of a struct, not to declare semantics. Semantics based on generics is a recipe for large binaries and/or a large number of matches in downcasting trait objects.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org