You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "aykut-bozkurt (via GitHub)" <gi...@apache.org> on 2024/08/24 01:33:25 UTC

[I] Safely Write IntervalMonthDayNanoArray to parquet or Throw error [arrow-rs]

aykut-bozkurt opened a new issue, #6298:
URL: https://github.com/apache/arrow-rs/issues/6298

   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   When the `nanoseconds` part of `IntervalMonthDayNano` value does not exceed `i32::MAX`, then it is safe to write it to parquet after truncating it to 4 bytes. But currently I always get an unimplemented error when I try to write the array to parquet.
   
   **Describe the solution you'd like**
   When the `nanoseconds` part does not exceed `i32::MAX`, then it is safe to write it to parquet after truncating it to 4 bytes. If it exceeds `i32::MAX`, then we can throw error.
   
   This unblocks the ones who needs to write arrow `IntervalMonthDayNano` with milliseconds precision to parquet. It currently always throws error even if it is safe to write the value to parquet.
   
   **Describe alternatives you've considered**
   There seems to be not many alternatives, other than adding new interval types to the spec. But this seems a huge change.
   
   **Additional context**
   related discussion #5849
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Safely Write IntervalMonthDayNanoArray to parquet or Throw error [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey closed issue #6298: Safely Write IntervalMonthDayNanoArray to parquet or Throw error
URL: https://github.com/apache/arrow-rs/issues/6298


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Safely Write IntervalMonthDayNanoArray to parquet or Throw error [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on issue #6298:
URL: https://github.com/apache/arrow-rs/issues/6298#issuecomment-4862725227

   per discussion in #5849 we probably wont proceed with this unless https://github.com/apache/parquet-format/issues/313 lands


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org