[GitHub] [arrow-rs] alamb commented on issue #1071: Offset Validation Nulls

[GitBox <gi...@apache.org>]

alamb commented on issue #1071:
URL: https://github.com/apache/arrow-rs/issues/1071#issuecomment-998314855


   I am not 100% sure if supporting non monotonic offsets is something we want to fix to be honest.  My rationale is if the offsets are monotonic, all arrays with offsets can calculate length as `offsets[i] - offsets[i-1]` which has better failure properties than if we ended up doing stuff like `0 - 32` or something because the offset had been left unset
   
   Thus
   
   Also for fun, I checked @jorgecarleitao 's implementation in arrow2 which also ensures that offsets are always monotonic. 
   https://github.com/jorgecarleitao/arrow2/blob/37a9c758826a92d98dc91e992b2a49ce9724095d/src/array/specification.rs#L102-L119
   
   I am pretty sure the C++ implementation also ensures the offsets are monotonic without first checking it is null: https://github.com/apache/arrow/blob/master/cpp/src/arrow/array/validate.cc#L568-L592


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org