[GitHub] [arrow] nevi-me commented on pull request #8971: ARROW-10987: [Rust] Interpret BinaryArray as JSON

[GitBox <gi...@apache.org>]

nevi-me commented on pull request #8971:
URL: https://github.com/apache/arrow/pull/8971#issuecomment-749333871


   Hi @kflansburg @jorgecarleitao , I offered the suggestion, so I should have been more explicit on how we could do this in a spec-compliant fashion. I apologise for that.
   
   The spec doesn't have a `JSONArray`, but it has an `ExtensionArray`, which would be one of the logical types that Arrow supports, but with extra metadata that defines that it's a specific type. JSON, UUID are good examples, where the datatype could be StringArray and FixedSizeBinaryArray respectively.
   
   In order to take advantage of the JSON conversion, we could:
   
   - Implement https://issues.apache.org/jira/browse/ARROW-10258, which is the PR for ExtensionArray (I still intend on getitng to this by this release, but help is welcome).
   - Add a function that converts a `StringArray` to `ExtensionArray` with the metadata indicating that it's a JSON array.
   
   We'd have to look at the other language implementations, and also plan out the API so it can cover its intended use-cases adequately.
   
   I personally see the need to be able to convert a `StringArray` that has JSON data, into a `StructArray`. One concrete example is that Postgres and other databases have a `JSON` type, which is externally represented as a `string with valid JSON`.
   Given that adding this functionality wouldn't introduce any dependencies, I think it can/should live in the `arrow` crate.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org