[I] [Format] Specify FIXED_SIZE_LIST Logical type [parquet-format]

["asfimport (via GitHub)" <gi...@apache.org>]

asfimport opened a new issue, #430:
URL: https://github.com/apache/parquet-format/issues/430

   [_Replicated from mailing list_](https://lists.apache.org/thread/xot5f3ghhtc82n1bf0wdl9zqwlrzqks3)
   
   Arrow recently introduced [FixedShapeTensor](https://arrow.apache.org/docs/format/CanonicalExtensions.html#fixed-shape-tensor) and [VariableShapeTensor](https://arrow.apache.org/docs/format/CanonicalExtensions.html#variable-shape-tensor) canonical extension types that use FixedSizeList and StructArray(List, FixedSizeList) as storage respectfully. These are targeted at machine learning and scientific applications that deal with large datasets and would benefit from using Parquet as on disk storage.
   
   However currently FixedSizeList is stored as List in Parquet which adds significant conversion overhead when reading and writing as [discussed here](https://github.com/apache/arrow/issues/34510). It would therefore be beneficial to introduce a FIXED_SIZE_LIST logical type to Parquet.
   
   **Reporter**: [Rok Mihevc](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=rok)
   #### PRs and other links:
   - [GitHub Pull Request #241](https://github.com/apache/parquet-format/pull/241)
   
   <sub>**Note**: *This issue was originally created as [PARQUET-2474](https://issues.apache.org/jira/browse/PARQUET-2474). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*</sub>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org