You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/06/25 22:37:00 UTC
[jira] [Updated] (ARROW-13191) [Go] Support external schema in ipc
readers
[ https://issues.apache.org/jira/browse/ARROW-13191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated ARROW-13191:
-----------------------------------
Labels: pull-request-available (was: )
> [Go] Support external schema in ipc readers
> -------------------------------------------
>
> Key: ARROW-13191
> URL: https://issues.apache.org/jira/browse/ARROW-13191
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Go
> Reporter: Seth Hollyman
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> (Apologies if I'm imprecise here, I'm still coming up to speed on the arrow details.)
>
> The IPC message format describes how data and metadata messages are encapsulated, but it is not a requirement that each message must include the schema.
>
> In Go, github.com/apache/arrow/go/arrow/ipc contains NewReader() for setting up reading of IPC messages, and accepts the option WithSchema to pass the schema into said reader. However, the implementation merely uses that information to compare that the schema it reads from the IPC stream matches the passed in reader. This request is to allow WithSchema to behave as expected, and use the option-provided Schema for performing reads.
>
> The one gotcha here appears to be the dictionary type map, which is currently retained independently of the schema but is part of the internal readSchema() setup. Completeness may warrant another option for communicating those externally as well? Or perhaps option-passed Schema should be documented to not support dictionary types?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)