You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by "Wes McKinney (JIRA)" <ji...@apache.org> on 2016/03/25 03:22:25 UTC
[jira] [Resolved] (ARROW-76) Revise format document to include null
count, defer non-nullable arrays to the domain of metadata
[ https://issues.apache.org/jira/browse/ARROW-76?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wes McKinney resolved ARROW-76.
-------------------------------
Resolution: Fixed
Assignee: Wes McKinney
Resolved in https://github.com/apache/arrow/commit/c06b7654bccfe8c461869a6e5922668896c27c45
> Revise format document to include null count, defer non-nullable arrays to the domain of metadata
> -------------------------------------------------------------------------------------------------
>
> Key: ARROW-76
> URL: https://issues.apache.org/jira/browse/ARROW-76
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Format
> Reporter: Wes McKinney
> Assignee: Wes McKinney
>
> After many mailing list discussions and a first draft of metadata serialization (https://github.com/apache/arrow/blob/master/format/Message.fbs), I propose we strike the notion of required fields from the physical memory layout and instead make this a property of the metadata (see the {{nullable}} member of {{Field}} in the serialized schema metadata).
> Note that Arrow implementations will be free to create specialized containers for data having {{nullable = false}} in their metadata. It will be the implementations' responsibility to validate that {{null_count == 0}} at IPC boundaries when interacting with "untrusted" data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)