[jira] [Commented] (ARROW-1224) [Format] Clarify language around buffer padding and alignment in IPC

["SIDDHARTH TEOTIA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16110055#comment-16110055 ] 

SIDDHARTH TEOTIA commented on ARROW-1224:
-----------------------------------------

[~wesmckinn], [~jnadeau], [~sphillips]

I understand that 64-byte alignment recommendation comes from the fact that largest (AFAIK) SIMD register available is Intel AVX-512. So we can load the column values from the 64-byte buffer into the 512-bit wide SIMD register and run operations in parallel -- higher degree of data-level parallelism

However, it looks like the minimum/strict alignment requirement should be 16-byte boundary as far as SIMD processing is concerned since that allows us to use 128-bit wide SIMD registers added as part of Intel SSE instruction set.

Am I missing something?

Thanks,
Sidd

> [Format] Clarify language around buffer padding and alignment in IPC
> --------------------------------------------------------------------
>
>                 Key: ARROW-1224
>                 URL: https://issues.apache.org/jira/browse/ARROW-1224
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Format
>            Reporter: Wes McKinney
>            Assignee: SIDDHARTH TEOTIA
>             Fix For: 0.6.0
>
>
> We recommend 64-byte padding in memory allocations, but in practice some systems may emit buffers that are only 8-byte aligned. We should take a look through the format documents and perhaps indicate that 64-byte padding is the _preferred_ padding but that 8-byte padding is the _required_ padding. 
> In particular, it should be clear that each buffer in an IPC payload must start on an aligned 8-byte offset, but it does not necessarily need to be 64-byte aligned. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)