You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/10/16 02:07:42 UTC
[GitHub] [arrow-rs] viirya opened a new issue, #2882: Memory alignment error in `RawPtrBox::new`
viirya opened a new issue, #2882:
URL: https://github.com/apache/arrow-rs/issues/2882
**Describe the bug**
<!--
A clear and concise description of what the bug is.
-->
When we read array data from IPC, IPC reader basically shares the same memory allocation between Buffers across arrays. These Buffers are just separate slices of an original Buffer.
But this possibly causes memory alignment check error in `RawPtrBox::new` where an assert verifies if the given pointer is aligned with type `T`. The alignment offset is computed based on original memory allocation, not these slices.
For example, there are three slices.
```
let x = [5u8, 6u8, 7u8, 8u8, 9u8];
let y = &x[0..x.len() -1];
let z = [5u8, 6u8, 7u8, 8u8];
```
Then we compute their alignment offset by:
```
let ptr = x.as_ptr() as *const u8;
let offset = ptr.align_offset(align_of::<u16>());
```
The offset of `x` and `y` are both 1, and the offset of `z` is 0.
**To Reproduce**
<!--
Steps to reproduce the behavior:
-->
Just run `cargo test read_generated_files_014` on M1 Macbook. Got:
```
running 1 test
thread 'read_generated_files_014' panicked at 'assertion failed: `(left == right)`
left: `8`,
right: `0`: memory is not aligned', arrow-array/src/raw_pointer.rs:40:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test read_generated_files_014 ... FAILED
failures:
failures:
read_generated_files_014
```
**Expected behavior**
<!--
A clear and concise description of what you expected to happen.
-->
**Additional context**
<!--
Add any other context about the problem here.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on issue #2882: Memory alignment error in `RawPtrBox::new`
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #2882:
URL: https://github.com/apache/arrow-rs/issues/2882#issuecomment-1294979272
`label_issue.py` automatically added labels {'arrow'} from #2883
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on issue #2882: Memory alignment error in `RawPtrBox::new`
Posted by GitBox <gi...@apache.org>.
tustvold commented on issue #2882:
URL: https://github.com/apache/arrow-rs/issues/2882#issuecomment-1279885278
What is the alignment requirement for i128 on an M1 mac, on x86 it is 8 bytes, which is what is guaranteed by the IPC format?
The error message would suggest the buffer is 8 byte aligned, but that this is insufficient for whatever type is being handled.
My hunch is arm requires 16-byte alignment for i128, and following #2857 we are only just now seeing this. I suspect for this case we will need to copy the buffer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] viirya closed issue #2882: Memory alignment error in `RawPtrBox::new`
Posted by GitBox <gi...@apache.org>.
viirya closed issue #2882: Memory alignment error in `RawPtrBox::new`
URL: https://github.com/apache/arrow-rs/issues/2882
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] viirya commented on issue #2882: Memory alignment error in `RawPtrBox::new`
Posted by GitBox <gi...@apache.org>.
viirya commented on issue #2882:
URL: https://github.com/apache/arrow-rs/issues/2882#issuecomment-1279886865
Yes, on M1 Macbook, the alignment for i128 is 16 bytes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org