You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/05/15 17:42:07 UTC
[GitHub] [arrow-rs] roee88 commented on pull request #287: Fix FFI and add support for Struct type
roee88 commented on pull request #287:
URL: https://github.com/apache/arrow-rs/pull/287#issuecomment-841699212
> I also wondered if this would fix #227 (aka miri reported issues):
>
> ```
> RUST_BACKTRACE=1 MIRIFLAGS="-Zmiri-disable-isolation" cargo +nightly miri test -p arrow -- ffi
> ```
>
> Sadly, it still fails on both this branch and master. On master it fails with:
>
> ```
> running 12 tests
> test array::ffi::tests::test_i64 ... error: Undefined Behavior: trying to reborrow for SharedReadOnly at alloc875771, but parent tag <2225913> does not have an appropriate item in the borrow stack
> --> /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/unique.rs:118:18
> |
> 118 | unsafe { &*self.as_ptr() }
> | ^^^^^^^^^^^^^^^ trying to reborrow for SharedReadOnly at alloc875771, but parent tag <2225913> does not have an appropriate item in the borrow stack
> |
> = help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
> = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
>
> = note: inside `std::ptr::Unique::<[*const std::ffi::c_void]>::as_ref` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/unique.rs:118:18
> = note: inside `alloc::alloc::box_free::<[*const std::ffi::c_void], std::alloc::Global>` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/alloc.rs:331:32
> = note: inside `std::ptr::drop_in_place::<std::boxed::Box<[*const std::ffi::c_void]>> - shim(Some(std::boxed::Box<[*const std::ffi::c_void]>))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> = note: inside `std::ptr::drop_in_place::<ffi::PrivateData> - shim(Some(ffi::PrivateData))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> = note: inside `std::ptr::drop_in_place::<std::boxed::Box<ffi::PrivateData>> - shim(Some(std::boxed::Box<ffi::PrivateData>))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> note: inside `ffi::release_array` at arrow/src/ffi.rs:396:58
> --> arrow/src/ffi.rs:396:58
> |
> 396 | Box::from_raw(array.private_data as *mut PrivateData);
> | ^
> note: inside `<ffi::FFI_ArrowArray as std::ops::Drop>::drop` at arrow/src/ffi.rs:517:39
> --> arrow/src/ffi.rs:517:39
> |
> 517 | Some(release) => unsafe { release(self) },
> | ^^^^^^^^^^^^^
> = note: inside `std::ptr::drop_in_place::<ffi::FFI_ArrowArray> - shim(Some(ffi::FFI_ArrowArray))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> = note: inside `std::sync::Arc::<ffi::FFI_ArrowArray>::drop_slow` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/sync.rs:1039:18
> = note: inside `<std::sync::Arc<ffi::FFI_ArrowArray> as std::ops::Drop>::drop` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/sync.rs:1571:13
> = note: inside `std::ptr::drop_in_place::<std::sync::Arc<ffi::FFI_ArrowArray>> - shim(Some(std::sync::Arc<ffi::FFI_ArrowArray>))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> = note: inside `std::ptr::drop_in_place::<bytes::Deallocation> - shim(Some(bytes::Deallocation))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> = note: inside `std::ptr::drop_in_place::<bytes::Bytes> - shim(Some(bytes::Bytes))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> = note: inside `std::sync::Arc::<bytes::Bytes>::drop_slow` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/sync.rs:1039:18
> = note: inside `<std::sync::Arc<bytes::Bytes> as std::ops::Drop>::drop` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/sync.rs:1571:13
> = note: inside `std::ptr::drop_in_place::<std::sync::Arc<bytes::Bytes>> - shim(Some(std::sync::Arc<bytes::Bytes>))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> = note: inside `std::ptr::drop_in_place::<buffer::immutable::Buffer> - shim(Some(buffer::immutable::Buffer))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> = note: inside `std::ptr::drop_in_place::<bitmap::Bitmap> - shim(Some(bitmap::Bitmap))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> = note: inside `std::ptr::drop_in_place::<std::option::Option<bitmap::Bitmap>> - shim(Some(std::option::Option<bitmap::Bitmap>))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> = note: inside `std::ptr::drop_in_place::<array::data::ArrayData> - shim(Some(array::data::ArrayData))` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:192:1
> note: inside `array::ffi::tests::test_round_trip` at arrow/src/array/ffi.rs:146:5
> --> arrow/src/array/ffi.rs:146:5
> |
> 146 | }
> | ^
> note: inside `array::ffi::tests::test_i64` at arrow/src/array/ffi.rs:166:9
> --> arrow/src/array/ffi.rs:166:9
> |
> 166 | test_round_trip(data)
> | ^^^^^^^^^^^^^^^^^^^^^
> note: inside closure at arrow/src/array/ffi.rs:163:5
> --> arrow/src/array/ffi.rs:163:5
> |
> 163 | / fn test_i64() -> Result<()> {
> 164 | | let array = Int64Array::from(vec![Some(2), None, Some(1), None]);
> 165 | | let data = array.data();
> 166 | | test_round_trip(data)
> 167 | | }
> | |_____^
> = note: inside `<[closure@arrow/src/array/ffi.rs:163:5: 167:6] as std::ops::FnOnce<()>>::call_once - shim` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
> = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
> = note: inside `test::__rust_begin_short_backtrace::<fn()>` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:577:5
> = note: inside closure at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:568:30
> = note: inside `<[closure@test::run_test::{closure#2}] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
> = note: inside `<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send> as std::ops::FnOnce<()>>::call_once` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1546:9
> = note: inside `<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>> as std::ops::FnOnce<()>>::call_once` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panic.rs:344:9
> = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:379:40
> = note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>>` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:343:19
> = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panic.rs:431:14
> = note: inside `test::run_test_in_process` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:600:18
> = note: inside closure at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:492:39
> = note: inside `test::run_test::run_test_inner` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:530:13
> = note: inside `test::run_test` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:564:28
> = note: inside `test::run_tests::<[closure@test::run_tests_console::{closure#2}]>` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:305:17
> = note: inside `test::run_tests_console` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/console.rs:289:5
> = note: inside `test::test_main` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:122:15
> = note: inside `test::test_main_static` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/test/src/lib.rs:141:5
> = note: inside `main`
> = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
> = note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:125:18
> = note: inside closure at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/rt.rs:49:18
> = note: inside `std::ops::function::impls::<impl std::ops::FnOnce<()> for &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>::call_once` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:259:13
> = note: inside `std::panicking::r#try::do_call::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:379:40
> = note: inside `std::panicking::r#try::<i32, &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:343:19
> = note: inside `std::panic::catch_unwind::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panic.rs:431:14
> = note: inside `std::rt::lang_start_internal` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/rt.rs:34:21
> = note: inside `std::rt::lang_start::<()>` at /Users/alamb/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/rt.rs:48:5
> = note: this error originates in an attribute macro (in Nightly builds, run with -Z macro-backtrace for more info)
>
> error: aborting due to previous error
>
> error: test failed, to rerun pass '-p arrow --lib'
> ```
I have written on slack about this. I spent several hours on this but couldn't understand why dereferencing the raw pointer (to get the inner raw pointer) pops the borrow. I did test with miri but had to disable stacked borrows as suggested by @jorgecarleitao.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org