[GitHub] [arrow-rs] alamb commented on a change in pull request #628: Add a note about arrow crate security / safety

[GitBox <gi...@apache.org>]

alamb commented on a change in pull request #628:
URL: https://github.com/apache/arrow-rs/pull/628#discussion_r678552891



##########
File path: arrow/README.md
##########
@@ -35,6 +39,26 @@ The arrow crate provides the following optional features:
   implementations of some [compute](https://github.com/apache/arrow/tree/master/rust/arrow/src/compute)
   kernels using explicit SIMD processor intrinsics.
 
+## Safety
+
+TLDR: You should avoid using the `alloc` and `buffer` and `bitmap` modules if at all possible. These modules contain `unsafe` code and are easy to misuse.
+
+As with all open source code, you should carefully evaluate the suitability of `arrow` for your project, taking into consideration your needs and risk tolerance prior to use.
+
+_Background_: There are various parts of the `arrow` crate which use `unsafe` and `transmute` code internally. We are actively working as a community to minimize undefined behavior and remove `unsafe` usage to align more with Rust's core principles of safety (e.g. the arrow2 project).
+
+As `arrow` exists today, it is fairly easy to misuse the APIs, leading to undefined behavior, and it is especially easy to misuse code in modules named above. For an example, as described in [the arrow2 crate](https://github.com/jorgecarleitao/arrow2#why), the following code compiles, does not panic, but results in undefined behavior:
+
+```rust
+let buffer = Buffer::from_slic_ref(&[0i32, 2i32])
+let data = ArrayData::new(DataType::Int64, 10, 0, None, 0, vec![buffer], vec![]);
+let array = Float64Array::from(Arc::new(data));
+
+println!("{:?}", array.value(1));
+```
+
+NOTE: We plan to deprecate and make these modules private as part of a follow on release, as part of our journey of redesigning this crate.

Review comment:
       I filed https://github.com/apache/arrow-rs/issues/629 as a proposal to mark these modules private. Feedback more than welcome

##########
File path: arrow/README.md
##########
@@ -35,6 +39,26 @@ The arrow crate provides the following optional features:
   implementations of some [compute](https://github.com/apache/arrow/tree/master/rust/arrow/src/compute)
   kernels using explicit SIMD processor intrinsics.
 
+## Safety
+
+TLDR: You should avoid using the `alloc` and `buffer` and `bitmap` modules if at all possible. These modules contain `unsafe` code and are easy to misuse.

Review comment:
       @jorgecarleitao @jhorstmann @nevi-me @houqp @ritchie46 @andygrove 
   
   Is this a fair assessment, in your opinion, about the risk of using the `arrow` crate?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org