You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2025/11/03 14:03:21 UTC
[PR] Update docs for zero copy conversion of ScalarBuffer [arrow-rs]
alamb opened a new pull request, #8772:
URL: https://github.com/apache/arrow-rs/pull/8772
# Which issue does this PR close?
- Similar to https://github.com/apache/arrow-rs/pull/8771
# Rationale for this change
As people write more arrow kernels, let's make it easier to understand how to go back/forth to Vec without copying
# What changes are included in this PR?
Add some additional information and examples about converting ScalarBuffer back/forth to Vec
# Are these changes tested?
Yes by CI
# Are there any user-facing changes?
Docs only. No functional change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update docs for zero copy conversion of ScalarBuffer [arrow-rs]
Posted by "kylebarron (via GitHub)" <gi...@apache.org>.
kylebarron commented on code in PR #8772:
URL: https://github.com/apache/arrow-rs/pull/8772#discussion_r2487385418
##########
arrow-buffer/src/buffer/scalar.rs:
##########
@@ -29,17 +29,35 @@ use std::ops::Deref;
/// with the following differences:
///
/// - slicing and cloning is O(1).
-/// - it supports external allocated memory
+/// - support for external allocated memory (e.g. via FFI).
///
+/// See [`Buffer`] for more low-level memory management details.
+///
+/// # Example: Convert to/from Vec (without copies)
/// ```
/// # use arrow_buffer::ScalarBuffer;
/// // Zero-copy conversion from Vec
/// let buffer = ScalarBuffer::from(vec![1, 2, 3]);
/// assert_eq!(&buffer, &[1, 2, 3]);
+/// // convert the buffer back to Vec without copy assuming:
+/// // 1. the inner buffer is not sliced
+/// // 2. the inner buffer uses standard allocation
Review Comment:
I think @mbrobbel 's comment applies here too https://github.com/apache/arrow-rs/pull/8771#discussion_r2486648522
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update docs for zero copy conversion of ScalarBuffer [arrow-rs]
Posted by "wjones127 (via GitHub)" <gi...@apache.org>.
wjones127 commented on code in PR #8772:
URL: https://github.com/apache/arrow-rs/pull/8772#discussion_r2487243809
##########
arrow-buffer/src/buffer/scalar.rs:
##########
@@ -29,17 +29,35 @@ use std::ops::Deref;
/// with the following differences:
///
/// - slicing and cloning is O(1).
-/// - it supports external allocated memory
+/// - support for external allocated memory (e.g. via FFI).
///
+/// See [`Buffer`] for more low-level memory management details.
+///
+/// # Example: Convert to/from Vec (without copies)
/// ```
/// # use arrow_buffer::ScalarBuffer;
/// // Zero-copy conversion from Vec
/// let buffer = ScalarBuffer::from(vec![1, 2, 3]);
/// assert_eq!(&buffer, &[1, 2, 3]);
+/// // convert the buffer back to Vec without copy assuming:
+/// // 1. the inner buffer is not sliced
+/// // 2. the inner buffer uses standard allocation
Review Comment:
Is there also a requirement for the reference count of the buffer to be 1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update docs for zero copy conversion of ScalarBuffer [arrow-rs]
Posted by "mbrobbel (via GitHub)" <gi...@apache.org>.
mbrobbel commented on PR #8772:
URL: https://github.com/apache/arrow-rs/pull/8772#issuecomment-3484464686
Thanks @alamb, @wjones127 and @kylebarron
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update docs for zero copy conversion of ScalarBuffer [arrow-rs]
Posted by "mbrobbel (via GitHub)" <gi...@apache.org>.
mbrobbel merged PR #8772:
URL: https://github.com/apache/arrow-rs/pull/8772
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update docs for zero copy conversion of ScalarBuffer [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #8772:
URL: https://github.com/apache/arrow-rs/pull/8772#discussion_r2487713032
##########
arrow-buffer/src/buffer/scalar.rs:
##########
@@ -29,17 +29,35 @@ use std::ops::Deref;
/// with the following differences:
///
/// - slicing and cloning is O(1).
-/// - it supports external allocated memory
+/// - support for external allocated memory (e.g. via FFI).
///
+/// See [`Buffer`] for more low-level memory management details.
+///
+/// # Example: Convert to/from Vec (without copies)
/// ```
/// # use arrow_buffer::ScalarBuffer;
/// // Zero-copy conversion from Vec
/// let buffer = ScalarBuffer::from(vec![1, 2, 3]);
/// assert_eq!(&buffer, &[1, 2, 3]);
+/// // convert the buffer back to Vec without copy assuming:
+/// // 1. the inner buffer is not sliced
+/// // 2. the inner buffer uses standard allocation
Review Comment:
Excellent call -- added in dac99aaac3. Thank you 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org