You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2026/01/15 13:12:11 UTC
[PR] Avoid a clone when creating `PrimitiveArray` from ArrayData [arrow-rs]
alamb opened a new pull request, #9190:
URL: https://github.com/apache/arrow-rs/pull/9190
# Which issue does this PR close?
- Part of https://github.com/apache/arrow-rs/issues/9061
- broken out of https://github.com/apache/arrow-rs/pull/9058
# Rationale for this change
Let's make arrow-rs the fastest we can and the fewer allocations the better
# What changes are included in this PR?
Apply pattern from https://github.com/apache/arrow-rs/pull/9114
# Are these changes tested?
Existing tests
# Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid a clone when creating `PrimitiveArray` from ArrayData [arrow-rs]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan merged PR #9190:
URL: https://github.com/apache/arrow-rs/pull/9190
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid a clone when creating `PrimitiveArray` from ArrayData [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #9190:
URL: https://github.com/apache/arrow-rs/pull/9190#discussion_r2694390283
##########
arrow-array/src/array/primitive_array.rs:
##########
@@ -1597,18 +1601,21 @@ impl<T: ArrowTimestampType> PrimitiveArray<T> {
/// Constructs a `PrimitiveArray` from an array data reference.
impl<T: ArrowPrimitiveType> From<ArrayData> for PrimitiveArray<T> {
fn from(data: ArrayData) -> Self {
- Self::assert_compatible(data.data_type());
+ let (data_type, len, nulls, offset, mut buffers, _child_data) = data.into_parts();
+
+ Self::assert_compatible(&data_type);
assert_eq!(
- data.buffers().len(),
+ buffers.len(),
1,
"PrimitiveArray data should contain a single buffer only (values buffer)"
);
+ let buffer = buffers.pop().expect("checked above");
- let values = ScalarBuffer::new(data.buffers()[0].clone(), data.offset(), data.len());
+ let values = ScalarBuffer::new(buffer, offset, len);
Self {
- data_type: data.data_type().clone(),
+ data_type,
values,
- nulls: data.nulls().cloned(),
Review Comment:
This saves a clone /call to DataType::drop as well as a Clone of `Buffer` (which is fairly inexpensive).
I don't think it will make any different in performance, but the code is cleaner and now consistent with other array types so I think this change is worth making
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid a clone when creating `PrimitiveArray` from ArrayData [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #9190:
URL: https://github.com/apache/arrow-rs/pull/9190#discussion_r2694390283
##########
arrow-array/src/array/primitive_array.rs:
##########
@@ -1597,18 +1601,21 @@ impl<T: ArrowTimestampType> PrimitiveArray<T> {
/// Constructs a `PrimitiveArray` from an array data reference.
impl<T: ArrowPrimitiveType> From<ArrayData> for PrimitiveArray<T> {
fn from(data: ArrayData) -> Self {
- Self::assert_compatible(data.data_type());
+ let (data_type, len, nulls, offset, mut buffers, _child_data) = data.into_parts();
+
+ Self::assert_compatible(&data_type);
assert_eq!(
- data.buffers().len(),
+ buffers.len(),
1,
"PrimitiveArray data should contain a single buffer only (values buffer)"
);
+ let buffer = buffers.pop().expect("checked above");
- let values = ScalarBuffer::new(data.buffers()[0].clone(), data.offset(), data.len());
+ let values = ScalarBuffer::new(buffer, offset, len);
Self {
- data_type: data.data_type().clone(),
+ data_type,
values,
- nulls: data.nulls().cloned(),
Review Comment:
The code looks easier and it saves a clone /call to DataType::drop as well as a Clone of `Buffer` (which is fairly inexpensive).
I don't think it will make any different in performance, but the code is cleaner and now consistent with other array types
##########
arrow-array/src/array/primitive_array.rs:
##########
@@ -1597,18 +1601,21 @@ impl<T: ArrowTimestampType> PrimitiveArray<T> {
/// Constructs a `PrimitiveArray` from an array data reference.
impl<T: ArrowPrimitiveType> From<ArrayData> for PrimitiveArray<T> {
fn from(data: ArrayData) -> Self {
- Self::assert_compatible(data.data_type());
+ let (data_type, len, nulls, offset, mut buffers, _child_data) = data.into_parts();
+
+ Self::assert_compatible(&data_type);
assert_eq!(
- data.buffers().len(),
+ buffers.len(),
1,
"PrimitiveArray data should contain a single buffer only (values buffer)"
);
+ let buffer = buffers.pop().expect("checked above");
- let values = ScalarBuffer::new(data.buffers()[0].clone(), data.offset(), data.len());
+ let values = ScalarBuffer::new(buffer, offset, len);
Self {
- data_type: data.data_type().clone(),
+ data_type,
values,
- nulls: data.nulls().cloned(),
Review Comment:
This saves a clone /call to DataType::drop as well as a Clone of `Buffer` (which is fairly inexpensive).
I don't think it will make any different in performance, but the code is cleaner and now consistent with other array types
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Avoid a clone when creating `PrimitiveArray` from ArrayData [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #9190:
URL: https://github.com/apache/arrow-rs/pull/9190#discussion_r2694390283
##########
arrow-array/src/array/primitive_array.rs:
##########
@@ -1597,18 +1601,21 @@ impl<T: ArrowTimestampType> PrimitiveArray<T> {
/// Constructs a `PrimitiveArray` from an array data reference.
impl<T: ArrowPrimitiveType> From<ArrayData> for PrimitiveArray<T> {
fn from(data: ArrayData) -> Self {
- Self::assert_compatible(data.data_type());
+ let (data_type, len, nulls, offset, mut buffers, _child_data) = data.into_parts();
+
+ Self::assert_compatible(&data_type);
assert_eq!(
- data.buffers().len(),
+ buffers.len(),
1,
"PrimitiveArray data should contain a single buffer only (values buffer)"
);
+ let buffer = buffers.pop().expect("checked above");
- let values = ScalarBuffer::new(data.buffers()[0].clone(), data.offset(), data.len());
+ let values = ScalarBuffer::new(buffer, offset, len);
Self {
- data_type: data.data_type().clone(),
+ data_type,
values,
- nulls: data.nulls().cloned(),
Review Comment:
The code looks easier and it saves a clone /call to DataType::drop as well as a Clone of buffer.
I don't think it will make any different in performance, but the code is cleaner and now consistent with other array types
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org