You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "dnsco (via GitHub)" <gi...@apache.org> on 2023/02/01 03:57:15 UTC
[GitHub] [arrow-rs] dnsco opened a new issue, #3644: A trait for append_value and append_null on ArrayBuilders
dnsco opened a new issue, #3644:
URL: https://github.com/apache/arrow-rs/issues/3644
I'm generating an arrow schema, and because there is not a macro, when it's time set the values I need to invoke a macro like this:
```rust
macro_rules! set_value {
($builder:expr,$i:expr,$typ:ty,$getter:ident,$value:expr) => {{
let b: &mut $typ = $builder.field_builder::<$typ>($i).unwrap();
match $value {
Some(cow) => {
let v = cow.$getter().unwrap();
b.append_value(v)
}
None => b.append_null(),
}
}};
}
```
```rust
fn append_value(
f: &Field,
builder: &mut StructBuilder,
i: usize,
value: Option<Cow<Value>>,
) -> Result<()> {
match f.data_type() {
DataType::Float64 => set_value!(builder, i, Float64Builder, as_f64, value),
DataType::Float32 => set_value!(builder, i, Float32Builder, as_f32, value),
DataType::Int64 => set_value!(builder, i, Int64Builder, as_i64, value),
DataType::Int32 => set_value!(builder, i, Int32Builder, as_i32, value),
DataType::UInt64 => set_value!(builder, i, UInt64Builder, as_u64, value),
DataType::UInt32 => set_value!(builder, i, UInt32Builder, as_u32, value),
DataType::Utf8 => set_value!(builder, i, StringBuilder, as_str, value),
DataType::LargeUtf8 => set_value!(builder, i, LargeStringBuilder, as_str, value),
DataType::Binary => set_value!(builder, i, BinaryBuilder, as_bytes, value),
DataType::LargeBinary => set_value!(builder, i, LargeBinaryBuilder, as_bytes, value),
DataType::Boolean => set_value!(builder, i, BooleanBuilder, as_bool, value),
...
}
```
If there was a trait that contained append_value and append_null this macro could be replaced with a generic function, which would be a much better developer experience.
Thanks for the library!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on issue #3644: A trait for append_value and append_null on ArrayBuilders
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #3644:
URL: https://github.com/apache/arrow-rs/issues/3644#issuecomment-1412580604
As of the most recent arrow release, the builders implement [`Extend`](https://doc.rust-lang.org/std/iter/trait.Extend.html) is this possibly what you are looking for?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] dnsco closed issue #3644: A trait for append_value and append_null on ArrayBuilders
Posted by "dnsco (via GitHub)" <gi...@apache.org>.
dnsco closed issue #3644: A trait for append_value and append_null on ArrayBuilders
URL: https://github.com/apache/arrow-rs/issues/3644
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] dnsco commented on issue #3644: A trait for append_value and append_null on ArrayBuilders
Posted by "dnsco (via GitHub)" <gi...@apache.org>.
dnsco commented on issue #3644:
URL: https://github.com/apache/arrow-rs/issues/3644#issuecomment-1412906085
So the builders implement extend but an &mut Builder does not have an implementation. I'm bashing my head against the wall trying to figure out how to make it work.
here's the new method:
```rust
pub fn set_val<T, R, V, F>(mut builder: T, value: Option<V>, getter: F) -> Result<()>
where
T: Extend<Option<R>>,
V: Deref<Target = Value>,
F: FnOnce(&V::Target) -> Option<R>,
{
let v = value
.as_deref()
.map(|v| {
getter(v).ok_or_else(|| {
let msg = format!("Could not cast {} to correct type", v);
Error::new(InvalidData, msg)
})
})
.transpose()?;
builder.extend(vec![v]);
Ok(())
}
```
called as:
```rust
DataType::Float64 => {
let b = struct_builder.field_builder::<Float64Builder>(i).unwrap();
set_val(b, value, Value::as_f64)?
}
```
Yielding the error:
```
96 | set_val(b, value, Value::as_f64)?
| ------- ^ the trait `Extend<std::option::Option<_>>` is not implemented for `&mut PrimitiveBuilder<Float64Type>`
| |
| required by a bound introduced by this call
|
= help: the trait `Extend<std::option::Option<<P as ArrowPrimitiveType>::Native>>` is implemented for `PrimitiveBuilder<P>`
note: required by a bound in `set_val`
--> project/src/record_conversion/builder_appending.rs:64:8
|
62 | pub fn set_val<T, R, V, F>(mut builder: T, value: Option<V>, getter: F) -> Result<()>
| ------- required by a bound in this
63 | where
64 | T: Extend<Option<R>>,
| ^^^^^^^^^^^^^^^^^ required by this bound in `set_val`
```
Am I missing something simple? If I try and reference ```*b``` at the callsite I get:
```
error[E0507]: cannot move out of `*b` which is behind a mutable reference
--> prost-arrow/src/record_conversion/builder_appending.rs:96:21
|
96 | set_val(*b, value, Value::as_f64)?
| ^^ move occurs because `*b` has type `PrimitiveBuilder<Float64Type>`, which does not implement the `Copy` trait
```
Is there a reason there can't be an implementation of Extend for &mut Builders?
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] dnsco commented on issue #3644: A trait for append_value and append_null on ArrayBuilders
Posted by "dnsco (via GitHub)" <gi...@apache.org>.
dnsco commented on issue #3644:
URL: https://github.com/apache/arrow-rs/issues/3644#issuecomment-1412939626
Per conversation on github I was missing an &mut T in the function's signature, extend should do what I need!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org