You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/05/05 07:25:27 UTC
[GitHub] [arrow-datafusion] tustvold opened a new issue, #2450: Preserve Element Name in ScalarValue::List
tustvold opened a new issue, #2450:
URL: https://github.com/apache/arrow-datafusion/issues/2450
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
`ScalarValue::List` lacks the ability to preserve the name of its element which prevents round-tripping it correctly
**Describe the solution you'd like**
`ScalarValue::List` should store a `Box<Field>` instead, much like `ScalarValue::Struct`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] tustvold commented on issue #2450: Preserve Element Name in ScalarValue::List
Posted by GitBox <gi...@apache.org>.
tustvold commented on issue #2450:
URL: https://github.com/apache/arrow-datafusion/issues/2450#issuecomment-1130202074
Yup precisely, when I took a stab at this previously I had to hack around https://github.com/apache/arrow-rs/issues/1649 which was a bit meh.
Let me know if you want to pick one or both of these issues up, as that would be awesome.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #2450: Preserve Element Name in ScalarValue::List
Posted by GitBox <gi...@apache.org>.
comphead commented on issue #2450:
URL: https://github.com/apache/arrow-datafusion/issues/2450#issuecomment-1129887886
Hi @tustvold is it to be consistent with arrow Datatype::List()
```
/// A list of some logical data type with variable length.
List(Box<Field>),
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #2450: Preserve Element Name in ScalarValue::List
Posted by GitBox <gi...@apache.org>.
comphead commented on issue #2450:
URL: https://github.com/apache/arrow-datafusion/issues/2450#issuecomment-1168068480
> The latter, you will need to both specify the name of the lists element, and its nullability
Thanks @tustvold for the quick reply. I was confused because we have the test like
```
ScalarValue::List(None, new_box_field("item", DataType::Boolean, true)),
```
which also looks like list of primitives but with other datatype provided
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] tustvold commented on issue #2450: Preserve Element Name in ScalarValue::List
Posted by GitBox <gi...@apache.org>.
tustvold commented on issue #2450:
URL: https://github.com/apache/arrow-datafusion/issues/2450#issuecomment-1165455444
The latter, you will need to both specify the name of the lists element, and its nullability
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed issue #2450: Preserve Element Name in ScalarValue::List
Posted by GitBox <gi...@apache.org>.
alamb closed issue #2450: Preserve Element Name in ScalarValue::List
URL: https://github.com/apache/arrow-datafusion/issues/2450
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] tustvold commented on issue #2450: Preserve Element Name in ScalarValue::List
Posted by GitBox <gi...@apache.org>.
tustvold commented on issue #2450:
URL: https://github.com/apache/arrow-datafusion/issues/2450#issuecomment-1168728762
This will be a breaking change, and will require updating said tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #2450: Preserve Element Name in ScalarValue::List
Posted by GitBox <gi...@apache.org>.
comphead commented on issue #2450:
URL: https://github.com/apache/arrow-datafusion/issues/2450#issuecomment-1163751034
@tustvold for the test, which was before. This represents roundtrip for the list
```
ScalarValue::List(
Some(vec![
ScalarValue::Float32(Some(-213.1)),
ScalarValue::Float32(None),
ScalarValue::Float32(Some(5.5)),
ScalarValue::Float32(Some(2.0)),
ScalarValue::Float32(Some(1.0)),
]),
Box::new(DataType::List(new_box_field(
"level1",
DataType::Float32,
true,
))),
```
Do you expect after change it has to be
```
ScalarValue::List(
Some(vec![
ScalarValue::Float32(Some(-213.1)),
ScalarValue::Float32(None),
ScalarValue::Float32(Some(5.5)),
ScalarValue::Float32(Some(2.0)),
ScalarValue::Float32(Some(1.0)),
]),
new_box_field("level1", DataType::Float32, true),
),
```
or
```
ScalarValue::List(
Some(vec![
ScalarValue::Float32(Some(-213.1)),
ScalarValue::Float32(None),
ScalarValue::Float32(Some(5.5)),
ScalarValue::Float32(Some(2.0)),
ScalarValue::Float32(Some(1.0)),
]),
new_box_field("item", DataType::List(new_box_field("level1", DataType::Float32, true)), true),
),
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #2450: Preserve Element Name in ScalarValue::List
Posted by GitBox <gi...@apache.org>.
comphead commented on issue #2450:
URL: https://github.com/apache/arrow-datafusion/issues/2450#issuecomment-1131793236
I can try with this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #2450: Preserve Element Name in ScalarValue::List
Posted by GitBox <gi...@apache.org>.
comphead commented on issue #2450:
URL: https://github.com/apache/arrow-datafusion/issues/2450#issuecomment-1134865657
@tustvold Protobuf serde is expected to pass the entire field value, not just datatype like it was before?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org