You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/02 20:13:02 UTC
[GitHub] [arrow] carols10cents opened a new pull request #8820: ARROW-10791: [Rust] StreamReader, read_dictionary duplicating schema info
carols10cents opened a new pull request #8820:
URL: https://github.com/apache/arrow/pull/8820
The purpose of this PR is refactoring `read_dictionary` to only need one kind of `Schema`, which lets us then remove the `find_dictionary_field` function and the `ipc_schema` field on `StreamReader` by adding a way to look up schema fields that use a particular dictionary by ID.
I'm also resubmitting a change to the `dict_id`/`dict_is_ordered` methods on `Field`; I had submitted this to @nevi-me to become part of #8200 but it looks like it got lost in a rebase or something? I think it's more correct to only return values if the fields have a dictionary as their datatype.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nevi-me closed pull request #8820: ARROW-10791: [Rust] StreamReader, read_dictionary duplicating schema info
Posted by GitBox <gi...@apache.org>.
nevi-me closed pull request #8820:
URL: https://github.com/apache/arrow/pull/8820
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] carols10cents commented on a change in pull request #8820: ARROW-10791: [Rust] StreamReader, read_dictionary duplicating schema info
Posted by GitBox <gi...@apache.org>.
carols10cents commented on a change in pull request #8820:
URL: https://github.com/apache/arrow/pull/8820#discussion_r535262084
##########
File path: rust/integration-testing/src/bin/arrow-json-integration-test.rs
##########
@@ -489,7 +489,12 @@ fn array_from_json(
Ok(Arc::new(array))
}
DataType::Dictionary(key_type, value_type) => {
- let dict_id = field.dict_id();
+ let dict_id = field.dict_id().ok_or_else(|| {
Review comment:
No worries! I remember I sent you a rebased PR to your branch that was... complicated... so I'm not surprised :) Thanks for the merge!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nevi-me commented on a change in pull request #8820: ARROW-10791: [Rust] StreamReader, read_dictionary duplicating schema info
Posted by GitBox <gi...@apache.org>.
nevi-me commented on a change in pull request #8820:
URL: https://github.com/apache/arrow/pull/8820#discussion_r534660714
##########
File path: rust/integration-testing/src/bin/arrow-json-integration-test.rs
##########
@@ -489,7 +489,12 @@ fn array_from_json(
Ok(Arc::new(array))
}
DataType::Dictionary(key_type, value_type) => {
- let dict_id = field.dict_id();
+ let dict_id = field.dict_id().ok_or_else(|| {
Review comment:
I apologise, I do remember this change, but I'm not sure of what happened; I presume while I was rebasing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8820: ARROW-10791: [Rust] StreamReader, read_dictionary duplicating schema info
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8820:
URL: https://github.com/apache/arrow/pull/8820#issuecomment-737475652
https://issues.apache.org/jira/browse/ARROW-10791
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org