You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/11/25 21:51:57 UTC
[GitHub] [arrow] carols10cents opened a new pull request #8773: ARROW-10268: [Rust] Write out non-nested dictionaries in the IPC format
carols10cents opened a new pull request #8773:
URL: https://github.com/apache/arrow/pull/8773
This adds support for writing non-nested dictionaries in the IPC format in Rust.
The first commit in this PR is sort of a bug fix related to correctly supporting dictionaries, but I wanted to draw attention to it specifically and make sure I'm handling that case correctly.
The second commit is the majority of the new functionality.
The third commit removes the TODO assertion in the integration test tool that errored if dictionaries were encountered in the JSON... which means nothing is reading from the `dictionaries` field of the `ArrowFile` struct. This might mean it can be removed? Or perhaps I'm missing something, or something isn't yet implemented?
The fourth commit fixes an issue when the integration testing tool reads JSON files with dictionaries; the null bit buffers weren't being set. This didn't matter when testing Rust-to-Rust as neither side was setting the null bit buffers, so the columns were equal, but this does matter when testing dictionaries with another language and Rust.
The fifth commit enables the dictionary and unsigned dictionary integration tests for Rust, but doesn't enable the nested dictionary test as that isn't working yet.
The sixth commit makes the Rust integration testing tool less noisy-- it was printing more output than the other languages' integration testing tools. Let me know if this should be done a different way.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nevi-me closed pull request #8773: ARROW-10268: [Rust] Write out non-nested dictionaries in the IPC format
Posted by GitBox <gi...@apache.org>.
nevi-me closed pull request #8773:
URL: https://github.com/apache/arrow/pull/8773
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8773: ARROW-10268: [Rust] Write out non-nested dictionaries in the IPC format
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8773:
URL: https://github.com/apache/arrow/pull/8773#issuecomment-733966792
https://issues.apache.org/jira/browse/ARROW-10268
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org