You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "wjones127 (via GitHub)" <gi...@apache.org> on 2023/04/10 18:30:58 UTC
[GitHub] [arrow] wjones127 opened a new pull request, #35013: GH-34983: [C++] Preserve map values nullability on C Data Interface import
wjones127 opened a new pull request, #35013:
URL: https://github.com/apache/arrow/pull/35013
### Rationale for this change
Map types with non-nullable values don't roundtrip right now through C Data interface, when they arguably should.
I also found that field names don't roundtrip, but I think that might be a feature, not a bug. Therefore I added unit tests enforcing it for now.
### What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
### Are these changes tested?
Yes, added various unit tests for export and roundtrip.
### Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please uncomment the line below and explain which changes are breaking.
-->
<!-- **This PR includes breaking changes to public APIs.** -->
<!--
Please uncomment the line below (and provide explanation) if the changes fix either (a) a security vulnerability, (b) a bug that caused incorrect or invalid data to be produced, or (c) a bug that causes a crash (even when the API contract is upheld). We use this to highlight fixes to issues that may affect users without their knowledge. For this reason, fixing bugs that cause errors don't count, since those are usually obvious.
-->
<!-- **This PR contains a "Critical Fix".** -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wjones127 merged pull request #35013: GH-34983: [C++] Preserve map values nullability on C Data Interface import
Posted by "wjones127 (via GitHub)" <gi...@apache.org>.
wjones127 merged PR #35013:
URL: https://github.com/apache/arrow/pull/35013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #35013: GH-34983: [C++] Preserve map values nullability on C Data Interface import
Posted by "ursabot (via GitHub)" <gi...@apache.org>.
ursabot commented on PR #35013:
URL: https://github.com/apache/arrow/pull/35013#issuecomment-1503625349
Benchmark runs are scheduled for baseline = e2ddf1c5687e13ef7a18cb4bdcac5d7c3652be81 and contender = c7ba0cd2cbdd89bce7d58edf573a9d3c16cf647d. c7ba0cd2cbdd89bce7d58edf573a9d3c16cf647d is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/3bfc9b39a5fc43e48d3cbe8b700352ce...7ee467436bd3404baad820150c9f2928/)
[Finished :arrow_down:0.59% :arrow_up:0.09%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/5487ccdc4e7e47af99be2e397017f8d8...688a40d8eaa8475a86c7bc15fb257c20/)
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/ca1e3e57a0d74646a5d2942f5132548d...627d67d7e32148619669b832f85121c6/)
[Failed :arrow_down:0.74% :arrow_up:0.46%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/062b823e8c7e41b886ca396e241c86de...4a25477e4edb4f2b97c2c6b129846060/)
Buildkite builds:
[Finished] [`c7ba0cd2` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/2670)
[Finished] [`c7ba0cd2` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/2703)
[Finished] [`c7ba0cd2` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/2668)
[Failed] [`c7ba0cd2` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/2694)
[Finished] [`e2ddf1c5` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/2669)
[Finished] [`e2ddf1c5` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/2702)
[Finished] [`e2ddf1c5` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/2667)
[Finished] [`e2ddf1c5` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/2693)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #35013: GH-34983: [C++] Preserve map values nullability on C Data Interface import
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35013:
URL: https://github.com/apache/arrow/pull/35013#issuecomment-1502156575
* Closes: #34983
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] paleolimbot commented on a diff in pull request #35013: GH-34983: [C++] Preserve map values nullability on C Data Interface import
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on code in PR #35013:
URL: https://github.com/apache/arrow/pull/35013#discussion_r1162214230
##########
cpp/src/arrow/c/bridge.cc:
##########
@@ -1102,7 +1102,13 @@ struct SchemaImporter {
}
bool keys_sorted = (c_struct_->flags & ARROW_FLAG_MAP_KEYS_SORTED);
- type_ = map(value_type->field(0)->type(), value_type->field(1)->type(), keys_sorted);
+ bool values_nullable = value_type->field(1)->nullable();
+ // Some implementations of Arrow (such as Rust) use a non-standard field name
+ // for key ("keys") and value ("values") fields. For simplicity, we override
+ // them on import.
Review Comment:
I need to fix this in the nanoarrow implementation (I think it currently errors for the case where the names are incorrect!)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org