You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/07/15 17:21:58 UTC
[GitHub] [arrow-rs] alamb opened a new pull request, #2081: Donate `object_store` code from object_store_rs to arrow-rs
alamb opened a new pull request, #2081:
URL: https://github.com/apache/arrow-rs/pull/2081
Draft until
- [ ] Port CI from .circleci
- [ ] IP Clearance
- [ ] Approved on mailing list
# Which issue does this PR close?
re https://github.com/apache/arrow-rs/issues/2030
# Rationale for this change
See https://github.com/apache/arrow-rs/issues/2030
# What changes are included in this PR?
These are kept in different commits for easier review:
- [ ] Copy Brings in the code from object_store_rs at https://github.com/influxdata/object_store_rs/commit/3c51870ac41a90942c2e45bb499a893d514ed1da
- [ ] Update documentation for new home
- [ ] Port CI (XXX) to github actons
# Are there any user-facing changes?
Next steps will be to sort out release strategy
<!---
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!---
If there are any breaking changes to public APIs, please add the `breaking change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb merged pull request #2081: Donate `object_store` code from object_store_rs to arrow-rs
Posted by GitBox <gi...@apache.org>.
alamb merged PR #2081:
URL: https://github.com/apache/arrow-rs/pull/2081
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on pull request #2081: Donate `object_store` code from object_store_rs to arrow-rs
Posted by GitBox <gi...@apache.org>.
tustvold commented on PR #2081:
URL: https://github.com/apache/arrow-rs/pull/2081#issuecomment-1185780188
> keep the history from object_store_rs
So unfortunately object_store_rs contains merge commits, so the only way to do this would break the linear history of arrow-rs, either by keeping the merge commits from object_store_rs/main or by adding a single merge commit. I therefore think its not worth it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on pull request #2081: Donate `object_store` code from object_store_rs to arrow-rs
Posted by GitBox <gi...@apache.org>.
tustvold commented on PR #2081:
URL: https://github.com/apache/arrow-rs/pull/2081#issuecomment-1187747842
Probably the Github Actions runner is running as root, and therefore the permission failure doesn't occur. I'll rework the test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on pull request #2081: Donate `object_store` code from object_store_rs to arrow-rs
Posted by GitBox <gi...@apache.org>.
alamb commented on PR #2081:
URL: https://github.com/apache/arrow-rs/pull/2081#issuecomment-1186154351
Update on test migration:
1. There appears to be one test that needs to be rewritten for github runners https://github.com/apache/arrow-rs/runs/7362535940?check_suite_focus=true (`bubble_up_io_errors`)
2. I have gotten the emulator tests running using github actions here: https://github.com/influxdata/object_store_rs/pull/51 (it needs some polishing but I propose merging this PR first and then adding in object_store tests after the merge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on pull request #2081: Donate `object_store` code from object_store_rs to arrow-rs
Posted by GitBox <gi...@apache.org>.
alamb commented on PR #2081:
URL: https://github.com/apache/arrow-rs/pull/2081#issuecomment-1187742353
> There appears to be one test that needs to be rewritten for github runners https://github.com/apache/arrow-rs/runs/7362535940?check_suite_focus=true (bubble_up_io_errors)
@tustvold sorry to bother you -- but I was wondering if you had any ideas of ways to port the above test to run on github actions -- specifically it appears that setting the file mode to `0o000` doesn't actually generate an Error 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on pull request #2081: Donate `object_store` code from object_store_rs to arrow-rs
Posted by GitBox <gi...@apache.org>.
alamb commented on PR #2081:
URL: https://github.com/apache/arrow-rs/pull/2081#issuecomment-1190693008
> Probably the Github Actions runner is running as root, and therefore the permission failure doesn't occur. I'll rework the test
I have `#[ignored]` the test in https://github.com/apache/arrow-rs/pull/2081/commits/ed0f926612adafe40ff0e0fef0eee29dd9e03418 and will file a follow on ticket to fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on pull request #2081: Donate `object_store` code from object_store_rs to arrow-rs
Posted by GitBox <gi...@apache.org>.
tustvold commented on PR #2081:
URL: https://github.com/apache/arrow-rs/pull/2081#issuecomment-1185756398
I wonder if we could preserve the history somehow, I'm mainly interested in not losing releases, but this would also help with in-flight PRs.
Perhaps you could add object_store_rs as a remote, add a commit moving it to a subdirectory, and then merge it into arrow-rs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] ursabot commented on pull request #2081: Donate `object_store` code from object_store_rs to arrow-rs
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #2081:
URL: https://github.com/apache/arrow-rs/pull/2081#issuecomment-1192836612
Benchmark runs are scheduled for baseline = add264979720a3500440da7b38b6f5ae6ac9eadb and contender = 866f1a102605d6f746b0e3ad7a009d6297efd8e8. 866f1a102605d6f746b0e3ad7a009d6297efd8e8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped :warning: Benchmarking of arrow-rs-commits is not supported on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/7c40b2b9c8194b999ca316d8cb804793...6df8a1cd8b3e4d7f8ee4626c631d7927/)
[Skipped :warning: Benchmarking of arrow-rs-commits is not supported on test-mac-arm] [test-mac-arm](https://conbench.ursa.dev/compare/runs/ce0ac8cd63d04171a11fa9f26b7197bd...78f4ffa1a6d745a4960684fe3873f5c1/)
[Skipped :warning: Benchmarking of arrow-rs-commits is not supported on ursa-i9-9960x] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/44d684f35fda49a484ccc80125c0a773...1a13a0bfa65046a4b9b21b343c88ba11/)
[Skipped :warning: Benchmarking of arrow-rs-commits is not supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/0f44a3029129467481799743509a4ae0...f713cb9ff15b4b0cb6baf11fdee6fedf/)
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on pull request #2081: Donate `object_store` code from object_store_rs to arrow-rs
Posted by GitBox <gi...@apache.org>.
alamb commented on PR #2081:
URL: https://github.com/apache/arrow-rs/pull/2081#issuecomment-1185767562
Update here: my git-fu is not strong enough to try and keep the history from object_store_rs -- @tustvold plans to give it a try. In the interim I will work on porting the tests to use github actions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] codecov-commenter commented on pull request #2081: Donate `object_store` code from object_store_rs to arrow-rs
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #2081:
URL: https://github.com/apache/arrow-rs/pull/2081#issuecomment-1185774349
# [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/2081?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2081](https://codecov.io/gh/apache/arrow-rs/pull/2081?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fb5ce62) into [master](https://codecov.io/gh/apache/arrow-rs/commit/cb7e5b0c8437c3f2ce2ae21951331599de4ccc7d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cb7e5b0) will **decrease** coverage by `0.78%`.
> The diff coverage is `55.62%`.
```diff
@@ Coverage Diff @@
## master #2081 +/- ##
==========================================
- Coverage 83.68% 82.89% -0.79%
==========================================
Files 224 236 +12
Lines 58826 60539 +1713
==========================================
+ Hits 49227 50184 +957
- Misses 9599 10355 +756
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-rs/pull/2081?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [object\_store/src/aws.rs](https://codecov.io/gh/apache/arrow-rs/pull/2081/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-b2JqZWN0X3N0b3JlL3NyYy9hd3MucnM=) | `0.00% <0.00%> (ø)` | |
| [object\_store/src/azure.rs](https://codecov.io/gh/apache/arrow-rs/pull/2081/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-b2JqZWN0X3N0b3JlL3NyYy9henVyZS5ycw==) | `0.00% <0.00%> (ø)` | |
| [object\_store/src/gcp.rs](https://codecov.io/gh/apache/arrow-rs/pull/2081/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-b2JqZWN0X3N0b3JlL3NyYy9nY3AucnM=) | `0.00% <0.00%> (ø)` | |
| [object\_store/src/oauth.rs](https://codecov.io/gh/apache/arrow-rs/pull/2081/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-b2JqZWN0X3N0b3JlL3NyYy9vYXV0aC5ycw==) | `0.00% <0.00%> (ø)` | |
| [object\_store/src/token.rs](https://codecov.io/gh/apache/arrow-rs/pull/2081/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-b2JqZWN0X3N0b3JlL3NyYy90b2tlbi5ycw==) | `0.00% <0.00%> (ø)` | |
| [object\_store/src/util.rs](https://codecov.io/gh/apache/arrow-rs/pull/2081/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-b2JqZWN0X3N0b3JlL3NyYy91dGlsLnJz) | `50.00% <50.00%> (ø)` | |
| [object\_store/src/local.rs](https://codecov.io/gh/apache/arrow-rs/pull/2081/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-b2JqZWN0X3N0b3JlL3NyYy9sb2NhbC5ycw==) | `84.39% <84.39%> (ø)` | |
| [object\_store/src/lib.rs](https://codecov.io/gh/apache/arrow-rs/pull/2081/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-b2JqZWN0X3N0b3JlL3NyYy9saWIucnM=) | `84.42% <84.42%> (ø)` | |
| [object\_store/src/path/mod.rs](https://codecov.io/gh/apache/arrow-rs/pull/2081/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-b2JqZWN0X3N0b3JlL3NyYy9wYXRoL21vZC5ycw==) | `90.90% <90.90%> (ø)` | |
| [object\_store/src/memory.rs](https://codecov.io/gh/apache/arrow-rs/pull/2081/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-b2JqZWN0X3N0b3JlL3NyYy9tZW1vcnkucnM=) | `92.17% <92.17%> (ø)` | |
| ... and [5 more](https://codecov.io/gh/apache/arrow-rs/pull/2081/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-rs/pull/2081?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/2081?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [cb7e5b0...fb5ce62](https://codecov.io/gh/apache/arrow-rs/pull/2081?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org