You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/10/19 12:07:22 UTC
[GitHub] [arrow] milesgranger opened a new pull request, #14459: ARROW-18052: [Python] Support passing create_dir thru pq.write_to_dataset
milesgranger opened a new pull request, #14459:
URL: https://github.com/apache/arrow/pull/14459
Will fix [ARROW-18052](https://issues.apache.org/jira/browse/ARROW-18052)
Should likely need a test, but curious if we should add another `write_dataset_kwargs`(?) collection parameter for `pq.write_to_dataset` for those kwargs that ought to be passed onto `ds.write_dataset`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #14459: ARROW-18052: [Python] Support passing create_dir thru pq.write_to_dataset
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #14459:
URL: https://github.com/apache/arrow/pull/14459#issuecomment-1293267303
Could you just rebase / merge master once more to ensure CI is passing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche merged pull request #14459: ARROW-18052: [Python] Support passing create_dir thru pq.write_to_dataset
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche merged PR #14459:
URL: https://github.com/apache/arrow/pull/14459
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on pull request #14459: ARROW-18052: [Python] Support passing create_dir thru pq.write_to_dataset
Posted by GitBox <gi...@apache.org>.
milesgranger commented on PR #14459:
URL: https://github.com/apache/arrow/pull/14459#issuecomment-1286642234
It became more apparent we're already mixing `kwargs`. Updated it to be more dynamic in matching kwargs to `ds.write_dataset` and remainder going to `make_write_options` when `use_legacy_dataset=False`. Along with a matching test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #14459: ARROW-18052: [Python] Support passing create_dir thru pq.write_to_dataset
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #14459:
URL: https://github.com/apache/arrow/pull/14459#issuecomment-1283954386
https://issues.apache.org/jira/browse/ARROW-18052
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #14459: ARROW-18052: [Python] Support passing create_dir thru pq.write_to_dataset
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #14459:
URL: https://github.com/apache/arrow/pull/14459#issuecomment-1295848274
Benchmark runs are scheduled for baseline = e2533ad08ff3b2e52c4da5ff66c5715ad415c026 and contender = 5d61c62008f2f6e02951670f8d0996d90638114c. 5d61c62008f2f6e02951670f8d0996d90638114c is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/0751367f357f42a7bed5fe2345f27494...42f70cb10dd540c994d2ef4d8ac06d0b/)
[Failed :arrow_down:0.0% :arrow_up:0.0%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/a4155b31b84a4931981f1885768b04b6...2660fcf3e0a941e7bfe2f23153b301ee/)
[Finished :arrow_down:0.54% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/8a6229c595514b128d578c6797be5ed5...2a3dac41cec940fa8136e6971db2e876/)
[Finished :arrow_down:0.04% :arrow_up:0.07%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/b23887f0d6294bb2bbfaa3b58bf87522...767e22c1d2f34fc28e6a0446a6144431/)
Buildkite builds:
[Finished] [`5d61c620` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/1777)
[Failed] [`5d61c620` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1798)
[Finished] [`5d61c620` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/1764)
[Finished] [`5d61c620` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/1790)
[Finished] [`e2533ad0` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/1776)
[Failed] [`e2533ad0` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1797)
[Finished] [`e2533ad0` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/1763)
[Finished] [`e2533ad0` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/1789)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #14459: ARROW-18052: [Python] Support passing create_dir thru pq.write_to_dataset
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #14459:
URL: https://github.com/apache/arrow/pull/14459#issuecomment-1290119255
@milesgranger you accidentally added some parquet data file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on pull request #14459: ARROW-18052: [Python] Support passing create_dir thru pq.write_to_dataset
Posted by GitBox <gi...@apache.org>.
milesgranger commented on PR #14459:
URL: https://github.com/apache/arrow/pull/14459#issuecomment-1293612611
@jorisvandenbossche seems okay, failure unrelated it appears.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org