You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/03/08 14:25:14 UTC
[GitHub] [arrow] jorisvandenbossche opened a new pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
jorisvandenbossche opened a new pull request #12584:
URL: https://github.com/apache/arrow/pull/12584
Currently, users can still specify `use_legacy_dataset=True` explicitly to get the old implementation/behaviour. But if we want to remove that implementation at some point (ARROW-15868), we should start deprecating that option, to futher nudge people to the new implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on pull request #12584:
URL: https://github.com/apache/arrow/pull/12584#issuecomment-1061972761
@github-actions crossbow submit -g integration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12584:
URL: https://github.com/apache/arrow/pull/12584#issuecomment-1062028852
Revision: 78ff0a58320f08330518e2e1dc13d6206fe64c03
Submitted crossbow builds: [ursacomputing/crossbow @ actions-1730](https://github.com/ursacomputing/crossbow/branches/all?query=actions-1730)
|Task|Status|
|----|------|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.7-turbodbc-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.7-turbodbc-latest)|
|test-conda-python-3.7-turbodbc-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.7-turbodbc-master)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1730-github-test-conda-python-3.9-spark-master)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #12584:
URL: https://github.com/apache/arrow/pull/12584#discussion_r823591435
##########
File path: python/pyarrow/parquet.py
##########
@@ -2006,6 +2007,12 @@ def read_table(source, columns=None, use_threads=True, metadata=None,
return dataset.read(columns=columns, use_threads=use_threads,
use_pandas_metadata=use_pandas_metadata)
+ warnings.warn(
+ "Passing 'use_legacy_dataset=True' to get the legacy behaviour is "
+ "deprecated as of pyarrow 8.0.0, and the legacy implementation will "
+ "be removed in a future version.",
+ DeprecationWarning, stacklevel=2)
Review comment:
Do we want to use DeprecationWarning or FutureWarning?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12584:
URL: https://github.com/apache/arrow/pull/12584#issuecomment-1068230968
Benchmark runs are scheduled for baseline = 2e314cbee42c7cf25c8e30934e945f11a070e54b and contender = 3bf061783f4e1ab447d2eb0f487c0c4fce6d5b15. 3bf061783f4e1ab447d2eb0f487c0c4fce6d5b15 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/52c35a4090a64edb89ead6afcea43d4e...2ae3f30bf3004859a24d6df93265536f/)
[Finished :arrow_down:0.25% :arrow_up:0.0%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/b1eb42950c194e3db852878336c8e316...9e6b792d1e544d76be495a2700811f2e/)
[Finished :arrow_down:0.36% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/e33a2d4e3e4347469d5ee75d97acf3ab...ade1ae6a21b64efabcc0fe4ade6b6e9c/)
[Finished :arrow_down:1.66% :arrow_up:0.21%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/875075d516924f928312ea7646d76308...fdf63750bd6f4fa3ac55edfe664aecb9/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on a change in pull request #12584:
URL: https://github.com/apache/arrow/pull/12584#discussion_r827200147
##########
File path: python/pyarrow/parquet.py
##########
@@ -2006,6 +2007,12 @@ def read_table(source, columns=None, use_threads=True, metadata=None,
return dataset.read(columns=columns, use_threads=use_threads,
use_pandas_metadata=use_pandas_metadata)
+ warnings.warn(
+ "Passing 'use_legacy_dataset=True' to get the legacy behaviour is "
+ "deprecated as of pyarrow 8.0.0, and the legacy implementation will "
+ "be removed in a future version.",
+ DeprecationWarning, stacklevel=2)
Review comment:
We can probably directly raise a FutureWarning, but all the others here are still raising DeprecationWarnings. Will do a separate PR to change those all from deprecation to future warning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12584:
URL: https://github.com/apache/arrow/pull/12584#issuecomment-1061877784
https://issues.apache.org/jira/browse/ARROW-15870
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12584:
URL: https://github.com/apache/arrow/pull/12584#issuecomment-1068230968
Benchmark runs are scheduled for baseline = 2e314cbee42c7cf25c8e30934e945f11a070e54b and contender = 3bf061783f4e1ab447d2eb0f487c0c4fce6d5b15. 3bf061783f4e1ab447d2eb0f487c0c4fce6d5b15 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/52c35a4090a64edb89ead6afcea43d4e...2ae3f30bf3004859a24d6df93265536f/)
[Scheduled] [test-mac-arm](https://conbench.ursa.dev/compare/runs/b1eb42950c194e3db852878336c8e316...9e6b792d1e544d76be495a2700811f2e/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/e33a2d4e3e4347469d5ee75d97acf3ab...ade1ae6a21b64efabcc0fe4ade6b6e9c/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/875075d516924f928312ea7646d76308...fdf63750bd6f4fa3ac55edfe664aecb9/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #12584:
URL: https://github.com/apache/arrow/pull/12584#issuecomment-1068230968
Benchmark runs are scheduled for baseline = 2e314cbee42c7cf25c8e30934e945f11a070e54b and contender = 3bf061783f4e1ab447d2eb0f487c0c4fce6d5b15. 3bf061783f4e1ab447d2eb0f487c0c4fce6d5b15 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/52c35a4090a64edb89ead6afcea43d4e...2ae3f30bf3004859a24d6df93265536f/)
[Scheduled] [test-mac-arm](https://conbench.ursa.dev/compare/runs/b1eb42950c194e3db852878336c8e316...9e6b792d1e544d76be495a2700811f2e/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/e33a2d4e3e4347469d5ee75d97acf3ab...ade1ae6a21b64efabcc0fe4ade6b6e9c/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/875075d516924f928312ea7646d76308...fdf63750bd6f4fa3ac55edfe664aecb9/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12584:
URL: https://github.com/apache/arrow/pull/12584#issuecomment-1068230968
Benchmark runs are scheduled for baseline = 2e314cbee42c7cf25c8e30934e945f11a070e54b and contender = 3bf061783f4e1ab447d2eb0f487c0c4fce6d5b15. 3bf061783f4e1ab447d2eb0f487c0c4fce6d5b15 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/52c35a4090a64edb89ead6afcea43d4e...2ae3f30bf3004859a24d6df93265536f/)
[Scheduled] [test-mac-arm](https://conbench.ursa.dev/compare/runs/b1eb42950c194e3db852878336c8e316...9e6b792d1e544d76be495a2700811f2e/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/e33a2d4e3e4347469d5ee75d97acf3ab...ade1ae6a21b64efabcc0fe4ade6b6e9c/)
[Finished :arrow_down:1.66% :arrow_up:0.21%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/875075d516924f928312ea7646d76308...fdf63750bd6f4fa3ac55edfe664aecb9/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12584:
URL: https://github.com/apache/arrow/pull/12584#issuecomment-1068230968
Benchmark runs are scheduled for baseline = 2e314cbee42c7cf25c8e30934e945f11a070e54b and contender = 3bf061783f4e1ab447d2eb0f487c0c4fce6d5b15. 3bf061783f4e1ab447d2eb0f487c0c4fce6d5b15 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/52c35a4090a64edb89ead6afcea43d4e...2ae3f30bf3004859a24d6df93265536f/)
[Finished :arrow_down:0.25% :arrow_up:0.0%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/b1eb42950c194e3db852878336c8e316...9e6b792d1e544d76be495a2700811f2e/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/e33a2d4e3e4347469d5ee75d97acf3ab...ade1ae6a21b64efabcc0fe4ade6b6e9c/)
[Finished :arrow_down:1.66% :arrow_up:0.21%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/875075d516924f928312ea7646d76308...fdf63750bd6f4fa3ac55edfe664aecb9/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche closed pull request #12584: ARROW-15870: [Python] Start to raise deprecation warnings for use_legacy_dataset=True in parquet.read_table
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche closed pull request #12584:
URL: https://github.com/apache/arrow/pull/12584
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org