You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "AlenkaF (via GitHub)" <gi...@apache.org> on 2023/05/10 12:05:00 UTC
[GitHub] [arrow] AlenkaF opened a new pull request, #35530: GH-35264: [Python] Interchange protocol: test clean-up
AlenkaF opened a new pull request, #35530:
URL: https://github.com/apache/arrow/pull/35530
The diff is a bit confusing so I will add some notes here:
- `test_categorical_roundtrip` is not removed but renamed to `test_pandas_roundtrip_categorical` so that all tests checking `pyarrow` -> `pandas` -> `pyarrow` start with `test_pandas_roundtip_*`
- the skip for the `test_pandas_roundtrip_categorical` is removed
- `test_pandas_to_pyarrow_categorical_with_missing` is removed as the conversion for categorical with missing values is now checked in `test_pandas_roundtrip_categorical`
- `test_roundtrip_pandas_boolean` is removed and the check for boolean has been added to `test_pandas_roundtrip`
- `test_pandas_assertion_error_large_string` and `test_pandas_to_pyarrow_string_with_missing` are removed as the update for these is done separately in https://github.com/apache/arrow/pull/35504
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35530:
URL: https://github.com/apache/arrow/pull/35530#issuecomment-1542086765
* Closes: #35264
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche merged pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche merged PR #35530:
URL: https://github.com/apache/arrow/pull/35530
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "ursabot (via GitHub)" <gi...@apache.org>.
ursabot commented on PR #35530:
URL: https://github.com/apache/arrow/pull/35530#issuecomment-1546702611
Benchmark runs are scheduled for baseline = b36ff71bfeddbd9ed99de58dabee382e0be3bcb5 and contender = 14f9bf923a5655c04130a2db0cabe15a0249dd79. 14f9bf923a5655c04130a2db0cabe15a0249dd79 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/7469f7640d4f4c1b8126a2c854af0628...1b5ded494dbf4f65b43328362c7fcca7/)
[Finished :arrow_down:0.84% :arrow_up:0.06%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/27b22857ccad4b6192c5cebf1f19b78d...50c295200bba44638ef77492ba2de447/)
[Finished :arrow_down:0.25% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/3702cca7e6cb421fa01379891897879f...80cbe13b10ca4d39b05e59e4b4d5037d/)
[Finished :arrow_down:0.39% :arrow_up:0.06%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/499dda40b6cd4c1cb0094765eda206e9...d6109567cf5e4da597c3d982ccbfe5d6/)
Buildkite builds:
[Finished] [`14f9bf92` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/2863)
[Finished] [`14f9bf92` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/2891)
[Finished] [`14f9bf92` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/2855)
[Finished] [`14f9bf92` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/2882)
[Finished] [`b36ff71b` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/2865)
[Finished] [`b36ff71b` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/2902)
[Finished] [`b36ff71b` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/2867)
[Finished] [`b36ff71b` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/2892)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #35530:
URL: https://github.com/apache/arrow/pull/35530#discussion_r1191374433
##########
python/pyarrow/tests/interchange/test_conversion.py:
##########
@@ -189,10 +143,9 @@ def test_pandas_roundtrip(uint, int, float, np_float):
@pytest.mark.pandas
-def test_roundtrip_pandas_string():
- # See https://github.com/pandas-dev/pandas/issues/50554
- if Version(pd.__version__) < Version("1.6"):
- pytest.skip(" Column.size() called as a method in pandas 2.0.0")
+def test_pandas_roundtrip_string():
+ if Version(pd.__version__) < Version("2.0.1"):
+ pytest.skip("Column.size() bug in pandas")
Review Comment:
Hm yes, I think this might be a wrong merge conflict. Will need to check and correct, thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35530:
URL: https://github.com/apache/arrow/pull/35530#issuecomment-1543659641
Revision: 0e74ace51b3668fca155163e0f887d83ba778ed5
Submitted crossbow builds: [ursacomputing/crossbow @ actions-894d083930](https://github.com/ursacomputing/crossbow/branches/all?query=actions-894d083930)
|Task|Status|
|----|------|
|test-conda-python-3.7-pandas-1.0|[](https://github.com/ursacomputing/crossbow/actions/runs/4946292349/jobs/8844170618)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/4946292892/jobs/8844171507)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/4946293474/jobs/8844172621)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions/runs/4946292555/jobs/8844170832)|
|test-conda-python-3.9-pandas-upstream_devel|[](https://github.com/ursacomputing/crossbow/actions/runs/4946293115/jobs/8844171934)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on PR #35530:
URL: https://github.com/apache/arrow/pull/35530#issuecomment-1543655388
@github-actions crossbow submit test-conda-python-*-pandas-*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #35530:
URL: https://github.com/apache/arrow/pull/35530#discussion_r1190874130
##########
python/pyarrow/tests/interchange/test_conversion.py:
##########
@@ -217,19 +170,28 @@ def test_roundtrip_pandas_string():
@pytest.mark.pandas
-def test_roundtrip_pandas_boolean():
- if Version(pd.__version__) < Version("1.5.0"):
- pytest.skip("__dataframe__ added to pandas in 1.5.0")
+def test_pandas_roundtrip_categorical():
+ if Version(pd.__version__) < Version("2.0.2"):
Review Comment:
I have, unfortunately, added this change in the rebasing process and is therefore not visible in the commit history.
The change can be seen here:
https://github.com/apache/arrow/blob/0e74ace51b3668fca155163e0f887d83ba778ed5/python/pyarrow/tests/interchange/test_conversion.py#L243-L245
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on code in PR #35530:
URL: https://github.com/apache/arrow/pull/35530#discussion_r1191199799
##########
python/pyarrow/tests/interchange/test_conversion.py:
##########
@@ -189,10 +143,9 @@ def test_pandas_roundtrip(uint, int, float, np_float):
@pytest.mark.pandas
-def test_roundtrip_pandas_string():
- # See https://github.com/pandas-dev/pandas/issues/50554
- if Version(pd.__version__) < Version("1.6"):
- pytest.skip(" Column.size() called as a method in pandas 2.0.0")
+def test_pandas_roundtrip_string():
+ if Version(pd.__version__) < Version("2.0.1"):
+ pytest.skip("Column.size() bug in pandas")
Review Comment:
And what's the reason that the skip version is changed in this PR? (I thought the string tests were already updated in the other, merged PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on code in PR #35530:
URL: https://github.com/apache/arrow/pull/35530#discussion_r1191609651
##########
python/pyarrow/tests/interchange/test_conversion.py:
##########
@@ -189,10 +143,9 @@ def test_pandas_roundtrip(uint, int, float, np_float):
@pytest.mark.pandas
-def test_roundtrip_pandas_string():
- # See https://github.com/pandas-dev/pandas/issues/50554
- if Version(pd.__version__) < Version("1.6"):
- pytest.skip(" Column.size() called as a method in pandas 2.0.0")
+def test_pandas_roundtrip_string():
+ if Version(pd.__version__) < Version("2.0.1"):
+ pytest.skip("Column.size() bug in pandas")
Review Comment:
> As for the message - why do you think it is wrong?
I though the specific size() related bug was already fixed earlier in pandas?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #35530:
URL: https://github.com/apache/arrow/pull/35530#discussion_r1191540371
##########
python/pyarrow/tests/interchange/test_conversion.py:
##########
@@ -189,10 +143,9 @@ def test_pandas_roundtrip(uint, int, float, np_float):
@pytest.mark.pandas
-def test_roundtrip_pandas_string():
- # See https://github.com/pandas-dev/pandas/issues/50554
- if Version(pd.__version__) < Version("1.6"):
- pytest.skip(" Column.size() called as a method in pandas 2.0.0")
+def test_pandas_roundtrip_string():
+ if Version(pd.__version__) < Version("2.0.1"):
+ pytest.skip("Column.size() bug in pandas")
Review Comment:
As for the message - why do you think it is wrong?
I have found "Column.size() called as a method in pandas 2.0.0" a bit confusing so I changed to "Column.size() bug in pandas" but can keep it if the first one is clearer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] danepitkin commented on pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "danepitkin (via GitHub)" <gi...@apache.org>.
danepitkin commented on PR #35530:
URL: https://github.com/apache/arrow/pull/35530#issuecomment-1542700055
Added one comment, but besides that LGTM!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on code in PR #35530:
URL: https://github.com/apache/arrow/pull/35530#discussion_r1191199192
##########
python/pyarrow/tests/interchange/test_conversion.py:
##########
@@ -189,10 +143,9 @@ def test_pandas_roundtrip(uint, int, float, np_float):
@pytest.mark.pandas
-def test_roundtrip_pandas_string():
- # See https://github.com/pandas-dev/pandas/issues/50554
- if Version(pd.__version__) < Version("1.6"):
- pytest.skip(" Column.size() called as a method in pandas 2.0.0")
+def test_pandas_roundtrip_string():
+ if Version(pd.__version__) < Version("2.0.1"):
+ pytest.skip("Column.size() bug in pandas")
Review Comment:
this skip message is wrong?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on code in PR #35530:
URL: https://github.com/apache/arrow/pull/35530#discussion_r1191610305
##########
python/pyarrow/tests/interchange/test_conversion.py:
##########
@@ -189,10 +143,9 @@ def test_pandas_roundtrip(uint, int, float, np_float):
@pytest.mark.pandas
-def test_roundtrip_pandas_string():
- # See https://github.com/pandas-dev/pandas/issues/50554
- if Version(pd.__version__) < Version("1.6"):
- pytest.skip(" Column.size() called as a method in pandas 2.0.0")
+def test_pandas_roundtrip_string():
+ if Version(pd.__version__) < Version("2.0.1"):
+ pytest.skip("Column.size() bug in pandas")
Review Comment:
Ah, but with the latest commit the version changed again, so now it's fine ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #35530:
URL: https://github.com/apache/arrow/pull/35530#discussion_r1190834427
##########
python/pyarrow/tests/interchange/test_conversion.py:
##########
@@ -217,19 +170,28 @@ def test_roundtrip_pandas_string():
@pytest.mark.pandas
-def test_roundtrip_pandas_boolean():
- if Version(pd.__version__) < Version("1.5.0"):
- pytest.skip("__dataframe__ added to pandas in 1.5.0")
+def test_pandas_roundtrip_categorical():
+ if Version(pd.__version__) < Version("2.0.2"):
Review Comment:
Good catch! =) There are also bitmasks involved here, so I have to change the skip message 😊
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on PR #35530:
URL: https://github.com/apache/arrow/pull/35530#issuecomment-1542173678
@github-actions crossbow submit test-conda-python-*-pandas-*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] danepitkin commented on a diff in pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "danepitkin (via GitHub)" <gi...@apache.org>.
danepitkin commented on code in PR #35530:
URL: https://github.com/apache/arrow/pull/35530#discussion_r1190318537
##########
python/pyarrow/tests/interchange/test_conversion.py:
##########
@@ -217,19 +170,28 @@ def test_roundtrip_pandas_string():
@pytest.mark.pandas
-def test_roundtrip_pandas_boolean():
- if Version(pd.__version__) < Version("1.5.0"):
- pytest.skip("__dataframe__ added to pandas in 1.5.0")
+def test_pandas_roundtrip_categorical():
+ if Version(pd.__version__) < Version("2.0.2"):
Review Comment:
Should the pandas version for skipping tests due to `Column.size() bug in pandas` match between tests `test_pandas_roundtrip_string` (v2.0.1) and `test_pandas_roundtrip_categorical` (v2.0.2)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #35530: GH-35264: [Python] Interchange protocol: test clean-up
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on PR #35530:
URL: https://github.com/apache/arrow/pull/35530#issuecomment-1543616226
This needs a rebase now I merged the other PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org