You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/11/24 13:55:38 UTC
[GitHub] [arrow] milesgranger opened a new pull request, #14729: ARROW-18399: [Python] Reduce warnings during tests
milesgranger opened a new pull request, #14729:
URL: https://github.com/apache/arrow/pull/14729
Will fix [ARROW-18399](https://issues.apache.org/jira/browse/ARROW-18399)
Also closes [ARROW-17651](https://issues.apache.org/jira/browse/ARROW-17651) and [ARROW-18125](https://issues.apache.org/jira/browse/ARROW-18125)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034496102
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
Review Comment:
Yes, separate PRs sound fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034580329
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -95,9 +96,13 @@ def _check_pandas_roundtrip(df, expected=None, use_threads=False,
if expected is None:
expected = df
- tm.assert_frame_equal(result, expected, check_dtype=check_dtype,
- check_index_type=('equiv' if preserve_index
- else False))
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ with warnings.catch_warnings():
+ warnings.simplefilter("ignore", DeprecationWarning)
+ tm.assert_frame_equal(result, expected, check_dtype=check_dtype,
+ check_index_type=('equiv' if preserve_index
+ else False))
Review Comment:
Could we be more specific in which warning we are catching? (so are not catching any future deprecation warning, but only specifically the current numpy one)
(for example with `filterwarnings()` instead of `simplefilter` where we can also specify the module / message, similarly to the `pytest.mark.filterwarnings`. Or just use that mark)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #14729:
URL: https://github.com/apache/arrow/pull/14729#issuecomment-1330647304
Can you merge master to get rid of the failure?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #14729:
URL: https://github.com/apache/arrow/pull/14729#issuecomment-1330648234
Actually, it's only the label workflow (the linter passed), so don't worry
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034584905
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -182,6 +187,8 @@ def test_column_index_names_are_preserved(self):
df.columns.names = ['a']
_check_pandas_roundtrip(df, preserve_index=True)
+ @pytest.mark.filterwarnings(
+ "ignore:parsing timezone aware datetimes:DeprecationWarning")
Review Comment:
Do you remember for which pandas version this happened? (I don't see this locally when running it with pandas 1.5.1)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034593323
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -182,6 +187,8 @@ def test_column_index_names_are_preserved(self):
df.columns.names = ['a']
_check_pandas_roundtrip(df, preserve_index=True)
+ @pytest.mark.filterwarnings(
+ "ignore:parsing timezone aware datetimes:DeprecationWarning")
Review Comment:
I don't I was trying locally between a couple Python version environments when making these changes and now I too cannot get it to recreate.. so trying in CI now https://github.com/apache/arrow/pull/14729/commits/a599d65fd09579c5d1a4195a7f41f356b958d544 (same for comment about 'Discarding nonzero nanoseconds')
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #14729:
URL: https://github.com/apache/arrow/pull/14729#issuecomment-1327441609
> I assume we don't want to turn warnings into errors? Maybe that's too frustrating since unrelated warnings can always pop up in subsequent PRs?
Yes, it would definitely be frustrating.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034580329
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -95,9 +96,13 @@ def _check_pandas_roundtrip(df, expected=None, use_threads=False,
if expected is None:
expected = df
- tm.assert_frame_equal(result, expected, check_dtype=check_dtype,
- check_index_type=('equiv' if preserve_index
- else False))
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ with warnings.catch_warnings():
+ warnings.simplefilter("ignore", DeprecationWarning)
+ tm.assert_frame_equal(result, expected, check_dtype=check_dtype,
+ check_index_type=('equiv' if preserve_index
+ else False))
Review Comment:
Could we be more specific in which warning we are catching? (so are not catching any future deprecation warning, but only specifically the current numpy one)
(for example with `filterwarnings()` instead of `simplefilter` where we can also specify the module / message, similarly to the `pytest.mark.filterwarnings`. Or just use that mark)
(and same for the other occurrences below)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034584905
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -182,6 +187,8 @@ def test_column_index_names_are_preserved(self):
df.columns.names = ['a']
_check_pandas_roundtrip(df, preserve_index=True)
+ @pytest.mark.filterwarnings(
+ "ignore:parsing timezone aware datetimes:DeprecationWarning")
Review Comment:
Do you remember for which pandas version this happened?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1033617447
##########
python/pyarrow/conftest.py:
##########
@@ -265,3 +267,19 @@ def add_fs(doctest_namespace, request, tmp_path):
doctest_namespace["local_path"] = str(tmp_path)
doctest_namespace["path"] = str(path)
yield
+
+
+@pytest.fixture(autouse=True)
+def _close_socket(monkeypatch):
+ """
+ There are occasions when a PytestUnraisableExceptionWarning is
+ raised from pyx files or other __del__ impls that doesn't properly
+ call the initial caller's .close when creating sockets. Here we'll
+ ensure sockets are closed.
+ """
+ class socket_(socket.socket):
+ def __del__(self):
+ if not self._closed:
+ self.close()
Review Comment:
Yeah, this felt dirty. Commit has been dropped, I'll poke around at it a bit more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034626310
##########
python/pyarrow/tests/parquet/test_datetime.py:
##########
@@ -297,6 +297,8 @@ def test_coerce_int96_timestamp_unit(unit):
@pytest.mark.pandas
@pytest.mark.parametrize('pq_reader_method', ['ParquetFile', 'read_table'])
+@pytest.mark.filterwarnings(
+ "ignore:Discarding nonzero nanoseconds in conversion:UserWarning")
Review Comment:
But I don't directly see how this test is causing such a warning, since it uses dates that doesn't include nanoseconds
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034593323
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -182,6 +187,8 @@ def test_column_index_names_are_preserved(self):
df.columns.names = ['a']
_check_pandas_roundtrip(df, preserve_index=True)
+ @pytest.mark.filterwarnings(
+ "ignore:parsing timezone aware datetimes:DeprecationWarning")
Review Comment:
I don't I was trying locally between a couple Python version environments and I too cannot get it to recreate.. so trying in CI now https://github.com/apache/arrow/pull/14729/commits/a599d65fd09579c5d1a4195a7f41f356b958d544 (same for comment about 'Discarding nonzero nanoseconds')
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034644581
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -95,9 +96,13 @@ def _check_pandas_roundtrip(df, expected=None, use_threads=False,
if expected is None:
expected = df
- tm.assert_frame_equal(result, expected, check_dtype=check_dtype,
- check_index_type=('equiv' if preserve_index
- else False))
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ with warnings.catch_warnings():
+ warnings.simplefilter("ignore", DeprecationWarning)
+ tm.assert_frame_equal(result, expected, check_dtype=check_dtype,
+ check_index_type=('equiv' if preserve_index
+ else False))
Review Comment:
Good point. Done in https://github.com/apache/arrow/pull/14729/commits/ee97c6bdc475b6e8e78c827cfea278272189f5c8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1033577836
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
Review Comment:
These warnings are happening inside of the `pandas.testing` module though. Unless I'm misunderstanding.. should we open an issue with pandas then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034517525
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
Review Comment:
Thanks @jorisvandenbossche! Okay, I'll add back the ignoring of this error then here, and also removed saying this closed [ARROW-17651](https://issues.apache.org/jira/browse/ARROW-17651)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034645696
##########
python/pyarrow/tests/parquet/test_datetime.py:
##########
@@ -297,6 +297,8 @@ def test_coerce_int96_timestamp_unit(unit):
@pytest.mark.pandas
@pytest.mark.parametrize('pq_reader_method', ['ParquetFile', 'read_table'])
+@pytest.mark.filterwarnings(
+ "ignore:Discarding nonzero nanoseconds in conversion:UserWarning")
Review Comment:
Should I move to a `warnings.catch` over this line specifically then to make it more clear where the warning is being raised?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche merged pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche merged PR #14729:
URL: https://github.com/apache/arrow/pull/14729
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034595116
##########
python/pyarrow/tests/test_substrait.py:
##########
@@ -223,7 +224,11 @@ def table_provider(names):
assert res_tb == test_table_1
+@pytest.mark.filterwarnings("ignore::pytest.PytestUnraisableExceptionWarning")
def test_named_table_invalid_table_name():
+ # Warning generated from pyx which is unable to be propagated to caller
+ # ref: https://docs.pytest.org/en/6.2.x/usage.html#warning-about
+ # -unraisable-exceptions-and-unhandled-thread-exceptions
Review Comment:
Is this something we should actually try to fix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on PR #14729:
URL: https://github.com/apache/arrow/pull/14729#issuecomment-1327380521
This appears to remove all errors in the CI [without](https://github.com/apache/arrow/actions/runs/3547620857/jobs/5957939770#step:6:5553) and [with pandas](https://github.com/apache/arrow/actions/runs/3547620857/jobs/5957939831#step:6:5696)
I assume we don't want to turn warnings into errors? Maybe that's too frustrating since unrelated warnings can always pop up in subsequent PRs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1031593419
##########
python/pyarrow/tests/parquet/test_basic.py:
##########
@@ -617,15 +618,15 @@ def test_read_non_existent_file(tempdir, use_legacy_dataset):
@parametrize_legacy_dataset
def test_read_table_doesnt_warn(datadir, use_legacy_dataset):
- with pytest.warns(None) as record:
- pq.read_table(datadir / 'v0.7.1.parquet',
- use_legacy_dataset=use_legacy_dataset)
-
if use_legacy_dataset:
- # FutureWarning: 'use_legacy_dataset=True'
- assert len(record) == 1
+ with pytest.warns(FutureWarning):
Review Comment:
Is it possible to match some relevant part of the warning message?
##########
python/pyarrow/conftest.py:
##########
@@ -265,3 +267,19 @@ def add_fs(doctest_namespace, request, tmp_path):
doctest_namespace["local_path"] = str(tmp_path)
doctest_namespace["path"] = str(path)
yield
+
+
+@pytest.fixture(autouse=True)
+def _close_socket(monkeypatch):
+ """
+ There are occasions when a PytestUnraisableExceptionWarning is
+ raised from pyx files or other __del__ impls that doesn't properly
+ call the initial caller's .close when creating sockets. Here we'll
+ ensure sockets are closed.
+ """
+ class socket_(socket.socket):
+ def __del__(self):
+ if not self._closed:
+ self.close()
Review Comment:
We definitely don't want to monkeypatch a core type just to avoid some warnings.
If some of our routines fail to close a socket we should try to improve them. Otherwise we should just live with the warnings.
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
Review Comment:
Since these are flagging behavior that is likely to change in the future, shouldn't we fix the underlying issues instead of simply silencing the warnings?
##########
python/pyarrow/parquet/core.py:
##########
@@ -1165,8 +1165,10 @@ def get_metadata(self):
-------
metadata : FileMetaData
"""
- with self.open() as parquet:
- return parquet.metadata
+ parquet = self.open()
+ meta = parquet.metadata
+ parquet.close(True)
Review Comment:
Hmm, isn't it a bug if a `with` statement fails to close the file it just opened?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1033581657
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
Review Comment:
Ah! Yes, perhaps open an issue with them, in case there's a workaround?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #14729:
URL: https://github.com/apache/arrow/pull/14729#issuecomment-1329102158
(sorry, some of those comments were actually written days ago, but I had forgotten to submit the review...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1033966041
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
Review Comment:
Hmm, actually, when making an example for the issue to report, I _think_ it maybe how our implementation of converting a variable list array to pandas:
```python
import pyarrow as pa
import pandas as pd
import pandas.testing as tm
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
# This will raise the warning
tm.assert_series_equal(
s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
check_names=False)
# but so will this:
s.equals(pd.Series([[[1, 2, 3], [4]], None], dtype=object))
# this will not
tm.assert_series_equal(
pd.Series([[[1, 2, 3], [4]], None], dtype=object),
pd.Series([[[1, 2, 3], [4]], None], dtype=object),
check_names=False)
```
Therefore I think a new issue to be made to correct this on our end after all. :thinking:
---
Edit: Dropped the commits relating to suppressing `DeprecationWarning`s in `test_pandas.py`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1033966041
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
Review Comment:
Hmm, actually, when making an example for the issue to report, I _think_ it maybe how our implementation of converting a variable list array to pandas:
```python
import pyarrow as pa
import pandas as pd
import pandas.testing as tm
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
# This will raise the warning
tm.assert_series_equal(
s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
check_names=False)
# but so will this:
s.equals(pd.Series([[[1, 2, 3], [4]], None], dtype=object))
# this will not
tm.assert_series_equal(
pd.Series([[[1, 2, 3], [4]], None], dtype=object),
pd.Series([[[1, 2, 3], [4]], None], dtype=object),
check_names=False)
```
Therefore I think a new issue to be made to correct this on our end after all. :thinking:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034694234
##########
python/pyarrow/tests/parquet/test_datetime.py:
##########
@@ -297,6 +297,8 @@ def test_coerce_int96_timestamp_unit(unit):
@pytest.mark.pandas
@pytest.mark.parametrize('pq_reader_method', ['ParquetFile', 'read_table'])
+@pytest.mark.filterwarnings(
+ "ignore:Discarding nonzero nanoseconds in conversion:UserWarning")
Review Comment:
https://github.com/apache/arrow/pull/14729/commits/ab4a31a4a1dbca63f8e35eb3553f8b43f9d0f71a
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1033620156
##########
python/pyarrow/parquet/core.py:
##########
@@ -1165,8 +1165,10 @@ def get_metadata(self):
-------
metadata : FileMetaData
"""
- with self.open() as parquet:
- return parquet.metadata
+ parquet = self.open()
+ meta = parquet.metadata
+ parquet.close(True)
Review Comment:
It's slightly awkward since ParquetFile distinguishes between opening and closing based on if it was the one who opened it. Indeed, `open()` opens the source for `ParquetFile` and thus should probably mark it as being responsible for closing. :+1: https://github.com/apache/arrow/pull/14729/commits/e845fc85c99ed3e484a320b48234c0dcf861a19e
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034569478
##########
python/pyarrow/tests/parquet/test_datetime.py:
##########
@@ -297,6 +297,8 @@ def test_coerce_int96_timestamp_unit(unit):
@pytest.mark.pandas
@pytest.mark.parametrize('pq_reader_method', ['ParquetFile', 'read_table'])
+@pytest.mark.filterwarnings(
+ "ignore:Discarding nonzero nanoseconds in conversion:UserWarning")
Review Comment:
Where is this warning coming from?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034627665
##########
python/pyarrow/tests/parquet/test_datetime.py:
##########
@@ -297,6 +297,8 @@ def test_coerce_int96_timestamp_unit(unit):
@pytest.mark.pandas
@pytest.mark.parametrize('pq_reader_method', ['ParquetFile', 'read_table'])
+@pytest.mark.filterwarnings(
+ "ignore:Discarding nonzero nanoseconds in conversion:UserWarning")
Review Comment:
Ah, or maybe from this line:
```
# with the default resolution of ns, we get wrong values for INT96
# that are out of bounds for nanosecond range
tab_error = get_table(pq_reader_method, filename)
assert tab_error["a"].to_pylist() != oob_dts
```
since you get "wrong" values, those have overflown, and the resulting wrong datetime will probably have nanoseconds, and thus can raise this warning. That should be OK to ignore then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034610079
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -182,6 +187,8 @@ def test_column_index_names_are_preserved(self):
df.columns.names = ['a']
_check_pandas_roundtrip(df, preserve_index=True)
+ @pytest.mark.filterwarnings(
+ "ignore:parsing timezone aware datetimes:DeprecationWarning")
Review Comment:
Okay, well it's not in CI that I can see.. not sure how I managed to get that. will remove the filter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034629424
##########
python/pyarrow/tests/test_substrait.py:
##########
@@ -223,7 +224,11 @@ def table_provider(names):
assert res_tb == test_table_1
+@pytest.mark.filterwarnings("ignore::pytest.PytestUnraisableExceptionWarning")
def test_named_table_invalid_table_name():
+ # Warning generated from pyx which is unable to be propagated to caller
+ # ref: https://docs.pytest.org/en/6.2.x/usage.html#warning-about
+ # -unraisable-exceptions-and-unhandled-thread-exceptions
Review Comment:
https://github.com/apache/arrow/issues/14766
##########
python/pyarrow/tests/test_substrait.py:
##########
@@ -223,7 +224,11 @@ def table_provider(names):
assert res_tb == test_table_1
+@pytest.mark.filterwarnings("ignore::pytest.PytestUnraisableExceptionWarning")
def test_named_table_invalid_table_name():
+ # Warning generated from pyx which is unable to be propagated to caller
+ # ref: https://docs.pytest.org/en/6.2.x/usage.html#warning-about
+ # -unraisable-exceptions-and-unhandled-thread-exceptions
Review Comment:
- https://github.com/apache/arrow/issues/14766
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034598573
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -182,6 +187,8 @@ def test_column_index_names_are_preserved(self):
df.columns.names = ['a']
_check_pandas_roundtrip(df, preserve_index=True)
+ @pytest.mark.filterwarnings(
+ "ignore:parsing timezone aware datetimes:DeprecationWarning")
Review Comment:
Yeah, it might be that bumping the minimum pandas version to 1.0 in the meantime has made that disappear already
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034661402
##########
python/pyarrow/tests/parquet/test_datetime.py:
##########
@@ -297,6 +297,8 @@ def test_coerce_int96_timestamp_unit(unit):
@pytest.mark.pandas
@pytest.mark.parametrize('pq_reader_method', ['ParquetFile', 'read_table'])
+@pytest.mark.filterwarnings(
+ "ignore:Discarding nonzero nanoseconds in conversion:UserWarning")
Review Comment:
> Should I move to a `warnings.catch` over this line specifically then to make it more clear where the warning is being raised?
That's probably most specific indeed (and also documents the root cause as well)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034457797
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
Review Comment:
For now, I've made issues
- https://github.com/apache/arrow/issues/14759
- https://github.com/apache/arrow/issues/14760.
I can create separate PRs for them I think if that's agreeable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034624379
##########
python/pyarrow/tests/parquet/test_datetime.py:
##########
@@ -297,6 +297,8 @@ def test_coerce_int96_timestamp_unit(unit):
@pytest.mark.pandas
@pytest.mark.parametrize('pq_reader_method', ['ParquetFile', 'read_table'])
+@pytest.mark.filterwarnings(
+ "ignore:Discarding nonzero nanoseconds in conversion:UserWarning")
Review Comment:
Searching for this phrase, pandas raises this warning when you convert a Timestamp object to a python datetime.datetime object:
```
In [22]: pd.Timestamp("2022-01-01 01:01:01.123456789").to_pydatetime()
<ipython-input-22-7cef94b12f00>:1: UserWarning: Discarding nonzero nanoseconds in conversion.
pd.Timestamp("2022-01-01 01:01:01.123456789").to_pydatetime()
Out[22]: datetime.datetime(2022, 1, 1, 1, 1, 1, 123456)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #14729:
URL: https://github.com/apache/arrow/pull/14729#issuecomment-1332320305
['Python', 'R'] benchmarks have high level of regressions.
[ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/4a05187130564eb4a0dd49de0d32bd3a...4a4e598aff8d4f599ed0753a830d0575/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #14729:
URL: https://github.com/apache/arrow/pull/14729#issuecomment-1326486660
https://issues.apache.org/jira/browse/ARROW-18399
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1033548908
##########
python/pyarrow/conftest.py:
##########
@@ -265,3 +267,19 @@ def add_fs(doctest_namespace, request, tmp_path):
doctest_namespace["local_path"] = str(tmp_path)
doctest_namespace["path"] = str(path)
yield
+
+
+@pytest.fixture(autouse=True)
+def _close_socket(monkeypatch):
+ """
+ There are occasions when a PytestUnraisableExceptionWarning is
+ raised from pyx files or other __del__ impls that doesn't properly
+ call the initial caller's .close when creating sockets. Here we'll
+ ensure sockets are closed.
+ """
+ class socket_(socket.socket):
+ def __del__(self):
+ if not self._closed:
+ self.close()
Review Comment:
We definitely don't want to monkeypatch a core type just to avoid some warnings.
If some of our routines fail to close a socket we should try to improve them. Otherwise we should just live with the warnings, or silence them selectively.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034619525
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
+ with warnings.catch_warnings():
+ warnings.simplefilter("ignore", np.VisibleDeprecationWarning)
Review Comment:
Given [our discussion about numpy 1.24](https://github.com/apache/arrow/issues/14759#issuecomment-1330436673) should I remove this warning since it'll be going away?
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
+ with warnings.catch_warnings():
+ warnings.simplefilter("ignore", np.VisibleDeprecationWarning)
Review Comment:
@jorisvandenbossche Given [our discussion about numpy 1.24](https://github.com/apache/arrow/issues/14759#issuecomment-1330436673) should I remove this warning since it'll be going away?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034622876
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
+ with warnings.catch_warnings():
+ warnings.simplefilter("ignore", np.VisibleDeprecationWarning)
Review Comment:
I would still catch/ignore it for now, since we will have test builds for some time that have older numpy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034506461
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -2164,9 +2172,17 @@ def test_nested_large_list(self):
s = (pa.array([[[1, 2, 3], [4]], None],
type=pa.large_list(pa.large_list(pa.int64())))
.to_pandas())
- tm.assert_series_equal(
- s, pd.Series([[[1, 2, 3], [4]], None], dtype=object),
- check_names=False)
+
+ # pandas.testing generates a
+ # DeprecationWarning: elementwise comparison failed
+ # numpy.VisibleDeprecationWarning: Creating an ndarray
+ # from ragged nested sequences ...
Review Comment:
See my answer in https://github.com/apache/arrow/issues/14759, for the warning coming from pandas' testing, I think we can ignore that warning on our side (and maybe report it to pandas)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034608925
##########
python/pyarrow/tests/test_substrait.py:
##########
@@ -223,7 +224,11 @@ def table_provider(names):
assert res_tb == test_table_1
+@pytest.mark.filterwarnings("ignore::pytest.PytestUnraisableExceptionWarning")
def test_named_table_invalid_table_name():
+ # Warning generated from pyx which is unable to be propagated to caller
+ # ref: https://docs.pytest.org/en/6.2.x/usage.html#warning-about
+ # -unraisable-exceptions-and-unhandled-thread-exceptions
Review Comment:
The ignored exception is:
```
Traceback (most recent call last):
File "/home/joris/scipy/repos/arrow/python/pyarrow/tests/test_substrait.py", line 238, in table_provider
raise Exception("Unrecognized table name")
Exception: Unrecognized table name
```
which is IMO something we should fix instead (the error should be raised to the user)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034607305
##########
python/pyarrow/tests/parquet/test_datetime.py:
##########
@@ -297,6 +297,8 @@ def test_coerce_int96_timestamp_unit(unit):
@pytest.mark.pandas
@pytest.mark.parametrize('pq_reader_method', ['ParquetFile', 'read_table'])
+@pytest.mark.filterwarnings(
+ "ignore:Discarding nonzero nanoseconds in conversion:UserWarning")
Review Comment:
Haivng a hard time getting it to show up agaiin locally, but it's iin CI: https://github.com/milesgranger/arrow/actions/runs/3573348441/jobs/6007303891#step:6:5679
I _think_ it's coming from `tab_correct.to_pandas` call. But not sure, will try more to get it to happen locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] milesgranger commented on a diff in pull request #14729: ARROW-18399: [Python] Reduce warnings during tests
Posted by GitBox <gi...@apache.org>.
milesgranger commented on code in PR #14729:
URL: https://github.com/apache/arrow/pull/14729#discussion_r1034617019
##########
python/pyarrow/tests/test_substrait.py:
##########
@@ -223,7 +224,11 @@ def table_provider(names):
assert res_tb == test_table_1
+@pytest.mark.filterwarnings("ignore::pytest.PytestUnraisableExceptionWarning")
def test_named_table_invalid_table_name():
+ # Warning generated from pyx which is unable to be propagated to caller
+ # ref: https://docs.pytest.org/en/6.2.x/usage.html#warning-about
+ # -unraisable-exceptions-and-unhandled-thread-exceptions
Review Comment:
That seems prudent. I'll make an issue for it separately, and have removed this filter in https://github.com/apache/arrow/pull/14729/commits/20b1a30622ecf722783c1f0485d5f37c110a8deb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org