You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "AlenkaF (via GitHub)" <gi...@apache.org> on 2024/01/15 14:38:58 UTC
[PR] GH-36412: Fix estra deprecation warnings in the pandas nightly build [arrow]
AlenkaF opened a new pull request, #39609:
URL: https://github.com/apache/arrow/pull/39609
Fixes left deprecation warnings coming from the pandas development version, by updating our test code to avoid the deprecated patterns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1893635304
Revision: 3adf04e8ed8f5875e51fd3807a6ac99e084c1877
Submitted crossbow builds: [ursacomputing/crossbow @ actions-5666ccb6e4](https://github.com/ursacomputing/crossbow/branches/all?query=actions-5666ccb6e4)
|Task|Status|
|----|------|
|test-conda-python-3.10-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions/runs/7541479487/job/20528289857)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1893208566
@github-actions crossbow submit test-conda-python-3.10-pandas-nightly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on code in PR #39609:
URL: https://github.com/apache/arrow/pull/39609#discussion_r1453085620
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -4312,10 +4314,13 @@ def test_array_to_pandas():
def test_roundtrip_empty_table_with_extension_dtype_index():
df = pd.DataFrame(index=pd.interval_range(start=0, end=3))
table = pa.table(df)
- table.to_pandas().index == pd.Index([{'left': 0, 'right': 1},
- {'left': 1, 'right': 2},
- {'left': 2, 'right': 3}],
- dtype='object')
+ if Version(pd.__version__) > Version("1.0"):
+ tm.assert_index_equal(table.to_pandas().index, df.index)
+ else:
+ assert table.to_pandas().index == pd.Index([{'left': 0, 'right': 1},
Review Comment:
Ah, sorry I missed the > (_strictly_ greater), not >=, ignore me then ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1893708994
@github-actions crossbow submit test-conda-python-3.10-pandas-nightly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1892380015
Revision: 0d7edf38b1735ff3306c2a8bde99b6043ce7add8
Submitted crossbow builds: [ursacomputing/crossbow @ actions-f2bde2fa12](https://github.com/ursacomputing/crossbow/branches/all?query=actions-f2bde2fa12)
|Task|Status|
|----|------|
|test-conda-python-3.10-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions/runs/7530964571/job/20498473323)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #39609:
URL: https://github.com/apache/arrow/pull/39609#discussion_r1453370816
##########
python/pyarrow/pandas_compat.py:
##########
@@ -950,7 +950,7 @@ def _reconstruct_index(table, index_descriptors, all_columns, types_mapper=None)
index = index_arrays[0]
if not isinstance(index, pd.Index):
# Box anything that wasn't boxed above
- index = pd.Index(index, name=index_names[0])
+ index = pd.Index(index.infer_objects(), name=index_names[0])
Review Comment:
https://github.com/apache/arrow/pull/39609/commits/01980c1baa76786f6bbae73d16ec04918c4b502b
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1893211017
Revision: 09f2f509b041732efa8c1520e683cc1273e2356b
Submitted crossbow builds: [ursacomputing/crossbow @ actions-3a8abeda7e](https://github.com/ursacomputing/crossbow/branches/all?query=actions-3a8abeda7e)
|Task|Status|
|----|------|
|test-conda-python-3.10-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions/runs/7538435838/job/20518961828)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1892376479
@github-actions crossbow submit test-conda-python-3.10-pandas-nightly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1893712697
Revision: 01980c1baa76786f6bbae73d16ec04918c4b502b
Submitted crossbow builds: [ursacomputing/crossbow @ actions-db434e524e](https://github.com/ursacomputing/crossbow/branches/all?query=actions-db434e524e)
|Task|Status|
|----|------|
|test-conda-python-3.10-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions/runs/7542048908/job/20530018141)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #39609:
URL: https://github.com/apache/arrow/pull/39609#discussion_r1453080124
##########
python/pyarrow/pandas_compat.py:
##########
@@ -950,7 +950,7 @@ def _reconstruct_index(table, index_descriptors, all_columns, types_mapper=None)
index = index_arrays[0]
if not isinstance(index, pd.Index):
# Box anything that wasn't boxed above
- index = pd.Index(index, name=index_names[0])
+ index = pd.Index(index.infer_objects(), name=index_names[0])
Review Comment:
It is `test_roundtrip_empty_table_with_extension_dtype_index`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1893631138
@github-actions crossbow submit test-conda-python-3.10-pandas-nightly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on code in PR #39609:
URL: https://github.com/apache/arrow/pull/39609#discussion_r1453060849
##########
python/pyarrow/pandas_compat.py:
##########
@@ -950,7 +950,7 @@ def _reconstruct_index(table, index_descriptors, all_columns, types_mapper=None)
index = index_arrays[0]
if not isinstance(index, pd.Index):
# Box anything that wasn't boxed above
- index = pd.Index(index, name=index_names[0])
+ index = pd.Index(index.infer_objects(), name=index_names[0])
Review Comment:
Do you know for which test this is needed? (I mean, which test triggers the warning here)
I am wondering what `index` would be then, because I would think we always already created a proper pandas object.
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -4312,10 +4314,13 @@ def test_array_to_pandas():
def test_roundtrip_empty_table_with_extension_dtype_index():
df = pd.DataFrame(index=pd.interval_range(start=0, end=3))
table = pa.table(df)
- table.to_pandas().index == pd.Index([{'left': 0, 'right': 1},
- {'left': 1, 'right': 2},
- {'left': 2, 'right': 3}],
- dtype='object')
+ if Version(pd.__version__) > Version("1.0"):
+ tm.assert_index_equal(table.to_pandas().index, df.index)
+ else:
+ assert table.to_pandas().index == pd.Index([{'left': 0, 'right': 1},
Review Comment:
I don't think we still support older pandas versions? (https://github.com/apache/arrow/pull/14631) So then this can be simplified
##########
python/pyarrow/tests/test_dataset.py:
##########
@@ -178,12 +178,14 @@ def multisourcefs(request):
# simply split the dataframe into four chunks to construct a data source
# from each chunk into its own directory
- df_a, df_b, df_c, df_d = np.array_split(df, 4)
+ n = len(df)
+ df_a, df_b, df_c, df_d = [df[i:i+n//4] for i in range(0, n, n//4)]
# create a directory containing a flat sequence of parquet files without
# any partitioning involved
mockfs.create_dir('plain')
- for i, chunk in enumerate(np.array_split(df_a, 10)):
+ n = len(df_a)
+ for i, chunk in enumerate([df_a[i:i+n//10] for i in range(0, n, n//10)]):
Review Comment:
```suggestion
for i, chunk in enumerate([df_a.iloc[i:i+n//10] for i in range(0, n, n//10)]):
```
##########
python/pyarrow/tests/test_dataset.py:
##########
@@ -178,12 +178,14 @@ def multisourcefs(request):
# simply split the dataframe into four chunks to construct a data source
# from each chunk into its own directory
- df_a, df_b, df_c, df_d = np.array_split(df, 4)
+ n = len(df)
+ df_a, df_b, df_c, df_d = [df[i:i+n//4] for i in range(0, n, n//4)]
Review Comment:
```suggestion
df_a, df_b, df_c, df_d = [df.iloc[i:i+n//4] for i in range(0, n, n//4)]
```
(should be exactly the same, but is a bit more explicit that it is doing positional indexing)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1892308355
Revision: 7230b6f6d01318369999f82633a5b3f7eb603177
Submitted crossbow builds: [ursacomputing/crossbow @ actions-40718f16dc](https://github.com/ursacomputing/crossbow/branches/all?query=actions-40718f16dc)
|Task|Status|
|----|------|
|test-conda-python-3.10-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions/runs/7530446666/job/20496845550)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1892304502
@github-actions crossbow submit test-conda-python-3.10-pandas-nightly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on code in PR #39609:
URL: https://github.com/apache/arrow/pull/39609#discussion_r1453093512
##########
python/pyarrow/pandas_compat.py:
##########
@@ -950,7 +950,7 @@ def _reconstruct_index(table, index_descriptors, all_columns, types_mapper=None)
index = index_arrays[0]
if not isinstance(index, pd.Index):
# Box anything that wasn't boxed above
- index = pd.Index(index, name=index_names[0])
+ index = pd.Index(index.infer_objects(), name=index_names[0])
Review Comment:
OK, so putting a breakpoint at the above location and then running that test, I see:
```
(Pdb) index
0 (0, 1]
1 (1, 2]
2 (2, 3]
dtype: object
(Pdb) index.values
array([Interval(0, 1, closed='right'), Interval(1, 2, closed='right'),
Interval(2, 3, closed='right')], dtype=object)
(Pdb) pd.Index(index, name=index_names[0])
IntervalIndex([(0, 1], (1, 2], (2, 3]], dtype='interval[int64, right]')
```
So `index` is an object-dtype Series with scalar Interval objects, and then converting that to an Index on the line above currently infers that it are interval objects, and converts that to interval dtype, while in the future it will preserve the object dtype.
Now, I do wonder why we have an object dtype series of interval objects, though. Because we created that data (converted from a column in the pyarrow table), so I would expect that we create proper interval dtype to start with.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1896078808
After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit 6eeee3b769f3ad6c724305b4182526307ab025d5.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/20579143438) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #39609:
URL: https://github.com/apache/arrow/pull/39609#discussion_r1453119674
##########
python/pyarrow/pandas_compat.py:
##########
@@ -950,7 +950,7 @@ def _reconstruct_index(table, index_descriptors, all_columns, types_mapper=None)
index = index_arrays[0]
if not isinstance(index, pd.Index):
# Box anything that wasn't boxed above
- index = pd.Index(index, name=index_names[0])
+ index = pd.Index(index.infer_objects(), name=index_names[0])
Review Comment:
I think we create a series here: https://github.com/apache/arrow/blob/ac50918a63f13088a5ea9e6c27a9fadbbb19d53f/python/pyarrow/pandas_compat.py#L983
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on code in PR #39609:
URL: https://github.com/apache/arrow/pull/39609#discussion_r1455064897
##########
python/pyarrow/pandas_compat.py:
##########
@@ -950,7 +950,7 @@ def _reconstruct_index(table, index_descriptors, all_columns, types_mapper=None)
index = index_arrays[0]
if not isinstance(index, pd.Index):
# Box anything that wasn't boxed above
- index = pd.Index(index, name=index_names[0])
+ index = pd.Index(index.infer_objects(), name=index_names[0])
Review Comment:
Nice cleanup!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #39609:
URL: https://github.com/apache/arrow/pull/39609#discussion_r1453068051
##########
python/pyarrow/tests/test_pandas.py:
##########
@@ -4312,10 +4314,13 @@ def test_array_to_pandas():
def test_roundtrip_empty_table_with_extension_dtype_index():
df = pd.DataFrame(index=pd.interval_range(start=0, end=3))
table = pa.table(df)
- table.to_pandas().index == pd.Index([{'left': 0, 'right': 1},
- {'left': 1, 'right': 2},
- {'left': 2, 'right': 3}],
- dtype='object')
+ if Version(pd.__version__) > Version("1.0"):
+ tm.assert_index_equal(table.to_pandas().index, df.index)
+ else:
+ assert table.to_pandas().index == pd.Index([{'left': 0, 'right': 1},
Review Comment:
It was failing on pandas 1.0 🤷♀️
https://github.com/apache/arrow/actions/runs/7530944983/job/20498420418#step:6:6647
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on code in PR #39609:
URL: https://github.com/apache/arrow/pull/39609#discussion_r1453098551
##########
python/pyarrow/pandas_compat.py:
##########
@@ -950,7 +950,7 @@ def _reconstruct_index(table, index_descriptors, all_columns, types_mapper=None)
index = index_arrays[0]
if not isinstance(index, pd.Index):
# Box anything that wasn't boxed above
- index = pd.Index(index, name=index_names[0])
+ index = pd.Index(index.infer_objects(), name=index_names[0])
Review Comment:
The index object is created in `_extract_index_level`, and it seems that for interval dtype, the `.values` call is causing conversion to object dtype:
```
(Pdb) l
971 return result_table, None, None
972
973 pd = _pandas_api.pd
974
975 col = table.column(i)
976 -> values = col.to_pandas(types_mapper=types_mapper).values
977
978 if hasattr(values, 'flags') and not values.flags.writeable:
979 # ARROW-1054: in pandas 0.19.2, factorize will reject
980 # non-writeable arrays when calling MultiIndex.from_arrays
981 values = values.copy()
(Pdb) col.to_pandas()
0 (0, 1]
1 (1, 2]
2 (2, 3]
Name: __index_level_0__, dtype: interval
(Pdb) col.to_pandas().values
array([Interval(0, 1, closed='right'), Interval(1, 2, closed='right'),
Interval(2, 3, closed='right')], dtype=object)
```
The reason we do the `.values` is because we check whether the data is writeable for old pandas versions. This if block can nowadays maybe be removed, and then we don't need to conversion of a Series to an array.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #39609:
URL: https://github.com/apache/arrow/pull/39609#discussion_r1453351154
##########
python/pyarrow/pandas_compat.py:
##########
@@ -950,7 +950,7 @@ def _reconstruct_index(table, index_descriptors, all_columns, types_mapper=None)
index = index_arrays[0]
if not isinstance(index, pd.Index):
# Box anything that wasn't boxed above
- index = pd.Index(index, name=index_names[0])
+ index = pd.Index(index.infer_objects(), name=index_names[0])
Review Comment:
Sorry, I have missed your last comment (the page didn't refresh because I was writing at the same time, I guess). Will remove the if block and check if it works 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on PR #39609:
URL: https://github.com/apache/arrow/pull/39609#issuecomment-1893736540
I think this PR looks good now.
The only missing thing I have to solve is this warning about comparison with empty arrays (coming from `pandas.testing.assert_frame_equal`):
```
=============================== warnings summary ===============================
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_pandas.py::TestConvertMetadata::test_empty_list_metadata
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_pandas.py::TestConvertListTypes::test_empty_list_roundtrip
/opt/conda/envs/arrow/lib/python3.10/site-packages/pandas/core/dtypes/missing.py:578: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.
return lib.array_equivalent_object(left, right)
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_pandas.py: 2 warnings
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/parquet/test_data_types.py: 3 warnings
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/parquet/test_pandas.py: 2 warnings
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/parquet/test_parquet_file.py: 6 warnings
/opt/conda/envs/arrow/lib/python3.10/site-packages/pandas/core/dtypes/missing.py:579: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.
if not lib.array_equivalent_object(left[~mask], right[~mask]):
```
But I would do this in a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-36412: [Python][CI] Fix extra deprecation warnings in the pandas nightly build [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche merged PR #39609:
URL: https://github.com/apache/arrow/pull/39609
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org