You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "Spenserrrr (via GitHub)" <gi...@apache.org> on 2026/08/11 23:15:10 UTC
[PR] [SPARK-58720][PYTHON][TESTS] Add tests for pa.Table.to_pandas [spark]
Spenserrrr opened a new pull request, #57939:
URL: https://github.com/apache/spark/pull/57939
### What changes were proposed in this pull request?
Adds golden-file drift tests for `pa.Table.to_pandas()` under the SPARK-54936 umbrella (monitoring upstream PyArrow / pandas behavior). New test file `python/pyspark/tests/upstream/pyarrow/test_pyarrow_table_to_pandas.py` with a shared testless base and two test classes:
- `PyArrowTableToPandasDefaultTests` — pins multi-column DataFrame assembly and the empty-table edges (0-column and 0-row) under a bare `to_pandas()`.
- `PyArrowTableToPandasCoerceTemporalTests` — pins `to_pandas(coerce_temporal_nanoseconds=True)` with `date_as_object` at its default (True) and with `date_as_object=False`, plus timestamp and duration overflow rows.
Because `Table.to_pandas()` returns a `DataFrame` (whereas `Array`/`ChunkedArray.to_pandas()` return a `Series`, already covered by `test_pyarrow_arrow_to_pandas_{default,non_default}.py`), two small helpers were added to `python/pyspark/testing/goldenutils.py`:
- a `pa.Table` branch in `repr_value` (`repr_arrow_table_value`) to render the input-table anchor column;
- `repr_pandas_value` now formats a DataFrame per-column via `tolist()` instead of `to_json()` — this is overflow-safe for out-of-range datetimes (e.g. a year-9999 date) and more readable. No existing golden produced a DataFrame, so no committed golden is affected by this change.
Table-specific coverage rationale: a Table's per-column conversion is byte-identical to the Array path (verified across all 121 inventory types), so these tests do not re-enumerate per-type conversion. They target what only a Table exercises — multi-column DataFrame assembly, the empty-table edges, and temporal coercion in an assembly context. Spark reaches `Table.to_pandas()` at `python/pyspark/sql/pandas/conversion.py` (the 0-column early return) and, in Spark Connect, at `python/pyspark/sql/connect/client/core.py` (a bare whole-Table conversion of a SQL command result).
### Why are the changes needed?
Part of SPARK-54936, which pins upstream PyArrow / pandas conversion behavior so that a library version bump changing it fails loudly in CI instead of silently returning wrong data. `pa.Table.to_pandas` is the remaining conversion in this family with no golden coverage. Writing the tests also surfaced a real cross-version divergence worth pinning: on pandas 3 the Table path converts string columns to the dedicated `str` dtype starting at PyArrow 19 (uniformly for empty, all-null, and populated columns), whereas the Array path switches its empty string columns only at PyArrow 24.
### Does this PR introduce _any_ user-facing change?
No. This is test-only, plus test-infrastructure helpers in `goldenutils.py`.
### How was this patch tested?
New golden tests. The goldens were generated on pandas 2.3.3 / PyArrow 24.0.0 and validated across a PyArrow 18-25 x pandas 2/3 sweep (16 combinations), each in a fresh virtualenv running the committed test files against the committed golden files. Version-legitimate differences are recorded as `LooseVersion`-guarded entries in the `overrides` dict rather than by regenerating. The sweep covers Linux/x86 only, so a platform-specific follow-up (e.g. ARM) may be needed once CI runs the full matrix.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58720][PYTHON][TESTS] Add tests for pa.Table.to_pandas [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #57939: [SPARK-58720][PYTHON][TESTS] Add tests for pa.Table.to_pandas
URL: https://github.com/apache/spark/pull/57939
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58720][PYTHON][TESTS] Add tests for pa.Table.to_pandas [spark]
Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang commented on code in PR #57939:
URL: https://github.com/apache/spark/pull/57939#discussion_r3764718942
##########
python/pyspark/tests/upstream/pyarrow/test_pyarrow_table_to_pandas.py:
##########
@@ -0,0 +1,340 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""
+Tests for PyArrow Table.to_pandas() using golden file comparison.
+
+Unlike Array/ChunkedArray.to_pandas() (which returns a Series and is covered by
+test_pyarrow_arrow_to_pandas_{default,non_default}.py), Table.to_pandas() returns a
+DataFrame. Its per-column conversion matches the Array tests, so this file pins the
+genuinely Table-specific behavior instead: multi-column DataFrame assembly and the
+empty-table edges (0 columns / 0 rows). Spark calls Table.to_pandas() at
+python/pyspark/sql/pandas/conversion.py:255 (the 0-column path) and, in Spark Connect,
+at python/pyspark/sql/connect/client/core.py:1423 (a bare whole-Table conversion).
+
+## Golden File Cell Format
+
+Each cell uses the value@type format:
+- pyarrow Table: "{col: [val1, val2, None], ...}@Table[name: type, ...]"
+- pandas DataFrame: "{col: [values], ...}@Dataframe[name dtype, ...]"
+- Error: "ERR@ExceptionClassName"
+
+Values are formatted via tolist() for stable, Python-native representation.
+
+## Regenerating Golden Files
+
+Set SPARK_GENERATE_GOLDEN_FILES=1 before running:
+
+ SPARK_GENERATE_GOLDEN_FILES=1 python -m pytest \\
+ python/pyspark/tests/upstream/pyarrow/test_pyarrow_table_to_pandas.py
+"""
+
+import datetime
+import unittest
+
+from pyspark.loose_version import LooseVersion
+from pyspark.testing.utils import (
+ have_pyarrow,
+ have_pandas,
+ pyarrow_requirement_message,
+ pandas_requirement_message,
+)
+from pyspark.testing.goldenutils import GoldenFileTestMixin
+
+if have_pandas:
+ import pandas as pd
+if have_pyarrow:
+ import pyarrow as pa
+
+
+class _PyArrowTableToPandasTestBase(GoldenFileTestMixin, unittest.TestCase):
+ """
+ Shared machinery for pa.Table.to_pandas() golden file tests.
+
+ Holds the conversion helper and the source-table inventory, split into group
+ methods that these and the (temporal-flag) tests reuse. Defines no ``test_*`` of
+ its own.
+ """
+
+ @staticmethod
+ def _repr_dataframe(df) -> str:
+ """
+ Format a pandas DataFrame result as a golden-file cell: per-column tolist()
+ for a stable, Python-native representation, mirroring goldenutils'
+ ``repr_pandas_series_value`` for the Series case.
+
+ This deliberately does NOT go through ``repr_value``/``repr_pandas_value``
+ (which use ``to_json``). With the default date_as_object=True a far-future
+ date (year 9999) comes back as an object column of Python ``datetime.date``
+ objects, and ``DataFrame.to_json`` overflows converting each to an epoch
+ nanosecond int (OverflowError); tolist() returns the objects as-is, so
+ nothing overflows.
+ """
+ body = str({name: col.tolist() for name, col in df.items()}).replace("\n", " ")
+ schema = ", ".join(f"{t} {d.name}" for t, d in df.dtypes.items())
+ return f"{body}@Dataframe[{schema}]"
Review Comment:
nit: not sure if we will have more golden file on table/dataframe, those can be moved to GoldenFileTestMixin for future reuse. Can also defer to later when we actually need to reuse it.
##########
python/pyspark/tests/upstream/pyarrow/test_pyarrow_table_to_pandas.py:
##########
@@ -0,0 +1,340 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""
+Tests for PyArrow Table.to_pandas() using golden file comparison.
+
+Unlike Array/ChunkedArray.to_pandas() (which returns a Series and is covered by
+test_pyarrow_arrow_to_pandas_{default,non_default}.py), Table.to_pandas() returns a
+DataFrame. Its per-column conversion matches the Array tests, so this file pins the
+genuinely Table-specific behavior instead: multi-column DataFrame assembly and the
+empty-table edges (0 columns / 0 rows). Spark calls Table.to_pandas() at
+python/pyspark/sql/pandas/conversion.py:255 (the 0-column path) and, in Spark Connect,
+at python/pyspark/sql/connect/client/core.py:1423 (a bare whole-Table conversion).
+
+## Golden File Cell Format
+
+Each cell uses the value@type format:
+- pyarrow Table: "{col: [val1, val2, None], ...}@Table[name: type, ...]"
+- pandas DataFrame: "{col: [values], ...}@Dataframe[name dtype, ...]"
+- Error: "ERR@ExceptionClassName"
+
+Values are formatted via tolist() for stable, Python-native representation.
+
+## Regenerating Golden Files
+
+Set SPARK_GENERATE_GOLDEN_FILES=1 before running:
+
+ SPARK_GENERATE_GOLDEN_FILES=1 python -m pytest \\
+ python/pyspark/tests/upstream/pyarrow/test_pyarrow_table_to_pandas.py
+"""
+
+import datetime
+import unittest
+
+from pyspark.loose_version import LooseVersion
+from pyspark.testing.utils import (
+ have_pyarrow,
+ have_pandas,
+ pyarrow_requirement_message,
+ pandas_requirement_message,
+)
+from pyspark.testing.goldenutils import GoldenFileTestMixin
+
+if have_pandas:
+ import pandas as pd
+if have_pyarrow:
+ import pyarrow as pa
+
+
+class _PyArrowTableToPandasTestBase(GoldenFileTestMixin, unittest.TestCase):
+ """
+ Shared machinery for pa.Table.to_pandas() golden file tests.
+
+ Holds the conversion helper and the source-table inventory, split into group
+ methods that these and the (temporal-flag) tests reuse. Defines no ``test_*`` of
+ its own.
+ """
+
+ @staticmethod
+ def _repr_dataframe(df) -> str:
+ """
+ Format a pandas DataFrame result as a golden-file cell: per-column tolist()
+ for a stable, Python-native representation, mirroring goldenutils'
+ ``repr_pandas_series_value`` for the Series case.
+
+ This deliberately does NOT go through ``repr_value``/``repr_pandas_value``
+ (which use ``to_json``). With the default date_as_object=True a far-future
+ date (year 9999) comes back as an object column of Python ``datetime.date``
+ objects, and ``DataFrame.to_json`` overflows converting each to an epoch
+ nanosecond int (OverflowError); tolist() returns the objects as-is, so
+ nothing overflows.
+ """
+ body = str({name: col.tolist() for name, col in df.items()}).replace("\n", " ")
+ schema = ", ".join(f"{t} {d.name}" for t, d in df.dtypes.items())
+ return f"{body}@Dataframe[{schema}]"
+
+ def _to_pandas_cell(self, table, **to_pandas_kwargs) -> str:
+ """
+ Convert ``table`` via ``to_pandas(**to_pandas_kwargs)`` and format the result
+ as a golden-file cell, returning ``ERR@<ExceptionClass>`` if it raises.
+ """
+ try:
+ return self._repr_dataframe(table.to_pandas(**to_pandas_kwargs))
Review Comment:
this way of try catch would also have wrong signal if self._repr_dataframe raises error. I think it's better to have try catch around to_pandas call only
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58720][PYTHON][TESTS] Add tests for pa.Table.to_pandas [spark]
Posted by "Spenserrrr (via GitHub)" <gi...@apache.org>.
Spenserrrr commented on code in PR #57939:
URL: https://github.com/apache/spark/pull/57939#discussion_r3765122665
##########
python/pyspark/tests/upstream/pyarrow/test_pyarrow_table_to_pandas.py:
##########
@@ -0,0 +1,340 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""
+Tests for PyArrow Table.to_pandas() using golden file comparison.
+
+Unlike Array/ChunkedArray.to_pandas() (which returns a Series and is covered by
+test_pyarrow_arrow_to_pandas_{default,non_default}.py), Table.to_pandas() returns a
+DataFrame. Its per-column conversion matches the Array tests, so this file pins the
+genuinely Table-specific behavior instead: multi-column DataFrame assembly and the
+empty-table edges (0 columns / 0 rows). Spark calls Table.to_pandas() at
+python/pyspark/sql/pandas/conversion.py:255 (the 0-column path) and, in Spark Connect,
+at python/pyspark/sql/connect/client/core.py:1423 (a bare whole-Table conversion).
+
+## Golden File Cell Format
+
+Each cell uses the value@type format:
+- pyarrow Table: "{col: [val1, val2, None], ...}@Table[name: type, ...]"
+- pandas DataFrame: "{col: [values], ...}@Dataframe[name dtype, ...]"
+- Error: "ERR@ExceptionClassName"
+
+Values are formatted via tolist() for stable, Python-native representation.
+
+## Regenerating Golden Files
+
+Set SPARK_GENERATE_GOLDEN_FILES=1 before running:
+
+ SPARK_GENERATE_GOLDEN_FILES=1 python -m pytest \\
+ python/pyspark/tests/upstream/pyarrow/test_pyarrow_table_to_pandas.py
+"""
+
+import datetime
+import unittest
+
+from pyspark.loose_version import LooseVersion
+from pyspark.testing.utils import (
+ have_pyarrow,
+ have_pandas,
+ pyarrow_requirement_message,
+ pandas_requirement_message,
+)
+from pyspark.testing.goldenutils import GoldenFileTestMixin
+
+if have_pandas:
+ import pandas as pd
+if have_pyarrow:
+ import pyarrow as pa
+
+
+class _PyArrowTableToPandasTestBase(GoldenFileTestMixin, unittest.TestCase):
+ """
+ Shared machinery for pa.Table.to_pandas() golden file tests.
+
+ Holds the conversion helper and the source-table inventory, split into group
+ methods that these and the (temporal-flag) tests reuse. Defines no ``test_*`` of
+ its own.
+ """
+
+ @staticmethod
+ def _repr_dataframe(df) -> str:
+ """
+ Format a pandas DataFrame result as a golden-file cell: per-column tolist()
+ for a stable, Python-native representation, mirroring goldenutils'
+ ``repr_pandas_series_value`` for the Series case.
+
+ This deliberately does NOT go through ``repr_value``/``repr_pandas_value``
+ (which use ``to_json``). With the default date_as_object=True a far-future
+ date (year 9999) comes back as an object column of Python ``datetime.date``
+ objects, and ``DataFrame.to_json`` overflows converting each to an epoch
+ nanosecond int (OverflowError); tolist() returns the objects as-is, so
+ nothing overflows.
+ """
+ body = str({name: col.tolist() for name, col in df.items()}).replace("\n", " ")
+ schema = ", ".join(f"{t} {d.name}" for t, d in df.dtypes.items())
+ return f"{body}@Dataframe[{schema}]"
+
+ def _to_pandas_cell(self, table, **to_pandas_kwargs) -> str:
+ """
+ Convert ``table`` via ``to_pandas(**to_pandas_kwargs)`` and format the result
+ as a golden-file cell, returning ``ERR@<ExceptionClass>`` if it raises.
+ """
+ try:
+ return self._repr_dataframe(table.to_pandas(**to_pandas_kwargs))
Review Comment:
Thanks for catching this and I just fixed it. The same both-wrapped pattern exists in the already-merged Array to_pandas tests as well and I'll unify this fix across them in a follow-up PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58720][PYTHON][TESTS] Add tests for pa.Table.to_pandas [spark]
Posted by "Spenserrrr (via GitHub)" <gi...@apache.org>.
Spenserrrr commented on code in PR #57939:
URL: https://github.com/apache/spark/pull/57939#discussion_r3765116805
##########
python/pyspark/tests/upstream/pyarrow/test_pyarrow_table_to_pandas.py:
##########
@@ -0,0 +1,340 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""
+Tests for PyArrow Table.to_pandas() using golden file comparison.
+
+Unlike Array/ChunkedArray.to_pandas() (which returns a Series and is covered by
+test_pyarrow_arrow_to_pandas_{default,non_default}.py), Table.to_pandas() returns a
+DataFrame. Its per-column conversion matches the Array tests, so this file pins the
+genuinely Table-specific behavior instead: multi-column DataFrame assembly and the
+empty-table edges (0 columns / 0 rows). Spark calls Table.to_pandas() at
+python/pyspark/sql/pandas/conversion.py:255 (the 0-column path) and, in Spark Connect,
+at python/pyspark/sql/connect/client/core.py:1423 (a bare whole-Table conversion).
+
+## Golden File Cell Format
+
+Each cell uses the value@type format:
+- pyarrow Table: "{col: [val1, val2, None], ...}@Table[name: type, ...]"
+- pandas DataFrame: "{col: [values], ...}@Dataframe[name dtype, ...]"
+- Error: "ERR@ExceptionClassName"
+
+Values are formatted via tolist() for stable, Python-native representation.
+
+## Regenerating Golden Files
+
+Set SPARK_GENERATE_GOLDEN_FILES=1 before running:
+
+ SPARK_GENERATE_GOLDEN_FILES=1 python -m pytest \\
+ python/pyspark/tests/upstream/pyarrow/test_pyarrow_table_to_pandas.py
+"""
+
+import datetime
+import unittest
+
+from pyspark.loose_version import LooseVersion
+from pyspark.testing.utils import (
+ have_pyarrow,
+ have_pandas,
+ pyarrow_requirement_message,
+ pandas_requirement_message,
+)
+from pyspark.testing.goldenutils import GoldenFileTestMixin
+
+if have_pandas:
+ import pandas as pd
+if have_pyarrow:
+ import pyarrow as pa
+
+
+class _PyArrowTableToPandasTestBase(GoldenFileTestMixin, unittest.TestCase):
+ """
+ Shared machinery for pa.Table.to_pandas() golden file tests.
+
+ Holds the conversion helper and the source-table inventory, split into group
+ methods that these and the (temporal-flag) tests reuse. Defines no ``test_*`` of
+ its own.
+ """
+
+ @staticmethod
+ def _repr_dataframe(df) -> str:
+ """
+ Format a pandas DataFrame result as a golden-file cell: per-column tolist()
+ for a stable, Python-native representation, mirroring goldenutils'
+ ``repr_pandas_series_value`` for the Series case.
+
+ This deliberately does NOT go through ``repr_value``/``repr_pandas_value``
+ (which use ``to_json``). With the default date_as_object=True a far-future
+ date (year 9999) comes back as an object column of Python ``datetime.date``
+ objects, and ``DataFrame.to_json`` overflows converting each to an epoch
+ nanosecond int (OverflowError); tolist() returns the objects as-is, so
+ nothing overflows.
+ """
+ body = str({name: col.tolist() for name, col in df.items()}).replace("\n", " ")
+ schema = ", ".join(f"{t} {d.name}" for t, d in df.dtypes.items())
+ return f"{body}@Dataframe[{schema}]"
Review Comment:
Thanks for the advice @Yicong-Huang! I'd like to do this in another PR, and I want to explain the reason behind it.
The reason why I use a local helper instead of modifying the shared `repr_pandas_value` is that it is already in use by another file. `repr_value` dispatches DataFrame to `repr_pandas_value`, and in `test_pandas_udf_return_type`, it runs a DataFrame through `repr_value` to build a golden column header. Thus, I added a local `tolist()` helper here. I actually tried changing r`epr_pandas_value` first, and it broke test_pandas_udf_return_type's goldens.
Also, the reason a different formatter is needed at all is that to_json() isn't safe for the temporal cases these Table tests cover. It raises OverflowError on out-of-nanosecond-range dates (the year-9999 rows, which are object-dtype datetime.date under the default date_as_object=True), and on pandas 2 it also misreads non-ns datetime64 units. tolist() avoids both.
To promote it into GoldenFileTestMixin cleanly, I plan to fix `repr_pandas_value`, regenerate test_pandas_udf_return_type's goldens, and switch these Table tests to call repr_value directly. Does this sound reasonable, or do you have any suggestions on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58720][PYTHON][TESTS] Add tests for pa.Table.to_pandas [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #57939:
URL: https://github.com/apache/spark/pull/57939#issuecomment-5268806368
Thank you @Spenserrrr and @zhengruifeng @Yicong-Huang!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58720][PYTHON][TESTS] Add tests for pa.Table.to_pandas [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #57939:
URL: https://github.com/apache/spark/pull/57939#issuecomment-5277457889
**Merge Summary:**
- merged into master https://github.com/apache/spark/commit/df1646c0ab325a631e6b00bf67be18d5955cefe2
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58720][PYTHON][TESTS] Add tests for pa.Table.to_pandas [spark]
Posted by "Spenserrrr (via GitHub)" <gi...@apache.org>.
Spenserrrr commented on PR #57939:
URL: https://github.com/apache/spark/pull/57939#issuecomment-5260098105
Hi @Yicong-Huang @zhengruifeng! This PR adds coverage for pa.Table.to_pandas. Could you take a look when you have a moment? Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58720][PYTHON][TESTS] Add tests for pa.Table.to_pandas [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #57939:
URL: https://github.com/apache/spark/pull/57939#issuecomment-5277464641
**Merge Summary:**
- merged into branch-4.x https://github.com/apache/spark/commit/b0200d1bae4d877e3f8814dfb4c7ad7fe9aaa03d
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org