You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/12/05 23:30:10 UTC
[GitHub] [spark] oeuf opened a new pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
oeuf opened a new pull request #34812:
URL: https://github.com/apache/spark/pull/34812
### What changes were proposed in this pull request?
- Adds code changes to allow for underscores in the elements for the `columns` arg and for the column names used for the `values` arg.
### Why are the changes needed?
Fixes a bug with the method `pyspark.pandas.frames.DataFrame.pivot_table` that causes a `KeyError` when an underscore is present (more details in [SPARK-37553](https://issues.apache.org/jira/browse/SPARK-37553)).
```python
>>> import numpy as np
>>> import pandas as pd
>>> from pyspark import pandas as ps
>>> pdf = pd.DataFrame(
{
"a": [4, 2, 3, 4, 8, 6],
"b_b": [1, 2, 2, 4, 2, 4],
"e": [10, 20, 20, 40, 20, 40],
"c": [1, 2, 9, 4, 7, 4],
"d": [-1, -2, -3, -4, -5, -6],
},
index=np.random.rand(6),
)
>>> psdf = ps.from_pandas(pdf)
>>> psdf.pivot_table(index=["c"], columns="a", values=["b_b", "e"])
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-8-32d5bb0e1166> in <module>
----> 1 psdf.pivot_table(index=["c"], columns="a", values=["b_b", "e"])
~/.pyenv/versions/3.7.9/envs/venv37/lib/python3.7/site-packages/pyspark/pandas/frame.py in pivot_table(self, values, index, columns, aggfunc, fill_value)
6053 column_labels = [
6054 tuple(list(column_name_to_index[name.split("_")[1]]) + [name.split("_")[0]])
-> 6055 for name in data_columns
6056 ]
6057 column_label_names = (
~/.pyenv/versions/3.7.9/envs/venv37/lib/python3.7/site-packages/pyspark/pandas/frame.py in <listcomp>(.0)
6053 column_labels = [
6054 tuple(list(column_name_to_index[name.split("_")[1]]) + [name.split("_")[0]])
-> 6055 for name in data_columns
6056 ]
6057 column_label_names = (
KeyError: 'b'
```
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- [x] Add unit tests for code changes
- [] Build package via Github Actions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986340422
**[Test build #145933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145933/testReport)** for PR 34812 at commit [`eff9357`](https://github.com/apache/spark/commit/eff9357d00590d5ed0242e4b432f428fa9338389).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] oeuf commented on a change in pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
oeuf commented on a change in pull request #34812:
URL: https://github.com/apache/spark/pull/34812#discussion_r763502607
##########
File path: python/pyspark/pandas/frame.py
##########
@@ -6054,17 +6056,21 @@ def pivot_table(
# E.g. if column is b and values is ['b','e'],
# then ['2_b', '2_e', '3_b', '3_e'].
- # We sort the columns of Spark DataFrame by values.
- data_columns.sort(key=lambda x: x.split("_", 1)[1])
Review comment:
I went ahead and added a slightly different approach that removes the usage of `Series.unique`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-987032207
**[Test build #145959 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145959/testReport)** for PR 34812 at commit [`3634265`](https://github.com/apache/spark/commit/3634265abd18059992bd79c576ff4b959d899063).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986331041
ok to test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986340422
**[Test build #145933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145933/testReport)** for PR 34812 at commit [`eff9357`](https://github.com/apache/spark/commit/eff9357d00590d5ed0242e4b432f428fa9338389).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986373005
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50409/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986325022
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986655010
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #34812:
URL: https://github.com/apache/spark/pull/34812#discussion_r762637825
##########
File path: python/pyspark/pandas/frame.py
##########
@@ -6054,17 +6056,21 @@ def pivot_table(
# E.g. if column is b and values is ['b','e'],
# then ['2_b', '2_e', '3_b', '3_e'].
- # We sort the columns of Spark DataFrame by values.
- data_columns.sort(key=lambda x: x.split("_", 1)[1])
Review comment:
Can we just simply use `-1` index? Calling `Series.unique` will trigger aggregation on each column that's pretty expensive. We could alternatively just subtract original column names instead of invoking `unique`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-987009310
**[Test build #145959 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145959/testReport)** for PR 34812 at commit [`3634265`](https://github.com/apache/spark/commit/3634265abd18059992bd79c576ff4b959d899063).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-987124424
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50433/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986364847
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145933/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-987042375
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50433/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ch-nickgustafson commented on a change in pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
ch-nickgustafson commented on a change in pull request #34812:
URL: https://github.com/apache/spark/pull/34812#discussion_r763500110
##########
File path: python/pyspark/pandas/frame.py
##########
@@ -6054,17 +6056,21 @@ def pivot_table(
# E.g. if column is b and values is ['b','e'],
# then ['2_b', '2_e', '3_b', '3_e'].
- # We sort the columns of Spark DataFrame by values.
- data_columns.sort(key=lambda x: x.split("_", 1)[1])
Review comment:
I went ahead and tried a slightly different approach that removes thee usage of `Series.unique`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #34812: [SPARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #34812:
URL: https://github.com/apache/spark/pull/34812
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986387197
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50409/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] oeuf commented on a change in pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
oeuf commented on a change in pull request #34812:
URL: https://github.com/apache/spark/pull/34812#discussion_r762681880
##########
File path: python/pyspark/pandas/frame.py
##########
@@ -6054,17 +6056,21 @@ def pivot_table(
# E.g. if column is b and values is ['b','e'],
# then ['2_b', '2_e', '3_b', '3_e'].
- # We sort the columns of Spark DataFrame by values.
- data_columns.sort(key=lambda x: x.split("_", 1)[1])
Review comment:
Thank you for the feedback, I appreciate it! :)
I tried using `-1` index, but it doesn't give the sort order expected by the tests. The `Series.unique` should only happen for a single column -- the docstring says only a single column is supported ([Link](https://github.com/apache/spark/blob/master/python/pyspark/pandas/frame.py#L5861-L5863)). Agreed that it's expensive, but I am not sure what else to do.
What do you think about using: ` _columns = [str(i) for i in set(self[columns].tolist())]` instead of `_columns = [str(i) for i in self[columns].unique().tolist()]`? Would this be less expensive?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986655010
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-987092412
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50433/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-987434655
cc @itholic and @xinrong-databricks if you guys find some time to review. Seems like we should fix this flaky implementation on column name at pivoting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-987058240
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145959/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-987124424
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50433/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986350850
**[Test build #145933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145933/testReport)** for PR 34812 at commit [`eff9357`](https://github.com/apache/spark/commit/eff9357d00590d5ed0242e4b432f428fa9338389).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986359188
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50409/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986364847
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145933/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #34812: [SPARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-1085259895
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-987009310
**[Test build #145959 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145959/testReport)** for PR 34812 at commit [`3634265`](https://github.com/apache/spark/commit/3634265abd18059992bd79c576ff4b959d899063).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-987058240
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145959/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986325022
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-986387197
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50409/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34812: [WIP][PARK-37553][PYTHON] Fix underscore (`_`) bug in pyspark.pandas.frames.DataFrame.pivot_table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34812:
URL: https://github.com/apache/spark/pull/34812#issuecomment-987517454
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org