You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Haejoon Lee (Jira)" <ji...@apache.org> on 2023/04/25 13:08:00 UTC
[jira] [Created] (SPARK-43282) Investigate DataFrame.sort_values with pandas behavior.
Haejoon Lee created SPARK-43282:
-----------------------------------
Summary: Investigate DataFrame.sort_values with pandas behavior.
Key: SPARK-43282
URL: https://issues.apache.org/jira/browse/SPARK-43282
Project: Spark
Issue Type: Sub-task
Components: Pandas API on Spark
Affects Versions: 3.5.0
Reporter: Haejoon Lee
{code:java}
import pandas as pd
pdf = pd.DataFrame(
{
"a": pd.Categorical([1, 2, 3, 1, 2, 3]),
"b": pd.Categorical(
["b", "a", "c", "c", "b", "a"], categories=["c", "b", "d", "a"]
),
},
)
pdf.groupby("a").apply(lambda x: x).sort_values(["a"])
Traceback (most recent call last):
...
ValueError: 'a' is both an index level and a column label, which is ambiguous. {code}
We should investigate this issue whether this is intended behavior or just bug in pandas.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org