You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "geoff langenderfer (Jira)" <ji...@apache.org> on 2023/10/22 00:08:00 UTC

[jira] [Commented] (SPARK-41762) Make `Column.__neg__` return the same column name as PySpark

    [ https://issues.apache.org/jira/browse/SPARK-41762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17778252#comment-17778252 ] 

geoff langenderfer commented on SPARK-41762:
--------------------------------------------

I'm trying to [reproduce this test error](https://github.com/apache/spark/blob/227cd8bb8f5d8d442f225057db39591b3c630f46/python/pyspark/sql/tests/connect/test_connect_column.py#L816), but I don't think I'm actually running the test. I get no output when I place a print statement. There is also no expected error.

how to reproduce:

```
~/work/spark master* ⇣
myenv ❯ python/run-tests --testnames pyspark.sql.tests.connect
✦ ~/work/spark master* ⇣ 20s
myenv ❯ bash tests.sh
Running PySpark tests. Output is in /home/g/work/spark/python/unit-tests.log
Will test against the following Python executables: ['/home/g/work/spark/python/myenv/bin/python3', 'python']
Will test the following Python tests: ['pyspark.sql.tests.connect.test_connect_column']
/home/g/work/spark/python/myenv/bin/python3 python_implementation is CPython
/home/g/work/spark/python/myenv/bin/python3 version is: Python 3.11.5
python python_implementation is CPython
python version is: Python 3.11.5
Starting test(/home/g/work/spark/python/myenv/bin/python3): pyspark.sql.tests.connect.test_connect_column (temp output: /home/g/work/spark/python/target/ef467634-9942-4870-88da-627971dc89ce/home_g_work_spark_python_myenv_bin_python3__pyspark.sql.tests.connect.test_connect_column__8ihwke8t.log)
Starting test(python): pyspark.sql.tests.connect.test_connect_column (temp output: /home/g/work/spark/python/target/2d173917-51fe-465a-bf5b-6ca87b64c9a6/python__pyspark.sql.tests.connect.test_connect_column__c5_jcwar.log)
Finished test(python): pyspark.sql.tests.connect.test_connect_column (0s)
Finished test(/home/g/work/spark/python/myenv/bin/python3): pyspark.sql.tests.connect.test_connect_column (0s)
Tests passed in 0 seconds

~/work/spark master* ⇣
myenv ❯ 
```


> Make `Column.__neg__` return the same column name as PySpark
> ------------------------------------------------------------
>
>                 Key: SPARK-41762
>                 URL: https://issues.apache.org/jira/browse/SPARK-41762
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Connect, PySpark
>    Affects Versions: 3.4.0
>            Reporter: Ruifeng Zheng
>            Priority: Major
>
> [left]:  Index(['negative(a)'], dtype='object')
> [right]: Index(['(- a)'], dtype='object')



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org