[jira] [Created] (SPARK-26393) Different behaviors of date_add when calling it inside expr

["Ahmed Kamal` (JIRA)" <ji...@apache.org>]

Ahmed Kamal` created SPARK-26393:
------------------------------------

             Summary: Different behaviors of date_add when calling it inside expr
                 Key: SPARK-26393
                 URL: https://issues.apache.org/jira/browse/SPARK-26393
             Project: Spark
          Issue Type: Bug
          Components: PySpark
    Affects Versions: 2.3.2
            Reporter: Ahmed Kamal`


When Calling date_add from pyspark.sql.functions directly without using expr, like this : 
{code:java}
df.withColumn("added", F.date_add(F.to_date(F.lit('1998-9-26')), F.col('days'))).toPandas(){code}
It will raise Error : `TypeError: Column is not iterable`

because it only taking a number not a column 

but when i try to use it inside an expr, like this :

 
{code:java}
df.withColumn("added", F.expr("date_add(to_date('1998-9-26'), days)")).toPandas()
{code}
 

it will work fine.

 

Shouldn't it behave the same way ? 

and i thin its logical to accept a column  here as well.

 

A python Notebook to demonstrate :

https://gist.github.com/AhmedKamal20/fec10337e815baa44f115d307e3b07eb



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org