Calender Obj to java.util.date conversion issue

[satish chandra j <js...@gmail.com>]

HI All,
Please find the below snippet referring to UDF which subtracts a day from
the given date value

*Snippet **:*

*val* sub_a_day = udf((d:Date) => {cal.setTime(d)

     cal.add(Calendar.DATE, -1)

     cal.getTime()

    })



*Error *:

Exception in thread "main" java.lang.UnsupportedOperationException: Schema
for type java.util.Date is not supported

        at
org.apache.spark.sql.catalyst.ScalaReflection$class.schemaFor(ScalaReflection.scala:152)

        at
org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:28)

        at
org.apache.spark.sql.catalyst.ScalaReflection$class.schemaFor(ScalaReflection.scala:63)

        at
org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:28)

        at org.apache.spark.sql.functions$.udf(functions.scala:1363)

        at
table_generation_mm_ref_df$.main(table_generation_mm_ref_df.scala:88)

        at table_generation_mm_ref_df.main(table_generation_mm_ref_df.scala)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:497)

        at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:665)

        at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170)

        at
org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193)

        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112)

        at
org.apache.spark.deploy.DseSparkSubmitBootstrapper$.main(DseSparkSubmitBootstrapper.scala:45)

        at
org.apache.spark.deploy.DseSparkSubmitBootstrapper.main(DseSparkSubmitBootstrapper.scala)



Please let me know if I am missing anything here


Regards,

Satish Chandra