[jira] [Created] (SPARK-22472) Datasets generate random values for null primitive types

["Vladislav Kuzemchik (JIRA)" <ji...@apache.org>]

Vladislav Kuzemchik created SPARK-22472:
-------------------------------------------

             Summary: Datasets generate random values for null primitive types
                 Key: SPARK-22472
                 URL: https://issues.apache.org/jira/browse/SPARK-22472
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.1.1
            Reporter: Vladislav Kuzemchik


Not sure if it ever were reported.

{code}
scala> val s = sc.parallelize(Seq[Option[Long]](None,Some(1L),Some(5))).toDF("v")
s: org.apache.spark.sql.DataFrame = [v: bigint]

scala> s.show(false)
+----+
|v   |
+----+
|null|
|1   |
|5   |
+----+


scala> s.as[Long].map(v => v*2).show(false)
+-----+
|value|
+-----+
|-2   |
|2    |
|10   |
+-----+


scala> s.select($"v"*2).show(false)
+-------+
|(v * 2)|
+-------+
|null   |
|2      |
|10     |
+-------+
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org