[jira] [Created] (SPARK-17681) Empty DataFrame with non-zero rows after using drop

["Ian Hellstrom (JIRA)" <ji...@apache.org>]

Ian Hellstrom created SPARK-17681:
-------------------------------------

             Summary: Empty DataFrame with non-zero rows after using drop
                 Key: SPARK-17681
                 URL: https://issues.apache.org/jira/browse/SPARK-17681
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 1.6.0, 1.4.1
            Reporter: Ian Hellstrom


It is possible to have a {DataFrame} with no columns to have a non-zero number of rows, even though the contents are empty:

{code}
val df = Seq((1,2)).toDF("a", "b")
df.drop("a").drop("b").count
{code}

I tried checking it for 2.0 but unfortunately the sbt console for Spark does not work on Windows. Anyway, I think the following snippet ought to be a simple copy-paste exercise:

{code}
import org.apache.spark._
import org.apache.spark.sql._

val conf = new SparkConf()
val sc = new SparkContext("local", "demo", conf)
val ss = SparkSession.builder.getOrCreate()

import ss.implicits._

case class Data(a: Int, b: Int)
val rdd = sc.parallelize(List(Data(1,2)))
val ds = ss.createDataset(rdd)

ds.drop("a").drop("b").count
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org