[jira] [Created] (SPARK-13167) JDBC data source does not include null value partition columns rows in the result.

["Suresh Thalamati (JIRA)" <ji...@apache.org>]

Suresh Thalamati created SPARK-13167:
----------------------------------------

             Summary: JDBC data source does not include null value partition columns rows in the result.
                 Key: SPARK-13167
                 URL: https://issues.apache.org/jira/browse/SPARK-13167
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 1.6.0, 2.0.0
            Reporter: Suresh Thalamati


Reading from am JDBC data source using a partition column that is nullable can return incorrect number of rows, if there are rows with null value for partition column.

{code}
val emp = sqlContext.read.jdbc("jdbc:h2:mem:testdb0;user=testUser;password=testPass", "TEST.EMP", "theid", 0, 4, 3, new Properties)
emp.count()
{code}

Above jdbc read call sets up the partitions of the following form. It does not include null predicate.

{code}
JDBCPartition(THEID < 1,0),JDBCPartition(THEID >= 1 AND THEID < 2,1),JDBCPartition(THEID >= 2,2)
{code}

Rows with null values in partition column are not included in the results because the partition predicate does not specify is null predicates.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org