You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Suresh Thalamati (JIRA)" <ji...@apache.org> on 2016/02/03 23:21:39 UTC

[jira] [Commented] (SPARK-13167) JDBC data source does not include null value partition columns rows in the result.

    [ https://issues.apache.org/jira/browse/SPARK-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131258#comment-15131258 ] 

Suresh Thalamati commented on SPARK-13167:
------------------------------------------

I am working on fix for this issue. 

> JDBC data source does not include null value partition columns rows in the result.
> ----------------------------------------------------------------------------------
>
>                 Key: SPARK-13167
>                 URL: https://issues.apache.org/jira/browse/SPARK-13167
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.6.0, 2.0.0
>            Reporter: Suresh Thalamati
>
> Reading from am JDBC data source using a partition column that is nullable can return incorrect number of rows, if there are rows with null value for partition column.
> {code}
> val emp = sqlContext.read.jdbc("jdbc:h2:mem:testdb0;user=testUser;password=testPass", "TEST.EMP", "theid", 0, 4, 3, new Properties)
> emp.count()
> {code}
> Above jdbc read call sets up the partitions of the following form. It does not include null predicate.
> {code}
> JDBCPartition(THEID < 1,0),JDBCPartition(THEID >= 1 AND THEID < 2,1),JDBCPartition(THEID >= 2,2)
> {code}
> Rows with null values in partition column are not included in the results because the partition predicate does not specify is null predicates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org