[jira] [Created] (PHOENIX-2567) phoenix-spark: DataFrame API should handle 'DATE' columns

["Josh Mahonin (JIRA)" <ji...@apache.org>]

Josh Mahonin created PHOENIX-2567:
-------------------------------------

             Summary: phoenix-spark: DataFrame API should handle 'DATE' columns
                 Key: PHOENIX-2567
                 URL: https://issues.apache.org/jira/browse/PHOENIX-2567
             Project: Phoenix
          Issue Type: Bug
    Affects Versions: 4.7.0
            Reporter: Josh Mahonin
            Assignee: Josh Mahonin
             Fix For: 4.7.0


The current implementation had the 'DATE' datatype bound to a Spark SQL 'TimestampType', which causes a casting error trying to convert from java.sql.Date to java.sql.Timestamp when using the DataFrame API with Phoenix DATE columns.

This patch modifies the schema handling to treat DATE columns as the Spark 'DateType' instead. Note that Spark *drops* the hour, minute and second values from these when interfacing using DataFrames. This follows the java.sql.Date spec, but might not useful to folks who rely on the hour/minute/second fields working using the DataFrame API and DATE columns. A future improvement would be to force these to be TimestampTypes instead to preserve information, but it's less intuitive and probably shouldn't be the default behaviour.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)