You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by jay vyas <ja...@gmail.com> on 2015/02/10 00:23:42 UTC

SparkSQL DateTime

Hi spark ! We are working on the bigpetstore-spark implementation in apache
bigtop, and want to implement idiomatic date/time usage for SparkSQL.

It appears that org.joda.time.DateTime  isnt in  SparkSQL's  rolodex of
reflection types.

I'd rather not force an artificial dependency on hive dates just for
dealing with time stamps.

Whats the simplest and cleanest way to map non-spark time values  into
SparkSQL friendly time values?

- One option could be a custom SparkSQL type, i guess?
- Any plan to have native spark sql support for Joda Time or (yikes)
java.util.Calendar ?

-- 
jay vyas

Re: SparkSQL DateTime

Posted by Michael Armbrust <mi...@databricks.com>.
The standard way to add timestamps is java.sql.Timestamp.

On Mon, Feb 9, 2015 at 3:23 PM, jay vyas <ja...@gmail.com>
wrote:

> Hi spark ! We are working on the bigpetstore-spark implementation in
> apache bigtop, and want to implement idiomatic date/time usage for SparkSQL.
>
> It appears that org.joda.time.DateTime  isnt in  SparkSQL's  rolodex of
> reflection types.
>
> I'd rather not force an artificial dependency on hive dates just for
> dealing with time stamps.
>
> Whats the simplest and cleanest way to map non-spark time values  into
> SparkSQL friendly time values?
>
> - One option could be a custom SparkSQL type, i guess?
> - Any plan to have native spark sql support for Joda Time or (yikes)
> java.util.Calendar ?
>
> --
> jay vyas
>