You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by jay vyas <ja...@gmail.com> on 2015/02/10 00:23:42 UTC
SparkSQL DateTime
Hi spark ! We are working on the bigpetstore-spark implementation in apache
bigtop, and want to implement idiomatic date/time usage for SparkSQL.
It appears that org.joda.time.DateTime isnt in SparkSQL's rolodex of
reflection types.
I'd rather not force an artificial dependency on hive dates just for
dealing with time stamps.
Whats the simplest and cleanest way to map non-spark time values into
SparkSQL friendly time values?
- One option could be a custom SparkSQL type, i guess?
- Any plan to have native spark sql support for Joda Time or (yikes)
java.util.Calendar ?
--
jay vyas
Re: SparkSQL DateTime
Posted by Michael Armbrust <mi...@databricks.com>.
The standard way to add timestamps is java.sql.Timestamp.
On Mon, Feb 9, 2015 at 3:23 PM, jay vyas <ja...@gmail.com>
wrote:
> Hi spark ! We are working on the bigpetstore-spark implementation in
> apache bigtop, and want to implement idiomatic date/time usage for SparkSQL.
>
> It appears that org.joda.time.DateTime isnt in SparkSQL's rolodex of
> reflection types.
>
> I'd rather not force an artificial dependency on hive dates just for
> dealing with time stamps.
>
> Whats the simplest and cleanest way to map non-spark time values into
> SparkSQL friendly time values?
>
> - One option could be a custom SparkSQL type, i guess?
> - Any plan to have native spark sql support for Joda Time or (yikes)
> java.util.Calendar ?
>
> --
> jay vyas
>