You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Damian Momot (JIRA)" <ji...@apache.org> on 2016/11/17 12:11:59 UTC
[jira] [Updated] (SPARK-18491) Spark uses mutable classes for
date/time types mapping
[ https://issues.apache.org/jira/browse/SPARK-18491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Damian Momot updated SPARK-18491:
---------------------------------
Description:
TimestampType is mapped to java.sql.Timestamp
DateType is mapped to java.sql.Date
Those both java types are mutable and thus their usage is highly discourage, especially in distributed computing which uses lazy, functional approach
Mapping to immutable joda times should be enough for now (until scala 2.12 + jdk8 java.time is available for spark)
It becomes more relevant now as schema is inferred from user provided case classes in datasets
was:
TimestampType is mapped to java.sql.Timestamp
DateType is mapped to java.sql.Date
Those both java types are mutable and thus their usage is highly discourage, especially in distributed computing which uses lazy, functional approach
Mapping to immutable joda times should be enough for now (until scala 2.12 + jdk8 java.time is available for spark)
> Spark uses mutable classes for date/time types mapping
> ------------------------------------------------------
>
> Key: SPARK-18491
> URL: https://issues.apache.org/jira/browse/SPARK-18491
> Project: Spark
> Issue Type: Improvement
> Reporter: Damian Momot
>
> TimestampType is mapped to java.sql.Timestamp
> DateType is mapped to java.sql.Date
> Those both java types are mutable and thus their usage is highly discourage, especially in distributed computing which uses lazy, functional approach
> Mapping to immutable joda times should be enough for now (until scala 2.12 + jdk8 java.time is available for spark)
> It becomes more relevant now as schema is inferred from user provided case classes in datasets
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org