You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by da...@apache.org on 2015/07/13 05:25:13 UTC
spark git commit: [SPARK-9006] [PYSPARK] fix microsecond loss in
Python 3
Repository: spark
Updated Branches:
refs/heads/master 30090884f -> 20b474335
[SPARK-9006] [PYSPARK] fix microsecond loss in Python 3
It may loss a microsecond if using timestamp as float, should be `int` instead.
Author: Davies Liu <da...@databricks.com>
Closes #7363 from davies/fix_microsecond and squashes the following commits:
36f6007 [Davies Liu] fix microsecond loss in Python 3
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/20b47433
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/20b47433
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/20b47433
Branch: refs/heads/master
Commit: 20b474335c68c644150fdc8443a2d0d2dad5e27d
Parents: 3009088
Author: Davies Liu <da...@databricks.com>
Authored: Sun Jul 12 20:25:06 2015 -0700
Committer: Davies Liu <da...@gmail.com>
Committed: Sun Jul 12 20:25:06 2015 -0700
----------------------------------------------------------------------
python/pyspark/sql/types.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/20b47433/python/pyspark/sql/types.py
----------------------------------------------------------------------
diff --git a/python/pyspark/sql/types.py b/python/pyspark/sql/types.py
index d638576..f75791f 100644
--- a/python/pyspark/sql/types.py
+++ b/python/pyspark/sql/types.py
@@ -188,7 +188,8 @@ class TimestampType(AtomicType):
def fromInternal(self, ts):
if ts is not None:
- return datetime.datetime.fromtimestamp(ts / 1e6)
+ # using int to avoid precision loss in float
+ return datetime.datetime.fromtimestamp(ts // 1000000).replace(microsecond=ts % 1000000)
class DecimalType(FractionalType):
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org