You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Michael Armbrust (JIRA)" <ji...@apache.org> on 2014/08/08 20:27:12 UTC
[jira] [Resolved] (SPARK-2854) Finalize _acceptable_types in
pyspark.sql
[ https://issues.apache.org/jira/browse/SPARK-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Armbrust resolved SPARK-2854.
-------------------------------------
Resolution: Fixed
Fix Version/s: 1.1.0
Assignee: Yin Huai
> Finalize _acceptable_types in pyspark.sql
> -----------------------------------------
>
> Key: SPARK-2854
> URL: https://issues.apache.org/jira/browse/SPARK-2854
> Project: Spark
> Issue Type: Task
> Components: SQL
> Reporter: Yin Huai
> Assignee: Yin Huai
> Priority: Blocker
> Fix For: 1.1.0
>
>
> In PySpark, _acceptable_types defines accepted Python data types for every Spark SQL data type. The list is shown below.
> {code}
> _acceptable_types = {
> BooleanType: (bool,),
> ByteType: (int, long),
> ShortType: (int, long),
> IntegerType: (int, long),
> LongType: (int, long),
> FloatType: (float,),
> DoubleType: (float,),
> DecimalType: (decimal.Decimal,),
> StringType: (str, unicode),
> TimestampType: (datetime.datetime, datetime.time, datetime.date),
> ArrayType: (list, tuple, array),
> MapType: (dict,),
> StructType: (tuple, list),
> }
> {code}
> Let's double check this mapping before 1.1 release.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org