You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Davies Liu (JIRA)" <ji...@apache.org> on 2014/09/18 19:33:33 UTC
[jira] [Created] (SPARK-3592) applySchema to an RDD of Row
Davies Liu created SPARK-3592:
---------------------------------
Summary: applySchema to an RDD of Row
Key: SPARK-3592
URL: https://issues.apache.org/jira/browse/SPARK-3592
Project: Spark
Issue Type: Bug
Components: PySpark, SQL
Reporter: Davies Liu
Priority: Critical
Right now, we can not appy schema to a RDD of Row, this should be a Bug,
{code}
>>> srdd = sqlCtx.jsonRDD(sc.parallelize(["""{"a":2}"""]))
>>> sqlCtx.applySchema(srdd.map(lambda x:x), srdd.schema())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/daviesliu/work/spark/python/pyspark/sql.py", line 1121,
in applySchema
_verify_type(row, schema)
File "/Users/daviesliu/work/spark/python/pyspark/sql.py", line 736,
in _verify_type
% (dataType, type(obj)))
TypeError: StructType(List(StructField(a,IntegerType,true))) can not
accept abject in type <class 'pyspark.sql.Row'>
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org