You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by afarahat <ay...@yahoo.com> on 2015/07/31 02:31:07 UTC

How do i specify the data types in a DF

Hello; 
I have a simple file of mobile IDFA. they look like
gregconv = ['00013FEE-7561-47F3-95BC-CA18D20BCF78',
'000D9B97-2B54-4B80-AAA1-C1CB42CFBF3A',
'000F9E1F-BC7E-47E1-BF68-C68F6D987B96']
I am trying to make this RDD into a data frame

ConvRecord = Row("IDFA")

gregconvdf = gregconv.map(lambda x: ConvRecord(*x)).toDF()
i get the following error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/homes/afarahat/aofspark/share/spark/python/pyspark/sql/context.py",
line 60, in toDF
    return sqlContext.createDataFrame(self, schema, sampleRatio)
  File "/homes/afarahat/aofspark/share/spark/python/pyspark/sql/context.py",
line 351, in createDataFrame
    _verify_type(row, schema)
  File "/homes/afarahat/aofspark/share/spark/python/pyspark/sql/types.py",
line 1027, in _verify_type
    "length of fields (%d)" % (len(obj), len(dataType.fields)))
ValueError: Length of object (36) does not match with length of fields (1)



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-do-i-specify-the-data-types-in-a-DF-tp24090.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org