"object cannot be cast to Double" using pipline with pyspark

[colin <co...@sina.cn>]

My colleagues use scala and I use python. 

They save a hive table ,which has doubletype columns. However there's no
double in python.

When I use /pipline.fit(dataframe)/, there occured an error:

java.lang.ClassCastException:  [Ljava.lang.Object: cnnot be cast to
java.lang.Double......

I guess it's because python doesn't have double type. So I cast these
columns to float using codes below:

/dateframe=dateframe.withColumn(dateframe.columns[0],dateframe[0].cast("float"))/

Then when I print the schema of the dataframe, there's no doubletype
columns.

However, when the programme goes into /pipline.fit(dataframe)/, the
ClassCastException still exists.

Why? How can I really modify the column type of a dateframe?



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/object-cannot-be-cast-to-Double-using-pipline-with-pyspark-tp27462.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org