You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Jerry Lam <ch...@gmail.com> on 2015/07/12 07:32:37 UTC
Re: [PySpark DataFrame] When a Row is not a Row
Hi guys,
I just hit the same problem. It is very confusing when Row is not the same
Row type at runtime. The worst thing is that when I use Spark in local mode,
the Row is the same Row type! so it passes the test cases but it fails when
I deploy the application.
Can someone suggest a workaround?
Best Regards,
Jerry
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/PySpark-DataFrame-When-a-Row-is-not-a-Row-tp12210p13153.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [PySpark DataFrame] When a Row is not a Row
Posted by Davies Liu <da...@databricks.com>.
We finally fix this in 1.5 (next release), see
https://github.com/apache/spark/pull/7301
On Sat, Jul 11, 2015 at 10:32 PM, Jerry Lam <ch...@gmail.com> wrote:
> Hi guys,
>
> I just hit the same problem. It is very confusing when Row is not the same
> Row type at runtime. The worst thing is that when I use Spark in local mode,
> the Row is the same Row type! so it passes the test cases but it fails when
> I deploy the application.
>
> Can someone suggest a workaround?
>
> Best Regards,
>
> Jerry
>
>
>
> --
> View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/PySpark-DataFrame-When-a-Row-is-not-a-Row-tp12210p13153.html
> Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org