You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by k0ala <k0...@gmail.com> on 2015/07/15 16:41:04 UTC
DataFrame more efficient than RDD?
Hi,
I have been working a bit with RDD, and am now taking a look at DataFrames.
The schema definition using case classes looks very attractive;
https://spark.apache.org/docs/1.4.0/sql-programming-guide.html#inferring-the-schema-using-reflection
<https://spark.apache.org/docs/1.4.0/sql-programming-guide.html#inferring-the-schema-using-reflection>
Is a DataFrame more efficient (space-wise) than an RDD for the same case
class?
And in general, when should DataFrames be preferred over RDDs, and vice
versa?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/DataFrame-more-efficient-than-RDD-tp23857.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: DataFrame more efficient than RDD?
Posted by Ted Yu <yu...@gmail.com>.
Here is a related thread:
http://search-hadoop.com/m/q3RTtPmjSJ1Dod92
> On Jul 15, 2015, at 7:41 AM, k0ala <k0...@gmail.com> wrote:
>
> Hi,
>
> I have been working a bit with RDD, and am now taking a look at DataFrames.
> The schema definition using case classes looks very attractive;
>
> https://spark.apache.org/docs/1.4.0/sql-programming-guide.html#inferring-the-schema-using-reflection
> <https://spark.apache.org/docs/1.4.0/sql-programming-guide.html#inferring-the-schema-using-reflection>
>
> Is a DataFrame more efficient (space-wise) than an RDD for the same case
> class?
>
> And in general, when should DataFrames be preferred over RDDs, and vice
> versa?
>
>
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/DataFrame-more-efficient-than-RDD-tp23857.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>