You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by tmoffwood <th...@growthintel.com> on 2016/02/16 17:33:39 UTC
In Spark Dataframes, does dropDuplicates retain the first row?
If we sort a Spark Dataframe by col1 and then dropDuplicates on col2, will it
retain the first row (which will have the highest col1 value for each col2)?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/In-Spark-Dataframes-does-dropDuplicates-retain-the-first-row-tp26236.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org