DataFrame.foreach(scala.Function1) example

[Mohammad Tariq <do...@gmail.com>]

Dear fellow spark users,

Could someone please point me to any example showcasing the usage of
*DataFrame.oreach(scala.Function1)* in *Java*?

*Problem statement :* I am reading data from a Kafka topic, and for each
RDD in the DStream I am creating a DataFrame in order to perform some
operations. After this I have to call *DataFrame.javaRDD* to convert the
resulting DF back into a *JavaRDD* so that I can perform further
computation on each record in this RDD through *JavaRDD<Row>.foreach*.

However, I wish to remove this additional hop of creating JavaRDD from the
resultant DF. I would like to use *DataFrame.oreach(scala.Function1)*
instead, and perform the computation directly on each *Row* of this DF
rather than having to first convert the DF into JavaRDD and then performing
any computations.

For interfaces like *Function* and *VoidFunction*(provided by
org.apache.spark.api.java.function API) I could implement named classes and
use them in my code. But having some hard time in figuring out how to use
*scala.Function1<scala.collection.Iterator<Row>,scala.runtime.BoxedUnit>*

Thank you so much for your valuable time. Really appreciate it!


[image: http://]

Tariq, Mohammad
about.me/mti
[image: http://]
<http://about.me/mti>