Spark scala examples

[Nirav Patel <np...@xactlycorp.com>]

Hi,

Can anyone point me to any spark-scala examples using livy as a spark
jobserver? I am trying to understand how much existing spark based work
(rdd, dataframe based transformations) we can leverage when using livy.
seems like all but it'd be nice to see examples.

Thanks

-- 


 <http://www.xactlycorp.com/email-click/>

 
<https://www.instagram.com/xactlycorp/>   
<https://www.linkedin.com/company/xactly-corporation>   
<https://twitter.com/Xactly>   <https://www.facebook.com/XactlyCorp>   
<http://www.youtube.com/xactlycorporation>

Re: Spark scala examples

["Harsch, Tim" <Ti...@Teradata.com>]

Hi Nirav,
There are a few ways to use Livy: Shared sessions and Batch sessions are covered in the REST API docs (https://livy.incubator.apache.org/docs/latest/rest-api.html) and GetHue.com wrote some cool articles.  I created a little bash framework for testing out some example code I found at GetHue here:  https://bitbucket.org/harschware/livy-shared-rdd/src/master/    Pointers to the articles are in the README.  Livy also has a programmatic API you’ve probably seen already ( https://livy.incubator.apache.org/docs/latest/programmatic-api.html )

With the shared session type “spark” the code that you execute with the Livy REST API is pretty much exactly what you would execute in a spark-shell.  There are a lot of examples via Google about running spark-shell and they translate well to running in Livy.

HTH,
Tim

From: Nirav Patel <np...@xactlycorp.com>
Reply-To: "user@livy.incubator.apache.org" <us...@livy.incubator.apache.org>
Date: Friday, May 25, 2018 at 11:25 PM
To: "user@livy.incubator.apache.org" <us...@livy.incubator.apache.org>
Subject: Spark scala examples

Hi,

Can anyone point me to any spark-scala examples using livy as a spark jobserver? I am trying to understand how much existing spark based work (rdd, dataframe based transformations) we can leverage when using livy. seems like all but it'd be nice to see examples.

Thanks



[Image removed by sender. What's New with Xactly]<http://www.xactlycorp.com/email-click/>

[Image removed by sender.]<https://www.instagram.com/xactlycorp/>  [Image removed by sender.] <https://www.linkedin.com/company/xactly-corporation>   [Image removed by sender.] <https://twitter.com/Xactly>   [Image removed by sender.] <https://www.facebook.com/XactlyCorp>   [Image removed by sender.] <http://www.youtube.com/xactlycorporation>