You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by "Md. Rezaul Karim" <re...@insight-centre.org> on 2017/02/16 22:00:10 UTC

Debugging Spark application

Hi,

I was looking for some URLs/documents for getting started on debugging
Spark applications.

I prefer developing Spark applications with Scala on Eclipse and then
package the application jar before submitting.



Kind regards,
Reza

Re: Debugging Spark application

Posted by "Md. Rezaul Karim" <re...@insight-centre.org>.

Thanks, Sam. I will have a look at it.

On Feb 16, 2017 10:06 PM, "Sam Elamin" <hu...@gmail.com> wrote:

> I recommend running spark in local mode when your first debugging your
> code just to understand what's happening and step through it, perhaps catch
> a few errors when you first start off
>
> I personally use intellij because it's my preference You can follow this
> guide.
> http://www.bigendiandata.com/2016-08-26-How-to-debug-
> remote-spark-jobs-with-IntelliJ/
>
> Although it's for intellij you can apply the same concepts to eclipse *I
> think*
>
>
> Regards
> Sam
>
>
> On Thu, 16 Feb 2017 at 22:00, Md. Rezaul Karim <
> rezaul.karim@insight-centre.org> wrote:
>
>> Hi,
>>
>> I was looking for some URLs/documents for getting started on debugging
>> Spark applications.
>>
>> I prefer developing Spark applications with Scala on Eclipse and then
>> package the application jar before submitting.
>>
>>
>>
>> Kind regards,
>> Reza
>>
>>
>>
>>
>

Re: Debugging Spark application

Posted by Sam Elamin <hu...@gmail.com>.

I recommend running spark in local mode when your first debugging your code
just to understand what's happening and step through it, perhaps catch a
few errors when you first start off

I personally use intellij because it's my preference You can follow this
guide.
http://www.bigendiandata.com/2016-08-26-How-to-debug-remote-spark-jobs-with-IntelliJ/

Although it's for intellij you can apply the same concepts to eclipse *I
think*


Regards
Sam


On Thu, 16 Feb 2017 at 22:00, Md. Rezaul Karim <
rezaul.karim@insight-centre.org> wrote:

> Hi,
>
> I was looking for some URLs/documents for getting started on debugging
> Spark applications.
>
> I prefer developing Spark applications with Scala on Eclipse and then
> package the application jar before submitting.
>
>
>
> Kind regards,
> Reza
>
>
>
>