You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Deep Pradhan <pr...@gmail.com> on 2015/02/06 05:28:08 UTC
Reg Job Server
Hi,
Can Spark Job Server be used for profiling Spark jobs?
Re: Reg Job Server
Posted by Deep Pradhan <pr...@gmail.com>.
I have a single node Spark standalone cluster. Will this also work for my
cluster?
Thank You
On Fri, Feb 6, 2015 at 11:02 AM, Mark Hamstra <ma...@clearstorydata.com>
wrote:
>
> https://cwiki.apache.org/confluence/display/SPARK/Profiling+Spark+Applications+Using+YourKit
>
> On Thu, Feb 5, 2015 at 9:18 PM, Deep Pradhan <pr...@gmail.com>
> wrote:
>
>> Yes, I want to know, the reason about the job being slow.
>> I will look at YourKit.
>> Can you redirect me to that, some tutorial in how to use?
>>
>> Thank You
>>
>> On Fri, Feb 6, 2015 at 10:44 AM, Kostas Sakellis <ko...@cloudera.com>
>> wrote:
>>
>>> When you say profiling, what are you trying to figure out? Why your
>>> spark job is slow? Gatling seems to be a load generating framework so I'm
>>> not sure how you'd use it (i've never used it before). Spark runs on the
>>> JVM so you can use any JVM profiling tools like YourKit.
>>>
>>> Kostas
>>>
>>> On Thu, Feb 5, 2015 at 9:03 PM, Deep Pradhan <pr...@gmail.com>
>>> wrote:
>>>
>>>> I read somewhere about Gatling. Can that be used to profile Spark jobs?
>>>>
>>>> On Fri, Feb 6, 2015 at 10:27 AM, Kostas Sakellis <ko...@cloudera.com>
>>>> wrote:
>>>>
>>>>> Which Spark Job server are you talking about?
>>>>>
>>>>> On Thu, Feb 5, 2015 at 8:28 PM, Deep Pradhan <
>>>>> pradhandeep1991@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>> Can Spark Job Server be used for profiling Spark jobs?
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Reg Job Server
Posted by Mark Hamstra <ma...@clearstorydata.com>.
https://cwiki.apache.org/confluence/display/SPARK/Profiling+Spark+Applications+Using+YourKit
On Thu, Feb 5, 2015 at 9:18 PM, Deep Pradhan <pr...@gmail.com>
wrote:
> Yes, I want to know, the reason about the job being slow.
> I will look at YourKit.
> Can you redirect me to that, some tutorial in how to use?
>
> Thank You
>
> On Fri, Feb 6, 2015 at 10:44 AM, Kostas Sakellis <ko...@cloudera.com>
> wrote:
>
>> When you say profiling, what are you trying to figure out? Why your spark
>> job is slow? Gatling seems to be a load generating framework so I'm not
>> sure how you'd use it (i've never used it before). Spark runs on the JVM so
>> you can use any JVM profiling tools like YourKit.
>>
>> Kostas
>>
>> On Thu, Feb 5, 2015 at 9:03 PM, Deep Pradhan <pr...@gmail.com>
>> wrote:
>>
>>> I read somewhere about Gatling. Can that be used to profile Spark jobs?
>>>
>>> On Fri, Feb 6, 2015 at 10:27 AM, Kostas Sakellis <ko...@cloudera.com>
>>> wrote:
>>>
>>>> Which Spark Job server are you talking about?
>>>>
>>>> On Thu, Feb 5, 2015 at 8:28 PM, Deep Pradhan <pradhandeep1991@gmail.com
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>> Can Spark Job Server be used for profiling Spark jobs?
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Reg Job Server
Posted by Deep Pradhan <pr...@gmail.com>.
Yes, I want to know, the reason about the job being slow.
I will look at YourKit.
Can you redirect me to that, some tutorial in how to use?
Thank You
On Fri, Feb 6, 2015 at 10:44 AM, Kostas Sakellis <ko...@cloudera.com>
wrote:
> When you say profiling, what are you trying to figure out? Why your spark
> job is slow? Gatling seems to be a load generating framework so I'm not
> sure how you'd use it (i've never used it before). Spark runs on the JVM so
> you can use any JVM profiling tools like YourKit.
>
> Kostas
>
> On Thu, Feb 5, 2015 at 9:03 PM, Deep Pradhan <pr...@gmail.com>
> wrote:
>
>> I read somewhere about Gatling. Can that be used to profile Spark jobs?
>>
>> On Fri, Feb 6, 2015 at 10:27 AM, Kostas Sakellis <ko...@cloudera.com>
>> wrote:
>>
>>> Which Spark Job server are you talking about?
>>>
>>> On Thu, Feb 5, 2015 at 8:28 PM, Deep Pradhan <pr...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>> Can Spark Job Server be used for profiling Spark jobs?
>>>>
>>>
>>>
>>
>
Re: Reg Job Server
Posted by Kostas Sakellis <ko...@cloudera.com>.
When you say profiling, what are you trying to figure out? Why your spark
job is slow? Gatling seems to be a load generating framework so I'm not
sure how you'd use it (i've never used it before). Spark runs on the JVM so
you can use any JVM profiling tools like YourKit.
Kostas
On Thu, Feb 5, 2015 at 9:03 PM, Deep Pradhan <pr...@gmail.com>
wrote:
> I read somewhere about Gatling. Can that be used to profile Spark jobs?
>
> On Fri, Feb 6, 2015 at 10:27 AM, Kostas Sakellis <ko...@cloudera.com>
> wrote:
>
>> Which Spark Job server are you talking about?
>>
>> On Thu, Feb 5, 2015 at 8:28 PM, Deep Pradhan <pr...@gmail.com>
>> wrote:
>>
>>> Hi,
>>> Can Spark Job Server be used for profiling Spark jobs?
>>>
>>
>>
>
Re: Reg Job Server
Posted by Deep Pradhan <pr...@gmail.com>.
I read somewhere about Gatling. Can that be used to profile Spark jobs?
On Fri, Feb 6, 2015 at 10:27 AM, Kostas Sakellis <ko...@cloudera.com>
wrote:
> Which Spark Job server are you talking about?
>
> On Thu, Feb 5, 2015 at 8:28 PM, Deep Pradhan <pr...@gmail.com>
> wrote:
>
>> Hi,
>> Can Spark Job Server be used for profiling Spark jobs?
>>
>
>
Re: Reg Job Server
Posted by Kostas Sakellis <ko...@cloudera.com>.
Which Spark Job server are you talking about?
On Thu, Feb 5, 2015 at 8:28 PM, Deep Pradhan <pr...@gmail.com>
wrote:
> Hi,
> Can Spark Job Server be used for profiling Spark jobs?
>