You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Mahmood Naderan <nt...@yahoo.com> on 2014/03/30 11:07:47 UTC

Profiling with visualvm

Hi,
I  profiled the Mahout command with visualvm and saw many threads. Some of them are related to the profiler and some other are communication threads. Interesting thing is that, the main thread is always in sleep state!

From the thread dump (which has been attached), the owner is Mahout. Isn't that strange?



 
Regards,
Mahmood

Re: Profiling with visualvm

Posted by Mahmood Naderan <nt...@yahoo.com>.
I tried with YourKit and a CPU sampling analysis shows only three threads!

org.apache.hadoop.mapred.LocalJobRunner$Job.run() 
org.apache.mahout.driver.MahoutDriver.main(String[])
java.lang.Thread.run()


I am trying to view somthing like http://www.yourkit.com/docs/yjp2013/help/cpu_intro.jsp
Whoever tried mahout/hadoop profiling, please let us know

 
Regards,
Mahmood
On Sunday, March 30, 2014 2:30 PM, Mahmood Naderan <nt...@yahoo.com> wrote:
 
> Profiled what exactly, a Hadoop job?
As soon as I run 
/mahout testclassifier -m
wikipediamodel -d wikipediainputI see a org.apache.mahout.driver.MahoutDriver in the visualvm and then I open it.




 
Regards,
Mahmood

Re: Profiling with visualvm

Posted by Mahmood Naderan <nt...@yahoo.com>.
> Profiled what exactly, a Hadoop job?
As soon as I run 
/mahout testclassifier -m
wikipediamodel -d wikipediainputI see a org.apache.mahout.driver.MahoutDriver in the visualvm and then I open it.



 
Regards,
Mahmood

Re: Profiling with visualvm

Posted by Sean Owen <sr...@gmail.com>.
Profiled what exactly, a Hadoop job? If you profile a client, you aren't
learning anything about the work, but just that the client process is
blocked waiting for Hadoop jobs to complete.
On Mar 30, 2014 10:08 AM, "Mahmood Naderan" <nt...@yahoo.com> wrote:

> Hi,
> I  profiled the Mahout command with visualvm and saw many threads. Some of
> them are related to the profiler and some other are communication threads.
> Interesting thing is that, the main thread is always in sleep state!
>
> From the thread dump (which has been attached), the owner is Mahout. Isn't
> that strange?
>
>
> Regards,
> Mahmood
>