You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Karim Awara <ka...@kaust.edu.sa> on 2013/12/16 14:07:31 UTC
Debug Hadoop Junit Test in Eclipse
Hi,
I want to trace how a file upload (-put) happens in hadoop. So Im junit
testing TestDFSShell.java. When I try to debug the test, It fails due to
test timed out exception. I believe this is because I am trying to stop one
thread while the rest are working. I have changed the breakpoint property
to suspend VM, but still same problem.
How can I trace calls made by datanode/namenode when running
TestDFSShell.java Junit test through eclipse?
I am using hadoop 2.2.0
--
Best Regards,
Karim Ahmed Awara
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: Debug Hadoop Junit Test in Eclipse
Posted by Jay Vyas <ja...@gmail.com>.
In that case i guess you will have to statically trace the code your self.
On Mon, Dec 16, 2013 at 10:32 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
> Useful post, however, I am not trying to debug mapreduce programs with
> its associated VMs. I want to modify HDFS source code on how it uploads
> files. So I am only looking to trace fs commands through the DFS shell. I
> believe this should be require less work in debugging than actually going
> to mapred VMs!
>
>
>
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Mon, Dec 16, 2013 at 5:57 PM, Jay Vyas <ja...@gmail.com> wrote:
>
>> Excellent question. its not trivial to debug a distributed app in
>> eclipse, but it is totally doable using javaagents . We've written it up
>> here:
>>
>>
>> http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
>>
>> FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
>> with the tutorial, he might have some extra advice also (cc'd on this
>> email), I've written up one way to do this using the bigtop VMs here.
>>
>>
>>
>>
>> On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>>
>>> Hi,
>>>
>>> I want to trace how a file upload (-put) happens in hadoop. So Im junit
>>> testing TestDFSShell.java. When I try to debug the test, It fails due to
>>> test timed out exception. I believe this is because I am trying to stop one
>>> thread while the rest are working. I have changed the breakpoint property
>>> to suspend VM, but still same problem.
>>>
>>>
>>> How can I trace calls made by datanode/namenode when running
>>> TestDFSShell.java Junit test through eclipse?
>>>
>>> I am using hadoop 2.2.0
>>>
>>> --
>>> Best Regards,
>>> Karim Ahmed Awara
>>>
>>> ------------------------------
>>> This message and its contents, including attachments are intended solely
>>> for the original recipient. If you are not the intended recipient or have
>>> received this message in error, please notify me immediately and delete
>>> this message from your computer system. Any unauthorized use or
>>> distribution is prohibited. Please consider the environment before printing
>>> this email.
>>
>>
>>
>>
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>>
>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Debug Hadoop Junit Test in Eclipse
Posted by Jay Vyas <ja...@gmail.com>.
In that case i guess you will have to statically trace the code your self.
On Mon, Dec 16, 2013 at 10:32 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
> Useful post, however, I am not trying to debug mapreduce programs with
> its associated VMs. I want to modify HDFS source code on how it uploads
> files. So I am only looking to trace fs commands through the DFS shell. I
> believe this should be require less work in debugging than actually going
> to mapred VMs!
>
>
>
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Mon, Dec 16, 2013 at 5:57 PM, Jay Vyas <ja...@gmail.com> wrote:
>
>> Excellent question. its not trivial to debug a distributed app in
>> eclipse, but it is totally doable using javaagents . We've written it up
>> here:
>>
>>
>> http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
>>
>> FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
>> with the tutorial, he might have some extra advice also (cc'd on this
>> email), I've written up one way to do this using the bigtop VMs here.
>>
>>
>>
>>
>> On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>>
>>> Hi,
>>>
>>> I want to trace how a file upload (-put) happens in hadoop. So Im junit
>>> testing TestDFSShell.java. When I try to debug the test, It fails due to
>>> test timed out exception. I believe this is because I am trying to stop one
>>> thread while the rest are working. I have changed the breakpoint property
>>> to suspend VM, but still same problem.
>>>
>>>
>>> How can I trace calls made by datanode/namenode when running
>>> TestDFSShell.java Junit test through eclipse?
>>>
>>> I am using hadoop 2.2.0
>>>
>>> --
>>> Best Regards,
>>> Karim Ahmed Awara
>>>
>>> ------------------------------
>>> This message and its contents, including attachments are intended solely
>>> for the original recipient. If you are not the intended recipient or have
>>> received this message in error, please notify me immediately and delete
>>> this message from your computer system. Any unauthorized use or
>>> distribution is prohibited. Please consider the environment before printing
>>> this email.
>>
>>
>>
>>
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>>
>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Debug Hadoop Junit Test in Eclipse
Posted by Jay Vyas <ja...@gmail.com>.
In that case i guess you will have to statically trace the code your self.
On Mon, Dec 16, 2013 at 10:32 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
> Useful post, however, I am not trying to debug mapreduce programs with
> its associated VMs. I want to modify HDFS source code on how it uploads
> files. So I am only looking to trace fs commands through the DFS shell. I
> believe this should be require less work in debugging than actually going
> to mapred VMs!
>
>
>
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Mon, Dec 16, 2013 at 5:57 PM, Jay Vyas <ja...@gmail.com> wrote:
>
>> Excellent question. its not trivial to debug a distributed app in
>> eclipse, but it is totally doable using javaagents . We've written it up
>> here:
>>
>>
>> http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
>>
>> FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
>> with the tutorial, he might have some extra advice also (cc'd on this
>> email), I've written up one way to do this using the bigtop VMs here.
>>
>>
>>
>>
>> On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>>
>>> Hi,
>>>
>>> I want to trace how a file upload (-put) happens in hadoop. So Im junit
>>> testing TestDFSShell.java. When I try to debug the test, It fails due to
>>> test timed out exception. I believe this is because I am trying to stop one
>>> thread while the rest are working. I have changed the breakpoint property
>>> to suspend VM, but still same problem.
>>>
>>>
>>> How can I trace calls made by datanode/namenode when running
>>> TestDFSShell.java Junit test through eclipse?
>>>
>>> I am using hadoop 2.2.0
>>>
>>> --
>>> Best Regards,
>>> Karim Ahmed Awara
>>>
>>> ------------------------------
>>> This message and its contents, including attachments are intended solely
>>> for the original recipient. If you are not the intended recipient or have
>>> received this message in error, please notify me immediately and delete
>>> this message from your computer system. Any unauthorized use or
>>> distribution is prohibited. Please consider the environment before printing
>>> this email.
>>
>>
>>
>>
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>>
>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Debug Hadoop Junit Test in Eclipse
Posted by Jay Vyas <ja...@gmail.com>.
In that case i guess you will have to statically trace the code your self.
On Mon, Dec 16, 2013 at 10:32 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
> Useful post, however, I am not trying to debug mapreduce programs with
> its associated VMs. I want to modify HDFS source code on how it uploads
> files. So I am only looking to trace fs commands through the DFS shell. I
> believe this should be require less work in debugging than actually going
> to mapred VMs!
>
>
>
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Mon, Dec 16, 2013 at 5:57 PM, Jay Vyas <ja...@gmail.com> wrote:
>
>> Excellent question. its not trivial to debug a distributed app in
>> eclipse, but it is totally doable using javaagents . We've written it up
>> here:
>>
>>
>> http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
>>
>> FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
>> with the tutorial, he might have some extra advice also (cc'd on this
>> email), I've written up one way to do this using the bigtop VMs here.
>>
>>
>>
>>
>> On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>>
>>> Hi,
>>>
>>> I want to trace how a file upload (-put) happens in hadoop. So Im junit
>>> testing TestDFSShell.java. When I try to debug the test, It fails due to
>>> test timed out exception. I believe this is because I am trying to stop one
>>> thread while the rest are working. I have changed the breakpoint property
>>> to suspend VM, but still same problem.
>>>
>>>
>>> How can I trace calls made by datanode/namenode when running
>>> TestDFSShell.java Junit test through eclipse?
>>>
>>> I am using hadoop 2.2.0
>>>
>>> --
>>> Best Regards,
>>> Karim Ahmed Awara
>>>
>>> ------------------------------
>>> This message and its contents, including attachments are intended solely
>>> for the original recipient. If you are not the intended recipient or have
>>> received this message in error, please notify me immediately and delete
>>> this message from your computer system. Any unauthorized use or
>>> distribution is prohibited. Please consider the environment before printing
>>> this email.
>>
>>
>>
>>
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>>
>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Debug Hadoop Junit Test in Eclipse
Posted by Karim Awara <ka...@kaust.edu.sa>.
Useful post, however, I am not trying to debug mapreduce programs with its
associated VMs. I want to modify HDFS source code on how it uploads files.
So I am only looking to trace fs commands through the DFS shell. I believe
this should be require less work in debugging than actually going to mapred
VMs!
--
Best Regards,
Karim Ahmed Awara
On Mon, Dec 16, 2013 at 5:57 PM, Jay Vyas <ja...@gmail.com> wrote:
> Excellent question. its not trivial to debug a distributed app in
> eclipse, but it is totally doable using javaagents . We've written it up
> here:
>
>
> http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
>
> FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
> with the tutorial, he might have some extra advice also (cc'd on this
> email), I've written up one way to do this using the bigtop VMs here.
>
>
>
>
> On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
>> Hi,
>>
>> I want to trace how a file upload (-put) happens in hadoop. So Im junit
>> testing TestDFSShell.java. When I try to debug the test, It fails due to
>> test timed out exception. I believe this is because I am trying to stop one
>> thread while the rest are working. I have changed the breakpoint property
>> to suspend VM, but still same problem.
>>
>>
>> How can I trace calls made by datanode/namenode when running
>> TestDFSShell.java Junit test through eclipse?
>>
>> I am using hadoop 2.2.0
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: Debug Hadoop Junit Test in Eclipse
Posted by Karim Awara <ka...@kaust.edu.sa>.
Useful post, however, I am not trying to debug mapreduce programs with its
associated VMs. I want to modify HDFS source code on how it uploads files.
So I am only looking to trace fs commands through the DFS shell. I believe
this should be require less work in debugging than actually going to mapred
VMs!
--
Best Regards,
Karim Ahmed Awara
On Mon, Dec 16, 2013 at 5:57 PM, Jay Vyas <ja...@gmail.com> wrote:
> Excellent question. its not trivial to debug a distributed app in
> eclipse, but it is totally doable using javaagents . We've written it up
> here:
>
>
> http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
>
> FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
> with the tutorial, he might have some extra advice also (cc'd on this
> email), I've written up one way to do this using the bigtop VMs here.
>
>
>
>
> On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
>> Hi,
>>
>> I want to trace how a file upload (-put) happens in hadoop. So Im junit
>> testing TestDFSShell.java. When I try to debug the test, It fails due to
>> test timed out exception. I believe this is because I am trying to stop one
>> thread while the rest are working. I have changed the breakpoint property
>> to suspend VM, but still same problem.
>>
>>
>> How can I trace calls made by datanode/namenode when running
>> TestDFSShell.java Junit test through eclipse?
>>
>> I am using hadoop 2.2.0
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: Debug Hadoop Junit Test in Eclipse
Posted by Karim Awara <ka...@kaust.edu.sa>.
Useful post, however, I am not trying to debug mapreduce programs with its
associated VMs. I want to modify HDFS source code on how it uploads files.
So I am only looking to trace fs commands through the DFS shell. I believe
this should be require less work in debugging than actually going to mapred
VMs!
--
Best Regards,
Karim Ahmed Awara
On Mon, Dec 16, 2013 at 5:57 PM, Jay Vyas <ja...@gmail.com> wrote:
> Excellent question. its not trivial to debug a distributed app in
> eclipse, but it is totally doable using javaagents . We've written it up
> here:
>
>
> http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
>
> FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
> with the tutorial, he might have some extra advice also (cc'd on this
> email), I've written up one way to do this using the bigtop VMs here.
>
>
>
>
> On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
>> Hi,
>>
>> I want to trace how a file upload (-put) happens in hadoop. So Im junit
>> testing TestDFSShell.java. When I try to debug the test, It fails due to
>> test timed out exception. I believe this is because I am trying to stop one
>> thread while the rest are working. I have changed the breakpoint property
>> to suspend VM, but still same problem.
>>
>>
>> How can I trace calls made by datanode/namenode when running
>> TestDFSShell.java Junit test through eclipse?
>>
>> I am using hadoop 2.2.0
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: Debug Hadoop Junit Test in Eclipse
Posted by Karim Awara <ka...@kaust.edu.sa>.
Useful post, however, I am not trying to debug mapreduce programs with its
associated VMs. I want to modify HDFS source code on how it uploads files.
So I am only looking to trace fs commands through the DFS shell. I believe
this should be require less work in debugging than actually going to mapred
VMs!
--
Best Regards,
Karim Ahmed Awara
On Mon, Dec 16, 2013 at 5:57 PM, Jay Vyas <ja...@gmail.com> wrote:
> Excellent question. its not trivial to debug a distributed app in
> eclipse, but it is totally doable using javaagents . We've written it up
> here:
>
>
> http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
>
> FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
> with the tutorial, he might have some extra advice also (cc'd on this
> email), I've written up one way to do this using the bigtop VMs here.
>
>
>
>
> On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
>> Hi,
>>
>> I want to trace how a file upload (-put) happens in hadoop. So Im junit
>> testing TestDFSShell.java. When I try to debug the test, It fails due to
>> test timed out exception. I believe this is because I am trying to stop one
>> thread while the rest are working. I have changed the breakpoint property
>> to suspend VM, but still same problem.
>>
>>
>> How can I trace calls made by datanode/namenode when running
>> TestDFSShell.java Junit test through eclipse?
>>
>> I am using hadoop 2.2.0
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: Debug Hadoop Junit Test in Eclipse
Posted by Jay Vyas <ja...@gmail.com>.
Excellent question. its not trivial to debug a distributed app in eclipse,
but it is totally doable using javaagents . We've written it up here:
http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
with the tutorial, he might have some extra advice also (cc'd on this
email), I've written up one way to do this using the bigtop VMs here.
On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> I want to trace how a file upload (-put) happens in hadoop. So Im junit
> testing TestDFSShell.java. When I try to debug the test, It fails due to
> test timed out exception. I believe this is because I am trying to stop one
> thread while the rest are working. I have changed the breakpoint property
> to suspend VM, but still same problem.
>
>
> How can I trace calls made by datanode/namenode when running
> TestDFSShell.java Junit test through eclipse?
>
> I am using hadoop 2.2.0
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Debug Hadoop Junit Test in Eclipse
Posted by Jay Vyas <ja...@gmail.com>.
Excellent question. its not trivial to debug a distributed app in eclipse,
but it is totally doable using javaagents . We've written it up here:
http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
with the tutorial, he might have some extra advice also (cc'd on this
email), I've written up one way to do this using the bigtop VMs here.
On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> I want to trace how a file upload (-put) happens in hadoop. So Im junit
> testing TestDFSShell.java. When I try to debug the test, It fails due to
> test timed out exception. I believe this is because I am trying to stop one
> thread while the rest are working. I have changed the breakpoint property
> to suspend VM, but still same problem.
>
>
> How can I trace calls made by datanode/namenode when running
> TestDFSShell.java Junit test through eclipse?
>
> I am using hadoop 2.2.0
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Debug Hadoop Junit Test in Eclipse
Posted by Jay Vyas <ja...@gmail.com>.
Excellent question. its not trivial to debug a distributed app in eclipse,
but it is totally doable using javaagents . We've written it up here:
http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
with the tutorial, he might have some extra advice also (cc'd on this
email), I've written up one way to do this using the bigtop VMs here.
On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> I want to trace how a file upload (-put) happens in hadoop. So Im junit
> testing TestDFSShell.java. When I try to debug the test, It fails due to
> test timed out exception. I believe this is because I am trying to stop one
> thread while the rest are working. I have changed the breakpoint property
> to suspend VM, but still same problem.
>
>
> How can I trace calls made by datanode/namenode when running
> TestDFSShell.java Junit test through eclipse?
>
> I am using hadoop 2.2.0
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Debug Hadoop Junit Test in Eclipse
Posted by Jay Vyas <ja...@gmail.com>.
Excellent question. its not trivial to debug a distributed app in eclipse,
but it is totally doable using javaagents . We've written it up here:
http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
with the tutorial, he might have some extra advice also (cc'd on this
email), I've written up one way to do this using the bigtop VMs here.
On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> I want to trace how a file upload (-put) happens in hadoop. So Im junit
> testing TestDFSShell.java. When I try to debug the test, It fails due to
> test timed out exception. I believe this is because I am trying to stop one
> thread while the rest are working. I have changed the breakpoint property
> to suspend VM, but still same problem.
>
>
> How can I trace calls made by datanode/namenode when running
> TestDFSShell.java Junit test through eclipse?
>
> I am using hadoop 2.2.0
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
--
Jay Vyas
http://jayunit100.blogspot.com