You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Kun Ren <re...@gmail.com> on 2016/04/19 17:16:41 UTC
Eclipse debug HDFS server side code
Hi All,
Currently I used eclipse to compile/debug the source code, and I configured
the "remote Java application" to debug the source code in eclipse, for
example, I can debug the client side code when I run the command
"./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
DistributedFileSystem---->DFSClient ....->send request to the server side.
Then it finish debugging, but I can not see how the server side handle the
request.
However, I want to debug and see how the server side(HDFS) handle the
request? Do you know how I can do this? Thanks so much. Also when I try to
add some debug information(such as print out something), but I can not see
anything printed out in eclipse, I don't know why. Thanks again.
Best,
Kun
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Thanks a lot.
I will explore more related to these.
On Tue, Apr 19, 2016 at 2:37 PM, Vinayakumar B <vi...@apache.org>
wrote:
> Usually namenode console logs will be in .out file.
>
> -vinay
> On Apr 20, 2016 12:03 AM, "Kun Ren" <re...@gmail.com> wrote:
>
>> Hi Vinay,
>>
>> Thanks a lot for your quick reply.
>>
>> My current configuration is: I run the pseudo-distributed mode in my one
>> Linux machine.
>> Yes, I changed the code in eclipse first, and then re-compiled the code,
>> and then restart HDFS using the new binary, and re-debug, but still can
>> not see any output? When you said "the console of remote process", do you
>> mean the console of the terminal that I run the command?
>>
>> If I want to change some code, Could you please explain a little more
>> about how to debug/run my new modified code? Thanks so much.
>>
>>
>>
>> On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
>> wrote:
>>
>>>
>>> -Vinay
>>>
>>> ---------- Forwarded message ----------
>>> From: Vinayakumar B <vi...@apache.org>
>>> Date: Tue, Apr 19, 2016 at 11:47 PM
>>> Subject: Re: Eclipse debug HDFS server side code
>>> To: Kun Ren <re...@gmail.com>
>>>
>>>
>>> 1. Since you are debugging remote code, you can't change the code
>>> dynamically during debugging. If you want to change, put the compiled code
>>> in remote process, and restart, reconnect and debug again.
>>> 2. All the console messages will be printed on the console of remote
>>> process, not in eclipse.
>>> 3. Though you cannot change the code, but you can change/inspect the
>>> value of value of an variable in 'display' window.
>>>
>>> You can explore more related to these.
>>>
>>> -Vinay
>>>
>>> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>>>
>>>> Hi Vinay,
>>>>
>>>> Thanks a lot, it works:)
>>>>
>>>> I have another question:
>>>> 1. When I add the following lines into the public boolean mkdirs(String
>>>> src, FsPermission masked, boolean createParent) in the
>>>> NameNodeRpcServer.java,
>>>>
>>>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>>>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>>>> actual work");
>>>>
>>>> But I can not see any output either from my terminal or Eclipse console
>>>> when I step-by-step debug this function, do you know why?
>>>>
>>>> 2. When I debug the client side code, for example FsShell, the default
>>>> log level is INFO, I changed the level to DEBUG using the
>>>> http://localhost:50070/logLevel, however, it still can not print out
>>>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>>>> out debug information even I set the log level to DEBUG.
>>>>
>>>>
>>>> Thanks a lot for your help,
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <
>>>> vinayakumarb@apache.org> wrote:
>>>>
>>>>> Hi Kun Ren,
>>>>>
>>>>> You can follow the below steps.
>>>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>>>> hadoop-env.sh
>>>>> 2. Start Namenode
>>>>> 3. Now Namenode will start debug port in 3988.
>>>>> 4. Configure Remote debug application to connect to
>>>>> <namenode_host>:3988 in eclipse
>>>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you
>>>>> want to debug. NameNodeRpcServer.java is the gateway between NameNode and
>>>>> IPC Server Layer.
>>>>> 6. Now you can do step-by-step execution to observe the detailed
>>>>> executions.
>>>>>
>>>>> Happy Debugging ;)
>>>>>
>>>>> -Vinay
>>>>>
>>>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> Currently I used eclipse to compile/debug the source code, and I
>>>>>> configured the "remote Java application" to debug the source code in
>>>>>> eclipse, for example, I can debug the client side code when I run the
>>>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>>>> Then it finish debugging, but I can not see how the server side handle the
>>>>>> request.
>>>>>>
>>>>>>
>>>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>>>> add some debug information(such as print out something), but I can not see
>>>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>>>
>>>>>> Best,
>>>>>> Kun
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Thanks a lot.
I will explore more related to these.
On Tue, Apr 19, 2016 at 2:37 PM, Vinayakumar B <vi...@apache.org>
wrote:
> Usually namenode console logs will be in .out file.
>
> -vinay
> On Apr 20, 2016 12:03 AM, "Kun Ren" <re...@gmail.com> wrote:
>
>> Hi Vinay,
>>
>> Thanks a lot for your quick reply.
>>
>> My current configuration is: I run the pseudo-distributed mode in my one
>> Linux machine.
>> Yes, I changed the code in eclipse first, and then re-compiled the code,
>> and then restart HDFS using the new binary, and re-debug, but still can
>> not see any output? When you said "the console of remote process", do you
>> mean the console of the terminal that I run the command?
>>
>> If I want to change some code, Could you please explain a little more
>> about how to debug/run my new modified code? Thanks so much.
>>
>>
>>
>> On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
>> wrote:
>>
>>>
>>> -Vinay
>>>
>>> ---------- Forwarded message ----------
>>> From: Vinayakumar B <vi...@apache.org>
>>> Date: Tue, Apr 19, 2016 at 11:47 PM
>>> Subject: Re: Eclipse debug HDFS server side code
>>> To: Kun Ren <re...@gmail.com>
>>>
>>>
>>> 1. Since you are debugging remote code, you can't change the code
>>> dynamically during debugging. If you want to change, put the compiled code
>>> in remote process, and restart, reconnect and debug again.
>>> 2. All the console messages will be printed on the console of remote
>>> process, not in eclipse.
>>> 3. Though you cannot change the code, but you can change/inspect the
>>> value of value of an variable in 'display' window.
>>>
>>> You can explore more related to these.
>>>
>>> -Vinay
>>>
>>> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>>>
>>>> Hi Vinay,
>>>>
>>>> Thanks a lot, it works:)
>>>>
>>>> I have another question:
>>>> 1. When I add the following lines into the public boolean mkdirs(String
>>>> src, FsPermission masked, boolean createParent) in the
>>>> NameNodeRpcServer.java,
>>>>
>>>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>>>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>>>> actual work");
>>>>
>>>> But I can not see any output either from my terminal or Eclipse console
>>>> when I step-by-step debug this function, do you know why?
>>>>
>>>> 2. When I debug the client side code, for example FsShell, the default
>>>> log level is INFO, I changed the level to DEBUG using the
>>>> http://localhost:50070/logLevel, however, it still can not print out
>>>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>>>> out debug information even I set the log level to DEBUG.
>>>>
>>>>
>>>> Thanks a lot for your help,
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <
>>>> vinayakumarb@apache.org> wrote:
>>>>
>>>>> Hi Kun Ren,
>>>>>
>>>>> You can follow the below steps.
>>>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>>>> hadoop-env.sh
>>>>> 2. Start Namenode
>>>>> 3. Now Namenode will start debug port in 3988.
>>>>> 4. Configure Remote debug application to connect to
>>>>> <namenode_host>:3988 in eclipse
>>>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you
>>>>> want to debug. NameNodeRpcServer.java is the gateway between NameNode and
>>>>> IPC Server Layer.
>>>>> 6. Now you can do step-by-step execution to observe the detailed
>>>>> executions.
>>>>>
>>>>> Happy Debugging ;)
>>>>>
>>>>> -Vinay
>>>>>
>>>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> Currently I used eclipse to compile/debug the source code, and I
>>>>>> configured the "remote Java application" to debug the source code in
>>>>>> eclipse, for example, I can debug the client side code when I run the
>>>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>>>> Then it finish debugging, but I can not see how the server side handle the
>>>>>> request.
>>>>>>
>>>>>>
>>>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>>>> add some debug information(such as print out something), but I can not see
>>>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>>>
>>>>>> Best,
>>>>>> Kun
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Thanks a lot.
I will explore more related to these.
On Tue, Apr 19, 2016 at 2:37 PM, Vinayakumar B <vi...@apache.org>
wrote:
> Usually namenode console logs will be in .out file.
>
> -vinay
> On Apr 20, 2016 12:03 AM, "Kun Ren" <re...@gmail.com> wrote:
>
>> Hi Vinay,
>>
>> Thanks a lot for your quick reply.
>>
>> My current configuration is: I run the pseudo-distributed mode in my one
>> Linux machine.
>> Yes, I changed the code in eclipse first, and then re-compiled the code,
>> and then restart HDFS using the new binary, and re-debug, but still can
>> not see any output? When you said "the console of remote process", do you
>> mean the console of the terminal that I run the command?
>>
>> If I want to change some code, Could you please explain a little more
>> about how to debug/run my new modified code? Thanks so much.
>>
>>
>>
>> On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
>> wrote:
>>
>>>
>>> -Vinay
>>>
>>> ---------- Forwarded message ----------
>>> From: Vinayakumar B <vi...@apache.org>
>>> Date: Tue, Apr 19, 2016 at 11:47 PM
>>> Subject: Re: Eclipse debug HDFS server side code
>>> To: Kun Ren <re...@gmail.com>
>>>
>>>
>>> 1. Since you are debugging remote code, you can't change the code
>>> dynamically during debugging. If you want to change, put the compiled code
>>> in remote process, and restart, reconnect and debug again.
>>> 2. All the console messages will be printed on the console of remote
>>> process, not in eclipse.
>>> 3. Though you cannot change the code, but you can change/inspect the
>>> value of value of an variable in 'display' window.
>>>
>>> You can explore more related to these.
>>>
>>> -Vinay
>>>
>>> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>>>
>>>> Hi Vinay,
>>>>
>>>> Thanks a lot, it works:)
>>>>
>>>> I have another question:
>>>> 1. When I add the following lines into the public boolean mkdirs(String
>>>> src, FsPermission masked, boolean createParent) in the
>>>> NameNodeRpcServer.java,
>>>>
>>>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>>>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>>>> actual work");
>>>>
>>>> But I can not see any output either from my terminal or Eclipse console
>>>> when I step-by-step debug this function, do you know why?
>>>>
>>>> 2. When I debug the client side code, for example FsShell, the default
>>>> log level is INFO, I changed the level to DEBUG using the
>>>> http://localhost:50070/logLevel, however, it still can not print out
>>>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>>>> out debug information even I set the log level to DEBUG.
>>>>
>>>>
>>>> Thanks a lot for your help,
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <
>>>> vinayakumarb@apache.org> wrote:
>>>>
>>>>> Hi Kun Ren,
>>>>>
>>>>> You can follow the below steps.
>>>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>>>> hadoop-env.sh
>>>>> 2. Start Namenode
>>>>> 3. Now Namenode will start debug port in 3988.
>>>>> 4. Configure Remote debug application to connect to
>>>>> <namenode_host>:3988 in eclipse
>>>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you
>>>>> want to debug. NameNodeRpcServer.java is the gateway between NameNode and
>>>>> IPC Server Layer.
>>>>> 6. Now you can do step-by-step execution to observe the detailed
>>>>> executions.
>>>>>
>>>>> Happy Debugging ;)
>>>>>
>>>>> -Vinay
>>>>>
>>>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> Currently I used eclipse to compile/debug the source code, and I
>>>>>> configured the "remote Java application" to debug the source code in
>>>>>> eclipse, for example, I can debug the client side code when I run the
>>>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>>>> Then it finish debugging, but I can not see how the server side handle the
>>>>>> request.
>>>>>>
>>>>>>
>>>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>>>> add some debug information(such as print out something), but I can not see
>>>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>>>
>>>>>> Best,
>>>>>> Kun
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Thanks a lot.
I will explore more related to these.
On Tue, Apr 19, 2016 at 2:37 PM, Vinayakumar B <vi...@apache.org>
wrote:
> Usually namenode console logs will be in .out file.
>
> -vinay
> On Apr 20, 2016 12:03 AM, "Kun Ren" <re...@gmail.com> wrote:
>
>> Hi Vinay,
>>
>> Thanks a lot for your quick reply.
>>
>> My current configuration is: I run the pseudo-distributed mode in my one
>> Linux machine.
>> Yes, I changed the code in eclipse first, and then re-compiled the code,
>> and then restart HDFS using the new binary, and re-debug, but still can
>> not see any output? When you said "the console of remote process", do you
>> mean the console of the terminal that I run the command?
>>
>> If I want to change some code, Could you please explain a little more
>> about how to debug/run my new modified code? Thanks so much.
>>
>>
>>
>> On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
>> wrote:
>>
>>>
>>> -Vinay
>>>
>>> ---------- Forwarded message ----------
>>> From: Vinayakumar B <vi...@apache.org>
>>> Date: Tue, Apr 19, 2016 at 11:47 PM
>>> Subject: Re: Eclipse debug HDFS server side code
>>> To: Kun Ren <re...@gmail.com>
>>>
>>>
>>> 1. Since you are debugging remote code, you can't change the code
>>> dynamically during debugging. If you want to change, put the compiled code
>>> in remote process, and restart, reconnect and debug again.
>>> 2. All the console messages will be printed on the console of remote
>>> process, not in eclipse.
>>> 3. Though you cannot change the code, but you can change/inspect the
>>> value of value of an variable in 'display' window.
>>>
>>> You can explore more related to these.
>>>
>>> -Vinay
>>>
>>> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>>>
>>>> Hi Vinay,
>>>>
>>>> Thanks a lot, it works:)
>>>>
>>>> I have another question:
>>>> 1. When I add the following lines into the public boolean mkdirs(String
>>>> src, FsPermission masked, boolean createParent) in the
>>>> NameNodeRpcServer.java,
>>>>
>>>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>>>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>>>> actual work");
>>>>
>>>> But I can not see any output either from my terminal or Eclipse console
>>>> when I step-by-step debug this function, do you know why?
>>>>
>>>> 2. When I debug the client side code, for example FsShell, the default
>>>> log level is INFO, I changed the level to DEBUG using the
>>>> http://localhost:50070/logLevel, however, it still can not print out
>>>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>>>> out debug information even I set the log level to DEBUG.
>>>>
>>>>
>>>> Thanks a lot for your help,
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <
>>>> vinayakumarb@apache.org> wrote:
>>>>
>>>>> Hi Kun Ren,
>>>>>
>>>>> You can follow the below steps.
>>>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>>>> hadoop-env.sh
>>>>> 2. Start Namenode
>>>>> 3. Now Namenode will start debug port in 3988.
>>>>> 4. Configure Remote debug application to connect to
>>>>> <namenode_host>:3988 in eclipse
>>>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you
>>>>> want to debug. NameNodeRpcServer.java is the gateway between NameNode and
>>>>> IPC Server Layer.
>>>>> 6. Now you can do step-by-step execution to observe the detailed
>>>>> executions.
>>>>>
>>>>> Happy Debugging ;)
>>>>>
>>>>> -Vinay
>>>>>
>>>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> Currently I used eclipse to compile/debug the source code, and I
>>>>>> configured the "remote Java application" to debug the source code in
>>>>>> eclipse, for example, I can debug the client side code when I run the
>>>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>>>> Then it finish debugging, but I can not see how the server side handle the
>>>>>> request.
>>>>>>
>>>>>>
>>>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>>>> add some debug information(such as print out something), but I can not see
>>>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>>>
>>>>>> Best,
>>>>>> Kun
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
Re: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
Usually namenode console logs will be in .out file.
-vinay
On Apr 20, 2016 12:03 AM, "Kun Ren" <re...@gmail.com> wrote:
> Hi Vinay,
>
> Thanks a lot for your quick reply.
>
> My current configuration is: I run the pseudo-distributed mode in my one
> Linux machine.
> Yes, I changed the code in eclipse first, and then re-compiled the code,
> and then restart HDFS using the new binary, and re-debug, but still can
> not see any output? When you said "the console of remote process", do you
> mean the console of the terminal that I run the command?
>
> If I want to change some code, Could you please explain a little more
> about how to debug/run my new modified code? Thanks so much.
>
>
>
> On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
> wrote:
>
>>
>> -Vinay
>>
>> ---------- Forwarded message ----------
>> From: Vinayakumar B <vi...@apache.org>
>> Date: Tue, Apr 19, 2016 at 11:47 PM
>> Subject: Re: Eclipse debug HDFS server side code
>> To: Kun Ren <re...@gmail.com>
>>
>>
>> 1. Since you are debugging remote code, you can't change the code
>> dynamically during debugging. If you want to change, put the compiled code
>> in remote process, and restart, reconnect and debug again.
>> 2. All the console messages will be printed on the console of remote
>> process, not in eclipse.
>> 3. Though you cannot change the code, but you can change/inspect the
>> value of value of an variable in 'display' window.
>>
>> You can explore more related to these.
>>
>> -Vinay
>>
>> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>>
>>> Hi Vinay,
>>>
>>> Thanks a lot, it works:)
>>>
>>> I have another question:
>>> 1. When I add the following lines into the public boolean mkdirs(String
>>> src, FsPermission masked, boolean createParent) in the
>>> NameNodeRpcServer.java,
>>>
>>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>>> actual work");
>>>
>>> But I can not see any output either from my terminal or Eclipse console
>>> when I step-by-step debug this function, do you know why?
>>>
>>> 2. When I debug the client side code, for example FsShell, the default
>>> log level is INFO, I changed the level to DEBUG using the
>>> http://localhost:50070/logLevel, however, it still can not print out
>>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>>> out debug information even I set the log level to DEBUG.
>>>
>>>
>>> Thanks a lot for your help,
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vinayakumarb@apache.org
>>> > wrote:
>>>
>>>> Hi Kun Ren,
>>>>
>>>> You can follow the below steps.
>>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>>> hadoop-env.sh
>>>> 2. Start Namenode
>>>> 3. Now Namenode will start debug port in 3988.
>>>> 4. Configure Remote debug application to connect to
>>>> <namenode_host>:3988 in eclipse
>>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>>>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>>>> Server Layer.
>>>> 6. Now you can do step-by-step execution to observe the detailed
>>>> executions.
>>>>
>>>> Happy Debugging ;)
>>>>
>>>> -Vinay
>>>>
>>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> Currently I used eclipse to compile/debug the source code, and I
>>>>> configured the "remote Java application" to debug the source code in
>>>>> eclipse, for example, I can debug the client side code when I run the
>>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>>> Then it finish debugging, but I can not see how the server side handle the
>>>>> request.
>>>>>
>>>>>
>>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>>> add some debug information(such as print out something), but I can not see
>>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>>
>>>>> Best,
>>>>> Kun
>>>>>
>>>>
>>>>
>>>
>>
>>
>
Re: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
Usually namenode console logs will be in .out file.
-vinay
On Apr 20, 2016 12:03 AM, "Kun Ren" <re...@gmail.com> wrote:
> Hi Vinay,
>
> Thanks a lot for your quick reply.
>
> My current configuration is: I run the pseudo-distributed mode in my one
> Linux machine.
> Yes, I changed the code in eclipse first, and then re-compiled the code,
> and then restart HDFS using the new binary, and re-debug, but still can
> not see any output? When you said "the console of remote process", do you
> mean the console of the terminal that I run the command?
>
> If I want to change some code, Could you please explain a little more
> about how to debug/run my new modified code? Thanks so much.
>
>
>
> On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
> wrote:
>
>>
>> -Vinay
>>
>> ---------- Forwarded message ----------
>> From: Vinayakumar B <vi...@apache.org>
>> Date: Tue, Apr 19, 2016 at 11:47 PM
>> Subject: Re: Eclipse debug HDFS server side code
>> To: Kun Ren <re...@gmail.com>
>>
>>
>> 1. Since you are debugging remote code, you can't change the code
>> dynamically during debugging. If you want to change, put the compiled code
>> in remote process, and restart, reconnect and debug again.
>> 2. All the console messages will be printed on the console of remote
>> process, not in eclipse.
>> 3. Though you cannot change the code, but you can change/inspect the
>> value of value of an variable in 'display' window.
>>
>> You can explore more related to these.
>>
>> -Vinay
>>
>> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>>
>>> Hi Vinay,
>>>
>>> Thanks a lot, it works:)
>>>
>>> I have another question:
>>> 1. When I add the following lines into the public boolean mkdirs(String
>>> src, FsPermission masked, boolean createParent) in the
>>> NameNodeRpcServer.java,
>>>
>>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>>> actual work");
>>>
>>> But I can not see any output either from my terminal or Eclipse console
>>> when I step-by-step debug this function, do you know why?
>>>
>>> 2. When I debug the client side code, for example FsShell, the default
>>> log level is INFO, I changed the level to DEBUG using the
>>> http://localhost:50070/logLevel, however, it still can not print out
>>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>>> out debug information even I set the log level to DEBUG.
>>>
>>>
>>> Thanks a lot for your help,
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vinayakumarb@apache.org
>>> > wrote:
>>>
>>>> Hi Kun Ren,
>>>>
>>>> You can follow the below steps.
>>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>>> hadoop-env.sh
>>>> 2. Start Namenode
>>>> 3. Now Namenode will start debug port in 3988.
>>>> 4. Configure Remote debug application to connect to
>>>> <namenode_host>:3988 in eclipse
>>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>>>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>>>> Server Layer.
>>>> 6. Now you can do step-by-step execution to observe the detailed
>>>> executions.
>>>>
>>>> Happy Debugging ;)
>>>>
>>>> -Vinay
>>>>
>>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> Currently I used eclipse to compile/debug the source code, and I
>>>>> configured the "remote Java application" to debug the source code in
>>>>> eclipse, for example, I can debug the client side code when I run the
>>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>>> Then it finish debugging, but I can not see how the server side handle the
>>>>> request.
>>>>>
>>>>>
>>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>>> add some debug information(such as print out something), but I can not see
>>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>>
>>>>> Best,
>>>>> Kun
>>>>>
>>>>
>>>>
>>>
>>
>>
>
Re: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
Usually namenode console logs will be in .out file.
-vinay
On Apr 20, 2016 12:03 AM, "Kun Ren" <re...@gmail.com> wrote:
> Hi Vinay,
>
> Thanks a lot for your quick reply.
>
> My current configuration is: I run the pseudo-distributed mode in my one
> Linux machine.
> Yes, I changed the code in eclipse first, and then re-compiled the code,
> and then restart HDFS using the new binary, and re-debug, but still can
> not see any output? When you said "the console of remote process", do you
> mean the console of the terminal that I run the command?
>
> If I want to change some code, Could you please explain a little more
> about how to debug/run my new modified code? Thanks so much.
>
>
>
> On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
> wrote:
>
>>
>> -Vinay
>>
>> ---------- Forwarded message ----------
>> From: Vinayakumar B <vi...@apache.org>
>> Date: Tue, Apr 19, 2016 at 11:47 PM
>> Subject: Re: Eclipse debug HDFS server side code
>> To: Kun Ren <re...@gmail.com>
>>
>>
>> 1. Since you are debugging remote code, you can't change the code
>> dynamically during debugging. If you want to change, put the compiled code
>> in remote process, and restart, reconnect and debug again.
>> 2. All the console messages will be printed on the console of remote
>> process, not in eclipse.
>> 3. Though you cannot change the code, but you can change/inspect the
>> value of value of an variable in 'display' window.
>>
>> You can explore more related to these.
>>
>> -Vinay
>>
>> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>>
>>> Hi Vinay,
>>>
>>> Thanks a lot, it works:)
>>>
>>> I have another question:
>>> 1. When I add the following lines into the public boolean mkdirs(String
>>> src, FsPermission masked, boolean createParent) in the
>>> NameNodeRpcServer.java,
>>>
>>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>>> actual work");
>>>
>>> But I can not see any output either from my terminal or Eclipse console
>>> when I step-by-step debug this function, do you know why?
>>>
>>> 2. When I debug the client side code, for example FsShell, the default
>>> log level is INFO, I changed the level to DEBUG using the
>>> http://localhost:50070/logLevel, however, it still can not print out
>>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>>> out debug information even I set the log level to DEBUG.
>>>
>>>
>>> Thanks a lot for your help,
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vinayakumarb@apache.org
>>> > wrote:
>>>
>>>> Hi Kun Ren,
>>>>
>>>> You can follow the below steps.
>>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>>> hadoop-env.sh
>>>> 2. Start Namenode
>>>> 3. Now Namenode will start debug port in 3988.
>>>> 4. Configure Remote debug application to connect to
>>>> <namenode_host>:3988 in eclipse
>>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>>>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>>>> Server Layer.
>>>> 6. Now you can do step-by-step execution to observe the detailed
>>>> executions.
>>>>
>>>> Happy Debugging ;)
>>>>
>>>> -Vinay
>>>>
>>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> Currently I used eclipse to compile/debug the source code, and I
>>>>> configured the "remote Java application" to debug the source code in
>>>>> eclipse, for example, I can debug the client side code when I run the
>>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>>> Then it finish debugging, but I can not see how the server side handle the
>>>>> request.
>>>>>
>>>>>
>>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>>> add some debug information(such as print out something), but I can not see
>>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>>
>>>>> Best,
>>>>> Kun
>>>>>
>>>>
>>>>
>>>
>>
>>
>
Re: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
Usually namenode console logs will be in .out file.
-vinay
On Apr 20, 2016 12:03 AM, "Kun Ren" <re...@gmail.com> wrote:
> Hi Vinay,
>
> Thanks a lot for your quick reply.
>
> My current configuration is: I run the pseudo-distributed mode in my one
> Linux machine.
> Yes, I changed the code in eclipse first, and then re-compiled the code,
> and then restart HDFS using the new binary, and re-debug, but still can
> not see any output? When you said "the console of remote process", do you
> mean the console of the terminal that I run the command?
>
> If I want to change some code, Could you please explain a little more
> about how to debug/run my new modified code? Thanks so much.
>
>
>
> On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
> wrote:
>
>>
>> -Vinay
>>
>> ---------- Forwarded message ----------
>> From: Vinayakumar B <vi...@apache.org>
>> Date: Tue, Apr 19, 2016 at 11:47 PM
>> Subject: Re: Eclipse debug HDFS server side code
>> To: Kun Ren <re...@gmail.com>
>>
>>
>> 1. Since you are debugging remote code, you can't change the code
>> dynamically during debugging. If you want to change, put the compiled code
>> in remote process, and restart, reconnect and debug again.
>> 2. All the console messages will be printed on the console of remote
>> process, not in eclipse.
>> 3. Though you cannot change the code, but you can change/inspect the
>> value of value of an variable in 'display' window.
>>
>> You can explore more related to these.
>>
>> -Vinay
>>
>> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>>
>>> Hi Vinay,
>>>
>>> Thanks a lot, it works:)
>>>
>>> I have another question:
>>> 1. When I add the following lines into the public boolean mkdirs(String
>>> src, FsPermission masked, boolean createParent) in the
>>> NameNodeRpcServer.java,
>>>
>>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>>> actual work");
>>>
>>> But I can not see any output either from my terminal or Eclipse console
>>> when I step-by-step debug this function, do you know why?
>>>
>>> 2. When I debug the client side code, for example FsShell, the default
>>> log level is INFO, I changed the level to DEBUG using the
>>> http://localhost:50070/logLevel, however, it still can not print out
>>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>>> out debug information even I set the log level to DEBUG.
>>>
>>>
>>> Thanks a lot for your help,
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vinayakumarb@apache.org
>>> > wrote:
>>>
>>>> Hi Kun Ren,
>>>>
>>>> You can follow the below steps.
>>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>>> hadoop-env.sh
>>>> 2. Start Namenode
>>>> 3. Now Namenode will start debug port in 3988.
>>>> 4. Configure Remote debug application to connect to
>>>> <namenode_host>:3988 in eclipse
>>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>>>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>>>> Server Layer.
>>>> 6. Now you can do step-by-step execution to observe the detailed
>>>> executions.
>>>>
>>>> Happy Debugging ;)
>>>>
>>>> -Vinay
>>>>
>>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> Currently I used eclipse to compile/debug the source code, and I
>>>>> configured the "remote Java application" to debug the source code in
>>>>> eclipse, for example, I can debug the client side code when I run the
>>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>>> Then it finish debugging, but I can not see how the server side handle the
>>>>> request.
>>>>>
>>>>>
>>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>>> add some debug information(such as print out something), but I can not see
>>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>>
>>>>> Best,
>>>>> Kun
>>>>>
>>>>
>>>>
>>>
>>
>>
>
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Hi Vinay,
Thanks a lot for your quick reply.
My current configuration is: I run the pseudo-distributed mode in my one
Linux machine.
Yes, I changed the code in eclipse first, and then re-compiled the code,
and then restart HDFS using the new binary, and re-debug, but still can
not see any output? When you said "the console of remote process", do you
mean the console of the terminal that I run the command?
If I want to change some code, Could you please explain a little more about
how to debug/run my new modified code? Thanks so much.
On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
wrote:
>
> -Vinay
>
> ---------- Forwarded message ----------
> From: Vinayakumar B <vi...@apache.org>
> Date: Tue, Apr 19, 2016 at 11:47 PM
> Subject: Re: Eclipse debug HDFS server side code
> To: Kun Ren <re...@gmail.com>
>
>
> 1. Since you are debugging remote code, you can't change the code
> dynamically during debugging. If you want to change, put the compiled code
> in remote process, and restart, reconnect and debug again.
> 2. All the console messages will be printed on the console of remote
> process, not in eclipse.
> 3. Though you cannot change the code, but you can change/inspect the value
> of value of an variable in 'display' window.
>
> You can explore more related to these.
>
> -Vinay
>
> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>
>> Hi Vinay,
>>
>> Thanks a lot, it works:)
>>
>> I have another question:
>> 1. When I add the following lines into the public boolean mkdirs(String
>> src, FsPermission masked, boolean createParent) in the
>> NameNodeRpcServer.java,
>>
>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>> actual work");
>>
>> But I can not see any output either from my terminal or Eclipse console
>> when I step-by-step debug this function, do you know why?
>>
>> 2. When I debug the client side code, for example FsShell, the default
>> log level is INFO, I changed the level to DEBUG using the
>> http://localhost:50070/logLevel, however, it still can not print out
>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>> out debug information even I set the log level to DEBUG.
>>
>>
>> Thanks a lot for your help,
>>
>>
>>
>>
>>
>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
>> wrote:
>>
>>> Hi Kun Ren,
>>>
>>> You can follow the below steps.
>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>> hadoop-env.sh
>>> 2. Start Namenode
>>> 3. Now Namenode will start debug port in 3988.
>>> 4. Configure Remote debug application to connect to <namenode_host>:3988
>>> in eclipse
>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>>> Server Layer.
>>> 6. Now you can do step-by-step execution to observe the detailed
>>> executions.
>>>
>>> Happy Debugging ;)
>>>
>>> -Vinay
>>>
>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> Currently I used eclipse to compile/debug the source code, and I
>>>> configured the "remote Java application" to debug the source code in
>>>> eclipse, for example, I can debug the client side code when I run the
>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>> Then it finish debugging, but I can not see how the server side handle the
>>>> request.
>>>>
>>>>
>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>> add some debug information(such as print out something), but I can not see
>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>
>>>> Best,
>>>> Kun
>>>>
>>>
>>>
>>
>
>
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Hi Vinay,
Thanks a lot for your quick reply.
My current configuration is: I run the pseudo-distributed mode in my one
Linux machine.
Yes, I changed the code in eclipse first, and then re-compiled the code,
and then restart HDFS using the new binary, and re-debug, but still can
not see any output? When you said "the console of remote process", do you
mean the console of the terminal that I run the command?
If I want to change some code, Could you please explain a little more about
how to debug/run my new modified code? Thanks so much.
On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
wrote:
>
> -Vinay
>
> ---------- Forwarded message ----------
> From: Vinayakumar B <vi...@apache.org>
> Date: Tue, Apr 19, 2016 at 11:47 PM
> Subject: Re: Eclipse debug HDFS server side code
> To: Kun Ren <re...@gmail.com>
>
>
> 1. Since you are debugging remote code, you can't change the code
> dynamically during debugging. If you want to change, put the compiled code
> in remote process, and restart, reconnect and debug again.
> 2. All the console messages will be printed on the console of remote
> process, not in eclipse.
> 3. Though you cannot change the code, but you can change/inspect the value
> of value of an variable in 'display' window.
>
> You can explore more related to these.
>
> -Vinay
>
> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>
>> Hi Vinay,
>>
>> Thanks a lot, it works:)
>>
>> I have another question:
>> 1. When I add the following lines into the public boolean mkdirs(String
>> src, FsPermission masked, boolean createParent) in the
>> NameNodeRpcServer.java,
>>
>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>> actual work");
>>
>> But I can not see any output either from my terminal or Eclipse console
>> when I step-by-step debug this function, do you know why?
>>
>> 2. When I debug the client side code, for example FsShell, the default
>> log level is INFO, I changed the level to DEBUG using the
>> http://localhost:50070/logLevel, however, it still can not print out
>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>> out debug information even I set the log level to DEBUG.
>>
>>
>> Thanks a lot for your help,
>>
>>
>>
>>
>>
>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
>> wrote:
>>
>>> Hi Kun Ren,
>>>
>>> You can follow the below steps.
>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>> hadoop-env.sh
>>> 2. Start Namenode
>>> 3. Now Namenode will start debug port in 3988.
>>> 4. Configure Remote debug application to connect to <namenode_host>:3988
>>> in eclipse
>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>>> Server Layer.
>>> 6. Now you can do step-by-step execution to observe the detailed
>>> executions.
>>>
>>> Happy Debugging ;)
>>>
>>> -Vinay
>>>
>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> Currently I used eclipse to compile/debug the source code, and I
>>>> configured the "remote Java application" to debug the source code in
>>>> eclipse, for example, I can debug the client side code when I run the
>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>> Then it finish debugging, but I can not see how the server side handle the
>>>> request.
>>>>
>>>>
>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>> add some debug information(such as print out something), but I can not see
>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>
>>>> Best,
>>>> Kun
>>>>
>>>
>>>
>>
>
>
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Hi Vinay,
Thanks a lot for your quick reply.
My current configuration is: I run the pseudo-distributed mode in my one
Linux machine.
Yes, I changed the code in eclipse first, and then re-compiled the code,
and then restart HDFS using the new binary, and re-debug, but still can
not see any output? When you said "the console of remote process", do you
mean the console of the terminal that I run the command?
If I want to change some code, Could you please explain a little more about
how to debug/run my new modified code? Thanks so much.
On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
wrote:
>
> -Vinay
>
> ---------- Forwarded message ----------
> From: Vinayakumar B <vi...@apache.org>
> Date: Tue, Apr 19, 2016 at 11:47 PM
> Subject: Re: Eclipse debug HDFS server side code
> To: Kun Ren <re...@gmail.com>
>
>
> 1. Since you are debugging remote code, you can't change the code
> dynamically during debugging. If you want to change, put the compiled code
> in remote process, and restart, reconnect and debug again.
> 2. All the console messages will be printed on the console of remote
> process, not in eclipse.
> 3. Though you cannot change the code, but you can change/inspect the value
> of value of an variable in 'display' window.
>
> You can explore more related to these.
>
> -Vinay
>
> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>
>> Hi Vinay,
>>
>> Thanks a lot, it works:)
>>
>> I have another question:
>> 1. When I add the following lines into the public boolean mkdirs(String
>> src, FsPermission masked, boolean createParent) in the
>> NameNodeRpcServer.java,
>>
>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>> actual work");
>>
>> But I can not see any output either from my terminal or Eclipse console
>> when I step-by-step debug this function, do you know why?
>>
>> 2. When I debug the client side code, for example FsShell, the default
>> log level is INFO, I changed the level to DEBUG using the
>> http://localhost:50070/logLevel, however, it still can not print out
>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>> out debug information even I set the log level to DEBUG.
>>
>>
>> Thanks a lot for your help,
>>
>>
>>
>>
>>
>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
>> wrote:
>>
>>> Hi Kun Ren,
>>>
>>> You can follow the below steps.
>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>> hadoop-env.sh
>>> 2. Start Namenode
>>> 3. Now Namenode will start debug port in 3988.
>>> 4. Configure Remote debug application to connect to <namenode_host>:3988
>>> in eclipse
>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>>> Server Layer.
>>> 6. Now you can do step-by-step execution to observe the detailed
>>> executions.
>>>
>>> Happy Debugging ;)
>>>
>>> -Vinay
>>>
>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> Currently I used eclipse to compile/debug the source code, and I
>>>> configured the "remote Java application" to debug the source code in
>>>> eclipse, for example, I can debug the client side code when I run the
>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>> Then it finish debugging, but I can not see how the server side handle the
>>>> request.
>>>>
>>>>
>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>> add some debug information(such as print out something), but I can not see
>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>
>>>> Best,
>>>> Kun
>>>>
>>>
>>>
>>
>
>
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Hi Vinay,
Thanks a lot for your quick reply.
My current configuration is: I run the pseudo-distributed mode in my one
Linux machine.
Yes, I changed the code in eclipse first, and then re-compiled the code,
and then restart HDFS using the new binary, and re-debug, but still can
not see any output? When you said "the console of remote process", do you
mean the console of the terminal that I run the command?
If I want to change some code, Could you please explain a little more about
how to debug/run my new modified code? Thanks so much.
On Tue, Apr 19, 2016 at 2:17 PM, Vinayakumar B <vi...@apache.org>
wrote:
>
> -Vinay
>
> ---------- Forwarded message ----------
> From: Vinayakumar B <vi...@apache.org>
> Date: Tue, Apr 19, 2016 at 11:47 PM
> Subject: Re: Eclipse debug HDFS server side code
> To: Kun Ren <re...@gmail.com>
>
>
> 1. Since you are debugging remote code, you can't change the code
> dynamically during debugging. If you want to change, put the compiled code
> in remote process, and restart, reconnect and debug again.
> 2. All the console messages will be printed on the console of remote
> process, not in eclipse.
> 3. Though you cannot change the code, but you can change/inspect the value
> of value of an variable in 'display' window.
>
> You can explore more related to these.
>
> -Vinay
>
> On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
>
>> Hi Vinay,
>>
>> Thanks a lot, it works:)
>>
>> I have another question:
>> 1. When I add the following lines into the public boolean mkdirs(String
>> src, FsPermission masked, boolean createParent) in the
>> NameNodeRpcServer.java,
>>
>> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
>> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
>> actual work");
>>
>> But I can not see any output either from my terminal or Eclipse console
>> when I step-by-step debug this function, do you know why?
>>
>> 2. When I debug the client side code, for example FsShell, the default
>> log level is INFO, I changed the level to DEBUG using the
>> http://localhost:50070/logLevel, however, it still can not print out
>> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
>> out debug information even I set the log level to DEBUG.
>>
>>
>> Thanks a lot for your help,
>>
>>
>>
>>
>>
>> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
>> wrote:
>>
>>> Hi Kun Ren,
>>>
>>> You can follow the below steps.
>>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>>> hadoop-env.sh
>>> 2. Start Namenode
>>> 3. Now Namenode will start debug port in 3988.
>>> 4. Configure Remote debug application to connect to <namenode_host>:3988
>>> in eclipse
>>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>>> Server Layer.
>>> 6. Now you can do step-by-step execution to observe the detailed
>>> executions.
>>>
>>> Happy Debugging ;)
>>>
>>> -Vinay
>>>
>>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> Currently I used eclipse to compile/debug the source code, and I
>>>> configured the "remote Java application" to debug the source code in
>>>> eclipse, for example, I can debug the client side code when I run the
>>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>>> Then it finish debugging, but I can not see how the server side handle the
>>>> request.
>>>>
>>>>
>>>> However, I want to debug and see how the server side(HDFS) handle the
>>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>>> add some debug information(such as print out something), but I can not see
>>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>>
>>>> Best,
>>>> Kun
>>>>
>>>
>>>
>>
>
>
Fwd: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
-Vinay
---------- Forwarded message ----------
From: Vinayakumar B <vi...@apache.org>
Date: Tue, Apr 19, 2016 at 11:47 PM
Subject: Re: Eclipse debug HDFS server side code
To: Kun Ren <re...@gmail.com>
1. Since you are debugging remote code, you can't change the code
dynamically during debugging. If you want to change, put the compiled code
in remote process, and restart, reconnect and debug again.
2. All the console messages will be printed on the console of remote
process, not in eclipse.
3. Though you cannot change the code, but you can change/inspect the value
of value of an variable in 'display' window.
You can explore more related to these.
-Vinay
On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
> Hi Vinay,
>
> Thanks a lot, it works:)
>
> I have another question:
> 1. When I add the following lines into the public boolean mkdirs(String
> src, FsPermission masked, boolean createParent) in the
> NameNodeRpcServer.java,
>
> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
> actual work");
>
> But I can not see any output either from my terminal or Eclipse console
> when I step-by-step debug this function, do you know why?
>
> 2. When I debug the client side code, for example FsShell, the default
> log level is INFO, I changed the level to DEBUG using the
> http://localhost:50070/logLevel, however, it still can not print out
> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
> out debug information even I set the log level to DEBUG.
>
>
> Thanks a lot for your help,
>
>
>
>
>
> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
> wrote:
>
>> Hi Kun Ren,
>>
>> You can follow the below steps.
>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>> hadoop-env.sh
>> 2. Start Namenode
>> 3. Now Namenode will start debug port in 3988.
>> 4. Configure Remote debug application to connect to <namenode_host>:3988
>> in eclipse
>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>> Server Layer.
>> 6. Now you can do step-by-step execution to observe the detailed
>> executions.
>>
>> Happy Debugging ;)
>>
>> -Vinay
>>
>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Currently I used eclipse to compile/debug the source code, and I
>>> configured the "remote Java application" to debug the source code in
>>> eclipse, for example, I can debug the client side code when I run the
>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>> Then it finish debugging, but I can not see how the server side handle the
>>> request.
>>>
>>>
>>> However, I want to debug and see how the server side(HDFS) handle the
>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>> add some debug information(such as print out something), but I can not see
>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>
>>> Best,
>>> Kun
>>>
>>
>>
>
Fwd: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
-Vinay
---------- Forwarded message ----------
From: Vinayakumar B <vi...@apache.org>
Date: Tue, Apr 19, 2016 at 11:47 PM
Subject: Re: Eclipse debug HDFS server side code
To: Kun Ren <re...@gmail.com>
1. Since you are debugging remote code, you can't change the code
dynamically during debugging. If you want to change, put the compiled code
in remote process, and restart, reconnect and debug again.
2. All the console messages will be printed on the console of remote
process, not in eclipse.
3. Though you cannot change the code, but you can change/inspect the value
of value of an variable in 'display' window.
You can explore more related to these.
-Vinay
On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
> Hi Vinay,
>
> Thanks a lot, it works:)
>
> I have another question:
> 1. When I add the following lines into the public boolean mkdirs(String
> src, FsPermission masked, boolean createParent) in the
> NameNodeRpcServer.java,
>
> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
> actual work");
>
> But I can not see any output either from my terminal or Eclipse console
> when I step-by-step debug this function, do you know why?
>
> 2. When I debug the client side code, for example FsShell, the default
> log level is INFO, I changed the level to DEBUG using the
> http://localhost:50070/logLevel, however, it still can not print out
> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
> out debug information even I set the log level to DEBUG.
>
>
> Thanks a lot for your help,
>
>
>
>
>
> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
> wrote:
>
>> Hi Kun Ren,
>>
>> You can follow the below steps.
>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>> hadoop-env.sh
>> 2. Start Namenode
>> 3. Now Namenode will start debug port in 3988.
>> 4. Configure Remote debug application to connect to <namenode_host>:3988
>> in eclipse
>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>> Server Layer.
>> 6. Now you can do step-by-step execution to observe the detailed
>> executions.
>>
>> Happy Debugging ;)
>>
>> -Vinay
>>
>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Currently I used eclipse to compile/debug the source code, and I
>>> configured the "remote Java application" to debug the source code in
>>> eclipse, for example, I can debug the client side code when I run the
>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>> Then it finish debugging, but I can not see how the server side handle the
>>> request.
>>>
>>>
>>> However, I want to debug and see how the server side(HDFS) handle the
>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>> add some debug information(such as print out something), but I can not see
>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>
>>> Best,
>>> Kun
>>>
>>
>>
>
Fwd: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
-Vinay
---------- Forwarded message ----------
From: Vinayakumar B <vi...@apache.org>
Date: Tue, Apr 19, 2016 at 11:47 PM
Subject: Re: Eclipse debug HDFS server side code
To: Kun Ren <re...@gmail.com>
1. Since you are debugging remote code, you can't change the code
dynamically during debugging. If you want to change, put the compiled code
in remote process, and restart, reconnect and debug again.
2. All the console messages will be printed on the console of remote
process, not in eclipse.
3. Though you cannot change the code, but you can change/inspect the value
of value of an variable in 'display' window.
You can explore more related to these.
-Vinay
On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
> Hi Vinay,
>
> Thanks a lot, it works:)
>
> I have another question:
> 1. When I add the following lines into the public boolean mkdirs(String
> src, FsPermission masked, boolean createParent) in the
> NameNodeRpcServer.java,
>
> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
> actual work");
>
> But I can not see any output either from my terminal or Eclipse console
> when I step-by-step debug this function, do you know why?
>
> 2. When I debug the client side code, for example FsShell, the default
> log level is INFO, I changed the level to DEBUG using the
> http://localhost:50070/logLevel, however, it still can not print out
> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
> out debug information even I set the log level to DEBUG.
>
>
> Thanks a lot for your help,
>
>
>
>
>
> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
> wrote:
>
>> Hi Kun Ren,
>>
>> You can follow the below steps.
>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>> hadoop-env.sh
>> 2. Start Namenode
>> 3. Now Namenode will start debug port in 3988.
>> 4. Configure Remote debug application to connect to <namenode_host>:3988
>> in eclipse
>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>> Server Layer.
>> 6. Now you can do step-by-step execution to observe the detailed
>> executions.
>>
>> Happy Debugging ;)
>>
>> -Vinay
>>
>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Currently I used eclipse to compile/debug the source code, and I
>>> configured the "remote Java application" to debug the source code in
>>> eclipse, for example, I can debug the client side code when I run the
>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>> Then it finish debugging, but I can not see how the server side handle the
>>> request.
>>>
>>>
>>> However, I want to debug and see how the server side(HDFS) handle the
>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>> add some debug information(such as print out something), but I can not see
>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>
>>> Best,
>>> Kun
>>>
>>
>>
>
Fwd: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
-Vinay
---------- Forwarded message ----------
From: Vinayakumar B <vi...@apache.org>
Date: Tue, Apr 19, 2016 at 11:47 PM
Subject: Re: Eclipse debug HDFS server side code
To: Kun Ren <re...@gmail.com>
1. Since you are debugging remote code, you can't change the code
dynamically during debugging. If you want to change, put the compiled code
in remote process, and restart, reconnect and debug again.
2. All the console messages will be printed on the console of remote
process, not in eclipse.
3. Though you cannot change the code, but you can change/inspect the value
of value of an variable in 'display' window.
You can explore more related to these.
-Vinay
On Tue, Apr 19, 2016 at 11:42 PM, Kun Ren <re...@gmail.com> wrote:
> Hi Vinay,
>
> Thanks a lot, it works:)
>
> I have another question:
> 1. When I add the following lines into the public boolean mkdirs(String
> src, FsPermission masked, boolean createParent) in the
> NameNodeRpcServer.java,
>
> LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
> System.out.println("In NameNodeRpcServer.java:mkdirs, before do
> actual work");
>
> But I can not see any output either from my terminal or Eclipse console
> when I step-by-step debug this function, do you know why?
>
> 2. When I debug the client side code, for example FsShell, the default
> log level is INFO, I changed the level to DEBUG using the
> http://localhost:50070/logLevel, however, it still can not print out
> LOG.debug(), but I can see LOG.info(), I don't know why it can not print
> out debug information even I set the log level to DEBUG.
>
>
> Thanks a lot for your help,
>
>
>
>
>
> On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
> wrote:
>
>> Hi Kun Ren,
>>
>> You can follow the below steps.
>> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
>> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
>> hadoop-env.sh
>> 2. Start Namenode
>> 3. Now Namenode will start debug port in 3988.
>> 4. Configure Remote debug application to connect to <namenode_host>:3988
>> in eclipse
>> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want
>> to debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
>> Server Layer.
>> 6. Now you can do step-by-step execution to observe the detailed
>> executions.
>>
>> Happy Debugging ;)
>>
>> -Vinay
>>
>> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> Currently I used eclipse to compile/debug the source code, and I
>>> configured the "remote Java application" to debug the source code in
>>> eclipse, for example, I can debug the client side code when I run the
>>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>>> Then it finish debugging, but I can not see how the server side handle the
>>> request.
>>>
>>>
>>> However, I want to debug and see how the server side(HDFS) handle the
>>> request? Do you know how I can do this? Thanks so much. Also when I try to
>>> add some debug information(such as print out something), but I can not see
>>> anything printed out in eclipse, I don't know why. Thanks again.
>>>
>>> Best,
>>> Kun
>>>
>>
>>
>
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Hi Vinay,
Thanks a lot, it works:)
I have another question:
1. When I add the following lines into the public boolean mkdirs(String
src, FsPermission masked, boolean createParent) in the
NameNodeRpcServer.java,
LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
System.out.println("In NameNodeRpcServer.java:mkdirs, before do
actual work");
But I can not see any output either from my terminal or Eclipse console
when I step-by-step debug this function, do you know why?
2. When I debug the client side code, for example FsShell, the default log
level is INFO, I changed the level to DEBUG using the
http://localhost:50070/logLevel, however, it still can not print out
LOG.debug(), but I can see LOG.info(), I don't know why it can not print
out debug information even I set the log level to DEBUG.
Thanks a lot for your help,
On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
wrote:
> Hi Kun Ren,
>
> You can follow the below steps.
> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
> hadoop-env.sh
> 2. Start Namenode
> 3. Now Namenode will start debug port in 3988.
> 4. Configure Remote debug application to connect to <namenode_host>:3988
> in eclipse
> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want to
> debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
> Server Layer.
> 6. Now you can do step-by-step execution to observe the detailed
> executions.
>
> Happy Debugging ;)
>
> -Vinay
>
> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>
>> Hi All,
>>
>> Currently I used eclipse to compile/debug the source code, and I
>> configured the "remote Java application" to debug the source code in
>> eclipse, for example, I can debug the client side code when I run the
>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>> Then it finish debugging, but I can not see how the server side handle the
>> request.
>>
>>
>> However, I want to debug and see how the server side(HDFS) handle the
>> request? Do you know how I can do this? Thanks so much. Also when I try to
>> add some debug information(such as print out something), but I can not see
>> anything printed out in eclipse, I don't know why. Thanks again.
>>
>> Best,
>> Kun
>>
>
>
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Hi Vinay,
Thanks a lot, it works:)
I have another question:
1. When I add the following lines into the public boolean mkdirs(String
src, FsPermission masked, boolean createParent) in the
NameNodeRpcServer.java,
LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
System.out.println("In NameNodeRpcServer.java:mkdirs, before do
actual work");
But I can not see any output either from my terminal or Eclipse console
when I step-by-step debug this function, do you know why?
2. When I debug the client side code, for example FsShell, the default log
level is INFO, I changed the level to DEBUG using the
http://localhost:50070/logLevel, however, it still can not print out
LOG.debug(), but I can see LOG.info(), I don't know why it can not print
out debug information even I set the log level to DEBUG.
Thanks a lot for your help,
On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
wrote:
> Hi Kun Ren,
>
> You can follow the below steps.
> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
> hadoop-env.sh
> 2. Start Namenode
> 3. Now Namenode will start debug port in 3988.
> 4. Configure Remote debug application to connect to <namenode_host>:3988
> in eclipse
> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want to
> debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
> Server Layer.
> 6. Now you can do step-by-step execution to observe the detailed
> executions.
>
> Happy Debugging ;)
>
> -Vinay
>
> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>
>> Hi All,
>>
>> Currently I used eclipse to compile/debug the source code, and I
>> configured the "remote Java application" to debug the source code in
>> eclipse, for example, I can debug the client side code when I run the
>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>> Then it finish debugging, but I can not see how the server side handle the
>> request.
>>
>>
>> However, I want to debug and see how the server side(HDFS) handle the
>> request? Do you know how I can do this? Thanks so much. Also when I try to
>> add some debug information(such as print out something), but I can not see
>> anything printed out in eclipse, I don't know why. Thanks again.
>>
>> Best,
>> Kun
>>
>
>
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Hi Vinay,
Thanks a lot, it works:)
I have another question:
1. When I add the following lines into the public boolean mkdirs(String
src, FsPermission masked, boolean createParent) in the
NameNodeRpcServer.java,
LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
System.out.println("In NameNodeRpcServer.java:mkdirs, before do
actual work");
But I can not see any output either from my terminal or Eclipse console
when I step-by-step debug this function, do you know why?
2. When I debug the client side code, for example FsShell, the default log
level is INFO, I changed the level to DEBUG using the
http://localhost:50070/logLevel, however, it still can not print out
LOG.debug(), but I can see LOG.info(), I don't know why it can not print
out debug information even I set the log level to DEBUG.
Thanks a lot for your help,
On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
wrote:
> Hi Kun Ren,
>
> You can follow the below steps.
> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
> hadoop-env.sh
> 2. Start Namenode
> 3. Now Namenode will start debug port in 3988.
> 4. Configure Remote debug application to connect to <namenode_host>:3988
> in eclipse
> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want to
> debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
> Server Layer.
> 6. Now you can do step-by-step execution to observe the detailed
> executions.
>
> Happy Debugging ;)
>
> -Vinay
>
> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>
>> Hi All,
>>
>> Currently I used eclipse to compile/debug the source code, and I
>> configured the "remote Java application" to debug the source code in
>> eclipse, for example, I can debug the client side code when I run the
>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>> Then it finish debugging, but I can not see how the server side handle the
>> request.
>>
>>
>> However, I want to debug and see how the server side(HDFS) handle the
>> request? Do you know how I can do this? Thanks so much. Also when I try to
>> add some debug information(such as print out something), but I can not see
>> anything printed out in eclipse, I don't know why. Thanks again.
>>
>> Best,
>> Kun
>>
>
>
Re: Eclipse debug HDFS server side code
Posted by Kun Ren <re...@gmail.com>.
Hi Vinay,
Thanks a lot, it works:)
I have another question:
1. When I add the following lines into the public boolean mkdirs(String
src, FsPermission masked, boolean createParent) in the
NameNodeRpcServer.java,
LOG.warn("NameNodeRpcServer.java:mkdirs, log level=warn");
System.out.println("In NameNodeRpcServer.java:mkdirs, before do
actual work");
But I can not see any output either from my terminal or Eclipse console
when I step-by-step debug this function, do you know why?
2. When I debug the client side code, for example FsShell, the default log
level is INFO, I changed the level to DEBUG using the
http://localhost:50070/logLevel, however, it still can not print out
LOG.debug(), but I can see LOG.info(), I don't know why it can not print
out debug information even I set the log level to DEBUG.
Thanks a lot for your help,
On Tue, Apr 19, 2016 at 12:21 PM, Vinayakumar B <vi...@apache.org>
wrote:
> Hi Kun Ren,
>
> You can follow the below steps.
> 1. configure HADOOP_NAMENODE_OPTS="-Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
> hadoop-env.sh
> 2. Start Namenode
> 3. Now Namenode will start debug port in 3988.
> 4. Configure Remote debug application to connect to <namenode_host>:3988
> in eclipse
> 5. Now add a breakpoint in NameNodeRpcServer.java's any method you want to
> debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
> Server Layer.
> 6. Now you can do step-by-step execution to observe the detailed
> executions.
>
> Happy Debugging ;)
>
> -Vinay
>
> On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
>
>> Hi All,
>>
>> Currently I used eclipse to compile/debug the source code, and I
>> configured the "remote Java application" to debug the source code in
>> eclipse, for example, I can debug the client side code when I run the
>> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
>> DistributedFileSystem---->DFSClient ....->send request to the server side.
>> Then it finish debugging, but I can not see how the server side handle the
>> request.
>>
>>
>> However, I want to debug and see how the server side(HDFS) handle the
>> request? Do you know how I can do this? Thanks so much. Also when I try to
>> add some debug information(such as print out something), but I can not see
>> anything printed out in eclipse, I don't know why. Thanks again.
>>
>> Best,
>> Kun
>>
>
>
Re: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
Hi Kun Ren,
You can follow the below steps.
1. configure HADOOP_NAMENODE_OPTS="-Xdebug
-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
hadoop-env.sh
2. Start Namenode
3. Now Namenode will start debug port in 3988.
4. Configure Remote debug application to connect to <namenode_host>:3988 in
eclipse
5. Now add a breakpoint in NameNodeRpcServer.java's any method you want to
debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
Server Layer.
6. Now you can do step-by-step execution to observe the detailed executions.
Happy Debugging ;)
-Vinay
On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
> Hi All,
>
> Currently I used eclipse to compile/debug the source code, and I
> configured the "remote Java application" to debug the source code in
> eclipse, for example, I can debug the client side code when I run the
> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
> DistributedFileSystem---->DFSClient ....->send request to the server side.
> Then it finish debugging, but I can not see how the server side handle the
> request.
>
>
> However, I want to debug and see how the server side(HDFS) handle the
> request? Do you know how I can do this? Thanks so much. Also when I try to
> add some debug information(such as print out something), but I can not see
> anything printed out in eclipse, I don't know why. Thanks again.
>
> Best,
> Kun
>
Re: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
Hi Kun Ren,
You can follow the below steps.
1. configure HADOOP_NAMENODE_OPTS="-Xdebug
-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
hadoop-env.sh
2. Start Namenode
3. Now Namenode will start debug port in 3988.
4. Configure Remote debug application to connect to <namenode_host>:3988 in
eclipse
5. Now add a breakpoint in NameNodeRpcServer.java's any method you want to
debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
Server Layer.
6. Now you can do step-by-step execution to observe the detailed executions.
Happy Debugging ;)
-Vinay
On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
> Hi All,
>
> Currently I used eclipse to compile/debug the source code, and I
> configured the "remote Java application" to debug the source code in
> eclipse, for example, I can debug the client side code when I run the
> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
> DistributedFileSystem---->DFSClient ....->send request to the server side.
> Then it finish debugging, but I can not see how the server side handle the
> request.
>
>
> However, I want to debug and see how the server side(HDFS) handle the
> request? Do you know how I can do this? Thanks so much. Also when I try to
> add some debug information(such as print out something), but I can not see
> anything printed out in eclipse, I don't know why. Thanks again.
>
> Best,
> Kun
>
Re: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
Hi Kun Ren,
You can follow the below steps.
1. configure HADOOP_NAMENODE_OPTS="-Xdebug
-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
hadoop-env.sh
2. Start Namenode
3. Now Namenode will start debug port in 3988.
4. Configure Remote debug application to connect to <namenode_host>:3988 in
eclipse
5. Now add a breakpoint in NameNodeRpcServer.java's any method you want to
debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
Server Layer.
6. Now you can do step-by-step execution to observe the detailed executions.
Happy Debugging ;)
-Vinay
On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
> Hi All,
>
> Currently I used eclipse to compile/debug the source code, and I
> configured the "remote Java application" to debug the source code in
> eclipse, for example, I can debug the client side code when I run the
> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
> DistributedFileSystem---->DFSClient ....->send request to the server side.
> Then it finish debugging, but I can not see how the server side handle the
> request.
>
>
> However, I want to debug and see how the server side(HDFS) handle the
> request? Do you know how I can do this? Thanks so much. Also when I try to
> add some debug information(such as print out something), but I can not see
> anything printed out in eclipse, I don't know why. Thanks again.
>
> Best,
> Kun
>
Re: Eclipse debug HDFS server side code
Posted by Vinayakumar B <vi...@apache.org>.
Hi Kun Ren,
You can follow the below steps.
1. configure HADOOP_NAMENODE_OPTS="-Xdebug
-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3988" in
hadoop-env.sh
2. Start Namenode
3. Now Namenode will start debug port in 3988.
4. Configure Remote debug application to connect to <namenode_host>:3988 in
eclipse
5. Now add a breakpoint in NameNodeRpcServer.java's any method you want to
debug. NameNodeRpcServer.java is the gateway between NameNode and IPC
Server Layer.
6. Now you can do step-by-step execution to observe the detailed executions.
Happy Debugging ;)
-Vinay
On Tue, Apr 19, 2016 at 8:46 PM, Kun Ren <re...@gmail.com> wrote:
> Hi All,
>
> Currently I used eclipse to compile/debug the source code, and I
> configured the "remote Java application" to debug the source code in
> eclipse, for example, I can debug the client side code when I run the
> command "./bin/hdfs dfs -mkdir test", it goes through "FsShell--->
> DistributedFileSystem---->DFSClient ....->send request to the server side.
> Then it finish debugging, but I can not see how the server side handle the
> request.
>
>
> However, I want to debug and see how the server side(HDFS) handle the
> request? Do you know how I can do this? Thanks so much. Also when I try to
> add some debug information(such as print out something), but I can not see
> anything printed out in eclipse, I don't know why. Thanks again.
>
> Best,
> Kun
>