You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Long Van Nguyen Dinh <mu...@gmail.com> on 2009/10/14 04:22:13 UTC
How to get IP address of the machine where map task runs
Hi all,
Given a map task, I need to know the IP address of the machine where
that task is running. Is there any existing method to get that
information?
Thank you,
Van
Re: How to get IP address of the machine where map task runs
Posted by Huy Phan <da...@gmail.com>.
Hi Long,
1. You can make it with a hardcore way: parsing the log file of job
tracker to get the information about `task id` and hostname of its node.
2. This is a sample code to get IP Address from hostname using InetAddress:
|java.net.InetAddress inetAdd = java.net.InetAddress.getByName("www.google.com");
String ipAdress = inetAdd.|getHostAddress()
Just for out of curiosity, what is your purpose of getting *location* of
map task when running a MapReduce job?
Long Van Nguyen Dinh wrote:
> Thanks Amogh. For my application, I want each map task reports to me
> where it's running. However, I have no idea how to use Java
> Inetaddress APIs to get that info. Could you explain more?
>
> Van
>
> On Wed, Oct 14, 2009 at 2:16 PM, Amogh Vasekar <am...@yahoo-inc.com> wrote:
>
>> For starters look at any monitoring tool like vaidya, hadoop UI ( ganglia too, haven't read much on it though ). Not sure if you need this for debugging purposes or for some other real-time app.. You should be able to get info on localhost of each of your map tasks in a pretty straightforward way using Java Inetaddress APIs( and use that info for search?)
>>
>> Thanks,
>> Amogh
>>
>>
>> On 10/15/09 12:11 AM, "Long Van Nguyen Dinh" <mu...@gmail.com> wrote:
>>
>> Hello again,
>>
>> Could you give me any hint to start with? I have no idea how to get
>> that information.
>>
>> Many thanks,
>> Van
>>
>> On Tue, Oct 13, 2009 at 9:22 PM, Long Van Nguyen Dinh <mu...@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> Given a map task, I need to know the IP address of the machine where
>>> that task is running. Is there any existing method to get that
>>> information?
>>>
>>> Thank you,
>>> Van
>>>
>>>
>>
>
>
Re: How to get IP address of the machine where map task runs
Posted by Amogh Vasekar <am...@yahoo-inc.com>.
InetAddress.getLocalHost() should give you that. If you are planning to make some decisions based on this, please do account for conditions arising from speculative executions ( they caused me some amount of trouble when I was designing my app )
Thanks,
Amogh
On 10/15/09 8:15 AM, "Long Van Nguyen Dinh" <mu...@gmail.com> wrote:
Thanks Amogh. For my application, I want each map task reports to me
where it's running. However, I have no idea how to use Java
Inetaddress APIs to get that info. Could you explain more?
Van
On Wed, Oct 14, 2009 at 2:16 PM, Amogh Vasekar <am...@yahoo-inc.com> wrote:
> For starters look at any monitoring tool like vaidya, hadoop UI ( ganglia too, haven't read much on it though ). Not sure if you need this for debugging purposes or for some other real-time app.. You should be able to get info on localhost of each of your map tasks in a pretty straightforward way using Java Inetaddress APIs( and use that info for search?)
>
> Thanks,
> Amogh
>
>
> On 10/15/09 12:11 AM, "Long Van Nguyen Dinh" <mu...@gmail.com> wrote:
>
> Hello again,
>
> Could you give me any hint to start with? I have no idea how to get
> that information.
>
> Many thanks,
> Van
>
> On Tue, Oct 13, 2009 at 9:22 PM, Long Van Nguyen Dinh <mu...@gmail.com> wrote:
>> Hi all,
>>
>> Given a map task, I need to know the IP address of the machine where
>> that task is running. Is there any existing method to get that
>> information?
>>
>> Thank you,
>> Van
>>
>
>
Re: How to get IP address of the machine where map task runs
Posted by Long Van Nguyen Dinh <mu...@gmail.com>.
Thanks Amogh. For my application, I want each map task reports to me
where it's running. However, I have no idea how to use Java
Inetaddress APIs to get that info. Could you explain more?
Van
On Wed, Oct 14, 2009 at 2:16 PM, Amogh Vasekar <am...@yahoo-inc.com> wrote:
> For starters look at any monitoring tool like vaidya, hadoop UI ( ganglia too, haven't read much on it though ). Not sure if you need this for debugging purposes or for some other real-time app.. You should be able to get info on localhost of each of your map tasks in a pretty straightforward way using Java Inetaddress APIs( and use that info for search?)
>
> Thanks,
> Amogh
>
>
> On 10/15/09 12:11 AM, "Long Van Nguyen Dinh" <mu...@gmail.com> wrote:
>
> Hello again,
>
> Could you give me any hint to start with? I have no idea how to get
> that information.
>
> Many thanks,
> Van
>
> On Tue, Oct 13, 2009 at 9:22 PM, Long Van Nguyen Dinh <mu...@gmail.com> wrote:
>> Hi all,
>>
>> Given a map task, I need to know the IP address of the machine where
>> that task is running. Is there any existing method to get that
>> information?
>>
>> Thank you,
>> Van
>>
>
>
Re: How to get IP address of the machine where map task runs
Posted by Amogh Vasekar <am...@yahoo-inc.com>.
For starters look at any monitoring tool like vaidya, hadoop UI ( ganglia too, haven't read much on it though ). Not sure if you need this for debugging purposes or for some other real-time app.. You should be able to get info on localhost of each of your map tasks in a pretty straightforward way using Java Inetaddress APIs( and use that info for search?)
Thanks,
Amogh
On 10/15/09 12:11 AM, "Long Van Nguyen Dinh" <mu...@gmail.com> wrote:
Hello again,
Could you give me any hint to start with? I have no idea how to get
that information.
Many thanks,
Van
On Tue, Oct 13, 2009 at 9:22 PM, Long Van Nguyen Dinh <mu...@gmail.com> wrote:
> Hi all,
>
> Given a map task, I need to know the IP address of the machine where
> that task is running. Is there any existing method to get that
> information?
>
> Thank you,
> Van
>
Re: How to get IP address of the machine where map task runs
Posted by Long Van Nguyen Dinh <mu...@gmail.com>.
Hello again,
Could you give me any hint to start with? I have no idea how to get
that information.
Many thanks,
Van
On Tue, Oct 13, 2009 at 9:22 PM, Long Van Nguyen Dinh <mu...@gmail.com> wrote:
> Hi all,
>
> Given a map task, I need to know the IP address of the machine where
> that task is running. Is there any existing method to get that
> information?
>
> Thank you,
> Van
>