You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by ram kumar <ra...@gmail.com> on 2015/10/29 06:54:52 UTC
check decommission status
Hi,
Is there a java api to get decommission status for a particular data node?
Thanks.
Re: check decommission status
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Surbhi,
There is no public, stable Java API for this. If you look at the
implementation of "hdfs dfsadmin -report", it involves downcasting from
the base FileSystem class to the concrete DistributedFileSystem, and then
calling DistributedFileSystem#getDataNodeStats.
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs
/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java#L523
You could potentially mimic this code from DFSAdmin, but please keep in
mind that you'd be calling internal APIs that do not come with a strong
stability guarantee. We might evolve those APIs across releases in ways
that would break your application after upgrading to a later version.
Another more stable approach would be to parse the shell output of "hdfs
dfsadmin -report". We offer a backward compatibility guarantee on shell
output.
A third approach is to call the NameNode's HTTP server to get a JSON dump
of NameNode JMX metrics. The NameNodeInfo MBean includes a "DecomNodes"
field with information about decommissioning nodes. Here is a sample curl
call, assuming a NameNode running on localhost with its HTTP server
listening on port 50070.
curl
'http://127.0.0.1:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo'
More details on our compatibility policy are listed here in case you're
interested.
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Co
mpatibility.html
--Chris Nauroth
On 10/29/15, 8:40 AM, "Surbhi Gupta" <su...@gmail.com> wrote:
>You can try nodetool netstats . If you find streaming happening from the
>decommission node to other node and vise versa then decommission is still
>going on .
>
>Sent from my iPhone
>
>> On Oct 28, 2015, at 10:54 PM, ram kumar <ra...@gmail.com> wrote:
>>
>> Hi,
>>
>> Is there a java api to get decommission status for a particular data
>>node?
>>
>> Thanks.
>
Re: check decommission status
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Surbhi,
There is no public, stable Java API for this. If you look at the
implementation of "hdfs dfsadmin -report", it involves downcasting from
the base FileSystem class to the concrete DistributedFileSystem, and then
calling DistributedFileSystem#getDataNodeStats.
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs
/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java#L523
You could potentially mimic this code from DFSAdmin, but please keep in
mind that you'd be calling internal APIs that do not come with a strong
stability guarantee. We might evolve those APIs across releases in ways
that would break your application after upgrading to a later version.
Another more stable approach would be to parse the shell output of "hdfs
dfsadmin -report". We offer a backward compatibility guarantee on shell
output.
A third approach is to call the NameNode's HTTP server to get a JSON dump
of NameNode JMX metrics. The NameNodeInfo MBean includes a "DecomNodes"
field with information about decommissioning nodes. Here is a sample curl
call, assuming a NameNode running on localhost with its HTTP server
listening on port 50070.
curl
'http://127.0.0.1:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo'
More details on our compatibility policy are listed here in case you're
interested.
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Co
mpatibility.html
--Chris Nauroth
On 10/29/15, 8:40 AM, "Surbhi Gupta" <su...@gmail.com> wrote:
>You can try nodetool netstats . If you find streaming happening from the
>decommission node to other node and vise versa then decommission is still
>going on .
>
>Sent from my iPhone
>
>> On Oct 28, 2015, at 10:54 PM, ram kumar <ra...@gmail.com> wrote:
>>
>> Hi,
>>
>> Is there a java api to get decommission status for a particular data
>>node?
>>
>> Thanks.
>
Re: check decommission status
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Surbhi,
There is no public, stable Java API for this. If you look at the
implementation of "hdfs dfsadmin -report", it involves downcasting from
the base FileSystem class to the concrete DistributedFileSystem, and then
calling DistributedFileSystem#getDataNodeStats.
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs
/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java#L523
You could potentially mimic this code from DFSAdmin, but please keep in
mind that you'd be calling internal APIs that do not come with a strong
stability guarantee. We might evolve those APIs across releases in ways
that would break your application after upgrading to a later version.
Another more stable approach would be to parse the shell output of "hdfs
dfsadmin -report". We offer a backward compatibility guarantee on shell
output.
A third approach is to call the NameNode's HTTP server to get a JSON dump
of NameNode JMX metrics. The NameNodeInfo MBean includes a "DecomNodes"
field with information about decommissioning nodes. Here is a sample curl
call, assuming a NameNode running on localhost with its HTTP server
listening on port 50070.
curl
'http://127.0.0.1:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo'
More details on our compatibility policy are listed here in case you're
interested.
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Co
mpatibility.html
--Chris Nauroth
On 10/29/15, 8:40 AM, "Surbhi Gupta" <su...@gmail.com> wrote:
>You can try nodetool netstats . If you find streaming happening from the
>decommission node to other node and vise versa then decommission is still
>going on .
>
>Sent from my iPhone
>
>> On Oct 28, 2015, at 10:54 PM, ram kumar <ra...@gmail.com> wrote:
>>
>> Hi,
>>
>> Is there a java api to get decommission status for a particular data
>>node?
>>
>> Thanks.
>
Re: check decommission status
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Surbhi,
There is no public, stable Java API for this. If you look at the
implementation of "hdfs dfsadmin -report", it involves downcasting from
the base FileSystem class to the concrete DistributedFileSystem, and then
calling DistributedFileSystem#getDataNodeStats.
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs
/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java#L523
You could potentially mimic this code from DFSAdmin, but please keep in
mind that you'd be calling internal APIs that do not come with a strong
stability guarantee. We might evolve those APIs across releases in ways
that would break your application after upgrading to a later version.
Another more stable approach would be to parse the shell output of "hdfs
dfsadmin -report". We offer a backward compatibility guarantee on shell
output.
A third approach is to call the NameNode's HTTP server to get a JSON dump
of NameNode JMX metrics. The NameNodeInfo MBean includes a "DecomNodes"
field with information about decommissioning nodes. Here is a sample curl
call, assuming a NameNode running on localhost with its HTTP server
listening on port 50070.
curl
'http://127.0.0.1:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo'
More details on our compatibility policy are listed here in case you're
interested.
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Co
mpatibility.html
--Chris Nauroth
On 10/29/15, 8:40 AM, "Surbhi Gupta" <su...@gmail.com> wrote:
>You can try nodetool netstats . If you find streaming happening from the
>decommission node to other node and vise versa then decommission is still
>going on .
>
>Sent from my iPhone
>
>> On Oct 28, 2015, at 10:54 PM, ram kumar <ra...@gmail.com> wrote:
>>
>> Hi,
>>
>> Is there a java api to get decommission status for a particular data
>>node?
>>
>> Thanks.
>
Re: check decommission status
Posted by Surbhi Gupta <su...@gmail.com>.
You can try nodetool netstats . If you find streaming happening from the decommission node to other node and vise versa then decommission is still going on .
Sent from my iPhone
> On Oct 28, 2015, at 10:54 PM, ram kumar <ra...@gmail.com> wrote:
>
> Hi,
>
> Is there a java api to get decommission status for a particular data node?
>
> Thanks.
Re: check decommission status
Posted by Surbhi Gupta <su...@gmail.com>.
You can try nodetool netstats . If you find streaming happening from the decommission node to other node and vise versa then decommission is still going on .
Sent from my iPhone
> On Oct 28, 2015, at 10:54 PM, ram kumar <ra...@gmail.com> wrote:
>
> Hi,
>
> Is there a java api to get decommission status for a particular data node?
>
> Thanks.
Re: check decommission status
Posted by Surbhi Gupta <su...@gmail.com>.
You can try nodetool netstats . If you find streaming happening from the decommission node to other node and vise versa then decommission is still going on .
Sent from my iPhone
> On Oct 28, 2015, at 10:54 PM, ram kumar <ra...@gmail.com> wrote:
>
> Hi,
>
> Is there a java api to get decommission status for a particular data node?
>
> Thanks.
Re: check decommission status
Posted by Surbhi Gupta <su...@gmail.com>.
You can try nodetool netstats . If you find streaming happening from the decommission node to other node and vise versa then decommission is still going on .
Sent from my iPhone
> On Oct 28, 2015, at 10:54 PM, ram kumar <ra...@gmail.com> wrote:
>
> Hi,
>
> Is there a java api to get decommission status for a particular data node?
>
> Thanks.