You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by norbi <no...@rocknob.de> on 2014/03/21 08:24:55 UTC
Hadoop upgrade, what happend on Datanode side and how long can it
takes?
Hi List,
we are using a hadoop cluster (v 0.20.2) and want to upgrade to cloudera
4.6 (hadoop 2.0).
Configured Capacity: 969329209806848 (881.6 TB)
Present Capacity: 969146545811532 (881.43 TB)
DFS Remaining: 219694195171526 (199.81 TB)
DFS Used: 749452350640006 (681.62 TB)
DFS Used%: 77.33%
Under replicated blocks: 0
Files are stored on 45 Datanodes.
I cant find any Information how long the upgrade will take, in our
Testsetup (only about 200GB with 3 Datanodes) the upgrade runs in a few
secons, but what will happen if i upgrade the "big" cluster?
Greats,
Norbert
Re: Hadoop upgrade, what happend on Datanode side and how long can it takes?
Posted by Harsh J <ha...@cloudera.com>.
Hello Norbert,
In 2.x, the upgrade process on the DN disk layout happens with greater
parallelism, and the overall DN upgrade is also a distributed task, so
you can expect it to complete very fast - certainly less than an hour
tops.
On Fri, Mar 21, 2014 at 12:54 PM, norbi <no...@rocknob.de> wrote:
> Hi List,
>
> we are using a hadoop cluster (v 0.20.2) and want to upgrade to cloudera 4.6
> (hadoop 2.0).
>
> Configured Capacity: 969329209806848 (881.6 TB)
> Present Capacity: 969146545811532 (881.43 TB)
> DFS Remaining: 219694195171526 (199.81 TB)
> DFS Used: 749452350640006 (681.62 TB)
> DFS Used%: 77.33%
> Under replicated blocks: 0
>
> Files are stored on 45 Datanodes.
>
> I cant find any Information how long the upgrade will take, in our Testsetup
> (only about 200GB with 3 Datanodes) the upgrade runs in a few secons, but
> what will happen if i upgrade the "big" cluster?
>
> Greats,
>
> Norbert
--
Harsh J
Re: Hadoop upgrade, what happend on Datanode side and how long can it takes?
Posted by Harsh J <ha...@cloudera.com>.
Hello Norbert,
In 2.x, the upgrade process on the DN disk layout happens with greater
parallelism, and the overall DN upgrade is also a distributed task, so
you can expect it to complete very fast - certainly less than an hour
tops.
On Fri, Mar 21, 2014 at 12:54 PM, norbi <no...@rocknob.de> wrote:
> Hi List,
>
> we are using a hadoop cluster (v 0.20.2) and want to upgrade to cloudera 4.6
> (hadoop 2.0).
>
> Configured Capacity: 969329209806848 (881.6 TB)
> Present Capacity: 969146545811532 (881.43 TB)
> DFS Remaining: 219694195171526 (199.81 TB)
> DFS Used: 749452350640006 (681.62 TB)
> DFS Used%: 77.33%
> Under replicated blocks: 0
>
> Files are stored on 45 Datanodes.
>
> I cant find any Information how long the upgrade will take, in our Testsetup
> (only about 200GB with 3 Datanodes) the upgrade runs in a few secons, but
> what will happen if i upgrade the "big" cluster?
>
> Greats,
>
> Norbert
--
Harsh J
Re: Hadoop upgrade, what happend on Datanode side and how long can it takes?
Posted by Harsh J <ha...@cloudera.com>.
Hello Norbert,
In 2.x, the upgrade process on the DN disk layout happens with greater
parallelism, and the overall DN upgrade is also a distributed task, so
you can expect it to complete very fast - certainly less than an hour
tops.
On Fri, Mar 21, 2014 at 12:54 PM, norbi <no...@rocknob.de> wrote:
> Hi List,
>
> we are using a hadoop cluster (v 0.20.2) and want to upgrade to cloudera 4.6
> (hadoop 2.0).
>
> Configured Capacity: 969329209806848 (881.6 TB)
> Present Capacity: 969146545811532 (881.43 TB)
> DFS Remaining: 219694195171526 (199.81 TB)
> DFS Used: 749452350640006 (681.62 TB)
> DFS Used%: 77.33%
> Under replicated blocks: 0
>
> Files are stored on 45 Datanodes.
>
> I cant find any Information how long the upgrade will take, in our Testsetup
> (only about 200GB with 3 Datanodes) the upgrade runs in a few secons, but
> what will happen if i upgrade the "big" cluster?
>
> Greats,
>
> Norbert
--
Harsh J
Re: Hadoop upgrade, what happend on Datanode side and how long can it takes?
Posted by Harsh J <ha...@cloudera.com>.
Hello Norbert,
In 2.x, the upgrade process on the DN disk layout happens with greater
parallelism, and the overall DN upgrade is also a distributed task, so
you can expect it to complete very fast - certainly less than an hour
tops.
On Fri, Mar 21, 2014 at 12:54 PM, norbi <no...@rocknob.de> wrote:
> Hi List,
>
> we are using a hadoop cluster (v 0.20.2) and want to upgrade to cloudera 4.6
> (hadoop 2.0).
>
> Configured Capacity: 969329209806848 (881.6 TB)
> Present Capacity: 969146545811532 (881.43 TB)
> DFS Remaining: 219694195171526 (199.81 TB)
> DFS Used: 749452350640006 (681.62 TB)
> DFS Used%: 77.33%
> Under replicated blocks: 0
>
> Files are stored on 45 Datanodes.
>
> I cant find any Information how long the upgrade will take, in our Testsetup
> (only about 200GB with 3 Datanodes) the upgrade runs in a few secons, but
> what will happen if i upgrade the "big" cluster?
>
> Greats,
>
> Norbert
--
Harsh J