You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by C G <pa...@yahoo.com> on 2009/09/07 07:45:28 UTC
discyp between different versions of Hadoop...
Hi All:
Does anybody know if it's possible to distcp between an old version of Hadoop (0.15.x, for example) and a modern version (0.19.2)?
A quick check trying to move from an "old" grid to a "new" grid shows a "Incorrect header or version mismatch" error in the new grid's NameNode log, and a SocketTimeOut exception on the distcp on the old grid.
Any help/info most appreciated.
Thanks,C G
Re: discyp between different versions of Hadoop...
Posted by Jason Venner <ja...@gmail.com>.
Thank you, I don't think of the ftp interface at all and had completely
forgotten it.
On Mon, Sep 7, 2009 at 12:00 AM, Erik Forsberg <fo...@opera.com> wrote:
> On Sun, 6 Sep 2009 22:45:28 -0700 (PDT)
> C G <pa...@yahoo.com> wrote:
>
> > Hi All:
> > Does anybody know if it's possible to distcp between an old version
> > of Hadoop (0.15.x, for example) and a modern version (0.19.2)?
>
> Yes:
>
> 1) Run the distcp job on the newer cluster.
> 2) Use the hftp method in the source URI.
>
> Example:
>
> hadoop distcp hftp://oldclusternamenode:50070/path/to/src \
> hdfs://newclusternamenode:8020/path/to/dst
>
> See http://hadoop.apache.org/common/docs/r0.20.0/distcp.html#cpver
>
> Cheers,
> \EF
> --
> Erik Forsberg <fo...@opera.com>
> Developer, Opera Software - http://www.opera.com/
>
--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals
Re: discyp between different versions of Hadoop...
Posted by Erik Forsberg <fo...@opera.com>.
On Sun, 6 Sep 2009 22:45:28 -0700 (PDT)
C G <pa...@yahoo.com> wrote:
> Hi All:
> Does anybody know if it's possible to distcp between an old version
> of Hadoop (0.15.x, for example) and a modern version (0.19.2)?
Yes:
1) Run the distcp job on the newer cluster.
2) Use the hftp method in the source URI.
Example:
hadoop distcp hftp://oldclusternamenode:50070/path/to/src \
hdfs://newclusternamenode:8020/path/to/dst
See http://hadoop.apache.org/common/docs/r0.20.0/distcp.html#cpver
Cheers,
\EF
--
Erik Forsberg <fo...@opera.com>
Developer, Opera Software - http://www.opera.com/
Re: discyp between different versions of Hadoop...
Posted by Jason Venner <ja...@gmail.com>.
You pretty much have to stage the files through somethime.
If you can make source version of hadoop's fuse mount work, you can copy in,
using the fuse mount as a source.
On Sun, Sep 6, 2009 at 10:50 PM, C G <pa...@yahoo.com> wrote:
> Sorry...subject should be "distcp" obviously...
> Also trying to pull from the new grid from the old yields a
> java.io.EOFException...
>
> --- On Mon, 9/7/09, C G <pa...@yahoo.com> wrote:
>
> From: C G <pa...@yahoo.com>
> Subject: discyp between different versions of Hadoop...
> To: core-user@hadoop.apache.org
> Date: Monday, September 7, 2009, 1:45 AM
>
> Hi All:
> Does anybody know if it's possible to distcp between an old version of
> Hadoop (0.15.x, for example) and a modern version (0.19.2)?
> A quick check trying to move from an "old" grid to a "new" grid shows a
> "Incorrect header or version mismatch" error in the new grid's NameNode log,
> and a SocketTimeOut exception on the distcp on the old grid.
> Any help/info most appreciated.
> Thanks,C G
>
>
>
>
>
>
>
>
--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals
Re: discyp between different versions of Hadoop...
Posted by C G <pa...@yahoo.com>.
Sorry...subject should be "distcp" obviously...
Also trying to pull from the new grid from the old yields a java.io.EOFException...
--- On Mon, 9/7/09, C G <pa...@yahoo.com> wrote:
From: C G <pa...@yahoo.com>
Subject: discyp between different versions of Hadoop...
To: core-user@hadoop.apache.org
Date: Monday, September 7, 2009, 1:45 AM
Hi All:
Does anybody know if it's possible to distcp between an old version of Hadoop (0.15.x, for example) and a modern version (0.19.2)?
A quick check trying to move from an "old" grid to a "new" grid shows a "Incorrect header or version mismatch" error in the new grid's NameNode log, and a SocketTimeOut exception on the distcp on the old grid.
Any help/info most appreciated.
Thanks,C G