You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Yabo-Arber Xu <ar...@gmail.com> on 2008/07/21 11:17:19 UTC
Retrying connect to server error
Greetings,
I am right now using hbase in our project in "stand-alone" mode. It worked
well until today I found the following message in the log:
2008-07-21 01:16:38,564 FATAL org.apache.hadoop.hbase.HMaster: Not starting
HMaster because: org.apache.hadoop.hbase.FileSystemVersionException: File
system needs to be upgraded. Run the '${HBASE_HOME}/bin/hbase migrate'
script.
I actually did not do any version change recently. But, anyway, i followed
the message and tried to do "hbase migrate", but it keeps showing me the
following error:
08/07/21 02:11:51 INFO ipc.Client: Retrying connect to server: localhost/
127.0.0.1:60000. Already tried 1 time(s).
The same error occurs when i try to do any query through hbase shell. This
is just wierd....I did not touch any part of the hbase, and the error just
sundenly came out. Any ideas why?
Thanks for your help! The whole project is just stuck here by this error.
Arber
--
Yabo-Arber Xu <yx...@cs.sfu.ca>
Web: http://www.cs.sfu.ca/~yxu/personal/<http://www.cs.sfu.ca/%7Eyxu/personal/>
Re: Retrying connect to server error
Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Well you can have the Namenode, Datanode, Master and RegionServer on the
same machine but there will be an overhead for sure. Scaling is the name of
the game in this type of distributed system so it is designed to be "easy"
to scale. For example, if you go from 1 to 2 machines, just adding the IP of
the second one in the Namenode and Master config and starting the Datanode
and RegionServer processes will do the job. Then you will have to balance
data but that's all described in the Hadoop documentation (and mailing
list). Rinse and repeat.
Eventually, if you have all your data replicated on many machines, you will
be able to shut down the Datanode and RegionServer of the first machine.
It gets more dirty than this but that's basically what you have to do.
J-D
On Mon, Jul 21, 2008 at 3:49 PM, Yabo-Arber Xu <ar...@gmail.com>
wrote:
> Thanks again. I set up HBase on an Amazon instance, which has never been
> rebooted. But anyway, i agree it may not be a good practice to use /tmp as
> the data directory.
>
> I do not have particular reason of using standalone mode. It's just the
> current size of data can be handled by one computer and i have just been
> lazy. Now i guess it's time to go "distributed". One question though.
>
> If i set up hbase in distributed mode, say putting HDFS and HBASE on the
> same instance, as the data grows, would it be possible to upgrade hbase to
> multiple instance, say using one instance for master, several more for
> regional servers, without re-importing the data?
>
> Thanks.
>
> On Mon, Jul 21, 2008 at 11:47 AM, Jean-Daniel Cryans <jd...@gmail.com>
> wrote:
>
> > Arber,
> >
> > /tmp is cleared when you reboot your machine and I think the OS reclaims
> it
> > once in a while (not sure). You should change that config ASAP. Is there
> a
> > particular reason why you want to use HBase in standalone?
> >
> > Sorry for your data.
> >
> > J-D
> >
> > On Mon, Jul 21, 2008 at 2:38 PM, Yabo-Arber, Xu <yx...@cs.sfu.ca> wrote:
> >
> > > Thanks for your reply, J-D. There is no such file hbase.version under
> the
> > > hbase root directory, and this morning i found something even worse
> > > happened. The data under hbase root directory ( i use the default
> > > /tmp/hbase-{user.home}/root ) disappeared......originally there was a
> > > folder
> > > corresponding to each table and i guess most of the data are physically
> > > stored there.
> > >
> > > This is not the first time happening to me. Last time similar things
> > > happened, and i addressed it using a naive way without thinking too
> much(
> > > re-building the whole database ) and then it runs well for around a
> week,
> > > and now it cames again. Do you have any idea what happened to the data?
> > > Would it be because i use the /tmp directory which happened to be
> > modified
> > > by the system?
> > >
> > >
> > >
> > > On Mon, Jul 21, 2008 at 7:58 AM, Jean-Daniel Cryans <
> jdcryans@gmail.com>
> > > wrote:
> > >
> > > > Arber,
> > > >
> > > > When the Master starts, it checks on the filesystem for a file named
> > > > "hbase.version" in your HBase root folder. If the version of your
> HBase
> > > > distribution changed OR if the file is missing, it will throw the
> error
> > > you
> > > > see. I would advise confirming that the file is still there.
> > > >
> > > > Also, the migrate script first tries to connect to the Master to make
> > > sure
> > > > it is offline. It retries a couple of times (the error you see) then
> > > > proceed
> > > > to doing the migration.
> > > >
> > > > Hope it helps,
> > > >
> > > > J-D
> > > >
> > > > On Mon, Jul 21, 2008 at 5:17 AM, Yabo-Arber Xu <
> > arber.research@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > Greetings,
> > > > >
> > > > > I am right now using hbase in our project in "stand-alone" mode. It
> > > > worked
> > > > > well until today I found the following message in the log:
> > > > >
> > > > > 2008-07-21 01:16:38,564 FATAL org.apache.hadoop.hbase.HMaster: Not
> > > > starting
> > > > > HMaster because:
> org.apache.hadoop.hbase.FileSystemVersionException:
> > > File
> > > > > system needs to be upgraded. Run the '${HBASE_HOME}/bin/hbase
> > migrate'
> > > > > script.
> > > > >
> > > > > I actually did not do any version change recently. But, anyway, i
> > > > followed
> > > > > the message and tried to do "hbase migrate", but it keeps showing
> me
> > > the
> > > > > following error:
> > > > >
> > > > > 08/07/21 02:11:51 INFO ipc.Client: Retrying connect to server:
> > > localhost/
> > > > > 127.0.0.1:60000. Already tried 1 time(s).
> > > > >
> > > > > The same error occurs when i try to do any query through hbase
> shell.
> > > > This
> > > > > is just wierd....I did not touch any part of the hbase, and the
> error
> > > > just
> > > > > sundenly came out. Any ideas why?
> > > > >
> > > > > Thanks for your help! The whole project is just stuck here by this
> > > error.
> > > > >
> > > > > Arber
> > > >
> > > >
> > >
> >
>
>
>
> --
> Yabo-Arber Xu <yx...@cs.sfu.ca>
> Web: http://www.cs.sfu.ca/~yxu/personal/<http://www.cs.sfu.ca/%7Eyxu/personal/>
> <http://www.cs.sfu.ca/%7Eyxu/personal/>
>
Re: Retrying connect to server error
Posted by Yabo-Arber Xu <ar...@gmail.com>.
Thanks again. I set up HBase on an Amazon instance, which has never been
rebooted. But anyway, i agree it may not be a good practice to use /tmp as
the data directory.
I do not have particular reason of using standalone mode. It's just the
current size of data can be handled by one computer and i have just been
lazy. Now i guess it's time to go "distributed". One question though.
If i set up hbase in distributed mode, say putting HDFS and HBASE on the
same instance, as the data grows, would it be possible to upgrade hbase to
multiple instance, say using one instance for master, several more for
regional servers, without re-importing the data?
Thanks.
On Mon, Jul 21, 2008 at 11:47 AM, Jean-Daniel Cryans <jd...@gmail.com>
wrote:
> Arber,
>
> /tmp is cleared when you reboot your machine and I think the OS reclaims it
> once in a while (not sure). You should change that config ASAP. Is there a
> particular reason why you want to use HBase in standalone?
>
> Sorry for your data.
>
> J-D
>
> On Mon, Jul 21, 2008 at 2:38 PM, Yabo-Arber, Xu <yx...@cs.sfu.ca> wrote:
>
> > Thanks for your reply, J-D. There is no such file hbase.version under the
> > hbase root directory, and this morning i found something even worse
> > happened. The data under hbase root directory ( i use the default
> > /tmp/hbase-{user.home}/root ) disappeared......originally there was a
> > folder
> > corresponding to each table and i guess most of the data are physically
> > stored there.
> >
> > This is not the first time happening to me. Last time similar things
> > happened, and i addressed it using a naive way without thinking too much(
> > re-building the whole database ) and then it runs well for around a week,
> > and now it cames again. Do you have any idea what happened to the data?
> > Would it be because i use the /tmp directory which happened to be
> modified
> > by the system?
> >
> >
> >
> > On Mon, Jul 21, 2008 at 7:58 AM, Jean-Daniel Cryans <jd...@gmail.com>
> > wrote:
> >
> > > Arber,
> > >
> > > When the Master starts, it checks on the filesystem for a file named
> > > "hbase.version" in your HBase root folder. If the version of your HBase
> > > distribution changed OR if the file is missing, it will throw the error
> > you
> > > see. I would advise confirming that the file is still there.
> > >
> > > Also, the migrate script first tries to connect to the Master to make
> > sure
> > > it is offline. It retries a couple of times (the error you see) then
> > > proceed
> > > to doing the migration.
> > >
> > > Hope it helps,
> > >
> > > J-D
> > >
> > > On Mon, Jul 21, 2008 at 5:17 AM, Yabo-Arber Xu <
> arber.research@gmail.com
> > >
> > > wrote:
> > >
> > > > Greetings,
> > > >
> > > > I am right now using hbase in our project in "stand-alone" mode. It
> > > worked
> > > > well until today I found the following message in the log:
> > > >
> > > > 2008-07-21 01:16:38,564 FATAL org.apache.hadoop.hbase.HMaster: Not
> > > starting
> > > > HMaster because: org.apache.hadoop.hbase.FileSystemVersionException:
> > File
> > > > system needs to be upgraded. Run the '${HBASE_HOME}/bin/hbase
> migrate'
> > > > script.
> > > >
> > > > I actually did not do any version change recently. But, anyway, i
> > > followed
> > > > the message and tried to do "hbase migrate", but it keeps showing me
> > the
> > > > following error:
> > > >
> > > > 08/07/21 02:11:51 INFO ipc.Client: Retrying connect to server:
> > localhost/
> > > > 127.0.0.1:60000. Already tried 1 time(s).
> > > >
> > > > The same error occurs when i try to do any query through hbase shell.
> > > This
> > > > is just wierd....I did not touch any part of the hbase, and the error
> > > just
> > > > sundenly came out. Any ideas why?
> > > >
> > > > Thanks for your help! The whole project is just stuck here by this
> > error.
> > > >
> > > > Arber
> > >
> > >
> >
>
--
Yabo-Arber Xu <yx...@cs.sfu.ca>
Web: http://www.cs.sfu.ca/~yxu/personal/<http://www.cs.sfu.ca/%7Eyxu/personal/>
Re: Retrying connect to server error
Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Arber,
/tmp is cleared when you reboot your machine and I think the OS reclaims it
once in a while (not sure). You should change that config ASAP. Is there a
particular reason why you want to use HBase in standalone?
Sorry for your data.
J-D
On Mon, Jul 21, 2008 at 2:38 PM, Yabo-Arber, Xu <yx...@cs.sfu.ca> wrote:
> Thanks for your reply, J-D. There is no such file hbase.version under the
> hbase root directory, and this morning i found something even worse
> happened. The data under hbase root directory ( i use the default
> /tmp/hbase-{user.home}/root ) disappeared......originally there was a
> folder
> corresponding to each table and i guess most of the data are physically
> stored there.
>
> This is not the first time happening to me. Last time similar things
> happened, and i addressed it using a naive way without thinking too much(
> re-building the whole database ) and then it runs well for around a week,
> and now it cames again. Do you have any idea what happened to the data?
> Would it be because i use the /tmp directory which happened to be modified
> by the system?
>
>
>
> On Mon, Jul 21, 2008 at 7:58 AM, Jean-Daniel Cryans <jd...@gmail.com>
> wrote:
>
> > Arber,
> >
> > When the Master starts, it checks on the filesystem for a file named
> > "hbase.version" in your HBase root folder. If the version of your HBase
> > distribution changed OR if the file is missing, it will throw the error
> you
> > see. I would advise confirming that the file is still there.
> >
> > Also, the migrate script first tries to connect to the Master to make
> sure
> > it is offline. It retries a couple of times (the error you see) then
> > proceed
> > to doing the migration.
> >
> > Hope it helps,
> >
> > J-D
> >
> > On Mon, Jul 21, 2008 at 5:17 AM, Yabo-Arber Xu <arber.research@gmail.com
> >
> > wrote:
> >
> > > Greetings,
> > >
> > > I am right now using hbase in our project in "stand-alone" mode. It
> > worked
> > > well until today I found the following message in the log:
> > >
> > > 2008-07-21 01:16:38,564 FATAL org.apache.hadoop.hbase.HMaster: Not
> > starting
> > > HMaster because: org.apache.hadoop.hbase.FileSystemVersionException:
> File
> > > system needs to be upgraded. Run the '${HBASE_HOME}/bin/hbase migrate'
> > > script.
> > >
> > > I actually did not do any version change recently. But, anyway, i
> > followed
> > > the message and tried to do "hbase migrate", but it keeps showing me
> the
> > > following error:
> > >
> > > 08/07/21 02:11:51 INFO ipc.Client: Retrying connect to server:
> localhost/
> > > 127.0.0.1:60000. Already tried 1 time(s).
> > >
> > > The same error occurs when i try to do any query through hbase shell.
> > This
> > > is just wierd....I did not touch any part of the hbase, and the error
> > just
> > > sundenly came out. Any ideas why?
> > >
> > > Thanks for your help! The whole project is just stuck here by this
> error.
> > >
> > > Arber
> >
> >
>
Re: Retrying connect to server error
Posted by "Yabo-Arber, Xu" <yx...@cs.sfu.ca>.
Thanks for your reply, J-D. There is no such file hbase.version under the
hbase root directory, and this morning i found something even worse
happened. The data under hbase root directory ( i use the default
/tmp/hbase-{user.home}/root ) disappeared......originally there was a folder
corresponding to each table and i guess most of the data are physically
stored there.
This is not the first time happening to me. Last time similar things
happened, and i addressed it using a naive way without thinking too much(
re-building the whole database ) and then it runs well for around a week,
and now it cames again. Do you have any idea what happened to the data?
Would it be because i use the /tmp directory which happened to be modified
by the system?
On Mon, Jul 21, 2008 at 7:58 AM, Jean-Daniel Cryans <jd...@gmail.com>
wrote:
> Arber,
>
> When the Master starts, it checks on the filesystem for a file named
> "hbase.version" in your HBase root folder. If the version of your HBase
> distribution changed OR if the file is missing, it will throw the error you
> see. I would advise confirming that the file is still there.
>
> Also, the migrate script first tries to connect to the Master to make sure
> it is offline. It retries a couple of times (the error you see) then
> proceed
> to doing the migration.
>
> Hope it helps,
>
> J-D
>
> On Mon, Jul 21, 2008 at 5:17 AM, Yabo-Arber Xu <ar...@gmail.com>
> wrote:
>
> > Greetings,
> >
> > I am right now using hbase in our project in "stand-alone" mode. It
> worked
> > well until today I found the following message in the log:
> >
> > 2008-07-21 01:16:38,564 FATAL org.apache.hadoop.hbase.HMaster: Not
> starting
> > HMaster because: org.apache.hadoop.hbase.FileSystemVersionException: File
> > system needs to be upgraded. Run the '${HBASE_HOME}/bin/hbase migrate'
> > script.
> >
> > I actually did not do any version change recently. But, anyway, i
> followed
> > the message and tried to do "hbase migrate", but it keeps showing me the
> > following error:
> >
> > 08/07/21 02:11:51 INFO ipc.Client: Retrying connect to server: localhost/
> > 127.0.0.1:60000. Already tried 1 time(s).
> >
> > The same error occurs when i try to do any query through hbase shell.
> This
> > is just wierd....I did not touch any part of the hbase, and the error
> just
> > sundenly came out. Any ideas why?
> >
> > Thanks for your help! The whole project is just stuck here by this error.
> >
> > Arber
>
>
Re: Retrying connect to server error
Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Arber,
When the Master starts, it checks on the filesystem for a file named
"hbase.version" in your HBase root folder. If the version of your HBase
distribution changed OR if the file is missing, it will throw the error you
see. I would advise confirming that the file is still there.
Also, the migrate script first tries to connect to the Master to make sure
it is offline. It retries a couple of times (the error you see) then proceed
to doing the migration.
Hope it helps,
J-D
On Mon, Jul 21, 2008 at 5:17 AM, Yabo-Arber Xu <ar...@gmail.com>
wrote:
> Greetings,
>
> I am right now using hbase in our project in "stand-alone" mode. It worked
> well until today I found the following message in the log:
>
> 2008-07-21 01:16:38,564 FATAL org.apache.hadoop.hbase.HMaster: Not starting
> HMaster because: org.apache.hadoop.hbase.FileSystemVersionException: File
> system needs to be upgraded. Run the '${HBASE_HOME}/bin/hbase migrate'
> script.
>
> I actually did not do any version change recently. But, anyway, i followed
> the message and tried to do "hbase migrate", but it keeps showing me the
> following error:
>
> 08/07/21 02:11:51 INFO ipc.Client: Retrying connect to server: localhost/
> 127.0.0.1:60000. Already tried 1 time(s).
>
> The same error occurs when i try to do any query through hbase shell. This
> is just wierd....I did not touch any part of the hbase, and the error just
> sundenly came out. Any ideas why?
>
> Thanks for your help! The whole project is just stuck here by this error.
>
> Arber
>
>
> --
> Yabo-Arber Xu <yx...@cs.sfu.ca>
> Web: http://www.cs.sfu.ca/~yxu/personal/<http://www.cs.sfu.ca/%7Eyxu/personal/>
> <http://www.cs.sfu.ca/%7Eyxu/personal/>
>