You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Liu Yan <gz...@gmail.com> on 2009/02/23 07:31:03 UTC
HBase Write Failure
hi,
When we run the java client to add data to hbase,
from the beginnng we can add the data to hbase. But running for some times,
when the client tries to add a record to hbase, it shows the exception:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
re
gion server Some server for region , row '176207513941646000', but failed
after
11 attempts.
Exceptions:
java.io.IOException: HRegionInfo was null or empty in .META.
java.io.IOException: HRegionInfo was null or empty in .META.
java.io.IOException: HRegionInfo was null or empty in .META.
java.io.IOException: HRegionInfo was null or empty in .META.
java.io.IOException: HRegionInfo was null or empty in .META.
java.io.IOException: HRegionInfo was null or empty in .META.
java.io.IOException: HRegionInfo was null or empty in .META.
java.io.IOException: HRegionInfo was null or empty in .META.
java.io.IOException: HRegionInfo was null or empty in .META.
java.io.IOException: HRegionInfo was null or empty in .META.
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getReg
ionLocationForRowWithRetries(HConnectionManager.java:900)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.proces
sBatchOfRows(HConnectionManager.java:915)
at
org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1372)
at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1316)
at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1296)
Could anybody help me out of this?
Regards,
Yan
Re: HBase Write Failure
Posted by Liu Yan <gz...@gmail.com>.
Thanks for your. We have 4 Fedora boxes running this Hadoop 0.19.0 and
HBase, they are all EC2 boxes.
We tried to do this on them:
append "- - nofile 32768"to /etc/security/limits.conf
append "fs.file-max=200000" to /etc/sysctl.conf
run "sysctl -p" to apply changes. We restarted HBase and Hadoop after
applying this.
But it again ran to the same error after 4 hours or so.
Regards,
Yan
2009/2/23 Amandeep Khurana <am...@gmail.com>
> I've been dealing with this myself over the last few days. Its got to do
> with the number of open files that hdfs can take. What is your cluster
> config?
>
>
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Sun, Feb 22, 2009 at 10:31 PM, Liu Yan <gz...@gmail.com> wrote:
>
> > hi,
> > When we run the java client to add data to hbase,
> > from the beginnng we can add the data to hbase. But running for some
> times,
> > when the client tries to add a record to hbase, it shows the exception:
> >
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
> contact
> > re
> > gion server Some server for region , row '176207513941646000', but failed
> > after
> > 11 attempts.
> > Exceptions:
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> >
> > at
> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getReg
> > ionLocationForRowWithRetries(HConnectionManager.java:900)
> > at
> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.proces
> > sBatchOfRows(HConnectionManager.java:915)
> > at
> > org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1372)
> > at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1316)
> > at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1296)
> >
> > Could anybody help me out of this?
> >
> > Regards,
> > Yan
> >
>
Re: HBase Write Failure
Posted by Liu Yan <gz...@gmail.com>.
We are trying to increase the max-file setting, and will post you the
results.
Thanks,
Yan
2009/2/23 Amandeep Khurana <am...@gmail.com>
> I've been dealing with this myself over the last few days. Its got to do
> with the number of open files that hdfs can take. What is your cluster
> config?
>
>
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Sun, Feb 22, 2009 at 10:31 PM, Liu Yan <gz...@gmail.com> wrote:
>
> > hi,
> > When we run the java client to add data to hbase,
> > from the beginnng we can add the data to hbase. But running for some
> times,
> > when the client tries to add a record to hbase, it shows the exception:
> >
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
> contact
> > re
> > gion server Some server for region , row '176207513941646000', but failed
> > after
> > 11 attempts.
> > Exceptions:
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> > java.io.IOException: HRegionInfo was null or empty in .META.
> >
> > at
> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getReg
> > ionLocationForRowWithRetries(HConnectionManager.java:900)
> > at
> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.proces
> > sBatchOfRows(HConnectionManager.java:915)
> > at
> > org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1372)
> > at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1316)
> > at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1296)
> >
> > Could anybody help me out of this?
> >
> > Regards,
> > Yan
> >
>
Re: HBase Write Failure
Posted by Amandeep Khurana <am...@gmail.com>.
I've been dealing with this myself over the last few days. Its got to do
with the number of open files that hdfs can take. What is your cluster
config?
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Sun, Feb 22, 2009 at 10:31 PM, Liu Yan <gz...@gmail.com> wrote:
> hi,
> When we run the java client to add data to hbase,
> from the beginnng we can add the data to hbase. But running for some times,
> when the client tries to add a record to hbase, it shows the exception:
>
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
> re
> gion server Some server for region , row '176207513941646000', but failed
> after
> 11 attempts.
> Exceptions:
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
>
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getReg
> ionLocationForRowWithRetries(HConnectionManager.java:900)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.proces
> sBatchOfRows(HConnectionManager.java:915)
> at
> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1372)
> at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1316)
> at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1296)
>
> Could anybody help me out of this?
>
> Regards,
> Yan
>
Re: HBase Write Failure
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Liu,
What's your cluster hardware like and how many machines?
What's your HBase configuration?
Which version of HBase?
Did you meet the requirements stated here
http://hadoop.apache.org/hbase/docs/r0.19.0/api/overview-summary.html#overview_description?
Have you look in your region servers logs and data node logs and do you see
errors stated in the Troubleshooting page of the wiki? (
http://wiki.apache.org/hadoop/Hbase/Troubleshooting)
Thanks!
J-D
On Mon, Feb 23, 2009 at 1:31 AM, Liu Yan <gz...@gmail.com> wrote:
> hi,
> When we run the java client to add data to hbase,
> from the beginnng we can add the data to hbase. But running for some times,
> when the client tries to add a record to hbase, it shows the exception:
>
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
> re
> gion server Some server for region , row '176207513941646000', but failed
> after
> 11 attempts.
> Exceptions:
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
> java.io.IOException: HRegionInfo was null or empty in .META.
>
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getReg
> ionLocationForRowWithRetries(HConnectionManager.java:900)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.proces
> sBatchOfRows(HConnectionManager.java:915)
> at
> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1372)
> at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1316)
> at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1296)
>
> Could anybody help me out of this?
>
> Regards,
> Yan
>