You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Prakrati Agrawal <Pr...@mu-sigma.com> on 2012/07/18 07:57:21 UTC
Hbase throwing error on batch inserts
Dear all,
I have an internal cluster of 11 machines with Hadoop and Hbase installed on it.
I created a table with 3 column families and was trying to do batch inserts of 75000 rows from a SQL database using Java.
I get the following error:
Exception in thread "main" org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 74999 actions: NotServingRegionException: 74999 times, servers with issues: ub2:60020,
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1485)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1499)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:859)
at com.musigma.hbaseTrigger.HBaseImport.batchInserts(HBaseImport.java:137)
at com.musigma.hbaseTrigger.HbaseTrigger.main(HbaseTrigger.java:64)
The machine on which the error occurs has a lot of free space(67 %) and Hbase is running successfully on it.
When I reduce the batch inserts from 75000 to 50000, my code works and the data is inserted successfully.
I am not understanding what exactly is happening in Hbase which is causing this error to occur.
Thanks and Regards
Prakrati
________________________________
This email message may contain proprietary, private and confidential information. The information transmitted is intended only for the person(s) or entities to which it is addressed. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited and may be illegal. If you received this in error, please contact the sender and delete the message from your system.
Mu Sigma takes all reasonable steps to ensure that its electronic communications are free from viruses. However, given Internet accessibility, the Company cannot accept liability for any virus introduced by this e-mail or any attachment and you are advised to use up-to-date virus checking software.
RE: Hbase throwing error on batch inserts
Posted by Prakrati Agrawal <Pr...@mu-sigma.com>.
Yes,
I am putting puts in a list before I call Table.put, but it was working fine when I was doing the same for a table which has only 2 column families.
Thanks
Prakrati
-----Original Message-----
From: John Hancock [mailto:jhancock1975@gmail.com]
Sent: Wednesday, July 18, 2012 3:00 PM
To: user@hbase.apache.org
Subject: Re: Hbase throwing error on batch inserts
Prakrati,
Are you putting all of the puts in a list and then doing them all at once?
Without having seen the code (which you may want to post to this list)
I am guessing you are putting to many puts in a list before you call Table.put, which is why it works for 50,000 but not 75,000.
-John
On Wed, Jul 18, 2012 at 1:57 AM, Prakrati Agrawal < Prakrati.Agrawal@mu-sigma.com> wrote:
> Dear all,
>
> I have an internal cluster of 11 machines with Hadoop and Hbase
> installed on it.
> I created a table with 3 column families and was trying to do batch
> inserts of 75000 rows from a SQL database using Java.
> I get the following error:
>
> Exception in thread "main"
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
> Failed
> 74999 actions: NotServingRegionException: 74999 times, servers with issues:
> ub2:60020,
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1485)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1499)
> at
> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:859)
> at
> com.musigma.hbaseTrigger.HBaseImport.batchInserts(HBaseImport.java:137)
> at
> com.musigma.hbaseTrigger.HbaseTrigger.main(HbaseTrigger.java:64)
>
> The machine on which the error occurs has a lot of free space(67 %)
> and Hbase is running successfully on it.
>
> When I reduce the batch inserts from 75000 to 50000, my code works and
> the data is inserted successfully.
> I am not understanding what exactly is happening in Hbase which is
> causing this error to occur.
>
> Thanks and Regards
> Prakrati
>
> ________________________________
> This email message may contain proprietary, private and confidential
> information. The information transmitted is intended only for the
> person(s) or entities to which it is addressed. Any review,
> retransmission, dissemination or other use of, or taking of any action
> in reliance upon, this information by persons or entities other than
> the intended recipient is prohibited and may be illegal. If you
> received this in error, please contact the sender and delete the message from your system.
>
> Mu Sigma takes all reasonable steps to ensure that its electronic
> communications are free from viruses. However, given Internet
> accessibility, the Company cannot accept liability for any virus
> introduced by this e-mail or any attachment and you are advised to use
> up-to-date virus checking software.
>
This email message may contain proprietary, private and confidential information. The information transmitted is intended only for the person(s) or entities to which it is addressed. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited and may be illegal. If you received this in error, please contact the sender and delete the message from your system.
Mu Sigma takes all reasonable steps to ensure that its electronic communications are free from viruses. However, given Internet accessibility, the Company cannot accept liability for any virus introduced by this e-mail or any attachment and you are advised to use up-to-date virus checking software.
Re: Hbase throwing error on batch inserts
Posted by John Hancock <jh...@gmail.com>.
Prakrati,
Are you putting all of the puts in a list and then doing them all at once?
Without having seen the code (which you may want to post to this list)
I am guessing you are putting to many puts in a list before you call
Table.put, which is why it works for 50,000 but not 75,000.
-John
On Wed, Jul 18, 2012 at 1:57 AM, Prakrati Agrawal <
Prakrati.Agrawal@mu-sigma.com> wrote:
> Dear all,
>
> I have an internal cluster of 11 machines with Hadoop and Hbase installed
> on it.
> I created a table with 3 column families and was trying to do batch
> inserts of 75000 rows from a SQL database using Java.
> I get the following error:
>
> Exception in thread "main"
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
> 74999 actions: NotServingRegionException: 74999 times, servers with issues:
> ub2:60020,
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1485)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1499)
> at
> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:859)
> at
> com.musigma.hbaseTrigger.HBaseImport.batchInserts(HBaseImport.java:137)
> at com.musigma.hbaseTrigger.HbaseTrigger.main(HbaseTrigger.java:64)
>
> The machine on which the error occurs has a lot of free space(67 %) and
> Hbase is running successfully on it.
>
> When I reduce the batch inserts from 75000 to 50000, my code works and the
> data is inserted successfully.
> I am not understanding what exactly is happening in Hbase which is causing
> this error to occur.
>
> Thanks and Regards
> Prakrati
>
> ________________________________
> This email message may contain proprietary, private and confidential
> information. The information transmitted is intended only for the person(s)
> or entities to which it is addressed. Any review, retransmission,
> dissemination or other use of, or taking of any action in reliance upon,
> this information by persons or entities other than the intended recipient
> is prohibited and may be illegal. If you received this in error, please
> contact the sender and delete the message from your system.
>
> Mu Sigma takes all reasonable steps to ensure that its electronic
> communications are free from viruses. However, given Internet
> accessibility, the Company cannot accept liability for any virus introduced
> by this e-mail or any attachment and you are advised to use up-to-date
> virus checking software.
>