You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jean-Marc Spaggiari <je...@spaggiari.org> on 2012/07/09 14:25:28 UTC
Improvement: Provide better feedback on Put to unknown CF
Hi,
When we try to add a value to a CF which does not exist on a table, we
are getting the error below. I think this is not really giving the
right information about the issue.
Should it not be better to provide an exception like
UnknownColumnFamillyException?
JM
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
phenom:60020,
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
at org.myapp.app.Integrator.main(Integrator.java:162)
Re: Improvement: Provide better feedback on Put to unknown CF
Posted by yu...@gmail.com.
I agree.
On Jul 9, 2012, at 5:25 AM, Jean-Marc Spaggiari <je...@spaggiari.org> wrote:
> Hi,
>
> When we try to add a value to a CF which does not exist on a table, we
> are getting the error below. I think this is not really giving the
> right information about the issue.
>
> Should it not be better to provide an exception like
> UnknownColumnFamillyException?
>
> JM
>
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
> phenom:60020,
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
> at org.myapp.app.Integrator.main(Integrator.java:162)
Re: Improvement: Provide better feedback on Put to unknown CF
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Hi Michael,
I agree that in the code we have access to all the information to
access the right column.
However, let's imagine the column family name is dynamically retrieved
from a property file, and there is a typo. Or, another process removed
the column family. Or there is a bug in the code, and so on.
There is many possibilities why an application might try to access a
CF which, at the end, doesn't exist in the table. I agree it should
have been checked from the meta before, but skeeping that step might
be required to improve performances.
Adding such exception will not have any negative impact on perfs,
readability, etc. It will simply help a lot the defect tracking when
someone will face the issue and see the stack trace.
JM
2012/7/9, Michael Segel <mi...@hotmail.com>:
> Jean-Marc,
>
> I think you mis understood.
> At run time, you can query HBase to find out the table schema and its column
> families.
>
> While I agree that you are seeing poorly written exceptions, IMHO its easier
> to avoid the problem in the first place.
>
> In a Map/Reduce in side the mapper class, you have everything you need to
> get the table's schema.
> From that you can see the column families.
>
>
> HTH
>
> -Mike
>
> On Jul 9, 2012, at 8:42 AM, Jean-Marc Spaggiari wrote:
>
>> In my case it was a codding issue. Used the wrong final byte array to
>> access the CF. So I agree, the CF is well known since you create the
>> table based on them. But maybe you have added some other CFs later and
>> something went wrong?
>>
>> It's just that based on the exception received, there is no indication
>> that there might be some issues with the CF. So you might end trying
>> to figure what the issue is far from where it's really.
>>
>> 2012/7/9, Michael Segel <mi...@hotmail.com>:
>>> This may beg the question ...
>>> Why do you not know the CF?
>>>
>>> Your table schemas only consist of tables and CFs. So you should know
>>> them
>>> at the start of your job or m/r Mapper.setup();
>>>
>>>
>>> On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:
>>>
>>>> Hi,
>>>>
>>>> When we try to add a value to a CF which does not exist on a table, we
>>>> are getting the error below. I think this is not really giving the
>>>> right information about the issue.
>>>>
>>>> Should it not be better to provide an exception like
>>>> UnknownColumnFamillyException?
>>>>
>>>> JM
>>>>
>>>> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>>>> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
>>>> phenom:60020,
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
>>>> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
>>>> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
>>>> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
>>>> at org.myapp.app.Integrator.main(Integrator.java:162)
>>>>
>>>
>>>
>>
>
>
Re: Improvement: Provide better feedback on Put to unknown CF
Posted by Michael Segel <mi...@hotmail.com>.
Jean-Marc,
I think you mis understood.
At run time, you can query HBase to find out the table schema and its column families.
While I agree that you are seeing poorly written exceptions, IMHO its easier to avoid the problem in the first place.
In a Map/Reduce in side the mapper class, you have everything you need to get the table's schema.
From that you can see the column families.
HTH
-Mike
On Jul 9, 2012, at 8:42 AM, Jean-Marc Spaggiari wrote:
> In my case it was a codding issue. Used the wrong final byte array to
> access the CF. So I agree, the CF is well known since you create the
> table based on them. But maybe you have added some other CFs later and
> something went wrong?
>
> It's just that based on the exception received, there is no indication
> that there might be some issues with the CF. So you might end trying
> to figure what the issue is far from where it's really.
>
> 2012/7/9, Michael Segel <mi...@hotmail.com>:
>> This may beg the question ...
>> Why do you not know the CF?
>>
>> Your table schemas only consist of tables and CFs. So you should know them
>> at the start of your job or m/r Mapper.setup();
>>
>>
>> On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:
>>
>>> Hi,
>>>
>>> When we try to add a value to a CF which does not exist on a table, we
>>> are getting the error below. I think this is not really giving the
>>> right information about the issue.
>>>
>>> Should it not be better to provide an exception like
>>> UnknownColumnFamillyException?
>>>
>>> JM
>>>
>>> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>>> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
>>> phenom:60020,
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
>>> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
>>> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
>>> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
>>> at org.myapp.app.Integrator.main(Integrator.java:162)
>>>
>>
>>
>
Re: Improvement: Provide better feedback on Put to unknown CF
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
In my case it was a codding issue. Used the wrong final byte array to
access the CF. So I agree, the CF is well known since you create the
table based on them. But maybe you have added some other CFs later and
something went wrong?
It's just that based on the exception received, there is no indication
that there might be some issues with the CF. So you might end trying
to figure what the issue is far from where it's really.
2012/7/9, Michael Segel <mi...@hotmail.com>:
> This may beg the question ...
> Why do you not know the CF?
>
> Your table schemas only consist of tables and CFs. So you should know them
> at the start of your job or m/r Mapper.setup();
>
>
> On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:
>
>> Hi,
>>
>> When we try to add a value to a CF which does not exist on a table, we
>> are getting the error below. I think this is not really giving the
>> right information about the issue.
>>
>> Should it not be better to provide an exception like
>> UnknownColumnFamillyException?
>>
>> JM
>>
>> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
>> phenom:60020,
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
>> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
>> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
>> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
>> at org.myapp.app.Integrator.main(Integrator.java:162)
>>
>
>
Re: Improvement: Provide better feedback on Put to unknown CF
Posted by Michael Segel <mi...@hotmail.com>.
This may beg the question ...
Why do you not know the CF?
Your table schemas only consist of tables and CFs. So you should know them at the start of your job or m/r Mapper.setup();
On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:
> Hi,
>
> When we try to add a value to a CF which does not exist on a table, we
> are getting the error below. I think this is not really giving the
> right information about the issue.
>
> Should it not be better to provide an exception like
> UnknownColumnFamillyException?
>
> JM
>
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
> phenom:60020,
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
> at org.myapp.app.Integrator.main(Integrator.java:162)
>