You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Steen Manniche <bo...@gmail.com> on 2017/01/27 08:55:15 UTC
On signalling clients from coprocessors
We have been using coprocessors to create secondary indexes as well as
doing some sanity checks on incoming data.
Especially in the last case, we planned to use the prePut event to reject
Puts to hbase that did not pass our sanity checks for incoming data. I had
expected being able to signal the client ingesting data into hbase via
throwing a custom exception, but learned that the regionserver the
coprocessor is installed on basically only have two options in the face of
a coprocessor throwing an exception: unloading the coprocessor or shutting
down.
This behaviour surprises me since coprocessors almost by definition contain
application logic, and exceptions are a standard way of signalling in
application logic. Are there any plans to make the signal handling work
better wrt. clients in coprocessors? Or have I simply misunderstood the
intention of coprocessors here?
Thanks in advance and best regards,
Steen Manniche
Re: On signalling clients from coprocessors
Posted by Gary Helmling <gh...@gmail.com>.
>
> A jira sounds like a good idea. Even if this is buried somewhere, it's
> clearly not prominent enough.
>
>
+1. Clarifying this in the javadoc and reference guide seems like a good
idea.
Re: On signalling clients from coprocessors
Posted by Sean Busbey <bu...@apache.org>.
A jira sounds like a good idea. Even if this is buried somewhere, it's
clearly not prominent enough.
On Jan 29, 2017 9:43 AM, "Steen Manniche" <bo...@gmail.com> wrote:
> Hi Gary,
>
> thanks for taking the time to answer this. Indeed, from my tests, it seems
> that throwing a CoprocessorException type is carried through to the client
> and will not cause the HBase RegionServer to crash in the process. This is
> great news, and will allow us to implement the application logic in a
> coprocessor instead of outside the HBase service.
>
> I can only wonder why this isn't documented better both in the API docs
> (which says nothing at all) as well as in the HBase book - not one mention
> of the semantics of the CoprocessorException type. Would it be good to open
> an issue on Jira regarding this? Or am I just overlooking where the
> CoprocessorException is described?
>
> On Fri, Jan 27, 2017 at 6:31 PM, Gary Helmling <gh...@gmail.com>
> wrote:
>
> > Did you try throwing CoprocessorException or making your custom
> exception a
> > subclass of it? These should be carried through to the client.
> >
> > Yes, for exceptions outside of this hierarchy, there is no way to know if
> > the exception is recoverable or not, so the safe route is chosen and
> either
> > the regionserver will abort or the coprocessor will be unloaded,
> depending
> > on configuration.
> >
> > On Fri, Jan 27, 2017 at 12:55 AM Steen Manniche <bo...@gmail.com>
> > wrote:
> >
> > > We have been using coprocessors to create secondary indexes as well as
> > > doing some sanity checks on incoming data.
> > >
> > > Especially in the last case, we planned to use the prePut event to
> reject
> > > Puts to hbase that did not pass our sanity checks for incoming data. I
> > had
> > > expected being able to signal the client ingesting data into hbase via
> > > throwing a custom exception, but learned that the regionserver the
> > > coprocessor is installed on basically only have two options in the face
> > of
> > > a coprocessor throwing an exception: unloading the coprocessor or
> > shutting
> > > down.
> > >
> > > This behaviour surprises me since coprocessors almost by definition
> > contain
> > > application logic, and exceptions are a standard way of signalling in
> > > application logic. Are there any plans to make the signal handling work
> > > better wrt. clients in coprocessors? Or have I simply misunderstood the
> > > intention of coprocessors here?
> > >
> > > Thanks in advance and best regards,
> > > Steen Manniche
> > >
> >
>
Re: On signalling clients from coprocessors
Posted by Steen Manniche <bo...@gmail.com>.
I was apparently a bit too naïve in my approach to the exception
handling here. I turns out that the exception bubbling out of hbase
will be the RetriesExhaustedWithDetailsException which contains one or
more originating exceptions. In this case, the
RetriesExhaustedWithDetailsException contained my
MissingIndexKeyException as its first wrapped exception - retrievable
by RetriesExhaustedWithDetailsException#getCause(int)
This works and does not cause the HBase RS to abort.
On Tue, Jan 31, 2017 at 1:00 PM, Steen Manniche <bo...@gmail.com> wrote:
> Hi Gary,
>
> I did some further testing of this in our docker setup, and to my
> surprise, I'm getting a
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException
> where I expected my custom exception (subclass of
> CoprocessorException) to appear. For illustration, I have the
> following in my coprocessor:
>
> if(failureKeys.contains(sourceColumn)) {
> String err = String.format("The key to write an entry in %s was
> empty! It came from %s={%s:%s}", destinationTable, new
> String(sourceKey), "e", sourceColumn);
> LOGGER.error(err);
> throw new MissingIndexKeyException(err);
> }
>
> Unfortunately I get a long timeout followed by the
> RetriesExhaustedWithDetailsException. If I do a try...catch in the
> client sending this put, I would expect that it didn't just continue
> retrying the Put? Is there something I need to signal to the
> regionserver in order for it to stop the processing of the Put and
> just return to the client?
>
> Thanks again for your time.
>
> Best regards,
> Steen
>
> On Sun, Jan 29, 2017 at 4:42 PM, Steen Manniche <bo...@gmail.com> wrote:
>> Hi Gary,
>>
>> thanks for taking the time to answer this. Indeed, from my tests, it seems
>> that throwing a CoprocessorException type is carried through to the client
>> and will not cause the HBase RegionServer to crash in the process. This is
>> great news, and will allow us to implement the application logic in a
>> coprocessor instead of outside the HBase service.
>>
>> I can only wonder why this isn't documented better both in the API docs
>> (which says nothing at all) as well as in the HBase book - not one mention
>> of the semantics of the CoprocessorException type. Would it be good to open
>> an issue on Jira regarding this? Or am I just overlooking where the
>> CoprocessorException is described?
>>
>> On Fri, Jan 27, 2017 at 6:31 PM, Gary Helmling <gh...@gmail.com> wrote:
>>>
>>> Did you try throwing CoprocessorException or making your custom exception
>>> a
>>> subclass of it? These should be carried through to the client.
>>>
>>> Yes, for exceptions outside of this hierarchy, there is no way to know if
>>> the exception is recoverable or not, so the safe route is chosen and
>>> either
>>> the regionserver will abort or the coprocessor will be unloaded, depending
>>> on configuration.
>>>
>>> On Fri, Jan 27, 2017 at 12:55 AM Steen Manniche <bo...@gmail.com>
>>> wrote:
>>>
>>> > We have been using coprocessors to create secondary indexes as well as
>>> > doing some sanity checks on incoming data.
>>> >
>>> > Especially in the last case, we planned to use the prePut event to
>>> > reject
>>> > Puts to hbase that did not pass our sanity checks for incoming data. I
>>> > had
>>> > expected being able to signal the client ingesting data into hbase via
>>> > throwing a custom exception, but learned that the regionserver the
>>> > coprocessor is installed on basically only have two options in the face
>>> > of
>>> > a coprocessor throwing an exception: unloading the coprocessor or
>>> > shutting
>>> > down.
>>> >
>>> > This behaviour surprises me since coprocessors almost by definition
>>> > contain
>>> > application logic, and exceptions are a standard way of signalling in
>>> > application logic. Are there any plans to make the signal handling work
>>> > better wrt. clients in coprocessors? Or have I simply misunderstood the
>>> > intention of coprocessors here?
>>> >
>>> > Thanks in advance and best regards,
>>> > Steen Manniche
>>> >
>>
>>
Re: On signalling clients from coprocessors
Posted by Steen Manniche <bo...@gmail.com>.
Hi Gary,
I did some further testing of this in our docker setup, and to my
surprise, I'm getting a
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException
where I expected my custom exception (subclass of
CoprocessorException) to appear. For illustration, I have the
following in my coprocessor:
if(failureKeys.contains(sourceColumn)) {
String err = String.format("The key to write an entry in %s was
empty! It came from %s={%s:%s}", destinationTable, new
String(sourceKey), "e", sourceColumn);
LOGGER.error(err);
throw new MissingIndexKeyException(err);
}
Unfortunately I get a long timeout followed by the
RetriesExhaustedWithDetailsException. If I do a try...catch in the
client sending this put, I would expect that it didn't just continue
retrying the Put? Is there something I need to signal to the
regionserver in order for it to stop the processing of the Put and
just return to the client?
Thanks again for your time.
Best regards,
Steen
On Sun, Jan 29, 2017 at 4:42 PM, Steen Manniche <bo...@gmail.com> wrote:
> Hi Gary,
>
> thanks for taking the time to answer this. Indeed, from my tests, it seems
> that throwing a CoprocessorException type is carried through to the client
> and will not cause the HBase RegionServer to crash in the process. This is
> great news, and will allow us to implement the application logic in a
> coprocessor instead of outside the HBase service.
>
> I can only wonder why this isn't documented better both in the API docs
> (which says nothing at all) as well as in the HBase book - not one mention
> of the semantics of the CoprocessorException type. Would it be good to open
> an issue on Jira regarding this? Or am I just overlooking where the
> CoprocessorException is described?
>
> On Fri, Jan 27, 2017 at 6:31 PM, Gary Helmling <gh...@gmail.com> wrote:
>>
>> Did you try throwing CoprocessorException or making your custom exception
>> a
>> subclass of it? These should be carried through to the client.
>>
>> Yes, for exceptions outside of this hierarchy, there is no way to know if
>> the exception is recoverable or not, so the safe route is chosen and
>> either
>> the regionserver will abort or the coprocessor will be unloaded, depending
>> on configuration.
>>
>> On Fri, Jan 27, 2017 at 12:55 AM Steen Manniche <bo...@gmail.com>
>> wrote:
>>
>> > We have been using coprocessors to create secondary indexes as well as
>> > doing some sanity checks on incoming data.
>> >
>> > Especially in the last case, we planned to use the prePut event to
>> > reject
>> > Puts to hbase that did not pass our sanity checks for incoming data. I
>> > had
>> > expected being able to signal the client ingesting data into hbase via
>> > throwing a custom exception, but learned that the regionserver the
>> > coprocessor is installed on basically only have two options in the face
>> > of
>> > a coprocessor throwing an exception: unloading the coprocessor or
>> > shutting
>> > down.
>> >
>> > This behaviour surprises me since coprocessors almost by definition
>> > contain
>> > application logic, and exceptions are a standard way of signalling in
>> > application logic. Are there any plans to make the signal handling work
>> > better wrt. clients in coprocessors? Or have I simply misunderstood the
>> > intention of coprocessors here?
>> >
>> > Thanks in advance and best regards,
>> > Steen Manniche
>> >
>
>
Re: On signalling clients from coprocessors
Posted by Steen Manniche <bo...@gmail.com>.
Hi Gary,
thanks for taking the time to answer this. Indeed, from my tests, it seems
that throwing a CoprocessorException type is carried through to the client
and will not cause the HBase RegionServer to crash in the process. This is
great news, and will allow us to implement the application logic in a
coprocessor instead of outside the HBase service.
I can only wonder why this isn't documented better both in the API docs
(which says nothing at all) as well as in the HBase book - not one mention
of the semantics of the CoprocessorException type. Would it be good to open
an issue on Jira regarding this? Or am I just overlooking where the
CoprocessorException is described?
On Fri, Jan 27, 2017 at 6:31 PM, Gary Helmling <gh...@gmail.com> wrote:
> Did you try throwing CoprocessorException or making your custom exception a
> subclass of it? These should be carried through to the client.
>
> Yes, for exceptions outside of this hierarchy, there is no way to know if
> the exception is recoverable or not, so the safe route is chosen and either
> the regionserver will abort or the coprocessor will be unloaded, depending
> on configuration.
>
> On Fri, Jan 27, 2017 at 12:55 AM Steen Manniche <bo...@gmail.com>
> wrote:
>
> > We have been using coprocessors to create secondary indexes as well as
> > doing some sanity checks on incoming data.
> >
> > Especially in the last case, we planned to use the prePut event to reject
> > Puts to hbase that did not pass our sanity checks for incoming data. I
> had
> > expected being able to signal the client ingesting data into hbase via
> > throwing a custom exception, but learned that the regionserver the
> > coprocessor is installed on basically only have two options in the face
> of
> > a coprocessor throwing an exception: unloading the coprocessor or
> shutting
> > down.
> >
> > This behaviour surprises me since coprocessors almost by definition
> contain
> > application logic, and exceptions are a standard way of signalling in
> > application logic. Are there any plans to make the signal handling work
> > better wrt. clients in coprocessors? Or have I simply misunderstood the
> > intention of coprocessors here?
> >
> > Thanks in advance and best regards,
> > Steen Manniche
> >
>
Re: On signalling clients from coprocessors
Posted by Gary Helmling <gh...@gmail.com>.
Did you try throwing CoprocessorException or making your custom exception a
subclass of it? These should be carried through to the client.
Yes, for exceptions outside of this hierarchy, there is no way to know if
the exception is recoverable or not, so the safe route is chosen and either
the regionserver will abort or the coprocessor will be unloaded, depending
on configuration.
On Fri, Jan 27, 2017 at 12:55 AM Steen Manniche <bo...@gmail.com> wrote:
> We have been using coprocessors to create secondary indexes as well as
> doing some sanity checks on incoming data.
>
> Especially in the last case, we planned to use the prePut event to reject
> Puts to hbase that did not pass our sanity checks for incoming data. I had
> expected being able to signal the client ingesting data into hbase via
> throwing a custom exception, but learned that the regionserver the
> coprocessor is installed on basically only have two options in the face of
> a coprocessor throwing an exception: unloading the coprocessor or shutting
> down.
>
> This behaviour surprises me since coprocessors almost by definition contain
> application logic, and exceptions are a standard way of signalling in
> application logic. Are there any plans to make the signal handling work
> better wrt. clients in coprocessors? Or have I simply misunderstood the
> intention of coprocessors here?
>
> Thanks in advance and best regards,
> Steen Manniche
>