You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Ramkrishna.S.Vasudevan" <ra...@huawei.com> on 2012/03/22 15:59:02 UTC
Providing coprocessor hooks in rollbackMemStore while doing puts
With coprocessors hooks while put happens we have the provision to create
new puts to other tables or regions. These puts can be done with writeToWal
as false.
In 0.94 and above the puts are first written to memstore and then to WAL.
If any failure in the WAL append or sync the memstore is rollbacked.
Now the problem is that if the put that happens in the main flow fails there
is no way to rollback the
puts that happened in the prePut.
Can we add hooks like preRollbackMemstore and postRollBackMemstore so that I
will be atleast able to rollback the entries?
Can we raise a JIRA for solving this.
RE: Providing coprocessor hooks in rollbackMemStore while doing puts
Posted by "Ramkrishna.S.Vasudevan" <ra...@huawei.com>.
Hi
Currently I too feel one is enough as per the use case that I could think
of.
Regards
Ram
> -----Original Message-----
> From: Ted Yu [mailto:yuzhihong@gmail.com]
> Sent: Thursday, March 22, 2012 8:40 PM
> To: dev@hbase.apache.org
> Cc: rama krishna
> Subject: Re: Providing coprocessor hooks in rollbackMemStore while
> doing puts
>
> Ram:
> Your proposal makes sense.
> Do we need two callbacks here ?
>
> On Thu, Mar 22, 2012 at 7:59 AM, Ramkrishna.S.Vasudevan <
> ramkrishna.vasudevan@huawei.com> wrote:
>
> > With coprocessors hooks while put happens we have the provision to
> create
> > new puts to other tables or regions. These puts can be done with
> > writeToWal
> > as false.
> > In 0.94 and above the puts are first written to memstore and then to
> WAL.
> > If any failure in the WAL append or sync the memstore is rollbacked.
> > Now the problem is that if the put that happens in the main flow
> fails
> > there
> > is no way to rollback the
> > puts that happened in the prePut.
> >
> > Can we add hooks like preRollbackMemstore and postRollBackMemstore so
> that
> > I
> > will be atleast able to rollback the entries?
> >
> > Can we raise a JIRA for solving this.
> >
> >
> >
> >
Re: Providing coprocessor hooks in rollbackMemStore while doing puts
Posted by Ted Yu <yu...@gmail.com>.
Ram:
Your proposal makes sense.
Do we need two callbacks here ?
On Thu, Mar 22, 2012 at 7:59 AM, Ramkrishna.S.Vasudevan <
ramkrishna.vasudevan@huawei.com> wrote:
> With coprocessors hooks while put happens we have the provision to create
> new puts to other tables or regions. These puts can be done with
> writeToWal
> as false.
> In 0.94 and above the puts are first written to memstore and then to WAL.
> If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails
> there
> is no way to rollback the
> puts that happened in the prePut.
>
> Can we add hooks like preRollbackMemstore and postRollBackMemstore so that
> I
> will be atleast able to rollback the entries?
>
> Can we raise a JIRA for solving this.
>
>
>
>
RE: Providing coprocessor hooks in rollbackMemStore while doing puts
Posted by Anoop Sam John <an...@huawei.com>.
Hi
What about providing more hooks with the batch put operation? A pre and post hooks for the entire batch operation. Before startRegionOperation() and after the closeRegionOperation()? [ In put(Pair<Put,Integer>[] putsAndLocks) ]
Also one more issue regarding the hooks at the miniBatchPut
During this doMiniBatchPut, the prePut () will be called for each and every Put in the list one after the other. Later in doMiniBatchPut when it tries to acquire the rowlocks, it will try to get atleast one lock. For those rows it is able to acquire locks, those Puts only will happen. Then the loop in the method, put(Pair<Put,Integer>[] putsAndLocks) will call doMiniBatchPut again. During this time also we will call prePut() for every puts in the original puts list. The pre hook is called for all rows not even checking whether the lock for the same is available or not. As per the comments in the prePut() call it is called outside the lock to avoid the possible deadlocks. But calling the prePut() more times for same Put, will it be an issue? If kernel is not handling this, may be the CP need to handle this.... Any comments?
As per the code in the HRegion. put() -> The prePut is called after acquiring the rowlock.
Thanks & Regards
-Anoop-
________________________________________
From: Andrew Purtell [apurtell@yahoo.com]
Sent: Friday, March 23, 2012 2:14 AM
To: dev@hbase.apache.org
Subject: Re: Providing coprocessor hooks in rollbackMemStore while doing puts
+1
Best regards,
- Andy
On Mar 22, 2012, at 9:24 AM, lars hofhansl <lh...@yahoo.com> wrote:
> +1
> That seems like a hole in the coprocessor framework that we missed when we did added the rollback logic.
>
>
>
> ________________________________
> From: Ramkrishna.S.Vasudevan <ra...@huawei.com>
> To: dev@hbase.apache.org
> Cc: 'rama krishna' <ra...@hotmail.com>
> Sent: Thursday, March 22, 2012 7:59 AM
> Subject: Providing coprocessor hooks in rollbackMemStore while doing puts
>
> With coprocessors hooks while put happens we have the provision to create
> new puts to other tables or regions. These puts can be done with writeToWal
> as false.
> In 0.94 and above the puts are first written to memstore and then to WAL.
> If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there
> is no way to rollback the
> puts that happened in the prePut.
>
> Can we add hooks like preRollbackMemstore and postRollBackMemstore so that I
> will be atleast able to rollback the entries?
>
> Can we raise a JIRA for solving this.
Re: Providing coprocessor hooks in rollbackMemStore while doing puts
Posted by Andrew Purtell <ap...@yahoo.com>.
+1
Best regards,
- Andy
On Mar 22, 2012, at 9:24 AM, lars hofhansl <lh...@yahoo.com> wrote:
> +1
> That seems like a hole in the coprocessor framework that we missed when we did added the rollback logic.
>
>
>
> ________________________________
> From: Ramkrishna.S.Vasudevan <ra...@huawei.com>
> To: dev@hbase.apache.org
> Cc: 'rama krishna' <ra...@hotmail.com>
> Sent: Thursday, March 22, 2012 7:59 AM
> Subject: Providing coprocessor hooks in rollbackMemStore while doing puts
>
> With coprocessors hooks while put happens we have the provision to create
> new puts to other tables or regions. These puts can be done with writeToWal
> as false.
> In 0.94 and above the puts are first written to memstore and then to WAL.
> If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there
> is no way to rollback the
> puts that happened in the prePut.
>
> Can we add hooks like preRollbackMemstore and postRollBackMemstore so that I
> will be atleast able to rollback the entries?
>
> Can we raise a JIRA for solving this.
Re: Providing coprocessor hooks in rollbackMemStore while doing puts
Posted by lars hofhansl <lh...@yahoo.com>.
+1
That seems like a hole in the coprocessor framework that we missed when we did added the rollback logic.
________________________________
From: Ramkrishna.S.Vasudevan <ra...@huawei.com>
To: dev@hbase.apache.org
Cc: 'rama krishna' <ra...@hotmail.com>
Sent: Thursday, March 22, 2012 7:59 AM
Subject: Providing coprocessor hooks in rollbackMemStore while doing puts
With coprocessors hooks while put happens we have the provision to create
new puts to other tables or regions. These puts can be done with writeToWal
as false.
In 0.94 and above the puts are first written to memstore and then to WAL.
If any failure in the WAL append or sync the memstore is rollbacked.
Now the problem is that if the put that happens in the main flow fails there
is no way to rollback the
puts that happened in the prePut.
Can we add hooks like preRollbackMemstore and postRollBackMemstore so that I
will be atleast able to rollback the entries?
Can we raise a JIRA for solving this.