Shared org.apache.hadoop.hbase.client.transactional.TransactionState?

[Chad Hardin <ce...@gmail.com>]

Hello all,

I'm trying to solve the problem of performing operations on a
HTransactionalTable under the context of a long-duration transaction.  My
problem is that I won't know what machines will be involved in the
long-duration transaction beforehand.  The solution I have in mind is to
make TransactionStates some type of remote resource that any machine can
reference at any time.  Is this possible.  Is this a bad idea in general?
How could I share a single TransactionState amongst several machines?


Thanks!
Chad Hardin

Re: Shared org.apache.hadoop.hbase.client.transactional.TransactionState?

[Clint Morgan <cl...@troove.net>]

Each regionserver participating in the transactions holds the writes for its
regions. It also holds info about the reads so it can determine if there is
a conflict with transactions that have recently committed.


On Thu, Jan 28, 2010 at 2:42 PM, Chad Hardin <ce...@gmail.com> wrote:

> I think I'm beginning to understand...
>
> So the state of the transaction (the puts, deletes, etc) is held in memory.
> Is it held in the memory of the HBase Master?
>
>
>
> On Thu, Jan 28, 2010 at 1:10 PM, Clint Morgan <clint.morgan@troove.net
> >wrote:
>
> > The transactional contrib stuff is more geared to short-duration
> > transactions. It relys on OCC, and the optimistic assumption breaks down
> as
> > the transaction gets longer (gives more time for another trx to put in a
> > write which will cause a conflict and then the long trx will have to
> > retry).
> > Also, all the transaction writes are stored in memory until commit-time,
> so
> > this could be an issue if you are write-heavy.
> >
> > That said, it should be possible to share transaction states across
> > machines. Just have to make sure that all regions in the get into the
> final
> > state that is used to commit and be sure that the commit process is only
> > triggered from a single client. I don't have any advice for the best way
> to
> > do this...
> >
> > cheers,
> > -clint
> >
> > On Thu, Jan 28, 2010 at 12:20 PM, Chad Hardin <ce...@gmail.com>
> wrote:
> >
> > > Hello all,
> > >
> > > I'm trying to solve the problem of performing operations on a
> > > HTransactionalTable under the context of a long-duration transaction.
>  My
> > > problem is that I won't know what machines will be involved in the
> > > long-duration transaction beforehand.  The solution I have in mind is
> to
> > > make TransactionStates some type of remote resource that any machine
> can
> > > reference at any time.  Is this possible.  Is this a bad idea in
> general?
> > > How could I share a single TransactionState amongst several machines?
> > >
> > >
> > > Thanks!
> > > Chad Hardin
> > >
> >
>

Re: Shared org.apache.hadoop.hbase.client.transactional.TransactionState?

[Chad Hardin <ce...@gmail.com>]

I think I'm beginning to understand...

So the state of the transaction (the puts, deletes, etc) is held in memory.
Is it held in the memory of the HBase Master?



On Thu, Jan 28, 2010 at 1:10 PM, Clint Morgan <cl...@troove.net>wrote:

> The transactional contrib stuff is more geared to short-duration
> transactions. It relys on OCC, and the optimistic assumption breaks down as
> the transaction gets longer (gives more time for another trx to put in a
> write which will cause a conflict and then the long trx will have to
> retry).
> Also, all the transaction writes are stored in memory until commit-time, so
> this could be an issue if you are write-heavy.
>
> That said, it should be possible to share transaction states across
> machines. Just have to make sure that all regions in the get into the final
> state that is used to commit and be sure that the commit process is only
> triggered from a single client. I don't have any advice for the best way to
> do this...
>
> cheers,
> -clint
>
> On Thu, Jan 28, 2010 at 12:20 PM, Chad Hardin <ce...@gmail.com> wrote:
>
> > Hello all,
> >
> > I'm trying to solve the problem of performing operations on a
> > HTransactionalTable under the context of a long-duration transaction.  My
> > problem is that I won't know what machines will be involved in the
> > long-duration transaction beforehand.  The solution I have in mind is to
> > make TransactionStates some type of remote resource that any machine can
> > reference at any time.  Is this possible.  Is this a bad idea in general?
> > How could I share a single TransactionState amongst several machines?
> >
> >
> > Thanks!
> > Chad Hardin
> >
>

Re: Shared org.apache.hadoop.hbase.client.transactional.TransactionState?

[Clint Morgan <cl...@troove.net>]

The transactional contrib stuff is more geared to short-duration
transactions. It relys on OCC, and the optimistic assumption breaks down as
the transaction gets longer (gives more time for another trx to put in a
write which will cause a conflict and then the long trx will have to retry).
Also, all the transaction writes are stored in memory until commit-time, so
this could be an issue if you are write-heavy.

That said, it should be possible to share transaction states across
machines. Just have to make sure that all regions in the get into the final
state that is used to commit and be sure that the commit process is only
triggered from a single client. I don't have any advice for the best way to
do this...

cheers,
-clint

On Thu, Jan 28, 2010 at 12:20 PM, Chad Hardin <ce...@gmail.com> wrote:

> Hello all,
>
> I'm trying to solve the problem of performing operations on a
> HTransactionalTable under the context of a long-duration transaction.  My
> problem is that I won't know what machines will be involved in the
> long-duration transaction beforehand.  The solution I have in mind is to
> make TransactionStates some type of remote resource that any machine can
> reference at any time.  Is this possible.  Is this a bad idea in general?
> How could I share a single TransactionState amongst several machines?
>
>
> Thanks!
> Chad Hardin
>