You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jaeyun Noh <me...@gmail.com> on 2008/10/06 20:19:56 UTC
Other CC examined in HBase transaction model?
Hi,
I looked at the source codes implemented to support Multi-region transaction
feature. (HBASE-669)
It uses optimistic concurrency control. Did any guys consider other CC like
timestamp-based CC or multiversion CC?
I think it's okay in read-dominated and relatively short transaction
workloads. In other case, is MVCC more scalable rather than OCC?
What do you think of pros/cons of other concurrency control schemes in Hbase
transactions?
Just I'm curious that hbase transaction uses other CCs or not.
Regards,
Jaeyun Noh.
Re: Re: What is the impact on RegionServer Crash?
Posted by "Qian, Ling" <la...@gmail.com>.
Stack,
Thanks for your explanations!
2008-10-13
Qian, Ling
发件人： stack
发送时间： 2008-10-11 12:07:42
收件人： hbase-user@hadoop.apache.org
抄送：
主题： Re: What is the impact on RegionServer Crash?
larryqian wrote:
> Dear All,
>
> During a data query or any of the data operation, if the coresponding Region Server crashes, will the query continue to work on another Region Server? Or the query fails and return error to Application?
>
Depends. Recovery from crash may take some time; a second or two and
even up to minutes if the crashed server was carrying catalog table
regions. Client may time out in the meantime. If the region the query
goes against comes back on line promptly, then client will return query
results as though the crash had not happend.
> If a free Region Server crashes, the master will find and reschdule the assigned task after 1 hearbeat, right?
>
Its all configurable but no, not after one heartbeat. Heartbeats are
every couple of seconds.
<property>
<name>hbase.regionserver.msginterval</name>
<value>3000</value>
<description>Interval between messages from the RegionServer to HMaster
in milliseconds. Default is 3 seconds.
</description>
</property>
Master will consider regionserver dead only after the regionservers'
lease expires in the master. This is the below configuration:
<property>
<name>hbase.master.lease.period</name>
<value>120000</value>
<description>HMaster server lease period in milliseconds. Default is
120 seconds. Region servers must report in within this period else
they are considered dead. On loaded cluster, may need to up this
period.</description>
</property>
St.Ack
Re: What is the impact on RegionServer Crash?
Posted by stack <st...@duboce.net>.
larryqian wrote:
> Dear All,
>
> During a data query or any of the data operation, if the coresponding Region Server crashes, will the query continue to work on another Region Server? Or the query fails and return error to Application?
>
Depends. Recovery from crash may take some time; a second or two and
even up to minutes if the crashed server was carrying catalog table
regions. Client may time out in the meantime. If the region the query
goes against comes back on line promptly, then client will return query
results as though the crash had not happend.
> If a free Region Server crashes, the master will find and reschdule the assigned task after 1 hearbeat, right?
>
Its all configurable but no, not after one heartbeat. Heartbeats are
every couple of seconds.
<property>
<name>hbase.regionserver.msginterval</name>
<value>3000</value>
<description>Interval between messages from the RegionServer to HMaster
in milliseconds. Default is 3 seconds.
</description>
</property>
Master will consider regionserver dead only after the regionservers'
lease expires in the master. This is the below configuration:
<property>
<name>hbase.master.lease.period</name>
<value>120000</value>
<description>HMaster server lease period in milliseconds. Default is
120 seconds. Region servers must report in within this period else
they are considered dead. On loaded cluster, may need to up this
period.</description>
</property>
St.Ack
What is the impact on RegionServer Crash?
Posted by larryqian <la...@gmail.com>.
Dear All,
During a data query or any of the data operation, if the coresponding Region Server crashes, will the query continue to work on another Region Server? Or the query fails and return error to Application?
If a free Region Server crashes, the master will find and reschdule the assigned task after 1 hearbeat, right?
Thanks
Qian, Ling
Re: Other CC examined in HBase transaction model?
Posted by Clint Morgan <cl...@gmail.com>.
OCC got into 0.18.0. See the package.html javadoc in
o.a.h.h.client.transactional for setup/usage instructions.
My secondary index patch is not in the trunk (or release) yet. You can try
it out here: https://issues.apache.org/jira/browse/HBASE-883. If you do try
it out, please comment in the jira how it worked, how you like the API, etc.
I attempt to respect transactional properties with the index updates, but
this has not been thoroughly tested.
cheers,
-clint
On Tue, Oct 7, 2008 at 3:54 PM, Ski Gh3 <sk...@gmail.com> wrote:
> Two quick follow-up questions:
>
> Which version of hbase do we need to download in order to use to OCC
> functionality?0.18.0?
> I guess no setting is required by the user, where hregionserver was used
> will now be transactionalregion, right?
>
> How can this be combined with secondary index? Didn't find the code for
> that
> in 0.18.0, is that still
> work in progress?
>
> Thanks a lot!
>
> On Mon, Oct 6, 2008 at 2:26 PM, Clint Morgan <cl...@gmail.com> wrote:
>
> > I implemented the OCC functionality. Went that route because it seemed
> > relatively straightforward and has acceptable performance for our
> > workloads.
> > Also I saw something somewhere that made me think that google was using
> OCC
> > for transactions in AppEngine.
> >
> > My intuition is that where the OCC approach really gets into trouble is
> > when
> > transactions start to frequently conflict (EG, when the "optimistic" hope
> > no
> > longer holds). Of course, as you increase the life of a transaction, you
> > increase the probability it will conflict with another.
> >
> > I don't know enough about timstamp-, or MV-CCs to compare "scalability"
> to
> > the OCC approach.
> >
> > -clint
> >
> > On Mon, Oct 6, 2008 at 11:19 AM, Jaeyun Noh <me...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I looked at the source codes implemented to support Multi-region
> > > transaction
> > > feature. (HBASE-669)
> > > It uses optimistic concurrency control. Did any guys consider other CC
> > like
> > > timestamp-based CC or multiversion CC?
> > >
> > > I think it's okay in read-dominated and relatively short transaction
> > > workloads. In other case, is MVCC more scalable rather than OCC?
> > > What do you think of pros/cons of other concurrency control schemes in
> > > Hbase
> > > transactions?
> > >
> > > Just I'm curious that hbase transaction uses other CCs or not.
> > >
> > > Regards,
> > > Jaeyun Noh.
> > >
> >
>
Re: Other CC examined in HBase transaction model?
Posted by Ski Gh3 <sk...@gmail.com>.
Two quick follow-up questions:
Which version of hbase do we need to download in order to use to OCC
functionality?0.18.0?
I guess no setting is required by the user, where hregionserver was used
will now be transactionalregion, right?
How can this be combined with secondary index? Didn't find the code for that
in 0.18.0, is that still
work in progress?
Thanks a lot!
On Mon, Oct 6, 2008 at 2:26 PM, Clint Morgan <cl...@gmail.com> wrote:
> I implemented the OCC functionality. Went that route because it seemed
> relatively straightforward and has acceptable performance for our
> workloads.
> Also I saw something somewhere that made me think that google was using OCC
> for transactions in AppEngine.
>
> My intuition is that where the OCC approach really gets into trouble is
> when
> transactions start to frequently conflict (EG, when the "optimistic" hope
> no
> longer holds). Of course, as you increase the life of a transaction, you
> increase the probability it will conflict with another.
>
> I don't know enough about timstamp-, or MV-CCs to compare "scalability" to
> the OCC approach.
>
> -clint
>
> On Mon, Oct 6, 2008 at 11:19 AM, Jaeyun Noh <me...@gmail.com> wrote:
>
> > Hi,
> >
> > I looked at the source codes implemented to support Multi-region
> > transaction
> > feature. (HBASE-669)
> > It uses optimistic concurrency control. Did any guys consider other CC
> like
> > timestamp-based CC or multiversion CC?
> >
> > I think it's okay in read-dominated and relatively short transaction
> > workloads. In other case, is MVCC more scalable rather than OCC?
> > What do you think of pros/cons of other concurrency control schemes in
> > Hbase
> > transactions?
> >
> > Just I'm curious that hbase transaction uses other CCs or not.
> >
> > Regards,
> > Jaeyun Noh.
> >
>
Re: Other CC examined in HBase transaction model?
Posted by Clint Morgan <cl...@gmail.com>.
I implemented the OCC functionality. Went that route because it seemed
relatively straightforward and has acceptable performance for our workloads.
Also I saw something somewhere that made me think that google was using OCC
for transactions in AppEngine.
My intuition is that where the OCC approach really gets into trouble is when
transactions start to frequently conflict (EG, when the "optimistic" hope no
longer holds). Of course, as you increase the life of a transaction, you
increase the probability it will conflict with another.
I don't know enough about timstamp-, or MV-CCs to compare "scalability" to
the OCC approach.
-clint
On Mon, Oct 6, 2008 at 11:19 AM, Jaeyun Noh <me...@gmail.com> wrote:
> Hi,
>
> I looked at the source codes implemented to support Multi-region
> transaction
> feature. (HBASE-669)
> It uses optimistic concurrency control. Did any guys consider other CC like
> timestamp-based CC or multiversion CC?
>
> I think it's okay in read-dominated and relatively short transaction
> workloads. In other case, is MVCC more scalable rather than OCC?
> What do you think of pros/cons of other concurrency control schemes in
> Hbase
> transactions?
>
> Just I'm curious that hbase transaction uses other CCs or not.
>
> Regards,
> Jaeyun Noh.
>