You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by kiran <ki...@gmail.com> on 2012/10/18 08:12:57 UTC
Checking major compaction
Hi all,
Is there a way to check if major compaction is running or not on a table.
--
Thank you
Kiran Sarvabhotla
-----Even a correct decision is wrong when it is taken late
RE: Checking major compaction
Posted by "Ramkrishna.S.Vasudevan" <ra...@huawei.com>.
HBASE-6033 does the work that you ask for. It is currently in Trunk version
of HBase.
Regards
Ram
> -----Original Message-----
> From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> Sent: Thursday, October 18, 2012 11:43 AM
> To: user@hbase.apache.org
> Subject: Checking major compaction
>
> Hi all,
>
> Is there a way to check if major compaction is running or not on a
> table.
>
> --
> Thank you
> Kiran Sarvabhotla
>
> -----Even a correct decision is wrong when it is taken late
RE: Checking major compaction
Posted by "Ramkrishna.S.Vasudevan" <ra...@huawei.com>.
>A simple test would be to just right some 10 rows
I meant to say write some 10 rows.(not right)
Regards
Ram
> -----Original Message-----
> From: Ramkrishna.S.Vasudevan [mailto:ramkrishna.vasudevan@huawei.com]
> Sent: Thursday, October 18, 2012 2:05 PM
> To: user@hbase.apache.org
> Subject: RE: Checking major compaction
>
> Hi
>
> Yes Kiran you can go thro the logs also.
>
> You will see some logs like
> 'Start major compaction for ..............
> 'Compacting file ................
> 'Compacting file ................
> And finally 'Completed major/minor compaction.....'
>
> I just don have some exact logs with me right now. But you can see log
> msgs
> but all comes in debug mode. So ensure you enable debug mode for your
> logs.
>
> A simple test would be to just right some 10 rows. In between do some 4
> to 5
> flushes.
>
> Just give major_compact(tableName) from the shell. You can see the
> logs.
> :)
>
> Regards
> Ram
>
> > -----Original Message-----
> > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> > Sent: Thursday, October 18, 2012 12:03 PM
> > To: user@hbase.apache.org
> > Subject: Re: Checking major compaction
> >
> > Thanks ram,
> >
> > Is there a way can I check it through region server logs. If it is
> > possible
> > what are the statements that I need to look for ??
> >
> > Thanks
> > Kiran
> >
> > On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
> > ramkrishna.vasudevan@huawei.com> wrote:
> >
> > > HBASE-6033 does the work that you ask for. It is currently in
> Trunk
> > > version
> > > of HBase.
> > >
> > > Regards
> > > Ram
> > >
> > > > -----Original Message-----
> > > > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> > > > Sent: Thursday, October 18, 2012 11:43 AM
> > > > To: user@hbase.apache.org
> > > > Subject: Checking major compaction
> > > >
> > > > Hi all,
> > > >
> > > > Is there a way to check if major compaction is running or not on
> a
> > > > table.
> > > >
> > > > --
> > > > Thank you
> > > > Kiran Sarvabhotla
> > > >
> > > > -----Even a correct decision is wrong when it is taken late
> > >
> > >
> >
> >
> > --
> > Thank you
> > Kiran Sarvabhotla
> >
> > -----Even a correct decision is wrong when it is taken late
RE: Checking major compaction
Posted by "Ramkrishna.S.Vasudevan" <ra...@huawei.com>.
Hi
Yes Kiran you can go thro the logs also.
You will see some logs like
'Start major compaction for ..............
'Compacting file ................
'Compacting file ................
And finally 'Completed major/minor compaction.....'
I just don have some exact logs with me right now. But you can see log msgs
but all comes in debug mode. So ensure you enable debug mode for your logs.
A simple test would be to just right some 10 rows. In between do some 4 to 5
flushes.
Just give major_compact(tableName) from the shell. You can see the logs.
:)
Regards
Ram
> -----Original Message-----
> From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> Sent: Thursday, October 18, 2012 12:03 PM
> To: user@hbase.apache.org
> Subject: Re: Checking major compaction
>
> Thanks ram,
>
> Is there a way can I check it through region server logs. If it is
> possible
> what are the statements that I need to look for ??
>
> Thanks
> Kiran
>
> On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
> ramkrishna.vasudevan@huawei.com> wrote:
>
> > HBASE-6033 does the work that you ask for. It is currently in Trunk
> > version
> > of HBase.
> >
> > Regards
> > Ram
> >
> > > -----Original Message-----
> > > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> > > Sent: Thursday, October 18, 2012 11:43 AM
> > > To: user@hbase.apache.org
> > > Subject: Checking major compaction
> > >
> > > Hi all,
> > >
> > > Is there a way to check if major compaction is running or not on a
> > > table.
> > >
> > > --
> > > Thank you
> > > Kiran Sarvabhotla
> > >
> > > -----Even a correct decision is wrong when it is taken late
> >
> >
>
>
> --
> Thank you
> Kiran Sarvabhotla
>
> -----Even a correct decision is wrong when it is taken late
Re: Checking major compaction
Posted by Jean-Daniel Cryans <jd...@apache.org>.
You could just disable then enable the table.
J-D
On Fri, Oct 19, 2012 at 9:10 AM, kiran <ki...@gmail.com> wrote:
> Kevin,
>
> Thanks for your reply,
>
> can restarting region servers help to kill the already existing compaction
> processes in region servers..
>
> Thanks
> Kiran
>
> On Fri, Oct 19, 2012 at 9:16 PM, Kevin O'dell <ke...@cloudera.com>wrote:
>
>> Kiran,
>>
>> I don't believe you can. If you have a sane region count/region size it
>> should be over soon.
>>
>> On Fri, Oct 19, 2012 at 11:37 AM, kiran <ki...@gmail.com>
>> wrote:
>>
>> > Hi all,
>> >
>> > I mistakenly execution major compaction on the entire table instead of
>> > region by region. It is causing so much delay in the write performance,
>> > also on the disk and networkI/O. How can i stop major compaction now ?
>> >
>> > Thanks
>> > Kiran
>> >
>> > On Thu, Oct 18, 2012 at 2:55 PM, Ramkrishna.S.Vasudevan <
>> > ramkrishna.vasudevan@huawei.com> wrote:
>> >
>> > > >A simple test would be to just right some 10 rows
>> > > I meant to say write some 10 rows.(not right)
>> > >
>> > > Regards
>> > > Ram
>> > >
>> > > > -----Original Message-----
>> > > > From: Ramkrishna.S.Vasudevan [mailto:ramkrishna.vasudevan@huawei.com
>> ]
>> > > > Sent: Thursday, October 18, 2012 2:05 PM
>> > > > To: user@hbase.apache.org
>> > > > Subject: RE: Checking major compaction
>> > > >
>> > > > Hi
>> > > >
>> > > > Yes Kiran you can go thro the logs also.
>> > > >
>> > > > You will see some logs like
>> > > > 'Start major compaction for ..............
>> > > > 'Compacting file ................
>> > > > 'Compacting file ................
>> > > > And finally 'Completed major/minor compaction.....'
>> > > >
>> > > > I just don have some exact logs with me right now. But you can see
>> log
>> > > > msgs
>> > > > but all comes in debug mode. So ensure you enable debug mode for
>> your
>> > > > logs.
>> > > >
>> > > > A simple test would be to just right some 10 rows. In between do
>> some 4
>> > > > to 5
>> > > > flushes.
>> > > >
>> > > > Just give major_compact(tableName) from the shell. You can see the
>> > > > logs.
>> > > > :)
>> > > >
>> > > > Regards
>> > > > Ram
>> > > >
>> > > > > -----Original Message-----
>> > > > > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
>> > > > > Sent: Thursday, October 18, 2012 12:03 PM
>> > > > > To: user@hbase.apache.org
>> > > > > Subject: Re: Checking major compaction
>> > > > >
>> > > > > Thanks ram,
>> > > > >
>> > > > > Is there a way can I check it through region server logs. If it is
>> > > > > possible
>> > > > > what are the statements that I need to look for ??
>> > > > >
>> > > > > Thanks
>> > > > > Kiran
>> > > > >
>> > > > > On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
>> > > > > ramkrishna.vasudevan@huawei.com> wrote:
>> > > > >
>> > > > > > HBASE-6033 does the work that you ask for. It is currently in
>> > > > Trunk
>> > > > > > version
>> > > > > > of HBase.
>> > > > > >
>> > > > > > Regards
>> > > > > > Ram
>> > > > > >
>> > > > > > > -----Original Message-----
>> > > > > > > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
>> > > > > > > Sent: Thursday, October 18, 2012 11:43 AM
>> > > > > > > To: user@hbase.apache.org
>> > > > > > > Subject: Checking major compaction
>> > > > > > >
>> > > > > > > Hi all,
>> > > > > > >
>> > > > > > > Is there a way to check if major compaction is running or not
>> on
>> > > > a
>> > > > > > > table.
>> > > > > > >
>> > > > > > > --
>> > > > > > > Thank you
>> > > > > > > Kiran Sarvabhotla
>> > > > > > >
>> > > > > > > -----Even a correct decision is wrong when it is taken late
>> > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Thank you
>> > > > > Kiran Sarvabhotla
>> > > > >
>> > > > > -----Even a correct decision is wrong when it is taken late
>> > >
>> > >
>> >
>> >
>> > --
>> > Thank you
>> > Kiran Sarvabhotla
>> >
>> > -----Even a correct decision is wrong when it is taken late
>> >
>>
>>
>>
>> --
>> Kevin O'Dell
>> Customer Operations Engineer, Cloudera
>>
>
>
>
> --
> Thank you
> Kiran Sarvabhotla
>
> -----Even a correct decision is wrong when it is taken late
Re: Checking major compaction
Posted by kiran <ki...@gmail.com>.
Kevin,
Thanks for your reply,
can restarting region servers help to kill the already existing compaction
processes in region servers..
Thanks
Kiran
On Fri, Oct 19, 2012 at 9:16 PM, Kevin O'dell <ke...@cloudera.com>wrote:
> Kiran,
>
> I don't believe you can. If you have a sane region count/region size it
> should be over soon.
>
> On Fri, Oct 19, 2012 at 11:37 AM, kiran <ki...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > I mistakenly execution major compaction on the entire table instead of
> > region by region. It is causing so much delay in the write performance,
> > also on the disk and networkI/O. How can i stop major compaction now ?
> >
> > Thanks
> > Kiran
> >
> > On Thu, Oct 18, 2012 at 2:55 PM, Ramkrishna.S.Vasudevan <
> > ramkrishna.vasudevan@huawei.com> wrote:
> >
> > > >A simple test would be to just right some 10 rows
> > > I meant to say write some 10 rows.(not right)
> > >
> > > Regards
> > > Ram
> > >
> > > > -----Original Message-----
> > > > From: Ramkrishna.S.Vasudevan [mailto:ramkrishna.vasudevan@huawei.com
> ]
> > > > Sent: Thursday, October 18, 2012 2:05 PM
> > > > To: user@hbase.apache.org
> > > > Subject: RE: Checking major compaction
> > > >
> > > > Hi
> > > >
> > > > Yes Kiran you can go thro the logs also.
> > > >
> > > > You will see some logs like
> > > > 'Start major compaction for ..............
> > > > 'Compacting file ................
> > > > 'Compacting file ................
> > > > And finally 'Completed major/minor compaction.....'
> > > >
> > > > I just don have some exact logs with me right now. But you can see
> log
> > > > msgs
> > > > but all comes in debug mode. So ensure you enable debug mode for
> your
> > > > logs.
> > > >
> > > > A simple test would be to just right some 10 rows. In between do
> some 4
> > > > to 5
> > > > flushes.
> > > >
> > > > Just give major_compact(tableName) from the shell. You can see the
> > > > logs.
> > > > :)
> > > >
> > > > Regards
> > > > Ram
> > > >
> > > > > -----Original Message-----
> > > > > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> > > > > Sent: Thursday, October 18, 2012 12:03 PM
> > > > > To: user@hbase.apache.org
> > > > > Subject: Re: Checking major compaction
> > > > >
> > > > > Thanks ram,
> > > > >
> > > > > Is there a way can I check it through region server logs. If it is
> > > > > possible
> > > > > what are the statements that I need to look for ??
> > > > >
> > > > > Thanks
> > > > > Kiran
> > > > >
> > > > > On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
> > > > > ramkrishna.vasudevan@huawei.com> wrote:
> > > > >
> > > > > > HBASE-6033 does the work that you ask for. It is currently in
> > > > Trunk
> > > > > > version
> > > > > > of HBase.
> > > > > >
> > > > > > Regards
> > > > > > Ram
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> > > > > > > Sent: Thursday, October 18, 2012 11:43 AM
> > > > > > > To: user@hbase.apache.org
> > > > > > > Subject: Checking major compaction
> > > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > Is there a way to check if major compaction is running or not
> on
> > > > a
> > > > > > > table.
> > > > > > >
> > > > > > > --
> > > > > > > Thank you
> > > > > > > Kiran Sarvabhotla
> > > > > > >
> > > > > > > -----Even a correct decision is wrong when it is taken late
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thank you
> > > > > Kiran Sarvabhotla
> > > > >
> > > > > -----Even a correct decision is wrong when it is taken late
> > >
> > >
> >
> >
> > --
> > Thank you
> > Kiran Sarvabhotla
> >
> > -----Even a correct decision is wrong when it is taken late
> >
>
>
>
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera
>
--
Thank you
Kiran Sarvabhotla
-----Even a correct decision is wrong when it is taken late
Re: Checking major compaction
Posted by Kevin O'dell <ke...@cloudera.com>.
Kiran,
I don't believe you can. If you have a sane region count/region size it
should be over soon.
On Fri, Oct 19, 2012 at 11:37 AM, kiran <ki...@gmail.com> wrote:
> Hi all,
>
> I mistakenly execution major compaction on the entire table instead of
> region by region. It is causing so much delay in the write performance,
> also on the disk and networkI/O. How can i stop major compaction now ?
>
> Thanks
> Kiran
>
> On Thu, Oct 18, 2012 at 2:55 PM, Ramkrishna.S.Vasudevan <
> ramkrishna.vasudevan@huawei.com> wrote:
>
> > >A simple test would be to just right some 10 rows
> > I meant to say write some 10 rows.(not right)
> >
> > Regards
> > Ram
> >
> > > -----Original Message-----
> > > From: Ramkrishna.S.Vasudevan [mailto:ramkrishna.vasudevan@huawei.com]
> > > Sent: Thursday, October 18, 2012 2:05 PM
> > > To: user@hbase.apache.org
> > > Subject: RE: Checking major compaction
> > >
> > > Hi
> > >
> > > Yes Kiran you can go thro the logs also.
> > >
> > > You will see some logs like
> > > 'Start major compaction for ..............
> > > 'Compacting file ................
> > > 'Compacting file ................
> > > And finally 'Completed major/minor compaction.....'
> > >
> > > I just don have some exact logs with me right now. But you can see log
> > > msgs
> > > but all comes in debug mode. So ensure you enable debug mode for your
> > > logs.
> > >
> > > A simple test would be to just right some 10 rows. In between do some 4
> > > to 5
> > > flushes.
> > >
> > > Just give major_compact(tableName) from the shell. You can see the
> > > logs.
> > > :)
> > >
> > > Regards
> > > Ram
> > >
> > > > -----Original Message-----
> > > > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> > > > Sent: Thursday, October 18, 2012 12:03 PM
> > > > To: user@hbase.apache.org
> > > > Subject: Re: Checking major compaction
> > > >
> > > > Thanks ram,
> > > >
> > > > Is there a way can I check it through region server logs. If it is
> > > > possible
> > > > what are the statements that I need to look for ??
> > > >
> > > > Thanks
> > > > Kiran
> > > >
> > > > On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
> > > > ramkrishna.vasudevan@huawei.com> wrote:
> > > >
> > > > > HBASE-6033 does the work that you ask for. It is currently in
> > > Trunk
> > > > > version
> > > > > of HBase.
> > > > >
> > > > > Regards
> > > > > Ram
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> > > > > > Sent: Thursday, October 18, 2012 11:43 AM
> > > > > > To: user@hbase.apache.org
> > > > > > Subject: Checking major compaction
> > > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Is there a way to check if major compaction is running or not on
> > > a
> > > > > > table.
> > > > > >
> > > > > > --
> > > > > > Thank you
> > > > > > Kiran Sarvabhotla
> > > > > >
> > > > > > -----Even a correct decision is wrong when it is taken late
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Thank you
> > > > Kiran Sarvabhotla
> > > >
> > > > -----Even a correct decision is wrong when it is taken late
> >
> >
>
>
> --
> Thank you
> Kiran Sarvabhotla
>
> -----Even a correct decision is wrong when it is taken late
>
--
Kevin O'Dell
Customer Operations Engineer, Cloudera
Re: Checking major compaction
Posted by kiran <ki...@gmail.com>.
Hi all,
I mistakenly execution major compaction on the entire table instead of
region by region. It is causing so much delay in the write performance,
also on the disk and networkI/O. How can i stop major compaction now ?
Thanks
Kiran
On Thu, Oct 18, 2012 at 2:55 PM, Ramkrishna.S.Vasudevan <
ramkrishna.vasudevan@huawei.com> wrote:
> >A simple test would be to just right some 10 rows
> I meant to say write some 10 rows.(not right)
>
> Regards
> Ram
>
> > -----Original Message-----
> > From: Ramkrishna.S.Vasudevan [mailto:ramkrishna.vasudevan@huawei.com]
> > Sent: Thursday, October 18, 2012 2:05 PM
> > To: user@hbase.apache.org
> > Subject: RE: Checking major compaction
> >
> > Hi
> >
> > Yes Kiran you can go thro the logs also.
> >
> > You will see some logs like
> > 'Start major compaction for ..............
> > 'Compacting file ................
> > 'Compacting file ................
> > And finally 'Completed major/minor compaction.....'
> >
> > I just don have some exact logs with me right now. But you can see log
> > msgs
> > but all comes in debug mode. So ensure you enable debug mode for your
> > logs.
> >
> > A simple test would be to just right some 10 rows. In between do some 4
> > to 5
> > flushes.
> >
> > Just give major_compact(tableName) from the shell. You can see the
> > logs.
> > :)
> >
> > Regards
> > Ram
> >
> > > -----Original Message-----
> > > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> > > Sent: Thursday, October 18, 2012 12:03 PM
> > > To: user@hbase.apache.org
> > > Subject: Re: Checking major compaction
> > >
> > > Thanks ram,
> > >
> > > Is there a way can I check it through region server logs. If it is
> > > possible
> > > what are the statements that I need to look for ??
> > >
> > > Thanks
> > > Kiran
> > >
> > > On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
> > > ramkrishna.vasudevan@huawei.com> wrote:
> > >
> > > > HBASE-6033 does the work that you ask for. It is currently in
> > Trunk
> > > > version
> > > > of HBase.
> > > >
> > > > Regards
> > > > Ram
> > > >
> > > > > -----Original Message-----
> > > > > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> > > > > Sent: Thursday, October 18, 2012 11:43 AM
> > > > > To: user@hbase.apache.org
> > > > > Subject: Checking major compaction
> > > > >
> > > > > Hi all,
> > > > >
> > > > > Is there a way to check if major compaction is running or not on
> > a
> > > > > table.
> > > > >
> > > > > --
> > > > > Thank you
> > > > > Kiran Sarvabhotla
> > > > >
> > > > > -----Even a correct decision is wrong when it is taken late
> > > >
> > > >
> > >
> > >
> > > --
> > > Thank you
> > > Kiran Sarvabhotla
> > >
> > > -----Even a correct decision is wrong when it is taken late
>
>
--
Thank you
Kiran Sarvabhotla
-----Even a correct decision is wrong when it is taken late
Re: Checking major compaction
Posted by kiran <ki...@gmail.com>.
Thanks ram,
Is there a way can I check it through region server logs. If it is possible
what are the statements that I need to look for ??
Thanks
Kiran
On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
ramkrishna.vasudevan@huawei.com> wrote:
> HBASE-6033 does the work that you ask for. It is currently in Trunk
> version
> of HBase.
>
> Regards
> Ram
>
> > -----Original Message-----
> > From: kiran [mailto:kiran.sarvabhotla@gmail.com]
> > Sent: Thursday, October 18, 2012 11:43 AM
> > To: user@hbase.apache.org
> > Subject: Checking major compaction
> >
> > Hi all,
> >
> > Is there a way to check if major compaction is running or not on a
> > table.
> >
> > --
> > Thank you
> > Kiran Sarvabhotla
> >
> > -----Even a correct decision is wrong when it is taken late
>
>
--
Thank you
Kiran Sarvabhotla
-----Even a correct decision is wrong when it is taken late