You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Natasha d'silva <na...@gmail.com> on 2016/05/16 17:30:40 UTC
Hbase v0.98 - does disabling autoflush affect Deletes or just Puts?
Hi, I would like some clarification on whether or not disabling autoflush
affects Put and Delete operations or just Puts.
The setAutoFlush doc states that it only affects Puts, but elsewhere in the
documentation for HTable, Puts and Writes are mentioned together: "In case
of writes (Put, Delete), the underlying write buffer can be corrupted if
multiple threads contend over a single HTable instance."
I would appreciate it if someone could please clarify.
Thanks!
--
-Natasha D'Silva
Re: Hbase v0.98 - does disabling autoflush affect Deletes or just Puts?
Posted by Stack <st...@duboce.net>.
I pushed HBASE-15846
St.Ack
On Tue, May 17, 2016 at 10:28 AM, Stack <st...@duboce.net> wrote:
> On Tue, May 17, 2016 at 7:18 AM, Natasha d'silva <na...@gmail.com>
> wrote:
>
>> Okay, good to know.
>>
>>
>> One last clarification: everytime I call:
>> HTableInterface table = connection.getTable(tableNameBytes);
>> *Must* there always be a call to table.close()?
>> I would like to avoid caching a reference to an HTableInterface object,
>> but
>> to do that means that I have to call getTable every time I need to
>> put/delete etc.
>> If, in turn, I must call table.close() every time, then it defeats the
>> purpose of disabling autoflush since each close() will trigger a call to
>> flushCommits().
>> So can I call connection.getTable() without closing the returned table, if
>> I ensure that I will call table.close() when I need to?
>>
>>
> Probably best if you read the code if you intend to take an unorthodox
> route. See what happens when you do getTable... see resources allocated and
> then how they'll allocate a buffer for the HTable instance. Calling close
> ensures all resources are returned and flushed safely. You need to ensure
> that whatever your strategy, the result must end up in the same cleanup.
>
>
>
>>
>> Lastly: this is what I was referring to when I mentioned that Puts and
>> Delets are mentioned together as Writes in the 0.98 doc:
>>
>> https://github.com/apache/hbase/blob/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java#L91
>>
>>
> Thanks. Let me fix.
>
> St.Ack
>
>
>
>> On Mon, May 16, 2016 at 8:24 PM, Stack <st...@duboce.net> wrote:
>>
>> > On Mon, May 16, 2016 at 5:20 PM, Natasha d'silva <na...@gmail.com>
>> > wrote:
>> >
>> > > Thanks for the quick response. What about check and put? The check is
>> > > atomic but is the put buffered?
>> > >
>> >
>> > No. checkAnd* are operations unto themselves that are sent to the
>> server to
>> > run; only it has the means of achieving a checkAnd* that has integrity.
>> > St.Ack
>> >
>> >
>> >
>> > > On May 16, 2016 6:30 PM, "Stack" <st...@duboce.net> wrote:
>> > >
>> > > > In 0.98, only Puts are buffered.
>> > > >
>> > > > I did a search of the above text but didn't find it in 0.98... I'm
>> > > > searching wrong? Mind fling an issue so we fix the above
>> > > misdocumentation?
>> > > >
>> > > > Thank you,
>> > > > S
>> > > >
>> > > >
>> > > >
>> > > > On Mon, May 16, 2016 at 10:30 AM, Natasha d'silva <
>> natashads@gmail.com
>> > >
>> > > > wrote:
>> > > >
>> > > > > Hi, I would like some clarification on whether or not disabling
>> > > autoflush
>> > > > > affects Put and Delete operations or just Puts.
>> > > > > The setAutoFlush doc states that it only affects Puts, but
>> elsewhere
>> > in
>> > > > the
>> > > > > documentation for HTable, Puts and Writes are mentioned together:
>> "In
>> > > > case
>> > > > > of writes (Put, Delete), the underlying write buffer can be
>> corrupted
>> > > if
>> > > > > multiple threads contend over a single HTable instance."
>> > > > > I would appreciate it if someone could please clarify.
>> > > > >
>> > > > > Thanks!
>> > > > >
>> > > > > --
>> > > > > -Natasha D'Silva
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> -Natasha D'Silva
>>
>
>
Re: Hbase v0.98 - does disabling autoflush affect Deletes or just Puts?
Posted by Stack <st...@duboce.net>.
On Tue, May 17, 2016 at 7:18 AM, Natasha d'silva <na...@gmail.com>
wrote:
> Okay, good to know.
>
>
> One last clarification: everytime I call:
> HTableInterface table = connection.getTable(tableNameBytes);
> *Must* there always be a call to table.close()?
> I would like to avoid caching a reference to an HTableInterface object, but
> to do that means that I have to call getTable every time I need to
> put/delete etc.
> If, in turn, I must call table.close() every time, then it defeats the
> purpose of disabling autoflush since each close() will trigger a call to
> flushCommits().
> So can I call connection.getTable() without closing the returned table, if
> I ensure that I will call table.close() when I need to?
>
>
Probably best if you read the code if you intend to take an unorthodox
route. See what happens when you do getTable... see resources allocated and
then how they'll allocate a buffer for the HTable instance. Calling close
ensures all resources are returned and flushed safely. You need to ensure
that whatever your strategy, the result must end up in the same cleanup.
>
> Lastly: this is what I was referring to when I mentioned that Puts and
> Delets are mentioned together as Writes in the 0.98 doc:
>
> https://github.com/apache/hbase/blob/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java#L91
>
>
Thanks. Let me fix.
St.Ack
> On Mon, May 16, 2016 at 8:24 PM, Stack <st...@duboce.net> wrote:
>
> > On Mon, May 16, 2016 at 5:20 PM, Natasha d'silva <na...@gmail.com>
> > wrote:
> >
> > > Thanks for the quick response. What about check and put? The check is
> > > atomic but is the put buffered?
> > >
> >
> > No. checkAnd* are operations unto themselves that are sent to the server
> to
> > run; only it has the means of achieving a checkAnd* that has integrity.
> > St.Ack
> >
> >
> >
> > > On May 16, 2016 6:30 PM, "Stack" <st...@duboce.net> wrote:
> > >
> > > > In 0.98, only Puts are buffered.
> > > >
> > > > I did a search of the above text but didn't find it in 0.98... I'm
> > > > searching wrong? Mind fling an issue so we fix the above
> > > misdocumentation?
> > > >
> > > > Thank you,
> > > > S
> > > >
> > > >
> > > >
> > > > On Mon, May 16, 2016 at 10:30 AM, Natasha d'silva <
> natashads@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hi, I would like some clarification on whether or not disabling
> > > autoflush
> > > > > affects Put and Delete operations or just Puts.
> > > > > The setAutoFlush doc states that it only affects Puts, but
> elsewhere
> > in
> > > > the
> > > > > documentation for HTable, Puts and Writes are mentioned together:
> "In
> > > > case
> > > > > of writes (Put, Delete), the underlying write buffer can be
> corrupted
> > > if
> > > > > multiple threads contend over a single HTable instance."
> > > > > I would appreciate it if someone could please clarify.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > --
> > > > > -Natasha D'Silva
> > > > >
> > > >
> > >
> >
>
>
>
> --
> -Natasha D'Silva
>
Re: Hbase v0.98 - does disabling autoflush affect Deletes or just Puts?
Posted by Natasha d'silva <na...@gmail.com>.
Okay, good to know.
One last clarification: everytime I call:
HTableInterface table = connection.getTable(tableNameBytes);
*Must* there always be a call to table.close()?
I would like to avoid caching a reference to an HTableInterface object, but
to do that means that I have to call getTable every time I need to
put/delete etc.
If, in turn, I must call table.close() every time, then it defeats the
purpose of disabling autoflush since each close() will trigger a call to
flushCommits().
So can I call connection.getTable() without closing the returned table, if
I ensure that I will call table.close() when I need to?
Lastly: this is what I was referring to when I mentioned that Puts and
Delets are mentioned together as Writes in the 0.98 doc:
https://github.com/apache/hbase/blob/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java#L91
On Mon, May 16, 2016 at 8:24 PM, Stack <st...@duboce.net> wrote:
> On Mon, May 16, 2016 at 5:20 PM, Natasha d'silva <na...@gmail.com>
> wrote:
>
> > Thanks for the quick response. What about check and put? The check is
> > atomic but is the put buffered?
> >
>
> No. checkAnd* are operations unto themselves that are sent to the server to
> run; only it has the means of achieving a checkAnd* that has integrity.
> St.Ack
>
>
>
> > On May 16, 2016 6:30 PM, "Stack" <st...@duboce.net> wrote:
> >
> > > In 0.98, only Puts are buffered.
> > >
> > > I did a search of the above text but didn't find it in 0.98... I'm
> > > searching wrong? Mind fling an issue so we fix the above
> > misdocumentation?
> > >
> > > Thank you,
> > > S
> > >
> > >
> > >
> > > On Mon, May 16, 2016 at 10:30 AM, Natasha d'silva <natashads@gmail.com
> >
> > > wrote:
> > >
> > > > Hi, I would like some clarification on whether or not disabling
> > autoflush
> > > > affects Put and Delete operations or just Puts.
> > > > The setAutoFlush doc states that it only affects Puts, but elsewhere
> in
> > > the
> > > > documentation for HTable, Puts and Writes are mentioned together: "In
> > > case
> > > > of writes (Put, Delete), the underlying write buffer can be corrupted
> > if
> > > > multiple threads contend over a single HTable instance."
> > > > I would appreciate it if someone could please clarify.
> > > >
> > > > Thanks!
> > > >
> > > > --
> > > > -Natasha D'Silva
> > > >
> > >
> >
>
--
-Natasha D'Silva
Re: Hbase v0.98 - does disabling autoflush affect Deletes or just Puts?
Posted by Stack <st...@duboce.net>.
On Mon, May 16, 2016 at 5:20 PM, Natasha d'silva <na...@gmail.com>
wrote:
> Thanks for the quick response. What about check and put? The check is
> atomic but is the put buffered?
>
No. checkAnd* are operations unto themselves that are sent to the server to
run; only it has the means of achieving a checkAnd* that has integrity.
St.Ack
> On May 16, 2016 6:30 PM, "Stack" <st...@duboce.net> wrote:
>
> > In 0.98, only Puts are buffered.
> >
> > I did a search of the above text but didn't find it in 0.98... I'm
> > searching wrong? Mind fling an issue so we fix the above
> misdocumentation?
> >
> > Thank you,
> > S
> >
> >
> >
> > On Mon, May 16, 2016 at 10:30 AM, Natasha d'silva <na...@gmail.com>
> > wrote:
> >
> > > Hi, I would like some clarification on whether or not disabling
> autoflush
> > > affects Put and Delete operations or just Puts.
> > > The setAutoFlush doc states that it only affects Puts, but elsewhere in
> > the
> > > documentation for HTable, Puts and Writes are mentioned together: "In
> > case
> > > of writes (Put, Delete), the underlying write buffer can be corrupted
> if
> > > multiple threads contend over a single HTable instance."
> > > I would appreciate it if someone could please clarify.
> > >
> > > Thanks!
> > >
> > > --
> > > -Natasha D'Silva
> > >
> >
>
Re: Hbase v0.98 - does disabling autoflush affect Deletes or just Puts?
Posted by Natasha d'silva <na...@gmail.com>.
Thanks for the quick response. What about check and put? The check is
atomic but is the put buffered?
On May 16, 2016 6:30 PM, "Stack" <st...@duboce.net> wrote:
> In 0.98, only Puts are buffered.
>
> I did a search of the above text but didn't find it in 0.98... I'm
> searching wrong? Mind fling an issue so we fix the above misdocumentation?
>
> Thank you,
> S
>
>
>
> On Mon, May 16, 2016 at 10:30 AM, Natasha d'silva <na...@gmail.com>
> wrote:
>
> > Hi, I would like some clarification on whether or not disabling autoflush
> > affects Put and Delete operations or just Puts.
> > The setAutoFlush doc states that it only affects Puts, but elsewhere in
> the
> > documentation for HTable, Puts and Writes are mentioned together: "In
> case
> > of writes (Put, Delete), the underlying write buffer can be corrupted if
> > multiple threads contend over a single HTable instance."
> > I would appreciate it if someone could please clarify.
> >
> > Thanks!
> >
> > --
> > -Natasha D'Silva
> >
>
Re: Hbase v0.98 - does disabling autoflush affect Deletes or just Puts?
Posted by Stack <st...@duboce.net>.
In 0.98, only Puts are buffered.
I did a search of the above text but didn't find it in 0.98... I'm
searching wrong? Mind fling an issue so we fix the above misdocumentation?
Thank you,
S
On Mon, May 16, 2016 at 10:30 AM, Natasha d'silva <na...@gmail.com>
wrote:
> Hi, I would like some clarification on whether or not disabling autoflush
> affects Put and Delete operations or just Puts.
> The setAutoFlush doc states that it only affects Puts, but elsewhere in the
> documentation for HTable, Puts and Writes are mentioned together: "In case
> of writes (Put, Delete), the underlying write buffer can be corrupted if
> multiple threads contend over a single HTable instance."
> I would appreciate it if someone could please clarify.
>
> Thanks!
>
> --
> -Natasha D'Silva
>