You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Mathias Herberts (JIRA)" <ji...@apache.org> on 2009/08/20 15:29:15 UTC
[jira] Created: (HBASE-1780) HTable.flushCommits clears write
buffer in finally clause
HTable.flushCommits clears write buffer in finally clause
---------------------------------------------------------
Key: HBASE-1780
URL: https://issues.apache.org/jira/browse/HBASE-1780
Project: Hadoop HBase
Issue Type: Bug
Components: client
Environment: All
Reporter: Mathias Herberts
Fix For: 0.20.1, 0.21.0
Metthod flushCommits clears the write buffer in a finally clause.
When using the write buffer, if the call to processBatchOfRows done in flushCommits throws an exception, the write buffer will be cleared thus potentially leading to loss of data on the client side.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1780) HTable.flushCommits clears write
buffer in finally clause
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell resolved HBASE-1780.
-----------------------------------
Resolution: Fixed
Fix Version/s: 0.21.0
Assignee: Andrew Purtell
Hadoop Flags: [Reviewed]
Committed to branch and trunk with synchronization bits removed and a comment that HTable is not MT safe for writes.
> HTable.flushCommits clears write buffer in finally clause
> ---------------------------------------------------------
>
> Key: HBASE-1780
> URL: https://issues.apache.org/jira/browse/HBASE-1780
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client
> Environment: All
> Reporter: Mathias Herberts
> Assignee: Andrew Purtell
> Fix For: 0.20.0, 0.21.0
>
> Attachments: HBASE-1780.patch, HBASE-1780.patch
>
>
> Metthod flushCommits clears the write buffer in a finally clause.
> When using the write buffer, if the call to processBatchOfRows done in flushCommits throws an exception, the write buffer will be cleared thus potentially leading to loss of data on the client side.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1780) HTable.flushCommits clears write
buffer in finally clause
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1780:
----------------------------------
Attachment: HBASE-1780.patch
Good point Mathias. Latest patch also synchronizes writeBuffer.
> HTable.flushCommits clears write buffer in finally clause
> ---------------------------------------------------------
>
> Key: HBASE-1780
> URL: https://issues.apache.org/jira/browse/HBASE-1780
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client
> Environment: All
> Reporter: Mathias Herberts
> Fix For: 0.20.1, 0.21.0
>
> Attachments: HBASE-1780.patch, HBASE-1780.patch
>
>
> Metthod flushCommits clears the write buffer in a finally clause.
> When using the write buffer, if the call to processBatchOfRows done in flushCommits throws an exception, the write buffer will be cleared thus potentially leading to loss of data on the client side.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1780) HTable.flushCommits clears write
buffer in finally clause
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1780:
----------------------------------
Attachment: HBASE-1780.patch
Testing patch now.
> HTable.flushCommits clears write buffer in finally clause
> ---------------------------------------------------------
>
> Key: HBASE-1780
> URL: https://issues.apache.org/jira/browse/HBASE-1780
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client
> Environment: All
> Reporter: Mathias Herberts
> Fix For: 0.20.1, 0.21.0
>
> Attachments: HBASE-1780.patch
>
>
> Metthod flushCommits clears the write buffer in a finally clause.
> When using the write buffer, if the call to processBatchOfRows done in flushCommits throws an exception, the write buffer will be cleared thus potentially leading to loss of data on the client side.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1780) HTable.flushCommits clears write
buffer in finally clause
Posted by "Mathias Herberts (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12748318#action_12748318 ]
Mathias Herberts commented on HBASE-1780:
-----------------------------------------
I think we should specify somewhere that HTable instances are not safe to use in MT environments for writes.
The manipulation of writebuffer could could lead to ConcurrentModificationException being raised if Puts are added to it while the new write buffer size is computed in flushCommits.
> HTable.flushCommits clears write buffer in finally clause
> ---------------------------------------------------------
>
> Key: HBASE-1780
> URL: https://issues.apache.org/jira/browse/HBASE-1780
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client
> Environment: All
> Reporter: Mathias Herberts
> Fix For: 0.20.1, 0.21.0
>
> Attachments: HBASE-1780.patch
>
>
> Metthod flushCommits clears the write buffer in a finally clause.
> When using the write buffer, if the call to processBatchOfRows done in flushCommits throws an exception, the write buffer will be cleared thus potentially leading to loss of data on the client side.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1780) HTable.flushCommits clears write
buffer in finally clause
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1780:
----------------------------------
Attachment: HBASE-1780.patch
> HTable.flushCommits clears write buffer in finally clause
> ---------------------------------------------------------
>
> Key: HBASE-1780
> URL: https://issues.apache.org/jira/browse/HBASE-1780
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client
> Environment: All
> Reporter: Mathias Herberts
> Fix For: 0.20.1, 0.21.0
>
> Attachments: HBASE-1780.patch
>
>
> Metthod flushCommits clears the write buffer in a finally clause.
> When using the write buffer, if the call to processBatchOfRows done in flushCommits throws an exception, the write buffer will be cleared thus potentially leading to loss of data on the client side.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1780) HTable.flushCommits clears write
buffer in finally clause
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12748404#action_12748404 ]
stack commented on HBASE-1780:
------------------------------
Do the put methods need to be synchronized now since internally its synchronizing on the write buffer? (Make currentWriteBufferSize transient so outside of the synch block its most recent value can be seen?)
Minor: Make the below final and do assigning at same time as declaration?
{code}
private ArrayList<Put> writeBuffer;
{code}
Otherwise patch looks great. Patch is important fix.
Now you've done the work, add a comment to HTable class javadoc to explicitly say its thread-safe?
(There are a bunch of unused imports in my HTable. Are they in yours? Remove as part of this patch? They look bad, especially in this most public of our classes).
> HTable.flushCommits clears write buffer in finally clause
> ---------------------------------------------------------
>
> Key: HBASE-1780
> URL: https://issues.apache.org/jira/browse/HBASE-1780
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client
> Environment: All
> Reporter: Mathias Herberts
> Fix For: 0.20.1, 0.21.0
>
> Attachments: HBASE-1780.patch, HBASE-1780.patch
>
>
> Metthod flushCommits clears the write buffer in a finally clause.
> When using the write buffer, if the call to processBatchOfRows done in flushCommits throws an exception, the write buffer will be cleared thus potentially leading to loss of data on the client side.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1780) HTable.flushCommits clears write
buffer in finally clause
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12748442#action_12748442 ]
stack commented on HBASE-1780:
------------------------------
Ryan makes the valid point offline that HTable should NOT be shared by threads. If its all synchronized on the write, then no performance gain sharing the HTable instance. That'd mean, remove all synchronization, flag HTable as not thread-safe and just add the bit where writes are not lost on exception.
> HTable.flushCommits clears write buffer in finally clause
> ---------------------------------------------------------
>
> Key: HBASE-1780
> URL: https://issues.apache.org/jira/browse/HBASE-1780
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client
> Environment: All
> Reporter: Mathias Herberts
> Fix For: 0.20.0
>
> Attachments: HBASE-1780.patch, HBASE-1780.patch
>
>
> Metthod flushCommits clears the write buffer in a finally clause.
> When using the write buffer, if the call to processBatchOfRows done in flushCommits throws an exception, the write buffer will be cleared thus potentially leading to loss of data on the client side.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1780) HTable.flushCommits clears write
buffer in finally clause
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12748432#action_12748432 ]
Andrew Purtell commented on HBASE-1780:
---------------------------------------
@stack:
bq. There are a bunch of unused imports in my HTable. Are they in yours? Remove as part of this patch?
If you're looking at Eclipse warnings, those are due to deprecated includes necessary for old client API, not unused imports.
I'll fold your suggestions into the commit. Thanks for the review.
> HTable.flushCommits clears write buffer in finally clause
> ---------------------------------------------------------
>
> Key: HBASE-1780
> URL: https://issues.apache.org/jira/browse/HBASE-1780
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client
> Environment: All
> Reporter: Mathias Herberts
> Fix For: 0.20.1, 0.21.0
>
> Attachments: HBASE-1780.patch, HBASE-1780.patch
>
>
> Metthod flushCommits clears the write buffer in a finally clause.
> When using the write buffer, if the call to processBatchOfRows done in flushCommits throws an exception, the write buffer will be cleared thus potentially leading to loss of data on the client side.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1780) HTable.flushCommits clears write
buffer in finally clause
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-1780:
----------------------------------
Attachment: (was: HBASE-1780.patch)
> HTable.flushCommits clears write buffer in finally clause
> ---------------------------------------------------------
>
> Key: HBASE-1780
> URL: https://issues.apache.org/jira/browse/HBASE-1780
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client
> Environment: All
> Reporter: Mathias Herberts
> Fix For: 0.20.1, 0.21.0
>
> Attachments: HBASE-1780.patch
>
>
> Metthod flushCommits clears the write buffer in a finally clause.
> When using the write buffer, if the call to processBatchOfRows done in flushCommits throws an exception, the write buffer will be cleared thus potentially leading to loss of data on the client side.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1780) HTable.flushCommits clears write
buffer in finally clause
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1780:
-------------------------
Fix Version/s: (was: 0.21.0)
(was: 0.20.1)
0.20.0
Bringing into 0.20.0
@Andrew You are right. I'm looking at them on different computer. Usually deprecated differs in that its strikethrough. Not so in this case.
> HTable.flushCommits clears write buffer in finally clause
> ---------------------------------------------------------
>
> Key: HBASE-1780
> URL: https://issues.apache.org/jira/browse/HBASE-1780
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client
> Environment: All
> Reporter: Mathias Herberts
> Fix For: 0.20.0
>
> Attachments: HBASE-1780.patch, HBASE-1780.patch
>
>
> Metthod flushCommits clears the write buffer in a finally clause.
> When using the write buffer, if the call to processBatchOfRows done in flushCommits throws an exception, the write buffer will be cleared thus potentially leading to loss of data on the client side.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.