You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Jonathan Gray (JIRA)" <ji...@apache.org> on 2008/08/06 06:30:44 UTC
[jira] Created: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Provide Client API to explicitly lock and unlock rows
-----------------------------------------------------
Key: HBASE-798
URL: https://issues.apache.org/jira/browse/HBASE-798
Project: Hadoop HBase
Issue Type: New Feature
Components: client, ipc, regionserver
Reporter: Jonathan Gray
Priority: Minor
We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-798:
--------------------------------
Attachment: hbase-798-v5.patch
Contains changes outlined by Jim. Also updates scanners in HRegion to use ConcurrentHashMap rather than synchronizedMap(HashMap).
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch, hbase-798-v3.patch, hbase-798-v4.patch, hbase-798-v5.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-798:
--------------------------------
Attachment: hbase-798-v1.patch
Here's my first go.
This adds lockRow and unlockRow methods to the client/HTable. The return type of lockRow is a new client object RowLock (contains long lockid and byte[] row). unlockRow takes a RowLock as argument.
Also adds new versions of commit, deleteAll, and deleteFamily that also can take RowLock's.
Within HRS I created new versions of the same functions which take the actual lock ids: long in HRS (randomized id as with scannerids, used for leases, etc), Integer in HR (existing type used for row locks). Existing functions, which do not explicitly take locks, now call the same HR functions but with nulls as row locks.
Internally HR checks if it was passed a valid row lock or not. If it's a null, it obtains a row lock and releases it at the end of the call. If it's valid, it makes use of the lock, and does nothing at the end. Otherwise an exception is thrown.
Code has not been completely cleaned up, but this demonstrates how the client row locks are being implemented.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Attachments: hbase-798-v1.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-798:
--------------------------------
Attachment: hbase-798-v2.patch
This patch should properly apply to trunk.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman updated HBASE-798:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
+1
Committed to trunk. Thanks for the patch Jonathan!
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch, hbase-798-v3.patch, hbase-798-v4.patch, hbase-798-v5.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620797#action_12620797 ]
Jonathan Gray commented on HBASE-798:
-------------------------------------
I'm going to work on another patch now which adds a kind of "upsert/increment" functionality. I have a few particular and frequent use cases in mind that others may have a need for. Rough APIs below...
incrementColumnValue(table,row,family:column) -> Reads last value at column (must be family:column), increments it, and writes new column version with incremented value (wrapped in a single row lock for atomicity)
columnWithIncrementedValue(table,row,family,newcolumn) -> This may be more limited to others, but we have many ordered column families where every column value inserted is lexicographically smaller than all previous (always inserted to the top). The value within each column is an incremented value. This grabs the value of the top column in the family, increments it, and uses it as the value for the newcolumn to be inserted at the top. There will be no checking for newcolumn being smaller (lexicographically) than the last one, this is the responsibility of the client, but this is assumed for correctness.
incrementColumn(table,row,family) -> This is also specialized. We assume a schema where a family has only one column. We read that column value, increment it, add the new column (reusing the old columns value), delete the old column, release the lock, return the new incremented column to the client. This is a way to provide a check-out mechanism for incrementing identifiers.
I'm going to build this patch on top of the previous patch because some helper functions and classes will be useful.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Attachments: hbase-798-v1.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-798:
--------------------------------
Status: Patch Available (was: Open)
Ready for review
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch, hbase-798-v3.patch, hbase-798-v4.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HBASE-798) Provide Client API to
explicitly lock and unlock rows
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620811#action_12620811 ]
jimk edited comment on HBASE-798 at 8/7/08 7:13 PM:
-------------------------------------------------------------
<removing extra text from original comment>
> Jonathan Gray - 07/Aug/08 05:41 PM
> I'm going to work on another patch now which adds a kind of "upsert/increment" functionality. I have a few
> particular and frequent use cases in mind that others may have a need for. Rough APIs below...
> incrementColumnValue(table,row,family:column) -> Reads last value at column (must be family:column),
<snip>
Shouldn't this work be targeted at HBASE-493 ?
Or link this patch to HBASE-493, saying that the patch for this issue is included in the patch for this issue.
was (Author: jimk):
Jonathan Gray - 07/Aug/08 05:41 PM
I'm going to work on another patch now which adds a kind of "upsert/increment" functionality. I have a few particular and frequent use cases in mind that others may have a need for. Rough APIs below...
incrementColumnValue(table,row,family:column) -> Reads last value at column (must be family:column), increments it, and writes new column version with incremented value (wrapped in a single row lock for atomicity)
columnWithIncrementedValue(table,row,family,newcolumn) -> This may be more limited to others, but we have many ordered column families where every column value inserted is lexicographically smaller than all previous (always inserted to the top). The value within each column is an incremented value. This grabs the value of the top column in the family, increments it, and uses it as the value for the newcolumn to be inserted at the top. There will be no checking for newcolumn being smaller (lexicographically) than the last one, this is the responsibility of the client, but this is assumed for correctness.
incrementColumn(table,row,family) -> This is also specialized. We assume a schema where a family has only one column. We read that column value, increment it, add the new column (reusing the old columns value), delete the old column, release the lock, return the new incremented column to the client. This is a way to provide a check-out mechanism for incrementing identifiers.
I'm going to build this patch on top of the previous patch because some helper functions and classes will be useful.
[ Show » ]
> Jonathan Gray - 07/Aug/08 05:41 PM
> I'm going to work on another patch now which adds a kind of "upsert/increment" functionality. I have a few
> particular and frequent use cases in mind that others may have a need for. Rough APIs below...
> incrementColumnValue(table,row,family:column) -> Reads last value at column (must be family:column),
<snip>
Shouldn't this work be targeted at HBASE-493 ?
Or link this patch to HBASE-493, saying that the patch for this issue is included in the patch for this issue.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Attachments: hbase-798-v1.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray reassigned HBASE-798:
-----------------------------------
Assignee: Jonathan Gray
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch, hbase-798-v3.patch, hbase-798-v4.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-798:
--------------------------------
Attachment: hbase-798-v6.patch
Adds RowLock.java and UnknownRowLockException.java
My apologies :)
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch, hbase-798-v3.patch, hbase-798-v4.patch, hbase-798-v5.patch, hbase-798-v6.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman updated HBASE-798:
--------------------------------
Fix Version/s: 0.3.0
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-798:
--------------------------------
Attachment: hbase-798-v3.patch
Applies to latest TRUNK. Were mismatches due to the HBASE-805 patch I submitted earlier.
This cleans up the code a bit and adds getRow/getFull methods with optional locks.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch, hbase-798-v3.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620813#action_12620813 ]
Jim Kellerman commented on HBASE-798:
-------------------------------------
On HRS side, I would not create new methods, but add RowLock as an additional parameter. Pass null from the client if no row lock was supplied, or something like that. Basically HRS methods usually expect all parameters and then deals with those that are not supplied.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Attachments: hbase-798-v1.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620431#action_12620431 ]
stack commented on HBASE-798:
-----------------------------
Suggestions: Add a subclass of HTable named something like LockingHTable and add your lock versions to this subclass.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman reopened HBASE-798:
---------------------------------
Neither RowLock nor UnknownRowLockException was included in the patch. This is causing compilation errors. Please supply ASAP!
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch, hbase-798-v3.patch, hbase-798-v4.patch, hbase-798-v5.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620433#action_12620433 ]
Jonathan Gray commented on HBASE-798:
-------------------------------------
After discussing with Jim on IRC, there won't be very many additional functions exposed through HTable.
It will actually just be one new version of each of: batchUpdate, deleteAll x 2, and deleteFamily.
So it would be those 4, plus the lockRow/unlockRow.
You think it would be okay to include just those in HTable w/o subclassing it?
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620823#action_12620823 ]
Jonathan Gray commented on HBASE-798:
-------------------------------------
Jim, I may have misspoken in my explanation.
There are only single HRS methods that take all arguments. Well, there are two of deleteAll but there's different HR implementations.
public void deleteAll(final byte [] regionName, final byte [] row,
final byte [] column, final long timestamp, final long lockId)
public void deleteAll(final byte [] regionName, final byte [] row,
final long timestamp, final long lockId)
public void deleteFamily(byte [] regionName, byte [] row, byte [] family,
long timestamp, final long lockId)
public void batchUpdate(final byte [] regionName, BatchUpdate b,
final long lockId)
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12621923#action_12621923 ]
Jim Kellerman commented on HBASE-798:
-------------------------------------
Reviewed patch. -1 because:
- HRegionServer.getLockFromId should renew the lease if it is passed an outstanding lock id
- HRegionServer.rowlocks should use java.util.concurrent.ConcurrentHashMap as it is much more efficient than Collections.synchronizedMap(HashMap)
- In HRegion, factor out the multiple occurrances of:
{code}
Integer lid = null;
if(lockid == null) {
lid = obtainRowLock(row);
} else {
if(!isRowLocked(lockid)) {
throw new IOException("Invalid row lock");
}
lid = lockid;
}
{code}
into a private or protected method, such as:
{code}
private Integer getLock(Integer lockid) throws IOException {
Integer lid = null;
if(lockid == null) {
lid = obtainRowLock(row);
} else {
if(!isRowLocked(lockid)) {
throw new IOException("Invalid row lock");
}
lid = lockid;
}
}
{code}
- isRowLocked should be protected or private
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch, hbase-798-v3.patch, hbase-798-v4.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12621929#action_12621929 ]
Jonathan Gray commented on HBASE-798:
-------------------------------------
Thanks for review Jim!
1) Agreed.
2) Okay. I was using the identical structure used to keep track of scanners:
Map<String, InternalScanner> scanners =
Collections.synchronizedMap(new HashMap<String, InternalScanner>());
Should that be changed as well?
3) Yes, I had it that way before but changed it for some reason. I think because I need to do a check at the end, but this checks the passed-in argument which is unchanged by the getLock function call. So you are correct, will change.
4) Yup.
Will wait to hear back regarding synchronizedMap(new Hashmap()) versus ConcurrentHashMap() before posting new patch.
Thanks again, Jim!
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch, hbase-798-v3.patch, hbase-798-v4.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman resolved HBASE-798.
---------------------------------
Resolution: Fixed
Added classes missing from final patch that were present in previous versions. Fix a number of compilation problems. All regression tests now pass.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch, hbase-798-v3.patch, hbase-798-v4.patch, hbase-798-v5.patch, hbase-798-v6.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620436#action_12620436 ]
stack commented on HBASE-798:
-----------------------------
I"m fine w/ that.
You might want to subclass anyways just because it groups this new functionality nicely.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620796#action_12620796 ]
Jean-Daniel Cryans commented on HBASE-798:
------------------------------------------
Patch seems nice Jonathan. Unfortunately I didn't have the chance to apply it since the path of the modification refers to "jonsrc" instead of "src".
Also, I think you forgot to include the class RowLock.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Attachments: hbase-798-v1.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620811#action_12620811 ]
Jim Kellerman commented on HBASE-798:
-------------------------------------
Jonathan Gray - 07/Aug/08 05:41 PM
I'm going to work on another patch now which adds a kind of "upsert/increment" functionality. I have a few particular and frequent use cases in mind that others may have a need for. Rough APIs below...
incrementColumnValue(table,row,family:column) -> Reads last value at column (must be family:column), increments it, and writes new column version with incremented value (wrapped in a single row lock for atomicity)
columnWithIncrementedValue(table,row,family,newcolumn) -> This may be more limited to others, but we have many ordered column families where every column value inserted is lexicographically smaller than all previous (always inserted to the top). The value within each column is an incremented value. This grabs the value of the top column in the family, increments it, and uses it as the value for the newcolumn to be inserted at the top. There will be no checking for newcolumn being smaller (lexicographically) than the last one, this is the responsibility of the client, but this is assumed for correctness.
incrementColumn(table,row,family) -> This is also specialized. We assume a schema where a family has only one column. We read that column value, increment it, add the new column (reusing the old columns value), delete the old column, release the lock, return the new incremented column to the client. This is a way to provide a check-out mechanism for incrementing identifiers.
I'm going to build this patch on top of the previous patch because some helper functions and classes will be useful.
[ Show » ]
> Jonathan Gray - 07/Aug/08 05:41 PM
> I'm going to work on another patch now which adds a kind of "upsert/increment" functionality. I have a few
> particular and frequent use cases in mind that others may have a need for. Rough APIs below...
> incrementColumnValue(table,row,family:column) -> Reads last value at column (must be family:column),
<snip>
Shouldn't this work be targeted at HBASE-493 ?
Or link this patch to HBASE-493, saying that the patch for this issue is included in the patch for this issue.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Attachments: hbase-798-v1.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620986#action_12620986 ]
Jonathan Gray commented on HBASE-798:
-------------------------------------
In our discussions we overlooked getRow, which calls HR.getFull, which does obtain row locks. Will make a new patch to add that.
Atomic increments discussed above have been moved to HBASE-803
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-798:
--------------------------------
Attachment: hbase-798-v4.patch
This patch is only a line break fix to allow this to apply against TRUNK (once HBASE-819 has been committed).
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Fix For: 0.3.0
>
> Attachments: hbase-798-v1.patch, hbase-798-v2.patch, hbase-798-v3.patch, hbase-798-v4.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-798) Provide Client API to explicitly lock
and unlock rows
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620798#action_12620798 ]
Jonathan Gray commented on HBASE-798:
-------------------------------------
Ah, I must have goofed my diff line. Will take a look and put the patch back up.
> Provide Client API to explicitly lock and unlock rows
> -----------------------------------------------------
>
> Key: HBASE-798
> URL: https://issues.apache.org/jira/browse/HBASE-798
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: client, ipc, regionserver
> Reporter: Jonathan Gray
> Priority: Minor
> Attachments: hbase-798-v1.patch
>
>
> We need to be able to perform a series of reads from and writes to a single row without any potential interference from other clients.
> Unfortunately this is a bit involved because normal reads currently do not acquire row locks, so it requires adding additional get/getRow calls that obtain and release a row lock.
> In addition, there will be two additional client calls, lockRow/unlockRow, which actually acquire and release the locks. Though each lock is associated with an HRegion, this will be tracked within the HRegionServer. When a lock is acquired from the client, it is handled much like a Scanner. We obtain the row lock from the HRegion, store the region name and lock identifier in a synchronized Map, and also obtain a lease to ensure that the lock will eventually be released even if the client dies.
> This also required adding a RowLockListener (implements LeaseListener) private class in HRS to handle row lock lease expiration.
> HRS.lockRow will return a long lockId (as openScanner does) that will be used in subsequent client calls to reuse this existing row lock. These calls will check that the lock is valid and perform the operations without any locking (wrappers around get*, new versions of batchUpdate, openScanner, etc).
> This is going to really add some noise to the list of available HTable/client methods so I'm not sure if it's something people would want to commit into a normal release. Regardless this does provide some very convenient functionality that may be useful to others.
> We are also looking into Clint Morgan's HBASE-669, but one major downside is that there is a significant amount of overhead involved. This row locking is already built in and this will only extend the API to allow clients to work with them directly. There is little to no overhead at all. The only (obvious) performance consideration is that this should only used where necessary as rows will not be locked and unlocked as quickly with round-trip client calls. In our design, we will have specific notes in our schema about which tables (or even which families or columns) must be accessed with row locks at all times and which do not.
> This is my first attempt at adding any additional functionality, so comments, criticism, code reviews are encouraged.
> I should have a patch up tomorrow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.