You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/06/24 11:20:00 UTC
[GitHub] [ozone] bharatviswa504 opened a new pull request #2364: HDDS-5349: Avoid usage of locks
bharatviswa504 opened a new pull request #2364:
URL: https://github.com/apache/ozone/pull/2364
## What changes were proposed in this pull request?
Avoid usage of locks when iterating rocksdb iterator.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5349
## How was this patch tested?
Existing UT's.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on pull request #2364: HDDS-5349: Avoid usage of lock in listStatus
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #2364:
URL: https://github.com/apache/ozone/pull/2364#issuecomment-868288591
Closing this, as branch name is changed in my fork.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 commented on pull request #2364: HDDS-5349: Avoid usage of lock in listStatus
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #2364:
URL: https://github.com/apache/ozone/pull/2364#issuecomment-868290145
Opened #2369 for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] bharatviswa504 closed pull request #2364: HDDS-5349: Avoid usage of lock in listStatus
Posted by GitBox <gi...@apache.org>.
bharatviswa504 closed pull request #2364:
URL: https://github.com/apache/ozone/pull/2364
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a change in pull request #2364: HDDS-5349: Avoid usage of lock in listStatus
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #2364:
URL: https://github.com/apache/ozone/pull/2364#discussion_r657867932
##########
File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
##########
@@ -2318,110 +2319,111 @@ private void listStatusFindKeyInTableCache(
startKey = OzoneFSUtils.addTrailingSlashIfNeeded(keyName);
}
+ // Note: eliminating the case where startCacheKey could end with '//'
+ String keyArgs = OzoneFSUtils.addTrailingSlashIfNeeded(
+ metadataManager.getOzoneKey(volumeName, bucketName, keyName));
+
metadataManager.getLock().acquireReadLock(BUCKET_LOCK, volumeName,
bucketName);
+
+ Table keyTable = metadataManager.getKeyTable();
try {
- Table keyTable = metadataManager.getKeyTable();
- Iterator<Map.Entry<CacheKey<String>, CacheValue<OmKeyInfo>>>
+ Iterator< Map.Entry< CacheKey< String >, CacheValue< OmKeyInfo > > >
cacheIter = keyTable.cacheIterator();
String startCacheKey = OZONE_URI_DELIMITER + volumeName +
OZONE_URI_DELIMITER + bucketName + OZONE_URI_DELIMITER +
((startKey.equals(OZONE_URI_DELIMITER)) ? "" : startKey);
- // Note: eliminating the case where startCacheKey could end with '//'
- String keyArgs = OzoneFSUtils.addTrailingSlashIfNeeded(
- metadataManager.getOzoneKey(volumeName, bucketName, keyName));
// First, find key in TableCache
listStatusFindKeyInTableCache(cacheIter, keyArgs, startCacheKey,
recursive, cacheKeyMap, deletedKeySet);
- // Then, find key in DB
- String seekKeyInDb =
- metadataManager.getOzoneKey(volumeName, bucketName, startKey);
- TableIterator<String, ? extends Table.KeyValue<String, OmKeyInfo>>
- iterator = keyTable.iterator();
- iterator.seek(seekKeyInDb);
- int countEntries = 0;
- if (iterator.hasNext()) {
- if (iterator.key().equals(keyArgs)) {
- // Skip the key itself, since we are listing inside the directory
- iterator.next();
- }
- // Iterate through seek results
- while (iterator.hasNext() && numEntries - countEntries > 0) {
- String entryInDb = iterator.key();
- OmKeyInfo omKeyInfo = iterator.value().getValue();
- if (entryInDb.startsWith(keyArgs)) {
- String entryKeyName = omKeyInfo.getKeyName();
- if (recursive) {
- // for recursive list all the entries
+ } finally {
+ metadataManager.getLock().acquireReadLock(BUCKET_LOCK, volumeName,
Review comment:
```suggestion
metadataManager.getLock().releaseReadLock(BUCKET_LOCK, volumeName,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org