You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "hemantk-12 (via GitHub)" <gi...@apache.org> on 2024/01/31 21:19:04 UTC
[PR] HDDS-10250 Use SnapshotId as key in SnapshotCache [ozone]
hemantk-12 opened a new pull request, #6139:
URL: https://github.com/apache/ozone/pull/6139
## What changes were proposed in this pull request?
Currently, `SnapshotName` (`/volumeName/bucketName/snapshotName`) is used as key in SnapshotCache which makes it hard for operations like Snapshot rename. Discussions: https://github.com/apache/ozone/pull/6006#pullrequestreview-1830469508
This change is to use `SnapshotId` as key in SnapshotCache rather than SnapshotName because `SnapshotId` is unique and is never changed throughout the snapshot's lifecycle.
## What is the link to the Apache JIRA
HDDS-10250
## How was this patch tested?
Updated existing unit tests to work with SnapshotId. No new tests were added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10250. Use SnapshotId as key in SnapshotCache [ozone]
Posted by "hemantk-12 (via GitHub)" <gi...@apache.org>.
hemantk-12 commented on PR #6139:
URL: https://github.com/apache/ozone/pull/6139#issuecomment-1937265192
Thanks @aswinshakil and @smengcl for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10250. Use SnapshotId as key in SnapshotCache [ozone]
Posted by "hemantk-12 (via GitHub)" <gi...@apache.org>.
hemantk-12 merged PR #6139:
URL: https://github.com/apache/ozone/pull/6139
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10250. Use SnapshotId as key in SnapshotCache [ozone]
Posted by "aswinshakil (via GitHub)" <gi...@apache.org>.
aswinshakil commented on code in PR #6139:
URL: https://github.com/apache/ozone/pull/6139#discussion_r1483517743
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java:
##########
@@ -325,14 +326,15 @@ public boolean canDisableFsSnapshot(OMMetadataManager ommm) {
return isSnapshotInfoTableEmpty;
}
- private CacheLoader<String, OmSnapshot> createCacheLoader() {
- return new CacheLoader<String, OmSnapshot>() {
+ private CacheLoader<UUID, OmSnapshot> createCacheLoader() {
+ return new CacheLoader<UUID, OmSnapshot>() {
@Nonnull
@Override
- public OmSnapshot load(@Nonnull String snapshotTableKey)
- throws IOException {
+ public OmSnapshot load(@Nonnull UUID snapshotId) throws IOException {
// Check if the snapshot exists
+ String snapshotTableKey = ((OmMetadataManagerImpl) ozoneManager.getMetadataManager()).getSnapshotChainManager()
Review Comment:
Should we add a null check here just as a sanity check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10250. Use SnapshotId as key in SnapshotCache [ozone]
Posted by "hemantk-12 (via GitHub)" <gi...@apache.org>.
hemantk-12 commented on code in PR #6139:
URL: https://github.com/apache/ozone/pull/6139#discussion_r1484896422
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java:
##########
@@ -325,14 +326,15 @@ public boolean canDisableFsSnapshot(OMMetadataManager ommm) {
return isSnapshotInfoTableEmpty;
}
- private CacheLoader<String, OmSnapshot> createCacheLoader() {
- return new CacheLoader<String, OmSnapshot>() {
+ private CacheLoader<UUID, OmSnapshot> createCacheLoader() {
+ return new CacheLoader<UUID, OmSnapshot>() {
@Nonnull
@Override
- public OmSnapshot load(@Nonnull String snapshotTableKey)
- throws IOException {
+ public OmSnapshot load(@Nonnull UUID snapshotId) throws IOException {
// Check if the snapshot exists
+ String snapshotTableKey = ((OmMetadataManagerImpl) ozoneManager.getMetadataManager()).getSnapshotChainManager()
Review Comment:
It makes sense to add a null check here. Added it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10250. Use SnapshotId as key in SnapshotCache [ozone]
Posted by "hemantk-12 (via GitHub)" <gi...@apache.org>.
hemantk-12 commented on code in PR #6139:
URL: https://github.com/apache/ozone/pull/6139#discussion_r1484899532
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java:
##########
@@ -325,19 +326,25 @@ public boolean canDisableFsSnapshot(OMMetadataManager ommm) {
return isSnapshotInfoTableEmpty;
}
- private CacheLoader<String, OmSnapshot> createCacheLoader() {
- return new CacheLoader<String, OmSnapshot>() {
+ private CacheLoader<UUID, OmSnapshot> createCacheLoader() {
+ return new CacheLoader<UUID, OmSnapshot>() {
@Nonnull
@Override
- public OmSnapshot load(@Nonnull String snapshotTableKey)
- throws IOException {
- // Check if the snapshot exists
- final SnapshotInfo snapshotInfo = getSnapshotInfo(snapshotTableKey);
+ public OmSnapshot load(@Nonnull UUID snapshotId) throws IOException {
+ String snapshotTableKey = ((OmMetadataManagerImpl) ozoneManager.getMetadataManager())
+ .getSnapshotChainManager()
+ .getTableKey(snapshotId);
+
+ // SnapshotChain maintains in-memory reverse mapping of snapshotId to snapshotName based on snapshotInfoTable.
+ // So it should not happen ideally.
+ // If it happens, then either snapshot has been purged in between or SnapshotChain is corrupted
+ // and missing some entries which needs investigation.
+ if (snapshotTableKey == null) {
+ throw new IOException("No snapshot exist with snapshotId: " + snapshotId);
+ }
- // Block snapshot from loading when it is no longer active e.g. DELETED,
- // unless this is called from SnapshotDeletingService.
- checkSnapshotActive(snapshotInfo, true);
Review Comment:
@aswinshakil this is not needed. Please correct me if I'm wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10250. Use SnapshotId as key in SnapshotCache [ozone]
Posted by "aswinshakil (via GitHub)" <gi...@apache.org>.
aswinshakil commented on code in PR #6139:
URL: https://github.com/apache/ozone/pull/6139#discussion_r1484909417
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java:
##########
@@ -325,19 +326,25 @@ public boolean canDisableFsSnapshot(OMMetadataManager ommm) {
return isSnapshotInfoTableEmpty;
}
- private CacheLoader<String, OmSnapshot> createCacheLoader() {
- return new CacheLoader<String, OmSnapshot>() {
+ private CacheLoader<UUID, OmSnapshot> createCacheLoader() {
+ return new CacheLoader<UUID, OmSnapshot>() {
@Nonnull
@Override
- public OmSnapshot load(@Nonnull String snapshotTableKey)
- throws IOException {
- // Check if the snapshot exists
- final SnapshotInfo snapshotInfo = getSnapshotInfo(snapshotTableKey);
+ public OmSnapshot load(@Nonnull UUID snapshotId) throws IOException {
+ String snapshotTableKey = ((OmMetadataManagerImpl) ozoneManager.getMetadataManager())
+ .getSnapshotChainManager()
+ .getTableKey(snapshotId);
+
+ // SnapshotChain maintains in-memory reverse mapping of snapshotId to snapshotName based on snapshotInfoTable.
+ // So it should not happen ideally.
+ // If it happens, then either snapshot has been purged in between or SnapshotChain is corrupted
+ // and missing some entries which needs investigation.
+ if (snapshotTableKey == null) {
+ throw new IOException("No snapshot exist with snapshotId: " + snapshotId);
+ }
- // Block snapshot from loading when it is no longer active e.g. DELETED,
- // unless this is called from SnapshotDeletingService.
- checkSnapshotActive(snapshotInfo, true);
Review Comment:
We needed it as a rail guard so that `DELETED` snapshot is not loaded by other services. But I don't think we would need it. Reason is that we already block FS calls to snapshot, We check snapshot status in snapDiff for each page. Make sense to remove this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10250. Use SnapshotId as key in SnapshotCache [ozone]
Posted by "aswinshakil (via GitHub)" <gi...@apache.org>.
aswinshakil commented on code in PR #6139:
URL: https://github.com/apache/ozone/pull/6139#discussion_r1483517743
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java:
##########
@@ -325,14 +326,15 @@ public boolean canDisableFsSnapshot(OMMetadataManager ommm) {
return isSnapshotInfoTableEmpty;
}
- private CacheLoader<String, OmSnapshot> createCacheLoader() {
- return new CacheLoader<String, OmSnapshot>() {
+ private CacheLoader<UUID, OmSnapshot> createCacheLoader() {
+ return new CacheLoader<UUID, OmSnapshot>() {
@Nonnull
@Override
- public OmSnapshot load(@Nonnull String snapshotTableKey)
- throws IOException {
+ public OmSnapshot load(@Nonnull UUID snapshotId) throws IOException {
// Check if the snapshot exists
+ String snapshotTableKey = ((OmMetadataManagerImpl) ozoneManager.getMetadataManager()).getSnapshotChainManager()
Review Comment:
Should we add a null check for `snapshotTableKey` here just as a sanity check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org