You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "xBis7 (via GitHub)" <gi...@apache.org> on 2023/11/09 14:53:46 UTC
[PR] HDDS-9662. Recon's keyContainerTable doesn't get cleaned up after key deletes [ozone]
xBis7 opened a new pull request, #5577:
URL: https://github.com/apache/ozone/pull/5577
## What changes were proposed in this pull request?
When Recon gets synced up with the OM, `ContainerKeyMapperTask` updates Recon's mapping tables. For key deletes on the OM, the tables are updated when the method `writeToTheDB()` calls
```
reconContainerMetadataManager
.batchDeleteContainerMapping(rdbBatchOperation, key);
```
`batchDeleteContainerMapping()` was only cleaning up the `containerKeyTable` leaving orphaned entries in the reverse mapping `keyContainerTable`. This patch fixes that.
There is a manual test on the jira, displaying the issue.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-9662
## How was this patch tested?
Existing unit tests were extended.
Green CI on my fork: https://github.com/xBis7/ozone/actions/runs/6811844924/job/18525926640
manual test in the docker-env:
```
> cd hadoop-ozone/dist/target/ozone-1.4.0-SNAPSHOT/compose/ozone
> docker-compose up --scale datanode=3 -d
> docker-compose exec recon bash
bash-4.2$ ozone freon ockg -t 10 -n 10
...
...
Total execution time (sec): 2
Failures: 0
Successful executions: 10
bash-4.2$ ozone fs -ls /vol1/bucket1
Found 1 items
drwxrwxrwx - hadoop hadoop 0 2023-11-09 14:48 /vol1/bucket1/dn0gs3wcx8
bash-4.2$ ozone fs -rm -r -skipTrash /vol1/bucket1/dn0gs3wcx8
Deleted /vol1/bucket1/dn0gs3wcx8
bash-4.2$ ozone sh key list /vol1/bucket1
[ ]
bash-4.2$ curl http://recon:9888/api/v1/triggerdbsync/om
truebash-4.2$
bash-4.2$
bash-4.2$ ozone debug ldb --db=/data/metadata/recon/recon-container-key.db_1699541279351/ scan --cf=containerKeyTable
{ }
bash-4.2$ ozone debug ldb --db=/data/metadata/recon/recon-container-key.db_1699541279351/ scan --cf=keyContainerTable
{ }
bash-4.2$ ozone debug ldb --db=/data/metadata/recon/recon-container-key.db_1699541279351/ scan --cf=containerKeyCountTable
{ 1: 0
, 2: 0
, 3: 0
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9662. Recon's keyContainerTable doesn't get cleaned up after key deletes [ozone]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 merged PR #5577:
URL: https://github.com/apache/ozone/pull/5577
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9662. Recon's keyContainerTable doesn't get cleaned up after key deletes [ozone]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #5577:
URL: https://github.com/apache/ozone/pull/5577#issuecomment-1812331519
Thanks @devmadhuu @symious @dombizita for the review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9662. Recon's keyContainerTable doesn't get cleaned up after key deletes [ozone]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #5577:
URL: https://github.com/apache/ozone/pull/5577#issuecomment-1803985121
@devmadhuu Can you take a look at this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9662. Recon's keyContainerTable doesn't get cleaned up after key deletes [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #5577:
URL: https://github.com/apache/ozone/pull/5577#issuecomment-1812333878
@xBis7 Please pay attention when merging PRs. If it has a single commit, GitHub uses the commit's message by default, not the PR title.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9662. Recon's keyContainerTable doesn't get cleaned up after key deletes [ozone]
Posted by "symious (via GitHub)" <gi...@apache.org>.
symious commented on PR #5577:
URL: https://github.com/apache/ozone/pull/5577#issuecomment-1811909706
LGTM, thanks for the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9662. Recon's keyContainerTable doesn't get cleaned up after key deletes [ozone]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #5577:
URL: https://github.com/apache/ozone/pull/5577#issuecomment-1812341031
@adoroszlai Sorry about that. I didn't notice because it was the same as the title. I'll be more careful next time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9662. Recon's keyContainerTable doesn't get cleaned up after key deletes [ozone]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #5577:
URL: https://github.com/apache/ozone/pull/5577#issuecomment-1812345792
@adoroszlai Is there a way to undo this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9662. Recon's keyContainerTable doesn't get cleaned up after key deletes [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #5577:
URL: https://github.com/apache/ozone/pull/5577#issuecomment-1812411122
> Is there a clean way to undo this?
Undo is possible only with forced push, which is dangerous, so let's avoid that. We could also revert and create a new commit, but then the old commit would still be there, and it would just create noise.
Fortunately the PR number is included in the message, so it's less of a problem.
I don't think we need to do anything about it now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org