You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "aswinshakil (via GitHub)" <gi...@apache.org> on 2025/07/30 23:02:53 UTC
[PR] HDDS-12990. Generate tree from metadata when it doesn't exist during getContainerChecksumInfo call [ozone]
aswinshakil opened a new pull request, #8881:
URL: https://github.com/apache/ozone/pull/8881
## What changes were proposed in this pull request?
Currently the client will get an [IO_EXCEPTION](https://github.com/apache/ozone/blob/229e5c94eaf1f16b9dcfdb19d7cc5451d104a6dd/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java#L794) result code when [requesting](https://github.com/apache/ozone/blob/8768d048160a6b3761ddc2938d7287f9cd609e04/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/checksum/ContainerChecksumTreeManager.java#L366) the merkle tree of a peer that has not been generated yet. If the container is not open, have the peer build the merkle tree from metadata and return this instead. This change builds merkle tree if it's not available.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-12990
## How was this patch tested?
Existing tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12990. Generate tree from metadata when it doesn't exist during getContainerChecksumInfo call [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #8881:
URL: https://github.com/apache/ozone/pull/8881#issuecomment-3162367687
@errose28 @aswinshakil This seems to break master. Please kindly help to rectify it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12990. Generate tree from metadata when it doesn't exist during getContainerChecksumInfo call [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 merged PR #8881:
URL: https://github.com/apache/ozone/pull/8881
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12990. Generate tree from metadata when it doesn't exist during getContainerChecksumInfo call [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #8881:
URL: https://github.com/apache/ozone/pull/8881#discussion_r2255429081
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java:
##########
@@ -760,14 +761,33 @@ ContainerCommandResponseProto handleGetContainerChecksumInfo(
try {
checksumTree = checksumManager.getContainerChecksumInfo(containerData);
} catch (IOException ex) {
- // For file not found or other inability to read the file, return an error to the client.
- LOG.error("Error occurred when reading checksum file for container {}", containerData.getContainerID(), ex);
- return ContainerCommandResponseProto.newBuilder()
- .setCmdType(request.getCmdType())
- .setTraceID(request.getTraceID())
- .setResult(IO_EXCEPTION)
- .setMessage(ex.getMessage())
- .build();
+ // Only build from metadata if the file doesn't exist
+ if (ex instanceof FileNotFoundException) {
+ try {
+ LOG.info("Checksum tree file not found for container {}. Building from merkle tree from metadata.",
Review Comment:
```suggestion
LOG.info("Checksum tree file not found for container {}. Building merkle tree from container metadata.",
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12990. Generate tree from metadata when it doesn't exist during getContainerChecksumInfo call [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #8881:
URL: https://github.com/apache/ozone/pull/8881#issuecomment-3166275348
Thanks @adoroszlai for helping to resolve the issue promptly and @errose28 for tracing the issue.
> I'm not sure if there's a way to prevent these sort of issues entirely, since there's always going to be a gap between when a PR's CI completes and when it is merged, during which any other change could show up in master.
Yes, IMO it's an acceptable tradeoff between occasional compile issue and needing to merge to latest master (which will slow down merge). I think it's an acceptable tradeoff if we can be rectify the issue quickly (e.g. checking master branch frequently + checking for notification for CI failures on master when the PR is meregd).
@adoroszlai By the way, can I check whether any committer / PMC has the privilege to push directly to master to fix compile issue or is there any SOP that we need to follow?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12990. Generate tree from metadata when it doesn't exist during getContainerChecksumInfo call [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #8881:
URL: https://github.com/apache/ozone/pull/8881#issuecomment-3166753694
> whether any committer / PMC has the privilege to push directly to master
Yes.
> a way to prevent these sort of issues entirely
Not completely, but here's a basic sanity check to be run locally before merge:
```
git fetch origin
gh pr checkout 8881
git merge --no-commit origin/master \
&& mvn -DskipRecon -DskipShade -DskipTests clean verify \
&& ./hadoop-ozone/dev-support/checks/checkstyle.sh \
&& ./hadoop-ozone/dev-support/checks/pmd.sh
```
(It uses GitHub CLI, but can also be done using few more lower level commands.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12990. Generate tree from metadata when it doesn't exist during getContainerChecksumInfo call [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #8881:
URL: https://github.com/apache/ozone/pull/8881#issuecomment-3164663911
Good catch @ivandika3 and @adoroszlai. Looks like the order of events was:
- CI green on this PR with master
- #8604 is merged, which updates master in such a way that compilation for this PR fails but doesn't generate a merge conflict
- This PR is merged based on the previous green CI
- The first CI run of this PR with the latest master changes happens on master itself and fails
I'm not sure if there's a way to prevent these sort of issues entirely, since there's always going to be a gap between when a PR's CI completes and when it is merged, during which any other change could show up in master. I guess if reviewers are aware of related changes in master they can re-trigger the CI before merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org