You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "SaketaChalamchala (via GitHub)" <gi...@apache.org> on 2023/08/15 21:28:48 UTC
[GitHub] [ozone] SaketaChalamchala opened a new pull request, #5189: HDDS-7275. Improve error logging when blocks cannot be allocated with RATIS THREE.
SaketaChalamchala opened a new pull request, #5189:
URL: https://github.com/apache/ozone/pull/5189
## What changes were proposed in this pull request?
Current implementation returns a null and later on ScmBlockLocationProtocolServerSideTranslatorPB#allocateScmBlock() throws an exception with #blocks allocated and requested.
```
# hdfs dfs -put /etc/krb5.conf ofs://ozone1/vol1/bucket/3
put: Allocated 0 blocks. Requested 1 blocks
```
Proposed changed returns an exception when a block cannot be allocated after trying all possible strategies
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7275
## How was this patch tested?
Manual.
```
sh-4.2$ ozone sh key put /vol1/bucket1/key1 CONTRIBUTING.md
INTERNAL_ERROR Unable to allocate a container to the block of size: 1048576, replicationConfig: RATIS/THREE
sh-4.2$
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] SaketaChalamchala commented on pull request #5189: HDDS-7275. Improve error logging when blocks cannot be allocated with RATIS THREE.
Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on PR #5189:
URL: https://github.com/apache/ozone/pull/5189#issuecomment-1692589503
> Thanks for improving this @SaketaChalamchala, this message has been confusing for users before. Could we also add a message summarizing why the allocation failed? For example, the log lines farther up in `getContainer` provide more information to the SCM log, but those messages could be returned to the client as well.
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] errose28 commented on a diff in pull request #5189: HDDS-7275. Improve error logging when blocks cannot be allocated with RATIS THREE.
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #5189:
URL: https://github.com/apache/ozone/pull/5189#discussion_r1307959592
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableRatisContainerProvider.java:
##########
@@ -167,11 +173,14 @@ public ContainerInfo getContainer(final long size,
}
// we have tried all strategies we know and but somehow we are not able
- // to get a container for this block. Log that info and return a null.
+ // to get a container for this block. Log that info and return a null and
+ // throw an exception.
Review Comment:
Suggested change:
```
// we have tried all strategies we know but somehow we are not able
// to get a container for this block. Log that info and throw an exception.
```
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableRatisContainerProvider.java:
##########
@@ -136,11 +137,16 @@ public ContainerInfo getContainer(final long size,
} catch (IOException e) {
LOG.warn("Waiting for one of pipelines {} to be OPEN failed. ",
allocatedPipelineIDs, e);
+ failureReason = "Waiting for one of pipelines to be OPEN failed. "
+ + e.getMessage();
Review Comment:
Should we include the allocated pipeline IDs in this message too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] errose28 merged pull request #5189: HDDS-7275. Improve error logging when blocks cannot be allocated with RATIS THREE.
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 merged PR #5189:
URL: https://github.com/apache/ozone/pull/5189
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] SaketaChalamchala commented on a diff in pull request #5189: HDDS-7275. Improve error logging when blocks cannot be allocated with RATIS THREE.
Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #5189:
URL: https://github.com/apache/ozone/pull/5189#discussion_r1309452471
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableRatisContainerProvider.java:
##########
@@ -167,11 +173,14 @@ public ContainerInfo getContainer(final long size,
}
// we have tried all strategies we know and but somehow we are not able
- // to get a container for this block. Log that info and return a null.
+ // to get a container for this block. Log that info and return a null and
+ // throw an exception.
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] errose28 commented on pull request #5189: HDDS-7275. Improve error logging when blocks cannot be allocated with RATIS THREE.
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #5189:
URL: https://github.com/apache/ozone/pull/5189#issuecomment-1707112639
Looks like #5217 is not progressing as fast as I thought, I will go ahead with the CI on this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] errose28 commented on pull request #5189: HDDS-7275. Improve error logging when blocks cannot be allocated with RATIS THREE.
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #5189:
URL: https://github.com/apache/ozone/pull/5189#issuecomment-1683151360
Thanks for improving this @SaketaChalamchala, this message has been confusing for users before. Could we also add a message summarizing why the allocation failed? For example, the log lines farther up in `getContainer` provide more information to the SCM log, but those messages could be returned to the client as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] SaketaChalamchala commented on a diff in pull request #5189: HDDS-7275. Improve error logging when blocks cannot be allocated with RATIS THREE.
Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #5189:
URL: https://github.com/apache/ozone/pull/5189#discussion_r1309451654
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableRatisContainerProvider.java:
##########
@@ -136,11 +137,16 @@ public ContainerInfo getContainer(final long size,
} catch (IOException e) {
LOG.warn("Waiting for one of pipelines {} to be OPEN failed. ",
allocatedPipelineIDs, e);
+ failureReason = "Waiting for one of pipelines to be OPEN failed. "
+ + e.getMessage();
Review Comment:
Would it provide more clarity than the error message? It might not be efficient to send allocated pipeline list to the client is it is too large.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] errose28 commented on a diff in pull request #5189: HDDS-7275. Improve error logging when blocks cannot be allocated with RATIS THREE.
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #5189:
URL: https://github.com/apache/ozone/pull/5189#discussion_r1310618026
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/WritableRatisContainerProvider.java:
##########
@@ -136,11 +137,16 @@ public ContainerInfo getContainer(final long size,
} catch (IOException e) {
LOG.warn("Waiting for one of pipelines {} to be OPEN failed. ",
allocatedPipelineIDs, e);
+ failureReason = "Waiting for one of pipelines to be OPEN failed. "
+ + e.getMessage();
Review Comment:
Sure we can omit it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] errose28 commented on pull request #5189: HDDS-7275. Improve error logging when blocks cannot be allocated with RATIS THREE.
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #5189:
URL: https://github.com/apache/ozone/pull/5189#issuecomment-1708800271
CI failures are fixed by #5217
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org