You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/06/17 17:00:27 UTC
[GitHub] [ozone] aswinshakil opened a new pull request, #3529: HDDS-6812. Fix INTERNAL_ERROR message on failed write.
aswinshakil opened a new pull request, #3529:
URL: https://github.com/apache/ozone/pull/3529
## What changes were proposed in this pull request?
When `allocateBlock()` fails, It throws a generic SCMException `Allocated 0 blocks. Requested 1 blocks`. Here the block allocation failed because there weren't enough DataNodes for EC replication. Proposing to throw an exception when `allocateContainer()` fails and propagate back the actual Exception.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-6812
## How was this patch tested?
This patch was tested manually with docker and Unit test. Now the error message is,
```
INTERNAL_ERROR No enough datanodes to choose. TotalNode = 3 AvailableNode = 3 RequiredNode = 5 ExcludedNode = 0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] umamaheswararao commented on pull request #3529: HDDS-6812. Fix INTERNAL_ERROR message on failed write.
Posted by GitBox <gi...@apache.org>.
umamaheswararao commented on PR #3529:
URL: https://github.com/apache/ozone/pull/3529#issuecomment-1163990217
Thanks @aswinshakil for working on this. Changes looks good to me. I have a general question:
try {
for (int i = 0; i < num; i++) {
AllocatedBlock block = scm.getScmBlockManager()
.allocateBlock(size, replicationConfig, owner, excludeList);
if (block != null) {
blocks.add(block);
}
}
return blocks;
} catch (Exception ex) {
Now here it will never return null since you are throwing exception? If yes, probably you may want to file a JIRA to revisit this to remove the if block!=null check and looping?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] flyphant commented on pull request #3529: HDDS-6812. Fix INTERNAL_ERROR message on failed write.
Posted by "flyphant (via GitHub)" <gi...@apache.org>.
flyphant commented on PR #3529:
URL: https://github.com/apache/ozone/pull/3529#issuecomment-1465455592
when i use cli command on the single node depolyment, input " bin/ozone sh key put /vol1/bucket1/key1 /tmp/myfile" , it's alse return the error: INTERNAL_ERROR Allocated 0 blocks. Requested 1 blocks
my version is 1.3.0, it's this bug fixed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] umamaheswararao merged pull request #3529: HDDS-6812. Fix INTERNAL_ERROR message on failed write.
Posted by GitBox <gi...@apache.org>.
umamaheswararao merged PR #3529:
URL: https://github.com/apache/ozone/pull/3529
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] umamaheswararao commented on a diff in pull request #3529: HDDS-6812. Fix INTERNAL_ERROR message on failed write.
Posted by GitBox <gi...@apache.org>.
umamaheswararao commented on code in PR #3529:
URL: https://github.com/apache/ozone/pull/3529#discussion_r904619448
##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestWritableECContainerProvider.java:
##########
@@ -261,12 +263,18 @@ public Pipeline createPipeline(ReplicationConfig repConf,
provider = new WritableECContainerProvider(
conf, pipelineManager, containerManager, pipelineChoosingPolicy);
- ContainerInfo container =
- provider.getContainer(1, repConfig, OWNER, new ExcludeList());
+ try {
+ provider.getContainer(1, repConfig, OWNER, new ExcludeList());
+ } catch (IOException ex) {
+ GenericTestUtils.assertExceptionContains("Cannot create pipelines", ex);
+ }
+
for (int i = 0; i < 5; i++) {
- ContainerInfo nextContainer =
- provider.getContainer(1, repConfig, OWNER, new ExcludeList());
- assertEquals(container, nextContainer);
+ try {
+ provider.getContainer(1, repConfig, OWNER, new ExcludeList());
Review Comment:
Same as above. Please few other places need to handle this.
##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestWritableECContainerProvider.java:
##########
@@ -234,9 +234,11 @@ public Pipeline createPipeline(ReplicationConfig repConf,
provider = new WritableECContainerProvider(
conf, pipelineManager, containerManager, pipelineChoosingPolicy);
- ContainerInfo container =
- provider.getContainer(1, repConfig, OWNER, new ExcludeList());
- assertNull(container);
+ try {
+ provider.getContainer(1, repConfig, OWNER, new ExcludeList());
Review Comment:
You have to add Assert.fail here. Lets say if above statement does not throw exception, then case will pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] aswinshakil commented on a diff in pull request #3529: HDDS-6812. Fix INTERNAL_ERROR message on failed write.
Posted by GitBox <gi...@apache.org>.
aswinshakil commented on code in PR #3529:
URL: https://github.com/apache/ozone/pull/3529#discussion_r905326264
##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestWritableECContainerProvider.java:
##########
@@ -234,9 +234,11 @@ public Pipeline createPipeline(ReplicationConfig repConf,
provider = new WritableECContainerProvider(
conf, pipelineManager, containerManager, pipelineChoosingPolicy);
- ContainerInfo container =
- provider.getContainer(1, repConfig, OWNER, new ExcludeList());
- assertNull(container);
+ try {
+ provider.getContainer(1, repConfig, OWNER, new ExcludeList());
Review Comment:
Thank for pointing it out. Updated the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] aswinshakil commented on pull request #3529: HDDS-6812. Fix INTERNAL_ERROR message on failed write.
Posted by GitBox <gi...@apache.org>.
aswinshakil commented on PR #3529:
URL: https://github.com/apache/ozone/pull/3529#issuecomment-1164724599
@umamaheswararao Thanks for the review. I will update the null checks and other changes in a follow-up JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org