You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "umamaheswararao (via GitHub)" <gi...@apache.org> on 2023/02/08 08:36:50 UTC
[GitHub] [ozone] umamaheswararao opened a new pull request, #4258: HDDS-7923. [EC] Reconstruction is failing with IndexOutOfBoundsException
umamaheswararao opened a new pull request, #4258:
URL: https://github.com/apache/ozone/pull/4258
## What changes were proposed in this pull request?
Handled the case when chunk list null, but getChunks internal method returning empty list. In this case, we have made wrong assumption and access array elements. This fixed the issue and added the necessary checks.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7923
## How was this patch tested?
Added the test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen merged pull request #4258: HDDS-7923. [EC] Reconstruction is failing with IndexOutOfBoundsException
Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen merged PR #4258:
URL: https://github.com/apache/ozone/pull/4258
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on a diff in pull request #4258: HDDS-7923. [EC] Reconstruction is failing with IndexOutOfBoundsException
Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen commented on code in PR #4258:
URL: https://github.com/apache/ozone/pull/4258#discussion_r1100458159
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/storage/TestContainerCommandsEC.java:
##########
@@ -349,9 +349,16 @@ private static byte[] getBytesWith(int singleDigitNumber, int total) {
@MethodSource("recoverableMissingIndexes")
void testECReconstructionCoordinatorWith(List<Integer> missingIndexes)
throws Exception {
- testECReconstructionCoordinator(missingIndexes);
+ testECReconstructionCoordinator(missingIndexes, 3);
}
+ @Test
+ void testECReconstructionWithPartialStripe()
Review Comment:
> It is failing when I run this test. Interestingly I noticed that, when I run the tests together with testECReconstructionCoordinatorWithMissingIndexes135, it passes. Can you try running this test alone?
Same. Maybe there is interference among the tests here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on pull request #4258: HDDS-7923. [EC] Reconstruction is failing with IndexOutOfBoundsException
Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen commented on PR #4258:
URL: https://github.com/apache/ozone/pull/4258#issuecomment-1423520840
Merged, thanks @umamaheswararao and @adoroszlai.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] umamaheswararao commented on a diff in pull request #4258: HDDS-7923. [EC] Reconstruction is failing with IndexOutOfBoundsException
Posted by "umamaheswararao (via GitHub)" <gi...@apache.org>.
umamaheswararao commented on code in PR #4258:
URL: https://github.com/apache/ozone/pull/4258#discussion_r1100467636
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/storage/TestContainerCommandsEC.java:
##########
@@ -349,9 +349,16 @@ private static byte[] getBytesWith(int singleDigitNumber, int total) {
@MethodSource("recoverableMissingIndexes")
void testECReconstructionCoordinatorWith(List<Integer> missingIndexes)
throws Exception {
- testECReconstructionCoordinator(missingIndexes);
+ testECReconstructionCoordinator(missingIndexes, 3);
}
+ @Test
+ void testECReconstructionWithPartialStripe()
Review Comment:
I figured out the problem. Test indeed tests the behavior, however because of using the inputchunks array which was might created by other tests, number of bytes written to file was more than one chunk. So, it's not meeting the expected situation when ran with other tests. If you run this test alone, it will consistently fail as input chunk initialization will happen as expected, that is just 1 chunk. I will update the patch shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on a diff in pull request #4258: HDDS-7923. [EC] Reconstruction is failing with IndexOutOfBoundsException
Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen commented on code in PR #4258:
URL: https://github.com/apache/ozone/pull/4258#discussion_r1099988731
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/storage/TestContainerCommandsEC.java:
##########
@@ -349,9 +349,16 @@ private static byte[] getBytesWith(int singleDigitNumber, int total) {
@MethodSource("recoverableMissingIndexes")
void testECReconstructionCoordinatorWith(List<Integer> missingIndexes)
throws Exception {
- testECReconstructionCoordinator(missingIndexes);
+ testECReconstructionCoordinator(missingIndexes, 3);
}
+ @Test
+ void testECReconstructionWithPartialStripe()
Review Comment:
Hi @umamaheswararao, the fix looks good.
However, seems the test here is not covering the bug.
Could you please take another look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] umamaheswararao commented on a diff in pull request #4258: HDDS-7923. [EC] Reconstruction is failing with IndexOutOfBoundsException
Posted by "umamaheswararao (via GitHub)" <gi...@apache.org>.
umamaheswararao commented on code in PR #4258:
URL: https://github.com/apache/ozone/pull/4258#discussion_r1100418750
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/storage/TestContainerCommandsEC.java:
##########
@@ -349,9 +349,16 @@ private static byte[] getBytesWith(int singleDigitNumber, int total) {
@MethodSource("recoverableMissingIndexes")
void testECReconstructionCoordinatorWith(List<Integer> missingIndexes)
throws Exception {
- testECReconstructionCoordinator(missingIndexes);
+ testECReconstructionCoordinator(missingIndexes, 3);
}
+ @Test
+ void testECReconstructionWithPartialStripe()
Review Comment:
<img width="1717" alt="Screen Shot 2023-02-08 at 8 39 49 AM" src="https://user-images.githubusercontent.com/1719507/217594332-898765cf-c936-4e23-b7c7-d9df5c6b1378.png">
It is failing when I run this test.
Interestingly I noticed that, when I run the tests together with testECReconstructionCoordinatorWithMissingIndexes135, it passes. Can you try running this test alone?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on a diff in pull request #4258: HDDS-7923. [EC] Reconstruction is failing with IndexOutOfBoundsException
Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen commented on code in PR #4258:
URL: https://github.com/apache/ozone/pull/4258#discussion_r1100458159
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/storage/TestContainerCommandsEC.java:
##########
@@ -349,9 +349,16 @@ private static byte[] getBytesWith(int singleDigitNumber, int total) {
@MethodSource("recoverableMissingIndexes")
void testECReconstructionCoordinatorWith(List<Integer> missingIndexes)
throws Exception {
- testECReconstructionCoordinator(missingIndexes);
+ testECReconstructionCoordinator(missingIndexes, 3);
}
+ @Test
+ void testECReconstructionWithPartialStripe()
Review Comment:
> It is failing when I run this test. Interestingly I noticed that, when I run the tests together with testECReconstructionCoordinatorWithMissingIndexes135, it passes. Can you try running this test alone?
Confirmed. Maybe there is interference among the tests here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org