You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/02/24 14:37:11 UTC
[GitHub] [hadoop-ozone] bshashikant opened a new pull request #592:
HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
bshashikant opened a new pull request #592: HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
URL: https://github.com/apache/hadoop-ozone/pull/592
## What changes were proposed in this pull request?
Added configs for rpc and watch request timeout for ratis client and server. Modified the unit tests
to remove conditions for asserting on write chunk and putblock metric count in tests after inducing failure as given the nature of retry and async write along wit pipeline destruction, these counters cannot be deterministic after failure.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3013
## How was this patch tested?
Ran the unit test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on issue #592: HDDS-3013. Fix
TestBlockOutputStreamWithFailures.java.
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on issue #592: HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
URL: https://github.com/apache/hadoop-ozone/pull/592#issuecomment-593855721
> can you please share log and output file for the failure?
Sure, it's available in the `integration` artifact from the [run page](https://github.com/adoroszlai/hadoop-ozone/runs/473056970):
https://github.com/adoroszlai/hadoop-ozone/suites/486485662/artifacts/2311772
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on issue #592: HDDS-3013. Fix
TestBlockOutputStreamWithFailures.java.
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on issue #592: HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
URL: https://github.com/apache/hadoop-ozone/pull/592#issuecomment-594056829
`TestRandomKeyGenerator` (freon) timeout is unrelated, reported in [HDDS-3086](https://issues.apache.org/jira/browse/HDDS-3086).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bshashikant commented on issue #592: HDDS-3013. Fix
TestBlockOutputStreamWithFailures.java.
Posted by GitBox <gi...@apache.org>.
bshashikant commented on issue #592: HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
URL: https://github.com/apache/hadoop-ozone/pull/592#issuecomment-593853059
> @bshashikant run with the latest update had 1 error:
>
> ```
> [ERROR] Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 644.647 s <<< FAILURE! - in org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures
> [ERROR] testFailureWithPrimeSizedData(org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures) Time elapsed: 55.401 s <<< ERROR!
> java.lang.NullPointerException
> at org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures.testFailureWithPrimeSizedData(TestBlockOutputStreamWithFailures.java:469)
> ```
>
> https://github.com/adoroszlai/hadoop-ozone/runs/473056970
@adoroszlai , can you please share log and output file for the failure?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bshashikant commented on issue #592: HDDS-3013. Fix
TestBlockOutputStreamWithFailures.java.
Posted by GitBox <gi...@apache.org>.
bshashikant commented on issue #592: HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
URL: https://github.com/apache/hadoop-ozone/pull/592#issuecomment-591848768
> Thanks @bshashikant for updating the patch. There are still some failures:
>
> ```
> [ERROR] testWatchForCommitDatanodeFailure(org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures) Time elapsed: 52.234 s <<< FAILURE!
> java.lang.AssertionError: expected:<800> but was:<450>
> ...
> at org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures.testWatchForCommitDatanodeFailure(TestBlockOutputStreamWithFailures.java:385)
> ```
>
> ```
> [ERROR] testWatchForCommitWithCloseContainerException(org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures) Time elapsed: 56.644 s <<< FAILURE!
> java.lang.AssertionError: expected:<80> but was:<84>
> ...
> at org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures.testWatchForCommitWithCloseContainerException(TestBlockOutputStreamWithFailures.java:175)
> ```
>
> ```
> [ERROR] testFailureWithPrimeSizedData(org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures) Time elapsed: 58.116 s <<< FAILURE!
> java.lang.AssertionError: expected:<2> but was:<3>
> ...
> at org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures.testFailureWithPrimeSizedData(TestBlockOutputStreamWithFailures.java:635)
> ```
>
> https://github.com/adoroszlai/hadoop-ozone/runs/468047506
Addressed failures.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on issue #592: HDDS-3013. Fix
TestBlockOutputStreamWithFailures.java.
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on issue #592: HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
URL: https://github.com/apache/hadoop-ozone/pull/592#issuecomment-590490211
@bshashikant there were various failures in 10 runs (at almost 10 minutes per run, this one cannot be run 20 times): https://github.com/adoroszlai/hadoop-ozone/runs/465142838
```
[ERROR] TestBlockOutputStreamWithFailures.test2DatanodesFailure:548 expected:<14> but was:<18>
[ERROR] TestBlockOutputStreamWithFailures.testWatchForCommitDatanodeFailure:390 expected:<800> but was:<450>
[ERROR] TestBlockOutputStreamWithFailures.testWatchForCommitWithCloseContainerException:273 expected:<94> but was:<92>
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai merged pull request #592: HDDS-3013. Fix
TestBlockOutputStreamWithFailures.java.
Posted by GitBox <gi...@apache.org>.
adoroszlai merged pull request #592: HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
URL: https://github.com/apache/hadoop-ozone/pull/592
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on issue #592: HDDS-3013. Fix
TestBlockOutputStreamWithFailures.java.
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on issue #592: HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
URL: https://github.com/apache/hadoop-ozone/pull/592#issuecomment-591281069
Thanks @bshashikant for updating the patch. There are still some failures:
```
[ERROR] testWatchForCommitDatanodeFailure(org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures) Time elapsed: 52.234 s <<< FAILURE!
java.lang.AssertionError: expected:<800> but was:<450>
...
at org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures.testWatchForCommitDatanodeFailure(TestBlockOutputStreamWithFailures.java:385)
```
```
[ERROR] testWatchForCommitWithCloseContainerException(org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures) Time elapsed: 56.644 s <<< FAILURE!
java.lang.AssertionError: expected:<80> but was:<84>
...
at org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures.testWatchForCommitWithCloseContainerException(TestBlockOutputStreamWithFailures.java:175)
```
```
[ERROR] testFailureWithPrimeSizedData(org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures) Time elapsed: 58.116 s <<< FAILURE!
java.lang.AssertionError: expected:<2> but was:<3>
...
at org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures.testFailureWithPrimeSizedData(TestBlockOutputStreamWithFailures.java:635)
```
https://github.com/adoroszlai/hadoop-ozone/runs/468047506
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bshashikant commented on issue #592: HDDS-3013. Fix
TestBlockOutputStreamWithFailures.java.
Posted by GitBox <gi...@apache.org>.
bshashikant commented on issue #592: HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
URL: https://github.com/apache/hadoop-ozone/pull/592#issuecomment-590967067
> @bshashikant there were various failures in 10 runs (at almost 10 minutes per run, this one cannot be run 20 times): https://github.com/adoroszlai/hadoop-ozone/runs/465142838
>
> ```
> [ERROR] TestBlockOutputStreamWithFailures.test2DatanodesFailure:548 expected:<14> but was:<18>
> [ERROR] TestBlockOutputStreamWithFailures.testWatchForCommitDatanodeFailure:390 expected:<800> but was:<450>
> [ERROR] TestBlockOutputStreamWithFailures.testWatchForCommitWithCloseContainerException:273 expected:<94> but was:<92>
> ```
Thanks @adoroszlai . The fix was not complete as metric count validation checks post inducing failure were not removed from few tests. Latest patch addresses these.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on issue #592: HDDS-3013. Fix
TestBlockOutputStreamWithFailures.java.
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on issue #592: HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
URL: https://github.com/apache/hadoop-ozone/pull/592#issuecomment-593777751
@bshashikant run with the latest update had 1 error:
```
[ERROR] Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 644.647 s <<< FAILURE! - in org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures
[ERROR] testFailureWithPrimeSizedData(org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures) Time elapsed: 55.401 s <<< ERROR!
java.lang.NullPointerException
at org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures.testFailureWithPrimeSizedData(TestBlockOutputStreamWithFailures.java:469)
```
https://github.com/adoroszlai/hadoop-ozone/runs/473056970
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on issue #592: HDDS-3013. Fix
TestBlockOutputStreamWithFailures.java.
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on issue #592: HDDS-3013. Fix TestBlockOutputStreamWithFailures.java.
URL: https://github.com/apache/hadoop-ozone/pull/592#issuecomment-591982649
Thanks @bshashikant. Yet another one:
```
[ERROR] test2DatanodesFailure(org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures) Time elapsed: 64.298 s <<< FAILURE!
java.lang.AssertionError: expected:<4> but was:<8>
...
at org.apache.hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures.test2DatanodesFailure(TestBlockOutputStreamWithFailures.java:406)
```
https://github.com/adoroszlai/hadoop-ozone/runs/472201293
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org