You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ratis.apache.org by "Hadoop QA (Jira)" <ji...@apache.org> on 2019/12/16 13:52:00 UTC
[jira] [Commented] (RATIS-776) Handle ResourceUnavailabeException
properly in Ratis Server
[ https://issues.apache.org/jira/browse/RATIS-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997309#comment-16997309 ]
Hadoop QA commented on RATIS-776:
---------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 5s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 0s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 18s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 55s{color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.logservice.server.TestMetaServer |
| | ratis.grpc.TestRaftAsyncWithGrpc |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/ratis:date2019-12-16 |
| JIRA Issue | RATIS-776 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12988928/RATIS-776.001.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs checkstyle compile |
| uname | Linux 900ef204a690 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh |
| git revision | master / 64c53ee |
| maven | version: Apache Maven 3.6.2 (40f52333136460af0dc0d7232c0dc0bcf0d9e117; 2019-08-27T15:06:16Z) |
| Default Java | 1.8.0_222 |
| unit | https://builds.apache.org/job/PreCommit-RATIS-Build/1187/artifact/out/patch-unit-root.txt |
| Test Results | https://builds.apache.org/job/PreCommit-RATIS-Build/1187/testReport/ |
| Max. process+thread count | 1641 (vs. ulimit of 5000) |
| modules | C: ratis-server ratis-grpc U: . |
| Console output | https://builds.apache.org/job/PreCommit-RATIS-Build/1187/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Handle ResourceUnavailabeException properly in Ratis Server
> -----------------------------------------------------------
>
> Key: RATIS-776
> URL: https://issues.apache.org/jira/browse/RATIS-776
> Project: Ratis
> Issue Type: Bug
> Components: server
> Reporter: Lokesh Jain
> Assignee: Lokesh Jain
> Priority: Major
> Attachments: RATIS-776.001.patch
>
>
> Ratis leader while processing a client request tries to create a pending request. If it is not able to do so it fails the request with ResourceUnavailableException. But the server keeps processing the other requests from the same client. The resources can be released when the other client requests are processed, resulting in out of order processing of client requests. On failure the server should ideally fail all the client requests which need to be processed.
> {code:java}
> 2019-12-10 19:50:31,846 [grpc-default-executor-5] INFO ratis.ContainerStateMachine (ContainerStateMachine.java:preAppendTransaction(311)) - append seqNum:2 WriteChunk
> 2019-12-10 19:50:31,860 [grpc-default-executor-5] INFO ratis.ContainerStateMachine (ContainerStateMachine.java:preAppendTransaction(311)) - append seqNum:3 WriteChunk
> Caused by: org.apache.ratis.protocol.exceptions.ResourceUnavailableException: 164293f2-68e3-4851-bc46-4a828bd79ffa@group-03010B1A5718: Failed to acquire a pending write request for RaftClientRequest:client-38E7254A5AF1->164293f2-68e3-4851-bc46-4a828bd79ffa@group-03010B1A5718, cid=3, seq=4, RW, Message:000000b2080612343362...(size=182)Caused by: org.apache.ratis.protocol.exceptions.ResourceUnavailableException: 164293f2-68e3-4851-bc46-4a828bd79ffa@group-03010B1A5718: Failed to acquire a pending write request for RaftClientRequest:client-38E7254A5AF1->164293f2-68e3-4851-bc46-4a828bd79ffa@group-03010B1A5718, cid=3, seq=4, RW, Message:000000b2080612343362...(size=182) at org.apache.ratis.server.impl.RaftServerImpl.appendTransaction(RaftServerImpl.java:514) at org.apache.ratis.server.impl.RaftServerImpl.submitClientRequestAsync(RaftServerImpl.java:589) at org.apache.ratis.server.impl.RaftServerProxy.lambda$submitClientRequestAsync$7(RaftServerProxy.java:333) at org.apache.ratis.server.impl.RaftServerProxy.lambda$null$5(RaftServerProxy.java:328) at org.apache.ratis.util.JavaUtils.callAsUnchecked(JavaUtils.java:109) at org.apache.ratis.server.impl.RaftServerProxy.lambda$submitRequest$6(RaftServerProxy.java:328) at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:981) at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2124) at org.apache.ratis.server.impl.RaftServerProxy.submitRequest(RaftServerProxy.java:327) at org.apache.ratis.server.impl.RaftServerProxy.submitClientRequestAsync(RaftServerProxy.java:333) at org.apache.ratis.grpc.client.GrpcClientProtocolService$RequestStreamObserver.processClientRequest(GrpcClientProtocolService.java:221) at org.apache.ratis.grpc.client.GrpcClientProtocolService$OrderedRequestStreamObserver.processClientRequest(GrpcClientProtocolService.java:327) at org.apache.ratis.util.SlidingWindow$Server.processRequestsFromHead(SlidingWindow.java:429) at org.apache.ratis.util.SlidingWindow$Server.receivedRequest(SlidingWindow.java:421) at org.apache.ratis.grpc.client.GrpcClientProtocolService$OrderedRequestStreamObserver.processClientRequest(GrpcClientProtocolService.java:346) at org.apache.ratis.grpc.client.GrpcClientProtocolService$RequestStreamObserver.onNext(GrpcClientProtocolService.java:241) at org.apache.ratis.grpc.client.GrpcClientProtocolService$RequestStreamObserver.onNext(GrpcClientProtocolService.java:168) at org.apache.ratis.thirdparty.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:251) at org.apache.ratis.thirdparty.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(ServerCallImpl.java:309) at org.apache.ratis.thirdparty.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:292) at org.apache.ratis.thirdparty.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:779) ... 5 more
> 2019-12-10 19:50:31,860 [grpc-default-executor-5] INFO ratis.ContainerStateMachine (ContainerStateMachine.java:preAppendTransaction(311)) - append seqNum:5 WriteChunk
> {code}
> Further while failing the client request server does not invalidate the retry cache. Any retries from the client are therefore ignored in the server.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)