You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ratis.apache.org by GitBox <gi...@apache.org> on 2022/01/11 10:32:55 UTC
[GitHub] [ratis] codings-dan opened a new pull request #576: RATIS-1485. Add take snapshot hadoop and netty related request and proto
codings-dan opened a new pull request #576:
URL: https://github.com/apache/ratis/pull/576
## What changes were proposed in this pull request?
subtask of Support snapshot command: Add take snapshot hadoop and netty related request and proto
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/RATIS-1486
## How was this patch tested?
No need
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo merged pull request #576: RATIS-1486. Add take snapshot hadoop and netty related request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo merged pull request #576:
URL: https://github.com/apache/ratis/pull/576
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan commented on pull request #576: RATIS-1485. Add take snapshot hadoop and netty related request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan commented on pull request #576:
URL: https://github.com/apache/ratis/pull/576#issuecomment-1009843579
@szetszwo I added the code for take snapshot hadoop netty proto and request, PTAL, after this patch, I wil add a shell command to manage snapshot. BTW, the ci test error has nothing to do with this code change.
```
org.junit.runners.model.TestTimedOutException: test timed out after 100 seconds
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:426)
at java.util.concurrent.FutureTask.get(FutureTask.java:204)
at org.junit.internal.runners.statements.FailOnTimeout.getResult(FailOnTimeout.java:141)
at org.junit.internal.runners.statements.FailOnTimeout.evaluate(FailOnTimeout.java:127)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan commented on pull request #576: RATIS-1485. Add take snapshot hadoop and netty related request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan commented on pull request #576:
URL: https://github.com/apache/ratis/pull/576#issuecomment-1010828193
@szetszwo Thanks for the review code, now we need to change RaftClient and grpc-related test code, which is not related to the content of this PR. Do I complete the above work in this PR, or create another PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo commented on pull request #576: RATIS-1486. Add take snapshot hadoop and netty related request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo commented on pull request #576:
URL: https://github.com/apache/ratis/pull/576#issuecomment-1010902942
@codings-dan , please create a subtask for the RaftClient change. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo commented on pull request #576: RATIS-1485. Add take snapshot hadoop and netty related request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo commented on pull request #576:
URL: https://github.com/apache/ratis/pull/576#issuecomment-1010791103
@codings-dan , just found that we have not changed the RaftClient. Let's change it first and also add a test for Grpc, i.e. TestSnapshotManagementWithGpc.
```
diff --git a/ratis-client/src/main/java/org/apache/ratis/client/RaftClient.java b/ratis-client/src/main/java/org/apache/ratis/client/RaftClient.java
index e7bd3bf1..8016a674 100644
--- a/ratis-client/src/main/java/org/apache/ratis/client/RaftClient.java
+++ b/ratis-client/src/main/java/org/apache/ratis/client/RaftClient.java
@@ -24,6 +24,7 @@ import org.apache.ratis.client.api.BlockingApi;
import org.apache.ratis.client.api.DataStreamApi;
import org.apache.ratis.client.api.GroupManagementApi;
import org.apache.ratis.client.api.MessageStreamApi;
+import org.apache.ratis.client.api.SnapshotManagementApi;
import org.apache.ratis.client.impl.ClientImplUtils;
import org.apache.ratis.conf.Parameters;
import org.apache.ratis.conf.RaftProperties;
@@ -57,6 +58,9 @@ public interface RaftClient extends Closeable {
/** Get the {@link GroupManagementApi} for the given server. */
GroupManagementApi getGroupManagementApi(RaftPeerId server);
+ /** Get the {@link SnapshotManagementApi}. */
+ SnapshotManagementApi getSnapshotManagementApi();
+
/** @return the {@link BlockingApi}. */
BlockingApi io();
diff --git a/ratis-client/src/main/java/org/apache/ratis/client/impl/RaftClientImpl.java b/ratis-client/src/main/java/org/apache/ratis/client/impl/RaftClientImpl.java
index ce7150b3..eae0606a 100644
--- a/ratis-client/src/main/java/org/apache/ratis/client/impl/RaftClientImpl.java
+++ b/ratis-client/src/main/java/org/apache/ratis/client/impl/RaftClientImpl.java
@@ -21,6 +21,7 @@ import org.apache.ratis.client.DataStreamClient;
import org.apache.ratis.client.RaftClient;
import org.apache.ratis.client.RaftClientRpc;
import org.apache.ratis.client.api.DataStreamApi;
+import org.apache.ratis.client.api.SnapshotManagementApi;
import org.apache.ratis.client.retry.ClientRetryEvent;
import org.apache.ratis.conf.RaftProperties;
import org.apache.ratis.proto.RaftProtos.SlidingWindowEntry;
@@ -140,6 +141,7 @@ public final class RaftClientImpl implements RaftClient {
private final Supplier<AdminImpl> adminApi;
private final ConcurrentMap<RaftPeerId, GroupManagementImpl> groupManagmenets = new ConcurrentHashMap<>();
+ private final Supplier<SnapshotManagementApi> snapshotManagemenet;
RaftClientImpl(ClientId clientId, RaftGroup group, RaftPeerId leaderId, RaftPeer primaryDataStreamServer,
RaftClientRpc clientRpc, RaftProperties properties, RetryPolicy retryPolicy) {
@@ -170,6 +172,7 @@ public final class RaftClientImpl implements RaftClient {
.setProperties(properties)
.build());
this.adminApi = JavaUtils.memoize(() -> new AdminImpl(this));
+ this.snapshotManagemenet= JavaUtils.memoize(() -> new SnapshotManagementImpl(this));
}
public RaftPeerId getLeaderId() {
@@ -243,6 +246,11 @@ public final class RaftClientImpl implements RaftClient {
return groupManagmenets.computeIfAbsent(server, id -> new GroupManagementImpl(id, this));
}
+ @Override
+ public SnapshotManagementApi getSnapshotManagementApi() {
+ return snapshotManagemenet.get();
+ }
+
@Override
public BlockingImpl io() {
return blockingApi.get();
diff --git a/ratis-server/src/test/java/org/apache/ratis/statemachine/SnapshotManagementTest.java b/ratis-server/src/test/java/org/apache/ratis/statemachine/SnapshotManagementTest.java
index 385b77ec..697aa748 100644
--- a/ratis-server/src/test/java/org/apache/ratis/statemachine/SnapshotManagementTest.java
+++ b/ratis-server/src/test/java/org/apache/ratis/statemachine/SnapshotManagementTest.java
@@ -100,10 +100,7 @@ public abstract class SnapshotManagementTest<CLUSTER extends MiniRaftCluster>
}
Assert.assertTrue(leader.getStateMachine().getLastAppliedTermIndex().getIndex()
< RaftServerConfigKeys.Snapshot.creationGap(getProperties()));
- final SnapshotManagementRequest r =
- SnapshotManagementRequest.newCreate(client.getId(), leaderId, cluster.getGroupId(),
- CallId.getAndIncrement(), 3000);
- snapshotReply = RaftServerTestUtil.takeSnapshotAsync(leader, r).join();
+ snapshotReply = client.getSnapshotManagementApi().create(3000);
Assert.assertTrue(snapshotReply.isSuccess());
Assert.assertEquals(0,snapshotReply.getLogIndex());
for (int i = 0; i < RaftServerConfigKeys.Snapshot.creationGap(getProperties())/2-1; i++) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan edited a comment on pull request #576: RATIS-1485. Add take snapshot hadoop and netty related request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan edited a comment on pull request #576:
URL: https://github.com/apache/ratis/pull/576#issuecomment-1010828193
@szetszwo Thanks for reviewing code, now we need to change RaftClient and grpc-related test code, which is not related to the content of this PR. Do I complete the above work in this PR, or create another PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org