You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ratis.apache.org by GitBox <gi...@apache.org> on 2021/11/19 08:12:42 UTC
[GitHub] [ratis] codings-dan opened a new pull request #538: Add snapshot manually request and proto
codings-dan opened a new pull request #538:
URL: https://github.com/apache/ratis/pull/538
## What changes were proposed in this pull request?
subtask of Support snapshot command: Add snapshotManually request and proto
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/RATIS-1437
## How was this patch tested?
No need
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-993244561
@codings-dan , we may use the GroupManagementRequest as an example for the snapshot request:
- SnapshotRequest should be similar to GroupManagementRequest.
- add a takeSnapshotAsync(..) method in RaftServerProxy for taking a new snapshot; see RaftServerProxy.groupAddAsync(..).
- add new unit tests to call RaftServerProxy.takeSnapshotAsync(..) directly.
The items above are the works for server change.
In the next JIRA, we will add RaftServerProxy.snapshotManagementAsync(SnapshotRequest) and also SnapshotRequestProto. Then, we can change the unit tests to call from the client.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-1004796115
@szetszwo I change the code, added the code in SimulatedRpc and the raft.proto, PTAL, thx! In addition, the CI test error seems to have nothing to do with this code change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-1007238165
@szetszwo In the next pull request, I will add gRPC, Netty and Hadoop related proto, and then I will add the corresponding shell command and add the corresponding unit test, what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-993230200
@szetszwo hi, since the implementation code on the server side depends on SnapshotRequest and some proto-related code, can we review and merge the proto-related commit first, and then complete the server-side modification, otherwise there will be many errors
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-1002588306
BTW, let's change the SimulatedRpc first and then change gRPC, Netty and Hadoop RPC later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo merged pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo merged pull request #538:
URL: https://github.com/apache/ratis/pull/538
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan commented on a change in pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan commented on a change in pull request #538:
URL: https://github.com/apache/ratis/pull/538#discussion_r754775498
##########
File path: ratis-server/src/main/java/org/apache/ratis/server/impl/RaftServerImpl.java
##########
@@ -963,6 +963,15 @@ void finishTransferLeadership() {
}
}
+ public RaftClientReply snapshotManually(SnapshotManuallyRequest request) throws IOException {
+ return waitForReply(request, snapshotManuallyAsync(request));
+ }
+
+ public CompletableFuture<RaftClientReply> snapshotManuallyAsync(SnapshotManuallyRequest request) throws IOException {
+ //TODO(codings-dan) : implement snapshot manually in server
Review comment:
Ok, I will try to implement this feature on the server side first
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo commented on a change in pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo commented on a change in pull request #538:
URL: https://github.com/apache/ratis/pull/538#discussion_r778793683
##########
File path: ratis-common/src/main/java/org/apache/ratis/protocol/SnapshotManagementAsyncProtocol.java
##########
@@ -0,0 +1,26 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.protocol;
+
+import java.io.IOException;
+import java.util.concurrent.CompletableFuture;
+
+public interface SnapshotManagementAsyncProtocol {
Review comment:
Similar to groupManagementAsync, let's create a snapshotManagementAsync method in AdminAsynchronousProtocol instead adding a new protocol.
```
//AdminAsynchronousProtocol.java
CompletableFuture<RaftClientReply> snapshotManagementAsync(SnapshotManagementRequest request);
```
##########
File path: ratis-common/src/main/java/org/apache/ratis/protocol/SnapshotManagementProtocol.java
##########
@@ -0,0 +1,24 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.protocol;
+
+import java.io.IOException;
+
+public interface SnapshotManagementProtocol {
Review comment:
Let's add a snapshotManagement method in AdminProtocol.
```
//AdminProtocol.java
RaftClientReply snapshotManagement(SnapshotManagementRequest request) throws IOException;
```
##########
File path: ratis-client/src/main/java/org/apache/ratis/client/api/SnapshotManagementApi.java
##########
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.client.api;
+
+import org.apache.ratis.protocol.RaftClientReply;
+
+import java.io.IOException;
+
+/**
+ * An API to support control snapshot
+ * such as create and list snapshot file.
+ */
+public interface SnapshotManagementApi {
+
+ /** trigger create snapshot file. */
+ RaftClientReply createSnapshot(long timeoutMs) throws IOException;
Review comment:
Let's simply call it "create".
##########
File path: ratis-proto/src/main/proto/Raft.proto
##########
@@ -413,6 +413,11 @@ message TransferLeadershipRequestProto {
RaftPeerProto newLeader = 2;
}
+// snapshot request
+message SnapshotRequestProto {
Review comment:
Let's call it SnapshotManagementRequestProto. We should also rename SnapshotRequest to SnapshotManagementRequest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan edited a comment on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan edited a comment on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-1007177210
@szetszwo I have added snapshotmanagement related code to the admin related protocol, PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-1007322644
@codings-dan , sure. Let's add gRPC first. We will add Netty and Hadoop later. Thanks a lot!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-975039535
@szetszwo My idea is to first complete this pr, add proto and request. Then manually trigger the snapshot on the server side, and finally add a command. Can you help review the code, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo merged pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo merged pull request #538:
URL: https://github.com/apache/ratis/pull/538
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-1002587391
@codings-dan , yes, it is a good time to talk about the API. Let's add a SnapshotManagementApi. Currently, we only have the create(..) method. In the future, we may consider adding other operations such as
- remove
- list
- download
- upload
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-991059990
@codings-dan , thanks for the update. The change is quite big and hard to be reviewed. Could you create another JIRA for the server only change? Then, move the server code change to a new pull request. We should also add new tests (just local call to the server methods).
This JIRA and pull request can keep the proto and API changes. Then, modify the tests to use RPC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-1002556436
@szetszwo hi, now the server-side code has been implemented, and some unit tests have been added. Can we discuss proto and API change, it seems that it is not appropriate to put it under AdminApi, can we create a new API interface, such as snapshotManagementApi?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-1007322644
@codings-dan , sure. Let's add gRPC first. We will add Netty and Hadoop later. Thanks a lot!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-1007177210
@szetszwo I have added snapshotmanagement related code to the admin related protocol, please take a look, thank youI have added snapshotmanagement related code to the admin related protocol, PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan edited a comment on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan edited a comment on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-1007177210
@szetszwo I have added snapshotmanagement related code to the admin related protocol, PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] szetszwo commented on a change in pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
szetszwo commented on a change in pull request #538:
URL: https://github.com/apache/ratis/pull/538#discussion_r754120126
##########
File path: ratis-server/src/main/java/org/apache/ratis/server/impl/RaftServerImpl.java
##########
@@ -963,6 +963,15 @@ void finishTransferLeadership() {
}
}
+ public RaftClientReply snapshotManually(SnapshotManuallyRequest request) throws IOException {
+ return waitForReply(request, snapshotManuallyAsync(request));
+ }
+
+ public CompletableFuture<RaftClientReply> snapshotManuallyAsync(SnapshotManuallyRequest request) throws IOException {
+ //TODO(codings-dan) : implement snapshot manually in server
Review comment:
@codings-dan , we should implement the feature (i.e. this TODO) in the server first and then change the API (this pull request).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-991101596
will do
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ratis] codings-dan commented on pull request #538: RATIS-1441.Add snapshot manually request and proto
Posted by GitBox <gi...@apache.org>.
codings-dan commented on pull request #538:
URL: https://github.com/apache/ratis/pull/538#issuecomment-1007177210
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ratis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org