You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ratis.apache.org by GitBox <gi...@apache.org> on 2020/10/11 14:33:50 UTC
[GitHub] [incubator-ratis] szetszwo opened a new pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
szetszwo opened a new pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215
See https://issues.apache.org/jira/browse/RATIS-1089
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] runzhiwang commented on pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
runzhiwang commented on pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#issuecomment-706787743
LGTM, just wait the fix of checkstyle.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] amaliujia commented on pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#issuecomment-706770710
LGTM overall. Thanks for adding detailed javadoc to interfaces.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] szetszwo commented on pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
szetszwo commented on pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#issuecomment-706912790
Thanks @runzhiwang , just have removed the unused import.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] szetszwo commented on pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
szetszwo commented on pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#issuecomment-707035593
@runzhiwang and @amaliujia , thanks a lot for reviewing this.
I suggest to refactor the other RaftClient APIs in a similar way. Filed https://issues.apache.org/jira/browse/RATIS-1091 .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] runzhiwang commented on a change in pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
runzhiwang commented on a change in pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#discussion_r502984470
##########
File path: ratis-client/src/main/java/org/apache/ratis/client/api/DataStreamApi.java
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.client.api;
+
+/**
+ * Stream data asynchronously to all the servers in the {@link org.apache.ratis.protocol.RaftGroup}.
+ * Clients may stream data to the nearest server and then the server will forward the data to the other servers.
+ * Once all the servers have received all the data of a request,
+ * the leader (may or may not be the nearest server) creates a log entry with a data ID generated by the state machine.
+ * Then, the leader sends the log entry to the followers.
+ * Since the followers already have received the data from the stream,
+ * they may lookup the data from the ID.
Review comment:
Because follower needs to look up the data from the ID, so when follower receives the data from the stream, it needs to store a map<ID, the data from the stream>, so this ID must be transfered to all servers in the first request of stream.
Can this ID generated by clientId_streamId ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] runzhiwang removed a comment on pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
runzhiwang removed a comment on pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#issuecomment-706787743
LGTM, just wait the fix of checkstyle.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] runzhiwang commented on pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
runzhiwang commented on pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#issuecomment-706973392
@szetszwo Thanks the patch. @amaliujia Thanks for review. I have merged the patch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] runzhiwang commented on a change in pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
runzhiwang commented on a change in pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#discussion_r503006741
##########
File path: ratis-client/src/main/java/org/apache/ratis/client/api/DataStreamApi.java
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.client.api;
+
+/**
+ * Stream data asynchronously to all the servers in the {@link org.apache.ratis.protocol.RaftGroup}.
+ * Clients may stream data to the nearest server and then the server will forward the data to the other servers.
+ * Once all the servers have received all the data of a request,
+ * the leader (may or may not be the nearest server) creates a log entry with a data ID generated by the state machine.
+ * Then, the leader sends the log entry to the followers.
+ * Since the followers already have received the data from the stream,
+ * they may lookup the data from the ID.
Review comment:
Thanks for explanation, Got it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] runzhiwang merged pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
runzhiwang merged pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] szetszwo commented on a change in pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
szetszwo commented on a change in pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#discussion_r502994540
##########
File path: ratis-client/src/main/java/org/apache/ratis/client/api/DataStreamApi.java
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.client.api;
+
+/**
+ * Stream data asynchronously to all the servers in the {@link org.apache.ratis.protocol.RaftGroup}.
+ * Clients may stream data to the nearest server and then the server will forward the data to the other servers.
+ * Once all the servers have received all the data of a request,
+ * the leader (may or may not be the nearest server) creates a log entry with a data ID generated by the state machine.
+ * Then, the leader sends the log entry to the followers.
+ * Since the followers already have received the data from the stream,
+ * they may lookup the data from the ID.
Review comment:
It depends on the StateMachine implementations. They may use clientId_streamId as the data ID and choose to generate the ID in some other way. For example, in Ozone, it may just use the Block/Chunk ID as the data ID.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] szetszwo commented on a change in pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
szetszwo commented on a change in pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#discussion_r502994540
##########
File path: ratis-client/src/main/java/org/apache/ratis/client/api/DataStreamApi.java
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.client.api;
+
+/**
+ * Stream data asynchronously to all the servers in the {@link org.apache.ratis.protocol.RaftGroup}.
+ * Clients may stream data to the nearest server and then the server will forward the data to the other servers.
+ * Once all the servers have received all the data of a request,
+ * the leader (may or may not be the nearest server) creates a log entry with a data ID generated by the state machine.
+ * Then, the leader sends the log entry to the followers.
+ * Since the followers already have received the data from the stream,
+ * they may lookup the data from the ID.
Review comment:
It depends on the StateMachine implementations. They may use clientId_streamId as the data ID or choose to generate the ID in some other way. For example, in Ozone, it may just use the Block/Chunk ID as the data ID.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] szetszwo commented on a change in pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
szetszwo commented on a change in pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#discussion_r502971645
##########
File path: ratis-client/src/main/java/org/apache/ratis/client/api/DataStreamApi.java
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.client.api;
+
+/**
+ * Stream data asynchronously to all the servers in the {@link org.apache.ratis.protocol.RaftGroup}.
+ * Clients may stream data to the nearest server and then the server will forward the data to the other servers.
+ * Once all the servers have received all the data of a request,
+ * the leader (may or may not be the nearest server) creates a log entry with the metadata but not the data.
+ * Then, the leader sends the metadata as a log entry to the followers.
+ * Since the followers already have received the data from the stream,
+ * they may lookup the data from the metadata.
Review comment:
The metadata is a data ID generated by the state machine so that the follower can lookup the data using the ID. Let me revise the javadoc.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-ratis] amaliujia commented on a change in pull request #215: RATIS-1089. Add getDataStreamApi() to RaftClient.
Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #215:
URL: https://github.com/apache/incubator-ratis/pull/215#discussion_r502967830
##########
File path: ratis-client/src/main/java/org/apache/ratis/client/api/DataStreamApi.java
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.client.api;
+
+/**
+ * Stream data asynchronously to all the servers in the {@link org.apache.ratis.protocol.RaftGroup}.
+ * Clients may stream data to the nearest server and then the server will forward the data to the other servers.
+ * Once all the servers have received all the data of a request,
+ * the leader (may or may not be the nearest server) creates a log entry with the metadata but not the data.
+ * Then, the leader sends the metadata as a log entry to the followers.
+ * Since the followers already have received the data from the stream,
+ * they may lookup the data from the metadata.
Review comment:
`they may lookup the data from the metadata`? Can you clarify this? Why look up data from metadata?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org