You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/06/23 05:01:40 UTC
[GitHub] [ozone] captainzmc opened a new pull request #2358: HDDS-5366. Implement stream method to ContainerStateMachine.
captainzmc opened a new pull request #2358:
URL: https://github.com/apache/ozone/pull/2358
## What changes were proposed in this pull request?
Use RATIS-Streaming to make ContainerStateLabine support writing streaming to the local.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5366
## How was this patch tested?
CI test will be added after the client is finished
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] szetszwo merged pull request #2358: HDDS-5366. [Ozone-Streaming] Implement stream method to ContainerStateMachine.
Posted by GitBox <gi...@apache.org>.
szetszwo merged pull request #2358:
URL: https://github.com/apache/ozone/pull/2358
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc commented on pull request #2358: HDDS-5366. [Ozone-Streaming] Implement stream method to ContainerStateMachine.
Posted by GitBox <gi...@apache.org>.
captainzmc commented on pull request #2358:
URL: https://github.com/apache/ozone/pull/2358#issuecomment-866961590
Thanks @szetszwo for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] szetszwo commented on a change in pull request #2358: HDDS-5366. [Ozone-Streaming] Implement stream method to ContainerStateMachine.
Posted by GitBox <gi...@apache.org>.
szetszwo commented on a change in pull request #2358:
URL: https://github.com/apache/ozone/pull/2358#discussion_r656905202
##########
File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java
##########
@@ -501,6 +503,30 @@ private ExecutorService getCommandExecutor(
return raftFuture;
}
+ @Override
+ public CompletableFuture<DataStream> stream(RaftClientRequest request) {
+ try {
+ ContainerCommandRequestProto requestProto =
+ getContainerCommandRequestProto(gid,
+ request.getMessage().getContent());
+ DispatcherContext context =
+ new DispatcherContext.Builder()
+ .setStage(DispatcherContext.WriteChunkStage.WRITE_DATA)
+ .setContainer2BCSIDMap(container2BCSIDMap)
+ .build();
+
+ ContainerCommandResponseProto response = runCommand(
+ requestProto, context);
+ String path = response.getMessage();
+ CompletableFuture<DataStream> dataStream = new CompletableFuture<>();
+ dataStream.complete(new LocalStream(new StreamDataChannel(
+ Paths.get(path))));
+ return dataStream;
+ } catch (IOException e) {
+ throw new CompletionException("Failed to create data stream", e);
+ }
Review comment:
We should make it async like below:
```
public CompletableFuture<DataStream> stream(RaftClientRequest request) {
return CompletableFuture.supplyAsync(() -> {
try {
ContainerCommandRequestProto requestProto =
getContainerCommandRequestProto(gid,
request.getMessage().getContent());
DispatcherContext context =
new DispatcherContext.Builder()
.setStage(DispatcherContext.WriteChunkStage.WRITE_DATA)
.setContainer2BCSIDMap(container2BCSIDMap)
.build();
ContainerCommandResponseProto response = runCommand(
requestProto, context);
String path = response.getMessage();
return new LocalStream(new StreamDataChannel(Paths.get(path))));
} catch (IOException e) {
throw new CompletionException("Failed to create data stream", e);
}
}, executor);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc closed pull request #2358: HDDS-5366. [Ozone-Streaming] Implement stream method to ContainerStateMachine.
Posted by GitBox <gi...@apache.org>.
captainzmc closed pull request #2358:
URL: https://github.com/apache/ozone/pull/2358
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org