You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/05/18 07:31:38 UTC
[GitHub] [ozone] neils-dev opened a new pull request #2259: HDDS-4440- s3 performance
neils-dev opened a new pull request #2259:
URL: https://github.com/apache/ozone/pull/2259
## What changes were proposed in this pull request?
Initial commit for s3g gRPC for command OmRequest and OmResponses. Creates:
1. a gRPC service for existing Om protocol - _protoc 2.5_ `hadoop-ozone/interface-client/pom.xml`, `hadoop-ozone/interface-client/target/generated-sources/protobuf/java/org/apache/hadoop/ozone/protocol/proto/OzoneManagerServiceGrpc.java` - _**HDDS-5210**_
2. starts OM gRPC server as part of OM bootstrap - `hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/GrpcOzoneManagerServer.java`, `OzoneManager.java` - _**HDDS-5211**_
3. creates implementation of OmTransport for gRPC - ` hadoop-ozone/interface-client/pom.xml` - _**HDDS-5212**_
4. Create specific OmTransportFactory for GrpcOmTransport - `hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/GrpcOmTransportFactory.java`; dynamically selected ServiceProvider loaded class for s3g `hadoop-ozone/s3gateway/src/main/resources/META-INF/services/org.apache.hadoop.ozone.om.protocolPB.OmTransportFactory` - _**HDDS-5213**_
Currently requires ratis _**disabled**_ for Om : set `ozone.om.ratis.enable=false` in _ozone-site.xml_; use _intellij_ `hadoop-ozone/dev-support/intellij/ozone-site.xml` and _intellij s3g_ together with invoke _s3g_ for simple bucket commands through _s3 cli_.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5210 - (see 1. previous section)
https://issues.apache.org/jira/browse/HDDS-5211 - (see 2. previous section)
https://issues.apache.org/jira/browse/HDDS-5212 - (see 3. previous section)
https://issues.apache.org/jira/browse/HDDS-5213 - (see 4. previous section)
## How was this patch tested?
Patch is initial POC for s3g gRPC OmRequest/OmResponse. Currently requires _**ratis disabled**_ for Om - (when invoking ratis server through _OzoneManagerRatisServer_ _submitRequest_, `createWriteRaftClientRequest(omRequest)` fails when checking preconditions due to _hadoop class Server.Call_ is _**null**_; requires attention from propagating the om _SubmitRequest_ through the _GrpcServer_).
To test - use intellij ozone development cluster running through runConfigurations for invoking SCM, OM, Recon, Datanodes and S3Gateway. Commit includes modified ozone-site.xml (`hadoop-ozone/dev-support/intellij/ozone-site.xml`) to disable ratis for om (`<property><name>ozone.om.ratis.enable</name><value>false</value></property>`). Run simple s3 file operations through aws cli:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai closed pull request #2259: HDDS-4440- s3 performance
Posted by GitBox <gi...@apache.org>.
adoroszlai closed pull request #2259:
URL: https://github.com/apache/ozone/pull/2259
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] elek commented on pull request #2259: HDDS-4440- s3 performance
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #2259:
URL: https://github.com/apache/ozone/pull/2259#issuecomment-847936452
Thanks the patch @neils-dev. Looks it has a lot of unrelated change. Is it possible that master is merged to your branch but not to the target branch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #2259: HDDS-4440- s3 performance
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #2259:
URL: https://github.com/apache/ozone/pull/2259#issuecomment-889149998
@neils-dev Given that
* HDDS-4440 is split into sub-tasks
* we had a separate PR for the first sub-task
* which had improvement over this one (as CSI test is failing here and was OK there)
* the separate PR just got merged
* we had a merge conflict even before that
I think we should close this mega PR and open a new one for the second sub-task.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org