You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ozone.apache.org by el...@apache.org on 2020/12/07 15:01:43 UTC
[ozone] branch HDDS-4440 created (now af94640)
This is an automated email from the ASF dual-hosted git repository.
elek pushed a change to branch HDDS-4440
in repository https://gitbox.apache.org/repos/asf/ozone.git.
at af94640 diagram about the old a new approach
This branch includes the following new commits:
new af94640 diagram about the old a new approach
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@ozone.apache.org
For additional commands, e-mail: commits-help@ozone.apache.org
[ozone] 01/01: diagram about the old a new approach
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-4440
in repository https://gitbox.apache.org/repos/asf/ozone.git
commit af946408c560e1ccbb5a1e6711775eb7e140113b
Author: Elek Márton <el...@apache.org>
AuthorDate: Mon Dec 7 16:01:28 2020 +0100
diagram about the old a new approach
---
hadoop-hdds/docs/content/design/s3-performance.md | 33 ++++++++++++++++++++-
.../docs/static/design/s3-performance-new.png | Bin 0 -> 33688 bytes
.../docs/static/design/s3-performance-old.png | Bin 0 -> 34927 bytes
3 files changed, 32 insertions(+), 1 deletion(-)
diff --git a/hadoop-hdds/docs/content/design/s3-performance.md b/hadoop-hdds/docs/content/design/s3-performance.md
index 8687563..7ce5683 100644
--- a/hadoop-hdds/docs/content/design/s3-performance.md
+++ b/hadoop-hdds/docs/content/design/s3-performance.md
@@ -184,7 +184,7 @@ public interface OmTransport {
...
```
- The proposal is to create a new **additional** transport, based on GRPC, which can do the per-request authentication. **Existing Hadoop clients will use the well-known Hadoop RPC client**, but S3g can start to use this specific transport to achieve better performance.
+The proposal is to create a new **additional** transport, based on GRPC, which can do the per-request authentication. **Existing Hadoop clients will use the well-known Hadoop RPC client**, but S3g can start to use this specific transport to achieve better performance.
As this is nothing more, just a transport: exactly the same messages (`OmRequest`) will be used, it's not a new RPC interface.
@@ -196,6 +196,37 @@ A new GRPC service should be started in Ozone Manager, which receives `OMRequest
With this approach the `OzoneClient` instances can be cached on S3g side (with persistent GRPC connections) as the authentication information is not part of the OzoneClient any more (added by the `OmTransport` implementation per request (in case of GRPC) or per connection (in case of HadoopRPC)).
+To make it easier to understand the implementation of this approach, let's compare the old (existing) and new (proposed) approach.
+
+### Old approach
+
+1. OzoneClientProvider creates a new OzoneClient for each of the HTTP requests (@RequestScope)
+
+2. OzoneToken is created based on the authentication header (signature, string to sign)
+
+3. OzoneClient creates new OM connection (Hadoop RPC) with the new OzoneToken
+
+4. OM extracts the information from OzoneToken and validates the signature
+
+5. OzoneClient is injected to the REST endpoints
+
+6. OzoneClient is used for any client calls
+
+7. When HTTP request is handled, the OzoneClient is closed
+
+ 
+
+### New approach
+
+1. OzoneClientProvider creates client with @ApplicationScope. Connection is always open to the OM (clients can be pooled)
+2. OM doesn't authentication the connection, each of the requests should be authenticated one-by-one
+3. OzoneClients are always injected to the REST endpoints
+4. For each new HTTP request the authorization header is extracted and added to the outgoing GRPC request as a token
+5. OM authenticate each of the request and calls the same request handler as before
+6. OzoneClient can be open long-term
+
+
+
# Possible alternatives
* It's possible to use pure Hadoop RPC client instead of Ozone Client which would make the client connection slightly cheaper (service discovery call is not required) but it's still require to create new connections for each requests (and downloading data without OzoneClient may have own challenges).
diff --git a/hadoop-hdds/docs/static/design/s3-performance-new.png b/hadoop-hdds/docs/static/design/s3-performance-new.png
new file mode 100644
index 0000000..f2cb405
Binary files /dev/null and b/hadoop-hdds/docs/static/design/s3-performance-new.png differ
diff --git a/hadoop-hdds/docs/static/design/s3-performance-old.png b/hadoop-hdds/docs/static/design/s3-performance-old.png
new file mode 100644
index 0000000..20ebfc2
Binary files /dev/null and b/hadoop-hdds/docs/static/design/s3-performance-old.png differ
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@ozone.apache.org
For additional commands, e-mail: commits-help@ozone.apache.org