You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "mridulm (via GitHub)" <gi...@apache.org> on 2024/04/16 06:54:40 UTC
[PR] [CELEBORN-1356] Split rpc module [celeborn]
mridulm opened a new pull request, #2459:
URL: https://github.com/apache/celeborn/pull/2459
### What changes were proposed in this pull request?
Split the `rpc` transport module into `rpc_app` and `rpc_server` to allow for them to be independently configured.
### Why are the changes needed?
We need the ability to independently configure communication between application components (driver/executors in spark applications) and those to/from Celeborn server (master/workers) components.
This is particularly relevant for TLS support where applications might be running with TLS disabled for their rpc services or using self-signed certificates (see CELEBORN-1354 for an example), while servers would have signed certs.
### Does this PR introduce _any_ user-facing change?
Yes, it allows users to independently configure rpc env within the application and those to/from servers.
Backward compatibility is maintained - and so existing `rpc` is the fallback in case `rpc_app` or `rpc_server` config is not found.
### How was this patch tested?
Unit tests were enhanced, existing tests pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CELEBORN-1356] Split rpc module [celeborn]
Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm closed pull request #2459: [CELEBORN-1356] Split rpc module
URL: https://github.com/apache/celeborn/pull/2459
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CELEBORN-1356] Split rpc module [celeborn]
Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on PR #2459:
URL: https://github.com/apache/celeborn/pull/2459#issuecomment-2058369799
If it makes it easier to review, I can split it into two - updates to `CelebornConf`, which is mostly config updates; and wiring up the new transport module to RpcEnv and usages.
Please let me know if that helps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CELEBORN-1356] Split rpc module [celeborn]
Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on PR #2459:
URL: https://github.com/apache/celeborn/pull/2459#issuecomment-2058390352
Let me close and recreate this PR ... looks messed up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org