You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "milenkovicm (via GitHub)" <gi...@apache.org> on 2026/01/06 10:09:00 UTC
[I] Provide support for SSL/TSL connections [datafusion-ballista]
milenkovicm opened a new issue, #1367:
URL: https://github.com/apache/datafusion-ballista/issues/1367
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Support optional TLS/SSL connections between clients schedulers and executors. Currently plain HTTP is used.
**Describe the solution you'd like**
I’m not an expert in this area but sensible security would make sense. Plain HTTP should remain the default. Good documentation should also cover security configuration.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Provide support for SSL/TSL connections [datafusion-ballista]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on issue #1367:
URL: https://github.com/apache/datafusion-ballista/issues/1367#issuecomment-3716605485
Open for any suggestion.
Also, if we can't find generic solution maybe just providing Chanel and Server factories will be sufficient
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Provide support for SSL/TSL connections [datafusion-ballista]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on issue #1367:
URL: https://github.com/apache/datafusion-ballista/issues/1367#issuecomment-3724512360
hey @mohit7705 please do not wait for approval, just do what you think its right, we can discuss when there is a concrete proposal in place
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Provide support for SSL/TSL connections [datafusion-ballista]
Posted by "mohit7705 (via GitHub)" <gi...@apache.org>.
mohit7705 commented on issue #1367:
URL: https://github.com/apache/datafusion-ballista/issues/1367#issuecomment-3720573118
## Design Proposal: Optional TLS via Channel and Server Factories
### Problem
Currently, communication between clients, schedulers, and executors uses plain HTTP, which does not provide transport-level encryption.
### Goal
Enable optional TLS support while keeping the default behavior unchanged (plain HTTP).
### Non-goals
- Enforcing TLS by default
- Implementing a full security or certificate management framework
### Proposed Solution
Introduce configurable abstractions for:
- `ChannelFactory` – responsible for creating outbound client channels
- `ServerFactory` – responsible for creating inbound server listeners
The existing HTTP-based implementation will remain the default.
Users who require TLS can provide custom implementations (for example, using `rustls`) without modifying core logic.
### Configuration
Factories can be selected via existing configuration mechanisms (e.g., config files or environment variables), allowing users to opt into TLS explicitly.
### Backward Compatibility
This approach is fully backward-compatible. Existing deployments will continue to work without any changes.
### Next Steps
After feedback and approval:
1. Identify instantiation points for channels and servers
2. Introduce factory interfaces
3. Wire configuration to select the desired factory implementation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Provide support for SSL/TSL connections [datafusion-ballista]
Posted by "mohit7705 (via GitHub)" <gi...@apache.org>.
mohit7705 commented on issue #1367:
URL: https://github.com/apache/datafusion-ballista/issues/1367#issuecomment-3718919192
Thanks for the clarification!
That makes sense — starting with a minimal, non-generic solution sounds reasonable.
I propose beginning with:
- Exposing configurable ChannelFactory and ServerFactory abstractions
- Keeping the default implementation as plain HTTP
- Allowing users to plug in TLS-enabled factories (for example, using rustls) without changing core logic
My next step would be to:
- Identify where channels and servers are currently instantiated
- Draft a small design note showing how factories could be injected via configuration
I’ll share the design proposal here before moving to implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Provide support for SSL/TSL connections [datafusion-ballista]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm closed issue #1367: Provide support for SSL/TSL connections
URL: https://github.com/apache/datafusion-ballista/issues/1367
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Provide support for SSL/TSL connections [datafusion-ballista]
Posted by "mohit7705 (via GitHub)" <gi...@apache.org>.
mohit7705 commented on issue #1367:
URL: https://github.com/apache/datafusion-ballista/issues/1367#issuecomment-3716596519
Hi @milenkovicm 👋
I’d like to help with this issue.
I’m planning to start by:
- Understanding the current HTTP communication flow between client, scheduler, and executors
- Researching how TLS is typically handled in similar distributed systems
- Proposing a minimal, optional TLS configuration (keeping plain HTTP as default)
Before proceeding further, I’d appreciate guidance on:
1. Whether TLS should be implemented at the transport layer only or also include mutual TLS (mTLS)
2. Any preferred libraries or existing patterns in the project for security-related configuration
3. Whether a design proposal document would be helpful before implementation
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Provide support for SSL/TSL connections [datafusion-ballista]
Posted by "phillipleblanc (via GitHub)" <gi...@apache.org>.
phillipleblanc commented on issue #1367:
URL: https://github.com/apache/datafusion-ballista/issues/1367#issuecomment-3788294348
@milenkovicm I've taken a shot at this in #1400
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org