You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@texera.apache.org by "chenlica (via GitHub)" <gi...@apache.org> on 2026/07/02 06:33:05 UTC
[D] Encrypting internal RESTful requests between micro services [texera]
GitHub user chenlica created a discussion: Encrypting internal RESTful requests between micro services
There are many internal RESTful requests between micro services in the Texera system. We want to know whether we need to encrypt these requests to make them more secure.
@parshimers @madhusudancs and others: any thoughts?
GitHub link: https://github.com/apache/texera/discussions/6059
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Encrypting internal RESTful requests between micro services [texera]
Posted by "parshimers (via GitHub)" <gi...@apache.org>.
GitHub user parshimers added a comment to the discussion: Encrypting internal RESTful requests between micro services
I agree with Dylan's points about focusing on wrapping everything (TLS/mTLS) rather than focusing solely on HTTP. I think giving linkerd a detailed look is also great direction. The earlier we can get something to work the better; it is going from a 0 to a 1.
GitHub link: https://github.com/apache/texera/discussions/6059#discussioncomment-18570583
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Encrypting internal RESTful requests between micro services [texera]
Posted by "dzueck (via GitHub)" <gi...@apache.org>.
GitHub user dzueck added a comment to the discussion: Encrypting internal RESTful requests between micro services
I am not assuming we are using http. I have shown that we are and that using http can leak information. https does seem like a valid solution, however we need to figure out how to do certificate distribution as https requires an authority to authenticate connections. Additionally, https would provide even more important qualities like integrity and authentication which we have not discussed much here, but are also very important.
GitHub link: https://github.com/apache/texera/discussions/6059#discussioncomment-17600542
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Encrypting internal RESTful requests between micro services [texera]
Posted by "dzueck (via GitHub)" <gi...@apache.org>.
GitHub user dzueck added a comment to the discussion: Encrypting internal RESTful requests between micro services
I have been able to show that using packet sniffing tools like tcpdump and Wireshark, the traffic between services does leak potentially private information such as data set names and contents. There is likely more, but I did not look for any more. In my case I had root access to the deployment to view the tcp communications, however as parshimers mentioned, you would not need root privileges to read these packets if services are spread out on multiple machines. In this setup you would only need a separate device on the same network as one of the services.
GitHub link: https://github.com/apache/texera/discussions/6059#discussioncomment-17552928
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Encrypting internal RESTful requests between micro services [texera]
Posted by "dzueck (via GitHub)" <gi...@apache.org>.
GitHub user dzueck added a comment to the discussion: Encrypting internal RESTful requests between micro services
Further looking into this, I believe that https is actually not what we are looking for here. The important features of https we need are:
1. **Encryption** to prevent external parties from reading sensitive information
2. **Authentication** to prevent man in the middle attacks where a third party pretends to be a CU or micro service. This would allow them to request sensitive data or provide fake data among other things.
3. **Integrity** to prevent external parties from modifying requests to provide fake data/requests.
While https provides all 3 of these, they are actually provided by TLS which https uses. For this reason we should be discussing providing TLS between containers instead. There are a few existing tools to do this (often called service meshes), however the most light weight and simple one I could find is linkerd which I will be looking into using.
Note: Due to the proposal to utilize VMs #8430 for container runtimes, any solution we choose to use should utilize sidecars instead of the sidecar less modern approaches. This is due to the fact that vms use a separate kernel with a separate networking stack which can interfere with non sidecar approaches.
GitHub link: https://github.com/apache/texera/discussions/6059#discussioncomment-18437847
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Encrypting internal RESTful requests between micro services [texera]
Posted by "parshimers (via GitHub)" <gi...@apache.org>.
GitHub user parshimers added a comment to the discussion: Encrypting internal RESTful requests between micro services
I think between services, there should be encryption, one way or another. It can easily be the case that different services are on different machines. In this case it can be important not only for privacy, but for stopping someone from injecting spoofed messages between the two services.
How this is done more concretely, I am a little less certain of. Asterix implements the ability to have certificate auth between all nodes in the cluster, which wraps all the IPC. However this was implemented before k8s was popular and basically assumes it can be running on bare metal. If you assume everything will always be running in k8s if it's distributed, then I think you can use a service mesh like istio that presents a proxy on each pod that manages the encryption separately. Which would be a lot more convenient than having to add this functionality internally and figure out some way to deal with all the certificate management. I'd be interested to hear Madhu's take on that idea however.
GitHub link: https://github.com/apache/texera/discussions/6059#discussioncomment-17512996
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Encrypting internal RESTful requests between micro services [texera]
Posted by "chenlica (via GitHub)" <gi...@apache.org>.
GitHub user chenlica added a comment to the discussion: Encrypting internal RESTful requests between micro services
@dzueck If you can include some diagram, that will be helpful.
GitHub link: https://github.com/apache/texera/discussions/6059#discussioncomment-18573903
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Encrypting internal RESTful requests between micro services [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan edited a comment on the discussion: Encrypting internal RESTful requests between micro services
Are you assuming that it's http? What if we use https? I don't know much about how does tcp layer work, but I guess the certificate can provide encryption at application layer.
GitHub link: https://github.com/apache/texera/discussions/6059#discussioncomment-17554800
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org
Re: [D] Encrypting internal RESTful requests between micro services [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
GitHub user aglinxinyuan added a comment to the discussion: Encrypting internal RESTful requests between micro services
Are you assuming that it's http? What if we use https? I don't know much about how does tcp layer work, but I guess the certificate can provide encryption.
GitHub link: https://github.com/apache/texera/discussions/6059#discussioncomment-17554800
----
This is an automatically sent email for dev@texera.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@texera.apache.org