You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by vinay patil <vi...@gmail.com> on 2016/09/15 00:36:59 UTC
Re: Data Transfer between TM should be encrypted
Hi Vijay,
Did you raise the PR for this task, I don't mind testing it out as well.
Regards,
Vinay Patil
On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <vi...@gmail.com>
wrote:
> Hi Vijay,
>
> That's a good news for me. Eagerly waiting for this change so that I can
> integrate and test it before going live.
>
> Regards,
> Vinay Patil
>
> On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache Flink
> User Mailing List archive.] <ml...@n4.nabble.com>
> wrote:
>
>> Hi Stephan,
>>
>> The dev work is almost complete except the Yarn mode deployment stuff
>> that needs to be patched. We are expecting to send a PR in a week or two.
>>
>> Regards
>> Vijay
>>
>>
>> On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=8801&i=0>> wrote:
>>
>>
>> Let me loop in Vijay, I think he is the one working on this and can
>> probably give the best estimate when it can be expected.
>>
>> @vijay: For the SSL/TLS transport encryption - do you have an estimate
>> for the timeline of that feature?
>>
>>
>> On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=8801&i=1>> wrote:
>>
>> Hi Stephan,
>>
>> Thank you for your reply.
>>
>> Till when can I expect this feature to be integrated in master or release
>> version ?
>>
>> We are going to get production data (financial data) in October end , so
>> want to have this feature before that.
>>
>> Regards,
>> Vinay Patil
>>
>> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User
>> Mailing List archive.] <[hidden email]> wrote:
>>
>> Hi!
>>
>> The way that the JIRA issue you linked will achieve this is by hooking
>> into the network stream pipeline directly, and encrypt the raw network byte
>> stream. We built the network stack on Netty, and will use Netty's SSL/TLS
>> handlers for that.
>>
>> That should be much more efficient than manual encryption/decryption in
>> each user function.
>>
>> Stephan
>>
>>
>>
>>
>>
>>
>> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote:
>>
>> Hi Ufuk,
>>
>> This is regarding this issue
>> https://issues.apache.org/jira /browse/FLINK-4404
>> <https://issues.apache.org/jira/browse/FLINK-4404>
>>
>> How can we achieve this, I am able to decrypt the data from Kafka coming
>> in, but I want to make sure that the data is encrypted when flowing between
>> TM's.
>>
>> One approach I can think of is to decrypt the data at the start of each
>> operator and encrypt it at the end of each operator, but I feel this is not
>> an efficient approach.
>>
>> I just want to check if there are alternatives to this and can this be
>> achieved by doing some configurations.
>>
>> Regards,
>> Vinay Patil
>>
>> ------------------------------
>> View this message in context: Data Transfer between TM should be
>> encrypted
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html>
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>> at Nabble.com.
>>
>>
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-flink-user-maili ng-list-archive.2336050.n4.
>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html>
>> To start a new topic under Apache Flink User Mailing List archive., email [hidden
>> email]
>> To unsubscribe from Apache Flink User Mailing List archive., click here.
>> NAML
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>>
>>
>> ------------------------------
>> View this message in context: Re: Data Transfer between TM should be
>> encrypted
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8786.html>
>>
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>> at Nabble.com.
>>
>>
>>
>>
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-flink-user-mailing-list-archive.2336050.n4.
>> nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8801.html
>> To start a new topic under Apache Flink User Mailing List archive., email
>> ml-node+s2336050n1h83@n4.nabble.com
>> To unsubscribe from Apache Flink User Mailing List archive., click here
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx>
>> .
>> NAML
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9033.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Data Transfer between TM should be encrypted
Posted by Vijay Srinivasaraghavan <vi...@yahoo.com>.
Hi Vinay,
There are some delays and we expect the PR to be created next week.
RegardsVijay
On Wednesday, September 14, 2016 5:41 PM, vinay patil <vi...@gmail.com> wrote:
Hi Vijay,
Did you raise the PR for this task, I don't mind testing it out as well.
Regards,Vinay Patil
On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote:
Hi Vijay,
That's a good news for me. Eagerly waiting for this change so that I can integrate and test it before going live.
Regards,Vinay Patil
On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache Flink User Mailing List archive.] <[hidden email]> wrote:
Hi Stephan,
The dev work is almost complete except the Yarn mode deployment stuff that needs to be patched. We are expecting to send a PR in a week or two.
RegardsVijay
On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]> wrote:
Let me loop in Vijay, I think he is the one working on this and can probably give the best estimate when it can be expected.
@vijay: For the SSL/TLS transport encryption - do you have an estimate for the timeline of that feature?
On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote:
Hi Stephan,
Thank you for your reply.
Till when can I expect this feature to be integrated in master or release version ?
We are going to get production data (financial data) in October end , so want to have this feature before that.
Regards,Vinay Patil
On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User Mailing List archive.] <[hidden email]> wrote:
Hi!
The way that the JIRA issue you linked will achieve this is by hooking into the network stream pipeline directly, and encrypt the raw network byte stream. We built the network stack on Netty, and will use Netty's SSL/TLS handlers for that.
That should be much more efficient than manual encryption/decryption in each user function.
Stephan
On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote:
Hi Ufuk,
This is regarding this issuehttps://issues.apache.org/jira /browse/FLINK-4404
How can we achieve this, I am able to decrypt the data from Kafka coming in, but I want to make sure that the data is encrypted when flowing between TM's.
One approach I can think of is to decrypt the data at the start of each operator and encrypt it at the end of each operator, but I feel this is not an efficient approach.
I just want to check if there are alternatives to this and can this be achieved by doing some configurations.
Regards,Vinay Patil
View this message in context: Data Transfer between TM should be encrypted
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
If you reply to this email, your message will be added to the discussion below: http://apache-flink-user-maili ng-list-archive.2336050.n4. nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html To start a new topic under Apache Flink User Mailing List archive., email [hidden email]
To unsubscribe from Apache Flink User Mailing List archive., click here.
NAML
View this message in context: Re: Data Transfer between TM should be encrypted
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
If you reply to this email, your message will be added to the discussion below: http://apache-flink-user-maili ng-list-archive.2336050.n4. nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8801.html To start a new topic under Apache Flink User Mailing List archive., email [hidden email]
To unsubscribe from Apache Flink User Mailing List archive., click here.
NAML
View this message in context: Re: Data Transfer between TM should be encrypted
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.