You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by vinay patil <vi...@gmail.com> on 2016/10/07 16:26:54 UTC
Re: Data Transfer between TM should be encrypted
Hi Stephan,
https://github.com/apache/flink/pull/2518
Is this pull request going to be part of 1.2 release ? Just wanted to get
an idea on timelines so that I can pass on to the team.
Regards,
Vinay Patil
On Thu, Sep 15, 2016 at 11:45 AM, Vijay Srinivasaraghavan <
vijikarthi@yahoo.com> wrote:
> Hi Vinay,
>
> There are some delays and we expect the PR to be created next week.
>
> Regards
> Vijay
>
> On Wednesday, September 14, 2016 5:41 PM, vinay patil <
> vinay18.patil@gmail.com> wrote:
>
>
> Hi Vijay,
>
> Did you raise the PR for this task, I don't mind testing it out as well.
>
> Regards,
> Vinay Patil
>
> On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote:
>
> Hi Vijay,
>
> That's a good news for me. Eagerly waiting for this change so that I can
> integrate and test it before going live.
>
> Regards,
> Vinay Patil
>
> On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache Flink
> User Mailing List archive.] <[hidden email]> wrote:
>
> Hi Stephan,
>
> The dev work is almost complete except the Yarn mode deployment stuff that
> needs to be patched. We are expecting to send a PR in a week or two.
>
> Regards
> Vijay
>
>
> On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]> wrote:
>
>
> Let me loop in Vijay, I think he is the one working on this and can
> probably give the best estimate when it can be expected.
>
> @vijay: For the SSL/TLS transport encryption - do you have an estimate for
> the timeline of that feature?
>
>
> On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote:
>
> Hi Stephan,
>
> Thank you for your reply.
>
> Till when can I expect this feature to be integrated in master or release
> version ?
>
> We are going to get production data (financial data) in October end , so
> want to have this feature before that.
>
> Regards,
> Vinay Patil
>
> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User
> Mailing List archive.] <[hidden email]> wrote:
>
> Hi!
>
> The way that the JIRA issue you linked will achieve this is by hooking
> into the network stream pipeline directly, and encrypt the raw network byte
> stream. We built the network stack on Netty, and will use Netty's SSL/TLS
> handlers for that.
>
> That should be much more efficient than manual encryption/decryption in
> each user function.
>
> Stephan
>
>
>
>
>
>
> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote:
>
> Hi Ufuk,
>
> This is regarding this issue
> https://issues.apache.org/jira /browse/FLINK-4404
> <https://issues.apache.org/jira/browse/FLINK-4404>
>
> How can we achieve this, I am able to decrypt the data from Kafka coming
> in, but I want to make sure that the data is encrypted when flowing between
> TM's.
>
> One approach I can think of is to decrypt the data at the start of each
> operator and encrypt it at the end of each operator, but I feel this is not
> an efficient approach.
>
> I just want to check if there are alternatives to this and can this be
> achieved by doing some configurations.
>
> Regards,
> Vinay Patil
>
> ------------------------------
> View this message in context: Data Transfer between TM should be encrypted
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html>
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at
> Nabble.com.
>
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-flink-user-maili ng-list-archive.2336050.n4.
> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html>
> To start a new topic under Apache Flink User Mailing List archive., email [hidden
> email]
> To unsubscribe from Apache Flink User Mailing List archive., click here.
> NAML
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
>
> ------------------------------
> View this message in context: Re: Data Transfer between TM should be
> encrypted
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8786.html>
>
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at
> Nabble.com.
>
>
>
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-flink-user-maili ng-list-archive.2336050.n4.
> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8801.html
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8801.html>
> To start a new topic under Apache Flink User Mailing List archive., email [hidden
> email]
> To unsubscribe from Apache Flink User Mailing List archive., click here.
> NAML
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
>
>
> ------------------------------
> View this message in context: Re: Data Transfer between TM should be
> encrypted
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9033.html>
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at
> Nabble.com.
>
>
>
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9392.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Data Transfer between TM should be encrypted
Posted by vinay patil <vi...@gmail.com>.
Thank you Stephan and Robert for your positive response
Regards,
Vinay Patil
On Thu, Oct 13, 2016 at 11:22 AM, Stephan Ewen [via Apache Flink User
Mailing List archive.] <ml...@n4.nabble.com> wrote:
> I agree with Robert. We should probably aim for the release to be end of
> November, so that it will come this year even in the presence of some delay.
>
> Best,
> Stephan
>
>
> On Thu, Oct 13, 2016 at 11:36 AM, Robert Metzger <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=9538&i=0>> wrote:
>
>> Hi,
>> the release dates depend on the community, when features are ready and so
>> on.
>> There was no discussion yet when we plan to do the release, because most
>> of the features we want to have in are not yet done yet.
>> I think its likely that we'll have a 1.2 release by end of this year.
>>
>> Regards,
>> Robert
>>
>> On Thu, Oct 13, 2016 at 5:07 AM, vinay patil <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=9538&i=1>> wrote:
>>
>>> Hi Robert,
>>>
>>> Thank you for that information, can you please let me know when 1.2 is
>>> planned to release ?
>>>
>>> Regards,
>>> Vinay Patil
>>>
>>> On Wed, Oct 12, 2016 at 4:17 AM, rmetzger0 [via Apache Flink User
>>> Mailing List archive.] <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=9508&i=0>> wrote:
>>>
>>>> Hi,
>>>> I think that pull request will be merged for 1.2.
>>>>
>>>> On Fri, Oct 7, 2016 at 6:26 PM, vinay patil <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=9473&i=0>> wrote:
>>>>
>>>>> Hi Stephan,
>>>>>
>>>>> https://github.com/apache/flink/pull/2518
>>>>> Is this pull request going to be part of 1.2 release ? Just wanted to
>>>>> get an idea on timelines so that I can pass on to the team.
>>>>>
>>>>>
>>>>> Regards,
>>>>> Vinay Patil
>>>>>
>>>>> On Thu, Sep 15, 2016 at 11:45 AM, Vijay Srinivasaraghavan <[hidden
>>>>> email] <http:///user/SendEmail.jtp?type=node&node=9392&i=0>> wrote:
>>>>>
>>>>>> Hi Vinay,
>>>>>>
>>>>>> There are some delays and we expect the PR to be created next week.
>>>>>>
>>>>>> Regards
>>>>>> Vijay
>>>>>>
>>>>>> On Wednesday, September 14, 2016 5:41 PM, vinay patil <[hidden email]
>>>>>> <http:///user/SendEmail.jtp?type=node&node=9392&i=1>> wrote:
>>>>>>
>>>>>>
>>>>>> Hi Vijay,
>>>>>>
>>>>>> Did you raise the PR for this task, I don't mind testing it out as
>>>>>> well.
>>>>>>
>>>>>> Regards,
>>>>>> Vinay Patil
>>>>>>
>>>>>> On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote:
>>>>>>
>>>>>> Hi Vijay,
>>>>>>
>>>>>> That's a good news for me. Eagerly waiting for this change so that I
>>>>>> can integrate and test it before going live.
>>>>>>
>>>>>> Regards,
>>>>>> Vinay Patil
>>>>>>
>>>>>> On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache
>>>>>> Flink User Mailing List archive.] <[hidden email]> wrote:
>>>>>>
>>>>>> Hi Stephan,
>>>>>>
>>>>>> The dev work is almost complete except the Yarn mode deployment stuff
>>>>>> that needs to be patched. We are expecting to send a PR in a week or two.
>>>>>>
>>>>>> Regards
>>>>>> Vijay
>>>>>>
>>>>>>
>>>>>> On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> Let me loop in Vijay, I think he is the one working on this and can
>>>>>> probably give the best estimate when it can be expected.
>>>>>>
>>>>>> @vijay: For the SSL/TLS transport encryption - do you have an
>>>>>> estimate for the timeline of that feature?
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote:
>>>>>>
>>>>>> Hi Stephan,
>>>>>>
>>>>>> Thank you for your reply.
>>>>>>
>>>>>> Till when can I expect this feature to be integrated in master or
>>>>>> release version ?
>>>>>>
>>>>>> We are going to get production data (financial data) in October end ,
>>>>>> so want to have this feature before that.
>>>>>>
>>>>>> Regards,
>>>>>> Vinay Patil
>>>>>>
>>>>>> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User
>>>>>> Mailing List archive.] <[hidden email]> wrote:
>>>>>>
>>>>>> Hi!
>>>>>>
>>>>>> The way that the JIRA issue you linked will achieve this is by
>>>>>> hooking into the network stream pipeline directly, and encrypt the raw
>>>>>> network byte stream. We built the network stack on Netty, and will use
>>>>>> Netty's SSL/TLS handlers for that.
>>>>>>
>>>>>> That should be much more efficient than manual encryption/decryption
>>>>>> in each user function.
>>>>>>
>>>>>> Stephan
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote:
>>>>>>
>>>>>> Hi Ufuk,
>>>>>>
>>>>>> This is regarding this issue
>>>>>> https://issues.apache.org/jira /browse/FLINK-4404
>>>>>> <https://issues.apache.org/jira/browse/FLINK-4404>
>>>>>>
>>>>>> How can we achieve this, I am able to decrypt the data from Kafka
>>>>>> coming in, but I want to make sure that the data is encrypted when flowing
>>>>>> between TM's.
>>>>>>
>>>>>> One approach I can think of is to decrypt the data at the start of
>>>>>> each operator and encrypt it at the end of each operator, but I feel this
>>>>>> is not an efficient approach.
>>>>>>
>>>>>> I just want to check if there are alternatives to this and can this
>>>>>> be achieved by doing some configurations.
>>>>>>
>>>>>> Regards,
>>>>>> Vinay Patil
>>>>>>
>>>>>> ------------------------------
>>>>>> View this message in context: Data Transfer between TM should be
>>>>>> encrypted
>>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html>
>>>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>>>> archive
>>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>>>>> at Nabble.com.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> If you reply to this email, your message will be added to the
>>>>>> discussion below:
>>>>>> http://apache-flink-user-maili ng-list-archive.2336050.n4.
>>>>>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html
>>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html>
>>>>>> To start a new topic under Apache Flink User Mailing List archive.,
>>>>>> email [hidden email]
>>>>>> To unsubscribe from Apache Flink User Mailing List archive., click
>>>>>> here.
>>>>>> NAML
>>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> View this message in context: Re: Data Transfer between TM should be
>>>>>> encrypted
>>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8786.html>
>>>>>>
>>>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>>>> archive
>>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>>>>> at Nabble.com.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> If you reply to this email, your message will be added to the
>>>>>> discussion below:
>>>>>> http://apache-flink-user-maili ng-list-archive.2336050.n4.
>>>>>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8801.html
>>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8801.html>
>>>>>> To start a new topic under Apache Flink User Mailing List archive.,
>>>>>> email [hidden email]
>>>>>> To unsubscribe from Apache Flink User Mailing List archive., click
>>>>>> here.
>>>>>> NAML
>>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> View this message in context: Re: Data Transfer between TM should be
>>>>>> encrypted
>>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9033.html>
>>>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>>>> archive
>>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>>>>> at Nabble.com.
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> ------------------------------
>>>>> View this message in context: Re: Data Transfer between TM should be
>>>>> encrypted
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9392.html>
>>>>>
>>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>>> archive
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>>>> at Nabble.com.
>>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the
>>>> discussion below:
>>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nab
>>>> ble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9473.html
>>>> To start a new topic under Apache Flink User Mailing List archive.,
>>>> email [hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=9508&i=1>
>>>> To unsubscribe from Apache Flink User Mailing List archive., click here
>>>> .
>>>> NAML
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>
>>>
>>> ------------------------------
>>> View this message in context: Re: Data Transfer between TM should be
>>> encrypted
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9508.html>
>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>> archive
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>> at Nabble.com.
>>>
>>
>>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/Data-Transfer-between-TM-should-be-
> encrypted-tp8781p9538.html
> To start a new topic under Apache Flink User Mailing List archive., email
> ml-node+s2336050n1h83@n4.nabble.com
> To unsubscribe from Apache Flink User Mailing List archive., click here
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx>
> .
> NAML
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9539.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Data Transfer between TM should be encrypted
Posted by Stephan Ewen <se...@apache.org>.
I agree with Robert. We should probably aim for the release to be end of
November, so that it will come this year even in the presence of some delay.
Best,
Stephan
On Thu, Oct 13, 2016 at 11:36 AM, Robert Metzger <rm...@apache.org>
wrote:
> Hi,
> the release dates depend on the community, when features are ready and so
> on.
> There was no discussion yet when we plan to do the release, because most
> of the features we want to have in are not yet done yet.
> I think its likely that we'll have a 1.2 release by end of this year.
>
> Regards,
> Robert
>
> On Thu, Oct 13, 2016 at 5:07 AM, vinay patil <vi...@gmail.com>
> wrote:
>
>> Hi Robert,
>>
>> Thank you for that information, can you please let me know when 1.2 is
>> planned to release ?
>>
>> Regards,
>> Vinay Patil
>>
>> On Wed, Oct 12, 2016 at 4:17 AM, rmetzger0 [via Apache Flink User Mailing
>> List archive.] <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=9508&i=0>> wrote:
>>
>>> Hi,
>>> I think that pull request will be merged for 1.2.
>>>
>>> On Fri, Oct 7, 2016 at 6:26 PM, vinay patil <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=9473&i=0>> wrote:
>>>
>>>> Hi Stephan,
>>>>
>>>> https://github.com/apache/flink/pull/2518
>>>> Is this pull request going to be part of 1.2 release ? Just wanted to
>>>> get an idea on timelines so that I can pass on to the team.
>>>>
>>>>
>>>> Regards,
>>>> Vinay Patil
>>>>
>>>> On Thu, Sep 15, 2016 at 11:45 AM, Vijay Srinivasaraghavan <[hidden
>>>> email] <http:///user/SendEmail.jtp?type=node&node=9392&i=0>> wrote:
>>>>
>>>>> Hi Vinay,
>>>>>
>>>>> There are some delays and we expect the PR to be created next week.
>>>>>
>>>>> Regards
>>>>> Vijay
>>>>>
>>>>> On Wednesday, September 14, 2016 5:41 PM, vinay patil <[hidden email]
>>>>> <http:///user/SendEmail.jtp?type=node&node=9392&i=1>> wrote:
>>>>>
>>>>>
>>>>> Hi Vijay,
>>>>>
>>>>> Did you raise the PR for this task, I don't mind testing it out as
>>>>> well.
>>>>>
>>>>> Regards,
>>>>> Vinay Patil
>>>>>
>>>>> On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote:
>>>>>
>>>>> Hi Vijay,
>>>>>
>>>>> That's a good news for me. Eagerly waiting for this change so that I
>>>>> can integrate and test it before going live.
>>>>>
>>>>> Regards,
>>>>> Vinay Patil
>>>>>
>>>>> On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache
>>>>> Flink User Mailing List archive.] <[hidden email]> wrote:
>>>>>
>>>>> Hi Stephan,
>>>>>
>>>>> The dev work is almost complete except the Yarn mode deployment stuff
>>>>> that needs to be patched. We are expecting to send a PR in a week or two.
>>>>>
>>>>> Regards
>>>>> Vijay
>>>>>
>>>>>
>>>>> On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]>
>>>>> wrote:
>>>>>
>>>>>
>>>>> Let me loop in Vijay, I think he is the one working on this and can
>>>>> probably give the best estimate when it can be expected.
>>>>>
>>>>> @vijay: For the SSL/TLS transport encryption - do you have an estimate
>>>>> for the timeline of that feature?
>>>>>
>>>>>
>>>>> On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote:
>>>>>
>>>>> Hi Stephan,
>>>>>
>>>>> Thank you for your reply.
>>>>>
>>>>> Till when can I expect this feature to be integrated in master or
>>>>> release version ?
>>>>>
>>>>> We are going to get production data (financial data) in October end ,
>>>>> so want to have this feature before that.
>>>>>
>>>>> Regards,
>>>>> Vinay Patil
>>>>>
>>>>> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User
>>>>> Mailing List archive.] <[hidden email]> wrote:
>>>>>
>>>>> Hi!
>>>>>
>>>>> The way that the JIRA issue you linked will achieve this is by hooking
>>>>> into the network stream pipeline directly, and encrypt the raw network byte
>>>>> stream. We built the network stack on Netty, and will use Netty's SSL/TLS
>>>>> handlers for that.
>>>>>
>>>>> That should be much more efficient than manual encryption/decryption
>>>>> in each user function.
>>>>>
>>>>> Stephan
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote:
>>>>>
>>>>> Hi Ufuk,
>>>>>
>>>>> This is regarding this issue
>>>>> https://issues.apache.org/jira /browse/FLINK-4404
>>>>> <https://issues.apache.org/jira/browse/FLINK-4404>
>>>>>
>>>>> How can we achieve this, I am able to decrypt the data from Kafka
>>>>> coming in, but I want to make sure that the data is encrypted when flowing
>>>>> between TM's.
>>>>>
>>>>> One approach I can think of is to decrypt the data at the start of
>>>>> each operator and encrypt it at the end of each operator, but I feel this
>>>>> is not an efficient approach.
>>>>>
>>>>> I just want to check if there are alternatives to this and can this be
>>>>> achieved by doing some configurations.
>>>>>
>>>>> Regards,
>>>>> Vinay Patil
>>>>>
>>>>> ------------------------------
>>>>> View this message in context: Data Transfer between TM should be
>>>>> encrypted
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html>
>>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>>> archive
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>>>> at Nabble.com.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> If you reply to this email, your message will be added to the
>>>>> discussion below:
>>>>> http://apache-flink-user-maili ng-list-archive.2336050.n4.
>>>>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html>
>>>>> To start a new topic under Apache Flink User Mailing List archive.,
>>>>> email [hidden email]
>>>>> To unsubscribe from Apache Flink User Mailing List archive., click
>>>>> here.
>>>>> NAML
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> View this message in context: Re: Data Transfer between TM should be
>>>>> encrypted
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8786.html>
>>>>>
>>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>>> archive
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>>>> at Nabble.com.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> If you reply to this email, your message will be added to the
>>>>> discussion below:
>>>>> http://apache-flink-user-maili ng-list-archive.2336050.n4.
>>>>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8801.html
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8801.html>
>>>>> To start a new topic under Apache Flink User Mailing List archive.,
>>>>> email [hidden email]
>>>>> To unsubscribe from Apache Flink User Mailing List archive., click
>>>>> here.
>>>>> NAML
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> View this message in context: Re: Data Transfer between TM should be
>>>>> encrypted
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9033.html>
>>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>>> archive
>>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>>>> at Nabble.com.
>>>>>
>>>>>
>>>>>
>>>>
>>>> ------------------------------
>>>> View this message in context: Re: Data Transfer between TM should be
>>>> encrypted
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9392.html>
>>>>
>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>> archive
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>>> at Nabble.com.
>>>>
>>>
>>>
>>>
>>> ------------------------------
>>> If you reply to this email, your message will be added to the discussion
>>> below:
>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nab
>>> ble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9473.html
>>> To start a new topic under Apache Flink User Mailing List archive.,
>>> email [hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=9508&i=1>
>>> To unsubscribe from Apache Flink User Mailing List archive., click here.
>>> NAML
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>>
>>
>> ------------------------------
>> View this message in context: Re: Data Transfer between TM should be
>> encrypted
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9508.html>
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>> at Nabble.com.
>>
>
>
Re: Data Transfer between TM should be encrypted
Posted by Robert Metzger <rm...@apache.org>.
Hi,
the release dates depend on the community, when features are ready and so
on.
There was no discussion yet when we plan to do the release, because most of
the features we want to have in are not yet done yet.
I think its likely that we'll have a 1.2 release by end of this year.
Regards,
Robert
On Thu, Oct 13, 2016 at 5:07 AM, vinay patil <vi...@gmail.com>
wrote:
> Hi Robert,
>
> Thank you for that information, can you please let me know when 1.2 is
> planned to release ?
>
> Regards,
> Vinay Patil
>
> On Wed, Oct 12, 2016 at 4:17 AM, rmetzger0 [via Apache Flink User Mailing
> List archive.] <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=9508&i=0>> wrote:
>
>> Hi,
>> I think that pull request will be merged for 1.2.
>>
>> On Fri, Oct 7, 2016 at 6:26 PM, vinay patil <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=9473&i=0>> wrote:
>>
>>> Hi Stephan,
>>>
>>> https://github.com/apache/flink/pull/2518
>>> Is this pull request going to be part of 1.2 release ? Just wanted to
>>> get an idea on timelines so that I can pass on to the team.
>>>
>>>
>>> Regards,
>>> Vinay Patil
>>>
>>> On Thu, Sep 15, 2016 at 11:45 AM, Vijay Srinivasaraghavan <[hidden
>>> email] <http:///user/SendEmail.jtp?type=node&node=9392&i=0>> wrote:
>>>
>>>> Hi Vinay,
>>>>
>>>> There are some delays and we expect the PR to be created next week.
>>>>
>>>> Regards
>>>> Vijay
>>>>
>>>> On Wednesday, September 14, 2016 5:41 PM, vinay patil <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=9392&i=1>> wrote:
>>>>
>>>>
>>>> Hi Vijay,
>>>>
>>>> Did you raise the PR for this task, I don't mind testing it out as well.
>>>>
>>>> Regards,
>>>> Vinay Patil
>>>>
>>>> On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote:
>>>>
>>>> Hi Vijay,
>>>>
>>>> That's a good news for me. Eagerly waiting for this change so that I
>>>> can integrate and test it before going live.
>>>>
>>>> Regards,
>>>> Vinay Patil
>>>>
>>>> On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache
>>>> Flink User Mailing List archive.] <[hidden email]> wrote:
>>>>
>>>> Hi Stephan,
>>>>
>>>> The dev work is almost complete except the Yarn mode deployment stuff
>>>> that needs to be patched. We are expecting to send a PR in a week or two.
>>>>
>>>> Regards
>>>> Vijay
>>>>
>>>>
>>>> On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]>
>>>> wrote:
>>>>
>>>>
>>>> Let me loop in Vijay, I think he is the one working on this and can
>>>> probably give the best estimate when it can be expected.
>>>>
>>>> @vijay: For the SSL/TLS transport encryption - do you have an estimate
>>>> for the timeline of that feature?
>>>>
>>>>
>>>> On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote:
>>>>
>>>> Hi Stephan,
>>>>
>>>> Thank you for your reply.
>>>>
>>>> Till when can I expect this feature to be integrated in master or
>>>> release version ?
>>>>
>>>> We are going to get production data (financial data) in October end ,
>>>> so want to have this feature before that.
>>>>
>>>> Regards,
>>>> Vinay Patil
>>>>
>>>> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User
>>>> Mailing List archive.] <[hidden email]> wrote:
>>>>
>>>> Hi!
>>>>
>>>> The way that the JIRA issue you linked will achieve this is by hooking
>>>> into the network stream pipeline directly, and encrypt the raw network byte
>>>> stream. We built the network stack on Netty, and will use Netty's SSL/TLS
>>>> handlers for that.
>>>>
>>>> That should be much more efficient than manual encryption/decryption in
>>>> each user function.
>>>>
>>>> Stephan
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote:
>>>>
>>>> Hi Ufuk,
>>>>
>>>> This is regarding this issue
>>>> https://issues.apache.org/jira /browse/FLINK-4404
>>>> <https://issues.apache.org/jira/browse/FLINK-4404>
>>>>
>>>> How can we achieve this, I am able to decrypt the data from Kafka
>>>> coming in, but I want to make sure that the data is encrypted when flowing
>>>> between TM's.
>>>>
>>>> One approach I can think of is to decrypt the data at the start of each
>>>> operator and encrypt it at the end of each operator, but I feel this is not
>>>> an efficient approach.
>>>>
>>>> I just want to check if there are alternatives to this and can this be
>>>> achieved by doing some configurations.
>>>>
>>>> Regards,
>>>> Vinay Patil
>>>>
>>>> ------------------------------
>>>> View this message in context: Data Transfer between TM should be
>>>> encrypted
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html>
>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>> archive
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>>> at Nabble.com.
>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the
>>>> discussion below:
>>>> http://apache-flink-user-maili ng-list-archive.2336050.n4.
>>>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html>
>>>> To start a new topic under Apache Flink User Mailing List archive.,
>>>> email [hidden email]
>>>> To unsubscribe from Apache Flink User Mailing List archive., click here
>>>> .
>>>> NAML
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> View this message in context: Re: Data Transfer between TM should be
>>>> encrypted
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8786.html>
>>>>
>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>> archive
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>>> at Nabble.com.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the
>>>> discussion below:
>>>> http://apache-flink-user-maili ng-list-archive.2336050.n4.
>>>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8801.html
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8801.html>
>>>> To start a new topic under Apache Flink User Mailing List archive.,
>>>> email [hidden email]
>>>> To unsubscribe from Apache Flink User Mailing List archive., click here
>>>> .
>>>> NAML
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> View this message in context: Re: Data Transfer between TM should be
>>>> encrypted
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9033.html>
>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>> archive
>>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>>> at Nabble.com.
>>>>
>>>>
>>>>
>>>
>>> ------------------------------
>>> View this message in context: Re: Data Transfer between TM should be
>>> encrypted
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9392.html>
>>>
>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>> archive
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>> at Nabble.com.
>>>
>>
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-flink-user-mailing-list-archive.2336050.n4.
>> nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9473.html
>> To start a new topic under Apache Flink User Mailing List archive., email [hidden
>> email] <http:///user/SendEmail.jtp?type=node&node=9508&i=1>
>> To unsubscribe from Apache Flink User Mailing List archive., click here.
>> NAML
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
> ------------------------------
> View this message in context: Re: Data Transfer between TM should be
> encrypted
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9508.html>
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at
> Nabble.com.
>
Re: Data Transfer between TM should be encrypted
Posted by vinay patil <vi...@gmail.com>.
Hi Robert,
Thank you for that information, can you please let me know when 1.2 is
planned to release ?
Regards,
Vinay Patil
On Wed, Oct 12, 2016 at 4:17 AM, rmetzger0 [via Apache Flink User Mailing
List archive.] <ml...@n4.nabble.com> wrote:
> Hi,
> I think that pull request will be merged for 1.2.
>
> On Fri, Oct 7, 2016 at 6:26 PM, vinay patil <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=9473&i=0>> wrote:
>
>> Hi Stephan,
>>
>> https://github.com/apache/flink/pull/2518
>> Is this pull request going to be part of 1.2 release ? Just wanted to get
>> an idea on timelines so that I can pass on to the team.
>>
>>
>> Regards,
>> Vinay Patil
>>
>> On Thu, Sep 15, 2016 at 11:45 AM, Vijay Srinivasaraghavan <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=9392&i=0>> wrote:
>>
>>> Hi Vinay,
>>>
>>> There are some delays and we expect the PR to be created next week.
>>>
>>> Regards
>>> Vijay
>>>
>>> On Wednesday, September 14, 2016 5:41 PM, vinay patil <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=9392&i=1>> wrote:
>>>
>>>
>>> Hi Vijay,
>>>
>>> Did you raise the PR for this task, I don't mind testing it out as well.
>>>
>>> Regards,
>>> Vinay Patil
>>>
>>> On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote:
>>>
>>> Hi Vijay,
>>>
>>> That's a good news for me. Eagerly waiting for this change so that I can
>>> integrate and test it before going live.
>>>
>>> Regards,
>>> Vinay Patil
>>>
>>> On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache
>>> Flink User Mailing List archive.] <[hidden email]> wrote:
>>>
>>> Hi Stephan,
>>>
>>> The dev work is almost complete except the Yarn mode deployment stuff
>>> that needs to be patched. We are expecting to send a PR in a week or two.
>>>
>>> Regards
>>> Vijay
>>>
>>>
>>> On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]>
>>> wrote:
>>>
>>>
>>> Let me loop in Vijay, I think he is the one working on this and can
>>> probably give the best estimate when it can be expected.
>>>
>>> @vijay: For the SSL/TLS transport encryption - do you have an estimate
>>> for the timeline of that feature?
>>>
>>>
>>> On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote:
>>>
>>> Hi Stephan,
>>>
>>> Thank you for your reply.
>>>
>>> Till when can I expect this feature to be integrated in master or
>>> release version ?
>>>
>>> We are going to get production data (financial data) in October end , so
>>> want to have this feature before that.
>>>
>>> Regards,
>>> Vinay Patil
>>>
>>> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User
>>> Mailing List archive.] <[hidden email]> wrote:
>>>
>>> Hi!
>>>
>>> The way that the JIRA issue you linked will achieve this is by hooking
>>> into the network stream pipeline directly, and encrypt the raw network byte
>>> stream. We built the network stack on Netty, and will use Netty's SSL/TLS
>>> handlers for that.
>>>
>>> That should be much more efficient than manual encryption/decryption in
>>> each user function.
>>>
>>> Stephan
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote:
>>>
>>> Hi Ufuk,
>>>
>>> This is regarding this issue
>>> https://issues.apache.org/jira /browse/FLINK-4404
>>> <https://issues.apache.org/jira/browse/FLINK-4404>
>>>
>>> How can we achieve this, I am able to decrypt the data from Kafka coming
>>> in, but I want to make sure that the data is encrypted when flowing between
>>> TM's.
>>>
>>> One approach I can think of is to decrypt the data at the start of each
>>> operator and encrypt it at the end of each operator, but I feel this is not
>>> an efficient approach.
>>>
>>> I just want to check if there are alternatives to this and can this be
>>> achieved by doing some configurations.
>>>
>>> Regards,
>>> Vinay Patil
>>>
>>> ------------------------------
>>> View this message in context: Data Transfer between TM should be
>>> encrypted
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html>
>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>> archive
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>> at Nabble.com.
>>>
>>>
>>>
>>>
>>> ------------------------------
>>> If you reply to this email, your message will be added to the discussion
>>> below:
>>> http://apache-flink-user-maili ng-list-archive.2336050.n4.
>>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html>
>>> To start a new topic under Apache Flink User Mailing List archive.,
>>> email [hidden email]
>>> To unsubscribe from Apache Flink User Mailing List archive., click here.
>>> NAML
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>>>
>>>
>>> ------------------------------
>>> View this message in context: Re: Data Transfer between TM should be
>>> encrypted
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8786.html>
>>>
>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>> archive
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>> at Nabble.com.
>>>
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------
>>> If you reply to this email, your message will be added to the discussion
>>> below:
>>> http://apache-flink-user-maili ng-list-archive.2336050.n4.
>>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8801.html
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8801.html>
>>> To start a new topic under Apache Flink User Mailing List archive.,
>>> email [hidden email]
>>> To unsubscribe from Apache Flink User Mailing List archive., click here.
>>> NAML
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>>>
>>>
>>>
>>> ------------------------------
>>> View this message in context: Re: Data Transfer between TM should be
>>> encrypted
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9033.html>
>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>> archive
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>>> at Nabble.com.
>>>
>>>
>>>
>>
>> ------------------------------
>> View this message in context: Re: Data Transfer between TM should be
>> encrypted
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9392.html>
>>
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>> at Nabble.com.
>>
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/Data-Transfer-between-TM-should-be-
> encrypted-tp8781p9473.html
> To start a new topic under Apache Flink User Mailing List archive., email
> ml-node+s2336050n1h83@n4.nabble.com
> To unsubscribe from Apache Flink User Mailing List archive., click here
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx>
> .
> NAML
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9508.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Data Transfer between TM should be encrypted
Posted by Robert Metzger <rm...@apache.org>.
Hi,
I think that pull request will be merged for 1.2.
On Fri, Oct 7, 2016 at 6:26 PM, vinay patil <vi...@gmail.com> wrote:
> Hi Stephan,
>
> https://github.com/apache/flink/pull/2518
> Is this pull request going to be part of 1.2 release ? Just wanted to get
> an idea on timelines so that I can pass on to the team.
>
>
> Regards,
> Vinay Patil
>
> On Thu, Sep 15, 2016 at 11:45 AM, Vijay Srinivasaraghavan <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=9392&i=0>> wrote:
>
>> Hi Vinay,
>>
>> There are some delays and we expect the PR to be created next week.
>>
>> Regards
>> Vijay
>>
>> On Wednesday, September 14, 2016 5:41 PM, vinay patil <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=9392&i=1>> wrote:
>>
>>
>> Hi Vijay,
>>
>> Did you raise the PR for this task, I don't mind testing it out as well.
>>
>> Regards,
>> Vinay Patil
>>
>> On Tue, Aug 30, 2016 at 6:28 PM, Vinay Patil <[hidden email]> wrote:
>>
>> Hi Vijay,
>>
>> That's a good news for me. Eagerly waiting for this change so that I can
>> integrate and test it before going live.
>>
>> Regards,
>> Vinay Patil
>>
>> On Tue, Aug 30, 2016 at 4:06 PM, Vijay Srinivasaraghavan [via Apache
>> Flink User Mailing List archive.] <[hidden email]> wrote:
>>
>> Hi Stephan,
>>
>> The dev work is almost complete except the Yarn mode deployment stuff
>> that needs to be patched. We are expecting to send a PR in a week or two.
>>
>> Regards
>> Vijay
>>
>>
>> On Tuesday, August 30, 2016 12:39 AM, Stephan Ewen <[hidden email]>
>> wrote:
>>
>>
>> Let me loop in Vijay, I think he is the one working on this and can
>> probably give the best estimate when it can be expected.
>>
>> @vijay: For the SSL/TLS transport encryption - do you have an estimate
>> for the timeline of that feature?
>>
>>
>> On Mon, Aug 29, 2016 at 8:54 PM, vinay patil <[hidden email]> wrote:
>>
>> Hi Stephan,
>>
>> Thank you for your reply.
>>
>> Till when can I expect this feature to be integrated in master or release
>> version ?
>>
>> We are going to get production data (financial data) in October end , so
>> want to have this feature before that.
>>
>> Regards,
>> Vinay Patil
>>
>> On Mon, Aug 29, 2016 at 11:15 AM, Stephan Ewen [via Apache Flink User
>> Mailing List archive.] <[hidden email]> wrote:
>>
>> Hi!
>>
>> The way that the JIRA issue you linked will achieve this is by hooking
>> into the network stream pipeline directly, and encrypt the raw network byte
>> stream. We built the network stack on Netty, and will use Netty's SSL/TLS
>> handlers for that.
>>
>> That should be much more efficient than manual encryption/decryption in
>> each user function.
>>
>> Stephan
>>
>>
>>
>>
>>
>>
>> On Mon, Aug 29, 2016 at 6:12 PM, vinay patil <[hidden email]> wrote:
>>
>> Hi Ufuk,
>>
>> This is regarding this issue
>> https://issues.apache.org/jira /browse/FLINK-4404
>> <https://issues.apache.org/jira/browse/FLINK-4404>
>>
>> How can we achieve this, I am able to decrypt the data from Kafka coming
>> in, but I want to make sure that the data is encrypted when flowing between
>> TM's.
>>
>> One approach I can think of is to decrypt the data at the start of each
>> operator and encrypt it at the end of each operator, but I feel this is not
>> an efficient approach.
>>
>> I just want to check if there are alternatives to this and can this be
>> achieved by doing some configurations.
>>
>> Regards,
>> Vinay Patil
>>
>> ------------------------------
>> View this message in context: Data Transfer between TM should be
>> encrypted
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781.html>
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>> at Nabble.com.
>>
>>
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-flink-user-maili ng-list-archive.2336050.n4.
>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8782.html
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8782.html>
>> To start a new topic under Apache Flink User Mailing List archive., email [hidden
>> email]
>> To unsubscribe from Apache Flink User Mailing List archive., click here.
>> NAML
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>>
>>
>> ------------------------------
>> View this message in context: Re: Data Transfer between TM should be
>> encrypted
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8786.html>
>>
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>> at Nabble.com.
>>
>>
>>
>>
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-flink-user-maili ng-list-archive.2336050.n4.
>> nabble.com/Data-Transfer-betwe en-TM-should-be-encrypted- tp8781p8801.html
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p8801.html>
>> To start a new topic under Apache Flink User Mailing List archive., email [hidden
>> email]
>> To unsubscribe from Apache Flink User Mailing List archive., click here.
>> NAML
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>>
>>
>>
>> ------------------------------
>> View this message in context: Re: Data Transfer between TM should be
>> encrypted
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9033.html>
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/>
>> at Nabble.com.
>>
>>
>>
>
> ------------------------------
> View this message in context: Re: Data Transfer between TM should be
> encrypted
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-Transfer-between-TM-should-be-encrypted-tp8781p9392.html>
>
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at
> Nabble.com.
>