Re: Using sensitive configuration/credentials

[vino yang <ya...@gmail.com>]

Hi Matt,

Flink is currently enhancing its security, such as the current data
transmission can be configured with SSL mode[1].
However, some problems involving configuration and web ui display do exist,
and they are still displayed in plain text.
I think a temporary way to do this is to keep your secret configuration in
encrypted form elsewhere, such as Zookeeper or RDBMS, and then dynamically
read it into the job in a UDF (in the open method).

https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/security-ssl.html

Thanks, vino.

Matt Moore <me...@mattdoescode.com> 于2018年8月9日周四 上午1:54写道：

> I'm wondering what the best practice is for using secrets in a Flink
> program, and I can't find any info in the docs or posted anywhere else.
>
> I need to store an access token to one of my APIs for flink to use to dump
> results into, and right now I'm passing it through as a configuration
> parameter, but that doesn't seem like the most secure thing to do and the
> value shows up in the Flink Dashboard under Configuration which is less
> than ideal.
>
> Has anyone else dealt with a situation like this?
>
> Thanks,
>
>

Re: Using sensitive configuration/credentials

[vino yang <ya...@gmail.com>]

Hi Chesnay,

Oh, I did not know this feature. Any more description in Flink official
documentation?

Thanks, vino.

Chesnay Schepler <ch...@apache.org> 于2018年8月9日周四 下午4:29写道：

> If you change the name of your configuration key ti include "secret" or
> "password" it should be hidden from the logs and UI.
>
> On 09.08.2018 04:28, vino yang wrote:
>
> Hi Matt,
>
> Flink is currently enhancing its security, such as the current data
> transmission can be configured with SSL mode[1].
> However, some problems involving configuration and web ui display do
> exist, and they are still displayed in plain text.
> I think a temporary way to do this is to keep your secret configuration in
> encrypted form elsewhere, such as Zookeeper or RDBMS, and then dynamically
> read it into the job in a UDF (in the open method).
>
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/security-ssl.html
>
> Thanks, vino.
>
> Matt Moore <me...@mattdoescode.com> 于2018年8月9日周四 上午1:54写道：
>
>> I'm wondering what the best practice is for using secrets in a Flink
>> program, and I can't find any info in the docs or posted anywhere else.
>>
>> I need to store an access token to one of my APIs for flink to use to
>> dump results into, and right now I'm passing it through as a configuration
>> parameter, but that doesn't seem like the most secure thing to do and the
>> value shows up in the Flink Dashboard under Configuration which is less
>> than ideal.
>>
>> Has anyone else dealt with a situation like this?
>>
>> Thanks,
>>
>>
>

Re: Using sensitive configuration/credentials

[Chesnay Schepler <ch...@apache.org>]

If you change the name of your configuration key ti include "secret" or 
"password" it should be hidden from the logs and UI.

On 09.08.2018 04:28, vino yang wrote:
> Hi Matt,
>
> Flink is currently enhancing its security, such as the current data 
> transmission can be configured with SSL mode[1].
> However, some problems involving configuration and web ui display do 
> exist, and they are still displayed in plain text.
> I think a temporary way to do this is to keep your secret 
> configuration in encrypted form elsewhere, such as Zookeeper or RDBMS, 
> and then dynamically read it into the job in a UDF (in the open method).
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/security-ssl.html
>
> Thanks, vino.
>
> Matt Moore <me@mattdoescode.com <ma...@mattdoescode.com>> 
> 于2018年8月9日周四 上午1:54写道：
>
>     I'm wondering what the best practice is for using secrets in a
>     Flink program, and I can't find any info in the docs or posted
>     anywhere else.
>
>     I need to store an access token to one of my APIs for flink to use
>     to dump results into, and right now I'm passing it through as a
>     configuration parameter, but that doesn't seem like the most
>     secure thing to do and the value shows up in the Flink Dashboard
>     under Configuration which is less than ideal.
>
>     Has anyone else dealt with a situation like this?
>
>     Thanks,
>