You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "Sahu, Sarthak 1. (Nokia - IN/Bangalore)" <sa...@nokia.com> on 2018/03/22 09:16:49 UTC
Issue in Flink/Zookeeper authentication via Kerberos
Hi Folks,
Environment Setup:
1. I have configured KDC 5 server.
2. Configured Kerberos in zookeeper-3.4.10 wherein I can able to connect ZooKeeper Server/Client via Kerberos authentication.
3. Now flink-1.4.0 has configured for Kerberos authentication as per below instruction.
* https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html#kerberos-based-security
* https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html#kerberos-based-security-1
Success Scenario:
1. All Kerberos configuration parameter is correct and flink/zookeeper able to connect trough TGT.
Problem:
1. Even if wrong Kerberos credentials given, flink able to connect ZooKeeper.
Please find the taskmanager/jobmanger logs and flink config file for both scenario attached.
Hoping for quick resolution.
Regards
Sarthak Sahu
Re: Issue in Flink/Zookeeper authentication via Kerberos
Posted by Shuyi Chen <su...@gmail.com>.
Hi Sarthak,
Happy to help. Could you please share the jobmanager/taskmanager log and
flink conf again?
Also, Flink 1.4.0 has a regression on kerberos security (keytab path in
TaskManager is set incorrectly) , which is fixed on 1.4.1. (see
https://issues.apache.org/jira/browse/FLINK-8275)
Shuyi
On Mon, Apr 2, 2018 at 3:44 PM, Shuyi Chen <su...@gmail.com> wrote:
> Hi Sarthak,
>
> Happy to help. Could you please share the jobmanager/taskmanager log and
> flink conf again?
>
> Also, Flink 1.4.0 has a regression on kerberos security (keytab path in
> TaskManager is set incorrectly) , which is fixed on 1.4.1. (see
> https://issues.apache.org/jira/browse/FLINK-8275)
>
> Shuyi
>
> On Mon, Mar 26, 2018 at 2:35 AM, Timo Walther <tw...@apache.org> wrote:
>
>> Hi Sarthak,
>>
>> I'm not a Kerberos expert but maybe Eron or Shuyi are more familiar with
>> the details?
>>
>> Would be great if somebody could help.
>>
>> Thanks,
>> Timo
>>
>> Am 22.03.18 um 10:16 schrieb Sahu, Sarthak 1. (Nokia - IN/Bangalore):
>>
>> Hi Folks,
>>
>>
>>
>> *Environment Setup:*
>>
>> 1. I have configured KDC 5 server.
>> 2. Configured Kerberos in zookeeper-3.4.10 wherein I can able to
>> connect ZooKeeper Server/Client via Kerberos authentication.
>> 3. Now flink-1.4.0 has configured for Kerberos authentication as per
>> below instruction.
>>
>>
>> - https://ci.apache.org/projects/flink/flink-docs-release-1.4/
>> ops/config.html#kerberos-based-security
>> <https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html#kerberos-based-security>
>> - https://ci.apache.org/projects/flink/flink-docs-release-1.4/
>> ops/config.html#kerberos-based-security-1
>> <https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html#kerberos-based-security-1>
>>
>> *Success Scenario:*
>>
>> 1. All Kerberos configuration parameter is correct and
>> flink/zookeeper able to connect trough TGT.
>>
>> *Problem:*
>>
>> 1. Even if wrong Kerberos credentials given, flink able to connect
>> ZooKeeper.
>>
>>
>>
>> Please find the taskmanager/jobmanger logs and flink config file for both
>> scenario attached.
>>
>>
>>
>> Hoping for quick resolution.
>>
>>
>>
>> Regards
>>
>> Sarthak Sahu
>>
>>
>>
>>
>>
>
>
> --
> "So you have to trust that the dots will somehow connect in your future."
>
--
"So you have to trust that the dots will somehow connect in your future."
Re: Issue in Flink/Zookeeper authentication via Kerberos
Posted by Eron Wright <er...@gmail.com>.
I believe that the solution here is to ensure that the znodes created by
Flink have an ACL that allows access only to the original creator. For
example, if a given Flink job has a Kerberos identity of "user1@example.com",
it should set the znode ACL appropriately to disallow access to any client
that doesn't successfully authenticate as that user. This may be
accomplished with the following Flink configuration setting:
high-availability.zookeeper.client.acl: creator
Some code links:
-
https://github.com/apache/flink/blob/release-1.4.2/flink-core/src/main/java/org/apache/flink/configuration/HighAvailabilityOptions.java#L171
-
https://github.com/apache/flink/blob/release-1.4.2/flink-runtime/src/main/java/org/apache/flink/runtime/util/ZooKeeperUtils.java#L93
Hope this helps!
Eron
On Sun, Apr 15, 2018 at 2:16 AM, Sahu, Sarthak 1. (Nokia - IN/Bangalore) <
sarthak.1.sahu@nokia.com> wrote:
> Glad to get the reply. With wrong Kerberos information I am expecting an
> ‘access denied’.
>
>
>
> As per flink log, it clear states that authentication failed due to
> Kerberos wrong information and trying to connect with zookeeper with
> unauthorised mode if zookeeper allows.
>
> And then it connected successfully!
>
>
>
> Do I missing any configuration in flink/zookeeper side.
>
> Expecting you suggestion here.
>
>
>
> Regards
>
> Sarthak Sahu
>
>
>
> *From:* Eron Wright [mailto:eronwright@gmail.com]
> *Sent:* Tuesday, April 3, 2018 3:07 AM
> *To:* Sahu, Sarthak 1. (Nokia - IN/Bangalore) <sa...@nokia.com>
> *Cc:* suez1224@gmail.com; Timo Walther <tw...@apache.org>
>
> *Subject:* Re: Issue in Flink/Zookeeper authentication via Kerberos
>
>
>
> Hello, I'm happy to help. Could you elaborate on the issue that you see?
> Are you saying that you expect to get 'access denied' but Zookeeper is
> allowing the connection anyway?
>
>
>
> My first thought is, maybe ZK allows unauthenticated connections but
> relies on the authorization layer to deny access to nodes based on the
> ACL. FLink has a configuration setting to set the 'owner' of the znode.
>
>
>
> -Eron
>
>
>
> On Mon, Apr 2, 2018 at 1:50 AM, Sahu, Sarthak 1. (Nokia - IN/Bangalore) <
> sarthak.1.sahu@nokia.com> wrote:
>
> Hi Eron/Shuyi
>
>
>
> Could you please help me on this below issue.
>
>
>
> Regards
>
> Sarthak Sahu
>
>
>
> *From:* Timo Walther [mailto:twalthr@apache.org]
> *Sent:* Monday, March 26, 2018 3:05 PM
> *To:* user@flink.apache.org
> *Cc:* eronwright@gmail.com; suez1224@gmail.com
> *Subject:* Re: Issue in Flink/Zookeeper authentication via Kerberos
>
>
>
> Hi Sarthak,
>
> I'm not a Kerberos expert but maybe Eron or Shuyi are more familiar with
> the details?
>
> Would be great if somebody could help.
>
> Thanks,
> Timo
>
> Am 22.03.18 um 10:16 schrieb Sahu, Sarthak 1. (Nokia - IN/Bangalore):
>
> Hi Folks,
>
>
>
> *Environment Setup:*
>
> 1. I have configured KDC 5 server.
> 2. Configured Kerberos in zookeeper-3.4.10 wherein I can able to
> connect ZooKeeper Server/Client via Kerberos authentication.
> 3. Now flink-1.4.0 has configured for Kerberos authentication as per
> below instruction.
>
> · https://ci.apache.org/projects/flink/flink-docs-
> release-1.4/ops/config.html#kerberos-based-security
>
> · https://ci.apache.org/projects/flink/flink-docs-
> release-1.4/ops/config.html#kerberos-based-security-1
>
> *Success Scenario:*
>
> 1. All Kerberos configuration parameter is correct and flink/zookeeper
> able to connect trough TGT.
>
> *Problem:*
>
> 1. Even if wrong Kerberos credentials given, flink able to connect
> ZooKeeper.
>
>
>
> Please find the taskmanager/jobmanger logs and flink config file for both
> scenario attached.
>
>
>
> Hoping for quick resolution.
>
>
>
> Regards
>
> Sarthak Sahu
>
>
>
>
>
>
>
Re: Issue in Flink/Zookeeper authentication via Kerberos
Posted by Timo Walther <tw...@apache.org>.
Hi Sarthak,
I'm not a Kerberos expert but maybe Eron or Shuyi are more familiar with
the details?
Would be great if somebody could help.
Thanks,
Timo
Am 22.03.18 um 10:16 schrieb Sahu, Sarthak 1. (Nokia - IN/Bangalore):
>
> Hi Folks,
>
> *_Environment Setup:_*
>
> 1. I have configured KDC 5 server.
> 2. Configured Kerberos in zookeeper-3.4.10 wherein I can able to
> connect ZooKeeper Server/Client via Kerberos authentication.
> 3. Now flink-1.4.0 has configured for Kerberos authentication as per
> below instruction.
>
> * https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html#kerberos-based-security
> * https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html#kerberos-based-security-1
>
> *_Success Scenario:_*
>
> 1. All Kerberos configuration parameter is correct and
> flink/zookeeper able to connect trough TGT.
>
> *_Problem:_*
>
> 1. Even if wrong Kerberos credentials given, flink able to connect
> ZooKeeper.*__*
>
> *__*
>
> Please find the taskmanager/jobmanger logs and flink config file for
> both scenario attached.
>
> Hoping for quick resolution.
>
> Regards
>
> Sarthak Sahu
>