You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "LINZ, Arnaud" <AL...@bouyguestelecom.fr> on 2022/11/22 16:18:22 UTC
"Authentication failed" in "ConnectionState" when enabling internal SSL on Yarn with self signed certificate
Hello,
I use Flink 1.11.2 in Yarn cluster mode.
I’ve followed the instructions listed here (https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/security/security-ssl/ <https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/security/security-ssl/%20> ) to turn on internal SSL:
$ keytool -genkeypair \
-alias flink.internal \
-keystore internal.keystore \
-dname "CN=flink.internal" \
-storepass internal_store_password \
-keyalg RSA \
-keysize 4096 \
-storetype PKCS12
security.ssl.internal.enabled: true
security.ssl.internal.keystore: /path/to/flink/conf/internal.keystore
security.ssl.internal.truststore: /path/to/flink/conf/internal.keystore
security.ssl.internal.keystore-password: internal_store_password
security.ssl.internal.truststore-password: internal_store_password
security.ssl.internal.key-password: internal_store_password
I’ve shipped the keystore on every node, and get no error from keystore reading.
However the application fails to start (stuck in initializing step), with the only error log in Yarn containers :
15:49:46.397 [main-EventThread] ERROR org.apache.flink.shaded.curator4.org.apache.curator.ConnectionState - Authentication failed
Could you please explain me what this “zookeeper” curator connection does and why it no longer works when enabling internal SSL ?
Best regards,
Arnaud
________________________________
L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur.
The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
RE: "Authentication failed" in "ConnectionState" when enabling internal SSL on Yarn with self signed certificate
Posted by "LINZ, Arnaud" <AL...@bouyguestelecom.fr>.
Last update :
My flink version is 1.14.3 in fact. The application works when enabling internal SSL in “local” intra-jvm cluster mode, so the certificate seems correct.
I see no log in Yarn server side, only that the application get killed.
I will try to take stack traces…
De : LINZ, Arnaud
Envoyé : mardi 22 novembre 2022 17:41
À : user <us...@flink.apache.org>
Objet : RE: "Authentication failed" in "ConnectionState" when enabling internal SSL on Yarn with self signed certificate
Update :
In fact this « Authentication failed” message also appears when SSL is turned off (and when the yarn application succeeds), so it’s more of a warning and has no link with the “freeze” when SSL is turned on.
Thus, when internal SSL is enabled, I have no error in the yarn log, and the only error I get is a “timed out error” like the one you get when you don’t have enough ressources :
(NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout)
But I do have enough resources.
De : LINZ, Arnaud
Envoyé : mardi 22 novembre 2022 17:18
À : user <us...@flink.apache.org>>
Objet : "Authentication failed" in "ConnectionState" when enabling internal SSL on Yarn with self signed certificate
Hello,
I use Flink 1.14.3 in Yarn cluster mode.
I’ve followed the instructions listed here (https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/security/security-ssl/ <https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/security/security-ssl/%20> ) to turn on internal SSL:
$ keytool -genkeypair \
-alias flink.internal \
-keystore internal.keystore \
-dname "CN=flink.internal" \
-storepass internal_store_password \
-keyalg RSA \
-keysize 4096 \
-storetype PKCS12
security.ssl.internal.enabled: true
security.ssl.internal.keystore: /path/to/flink/conf/internal.keystore
security.ssl.internal.truststore: /path/to/flink/conf/internal.keystore
security.ssl.internal.keystore-password: internal_store_password
security.ssl.internal.truststore-password: internal_store_password
security.ssl.internal.key-password: internal_store_password
I’ve shipped the keystore on every node, and get no error from keystore reading.
However the application fails to start (stuck in initializing step), with the only error log in Yarn containers :
15:49:46.397 [main-EventThread] ERROR org.apache.flink.shaded.curator4.org.apache.curator.ConnectionState - Authentication failed
Could you please explain me what this “zookeeper” curator connection does and why it no longer works when enabling internal SSL ?
Best regards,
Arnaud
________________________________
L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur.
The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
RE: "Authentication failed" in "ConnectionState" when enabling internal SSL on Yarn with self signed certificate
Posted by "LINZ, Arnaud" <AL...@bouyguestelecom.fr>.
Update :
In fact this « Authentication failed” message also appears when SSL is turned off (and when the yarn application succeeds), so it’s more of a warning and has no link with the “freeze” when SSL is turned on.
Thus, when internal SSL is enabled, I have no error in the yarn log, and the only error I get is a “timed out error” like the one you get when you don’t have enough ressources :
(NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout)
But I do have enough resources.
De : LINZ, Arnaud
Envoyé : mardi 22 novembre 2022 17:18
À : user <us...@flink.apache.org>
Objet : "Authentication failed" in "ConnectionState" when enabling internal SSL on Yarn with self signed certificate
Hello,
I use Flink 1.11.2 in Yarn cluster mode.
I’ve followed the instructions listed here (https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/security/security-ssl/ <https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/security/security-ssl/%20> ) to turn on internal SSL:
$ keytool -genkeypair \
-alias flink.internal \
-keystore internal.keystore \
-dname "CN=flink.internal" \
-storepass internal_store_password \
-keyalg RSA \
-keysize 4096 \
-storetype PKCS12
security.ssl.internal.enabled: true
security.ssl.internal.keystore: /path/to/flink/conf/internal.keystore
security.ssl.internal.truststore: /path/to/flink/conf/internal.keystore
security.ssl.internal.keystore-password: internal_store_password
security.ssl.internal.truststore-password: internal_store_password
security.ssl.internal.key-password: internal_store_password
I’ve shipped the keystore on every node, and get no error from keystore reading.
However the application fails to start (stuck in initializing step), with the only error log in Yarn containers :
15:49:46.397 [main-EventThread] ERROR org.apache.flink.shaded.curator4.org.apache.curator.ConnectionState - Authentication failed
Could you please explain me what this “zookeeper” curator connection does and why it no longer works when enabling internal SSL ?
Best regards,
Arnaud
________________________________
L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur.
The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.