You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "SunShun (Jira)" <ji...@apache.org> on 2022/06/15 04:45:00 UTC
[jira] [Updated] (FLINK-28069) Cannot attach SSL JKS file for Kafka connector
[ https://issues.apache.org/jira/browse/FLINK-28069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
SunShun updated FLINK-28069:
----------------------------
Description:
Hi, I intend to connect to a SSL enabled Kafka, which require to attach JKS file for truststore and keystore, then I am trying to use the keyword {*}-yt, yarnship{*}, to pass the JKS files, and it's expected to find the find under the classpath of job manager or task manager.
However, the FileNotFound exception is given when the job initiates, saying that the JKS files is not found from job manager.
val truststore_path = "client.truststore.jks"
val keystore_path = "client.keystore.jks"
val source_ddl =
s"""
CREATE TABLE source_table(
`id` BIGINT,
`time` TIMESTAMP(3)
) WITH (
'connector' = 'kafka',
...
'properties.security.protocol' = 'SSL',
'properties.ssl.truststore.location' = '$truststore_path',
'properties.ssl.truststore.password' = '$truststore_pwd',
'properties.ssl.keystore.location' = '$keystore_path',
'properties.ssl.keystore.password' = '$keystore_pwd',
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
)
"""
I am using the session mode to launch the job as below:
./bin/yarn-session.sh --detached
./bin/flink run ./job/finance-libra-stats-1.0.jar -yt ./client.keystore.jks ./client.truststore.jks
Anyone can help tell what's the best practice for such case, and any mistake I meet during the process.
was:
Hi, I intend to connect to a SSL enabled Kafka, which require to attach JKS file for truststore and keystore, then I am trying to use the keyword {*}-yt, yarnship{*}, to pass the JKS files, and it's expected to find the find under the classpath of job manager or task manager.
However, the FileNotFound exception is given when the job initiates, saying that the JKS files is not found from job manager.
```
val truststore_path = "client.truststore.jks"
val keystore_path = "client.keystore.jks"
val source_ddl =
s"""
CREATE TABLE source_table(
`id` BIGINT,
`time` TIMESTAMP(3)
) WITH (
'connector' = 'kafka',
...
'properties.security.protocol' = 'SSL',
'properties.ssl.truststore.location' = '$truststore_path',
'properties.ssl.truststore.password' = '$truststore_pwd',
'properties.ssl.keystore.location' = '$keystore_path',
'properties.ssl.keystore.password' = '$keystore_pwd',
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
)
"""
```
Anyone can help tell what's the best practice for such case, and any mistake I meet during the process.
> Cannot attach SSL JKS file for Kafka connector
> ----------------------------------------------
>
> Key: FLINK-28069
> URL: https://issues.apache.org/jira/browse/FLINK-28069
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.14.4
> Environment: I am using the session mode to launch the job as below:
> ```
> ./bin/yarn-session.sh --detached
> ./bin/flink run ./job/finance-libra-stats-1.0.jar -yt ./client.keystore.jks ./client.truststore.jks
> ```
> Reporter: SunShun
> Priority: Blocker
>
> Hi, I intend to connect to a SSL enabled Kafka, which require to attach JKS file for truststore and keystore, then I am trying to use the keyword {*}-yt, yarnship{*}, to pass the JKS files, and it's expected to find the find under the classpath of job manager or task manager.
> However, the FileNotFound exception is given when the job initiates, saying that the JKS files is not found from job manager.
>
> val truststore_path = "client.truststore.jks"
> val keystore_path = "client.keystore.jks"
> val source_ddl =
> s"""
> CREATE TABLE source_table(
> `id` BIGINT,
> `time` TIMESTAMP(3)
> ) WITH (
> 'connector' = 'kafka',
> ...
> 'properties.security.protocol' = 'SSL',
> 'properties.ssl.truststore.location' = '$truststore_path',
> 'properties.ssl.truststore.password' = '$truststore_pwd',
> 'properties.ssl.keystore.location' = '$keystore_path',
> 'properties.ssl.keystore.password' = '$keystore_pwd',
> 'scan.startup.mode' = 'latest-offset',
> 'format' = 'json'
> )
> """
> I am using the session mode to launch the job as below:
> ./bin/yarn-session.sh --detached
> ./bin/flink run ./job/finance-libra-stats-1.0.jar -yt ./client.keystore.jks ./client.truststore.jks
>
> Anyone can help tell what's the best practice for such case, and any mistake I meet during the process.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)