You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Ada Wong (Jira)" <ji...@apache.org> on 2022/01/27 03:59:00 UTC
[jira] [Updated] (FLINK-25834) 'flink run' command can not use 'pipeline.classpaths' in flink-conf.yaml
[ https://issues.apache.org/jira/browse/FLINK-25834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ada Wong updated FLINK-25834:
-----------------------------
Description:
When we use 'flink run' or CliFrontend class to submit job. If not set -C/-classpaths, it disable 'pipeline.classpaths'.
Example:
flink-conf.yaml content :
{code:java}
pipeline.classpaths: file:///opt/flink-1.14.2/other/flink-sql-connector-elasticsearch7_2.12-1.14.2.jar{code}
submit command:
{code:java}
bin/flink run /flink14-sql/target/flink14-sql-1.0-SNAPSHOT-jar-with-dependencies.jar{code}
It will throw elasticsearch7 class not found exception.
There are two reasons for this:
# ProgramOptions#applyToConfiguration will use a list which size is zero to overwrite 'pipeline.classpaths' value in configuration.
# ProgramOptions#buildProgram do not set 'pipeline.classpaths' into PackagedProgram.
To solve the 1) problem, could we add a directly return judgement when list size is zero in ConfigUtils#encodeCollectionToConfig()
To solve the 2) problem, could we append 'pipeline.classpaths' values into classpaths and pass setUserClassPaths together.
was:
When we use 'flink run' or CliFrontend class to submit job. If not set -C/-classpaths, it disable 'pipeline.classpaths'.
Example:
flink-conf.yaml content :
{code:java}
pipeline.classpaths: file:///opt/flink-1.14.2/other/flink-sql-connector-elasticsearch7_2.12-1.14.2.jar{code}
submit command:
{code:java}
bin/flink run /flink14-sql/target/flink14-sql-1.0-SNAPSHOT-jar-with-dependencies.jar{code}
it will throw elasticsearch7 class not found exception.
There are two reasons for this:
# ProgramOptions#applyToConfiguration will use a list which size is zero to overwrite 'pipeline.classpaths' value in configuration.
# ProgramOptions#buildProgram do not set 'pipeline.classpaths' into PackagedProgram.
To solve the 1) problem, could we add a directly return judgement when list size is zero in ConfigUtils#encodeCollectionToConfig()
To solve the 2) problem, could we append 'pipeline.classpaths' values into classpaths and pass setUserClassPaths together.
> 'flink run' command can not use 'pipeline.classpaths' in flink-conf.yaml
> ------------------------------------------------------------------------
>
> Key: FLINK-25834
> URL: https://issues.apache.org/jira/browse/FLINK-25834
> Project: Flink
> Issue Type: Bug
> Components: Client / Job Submission, Command Line Client
> Affects Versions: 1.14.3
> Reporter: Ada Wong
> Priority: Major
>
> When we use 'flink run' or CliFrontend class to submit job. If not set -C/-classpaths, it disable 'pipeline.classpaths'.
> Example:
> flink-conf.yaml content :
> {code:java}
> pipeline.classpaths: file:///opt/flink-1.14.2/other/flink-sql-connector-elasticsearch7_2.12-1.14.2.jar{code}
> submit command:
> {code:java}
> bin/flink run /flink14-sql/target/flink14-sql-1.0-SNAPSHOT-jar-with-dependencies.jar{code}
> It will throw elasticsearch7 class not found exception.
> There are two reasons for this:
> # ProgramOptions#applyToConfiguration will use a list which size is zero to overwrite 'pipeline.classpaths' value in configuration.
> # ProgramOptions#buildProgram do not set 'pipeline.classpaths' into PackagedProgram.
> To solve the 1) problem, could we add a directly return judgement when list size is zero in ConfigUtils#encodeCollectionToConfig()
> To solve the 2) problem, could we append 'pipeline.classpaths' values into classpaths and pass setUserClassPaths together.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)