You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Marke Builder <ma...@gmail.com> on 2018/12/07 09:55:07 UTC
Flink with Docker: docker-compose and FLINK_JOB_ARGUMENT exception
Hi,
I'm trying to run flink with docker (docker-compose) and job arguments
"config-dev.properties". But it seams that the job arguments are not
available:
docker-compose.yml
version: '2'
services:
job-cluster:
image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
ports:
- '8081:8081'
command: job-cluster --job-classname
-Djobmanager.rpc.address=job-cluster -Dparallelism.default=1 --config
config-dev.properties
taskmanager:
image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
command: task-manager -Djobmanager.rpc.address=job-cluster
scale: 1
Excpetion:
4:32 AMorg.apache.flink.runtime.entrypoint.FlinkParseException: Failed to
parse the command line arguments.
12/7/2018 10:44:32 AM at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
12/7/2018 10:44:32 AM at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
12/7/2018 10:44:32 AMCaused by:
org.apache.commons.cli.MissingArgumentException: Missing argument for
option: j
12/7/2018 10:44:32 AM at
org.apache.commons.cli.DefaultParser.checkRequiredArgs(DefaultParser.java:211)
12/7/2018 10:44:32 AM at
org.apache.commons.cli.DefaultParser.handleOption(DefaultParser.java:599)
12/7/2018 10:44:32 AM at
org.apache.commons.cli.DefaultParser.handleShortAndLongOption(DefaultParser.java:548)
12/7/2018 10:44:32 AM at
org.apache.commons.cli.DefaultParser.handleToken(DefaultParser.java:243)
12/7/2018 10:44:32 AM at
org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:120)
12/7/2018 10:44:32 AM at
org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
12/7/2018 10:44:32 AM at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
12/7/2018 10:44:32 AM ... 1 more
12/7/2018 10:44:32 AMException in thread "main"
java.lang.NoSuchMethodError:
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp()V
12/7/2018 10:44:32 AM at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
Re: Flink with Docker: docker-compose and FLINK_JOB_ARGUMENT exception
Posted by Jeff Zhang <zj...@gmail.com>.
I didn't use the built-in docker of flink, but the following flink docker
works for me pretty well.
https://github.com/big-data-europe/docker-flink
Piotr Nowojski <pi...@data-artisans.com> 于2018年12月7日周五 下午6:20写道：
> Hi,
>
> I have never used flink and docker together, so I’m not sure if I will be
> able to help, however have you seen this README:
> https://github.com/apache/flink/tree/master/flink-container/docker
> ?
> Shouldn’t you be passing your arguments via `FLINK_JOB_ARGUMENTS`
> environment variable?
>
> Piotrek
>
> On 7 Dec 2018, at 10:55, Marke Builder <ma...@gmail.com> wrote:
>
> Hi,
>
> I'm trying to run flink with docker (docker-compose) and job arguments
> "config-dev.properties". But it seams that the job arguments are not
> available:
>
> docker-compose.yml
>
> version: '2'
> services:
> job-cluster:
> image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
> ports:
> - '8081:8081'
> command: job-cluster --job-classname
> -Djobmanager.rpc.address=job-cluster -Dparallelism.default=1 --config
> config-dev.properties
>
> taskmanager:
> image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
> command: task-manager -Djobmanager.rpc.address=job-cluster
> scale: 1
>
>
> Excpetion:
> 4:32 AMorg.apache.flink.runtime.entrypoint.FlinkParseException: Failed to
> parse the command line arguments.
> 12/7/2018 10:44:32 AM at
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
> 12/7/2018 10:44:32 AM at
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
> 12/7/2018 10:44:32 AMCaused by:
> org.apache.commons.cli.MissingArgumentException: Missing argument for
> option: j
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.checkRequiredArgs(DefaultParser.java:211)
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.handleOption(DefaultParser.java:599)
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.handleShortAndLongOption(DefaultParser.java:548)
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.handleToken(DefaultParser.java:243)
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:120)
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
> 12/7/2018 10:44:32 AM at
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
> 12/7/2018 10:44:32 AM ... 1 more
> 12/7/2018 10:44:32 AMException in thread "main"
> java.lang.NoSuchMethodError:
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp()V
> 12/7/2018 10:44:32 AM at
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
>
>
>
--
Best Regards
Jeff Zhang
Re: Flink with Docker: docker-compose and FLINK_JOB_ARGUMENT exception
Posted by Spico Florin <sp...@gmail.com>.
Hello!
I have used with success Flink with Docker, the approach presented in this
article:
http://diegoreico.com/environments/runningflinkgclusterwithzeppelin/
It it using a older version of Flink (1.4.2) in order to be compatible with
Zeppelin client, but you can check if it works with new versions.
I hope it helps.
Regards,
Florin
On Fri, Dec 7, 2018 at 12:20 PM Piotr Nowojski <pi...@data-artisans.com>
wrote:
> Hi,
>
> I have never used flink and docker together, so I’m not sure if I will be
> able to help, however have you seen this README:
> https://github.com/apache/flink/tree/master/flink-container/docker
> ?
> Shouldn’t you be passing your arguments via `FLINK_JOB_ARGUMENTS`
> environment variable?
>
> Piotrek
>
> On 7 Dec 2018, at 10:55, Marke Builder <ma...@gmail.com> wrote:
>
> Hi,
>
> I'm trying to run flink with docker (docker-compose) and job arguments
> "config-dev.properties". But it seams that the job arguments are not
> available:
>
> docker-compose.yml
>
> version: '2'
> services:
> job-cluster:
> image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
> ports:
> - '8081:8081'
> command: job-cluster --job-classname
> -Djobmanager.rpc.address=job-cluster -Dparallelism.default=1 --config
> config-dev.properties
>
> taskmanager:
> image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
> command: task-manager -Djobmanager.rpc.address=job-cluster
> scale: 1
>
>
> Excpetion:
> 4:32 AMorg.apache.flink.runtime.entrypoint.FlinkParseException: Failed to
> parse the command line arguments.
> 12/7/2018 10:44:32 AM at
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
> 12/7/2018 10:44:32 AM at
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
> 12/7/2018 10:44:32 AMCaused by:
> org.apache.commons.cli.MissingArgumentException: Missing argument for
> option: j
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.checkRequiredArgs(DefaultParser.java:211)
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.handleOption(DefaultParser.java:599)
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.handleShortAndLongOption(DefaultParser.java:548)
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.handleToken(DefaultParser.java:243)
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:120)
> 12/7/2018 10:44:32 AM at
> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
> 12/7/2018 10:44:32 AM at
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
> 12/7/2018 10:44:32 AM ... 1 more
> 12/7/2018 10:44:32 AMException in thread "main"
> java.lang.NoSuchMethodError:
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp()V
> 12/7/2018 10:44:32 AM at
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
>
>
>
Re: Flink with Docker: docker-compose and FLINK_JOB_ARGUMENT
exception
Posted by Piotr Nowojski <pi...@data-artisans.com>.
Hi,
I have never used flink and docker together, so I’m not sure if I will be able to help, however have you seen this README:
https://github.com/apache/flink/tree/master/flink-container/docker
?
Shouldn’t you be passing your arguments via `FLINK_JOB_ARGUMENTS` environment variable?
Piotrek
> On 7 Dec 2018, at 10:55, Marke Builder <ma...@gmail.com> wrote:
>
> Hi,
>
> I'm trying to run flink with docker (docker-compose) and job arguments "config-dev.properties". But it seams that the job arguments are not available:
>
> docker-compose.yml
>
> version: '2'
> services:
> job-cluster:
> image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
> ports:
> - '8081:8081'
> command: job-cluster --job-classname -Djobmanager.rpc.address=job-cluster -Dparallelism.default=1 --config config-dev.properties
>
> taskmanager:
> image: ${FLINK_DOCKER_IMAGE_NAME:-timeseries-v1}
> command: task-manager -Djobmanager.rpc.address=job-cluster
> scale: 1
>
>
> Excpetion:
> <>4:32 AMorg.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse the command line arguments.
> 12/7/2018 10:44:32 AM at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
> 12/7/2018 10:44:32 AM at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
> 12/7/2018 10:44:32 AMCaused by: org.apache.commons.cli.MissingArgumentException: Missing argument for option: j
> 12/7/2018 10:44:32 AM at org.apache.commons.cli.DefaultParser.checkRequiredArgs(DefaultParser.java:211)
> 12/7/2018 10:44:32 AM at org.apache.commons.cli.DefaultParser.handleOption(DefaultParser.java:599)
> 12/7/2018 10:44:32 AM at org.apache.commons.cli.DefaultParser.handleShortAndLongOption(DefaultParser.java:548)
> 12/7/2018 10:44:32 AM at org.apache.commons.cli.DefaultParser.handleToken(DefaultParser.java:243)
> 12/7/2018 10:44:32 AM at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:120)
> 12/7/2018 10:44:32 AM at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
> 12/7/2018 10:44:32 AM at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
> 12/7/2018 10:44:32 AM ... 1 more
> 12/7/2018 10:44:32 AMException in thread "main" java.lang.NoSuchMethodError: org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp()V
> 12/7/2018 10:44:32 AM at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)