You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Hao Sun <ha...@zendesk.com> on 2018/08/24 22:37:00 UTC
anybody can start flink with job mode?
I got an error like this.
$ docker run -it flink-job:latest job-cluster
Starting the job-cluster
config file:
jobmanager.rpc.address: localhost
jobmanager.rpc.port: 6123
jobmanager.heap.size: 1024m
taskmanager.heap.size: 1024m
taskmanager.numberOfTaskSlots: 1
parallelism.default: 1
rest.port: 8081
Starting standalonejob as a console application on host cf9bd047082c.
2018-08-24 22:33:00,773 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
--------------------------------------------------------------------------------
2018-08-24 22:33:00,774 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting
StandaloneJobClusterEntryPoint (Version: 1.6.0, Rev:ff472b4,
Date:07.08.2018 @ 13:31:13 UTC)
2018-08-24 22:33:00,775 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - OS current user:
flink
2018-08-24 22:33:01,168 WARN org.apache.hadoop.util.NativeCodeLoader -
Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
2018-08-24 22:33:01,232 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Current
Hadoop/Kerberos user: flink
2018-08-24 22:33:01,232 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM: OpenJDK 64-Bit
Server VM - Oracle Corporation - 1.8/25.111-b14
2018-08-24 22:33:01,232 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Maximum heap size:
981 MiBytes
2018-08-24 22:33:01,232 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JAVA_HOME:
/usr/lib/jvm/java-1.8-openjdk/jre
2018-08-24 22:33:01,236 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Hadoop version:
2.8.3
2018-08-24 22:33:01,236 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM Options:
2018-08-24 22:33:01,236 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xms1024m
2018-08-24 22:33:01,236 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xmx1024m
2018-08-24 22:33:01,237 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
-Dlog4j.configuration=file:/opt/flink-1.6.0/conf/log4j-console.properties
2018-08-24 22:33:01,237 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
-Dlogback.configurationFile=file:/opt/flink-1.6.0/conf/logback-console.xml
2018-08-24 22:33:01,237 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Program Arguments:
2018-08-24 22:33:01,237 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - --configDir
2018-08-24 22:33:01,238 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
/opt/flink-1.6.0/conf
2018-08-24 22:33:01,238 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Classpath:
/opt/flink-1.6.0/lib/flink-python_2.11-1.6.0.jar:/opt/flink-1.6.0/lib/flink-shaded-hadoop2-uber-1.6.0.jar:/opt/flink-1.6.0/lib/job.jar:/opt/flink-1.6.0/lib/log4j-1.2.17.jar:/opt/flink-1.6.0/lib/slf4j-log4j12-1.7.7.jar:/opt/flink-1.6.0/lib/flink-dist_2.11-1.6.0.jar:::
2018-08-24 22:33:01,238 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
--------------------------------------------------------------------------------
2018-08-24 22:33:01,240 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Registered UNIX
signal handlers for [TERM, HUP, INT]
2018-08-24 22:33:01,248 ERROR
*org.apache.flink.runtime.entrypoint.ClusterEntrypoint
- Could not parse command line arguments [--configDir,
/opt/flink-1.6.0/conf].*
org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse
the command line arguments.
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
Caused by: org.apache.commons.cli.MissingOptionException: *Missing required
option: j*
at
org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
... 1 more
Exception in thread "main" java.lang.IllegalArgumentException:
cmdLineSyntax not provided
at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:546)
at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:492)
at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:408)
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp(CommandLineParser.java:60)
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
Re: anybody can start flink with job mode?
Posted by Till Rohrmann <tr...@apache.org>.
Great to hear :-)
On Tue, Aug 28, 2018, 14:59 Hao Sun <ha...@zendesk.com> wrote:
> Thanks Till for the follow up, I can run my job now.
>
> On Tue, Aug 28, 2018, 00:57 Till Rohrmann <tr...@apache.org> wrote:
>
>> Hi Hao,
>>
>> Vino is right, you need to specify the -j/--job-classname option which
>> specifies the job name you want to execute. Please make sure that the jar
>> containing this class is on the class path.
>>
>> I recently pushed some fixes which generate a better error message than
>> the one you've received. If you check out the latest master branch, then it
>> should work better.
>>
>> Let me know if you should run into other problems.
>>
>> Cheers,
>> Till
>>
>> On Sat, Aug 25, 2018 at 5:11 AM Hao Sun <ha...@zendesk.com> wrote:
>>
>>> Thanks, I'll look into it.
>>>
>>> On Fri, Aug 24, 2018, 19:44 vino yang <ya...@gmail.com> wrote:
>>>
>>>> Hi Hao Sun,
>>>>
>>>> From the error log, it seems that the jar package for the job was not
>>>> found.
>>>> You must make sure your Jar is in the classpath.
>>>> Related documentation may not be up-to-date, and there is a discussion
>>>> on this issue on this mailing list. [1]
>>>>
>>>> I see that the status of FLINK-10001 [2] is closed and it will be
>>>> updated with the release of 1.6.1 and 1.7.0.
>>>>
>>>> [1]:
>>>> http://mail-archives.apache.org/mod_mbox/flink-dev/201808.mbox/%3CCAC27z=OaohMbmcryB-+m3GBmZP=XPHa8miHv7ZS1gRGSEkkfdg@mail.gmail.com%3E
>>>> [2]: https://issues.apache.org/jira/browse/FLINK-10001
>>>>
>>>> Thanks, vino.
>>>>
>>>>
>>>> Hao Sun <ha...@zendesk.com> 于2018年8月25日周六 上午6:37写道：
>>>>
>>>>> I got an error like this.
>>>>>
>>>>> $ docker run -it flink-job:latest job-cluster
>>>>> Starting the job-cluster
>>>>> config file:
>>>>> jobmanager.rpc.address: localhost
>>>>> jobmanager.rpc.port: 6123
>>>>> jobmanager.heap.size: 1024m
>>>>> taskmanager.heap.size: 1024m
>>>>> taskmanager.numberOfTaskSlots: 1
>>>>> parallelism.default: 1
>>>>> rest.port: 8081
>>>>> Starting standalonejob as a console application on host cf9bd047082c.
>>>>> 2018-08-24 22:33:00,773 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>>>> --------------------------------------------------------------------------------
>>>>> 2018-08-24 22:33:00,774 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting
>>>>> StandaloneJobClusterEntryPoint (Version: 1.6.0, Rev:ff472b4,
>>>>> Date:07.08.2018 @ 13:31:13 UTC)
>>>>> 2018-08-24 22:33:00,775 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - OS current user:
>>>>> flink
>>>>> 2018-08-24 22:33:01,168 WARN org.apache.hadoop.util.NativeCodeLoader -
>>>>> Unable to load native-hadoop library for your platform... using
>>>>> builtin-java classes where applicable
>>>>> 2018-08-24 22:33:01,232 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Current
>>>>> Hadoop/Kerberos user: flink
>>>>> 2018-08-24 22:33:01,232 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM: OpenJDK 64-Bit
>>>>> Server VM - Oracle Corporation - 1.8/25.111-b14
>>>>> 2018-08-24 22:33:01,232 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Maximum heap size:
>>>>> 981 MiBytes
>>>>> 2018-08-24 22:33:01,232 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JAVA_HOME:
>>>>> /usr/lib/jvm/java-1.8-openjdk/jre
>>>>> 2018-08-24 22:33:01,236 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Hadoop version:
>>>>> 2.8.3
>>>>> 2018-08-24 22:33:01,236 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM Options:
>>>>> 2018-08-24 22:33:01,236 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xms1024m
>>>>> 2018-08-24 22:33:01,236 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xmx1024m
>>>>> 2018-08-24 22:33:01,237 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>>>> -Dlog4j.configuration=file:/opt/flink-1.6.0/conf/log4j-console.properties
>>>>> 2018-08-24 22:33:01,237 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>>>> -Dlogback.configurationFile=file:/opt/flink-1.6.0/conf/logback-console.xml
>>>>> 2018-08-24 22:33:01,237 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Program Arguments:
>>>>> 2018-08-24 22:33:01,237 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - --configDir
>>>>> 2018-08-24 22:33:01,238 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>>>> /opt/flink-1.6.0/conf
>>>>> 2018-08-24 22:33:01,238 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Classpath:
>>>>> /opt/flink-1.6.0/lib/flink-python_2.11-1.6.0.jar:/opt/flink-1.6.0/lib/flink-shaded-hadoop2-uber-1.6.0.jar:/opt/flink-1.6.0/lib/job.jar:/opt/flink-1.6.0/lib/log4j-1.2.17.jar:/opt/flink-1.6.0/lib/slf4j-log4j12-1.7.7.jar:/opt/flink-1.6.0/lib/flink-dist_2.11-1.6.0.jar:::
>>>>> 2018-08-24 22:33:01,238 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>>>> --------------------------------------------------------------------------------
>>>>> 2018-08-24 22:33:01,240 INFO
>>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Registered UNIX
>>>>> signal handlers for [TERM, HUP, INT]
>>>>> 2018-08-24 22:33:01,248 ERROR *org.apache.flink.runtime.entrypoint.ClusterEntrypoint
>>>>> - Could not parse command line arguments [--configDir,
>>>>> /opt/flink-1.6.0/conf].*
>>>>> org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to
>>>>> parse the command line arguments.
>>>>> at
>>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
>>>>> at
>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
>>>>> Caused by: org.apache.commons.cli.MissingOptionException: *Missing
>>>>> required option: j*
>>>>> at
>>>>> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
>>>>> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
>>>>> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
>>>>> at
>>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
>>>>> ... 1 more
>>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>>> cmdLineSyntax not provided
>>>>> at
>>>>> org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:546)
>>>>> at
>>>>> org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:492)
>>>>> at
>>>>> org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:408)
>>>>> at
>>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp(CommandLineParser.java:60)
>>>>> at
>>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
>>>>>
>>>>
Re: anybody can start flink with job mode?
Posted by Hao Sun <ha...@zendesk.com>.
Thanks Till for the follow up, I can run my job now.
On Tue, Aug 28, 2018, 00:57 Till Rohrmann <tr...@apache.org> wrote:
> Hi Hao,
>
> Vino is right, you need to specify the -j/--job-classname option which
> specifies the job name you want to execute. Please make sure that the jar
> containing this class is on the class path.
>
> I recently pushed some fixes which generate a better error message than
> the one you've received. If you check out the latest master branch, then it
> should work better.
>
> Let me know if you should run into other problems.
>
> Cheers,
> Till
>
> On Sat, Aug 25, 2018 at 5:11 AM Hao Sun <ha...@zendesk.com> wrote:
>
>> Thanks, I'll look into it.
>>
>> On Fri, Aug 24, 2018, 19:44 vino yang <ya...@gmail.com> wrote:
>>
>>> Hi Hao Sun,
>>>
>>> From the error log, it seems that the jar package for the job was not
>>> found.
>>> You must make sure your Jar is in the classpath.
>>> Related documentation may not be up-to-date, and there is a discussion
>>> on this issue on this mailing list. [1]
>>>
>>> I see that the status of FLINK-10001 [2] is closed and it will be
>>> updated with the release of 1.6.1 and 1.7.0.
>>>
>>> [1]:
>>> http://mail-archives.apache.org/mod_mbox/flink-dev/201808.mbox/%3CCAC27z=OaohMbmcryB-+m3GBmZP=XPHa8miHv7ZS1gRGSEkkfdg@mail.gmail.com%3E
>>> <http://mail-archives.apache.org/mod_mbox/flink-dev/201808.mbox/%3CCAC27z=OaohMbmcryB-+m3GBmZP=XPHa8miHv7ZS1gRGSEkkfdg@mail.gmail.com%3E>
>>> [2]: https://issues.apache.org/jira/browse/FLINK-10001
>>> <https://issues.apache.org/jira/browse/FLINK-10001>
>>>
>>> Thanks, vino.
>>>
>>>
>>> Hao Sun <ha...@zendesk.com> 于2018年8月25日周六 上午6:37写道：
>>>
>>>> I got an error like this.
>>>>
>>>> $ docker run -it flink-job:latest job-cluster
>>>> Starting the job-cluster
>>>> config file:
>>>> jobmanager.rpc.address: localhost
>>>> jobmanager.rpc.port: 6123
>>>> jobmanager.heap.size: 1024m
>>>> taskmanager.heap.size: 1024m
>>>> taskmanager.numberOfTaskSlots: 1
>>>> parallelism.default: 1
>>>> rest.port: 8081
>>>> Starting standalonejob as a console application on host cf9bd047082c.
>>>> 2018-08-24 22:33:00,773 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>>> --------------------------------------------------------------------------------
>>>> 2018-08-24 22:33:00,774 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting
>>>> StandaloneJobClusterEntryPoint (Version: 1.6.0, Rev:ff472b4,
>>>> Date:07.08.2018 @ 13:31:13 UTC)
>>>> 2018-08-24 22:33:00,775 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - OS current user:
>>>> flink
>>>> 2018-08-24 22:33:01,168 WARN org.apache.hadoop.util.NativeCodeLoader -
>>>> Unable to load native-hadoop library for your platform... using
>>>> builtin-java classes where applicable
>>>> 2018-08-24 22:33:01,232 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Current
>>>> Hadoop/Kerberos user: flink
>>>> 2018-08-24 22:33:01,232 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM: OpenJDK 64-Bit
>>>> Server VM - Oracle Corporation - 1.8/25.111-b14
>>>> 2018-08-24 22:33:01,232 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Maximum heap size:
>>>> 981 MiBytes
>>>> 2018-08-24 22:33:01,232 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JAVA_HOME:
>>>> /usr/lib/jvm/java-1.8-openjdk/jre
>>>> 2018-08-24 22:33:01,236 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Hadoop version:
>>>> 2.8.3
>>>> 2018-08-24 22:33:01,236 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM Options:
>>>> 2018-08-24 22:33:01,236 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xms1024m
>>>> 2018-08-24 22:33:01,236 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xmx1024m
>>>> 2018-08-24 22:33:01,237 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>>> -Dlog4j.configuration=file:/opt/flink-1.6.0/conf/log4j-console.properties
>>>> 2018-08-24 22:33:01,237 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>>> -Dlogback.configurationFile=file:/opt/flink-1.6.0/conf/logback-console.xml
>>>> 2018-08-24 22:33:01,237 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Program Arguments:
>>>> 2018-08-24 22:33:01,237 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - --configDir
>>>> 2018-08-24 22:33:01,238 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>>> /opt/flink-1.6.0/conf
>>>> 2018-08-24 22:33:01,238 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Classpath:
>>>> /opt/flink-1.6.0/lib/flink-python_2.11-1.6.0.jar:/opt/flink-1.6.0/lib/flink-shaded-hadoop2-uber-1.6.0.jar:/opt/flink-1.6.0/lib/job.jar:/opt/flink-1.6.0/lib/log4j-1.2.17.jar:/opt/flink-1.6.0/lib/slf4j-log4j12-1.7.7.jar:/opt/flink-1.6.0/lib/flink-dist_2.11-1.6.0.jar:::
>>>> 2018-08-24 22:33:01,238 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>>> --------------------------------------------------------------------------------
>>>> 2018-08-24 22:33:01,240 INFO
>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Registered UNIX
>>>> signal handlers for [TERM, HUP, INT]
>>>> 2018-08-24 22:33:01,248 ERROR *org.apache.flink.runtime.entrypoint.ClusterEntrypoint
>>>> - Could not parse command line arguments [--configDir,
>>>> /opt/flink-1.6.0/conf].*
>>>> org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to
>>>> parse the command line arguments.
>>>> at
>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
>>>> at
>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
>>>> Caused by: org.apache.commons.cli.MissingOptionException: *Missing
>>>> required option: j*
>>>> at
>>>> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
>>>> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
>>>> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
>>>> at
>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
>>>> ... 1 more
>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>> cmdLineSyntax not provided
>>>> at
>>>> org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:546)
>>>> at
>>>> org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:492)
>>>> at
>>>> org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:408)
>>>> at
>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp(CommandLineParser.java:60)
>>>> at
>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
>>>>
>>>
Re: anybody can start flink with job mode?
Posted by Till Rohrmann <tr...@apache.org>.
Hi Hao,
Vino is right, you need to specify the -j/--job-classname option which
specifies the job name you want to execute. Please make sure that the jar
containing this class is on the class path.
I recently pushed some fixes which generate a better error message than the
one you've received. If you check out the latest master branch, then it
should work better.
Let me know if you should run into other problems.
Cheers,
Till
On Sat, Aug 25, 2018 at 5:11 AM Hao Sun <ha...@zendesk.com> wrote:
> Thanks, I'll look into it.
>
> On Fri, Aug 24, 2018, 19:44 vino yang <ya...@gmail.com> wrote:
>
>> Hi Hao Sun,
>>
>> From the error log, it seems that the jar package for the job was not
>> found.
>> You must make sure your Jar is in the classpath.
>> Related documentation may not be up-to-date, and there is a discussion on
>> this issue on this mailing list. [1]
>>
>> I see that the status of FLINK-10001 [2] is closed and it will be updated
>> with the release of 1.6.1 and 1.7.0.
>>
>> [1]:
>> http://mail-archives.apache.org/mod_mbox/flink-dev/201808.mbox/%3CCAC27z=OaohMbmcryB-+m3GBmZP=XPHa8miHv7ZS1gRGSEkkfdg@mail.gmail.com%3E
>> [2]: https://issues.apache.org/jira/browse/FLINK-10001
>>
>> Thanks, vino.
>>
>>
>> Hao Sun <ha...@zendesk.com> 于2018年8月25日周六 上午6:37写道：
>>
>>> I got an error like this.
>>>
>>> $ docker run -it flink-job:latest job-cluster
>>> Starting the job-cluster
>>> config file:
>>> jobmanager.rpc.address: localhost
>>> jobmanager.rpc.port: 6123
>>> jobmanager.heap.size: 1024m
>>> taskmanager.heap.size: 1024m
>>> taskmanager.numberOfTaskSlots: 1
>>> parallelism.default: 1
>>> rest.port: 8081
>>> Starting standalonejob as a console application on host cf9bd047082c.
>>> 2018-08-24 22:33:00,773 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>> --------------------------------------------------------------------------------
>>> 2018-08-24 22:33:00,774 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting
>>> StandaloneJobClusterEntryPoint (Version: 1.6.0, Rev:ff472b4,
>>> Date:07.08.2018 @ 13:31:13 UTC)
>>> 2018-08-24 22:33:00,775 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - OS current user:
>>> flink
>>> 2018-08-24 22:33:01,168 WARN org.apache.hadoop.util.NativeCodeLoader -
>>> Unable to load native-hadoop library for your platform... using
>>> builtin-java classes where applicable
>>> 2018-08-24 22:33:01,232 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Current
>>> Hadoop/Kerberos user: flink
>>> 2018-08-24 22:33:01,232 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM: OpenJDK 64-Bit
>>> Server VM - Oracle Corporation - 1.8/25.111-b14
>>> 2018-08-24 22:33:01,232 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Maximum heap size:
>>> 981 MiBytes
>>> 2018-08-24 22:33:01,232 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JAVA_HOME:
>>> /usr/lib/jvm/java-1.8-openjdk/jre
>>> 2018-08-24 22:33:01,236 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Hadoop version:
>>> 2.8.3
>>> 2018-08-24 22:33:01,236 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM Options:
>>> 2018-08-24 22:33:01,236 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xms1024m
>>> 2018-08-24 22:33:01,236 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xmx1024m
>>> 2018-08-24 22:33:01,237 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>> -Dlog4j.configuration=file:/opt/flink-1.6.0/conf/log4j-console.properties
>>> 2018-08-24 22:33:01,237 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>> -Dlogback.configurationFile=file:/opt/flink-1.6.0/conf/logback-console.xml
>>> 2018-08-24 22:33:01,237 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Program Arguments:
>>> 2018-08-24 22:33:01,237 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - --configDir
>>> 2018-08-24 22:33:01,238 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>> /opt/flink-1.6.0/conf
>>> 2018-08-24 22:33:01,238 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Classpath:
>>> /opt/flink-1.6.0/lib/flink-python_2.11-1.6.0.jar:/opt/flink-1.6.0/lib/flink-shaded-hadoop2-uber-1.6.0.jar:/opt/flink-1.6.0/lib/job.jar:/opt/flink-1.6.0/lib/log4j-1.2.17.jar:/opt/flink-1.6.0/lib/slf4j-log4j12-1.7.7.jar:/opt/flink-1.6.0/lib/flink-dist_2.11-1.6.0.jar:::
>>> 2018-08-24 22:33:01,238 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>>> --------------------------------------------------------------------------------
>>> 2018-08-24 22:33:01,240 INFO
>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Registered UNIX
>>> signal handlers for [TERM, HUP, INT]
>>> 2018-08-24 22:33:01,248 ERROR *org.apache.flink.runtime.entrypoint.ClusterEntrypoint
>>> - Could not parse command line arguments [--configDir,
>>> /opt/flink-1.6.0/conf].*
>>> org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse
>>> the command line arguments.
>>> at
>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
>>> at
>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
>>> Caused by: org.apache.commons.cli.MissingOptionException: *Missing
>>> required option: j*
>>> at
>>> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
>>> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
>>> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
>>> at
>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
>>> ... 1 more
>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>> cmdLineSyntax not provided
>>> at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:546)
>>> at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:492)
>>> at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:408)
>>> at
>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp(CommandLineParser.java:60)
>>> at
>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
>>>
>>
Re: anybody can start flink with job mode?
Posted by Hao Sun <ha...@zendesk.com>.
Thanks, I'll look into it.
On Fri, Aug 24, 2018, 19:44 vino yang <ya...@gmail.com> wrote:
> Hi Hao Sun,
>
> From the error log, it seems that the jar package for the job was not
> found.
> You must make sure your Jar is in the classpath.
> Related documentation may not be up-to-date, and there is a discussion on
> this issue on this mailing list. [1]
>
> I see that the status of FLINK-10001 [2] is closed and it will be updated
> with the release of 1.6.1 and 1.7.0.
>
> [1]:
> http://mail-archives.apache.org/mod_mbox/flink-dev/201808.mbox/%3CCAC27z=OaohMbmcryB-+m3GBmZP=XPHa8miHv7ZS1gRGSEkkfdg@mail.gmail.com%3E
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201808.mbox/%3CCAC27z=OaohMbmcryB-+m3GBmZP=XPHa8miHv7ZS1gRGSEkkfdg@mail.gmail.com%3E>
> [2]: https://issues.apache.org/jira/browse/FLINK-10001
> <https://issues.apache.org/jira/browse/FLINK-10001>
>
> Thanks, vino.
>
>
> Hao Sun <ha...@zendesk.com> 于2018年8月25日周六 上午6:37写道：
>
>> I got an error like this.
>>
>> $ docker run -it flink-job:latest job-cluster
>> Starting the job-cluster
>> config file:
>> jobmanager.rpc.address: localhost
>> jobmanager.rpc.port: 6123
>> jobmanager.heap.size: 1024m
>> taskmanager.heap.size: 1024m
>> taskmanager.numberOfTaskSlots: 1
>> parallelism.default: 1
>> rest.port: 8081
>> Starting standalonejob as a console application on host cf9bd047082c.
>> 2018-08-24 22:33:00,773 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>> --------------------------------------------------------------------------------
>> 2018-08-24 22:33:00,774 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting
>> StandaloneJobClusterEntryPoint (Version: 1.6.0, Rev:ff472b4,
>> Date:07.08.2018 @ 13:31:13 UTC)
>> 2018-08-24 22:33:00,775 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - OS current user:
>> flink
>> 2018-08-24 22:33:01,168 WARN org.apache.hadoop.util.NativeCodeLoader -
>> Unable to load native-hadoop library for your platform... using
>> builtin-java classes where applicable
>> 2018-08-24 22:33:01,232 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Current
>> Hadoop/Kerberos user: flink
>> 2018-08-24 22:33:01,232 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM: OpenJDK 64-Bit
>> Server VM - Oracle Corporation - 1.8/25.111-b14
>> 2018-08-24 22:33:01,232 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Maximum heap size:
>> 981 MiBytes
>> 2018-08-24 22:33:01,232 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JAVA_HOME:
>> /usr/lib/jvm/java-1.8-openjdk/jre
>> 2018-08-24 22:33:01,236 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Hadoop version:
>> 2.8.3
>> 2018-08-24 22:33:01,236 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM Options:
>> 2018-08-24 22:33:01,236 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xms1024m
>> 2018-08-24 22:33:01,236 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xmx1024m
>> 2018-08-24 22:33:01,237 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>> -Dlog4j.configuration=file:/opt/flink-1.6.0/conf/log4j-console.properties
>> 2018-08-24 22:33:01,237 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>> -Dlogback.configurationFile=file:/opt/flink-1.6.0/conf/logback-console.xml
>> 2018-08-24 22:33:01,237 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Program Arguments:
>> 2018-08-24 22:33:01,237 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - --configDir
>> 2018-08-24 22:33:01,238 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>> /opt/flink-1.6.0/conf
>> 2018-08-24 22:33:01,238 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Classpath:
>> /opt/flink-1.6.0/lib/flink-python_2.11-1.6.0.jar:/opt/flink-1.6.0/lib/flink-shaded-hadoop2-uber-1.6.0.jar:/opt/flink-1.6.0/lib/job.jar:/opt/flink-1.6.0/lib/log4j-1.2.17.jar:/opt/flink-1.6.0/lib/slf4j-log4j12-1.7.7.jar:/opt/flink-1.6.0/lib/flink-dist_2.11-1.6.0.jar:::
>> 2018-08-24 22:33:01,238 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
>> --------------------------------------------------------------------------------
>> 2018-08-24 22:33:01,240 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Registered UNIX
>> signal handlers for [TERM, HUP, INT]
>> 2018-08-24 22:33:01,248 ERROR *org.apache.flink.runtime.entrypoint.ClusterEntrypoint
>> - Could not parse command line arguments [--configDir,
>> /opt/flink-1.6.0/conf].*
>> org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse
>> the command line arguments.
>> at
>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
>> at
>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
>> Caused by: org.apache.commons.cli.MissingOptionException: *Missing
>> required option: j*
>> at
>> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
>> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
>> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
>> at
>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
>> ... 1 more
>> Exception in thread "main" java.lang.IllegalArgumentException:
>> cmdLineSyntax not provided
>> at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:546)
>> at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:492)
>> at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:408)
>> at
>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp(CommandLineParser.java:60)
>> at
>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
>>
>
Re: anybody can start flink with job mode?
Posted by vino yang <ya...@gmail.com>.
Hi Hao Sun,
From the error log, it seems that the jar package for the job was not
found.
You must make sure your Jar is in the classpath.
Related documentation may not be up-to-date, and there is a discussion on
this issue on this mailing list. [1]
I see that the status of FLINK-10001 [2] is closed and it will be updated
with the release of 1.6.1 and 1.7.0.
[1]:
http://mail-archives.apache.org/mod_mbox/flink-dev/201808.mbox/%3CCAC27z=OaohMbmcryB-+m3GBmZP=XPHa8miHv7ZS1gRGSEkkfdg@mail.gmail.com%3E
[2]: https://issues.apache.org/jira/browse/FLINK-10001
Thanks, vino.
Hao Sun <ha...@zendesk.com> 于2018年8月25日周六 上午6:37写道：
> I got an error like this.
>
> $ docker run -it flink-job:latest job-cluster
> Starting the job-cluster
> config file:
> jobmanager.rpc.address: localhost
> jobmanager.rpc.port: 6123
> jobmanager.heap.size: 1024m
> taskmanager.heap.size: 1024m
> taskmanager.numberOfTaskSlots: 1
> parallelism.default: 1
> rest.port: 8081
> Starting standalonejob as a console application on host cf9bd047082c.
> 2018-08-24 22:33:00,773 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
> --------------------------------------------------------------------------------
> 2018-08-24 22:33:00,774 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting
> StandaloneJobClusterEntryPoint (Version: 1.6.0, Rev:ff472b4,
> Date:07.08.2018 @ 13:31:13 UTC)
> 2018-08-24 22:33:00,775 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - OS current user:
> flink
> 2018-08-24 22:33:01,168 WARN org.apache.hadoop.util.NativeCodeLoader -
> Unable to load native-hadoop library for your platform... using
> builtin-java classes where applicable
> 2018-08-24 22:33:01,232 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Current
> Hadoop/Kerberos user: flink
> 2018-08-24 22:33:01,232 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM: OpenJDK 64-Bit
> Server VM - Oracle Corporation - 1.8/25.111-b14
> 2018-08-24 22:33:01,232 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Maximum heap size:
> 981 MiBytes
> 2018-08-24 22:33:01,232 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JAVA_HOME:
> /usr/lib/jvm/java-1.8-openjdk/jre
> 2018-08-24 22:33:01,236 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Hadoop version:
> 2.8.3
> 2018-08-24 22:33:01,236 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM Options:
> 2018-08-24 22:33:01,236 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xms1024m
> 2018-08-24 22:33:01,236 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xmx1024m
> 2018-08-24 22:33:01,237 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
> -Dlog4j.configuration=file:/opt/flink-1.6.0/conf/log4j-console.properties
> 2018-08-24 22:33:01,237 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
> -Dlogback.configurationFile=file:/opt/flink-1.6.0/conf/logback-console.xml
> 2018-08-24 22:33:01,237 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Program Arguments:
> 2018-08-24 22:33:01,237 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - --configDir
> 2018-08-24 22:33:01,238 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
> /opt/flink-1.6.0/conf
> 2018-08-24 22:33:01,238 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Classpath:
> /opt/flink-1.6.0/lib/flink-python_2.11-1.6.0.jar:/opt/flink-1.6.0/lib/flink-shaded-hadoop2-uber-1.6.0.jar:/opt/flink-1.6.0/lib/job.jar:/opt/flink-1.6.0/lib/log4j-1.2.17.jar:/opt/flink-1.6.0/lib/slf4j-log4j12-1.7.7.jar:/opt/flink-1.6.0/lib/flink-dist_2.11-1.6.0.jar:::
> 2018-08-24 22:33:01,238 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
> --------------------------------------------------------------------------------
> 2018-08-24 22:33:01,240 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Registered UNIX
> signal handlers for [TERM, HUP, INT]
> 2018-08-24 22:33:01,248 ERROR *org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> - Could not parse command line arguments [--configDir,
> /opt/flink-1.6.0/conf].*
> org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to parse
> the command line arguments.
> at
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
> at
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
> Caused by: org.apache.commons.cli.MissingOptionException: *Missing
> required option: j*
> at
> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
> at
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
> ... 1 more
> Exception in thread "main" java.lang.IllegalArgumentException:
> cmdLineSyntax not provided
> at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:546)
> at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:492)
> at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:408)
> at
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp(CommandLineParser.java:60)
> at
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:146)
>