You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Felipe Gutierrez <fe...@gmail.com> on 2020/09/18 09:52:39 UTC
Error on deploying Flink docker image with Kubernetes (minikube) and
automatically launch a stream WordCount job.
Hi community,
I am trying to deploy the default WordCount stream application on
minikube using the official documentation at [1]. I am using minikube
v1.13.0 on Ubuntu 18.04 and Kubernetes v1.19.0 on Docker 19.03.8. I
could sucessfully start 1 job manager and 3 task managers using the
yaml files flink-configuration-configmap.yaml,
jobmanager-service.yaml, jobmanager-rest-service.yaml,
jobmanager-session-deployment.yaml, and
taskmanager-session-deployment.yaml (all available on the Apendix of
this link [1]).
Then I would like to start the word-count stream job available on the
flink jar image [2], which I believe is available since it is built
inside the default flink jar distribution. What I understood that I
have to do is to create the objects based on the files
jobmanager-job.yaml and taskmanager-job-deployment.yaml (also
available on the link [1]). And, I think that I have to replace this
line below on the object jobmanager-job.yaml
(spec.template.spec.containers.name[jobmanager]):
args: ["standalone-job", "--job-classname",
"org.apache.flink.streaming.examples.wordcount.WordCount"]
Is this correct? I am not sure if this is my entire error. I am
getting the message "Could not find the provided job class
(org.apache.flink.streaming.examples.wordcount.WordCount) in the user
lib directory (/opt/flink/usrlib)". As far as I know
"/opt/flink/usrlib" is the default directory. I am not sure if I have
to change the property: path /host/path/to/job/artifacts. This is my
log message of the pod error.
Do you guys have any idea of what I am missing in my configuration?
Thanks, Felipe
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
flink-jobmanager-ftgg9 0/1 CrashLoopBackOff 3 83s
$ kubectl logs flink-jobmanager-ftgg9
Starting Job Manager
sed: couldn't open temporary file /opt/flink/conf/sedA699Jt: Read-only
file system
sed: couldn't open temporary file /opt/flink/conf/sedvZhs0w: Read-only
file system
/docker-entrypoint.sh: 72: /docker-entrypoint.sh: cannot create
/opt/flink/conf/flink-conf.yaml: Permission denied
/docker-entrypoint.sh: 91: /docker-entrypoint.sh: cannot create
/opt/flink/conf/flink-conf.yaml.tmp: Read-only file system
Starting standalonejob as a console application on host flink-jobmanager-ftgg9.
2020-09-18 09:15:58,801 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--------------------------------------------------------------------------------
2020-09-18 09:15:58,804 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Preconfiguration:
2020-09-18 09:15:58,804 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
JM_RESOURCE_PARAMS extraction logs:
jvm_params: -Xmx1073741824 -Xms1073741824 -XX:MaxMetaspaceSize=268435456
logs: INFO [] - Loading configuration property:
jobmanager.rpc.address, flink-jobmanager
INFO [] - Loading configuration property: taskmanager.numberOfTaskSlots, 4
INFO [] - Loading configuration property: blob.server.port, 6124
INFO [] - Loading configuration property: jobmanager.rpc.port, 6123
INFO [] - Loading configuration property: taskmanager.rpc.port, 6122
INFO [] - Loading configuration property: queryable-state.proxy.ports, 6125
INFO [] - Loading configuration property: jobmanager.memory.process.size, 1600m
INFO [] - Loading configuration property:
taskmanager.memory.process.size, 1728m
INFO [] - Loading configuration property: parallelism.default, 2
INFO [] - The derived from fraction jvm overhead memory (160.000mb
(167772162 bytes)) is less than its min value 192.000mb (201326592
bytes), min value will be used instead
INFO [] - Final Master Memory configuration:
INFO [] - Total Process Memory: 1.563gb (1677721600 bytes)
INFO [] - Total Flink Memory: 1.125gb (1207959552 bytes)
INFO [] - JVM Heap: 1024.000mb (1073741824 bytes)
INFO [] - Off-heap: 128.000mb (134217728 bytes)
INFO [] - JVM Metaspace: 256.000mb (268435456 bytes)
INFO [] - JVM Overhead: 192.000mb (201326592 bytes)
2020-09-18 09:15:58,805 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--------------------------------------------------------------------------------
2020-09-18 09:15:58,805 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Starting StandaloneApplicationClusterEntryPoint (Version: 1.11.0,
Scala: 2.11, Rev:d04872d, Date:2020-06-29T16:13:14+02:00)
2020-09-18 09:15:58,805 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - OS
current user: flink
2020-09-18 09:15:58,805 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Current Hadoop/Kerberos user: <no hadoop dependency found>
2020-09-18 09:15:58,805 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.262-b10
2020-09-18 09:15:58,806 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Maximum heap size: 989 MiBytes
2020-09-18 09:15:58,806 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
JAVA_HOME: /usr/local/openjdk-8
2020-09-18 09:15:58,806 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - No
Hadoop Dependency available
2020-09-18 09:15:58,806 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM
Options:
2020-09-18 09:15:58,806 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Xmx1073741824
2020-09-18 09:15:58,806 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Xms1073741824
2020-09-18 09:15:58,807 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:MaxMetaspaceSize=268435456
2020-09-18 09:15:58,807 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog.file=/opt/flink/log/flink--standalonejob-0-flink-jobmanager-ftgg9.log
2020-09-18 09:15:58,807 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties
2020-09-18 09:15:58,807 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog4j.configurationFile=file:/opt/flink/conf/log4j-console.properties
2020-09-18 09:15:58,807 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml
2020-09-18 09:15:58,807 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Program Arguments:
2020-09-18 09:15:58,807 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--configDir
2020-09-18 09:15:58,808 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
/opt/flink/conf
2020-09-18 09:15:58,808 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--job-classname
2020-09-18 09:15:58,808 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
org.apache.flink.streaming.examples.wordcount.WordCount
2020-09-18 09:15:58,808 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Classpath: /opt/flink/lib/flink-csv-1.11.0.jar:/opt/flink/lib/flink-json-1.11.0.jar:/opt/flink/lib/flink-shaded-zookeeper-3.4.14.jar:/opt/flink/lib/flink-table-blink_2.11-1.11.0.jar:/opt/flink/lib/flink-table_2.11-1.11.0.jar:/opt/flink/lib/log4j-1.2-api-2.12.1.jar:/opt/flink/lib/log4j-api-2.12.1.jar:/opt/flink/lib/log4j-core-2.12.1.jar:/opt/flink/lib/log4j-slf4j-impl-2.12.1.jar:/opt/flink/lib/flink-dist_2.11-1.11.0.jar:::
2020-09-18 09:15:58,808 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--------------------------------------------------------------------------------
2020-09-18 09:15:58,809 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Registered UNIX signal handlers for [TERM, HUP, INT]
2020-09-18 09:15:58,849 ERROR
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Could not create application program.
org.apache.flink.util.FlinkException: Could not find the provided job
class (org.apache.flink.streaming.examples.wordcount.WordCount) in the
user lib directory (/opt/flink/usrlib).
at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getJobClassNameOrScanClassPath(ClassPathPackagedProgramRetriever.java:140)
~[flink-dist_2.11-1.11.0.jar:1.11.0]
at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getPackagedProgram(ClassPathPackagedProgramRetriever.java:123)
~[flink-dist_2.11-1.11.0.jar:1.11.0]
at org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.getPackagedProgram(StandaloneApplicationClusterEntryPoint.java:110)
~[flink-dist_2.11-1.11.0.jar:1.11.0]
at org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.main(StandaloneApplicationClusterEntryPoint.java:78)
[flink-dist_2.11-1.11.0.jar:1.11.0]
[1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/kubernetes.html
[2] https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/wordcount/WordCount.java
--
-- Felipe Gutierrez
-- skype: felipe.o.gutierrez
-- https://felipeogutierrez.blogspot.com
Re: Error on deploying Flink docker image with Kubernetes (minikube)
and automatically launch a stream WordCount job.
Posted by Yang Wang <da...@gmail.com>.
Glad to hear that. And the repository you shared is very helpful for the
users who are trying
to deploy Flink clusters on K8s.
Best,
Yang
Felipe Gutierrez <fe...@gmail.com> 于2020年9月24日周四 下午2:04写道：
> thanks Yang,
> I got to put it to work in the way that you said.
> https://github.com/felipegutierrez/explore-flink
>
> Best,
> Felipe
> --
> -- Felipe Gutierrez
> -- skype: felipe.o.gutierrez
> -- https://felipeogutierrez.blogspot.com
>
> On Thu, Sep 24, 2020 at 6:59 AM Yang Wang <da...@gmail.com> wrote:
> >
> > Hi Felipe,
> >
> > Currently, if you want to deploy a standalone job/application Flink
> cluster on K8s via yamls.
> > You should have your own image with user jar baked located at
> /opt/flink/usrlib. It could not be
> > specified via config option. Usually, you could add new layer on the
> official docker image to
> > build in the user jar.
> >
> > A more graceful solution is init container[1], which could download the
> user jar from remote
> > storage or just copy from local directory /opt/flink/opt to
> /opt/flink/usrlib.
> >
> > [1].
> https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-initialization/
> >
> > Best,
> > Yang
> >
> >
> > Felipe Gutierrez <fe...@gmail.com> 于2020年9月18日周五 下午5:52写道：
> >>
> >> Hi community,
> >>
> >> I am trying to deploy the default WordCount stream application on
> >> minikube using the official documentation at [1]. I am using minikube
> >> v1.13.0 on Ubuntu 18.04 and Kubernetes v1.19.0 on Docker 19.03.8. I
> >> could sucessfully start 1 job manager and 3 task managers using the
> >> yaml files flink-configuration-configmap.yaml,
> >> jobmanager-service.yaml, jobmanager-rest-service.yaml,
> >> jobmanager-session-deployment.yaml, and
> >> taskmanager-session-deployment.yaml (all available on the Apendix of
> >> this link [1]).
> >>
> >> Then I would like to start the word-count stream job available on the
> >> flink jar image [2], which I believe is available since it is built
> >> inside the default flink jar distribution. What I understood that I
> >> have to do is to create the objects based on the files
> >> jobmanager-job.yaml and taskmanager-job-deployment.yaml (also
> >> available on the link [1]). And, I think that I have to replace this
> >> line below on the object jobmanager-job.yaml
> >> (spec.template.spec.containers.name[jobmanager]):
> >>
> >> args: ["standalone-job", "--job-classname",
> >> "org.apache.flink.streaming.examples.wordcount.WordCount"]
> >>
> >> Is this correct? I am not sure if this is my entire error. I am
> >> getting the message "Could not find the provided job class
> >> (org.apache.flink.streaming.examples.wordcount.WordCount) in the user
> >> lib directory (/opt/flink/usrlib)". As far as I know
> >> "/opt/flink/usrlib" is the default directory. I am not sure if I have
> >> to change the property: path /host/path/to/job/artifacts. This is my
> >> log message of the pod error.
> >> Do you guys have any idea of what I am missing in my configuration?
> >>
> >> Thanks, Felipe
> >>
> >> $ kubectl get pods
> >> NAME READY STATUS RESTARTS AGE
> >> flink-jobmanager-ftgg9 0/1 CrashLoopBackOff 3 83s
> >> $ kubectl logs flink-jobmanager-ftgg9
> >> Starting Job Manager
> >> sed: couldn't open temporary file /opt/flink/conf/sedA699Jt: Read-only
> >> file system
> >> sed: couldn't open temporary file /opt/flink/conf/sedvZhs0w: Read-only
> >> file system
> >> /docker-entrypoint.sh: 72: /docker-entrypoint.sh: cannot create
> >> /opt/flink/conf/flink-conf.yaml: Permission denied
> >> /docker-entrypoint.sh: 91: /docker-entrypoint.sh: cannot create
> >> /opt/flink/conf/flink-conf.yaml.tmp: Read-only file system
> >> Starting standalonejob as a console application on host
> flink-jobmanager-ftgg9.
> >> 2020-09-18 09:15:58,801 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >>
> --------------------------------------------------------------------------------
> >> 2020-09-18 09:15:58,804 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> Preconfiguration:
> >> 2020-09-18 09:15:58,804 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >>
> >>
> >> JM_RESOURCE_PARAMS extraction logs:
> >> jvm_params: -Xmx1073741824 -Xms1073741824 -XX:MaxMetaspaceSize=268435456
> >> logs: INFO [] - Loading configuration property:
> >> jobmanager.rpc.address, flink-jobmanager
> >> INFO [] - Loading configuration property:
> taskmanager.numberOfTaskSlots, 4
> >> INFO [] - Loading configuration property: blob.server.port, 6124
> >> INFO [] - Loading configuration property: jobmanager.rpc.port, 6123
> >> INFO [] - Loading configuration property: taskmanager.rpc.port, 6122
> >> INFO [] - Loading configuration property: queryable-state.proxy.ports,
> 6125
> >> INFO [] - Loading configuration property:
> jobmanager.memory.process.size, 1600m
> >> INFO [] - Loading configuration property:
> >> taskmanager.memory.process.size, 1728m
> >> INFO [] - Loading configuration property: parallelism.default, 2
> >> INFO [] - The derived from fraction jvm overhead memory (160.000mb
> >> (167772162 bytes)) is less than its min value 192.000mb (201326592
> >> bytes), min value will be used instead
> >> INFO [] - Final Master Memory configuration:
> >> INFO [] - Total Process Memory: 1.563gb (1677721600 bytes)
> >> INFO [] - Total Flink Memory: 1.125gb (1207959552 bytes)
> >> INFO [] - JVM Heap: 1024.000mb (1073741824 bytes)
> >> INFO [] - Off-heap: 128.000mb (134217728 bytes)
> >> INFO [] - JVM Metaspace: 256.000mb (268435456 bytes)
> >> INFO [] - JVM Overhead: 192.000mb (201326592 bytes)
> >>
> >> 2020-09-18 09:15:58,805 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >>
> --------------------------------------------------------------------------------
> >> 2020-09-18 09:15:58,805 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> Starting StandaloneApplicationClusterEntryPoint (Version: 1.11.0,
> >> Scala: 2.11, Rev:d04872d, Date:2020-06-29T16:13:14+02:00)
> >> 2020-09-18 09:15:58,805 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - OS
> >> current user: flink
> >> 2020-09-18 09:15:58,805 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> Current Hadoop/Kerberos user: <no hadoop dependency found>
> >> 2020-09-18 09:15:58,805 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.262-b10
> >> 2020-09-18 09:15:58,806 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> Maximum heap size: 989 MiBytes
> >> 2020-09-18 09:15:58,806 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> JAVA_HOME: /usr/local/openjdk-8
> >> 2020-09-18 09:15:58,806 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - No
> >> Hadoop Dependency available
> >> 2020-09-18 09:15:58,806 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM
> >> Options:
> >> 2020-09-18 09:15:58,806 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> -Xmx1073741824
> >> 2020-09-18 09:15:58,806 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> -Xms1073741824
> >> 2020-09-18 09:15:58,807 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> -XX:MaxMetaspaceSize=268435456
> >> 2020-09-18 09:15:58,807 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >>
> -Dlog.file=/opt/flink/log/flink--standalonejob-0-flink-jobmanager-ftgg9.log
> >> 2020-09-18 09:15:58,807 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> -Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties
> >> 2020-09-18 09:15:58,807 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> -Dlog4j.configurationFile=file:/opt/flink/conf/log4j-console.properties
> >> 2020-09-18 09:15:58,807 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> -Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml
> >> 2020-09-18 09:15:58,807 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> Program Arguments:
> >> 2020-09-18 09:15:58,807 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> --configDir
> >> 2020-09-18 09:15:58,808 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> /opt/flink/conf
> >> 2020-09-18 09:15:58,808 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> --job-classname
> >> 2020-09-18 09:15:58,808 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> org.apache.flink.streaming.examples.wordcount.WordCount
> >> 2020-09-18 09:15:58,808 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> Classpath:
> /opt/flink/lib/flink-csv-1.11.0.jar:/opt/flink/lib/flink-json-1.11.0.jar:/opt/flink/lib/flink-shaded-zookeeper-3.4.14.jar:/opt/flink/lib/flink-table-blink_2.11-1.11.0.jar:/opt/flink/lib/flink-table_2.11-1.11.0.jar:/opt/flink/lib/log4j-1.2-api-2.12.1.jar:/opt/flink/lib/log4j-api-2.12.1.jar:/opt/flink/lib/log4j-core-2.12.1.jar:/opt/flink/lib/log4j-slf4j-impl-2.12.1.jar:/opt/flink/lib/flink-dist_2.11-1.11.0.jar:::
> >> 2020-09-18 09:15:58,808 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >>
> --------------------------------------------------------------------------------
> >> 2020-09-18 09:15:58,809 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> Registered UNIX signal handlers for [TERM, HUP, INT]
> >> 2020-09-18 09:15:58,849 ERROR
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> >> Could not create application program.
> >> org.apache.flink.util.FlinkException: Could not find the provided job
> >> class (org.apache.flink.streaming.examples.wordcount.WordCount) in the
> >> user lib directory (/opt/flink/usrlib).
> >> at
> org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getJobClassNameOrScanClassPath(ClassPathPackagedProgramRetriever.java:140)
> >> ~[flink-dist_2.11-1.11.0.jar:1.11.0]
> >> at
> org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getPackagedProgram(ClassPathPackagedProgramRetriever.java:123)
> >> ~[flink-dist_2.11-1.11.0.jar:1.11.0]
> >> at
> org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.getPackagedProgram(StandaloneApplicationClusterEntryPoint.java:110)
> >> ~[flink-dist_2.11-1.11.0.jar:1.11.0]
> >> at
> org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.main(StandaloneApplicationClusterEntryPoint.java:78)
> >> [flink-dist_2.11-1.11.0.jar:1.11.0]
> >>
> >> [1]
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/kubernetes.html
> >> [2]
> https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/wordcount/WordCount.java
> >> --
> >> -- Felipe Gutierrez
> >> -- skype: felipe.o.gutierrez
> >> -- https://felipeogutierrez.blogspot.com
>
Re: Error on deploying Flink docker image with Kubernetes (minikube)
and automatically launch a stream WordCount job.
Posted by Felipe Gutierrez <fe...@gmail.com>.
thanks Yang,
I got to put it to work in the way that you said.
https://github.com/felipegutierrez/explore-flink
Best,
Felipe
--
-- Felipe Gutierrez
-- skype: felipe.o.gutierrez
-- https://felipeogutierrez.blogspot.com
On Thu, Sep 24, 2020 at 6:59 AM Yang Wang <da...@gmail.com> wrote:
>
> Hi Felipe,
>
> Currently, if you want to deploy a standalone job/application Flink cluster on K8s via yamls.
> You should have your own image with user jar baked located at /opt/flink/usrlib. It could not be
> specified via config option. Usually, you could add new layer on the official docker image to
> build in the user jar.
>
> A more graceful solution is init container[1], which could download the user jar from remote
> storage or just copy from local directory /opt/flink/opt to /opt/flink/usrlib.
>
> [1]. https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-initialization/
>
> Best,
> Yang
>
>
> Felipe Gutierrez <fe...@gmail.com> 于2020年9月18日周五 下午5:52写道：
>>
>> Hi community,
>>
>> I am trying to deploy the default WordCount stream application on
>> minikube using the official documentation at [1]. I am using minikube
>> v1.13.0 on Ubuntu 18.04 and Kubernetes v1.19.0 on Docker 19.03.8. I
>> could sucessfully start 1 job manager and 3 task managers using the
>> yaml files flink-configuration-configmap.yaml,
>> jobmanager-service.yaml, jobmanager-rest-service.yaml,
>> jobmanager-session-deployment.yaml, and
>> taskmanager-session-deployment.yaml (all available on the Apendix of
>> this link [1]).
>>
>> Then I would like to start the word-count stream job available on the
>> flink jar image [2], which I believe is available since it is built
>> inside the default flink jar distribution. What I understood that I
>> have to do is to create the objects based on the files
>> jobmanager-job.yaml and taskmanager-job-deployment.yaml (also
>> available on the link [1]). And, I think that I have to replace this
>> line below on the object jobmanager-job.yaml
>> (spec.template.spec.containers.name[jobmanager]):
>>
>> args: ["standalone-job", "--job-classname",
>> "org.apache.flink.streaming.examples.wordcount.WordCount"]
>>
>> Is this correct? I am not sure if this is my entire error. I am
>> getting the message "Could not find the provided job class
>> (org.apache.flink.streaming.examples.wordcount.WordCount) in the user
>> lib directory (/opt/flink/usrlib)". As far as I know
>> "/opt/flink/usrlib" is the default directory. I am not sure if I have
>> to change the property: path /host/path/to/job/artifacts. This is my
>> log message of the pod error.
>> Do you guys have any idea of what I am missing in my configuration?
>>
>> Thanks, Felipe
>>
>> $ kubectl get pods
>> NAME READY STATUS RESTARTS AGE
>> flink-jobmanager-ftgg9 0/1 CrashLoopBackOff 3 83s
>> $ kubectl logs flink-jobmanager-ftgg9
>> Starting Job Manager
>> sed: couldn't open temporary file /opt/flink/conf/sedA699Jt: Read-only
>> file system
>> sed: couldn't open temporary file /opt/flink/conf/sedvZhs0w: Read-only
>> file system
>> /docker-entrypoint.sh: 72: /docker-entrypoint.sh: cannot create
>> /opt/flink/conf/flink-conf.yaml: Permission denied
>> /docker-entrypoint.sh: 91: /docker-entrypoint.sh: cannot create
>> /opt/flink/conf/flink-conf.yaml.tmp: Read-only file system
>> Starting standalonejob as a console application on host flink-jobmanager-ftgg9.
>> 2020-09-18 09:15:58,801 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> --------------------------------------------------------------------------------
>> 2020-09-18 09:15:58,804 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> Preconfiguration:
>> 2020-09-18 09:15:58,804 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>>
>>
>> JM_RESOURCE_PARAMS extraction logs:
>> jvm_params: -Xmx1073741824 -Xms1073741824 -XX:MaxMetaspaceSize=268435456
>> logs: INFO [] - Loading configuration property:
>> jobmanager.rpc.address, flink-jobmanager
>> INFO [] - Loading configuration property: taskmanager.numberOfTaskSlots, 4
>> INFO [] - Loading configuration property: blob.server.port, 6124
>> INFO [] - Loading configuration property: jobmanager.rpc.port, 6123
>> INFO [] - Loading configuration property: taskmanager.rpc.port, 6122
>> INFO [] - Loading configuration property: queryable-state.proxy.ports, 6125
>> INFO [] - Loading configuration property: jobmanager.memory.process.size, 1600m
>> INFO [] - Loading configuration property:
>> taskmanager.memory.process.size, 1728m
>> INFO [] - Loading configuration property: parallelism.default, 2
>> INFO [] - The derived from fraction jvm overhead memory (160.000mb
>> (167772162 bytes)) is less than its min value 192.000mb (201326592
>> bytes), min value will be used instead
>> INFO [] - Final Master Memory configuration:
>> INFO [] - Total Process Memory: 1.563gb (1677721600 bytes)
>> INFO [] - Total Flink Memory: 1.125gb (1207959552 bytes)
>> INFO [] - JVM Heap: 1024.000mb (1073741824 bytes)
>> INFO [] - Off-heap: 128.000mb (134217728 bytes)
>> INFO [] - JVM Metaspace: 256.000mb (268435456 bytes)
>> INFO [] - JVM Overhead: 192.000mb (201326592 bytes)
>>
>> 2020-09-18 09:15:58,805 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> --------------------------------------------------------------------------------
>> 2020-09-18 09:15:58,805 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> Starting StandaloneApplicationClusterEntryPoint (Version: 1.11.0,
>> Scala: 2.11, Rev:d04872d, Date:2020-06-29T16:13:14+02:00)
>> 2020-09-18 09:15:58,805 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - OS
>> current user: flink
>> 2020-09-18 09:15:58,805 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> Current Hadoop/Kerberos user: <no hadoop dependency found>
>> 2020-09-18 09:15:58,805 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.262-b10
>> 2020-09-18 09:15:58,806 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> Maximum heap size: 989 MiBytes
>> 2020-09-18 09:15:58,806 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> JAVA_HOME: /usr/local/openjdk-8
>> 2020-09-18 09:15:58,806 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - No
>> Hadoop Dependency available
>> 2020-09-18 09:15:58,806 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM
>> Options:
>> 2020-09-18 09:15:58,806 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> -Xmx1073741824
>> 2020-09-18 09:15:58,806 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> -Xms1073741824
>> 2020-09-18 09:15:58,807 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> -XX:MaxMetaspaceSize=268435456
>> 2020-09-18 09:15:58,807 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> -Dlog.file=/opt/flink/log/flink--standalonejob-0-flink-jobmanager-ftgg9.log
>> 2020-09-18 09:15:58,807 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> -Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties
>> 2020-09-18 09:15:58,807 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> -Dlog4j.configurationFile=file:/opt/flink/conf/log4j-console.properties
>> 2020-09-18 09:15:58,807 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> -Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml
>> 2020-09-18 09:15:58,807 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> Program Arguments:
>> 2020-09-18 09:15:58,807 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> --configDir
>> 2020-09-18 09:15:58,808 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> /opt/flink/conf
>> 2020-09-18 09:15:58,808 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> --job-classname
>> 2020-09-18 09:15:58,808 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> org.apache.flink.streaming.examples.wordcount.WordCount
>> 2020-09-18 09:15:58,808 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> Classpath: /opt/flink/lib/flink-csv-1.11.0.jar:/opt/flink/lib/flink-json-1.11.0.jar:/opt/flink/lib/flink-shaded-zookeeper-3.4.14.jar:/opt/flink/lib/flink-table-blink_2.11-1.11.0.jar:/opt/flink/lib/flink-table_2.11-1.11.0.jar:/opt/flink/lib/log4j-1.2-api-2.12.1.jar:/opt/flink/lib/log4j-api-2.12.1.jar:/opt/flink/lib/log4j-core-2.12.1.jar:/opt/flink/lib/log4j-slf4j-impl-2.12.1.jar:/opt/flink/lib/flink-dist_2.11-1.11.0.jar:::
>> 2020-09-18 09:15:58,808 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> --------------------------------------------------------------------------------
>> 2020-09-18 09:15:58,809 INFO
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> Registered UNIX signal handlers for [TERM, HUP, INT]
>> 2020-09-18 09:15:58,849 ERROR
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>> Could not create application program.
>> org.apache.flink.util.FlinkException: Could not find the provided job
>> class (org.apache.flink.streaming.examples.wordcount.WordCount) in the
>> user lib directory (/opt/flink/usrlib).
>> at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getJobClassNameOrScanClassPath(ClassPathPackagedProgramRetriever.java:140)
>> ~[flink-dist_2.11-1.11.0.jar:1.11.0]
>> at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getPackagedProgram(ClassPathPackagedProgramRetriever.java:123)
>> ~[flink-dist_2.11-1.11.0.jar:1.11.0]
>> at org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.getPackagedProgram(StandaloneApplicationClusterEntryPoint.java:110)
>> ~[flink-dist_2.11-1.11.0.jar:1.11.0]
>> at org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.main(StandaloneApplicationClusterEntryPoint.java:78)
>> [flink-dist_2.11-1.11.0.jar:1.11.0]
>>
>> [1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/kubernetes.html
>> [2] https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/wordcount/WordCount.java
>> --
>> -- Felipe Gutierrez
>> -- skype: felipe.o.gutierrez
>> -- https://felipeogutierrez.blogspot.com
Re: Error on deploying Flink docker image with Kubernetes (minikube)
and automatically launch a stream WordCount job.
Posted by Yang Wang <da...@gmail.com>.
Hi Felipe,
Currently, if you want to deploy a standalone job/application Flink cluster
on K8s via yamls.
You should have your own image with user jar baked located at
/opt/flink/usrlib. It could not be
specified via config option. Usually, you could add new layer on the
official docker image to
build in the user jar.
A more graceful solution is init container[1], which could download the
user jar from remote
storage or just copy from local directory /opt/flink/opt to
/opt/flink/usrlib.
[1].
https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-initialization/
Best,
Yang
Felipe Gutierrez <fe...@gmail.com> 于2020年9月18日周五 下午5:52写道：
> Hi community,
>
> I am trying to deploy the default WordCount stream application on
> minikube using the official documentation at [1]. I am using minikube
> v1.13.0 on Ubuntu 18.04 and Kubernetes v1.19.0 on Docker 19.03.8. I
> could sucessfully start 1 job manager and 3 task managers using the
> yaml files flink-configuration-configmap.yaml,
> jobmanager-service.yaml, jobmanager-rest-service.yaml,
> jobmanager-session-deployment.yaml, and
> taskmanager-session-deployment.yaml (all available on the Apendix of
> this link [1]).
>
> Then I would like to start the word-count stream job available on the
> flink jar image [2], which I believe is available since it is built
> inside the default flink jar distribution. What I understood that I
> have to do is to create the objects based on the files
> jobmanager-job.yaml and taskmanager-job-deployment.yaml (also
> available on the link [1]). And, I think that I have to replace this
> line below on the object jobmanager-job.yaml
> (spec.template.spec.containers.name[jobmanager]):
>
> args: ["standalone-job", "--job-classname",
> "org.apache.flink.streaming.examples.wordcount.WordCount"]
>
> Is this correct? I am not sure if this is my entire error. I am
> getting the message "Could not find the provided job class
> (org.apache.flink.streaming.examples.wordcount.WordCount) in the user
> lib directory (/opt/flink/usrlib)". As far as I know
> "/opt/flink/usrlib" is the default directory. I am not sure if I have
> to change the property: path /host/path/to/job/artifacts. This is my
> log message of the pod error.
> Do you guys have any idea of what I am missing in my configuration?
>
> Thanks, Felipe
>
> $ kubectl get pods
> NAME READY STATUS RESTARTS AGE
> flink-jobmanager-ftgg9 0/1 CrashLoopBackOff 3 83s
> $ kubectl logs flink-jobmanager-ftgg9
> Starting Job Manager
> sed: couldn't open temporary file /opt/flink/conf/sedA699Jt: Read-only
> file system
> sed: couldn't open temporary file /opt/flink/conf/sedvZhs0w: Read-only
> file system
> /docker-entrypoint.sh: 72: /docker-entrypoint.sh: cannot create
> /opt/flink/conf/flink-conf.yaml: Permission denied
> /docker-entrypoint.sh: 91: /docker-entrypoint.sh: cannot create
> /opt/flink/conf/flink-conf.yaml.tmp: Read-only file system
> Starting standalonejob as a console application on host
> flink-jobmanager-ftgg9.
> 2020-09-18 09:15:58,801 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>
> --------------------------------------------------------------------------------
> 2020-09-18 09:15:58,804 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> Preconfiguration:
> 2020-09-18 09:15:58,804 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>
>
> JM_RESOURCE_PARAMS extraction logs:
> jvm_params: -Xmx1073741824 -Xms1073741824 -XX:MaxMetaspaceSize=268435456
> logs: INFO [] - Loading configuration property:
> jobmanager.rpc.address, flink-jobmanager
> INFO [] - Loading configuration property: taskmanager.numberOfTaskSlots, 4
> INFO [] - Loading configuration property: blob.server.port, 6124
> INFO [] - Loading configuration property: jobmanager.rpc.port, 6123
> INFO [] - Loading configuration property: taskmanager.rpc.port, 6122
> INFO [] - Loading configuration property: queryable-state.proxy.ports,
> 6125
> INFO [] - Loading configuration property: jobmanager.memory.process.size,
> 1600m
> INFO [] - Loading configuration property:
> taskmanager.memory.process.size, 1728m
> INFO [] - Loading configuration property: parallelism.default, 2
> INFO [] - The derived from fraction jvm overhead memory (160.000mb
> (167772162 bytes)) is less than its min value 192.000mb (201326592
> bytes), min value will be used instead
> INFO [] - Final Master Memory configuration:
> INFO [] - Total Process Memory: 1.563gb (1677721600 bytes)
> INFO [] - Total Flink Memory: 1.125gb (1207959552 bytes)
> INFO [] - JVM Heap: 1024.000mb (1073741824 bytes)
> INFO [] - Off-heap: 128.000mb (134217728 bytes)
> INFO [] - JVM Metaspace: 256.000mb (268435456 bytes)
> INFO [] - JVM Overhead: 192.000mb (201326592 bytes)
>
> 2020-09-18 09:15:58,805 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>
> --------------------------------------------------------------------------------
> 2020-09-18 09:15:58,805 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> Starting StandaloneApplicationClusterEntryPoint (Version: 1.11.0,
> Scala: 2.11, Rev:d04872d, Date:2020-06-29T16:13:14+02:00)
> 2020-09-18 09:15:58,805 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - OS
> current user: flink
> 2020-09-18 09:15:58,805 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> Current Hadoop/Kerberos user: <no hadoop dependency found>
> 2020-09-18 09:15:58,805 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.262-b10
> 2020-09-18 09:15:58,806 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> Maximum heap size: 989 MiBytes
> 2020-09-18 09:15:58,806 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> JAVA_HOME: /usr/local/openjdk-8
> 2020-09-18 09:15:58,806 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - No
> Hadoop Dependency available
> 2020-09-18 09:15:58,806 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM
> Options:
> 2020-09-18 09:15:58,806 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Xmx1073741824
> 2020-09-18 09:15:58,806 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Xms1073741824
> 2020-09-18 09:15:58,807 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -XX:MaxMetaspaceSize=268435456
> 2020-09-18 09:15:58,807 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Dlog.file=/opt/flink/log/flink--standalonejob-0-flink-jobmanager-ftgg9.log
> 2020-09-18 09:15:58,807 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties
> 2020-09-18 09:15:58,807 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Dlog4j.configurationFile=file:/opt/flink/conf/log4j-console.properties
> 2020-09-18 09:15:58,807 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml
> 2020-09-18 09:15:58,807 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> Program Arguments:
> 2020-09-18 09:15:58,807 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> --configDir
> 2020-09-18 09:15:58,808 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> /opt/flink/conf
> 2020-09-18 09:15:58,808 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> --job-classname
> 2020-09-18 09:15:58,808 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> org.apache.flink.streaming.examples.wordcount.WordCount
> 2020-09-18 09:15:58,808 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> Classpath:
> /opt/flink/lib/flink-csv-1.11.0.jar:/opt/flink/lib/flink-json-1.11.0.jar:/opt/flink/lib/flink-shaded-zookeeper-3.4.14.jar:/opt/flink/lib/flink-table-blink_2.11-1.11.0.jar:/opt/flink/lib/flink-table_2.11-1.11.0.jar:/opt/flink/lib/log4j-1.2-api-2.12.1.jar:/opt/flink/lib/log4j-api-2.12.1.jar:/opt/flink/lib/log4j-core-2.12.1.jar:/opt/flink/lib/log4j-slf4j-impl-2.12.1.jar:/opt/flink/lib/flink-dist_2.11-1.11.0.jar:::
> 2020-09-18 09:15:58,808 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
>
> --------------------------------------------------------------------------------
> 2020-09-18 09:15:58,809 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> Registered UNIX signal handlers for [TERM, HUP, INT]
> 2020-09-18 09:15:58,849 ERROR
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> Could not create application program.
> org.apache.flink.util.FlinkException: Could not find the provided job
> class (org.apache.flink.streaming.examples.wordcount.WordCount) in the
> user lib directory (/opt/flink/usrlib).
> at
> org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getJobClassNameOrScanClassPath(ClassPathPackagedProgramRetriever.java:140)
> ~[flink-dist_2.11-1.11.0.jar:1.11.0]
> at
> org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getPackagedProgram(ClassPathPackagedProgramRetriever.java:123)
> ~[flink-dist_2.11-1.11.0.jar:1.11.0]
> at
> org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.getPackagedProgram(StandaloneApplicationClusterEntryPoint.java:110)
> ~[flink-dist_2.11-1.11.0.jar:1.11.0]
> at
> org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.main(StandaloneApplicationClusterEntryPoint.java:78)
> [flink-dist_2.11-1.11.0.jar:1.11.0]
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/kubernetes.html
> [2]
> https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/wordcount/WordCount.java
> --
> -- Felipe Gutierrez
> -- skype: felipe.o.gutierrez
> -- https://felipeogutierrez.blogspot.com
>