You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "kksxf (via GitHub)" <gi...@apache.org> on 2023/05/19 12:40:29 UTC
[GitHub] [incubator-seatunnel] kksxf opened a new issue, #4788: [Bug] failed to run seatunnel with flink operator following the guidance of document
kksxf opened a new issue, #4788:
URL: https://github.com/apache/incubator-seatunnel/issues/4788
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
I tried to run seatunnel demo following the guidance of [seatunnel document](https://seatunnel.apache.org/docs/2.3.1/start-v2/kubernetes/), but an error occurred after deployment using `kubectl apply -f seatunnel-flink.yaml`
```
2023-05-19 12:35:17,812 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Could not create application program.
org.apache.flink.util.FlinkException: Could not load the provided entrypoint class.
at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getPackagedProgram(ClassPathPackagedProgramRetriever.java:178) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint.getPackagedProgram(KubernetesApplicationClusterEntrypoint.java:101) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint.main(KubernetesApplicationClusterEntrypoint.java:71) [flink-dist_2.12-1.13.6.jar:1.13.6]
Caused by: org.apache.flink.client.program.ProgramInvocationException: JAR file does not exist '/opt/seatunnel/starter/seatunnel-flink-starter.jar'
at org.apache.flink.client.program.PackagedProgram.checkJarFile(PackagedProgram.java:617) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.loadJarFile(PackagedProgram.java:465) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:135) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:65) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram$Builder.build(PackagedProgram.java:691) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getPackagedProgram(ClassPathPackagedProgramRetriever.java:167) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
... 2 more
Caused by: java.io.IOException: JAR file does not exist '/opt/seatunnel/starter/seatunnel-flink-starter.jar'
at org.apache.flink.util.JarUtils.checkJarFile(JarUtils.java:46) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.checkJarFile(PackagedProgram.java:615) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.loadJarFile(PackagedProgram.java:465) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:135) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:65) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram$Builder.build(PackagedProgram.java:691) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getPackagedProgram(ClassPathPackagedProgramRetriever.java:167) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
... 2 more
```
**Dockerfile**
```
FROM flink:1.13
ENV SEATUNNEL_VERSION="2.3.1"
ENV SEATUNNEL_HOME = "/opt/seatunnel"
RUN mkdir -p $SEATUNNEL_HOME
RUN wget https://archive.apache.org/dist/incubator/seatunnel/${SEATUNNEL_VERSION}/apache-seatunnel-incubating-${SEATUNNEL_VERSION}-bin.tar.gz
RUN tar -xzvf apache-seatunnel-incubating-${SEATUNNEL_VERSION}-bin.tar.gz
RUN cp -r apache-seatunnel-incubating-${SEATUNNEL_VERSION}/* $SEATUNNEL_HOME/
RUN rm -rf apache-seatunnel-incubating-${SEATUNNEL_VERSION}*
```
**FlinkDeployment**
```
apiVersion: flink.apache.org/v1beta1
kind: FlinkDeployment
metadata:
name: seatunnel-flink-streaming-example
spec:
image: seatunnel:2.3.1-flink-1.13
flinkVersion: v1_13
flinkConfiguration:
taskmanager.numberOfTaskSlots: "2"
serviceAccount: flink
jobManager:
replicas: 1
resource:
memory: "1024m"
cpu: 1
taskManager:
resource:
memory: "1024m"
cpu: 1
podTemplate:
spec:
containers:
- name: flink-main-container
volumeMounts:
- name: seatunnel-config
mountPath: /data/seatunnel.streaming.conf
subPath: seatunnel.streaming.conf
volumes:
- name: seatunnel-config
configMap:
name: seatunnel-config
items:
- key: seatunnel.streaming.conf
path: seatunnel.streaming.conf
job:
jarURI: local:///opt/seatunnel/starter/seatunnel-flink-starter.jar
entryClass: org.apache.seatunnel.core.starter.flink.SeatunnelFlink
args: ["--config", "/data/seatunnel.streaming.conf"]
parallelism: 2
upgradeMode: stateless
```
I believe there are several places in the document need to be updated or corrected in SeaTunnel 2.3.1
I will summarize theses places to help the demo work in SeaTunnel 2.3.1 and maybe submit a PR later if that helps
### SeaTunnel Version
2.3.1
### SeaTunnel Config
```conf
env {
execution.parallelism = 1
job.mode = "STREAMING"
checkpoint.interval = 2000
}
source {
FakeSource {
result_table_name = "fake"
row.num = 160000
schema = {
fields {
name = "string"
age = "int"
}
}
}
}
transform {
}
sink {
Console {}
}
```
### Running Command
```shell
kubectl apply -f seatunnel-flink.yaml
```
### Error Exception
```log
2023-05-19 12:35:17,812 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Could not create application program.
org.apache.flink.util.FlinkException: Could not load the provided entrypoint class.
at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getPackagedProgram(ClassPathPackagedProgramRetriever.java:178) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint.getPackagedProgram(KubernetesApplicationClusterEntrypoint.java:101) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint.main(KubernetesApplicationClusterEntrypoint.java:71) [flink-dist_2.12-1.13.6.jar:1.13.6]
Caused by: org.apache.flink.client.program.ProgramInvocationException: JAR file does not exist '/opt/seatunnel/starter/seatunnel-flink-starter.jar'
at org.apache.flink.client.program.PackagedProgram.checkJarFile(PackagedProgram.java:617) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.loadJarFile(PackagedProgram.java:465) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:135) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:65) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram$Builder.build(PackagedProgram.java:691) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getPackagedProgram(ClassPathPackagedProgramRetriever.java:167) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
... 2 more
Caused by: java.io.IOException: JAR file does not exist '/opt/seatunnel/starter/seatunnel-flink-starter.jar'
at org.apache.flink.util.JarUtils.checkJarFile(JarUtils.java:46) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.checkJarFile(PackagedProgram.java:615) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.loadJarFile(PackagedProgram.java:465) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:135) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:65) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.program.PackagedProgram$Builder.build(PackagedProgram.java:691) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getPackagedProgram(ClassPathPackagedProgramRetriever.java:167) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
... 2 more
```
### Flink or Spark Version
flink version 1.13
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on issue #4788: [Bug] failed to run seatunnel with flink operator following the guidance of document
Posted by "liugddx (via GitHub)" <gi...@apache.org>.
liugddx commented on issue #4788:
URL: https://github.com/apache/incubator-seatunnel/issues/4788#issuecomment-1555787190
> As mentioned earlier, there are several places in the document: [Set Up with Kubernetes v2.3.1](https://seatunnel.apache.org/docs/2.3.1/start-v2/kubernetes/) that have to be modified if you want to run the demo on flink operator (flink version 1.13)
>
> 1. The jarURI in FlinkDeployment should be modified to `local:///opt/seatunnel/starter/seatunnel-flink-13-starter.jar`
> 2. The entryClass in FlinkDeployment should be modified to `org.apache.seatunnel.core.starter.flink.SeaTunnelFlink`
> 3. The redundant whitespace around the equal sign have to be removed while setting `SEATUNNEL_HOME` in Dockerfile, otherwise the seatunnel cannot find the plugins in the right path
>
> If i am wrong about something, please let me know
Great! Can you submit a pr?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] kksxf commented on issue #4788: [Bug] failed to run seatunnel with flink operator following the guidance of document
Posted by "kksxf (via GitHub)" <gi...@apache.org>.
kksxf commented on issue #4788:
URL: https://github.com/apache/incubator-seatunnel/issues/4788#issuecomment-1555904260
> > As mentioned earlier, there are several places in the document: [Set Up with Kubernetes v2.3.1](https://seatunnel.apache.org/docs/2.3.1/start-v2/kubernetes/) that have to be modified if you want to run the demo on flink operator (flink version 1.13)
> >
> > 1. The jarURI in FlinkDeployment should be modified to `local:///opt/seatunnel/starter/seatunnel-flink-13-starter.jar`
> > 2. The entryClass in FlinkDeployment should be modified to `org.apache.seatunnel.core.starter.flink.SeaTunnelFlink`
> > 3. The redundant whitespace around the equal sign have to be removed while setting `SEATUNNEL_HOME` in Dockerfile, otherwise the seatunnel cannot find the plugins in the right path
> >
> > If i am wrong about something, please let me know
>
> Great! Can you submit a pr?
Sure, i will submit a PR later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] github-actions[bot] commented on issue #4788: [Bug] failed to run seatunnel with flink operator following the guidance of document
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4788:
URL: https://github.com/apache/seatunnel/issues/4788#issuecomment-1608526255
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] github-actions[bot] commented on issue #4788: [Bug] failed to run seatunnel with flink operator following the guidance of document
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4788:
URL: https://github.com/apache/seatunnel/issues/4788#issuecomment-1597905493
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] github-actions[bot] closed issue #4788: [Bug] failed to run seatunnel with flink operator following the guidance of document
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #4788: [Bug] failed to run seatunnel with flink operator following the guidance of document
URL: https://github.com/apache/seatunnel/issues/4788
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] kksxf commented on issue #4788: [Bug] failed to run seatunnel with flink operator following the guidance of document
Posted by "kksxf (via GitHub)" <gi...@apache.org>.
kksxf commented on issue #4788:
URL: https://github.com/apache/incubator-seatunnel/issues/4788#issuecomment-1555773632
As mentioned earlier, there are several places in the document: [Set Up with Kubernetes v2.3.1](https://seatunnel.apache.org/docs/2.3.1/start-v2/kubernetes/) that have to be modified if you want to run the demo on flink operator (flink version 1.13)
1. The jarURI in FlinkDeployment should be modified to `local:///opt/seatunnel/starter/seatunnel-flink-13-starter.jar`
2. The entryClass in FlinkDeployment should be modified to `org.apache.seatunnel.core.starter.flink.SeaTunnelFlink`
3. The redundant whitespace around the equal sign have to be removed while setting `SEATUNNEL_HOME` in Dockerfile, otherwise the seatunnel cannot find the plugins in the right path
If i am wrong about something, please let me know
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org