You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/05/11 10:30:58 UTC
[GitHub] [incubator-seatunnel] mosence opened a new issue, #1852: [Bug] [Docs] Seatunnel Version: 2.1.1 donnot use flink version 1.14.4 engine, but the quick start support more than 1.9.0.
mosence opened a new issue, #1852:
URL: https://github.com/apache/incubator-seatunnel/issues/1852
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
When I use flink 1.14.4 engine, cannot run quick start command.
It that the document all right ?
### SeaTunnel Version
2.1.1
### SeaTunnel Config
```conf
env {
# You can set flink configuration here
execution.parallelism = 1
#execution.checkpoint.interval = 10000
#execution.checkpoint.data-uri = "hdfs://localhost:9000/checkpoint"
}
source {
# This is a example input plugin **only for test and demonstrate the feature input plugin**
FakeSourceStream {
result_table_name = "fake"
field_name = "name,age"
}
# If you would like to get more information about how to configure seatunnel and see full list of input plugins,
# please go to https://seatunnel.apache.org/docs/flink/configuration/source-plugins/Fake
}
transform {
sql {
sql = "select name,age from fake"
}
# If you would like to get more information about how to configure seatunnel and see full list of filter plugins,
# please go to https://seatunnel.apache.org/docs/flink/configuration/transform-plugins/Sql
}
sink {
ConsoleSink {}
# If you would like to get more information about how to configure seatunnel and see full list of output plugins,
# please go to https://seatunnel.apache.org/docs/flink/configuration/sink-plugins/Console
}
```
### Running Command
```shell
./bin/start-seatunnel-flink.sh --config ./config/flink.streaming.conf.template
```
### Error Exception
```log
------------------------------------------------------------
The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: The old planner has been removed in Flink 1.14. Please upgrade your table program to use the default planner (previously called the 'blink' planner).
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: org.apache.flink.table.api.TableException: The old planner has been removed in Flink 1.14. Please upgrade your table program to use the default planner (previously called the 'blink' planner).
at org.apache.flink.table.api.EnvironmentSettings$Builder.useOldPlanner(EnvironmentSettings.java:238)
at org.apache.seatunnel.flink.FlinkEnvironment.createStreamTableEnvironment(FlinkEnvironment.java:129)
at org.apache.seatunnel.flink.FlinkEnvironment.prepare(FlinkEnvironment.java:83)
at org.apache.seatunnel.flink.FlinkEnvironment.prepare(FlinkEnvironment.java:45)
at org.apache.seatunnel.config.EnvironmentFactory.getEnvironment(EnvironmentFactory.java:61)
at org.apache.seatunnel.config.ExecutionContext.<init>(ExecutionContext.java:49)
at org.apache.seatunnel.command.flink.FlinkTaskExecuteCommand.execute(FlinkTaskExecuteCommand.java:47)
at org.apache.seatunnel.command.flink.FlinkTaskExecuteCommand.execute(FlinkTaskExecuteCommand.java:39)
at org.apache.seatunnel.Seatunnel.run(Seatunnel.java:48)
at org.apache.seatunnel.SeatunnelFlink.main(SeatunnelFlink.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
... 8 more
```
### Flink or Spark Version
flink 1.14.4
### Java or Scala Version
1.8
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ruanwenjun commented on issue #1852: [Bug] [Docs] Seatunnel Version: 2.1.1 donnot use flink version 1.14.4 engine, but the quick start support more than 1.9.0.
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #1852:
URL: https://github.com/apache/incubator-seatunnel/issues/1852#issuecomment-1124053334
@mosence You can try to set `execution.planner=blink` to set the planner to `blink`, and I think we can set the default planner to `blink`, are you interested to submit a PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ruanwenjun commented on issue #1852: [Bug] [Docs] Seatunnel Version: 2.1.1 donnot use flink version 1.14.4 engine, but the quick start support more than 1.9.0.
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #1852:
URL: https://github.com/apache/incubator-seatunnel/issues/1852#issuecomment-1125044138
Closed by #1860
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ruanwenjun commented on issue #1852: [Bug] [Docs] Seatunnel Version: 2.1.1 donnot use flink version 1.14.4 engine, but the quick start support more than 1.9.0.
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #1852:
URL: https://github.com/apache/incubator-seatunnel/issues/1852#issuecomment-1124468915
@mosence This may be caused by flink version conflict.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] legendtkl commented on issue #1852: [Bug] [Docs] Seatunnel Version: 2.1.1 donnot use flink version 1.14.4 engine, but the quick start support more than 1.9.0.
Posted by GitBox <gi...@apache.org>.
legendtkl commented on issue #1852:
URL: https://github.com/apache/incubator-seatunnel/issues/1852#issuecomment-1124600916
I think flink APIs upgrade is necessary. @ruanwenjun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] mosence commented on issue #1852: [Bug] [Docs] Seatunnel Version: 2.1.1 donnot use flink version 1.14.4 engine, but the quick start support more than 1.9.0.
Posted by GitBox <gi...@apache.org>.
mosence commented on issue #1852:
URL: https://github.com/apache/incubator-seatunnel/issues/1852#issuecomment-1124456125
@ruanwenjun
I set the SeaTunnel config 'execution.planner=blink' in env, but the NPE exception throws:
It seem that the Flink BatchAPI not distribute in flink-dist.jar.
It that mean we must rewrite BatchAPI to StreamAPI?
```log
java.lang.NoClassDefFoundError: org/apache/flink/table/api/bridge/java/BatchTableEnvironment
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.getDeclaredMethods(Class.java:1975)
at org.apache.flink.api.java.typeutils.TypeExtractionUtils.getAllDeclaredMethods(TypeExtractionUtils.java:269)
at org.apache.flink.api.java.typeutils.TypeExtractionUtils.checkAndExtractLambda(TypeExtractionUtils.java:149)
at org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:536)
at org.apache.flink.api.java.typeutils.TypeExtractor.getMapReturnTypes(TypeExtractor.java:151)
at org.apache.flink.streaming.api.datastream.DataStream.map(DataStream.java:576)
at org.apache.seatunnel.flink.util.TableUtil.tableToDataStream(TableUtil.java:45)
at org.apache.seatunnel.flink.transform.Sql.processStream(Sql.java:53)
at org.apache.seatunnel.flink.stream.FlinkStreamExecution.start(FlinkStreamExecution.java:64)
at org.apache.seatunnel.command.flink.FlinkTaskExecuteCommand.execute(FlinkTaskExecuteCommand.java:60)
at org.apache.seatunnel.command.flink.FlinkTaskExecuteCommand.execute(FlinkTaskExecuteCommand.java:39)
at org.apache.seatunnel.Seatunnel.run(Seatunnel.java:48)
at org.apache.seatunnel.SeatunnelFlink.main(SeatunnelFlink.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: java.lang.ClassNotFoundException: org.apache.flink.table.api.bridge.java.BatchTableEnvironment
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:65)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 28 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] mosence commented on issue #1852: [Bug] [Docs] Seatunnel Version: 2.1.1 donnot use flink version 1.14.4 engine, but the quick start support more than 1.9.0.
Posted by GitBox <gi...@apache.org>.
mosence commented on issue #1852:
URL: https://github.com/apache/incubator-seatunnel/issues/1852#issuecomment-1124481951
@ruanwenjun I create a PR in docs‘s project, add flink version limit. And this problem need to resolve in future? I think it must bean resolve after the unified API .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ruanwenjun commented on issue #1852: [Bug] [Docs] Seatunnel Version: 2.1.1 donnot use flink version 1.14.4 engine, but the quick start support more than 1.9.0.
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #1852:
URL: https://github.com/apache/incubator-seatunnel/issues/1852#issuecomment-1124856275
> I think flink APIs upgrade is necessary. @ruanwenjun
Yes, we should remove the deprecated API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ruanwenjun commented on issue #1852: [Bug] [Docs] Seatunnel Version: 2.1.1 donnot use flink version 1.14.4 engine, but the quick start support more than 1.9.0.
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #1852:
URL: https://github.com/apache/incubator-seatunnel/issues/1852#issuecomment-1124510556
Yes, after unified API, we can easy to do different version distriibution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ruanwenjun closed issue #1852: [Bug] [Docs] Seatunnel Version: 2.1.1 donnot use flink version 1.14.4 engine, but the quick start support more than 1.9.0.
Posted by GitBox <gi...@apache.org>.
ruanwenjun closed issue #1852: [Bug] [Docs] Seatunnel Version: 2.1.1 donnot use flink version 1.14.4 engine, but the quick start support more than 1.9.0.
URL: https://github.com/apache/incubator-seatunnel/issues/1852
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org