You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/06/17 03:58:38 UTC
[GitHub] [incubator-seatunnel] ic4y opened a new issue, #2026: [bug][api-draft] Disabling sql transform plugin in batch mode causes task execution to fail
ic4y opened a new issue, #2026:
URL: https://github.com/apache/incubator-seatunnel/issues/2026
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
When testing with flink `SeaTunnelApiExample`.
If cancel the transform sql configuration in the `fake_to_console.conf` configuration file. Like this
<img width="966" alt="image" src="https://user-images.githubusercontent.com/83933160/174221224-e875bfc4-f162-4737-a250-832eb48dcdc7.png">
This will cause the task to fail with the following error
```
Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: Flink job executed failed
at org.apache.seatunnel.core.starter.flink.command.FlinkApiTaskExecuteCommand.execute(FlinkApiTaskExecuteCommand.java:57)
at org.apache.seatunnel.core.starter.Seatunnel.run(Seatunnel.java:40)
at org.apache.seatunnel.example.flink.SeaTunnelApiExample.main(SeaTunnelApiExample.java:41)
Caused by: java.lang.IllegalStateException: Detected an UNBOUNDED source with the 'execution.runtime-mode' set to 'BATCH'. This combination is not allowed, please set the 'execution.runtime-mode' to STREAMING or AUTOMATIC
at org.apache.flink.util.Preconditions.checkState(Preconditions.java:193)
at org.apache.flink.streaming.api.graph.StreamGraphGenerator.shouldExecuteInBatchMode(StreamGraphGenerator.java:390)
at org.apache.flink.streaming.api.graph.StreamGraphGenerator.generate(StreamGraphGenerator.java:297)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:2009)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionPlan(StreamExecutionEnvironment.java:2042)
at org.apache.seatunnel.core.starter.flink.execution.FlinkExecution.execute(FlinkExecution.java:65)
at org.apache.seatunnel.core.starter.flink.command.FlinkApiTaskExecuteCommand.execute(FlinkApiTaskExecuteCommand.java:55)
... 2 more
```
### SeaTunnel Version
api-draft branch
### SeaTunnel Config
```conf
env {
# You can set flink configuration here
execution.parallelism = 1
job.mode = "BATCH"
}
source {
# This is a example source plugin **only for test and demonstrate the feature source plugin**
FakeSource {
result_table_name = "fake"
field_name = "name,age"
}
}
transform {
# sql {
# sql = "select name,age from fake"
# }
}
sink {
Console {}
}
```
### Running Command
```shell
local test
```
### Error Exception
```log
22/06/17 10:23:17 ERROR Seatunnel: Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: Flink job executed failed
at org.apache.seatunnel.core.starter.flink.command.FlinkApiTaskExecuteCommand.execute(FlinkApiTaskExecuteCommand.java:57)
at org.apache.seatunnel.core.starter.Seatunnel.run(Seatunnel.java:40)
at org.apache.seatunnel.example.flink.SeaTunnelApiExample.main(SeaTunnelApiExample.java:41)
Caused by: java.lang.IllegalStateException: Detected an UNBOUNDED source with the 'execution.runtime-mode' set to 'BATCH'. This combination is not allowed, please set the 'execution.runtime-mode' to STREAMING or AUTOMATIC
at org.apache.flink.util.Preconditions.checkState(Preconditions.java:193)
at org.apache.flink.streaming.api.graph.StreamGraphGenerator.shouldExecuteInBatchMode(StreamGraphGenerator.java:390)
at org.apache.flink.streaming.api.graph.StreamGraphGenerator.generate(StreamGraphGenerator.java:297)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:2009)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionPlan(StreamExecutionEnvironment.java:2042)
at org.apache.seatunnel.core.starter.flink.execution.FlinkExecution.execute(FlinkExecution.java:65)
at org.apache.seatunnel.core.starter.flink.command.FlinkApiTaskExecuteCommand.execute(FlinkApiTaskExecuteCommand.java:55)
... 2 more
```
### Flink or Spark Version
_No response_
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2026: [bug][api-draft] Disabling sql transform plugin in batch mode causes task execution to fail
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2026:
URL: https://github.com/apache/incubator-seatunnel/issues/2026#issuecomment-1166769532
Close by #2038
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X closed issue #2026: [bug][api-draft] Disabling sql transform plugin in batch mode causes task execution to fail
Posted by GitBox <gi...@apache.org>.
Hisoka-X closed issue #2026: [bug][api-draft] Disabling sql transform plugin in batch mode causes task execution to fail
URL: https://github.com/apache/incubator-seatunnel/issues/2026
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] wuchunfu commented on issue #2026: [bug][api-draft] Disabling sql transform plugin in batch mode causes task execution to fail
Posted by GitBox <gi...@apache.org>.
wuchunfu commented on issue #2026:
URL: https://github.com/apache/incubator-seatunnel/issues/2026#issuecomment-1158741462
> This is a bug. `org.apache.seatunnel.core.starter.flink.execution.SourceExecuteProcessor` don't determine whether the source is bounded when adding the source. It uses `org.apache.flink.streaming.api.environment.StreamExecutionEnvironment#addSource(org.apache.flink.streaming.api.functions.source.SourceFunction<OUT>)` instead of `org.apache.flink.streaming.api.datastream.DataStreamSource#DataStreamSource(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment, org.apache.flink.api.common.typeinfo.TypeInformation<T>, org.apache.flink.streaming.api.operators.StreamSource<T,?>, boolean, java.lang.String, org.apache.flink.api.connector.source.Boundedness)`.
@ashulin I also encountered this problem, do you have a good solution?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ashulin commented on issue #2026: [bug][api-draft] Disabling sql transform plugin in batch mode causes task execution to fail
Posted by GitBox <gi...@apache.org>.
ashulin commented on issue #2026:
URL: https://github.com/apache/incubator-seatunnel/issues/2026#issuecomment-1160376136
https://github.com/apache/incubator-seatunnel/pull/2038
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ashulin commented on issue #2026: [bug][api-draft] Disabling sql transform plugin in batch mode causes task execution to fail
Posted by GitBox <gi...@apache.org>.
ashulin commented on issue #2026:
URL: https://github.com/apache/incubator-seatunnel/issues/2026#issuecomment-1158470400
This is a bug. `org.apache.seatunnel.core.starter.flink.execution.SourceExecuteProcessor` don't determine whether the source is bounded when adding the source. It uses `org.apache.flink.streaming.api.environment.StreamExecutionEnvironment#addSource(org.apache.flink.streaming.api.functions.source.SourceFunction<OUT>)` instead of `org.apache.flink.streaming.api.datastream.DataStreamSource#DataStreamSource(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment, org.apache.flink.api.common.typeinfo.TypeInformation<T>, org.apache.flink.streaming.api.operators.StreamSource<T,?>, boolean, java.lang.String, org.apache.flink.api.connector.source.Boundedness)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org