You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "wanwei120 (via GitHub)" <gi...@apache.org> on 2023/04/13 11:10:45 UTC
[GitHub] [incubator-seatunnel] wanwei120 opened a new issue, #4568: [Bug] [seatunnel-translation] java.lang.NullPointerException at org.apache.seatunnel.translation.flink.source.BaseSeatunnelSourceFunction.lambda$initializeState$0(BaseSeatunnelSourceFunction.java 155)
wanwei120 opened a new issue, #4568:
URL: https://github.com/apache/incubator-seatunnel/issues/4568
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
The function implemented by the seatunnel task is to synchronize data from Kafka to ES. When the Seatunnel program restarts abnormally, it cannot restart the task from the checkpoint of Flink. The seatunnel task will continue to restart until the taskmanager crashes. This issue occurs when deploying Flink in both k8s and non k8s scenarios.
### SeaTunnel Version
SeaTunnel 2.3.0
kafka 2.12-2.2.0
es 8.5
### SeaTunnel Config
```conf
env
{
job.name="xxx"
job.mode="STREAMING"
}
source
{
parallelism=3
topic = "xxx"
bootstrap.servers ="xxxx:9092"
consumer.group = "seatunnel2"
kafka.auto.offset.reset ="earliest"
kafka.enable.auto.commit="true"
start_mode="earliest"
result_table_name="xxx"
schema ={fields{xxxx=xxxx}}
}
sink
{
Elasticesarch
{
parallelism=10
es.cluster.name = "xxx"
hosts=[xxx]
username = "xx"
password = "xx"
index = "xx"
max_batch_size =10000
}
}
```
### Running Command
```shell
flink run-application --target kubernetes-application -Dkuburnetes.namespace=flinkapp -Dkubernetes.jobmanager.service-account = flinkapp -Dkubernetes.rest-service.exposed.type=NodePort -Dkubernetes.container.image=xxx -Dkubernetes.pod-template-file= xxx -Dkubernetes.cluster-id =xx -c org.apache.seatunnel.core.starter.flink.SeatunnelFlink local:///opt/seatunnel/starter/seatunnel-flink-starter.jar --config xxx
```
### Error Exception
```log
java.lang.NullPointerException at org.apache.seatunnel.translation.flink.source.BaseSeatunnelSourceFunction.lambda$initializeState$0(BaseSeatunnelSourceFunction.java 155)
at java.util.ArrayList.foreach(ArrayList.java:1259)
...
```
### Flink or Spark Version
flink:1.13.6
### Java or Scala Version
java:1.8
scala:2.11
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #4568: [Bug] [seatunnel-translation] java.lang.NullPointerException at org.apache.seatunnel.translation.flink.source.BaseSeatunnelSourceFunction.lambda$initializeState$0(BaseSeatunnelSourceFunction.java 155)
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4568:
URL: https://github.com/apache/incubator-seatunnel/issues/4568#issuecomment-1546775967
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org