You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "chestnut-c (via GitHub)" <gi...@apache.org> on 2025/07/07 07:36:10 UTC
[I] [Bug] [zeta] 'checkpoint.interval' configuration of env is missing, checkpoint be disabled, but still create storage. [seatunnel]
chestnut-c opened a new issue, #9542:
URL: https://github.com/apache/seatunnel/issues/9542
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
Remove the 'checkpoint.interval' from the task configuration environment variable. Also, checkpointEnable=false is still triggering the creation of checkpoint storage.
### SeaTunnel Version
2.3.11
### SeaTunnel Config
```conf
env {
# You can set SeaTunnel environment configuration here
parallelism = 2
job.mode = "BATCH"
}
source {
# This is a example source plugin **only for test and demonstrate the feature source plugin**
FakeSource {
parallelism = 2
plugin_output = "fake"
row.num = 16
schema = {
fields {
name = "string"
age = "int"
}
}
}
# If you would like to get more information about how to configure SeaTunnel and see full list of source plugins,
# please go to https://seatunnel.apache.org/docs/connector-v2/source
}
sink {
Console {
}
# If you would like to get more information about how to configure SeaTunnel and see full list of sink plugins,
# please go to https://seatunnel.apache.org/docs/connector-v2/sink
}
```
### Running Command
```shell
--config ./config/v2.batch.config.template -m local
```
### Error Exception
```log
Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:228)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:40)
Caused by: java.util.concurrent.CompletionException: org.apache.seatunnel.engine.common.exception.JobException: java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream
at org.apache.seatunnel.engine.checkpoint.storage.hdfs.common.HdfsFileStorageInstance.getOrCreateStorage(HdfsFileStorageInstance.java:53)
at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorageFactory.create(HdfsStorageFactory.java:74)
at org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.<init>(CheckpointManager.java:107)
at org.apache.seatunnel.engine.server.master.JobMaster.initCheckPointManager(JobMaster.java:329)
at org.apache.seatunnel.engine.server.master.JobMaster.init(JobMaster.java:312)
at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$6(CoordinatorService.java:656)
at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:43)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FSDataInputStream
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:359)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
```
### Zeta or Flink or Spark Version
zeta
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Zeta] 'checkpoint.interval' configuration of env is missing, checkpoint be disabled, but still create storage. [seatunnel]
Posted by "corgy-w (via GitHub)" <gi...@apache.org>.
corgy-w closed issue #9542: [Bug] [Zeta] 'checkpoint.interval' configuration of env is missing, checkpoint be disabled, but still create storage.
URL: https://github.com/apache/seatunnel/issues/9542
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Zeta] 'checkpoint.interval' configuration of env is missing, checkpoint be disabled, but still create storage. [seatunnel]
Posted by "chestnut-c (via GitHub)" <gi...@apache.org>.
chestnut-c commented on issue #9542:
URL: https://github.com/apache/seatunnel/issues/9542#issuecomment-3055768866
> Hi [@chestnut-c](https://github.com/chestnut-c) , I'm a little curious why this error occurs. I download 2.3.11 and worked fine.
>
> <img alt="Image" width="1118" height="686" src="https://private-user-images.githubusercontent.com/32387433/463960348-2692de4e-d9bd-4a3b-b497-84e144d3127a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTIxMjg2NjAsIm5iZiI6MTc1MjEyODM2MCwicGF0aCI6Ii8zMjM4NzQzMy80NjM5NjAzNDgtMjY5MmRlNGUtZDliZC00YTNiLWI0OTctODRlMTQ0ZDMxMjdhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA3MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNzEwVDA2MTkyMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTNkODBjM2NjYTMxYmFhNGEzMmYwNTBmODlhNTQxNmFjOWY5MmRjMzczMTY4ZmQ3MzRiOGQ4ZGViMDJmMzY3ZTImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.i4OHV7KtyBMzvlW7hKIfuGaAvsDh7LFNfSQ5mJ_KXY8">
> Have you check seatunnel/lib contains hadoop jar?
hi @Hisoka-X Yes, I didn't report this error later, but here we need to see whether we need to reset the checkpoint storage after checkpoint will be disabled, and no longer use the checkpoint storage configuration in seatunnel.yaml. This is my confusion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Zeta] 'checkpoint.interval' configuration of env is missing, checkpoint be disabled, but still create storage. [seatunnel]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on issue #9542:
URL: https://github.com/apache/seatunnel/issues/9542#issuecomment-3050917474
Hi @chestnut-c , I'm a little curious why this error occurs. I download 2.3.11 and worked fine.
<img width="1118" height="686" alt="Image" src="https://github.com/user-attachments/assets/2692de4e-d9bd-4a3b-b497-84e144d3127a" />
Have you check seatunnel/lib contains hadoop jar?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org