You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "liuyongfei1 (via GitHub)" <gi...@apache.org> on 2023/05/05 08:18:58 UTC
[GitHub] [incubator-seatunnel] liuyongfei1 opened a new issue, #4707: [Bug] [connector-v2] Can't find config file when spark standalone cluster submit
liuyongfei1 opened a new issue, #4707:
URL: https://github.com/apache/incubator-seatunnel/issues/4707
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
when spark standalone cluster submit seatunnel job，Can't find config file。
一：config.file:
env {
execution.parallelism = 1
job.mode = "BATCH"
}
source {
FakeSource {
result_table_name = "fake"
row.num = 16
schema = {
fields {
name = "string"
age = "int"
}
}
}
}
sink {
Console {}
}
二. command
/data/app/apache-seatunnel-incubating-2.3.1/bin/start-seatunnel-spark-2-connector-v2.sh --master spark://starrocks-33:7077 --deploy-mode cluster --config /data/app/apache-seatunnel-incubating-2.3.1/config/seatunnel.streaming.conf_lyf_test_0505.template
三.Following error occurred:
23/05/05 16:10:35 ERROR SeaTunnel:
===============================================================================
23/05/05 16:10:35 ERROR SeaTunnel: Fatal Error,
23/05/05 16:10:35 ERROR SeaTunnel: Please submit bug report in https://github.com/apache/incubator-seatunnel/issues
23/05/05 16:10:35 ERROR SeaTunnel: Reason:ErrorCode:[COMMON-01], ErrorDescription:[File operation failed, such as (read,list,write,move,copy,sync) etc...] - Can't find config file: seatunnel.streaming.conf_lyf_test_0505.template
23/05/05 16:10:35 ERROR SeaTunnel: Exception StackTrace:org.apache.seatunnel.common.exception.SeaTunnelRuntimeException: ErrorCode:[COMMON-01], ErrorDescription:[File operation failed, such as (read,list,write,move,copy,sync) etc...] - Can't find config file: seatunnel.streaming.conf_lyf_test_0505.template
at org.apache.seatunnel.core.starter.utils.FileUtils.checkConfigExist(FileUtils.java:68)
at org.apache.seatunnel.core.starter.spark.command.SparkTaskExecuteCommand.execute(SparkTaskExecuteCommand.java:50)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.spark.SeaTunnelSpark.main(SeaTunnelSpark.java:35)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.worker.DriverWrapper$.main(DriverWrapper.scala:65)
at org.apache.spark.deploy.worker.DriverWrapper.main(DriverWrapper.scala)
23/05/05 16:10:35 ERROR SeaTunnel:
### SeaTunnel Version
2.3.1
### SeaTunnel Config
```conf
env {
execution.parallelism = 1
job.mode = "BATCH"
}
source {
FakeSource {
result_table_name = "fake"
row.num = 16
schema = {
fields {
name = "string"
age = "int"
}
}
}
}
sink {
Console {}
}
```
### Running Command
```shell
/data/app/apache-seatunnel-incubating-2.3.1/bin/start-seatunnel-spark-2-connector-v2.sh --master spark://starrocks-33:7077 --deploy-mode cluster --config /data/app/apache-seatunnel-incubating-2.3.1/config/seatunnel.streaming.conf_lyf_test_0505.template
```
### Error Exception
```log
23/05/05 16:10:35 ERROR SeaTunnel:
===============================================================================
23/05/05 16:10:35 ERROR SeaTunnel: Fatal Error,
23/05/05 16:10:35 ERROR SeaTunnel: Please submit bug report in https://github.com/apache/incubator-seatunnel/issues
23/05/05 16:10:35 ERROR SeaTunnel: Reason:ErrorCode:[COMMON-01], ErrorDescription:[File operation failed, such as (read,list,write,move,copy,sync) etc...] - Can't find config file: seatunnel.streaming.conf_lyf_test_0505.template
23/05/05 16:10:35 ERROR SeaTunnel: Exception StackTrace:org.apache.seatunnel.common.exception.SeaTunnelRuntimeException: ErrorCode:[COMMON-01], ErrorDescription:[File operation failed, such as (read,list,write,move,copy,sync) etc...] - Can't find config file: seatunnel.streaming.conf_lyf_test_0505.template
at org.apache.seatunnel.core.starter.utils.FileUtils.checkConfigExist(FileUtils.java:68)
at org.apache.seatunnel.core.starter.spark.command.SparkTaskExecuteCommand.execute(SparkTaskExecuteCommand.java:50)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.spark.SeaTunnelSpark.main(SeaTunnelSpark.java:35)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.worker.DriverWrapper$.main(DriverWrapper.scala:65)
at org.apache.spark.deploy.worker.DriverWrapper.main(DriverWrapper.scala)
23/05/05 16:10:35 ERROR SeaTunnel:
```
### Flink or Spark Version
spark 2.4.8，standalone模式，且是单节点
### Java or Scala Version
java 1.8
### Screenshots
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer closed issue #4707: [Bug] [connector-v2] Can't find config file when spark standalone cluster submit
Posted by "TyrantLucifer (via GitHub)" <gi...@apache.org>.
TyrantLucifer closed issue #4707: [Bug] [connector-v2] Can't find config file when spark standalone cluster submit
URL: https://github.com/apache/incubator-seatunnel/issues/4707
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on issue #4707: [Bug] [connector-v2] Can't find config file when spark standalone cluster submit
Posted by "TyrantLucifer (via GitHub)" <gi...@apache.org>.
TyrantLucifer commented on issue #4707:
URL: https://github.com/apache/incubator-seatunnel/issues/4707#issuecomment-1535910941
Use client mode submit job，not cluster mode
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on issue #4707: [Bug] [connector-v2] Can't find config file when spark standalone cluster submit
Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 commented on issue #4707:
URL: https://github.com/apache/incubator-seatunnel/issues/4707#issuecomment-1535908002
@TyrantLucifer PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org