You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/05/24 01:35:44 UTC
[GitHub] [incubator-seatunnel] jersy opened a new issue, #1944: [Bug] [Config] seatunnel config file do not use include file do substitutions
jersy opened a new issue, #1944:
URL: https://github.com/apache/incubator-seatunnel/issues/1944
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
seatunnel config file do not use include file do substitutions
### SeaTunnel Version
2.1.1
### SeaTunnel Config
```conf
#source_sql_db.conf
ems {
driver = "com.mysql.jdbc.Driver"
url = "jdbc:mysql://localhost:3306/ems"
user = "ems"
password = "emspass"
}
#spark_application4.conf
driver="com.mysql.jdbc.Driver"
refer_driver=${driver}
include "source_sql_db.conf"
source {
jdbc {
driver = ${ems.driver}
url = ${ems.url}
table =${ems.driver}
user = ${ems.user}
password = ${ems.password}
result_table_name = "test_table"
}
}
transform {
}
sink {
}
```
### Running Command
```shell
import org.apache.seatunnel.shade.com.typesafe.config.Config;
import org.apache.seatunnel.shade.com.typesafe.config.ConfigFactory;
import org.apache.seatunnel.shade.com.typesafe.config.ConfigRenderOptions;
import org.apache.seatunnel.shade.com.typesafe.config.ConfigResolveOptions;
import java.io.File;
import java.io.FileNotFoundException;
public class TestConfig4 {
public static void main(String[] args) throws FileNotFoundException {
String configFile = "config/spark_application4.conf";
File file = new File(configFile);
if (!file.exists()) {
throw new FileNotFoundException("config file '" + file + "' does not exists!");
}
Config appConfig = ConfigFactory.parseFile(file)
.resolve(ConfigResolveOptions.defaults().setAllowUnresolved(true))
.resolveWith( ConfigFactory.systemEnvironment(), ConfigResolveOptions.defaults().setAllowUnresolved(true));
ConfigRenderOptions options = ConfigRenderOptions.concise().setFormatted(true);
System.out.println(appConfig.root().render(options));;
}
}
```
### Error Exception
```log
{
"transform" : [],
"driver" : "com.mysql.jdbc.Driver",
"sink" : [],
"ems" : {
"driver" : "com.mysql.jdbc.Driver",
"url" : "jdbc:mysql://localhost:3306/ems",
"user" : "ems",
"password" : "emspass"
},
"source" : [
{
"password" : ${ems.password},
"driver" : ${ems.driver},
"result_table_name" : "test_table",
"plugin_name" : "jdbc",
"user" : ${ems.user},
"url" : ${ems.url},
"table" : ${ems.driver}
}
],
"refer_driver" : "com.mysql.jdbc.Driver"
}
```
### Flink or Spark Version
_No response_
### Java or Scala Version
1.8
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] jersy commented on issue #1944: [Bug] [Config] seatunnel config file do not use include file do substitutions
Posted by GitBox <gi...@apache.org>.
jersy commented on issue #1944:
URL: https://github.com/apache/incubator-seatunnel/issues/1944#issuecomment-1135326073
> Yeap, SeaTunnel not support include other file. If you want, you can contribute this.
If I had 100 Mysql tables to import, it would be crazy to configure JDBC information in 100 files. I think should support this feature。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] BenJFan commented on issue #1944: [Bug] [Config] seatunnel config file do not use include file do substitutions
Posted by GitBox <gi...@apache.org>.
BenJFan commented on issue #1944:
URL: https://github.com/apache/incubator-seatunnel/issues/1944#issuecomment-1135320378
Yeap, SeaTunnel not support include other file. If you want, you can contribute this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] BenJFan commented on issue #1944: [Bug] [Config] seatunnel config file do not use include file do substitutions
Posted by GitBox <gi...@apache.org>.
BenJFan commented on issue #1944:
URL: https://github.com/apache/incubator-seatunnel/issues/1944#issuecomment-1135341549
> > Yeap, SeaTunnel not support include other file. If you want, you can contribute this.
>
> If I had 100 Mysql tables to import, it would be crazy to configure JDBC information in 100 files. I think should support this feature。
Looking forwar your contribute！
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org