You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/09/27 07:09:05 UTC
[GitHub] [incubator-seatunnel] Carl-Zhou-CN opened a new issue, #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Carl-Zhou-CN opened a new issue, #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912
### Search before asking
- [X] I had searched in the [feature](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
### Description
At present, seatunnel has a variety of engines. Each engine has its own unique parameters. Some parameters can be adapted to all engines, such as parallelism. We can find more such parameters
### Usage Scenario
Connector-V2
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
URL: https://github.com/apache/incubator-seatunnel/issues/2912
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1297974171
Very helpful issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1298132741
> At present, there are some differences between the startup methods of Flink and Spark in seatunnel. Spark pays more attention to the env parameter in config when it starts. Flink depends more on the command line parameters. The parameters that cannot be recognized by the command line are used as the startup parameters of Flink. Does this need to be unified?
In my view, we should focus on env parameter at now. Provide a unified user experience to users. The different about how to handle these parameter, if can be unified would be better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Carl-Zhou-CN commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1299626752
> > At present, there are some differences between the startup methods of Flink and Spark in seatunnel. Spark pays more attention to the env parameter in config when it starts. Flink depends more on the command line parameters. The parameters that cannot be recognized by the command line are used as the startup parameters of Flink. Does this need to be unified?
>
> In my view, we should focus on env parameter at now. Provide a unified user experience to users. The different about how to handle these parameter, if can be unified would be better.
> > > At present, there are some differences between the startup methods of Flink and Spark in seatunnel. Spark pays more attention to the env parameter in config when it starts. Flink depends more on the command line parameters. The parameters that cannot be recognized by the command line are used as the startup parameters of Flink. Does this need to be unified?
> >
> >
> > In my view, we should focus on env parameter at now. Provide a unified user experience to users. The different about how to handle these parameter, if can be unified would be better.
>
> agree
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1384694285
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1426525487
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1350157595
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
hailin0 commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1299622762
> > At present, there are some differences between the startup methods of Flink and Spark in seatunnel. Spark pays more attention to the env parameter in config when it starts. Flink depends more on the command line parameters. The parameters that cannot be recognized by the command line are used as the startup parameters of Flink. Does this need to be unified?
>
> In my view, we should focus on env parameter at now. Provide a unified user experience to users. The different about how to handle these parameter, if can be unified would be better.
agree
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1294265471
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
hailin0 commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1299628214
command line parameters reference
https://seatunnel.apache.org/docs/command/usage/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 closed issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 closed issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
URL: https://github.com/apache/incubator-seatunnel/issues/2912
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
hailin0 commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1297978694
flink example:
```
env {
# You can set flink configuration here
execution.parallelism = 1
job.mode = "BATCH"
execution.checkpoint.interval = 10000
execution.checkpoint.data-uri = "hdfs://localhost:9000/checkpoint"
}
```
spark example:
```
env {
job.mode = "BATCH"
job.name = "SeaTunnel"
spark.executor.instances = 2
spark.executor.cores = 1
spark.executor.memory = "1g"
spark.master = local
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
hailin0 commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1297974902
good idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Carl-Zhou-CN commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1298086998
At present, there are some differences between the startup methods of Flink and Spark in seatunnel. Spark pays more attention to the env parameter in config when it starts. Flink depends more on the command line parameters. The parameters that cannot be recognized by the command line are used as the startup parameters of Flink. Does this need to be unified?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Carl-Zhou-CN commented on issue #2912: [Feature][Discuss] Uniform parameters are available for multiple engines
Posted by GitBox <gi...@apache.org>.
Carl-Zhou-CN commented on issue #2912:
URL: https://github.com/apache/incubator-seatunnel/issues/2912#issuecomment-1350186191
underway
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org