You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@linkis.apache.org by GitBox <gi...@apache.org> on 2022/08/23 10:52:14 UTC
[GitHub] [incubator-linkis] chenmutime opened a new pull request, #2961: Dev 1.3.1 muli manager
chenmutime opened a new pull request, #2961:
URL: https://github.com/apache/incubator-linkis/pull/2961
<!--
Thank you for sending the PR! We appreciate you spending the time to work on these changes.
You can learn more about contributing to Apache Linkis here: https://linkis.apache.org/community/how-to-contribute
Happy contributing!
-->
### What is the purpose of the change
EngineConn-Core defines the the abstractions and interfaces of the EngineConn core functions.
The Engine Service in Linkis 0.x is refactored, EngineConn will handle the engine connection
and session management.
### Related issues/PRs
Related issues: #590
Related pr:#591
### Brief change log
- Define the core abstraction and interfaces of the EngineConn Factory;
- Define the core abstraction and interfaces of Executor Manager.
### Checklist
- [x] I have read the [Contributing Guidelines on pull requests](https://github.com/facebook/docusaurus/blob/main/CONTRIBUTING.md#pull-requests).
- [ ] I have explained the need for this PR and the problem it solves
- [ ] I have explained the changes or the new features added to this PR
- [ ] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [ ] I have verified that this change is backward compatible (If not, please discuss on the [Linkis mailing list](https://linkis.apache.org/community/how-to-subscribe) first)
- [ ] **If this is a code change**: I have written unit tests to fully verify the new behavior.
<!--
Note
1. Mark the PR title as `[WIP] title` until it's ready to be reviewed.
如果PR还未准备好被review，请在标题上添加[WIP]标识(WIP work in progress)
2. Always add/update tests for any changes unless you have a good reason.
除非您有充分的理由，否则任何修改都需要添加/更新测试
3. Always update the documentation to reflect the changes made in the PR.
始终更新文档以反映 PR 中所做的更改
4. After the PR is submitted, please pay attention to the execution result of git action check.
If there is any failure, please adjust it in time
PR提交后，请关注git action check 执行结果，关键的check失败时，请及时修正
5. Before the pr is merged, if the commit is missing, you can continue to commit the code
在未合并前，如果提交有遗漏，您可以继续提交代码
6. After you submit PR, you can add assistant WeChat, the WeChat QR code is
https://user-images.githubusercontent.com/7869972/176336986-d6b9be8f-d1d3-45f1-aa45-8e6adf5dd244.png
您提交pr后，可以添加助手微信，微信二维码为
https://user-images.githubusercontent.com/7869972/176336986-d6b9be8f-d1d3-45f1-aa45-8e6adf5dd244.png
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] casionone commented on a diff in pull request #2961: Dev 1.3.1 muli manager
Posted by GitBox <gi...@apache.org>.
casionone commented on code in PR #2961:
URL: https://github.com/apache/incubator-linkis/pull/2961#discussion_r953390253
##########
linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceEnvMapper.xml:
##########
@@ -123,4 +123,14 @@
`id` = #{id}
</where>
</update>
+
+ <select id="selectOneByName" resultMap="dataSourceEnvMap">
+ <![CDATA[SELECT ]]>
Review Comment:
需要加上dataSourceTypeId参数 单独的 envName 容易出现冲突
表linkis_ps_dm_datasource_env 需要新增unique 索引键 envName +dataSourceTypeId
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] casionone commented on a diff in pull request #2961: Dev 1.3.1 muli manager
Posted by GitBox <gi...@apache.org>.
casionone commented on code in PR #2961:
URL: https://github.com/apache/incubator-linkis/pull/2961#discussion_r953396019
##########
linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/DataSourceInfoServiceImpl.java:
##########
@@ -596,4 +676,18 @@ private void deleteResources(String userName, List<String> uploadedResources) {
}
}
}
+
+ private String mergeDataSourceVersionParameter(DataSource dataSource, Long version)
+ throws JsonErrorException {
+ Map<String, Object> connectParams = dataSource.getConnectParams();
+ if (Objects.isNull(version) || version <= 0) {
Review Comment:
这里建议只做数据合法性校验 不做默认值的赋值
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] casionone commented on a diff in pull request #2961: Dev 1.3.1 muli manager
Posted by GitBox <gi...@apache.org>.
casionone commented on code in PR #2961:
URL: https://github.com/apache/incubator-linkis/pull/2961#discussion_r953390543
##########
linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceAdminRestfulApi.java:
##########
@@ -57,6 +55,8 @@
produces = {"application/json"})
public class DataSourceAdminRestfulApi {
+ private final List<String> permitSystemList =
+ Lists.newCopyOnWriteArrayList(Arrays.asList("Qualitis"));
Review Comment:
just
` private final List<String> permitSystemList =Arrays.asList("Qualitis");`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] casionone merged pull request #2961: [Feature] datasource manager support mulit env
Posted by GitBox <gi...@apache.org>.
casionone merged PR #2961:
URL: https://github.com/apache/incubator-linkis/pull/2961
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] casionone commented on a diff in pull request #2961: Dev 1.3.1 muli manager
Posted by GitBox <gi...@apache.org>.
casionone commented on code in PR #2961:
URL: https://github.com/apache/incubator-linkis/pull/2961#discussion_r953392302
##########
linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java:
##########
@@ -216,6 +217,8 @@ public Message updateDataSourceInJson(
+ dataSourceName
+ " 已经存在]");
}
+ String parameter = Json.toJson(dataSource.getConnectParams(), null);
Review Comment:
这里加一次转换的目的是? 是否必须呢
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] casionone commented on a diff in pull request #2961: Dev 1.3.1 muli manager
Posted by GitBox <gi...@apache.org>.
casionone commented on code in PR #2961:
URL: https://github.com/apache/incubator-linkis/pull/2961#discussion_r953391271
##########
linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceAdminRestfulApi.java:
##########
@@ -89,13 +89,122 @@ public Message insertJsonEnv(@RequestBody DataSourceEnv dataSourceEnv, HttpServl
if (result.size() > 0) {
throw new ConstraintViolationException(result);
}
+ if (dataSourceInfoService.existDataSourceEnv(dataSourceEnv.getEnvName())) {
+ return Message.error(
+ "The data source env named: "
+ + dataSourceEnv.getEnvName()
+ + " has been existed [数据源环境: "
+ + dataSourceEnv.getEnvName()
+ + " 已经存在]");
+ }
dataSourceEnv.setCreateUser(userName);
insertDataSourceEnv(dataSourceEnv);
return Message.ok().data("insertId", dataSourceEnv.getId());
},
"Fail to insert data source environment[新增数据源环境失败]");
}
+ @RequestMapping(value = "/env/json/batch", method = RequestMethod.POST)
Review Comment:
补充swagger 注解
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] codecov[bot] commented on pull request #2961: [Feature] datasource manager support mulit env
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #2961:
URL: https://github.com/apache/incubator-linkis/pull/2961#issuecomment-1228536953
# [Codecov](https://codecov.io/gh/apache/incubator-linkis/pull/2961?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2961](https://codecov.io/gh/apache/incubator-linkis/pull/2961?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2d0d50c) into [dev-1.3.1](https://codecov.io/gh/apache/incubator-linkis/commit/256a175400393172e6cde831a8ccb8855eaf2f1e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (256a175) will **decrease** coverage by `0.00%`.
> The diff coverage is `34.40%`.
```diff
@@ Coverage Diff @@
## dev-1.3.1 #2961 +/- ##
===============================================
- Coverage 15.81% 15.80% -0.01%
- Complexity 1330 1332 +2
===============================================
Files 742 742
Lines 24520 24638 +118
Branches 3480 3502 +22
===============================================
+ Hits 3877 3894 +17
- Misses 20154 20248 +94
- Partials 489 496 +7
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-linkis/pull/2961?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...on/errorcode/LinkisEngineConnErrorCodeSummary.java](https://codecov.io/gh/apache/incubator-linkis/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbW1vbnMvbGlua2lzLWNvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvbGlua2lzL2NvbW1vbi9lcnJvcmNvZGUvTGlua2lzRW5naW5lQ29ubkVycm9yQ29kZVN1bW1hcnkuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...mon/errorcode/LinkisExtensionErrorCodeSummary.java](https://codecov.io/gh/apache/incubator-linkis/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbW1vbnMvbGlua2lzLWNvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvbGlua2lzL2NvbW1vbi9lcnJvcmNvZGUvTGlua2lzRXh0ZW5zaW9uRXJyb3JDb2RlU3VtbWFyeS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...rcode/LinkisPublicEnhancementErrorCodeSummary.java](https://codecov.io/gh/apache/incubator-linkis/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbW1vbnMvbGlua2lzLWNvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvbGlua2lzL2NvbW1vbi9lcnJvcmNvZGUvTGlua2lzUHVibGljRW5oYW5jZW1lbnRFcnJvckNvZGVTdW1tYXJ5LmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...code/LinkisSpringCloudServiceErrorCodeSummary.java](https://codecov.io/gh/apache/incubator-linkis/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbW1vbnMvbGlua2lzLWNvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvbGlua2lzL2NvbW1vbi9lcnJvcmNvZGUvTGlua2lzU3ByaW5nQ2xvdWRTZXJ2aWNlRXJyb3JDb2RlU3VtbWFyeS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...on/exception/VariableOperationFailedException.java](https://codecov.io/gh/apache/incubator-linkis/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbW1vbnMvbGlua2lzLWNvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvbGlua2lzL2NvbW1vbi9leGNlcHRpb24vVmFyaWFibGVPcGVyYXRpb25GYWlsZWRFeGNlcHRpb24uamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [.../main/java/org/apache/linkis/common/io/FsPath.java](https://codecov.io/gh/apache/incubator-linkis/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbW1vbnMvbGlua2lzLWNvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvbGlua2lzL2NvbW1vbi9pby9Gc1BhdGguamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [.../org/apache/linkis/common/utils/ByteTimeUtils.java](https://codecov.io/gh/apache/incubator-linkis/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbW1vbnMvbGlua2lzLWNvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvbGlua2lzL2NvbW1vbi91dGlscy9CeXRlVGltZVV0aWxzLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...n/java/org/apache/linkis/common/utils/DESUtil.java](https://codecov.io/gh/apache/incubator-linkis/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbW1vbnMvbGlua2lzLWNvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvbGlua2lzL2NvbW1vbi91dGlscy9ERVNVdGlsLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ala/org/apache/linkis/common/ServiceInstance.scala](https://codecov.io/gh/apache/incubator-linkis/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbW1vbnMvbGlua2lzLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpbmtpcy9jb21tb24vU2VydmljZUluc3RhbmNlLnNjYWxh) | `0.00% <0.00%> (ø)` | |
| [...scala/org/apache/linkis/common/conf/ByteType.scala](https://codecov.io/gh/apache/incubator-linkis/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bGlua2lzLWNvbW1vbnMvbGlua2lzLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpbmtpcy9jb21tb24vY29uZi9CeXRlVHlwZS5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| ... and [156 more](https://codecov.io/gh/apache/incubator-linkis/pull/2961/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org