You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@linkis.apache.org by pe...@apache.org on 2022/02/23 07:47:06 UTC
[incubator-linkis] branch dev-1.1.0-datasource updated (1de2c91 -> b4d45d3)
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a change to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git.
from 1de2c91 adjust code style
new e2d9fee clear Redundant code for datasource
new acace79 add datasource and metadata manager to assembly package
new 4554c8a add datasource related properties
new 2b1234e add datasource statup script
new 7217ab5 add datasource ddl and dml
new 390c49f fix fastxml can't convert
new f53e04e fix datasource web visit failure
new 4ceec6a add receiver for metadata
new 90c2549 remove webank flag
new 84286f1 mvn spotless format
new b4d45d3 fix code format
The 11 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../conf/linkis-ps-data-source-manager.properties | 38 ++-
.../conf/linkis-ps-metadatamanager.properties | 16 +-
...{linkis-ps-cs => linkis-ps-data-source-manager} | 12 +-
...nkis-common-start => linkis-ps-metadatamanager} | 32 +-
.../src/main/assembly/assembly.xml | 25 ++
db/linkis_ddl.sql | 111 ++++++-
db/linkis_dml.sql | 18 +
db/module/linkis_datasource.sql | 127 +++++++
.../datasourcemanager/common/pom.xml | 73 ----
.../datasourcemanager/common/DsmConfiguration.java | 31 --
.../datasourcemanager/common/ServiceErrorCode.java | 38 ---
.../common/domain/DataSource.java | 245 --------------
.../common/domain/DataSourceEnv.java | 189 -----------
.../domain/DataSourceParamKeyDefinition.java | 233 -------------
.../common/domain/DataSourceType.java | 78 -----
.../datasourcemanager/common/util/CryptoUtils.java | 104 ------
.../common/util/PatternInjectUtils.java | 177 ----------
.../datasourcemanager/common/util/json/Json.java | 114 -------
.../common/protocol/DsmQueryProtocol.scala | 40 ---
.../datasourcemanager/server/pom.xml | 134 --------
.../datasourcemanager/core/dao/DataSourceDao.java | 77 -----
.../core/dao/DataSourceEnvDao.java | 69 ----
.../core/dao/DataSourceParamKeyDao.java | 42 ---
.../core/dao/DataSourceTypeDao.java | 37 ---
.../core/dao/DataSourceTypeEnvDao.java | 38 ---
.../core/dao/mapper/DataSouceMapper.xml | 182 ----------
.../core/dao/mapper/DataSourceEnvMapper.xml | 101 ------
.../core/dao/mapper/DataSourceParamKeyMapper.xml | 50 ---
.../core/dao/mapper/DataSourceTypeEnvMapper.xml | 33 --
.../core/dao/mapper/DataSourceTypeMapper.xml | 36 --
.../CustomMultiPartFormDataTransformer.java | 315 ------------------
.../core/formdata/FormDataTransformerFactory.java | 38 ---
.../core/formdata/FormStreamContent.java | 86 -----
.../formdata/MultiPartFormDataTransformer.java | 43 ---
.../core/restful/DataSourceAdminRestfulApi.java | 220 ------------
.../core/restful/DataSourceCoreRestfulApi.java | 257 --------------
.../core/restful/DataSourceOperateRestfulApi.java | 118 -------
.../core/restful/RestfulApiHelper.java | 115 -------
.../exception/BeanValidationExceptionMapper.java | 41 ---
.../core/service/BmlAppService.java | 50 ---
.../core/service/DataSourceInfoService.java | 120 -------
.../core/service/DataSourceOpService.java | 21 --
.../core/service/DataSourceRelateService.java | 54 ---
.../core/service/MetadataOperateService.java | 38 ---
.../core/service/impl/BmlAppServiceImpl.java | 104 ------
.../service/impl/DataSourceInfoServiceImpl.java | 369 ---------------------
.../service/impl/DataSourceRelateServiceImpl.java | 58 ----
.../service/impl/MetadataOperateServiceImpl.java | 110 ------
.../validate/DataSourceParameterValidator.java | 82 -----
.../core/validate/ParameterValidateException.java | 32 --
.../core/validate/ParameterValidateStrategy.java | 41 ---
.../core/validate/ParameterValidator.java | 45 ---
.../strategy/RegExpParameterValidateStrategy.java | 64 ----
.../strategy/TypeParameterValidateStrategy.java | 71 ----
.../datasourcemanager/core/vo/DataSourceEnvVo.java | 51 ---
.../datasourcemanager/core/vo/DataSourceVo.java | 82 -----
.../datasourcemanager/core/vo/PageViewVo.java | 39 ---
.../server/src/main/resources/application.yml | 39 ---
.../server/src/main/resources/log4j2.xml | 39 ---
.../core/receivers/DsmReceiver.scala | 78 -----
.../common/domain/DataSource.java | 6 +-
.../common/domain/DataSourceEnv.java | 6 +-
.../domain/DataSourceParamKeyDefinition.java | 4 +-
.../common/domain/DataSourceParameter.java | 4 +-
.../common/domain/DataSourceType.java | 4 +-
.../common/domain/DatasourceVersion.java | 4 +-
.../core/restful/DataSourceCoreRestfulApi.java | 10 +-
.../server/loader/MetaClassLoaderManager.java | 2 +-
.../server/receiver/BaseMetaReceiver.scala | 68 ++++
.../linkis-metadata-manager/service/hive/pom.xml | 2 +-
.../metadatamanager/common/pom.xml | 77 -----
.../apache/linkis/metadatamanager/common/Json.java | 114 -------
.../metadatamanager/common/MdmConfiguration.java | 29 --
.../common/cache/CacheConfiguration.java | 29 --
.../metadatamanager/common/cache/CacheManager.java | 43 ---
.../common/cache/ConnCacheManager.java | 57 ----
.../common/domain/MetaColumnInfo.java | 67 ----
.../common/domain/MetaPartitionInfo.java | 96 ------
.../common/exception/MetaRuntimeException.java | 32 --
.../common/service/AbstractMetaService.java | 214 ------------
.../common/service/BaseMetadataService.java | 32 --
.../common/service/MetadataConnection.java | 58 ----
.../common/service/MetadataDbService.java | 68 ----
.../common/service/MetadataService.java | 21 --
.../common/protocol/MetadataOperateProtocol.scala | 31 --
.../common/protocol/MetadataProtocol.scala | 24 --
.../common/protocol/MetadataQueryProtocol.scala | 61 ----
.../common/receiver/BaseMetaReceiver.scala | 64 ----
.../metadatamanager/server/pom.xml | 94 ------
.../server/restful/MetadataCoreRestful.java | 134 --------
.../server/service/MetadataAppService.java | 69 ----
.../service/impl/MetadataAppServiceImpl.java | 153 ---------
.../server/src/main/resources/application.yml | 44 ---
.../server/src/main/resources/log4j2.xml | 39 ---
.../metadatamanager/service/elasticsearch/pom.xml | 81 -----
.../metadatamanager/service/ElasticConnection.java | 140 --------
.../service/ElasticMetaService.java | 87 -----
.../service/ElasticParamsMapper.java | 34 --
.../src/main/resources/application.yml | 38 ---
.../src/main/resources/linkis.properties | 22 --
.../elasticsearch/src/main/resources/log4j2.xml | 39 ---
.../service/receiver/ElasticReceiver.scala | 33 --
.../metadatamanager/service/hive/pom.xml | 161 ---------
.../metadatamanager/service/HiveConnection.java | 139 --------
.../metadatamanager/service/HiveMetaService.java | 197 -----------
.../metadatamanager/service/HiveParamsMapper.java | 33 --
.../hive/src/main/resources/application.yml | 38 ---
.../hive/src/main/resources/linkis.properties | 25 --
.../service/hive/src/main/resources/log4j2.xml | 39 ---
.../service/receiver/HiveReceiver.scala | 33 --
.../metadatamanager/service/mysql/pom.xml | 86 -----
.../metadatamanager/service/SqlConnection.java | 210 ------------
.../metadatamanager/service/SqlMetaService.java | 79 -----
.../metadatamanager/service/SqlParamsMapper.java | 38 ---
.../mysql/src/main/resources/application.yml | 38 ---
.../mysql/src/main/resources/linkis.properties | 22 --
.../service/mysql/src/main/resources/log4j2.xml | 39 ---
.../service/receiver/SqlReceiver.scala | 33 --
118 files changed, 414 insertions(+), 8350 deletions(-)
rename linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/linkis.properties => assembly-combined-package/assembly-combined/conf/linkis-ps-data-source-manager.properties (72%)
rename linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/linkis.properties => assembly-combined-package/assembly-combined/conf/linkis-ps-metadatamanager.properties (79%)
copy assembly-combined-package/assembly-combined/sbin/ext/{linkis-ps-cs => linkis-ps-data-source-manager} (81%)
copy assembly-combined-package/assembly-combined/sbin/ext/{linkis-common-start => linkis-ps-metadatamanager} (69%)
create mode 100644 db/module/linkis_datasource.sql
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/common/pom.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/DsmConfiguration.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/ServiceErrorCode.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParamKeyDefinition.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceType.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/CryptoUtils.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/PatternInjectUtils.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/json/Json.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/scala/org/apache/linkis/datasourcemanager/common/protocol/DsmQueryProtocol.scala
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/pom.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceDao.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceEnvDao.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceParamKeyDao.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceTypeDao.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceTypeEnvDao.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSouceMapper.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceEnvMapper.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceParamKeyMapper.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceTypeEnvMapper.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceTypeMapper.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/CustomMultiPartFormDataTransformer.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/FormDataTransformerFactory.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/FormStreamContent.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/MultiPartFormDataTransformer.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceAdminRestfulApi.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceOperateRestfulApi.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/RestfulApiHelper.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/exception/BeanValidationExceptionMapper.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/BmlAppService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceInfoService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceOpService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceRelateService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/MetadataOperateService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/BmlAppServiceImpl.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/DataSourceInfoServiceImpl.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/DataSourceRelateServiceImpl.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/MetadataOperateServiceImpl.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/DataSourceParameterValidator.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidateException.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidateStrategy.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidator.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/strategy/RegExpParameterValidateStrategy.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/strategy/TypeParameterValidateStrategy.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/DataSourceEnvVo.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/DataSourceVo.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/PageViewVo.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/application.yml
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/log4j2.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/scala/org/apache/linkis/datasourcemanager/core/receivers/DsmReceiver.scala
create mode 100644 linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/scala/org/apache/linkis/metadatamanager/server/receiver/BaseMetaReceiver.scala
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/pom.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/Json.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/MdmConfiguration.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/CacheConfiguration.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/CacheManager.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/ConnCacheManager.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/domain/MetaColumnInfo.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/domain/MetaPartitionInfo.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/exception/MetaRuntimeException.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/AbstractMetaService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/BaseMetadataService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataConnection.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataDbService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataOperateProtocol.scala
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataProtocol.scala
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataQueryProtocol.scala
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/receiver/BaseMetaReceiver.scala
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/server/pom.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/restful/MetadataCoreRestful.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/service/MetadataAppService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/service/impl/MetadataAppServiceImpl.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/application.yml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/log4j2.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/pom.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticConnection.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticMetaService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticParamsMapper.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/application.yml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/linkis.properties
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/log4j2.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/ElasticReceiver.scala
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/pom.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveConnection.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveMetaService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveParamsMapper.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/application.yml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/linkis.properties
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/log4j2.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/HiveReceiver.scala
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/pom.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlConnection.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlMetaService.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlParamsMapper.java
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/application.yml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/linkis.properties
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/log4j2.xml
delete mode 100644 linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/SqlReceiver.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[incubator-linkis] 09/11: remove webank flag
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git
commit 90c2549eaa0222c8e9d00aaf3ba13d2413ffc17d
Author: xiaojie19852006 <xi...@163.com>
AuthorDate: Wed Feb 23 15:11:04 2022 +0800
remove webank flag
---
.../linkis/metadatamanager/server/loader/MetaClassLoaderManager.java | 2 +-
.../linkis-datasource/linkis-metadata-manager/service/hive/pom.xml | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/java/org/apache/linkis/metadatamanager/server/loader/MetaClassLoaderManager.java b/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/java/org/apache/linkis/metadatamanager/server/loader/MetaClassLoaderManager.java
index 6fbdb1a..62aaf28 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/java/org/apache/linkis/metadatamanager/server/loader/MetaClassLoaderManager.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/java/org/apache/linkis/metadatamanager/server/loader/MetaClassLoaderManager.java
@@ -57,7 +57,7 @@ public class MetaClassLoaderManager {
CommonVars.apply("wds.linkis.server.mdm.service.instance.expire-in-seconds", 60);
private static final String META_CLASS_NAME =
- "com.webank.wedatasphere.linkis.metadatamanager.service.%sMetaService";
+ "org.apache.linkis.metadatamanager.service.%sMetaService";
private static final Logger LOG = LoggerFactory.getLogger(MetaClassLoaderManager.class);
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/service/hive/pom.xml b/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/service/hive/pom.xml
index 7a7e034..55b70c0 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/service/hive/pom.xml
+++ b/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/service/hive/pom.xml
@@ -247,7 +247,7 @@
<exclusions>
<exclusion>
<artifactId>linkis-httpclient</artifactId>
- <groupId>com.webank.wedatasphere.linkis</groupId>
+ <groupId>org.apache.linkis</groupId>
</exclusion>
</exclusions>
</dependency>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[incubator-linkis] 10/11: mvn spotless format
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git
commit 84286f1f8ec261867d0c6b2fb1efdf07dcf71002
Author: xiaojie19852006 <xi...@163.com>
AuthorDate: Wed Feb 23 15:11:31 2022 +0800
mvn spotless format
---
.../linkis/datasourcemanager/common/domain/DataSource.java | 11 +++++++----
.../linkis/datasourcemanager/common/domain/DataSourceEnv.java | 11 +++++++----
.../datasourcemanager/common/domain/DatasourceVersion.java | 8 +++++---
.../core/restful/DataSourceCoreRestfulApi.java | 9 ++++++---
4 files changed, 25 insertions(+), 14 deletions(-)
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java
index 5b140a1..7b68ccf 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java
@@ -17,16 +17,19 @@
package org.apache.linkis.datasourcemanager.common.domain;
-import com.fasterxml.jackson.annotation.JsonIgnore;
-import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
-import com.fasterxml.jackson.databind.annotation.JsonSerialize;
-import org.apache.commons.lang.StringUtils;
import org.apache.linkis.datasourcemanager.common.exception.JsonErrorException;
import org.apache.linkis.datasourcemanager.common.util.json.Json;
+import org.apache.commons.lang.StringUtils;
+
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Pattern;
import javax.validation.constraints.Size;
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+
import java.util.*;
/** Store the data source information */
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java
index 32b1639..d12dce3 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java
@@ -17,15 +17,18 @@
package org.apache.linkis.datasourcemanager.common.domain;
-import com.fasterxml.jackson.annotation.JsonIgnore;
-import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
-import com.fasterxml.jackson.databind.annotation.JsonSerialize;
-import org.apache.commons.lang.StringUtils;
import org.apache.linkis.datasourcemanager.common.exception.JsonErrorException;
import org.apache.linkis.datasourcemanager.common.util.json.Json;
+import org.apache.commons.lang.StringUtils;
+
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Size;
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+
import java.util.*;
/** Store the data source environment information */
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DatasourceVersion.java b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DatasourceVersion.java
index c009c1a..cd7a6e5 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DatasourceVersion.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DatasourceVersion.java
@@ -17,12 +17,14 @@
package org.apache.linkis.datasourcemanager.common.domain;
-import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
-import com.fasterxml.jackson.databind.annotation.JsonSerialize;
-import org.apache.commons.lang.StringUtils;
import org.apache.linkis.datasourcemanager.common.exception.JsonErrorException;
import org.apache.linkis.datasourcemanager.common.util.json.Json;
+import org.apache.commons.lang.StringUtils;
+
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+
import java.util.Date;
import java.util.HashMap;
import java.util.Map;
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java
index 7cdc008..bf1d0ab 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java
@@ -17,7 +17,6 @@
package org.apache.linkis.datasourcemanager.core.restful;
-import com.github.pagehelper.PageInfo;
import org.apache.linkis.common.exception.ErrorException;
import org.apache.linkis.datasourcemanager.common.ServiceErrorCode;
import org.apache.linkis.datasourcemanager.common.auth.AuthContext;
@@ -36,8 +35,7 @@ import org.apache.linkis.datasourcemanager.core.vo.DataSourceVo;
import org.apache.linkis.metadatamanager.common.MdmConfiguration;
import org.apache.linkis.server.Message;
import org.apache.linkis.server.security.SecurityFilter;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
@@ -47,6 +45,11 @@ import javax.validation.ConstraintViolation;
import javax.validation.ConstraintViolationException;
import javax.validation.Validator;
import javax.validation.groups.Default;
+
+import com.github.pagehelper.PageInfo;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.*;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[incubator-linkis] 08/11: add receiver for metadata
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git
commit 4ceec6aca15337ea26b17c297306d79cd45d53b2
Author: xiaojie19852006 <xi...@163.com>
AuthorDate: Wed Feb 23 15:09:51 2022 +0800
add receiver for metadata
---
.../server/receiver/BaseMetaReceiver.scala | 68 ++++++++++++++++++++++
1 file changed, 68 insertions(+)
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/scala/org/apache/linkis/metadatamanager/server/receiver/BaseMetaReceiver.scala b/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/scala/org/apache/linkis/metadatamanager/server/receiver/BaseMetaReceiver.scala
new file mode 100644
index 0000000..91b3eac
--- /dev/null
+++ b/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/scala/org/apache/linkis/metadatamanager/server/receiver/BaseMetaReceiver.scala
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.linkis.metadatamanager.server.receiver
+
+import org.apache.linkis.common.exception.WarnException
+import org.apache.linkis.common.utils.{Logging, Utils}
+import org.apache.linkis.metadatamanager.common.exception.MetaMethodInvokeException
+import org.apache.linkis.metadatamanager.common.protocol.{MetadataConnect, MetadataResponse}
+import org.apache.linkis.metadatamanager.server.service.MetadataAppService
+import org.apache.linkis.rpc.{Receiver, Sender}
+import org.apache.linkis.server.BDPJettyServerHelper
+import org.springframework.beans.factory.annotation.Autowired
+import org.springframework.stereotype.Component
+
+import scala.concurrent.duration.Duration
+
+@Component
+class BaseMetaReceiver extends Receiver with Logging{
+ @Autowired
+ private var metadataAppService:MetadataAppService = _
+
+
+ override def receive(message: Any, sender: Sender): Unit = ???
+
+ override def receiveAndReply(message: Any, sender: Sender): Any = invoke(metadataAppService, message)
+
+ override def receiveAndReply(message: Any, duration: Duration, sender: Sender): Any = invoke(metadataAppService, message)
+
+
+ def invoke(service: MetadataAppService, message: Any): Any = Utils.tryCatch{
+ val data = message match {
+ case MetadataConnect(dataSourceType, operator, params, version) =>
+ service.getConnection(dataSourceType, operator, params)
+ // MetadataConnection is not scala class
+ null
+ case _ => new Object()
+ }
+ MetadataResponse(status = true, BDPJettyServerHelper.gson.toJson(data))
+ }{
+ case e: WarnException => val errorMsg = e.getMessage
+ trace(s"Fail to invoke meta service: [$errorMsg]")
+ MetadataResponse(status = false, errorMsg)
+ case t: Exception =>
+ t match {
+ case exception: MetaMethodInvokeException =>
+ MetadataResponse(status = false, exception.getCause.getMessage)
+ case _ =>
+ error(s"Fail to invoke meta service", t)
+ MetadataResponse(status = false, t.getMessage)
+ }
+ }
+}
+
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[incubator-linkis] 11/11: fix code format
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git
commit b4d45d3343c2e189ee300702b1041b955ab03c15
Author: xiaojie19852006 <xi...@163.com>
AuthorDate: Wed Feb 23 15:28:37 2022 +0800
fix code format
---
.../linkis/metadatamanager/server/receiver/BaseMetaReceiver.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/scala/org/apache/linkis/metadatamanager/server/receiver/BaseMetaReceiver.scala b/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/scala/org/apache/linkis/metadatamanager/server/receiver/BaseMetaReceiver.scala
index 91b3eac..65ff091 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/scala/org/apache/linkis/metadatamanager/server/receiver/BaseMetaReceiver.scala
+++ b/linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/src/main/scala/org/apache/linkis/metadatamanager/server/receiver/BaseMetaReceiver.scala
@@ -35,7 +35,7 @@ class BaseMetaReceiver extends Receiver with Logging{
private var metadataAppService:MetadataAppService = _
- override def receive(message: Any, sender: Sender): Unit = ???
+ override def receive(message: Any, sender: Sender): Unit = {}
override def receiveAndReply(message: Any, sender: Sender): Any = invoke(metadataAppService, message)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[incubator-linkis] 04/11: add datasource statup script
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git
commit 2b1234ed6eda4523dc288455602b430c71fa2f4c
Author: xiaojie19852006 <xi...@163.com>
AuthorDate: Wed Feb 23 15:04:19 2022 +0800
add datasource statup script
---
.../sbin/ext/linkis-ps-data-source-manager | 35 ++++++++++
.../sbin/ext/linkis-ps-metadatamanager | 75 ++++++++++++++++++++++
2 files changed, 110 insertions(+)
diff --git a/assembly-combined-package/assembly-combined/sbin/ext/linkis-ps-data-source-manager b/assembly-combined-package/assembly-combined/sbin/ext/linkis-ps-data-source-manager
new file mode 100644
index 0000000..2b95cfa
--- /dev/null
+++ b/assembly-combined-package/assembly-combined/sbin/ext/linkis-ps-data-source-manager
@@ -0,0 +1,35 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+# http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# description: cs start cmd
+#
+# Modified for Linkis 1.1.0
+#
+
+
+export SERVER_SUFFIX="linkis-public-enhancements/linkis-ps-data-source-manager"
+
+
+export DEBUG_PORT=10019
+
+export SERVER_CLASS=org.apache.linkis.datasourcemanager.core.LinkisDataSourceManagerApplication
+
+export COMMON_START_BIN=$LINKIS_HOME/sbin/ext/linkis-common-start
+if [[ ! -f "${COMMON_START_BIN}" ]]; then
+ echo "The $COMMON_START_BIN does not exist!"
+ exit 1
+else
+ sh $COMMON_START_BIN
+fi
diff --git a/assembly-combined-package/assembly-combined/sbin/ext/linkis-ps-metadatamanager b/assembly-combined-package/assembly-combined/sbin/ext/linkis-ps-metadatamanager
new file mode 100644
index 0000000..c6a666f
--- /dev/null
+++ b/assembly-combined-package/assembly-combined/sbin/ext/linkis-ps-metadatamanager
@@ -0,0 +1,75 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+# http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# description: cs start cmd
+#
+# Modified for Linkis 1.1.0
+#
+
+source $LINKIS_CONF_DIR/linkis-env.sh
+
+SERVER_SUFFIX="linkis-public-enhancements/linkis-ps-metadatamanager"
+
+
+export SERVER_CLASS=org.apache.linkis.metadatamanager.server.LinkisMetadataManagerApplication
+export LINKIS_LOG_DIR=$LINKIS_HOME/logs
+if test -z "$SERVER_HEAP_SIZE"
+then
+ export SERVER_HEAP_SIZE="512M"
+fi
+
+if [ "$DEBUG_PORT" ];
+then
+ export DEBUG_CMD="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=$DEBUG_PORT"
+fi
+
+if test -z "$SERVER_JAVA_OPTS"
+then
+ export SERVER_JAVA_OPTS=" -DserviceName=$SERVER_NAME -Xmx$SERVER_HEAP_SIZE -XX:+UseG1GC -Xloggc:$LINKIS_LOG_DIR/${SERVER_NAME}-gc.log $DEBUG_CMD "
+fi
+
+## conf dir
+export SERVER_CONF_PATH=$LINKIS_CONF_DIR
+
+## commons lib
+export LINKIS_COMMONS_LIB=$LINKIS_HOME/$LINKIS_PUBLIC_MODULE
+if [ ! -r "$LINKIS_COMMONS_LIB" ] ; then
+ echo "linkis commons lib not exists $LINKIS_COMMONS_LIB"
+ exit 1
+fi
+
+## server lib
+export SERVER_LIB=$LINKIS_HOME/lib/$SERVER_SUFFIX
+if [ ! -r "$SERVER_LIB" ] ; then
+ echo "server lib not exists $SERVER_LIB"
+ exit 1
+fi
+# set the wds.linkis.server.mdm.service.lib.dir to ${SERVER_LIB}/service
+SERVER_JAVA_OPTS=" -Dwds.linkis.server.mdm.service.lib.dir=${SERVER_LIB}/service"${SERVER_JAVA_OPTS}
+
+## set class path
+export SERVER_CLASS_PATH=$SERVER_CONF_PATH:$LINKIS_COMMONS_LIB/*:$SERVER_LIB/*
+
+nohup java $SERVER_JAVA_OPTS -cp $SERVER_CLASS_PATH $SERVER_CLASS $SPRING_ARGS 2>&1 > $LINKIS_LOG_DIR/${SERVER_NAME}.out &
+
+pid=$!
+sleep 2
+if [[ -z "${pid}" ]]; then
+ echo "server $SERVER_NAME start failed!"
+ exit 1
+else
+ echo "server $SERVER_NAME start succeeded!"
+ echo $pid > $SERVER_PID
+fi
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[incubator-linkis] 06/11: fix fastxml can't convert
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git
commit 390c49f227391ede50e22b29344dcb040c378bcf
Author: xiaojie19852006 <xi...@163.com>
AuthorDate: Wed Feb 23 15:06:47 2022 +0800
fix fastxml can't convert
---
.../linkis/datasourcemanager/common/domain/DataSource.java | 11 ++++-------
.../linkis/datasourcemanager/common/domain/DataSourceEnv.java | 11 ++++-------
.../common/domain/DataSourceParamKeyDefinition.java | 4 ++--
.../datasourcemanager/common/domain/DataSourceParameter.java | 4 ++--
.../datasourcemanager/common/domain/DataSourceType.java | 4 ++--
.../datasourcemanager/common/domain/DatasourceVersion.java | 8 +++-----
6 files changed, 17 insertions(+), 25 deletions(-)
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java
index b2a5b2f..5b140a1 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java
@@ -17,19 +17,16 @@
package org.apache.linkis.datasourcemanager.common.domain;
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+import org.apache.commons.lang.StringUtils;
import org.apache.linkis.datasourcemanager.common.exception.JsonErrorException;
import org.apache.linkis.datasourcemanager.common.util.json.Json;
-import org.apache.commons.lang.StringUtils;
-
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Pattern;
import javax.validation.constraints.Size;
-
-import org.codehaus.jackson.annotate.JsonIgnore;
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-
import java.util.*;
/** Store the data source information */
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java
index 8645300..32b1639 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java
@@ -17,18 +17,15 @@
package org.apache.linkis.datasourcemanager.common.domain;
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+import org.apache.commons.lang.StringUtils;
import org.apache.linkis.datasourcemanager.common.exception.JsonErrorException;
import org.apache.linkis.datasourcemanager.common.util.json.Json;
-import org.apache.commons.lang.StringUtils;
-
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Size;
-
-import org.codehaus.jackson.annotate.JsonIgnore;
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-
import java.util.*;
/** Store the data source environment information */
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParamKeyDefinition.java b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParamKeyDefinition.java
index d8fe841..21573f9 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParamKeyDefinition.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParamKeyDefinition.java
@@ -17,8 +17,8 @@
package org.apache.linkis.datasourcemanager.common.domain;
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import java.io.InputStream;
import java.util.Date;
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParameter.java b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParameter.java
index 9e40bc3..6c7271a 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParameter.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParameter.java
@@ -17,8 +17,8 @@
package org.apache.linkis.datasourcemanager.common.domain;
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import java.util.HashMap;
import java.util.Map;
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceType.java b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceType.java
index 07920cf..e4280b6 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceType.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceType.java
@@ -17,8 +17,8 @@
package org.apache.linkis.datasourcemanager.common.domain;
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
/** Data source type entity */
@JsonSerialize(include = JsonSerialize.Inclusion.NON_EMPTY)
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DatasourceVersion.java b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DatasourceVersion.java
index cbaeb3b..c009c1a 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DatasourceVersion.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DatasourceVersion.java
@@ -17,14 +17,12 @@
package org.apache.linkis.datasourcemanager.common.domain;
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+import org.apache.commons.lang.StringUtils;
import org.apache.linkis.datasourcemanager.common.exception.JsonErrorException;
import org.apache.linkis.datasourcemanager.common.util.json.Json;
-import org.apache.commons.lang.StringUtils;
-
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-
import java.util.Date;
import java.util.HashMap;
import java.util.Map;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[incubator-linkis] 05/11: add datasource ddl and dml
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git
commit 7217ab5a9f97103e332d7e426f58d8a8ee03dc70
Author: xiaojie19852006 <xi...@163.com>
AuthorDate: Wed Feb 23 15:05:58 2022 +0800
add datasource ddl and dml
---
db/linkis_ddl.sql | 111 ++++++++++++++++++++++++++++++++++-
db/linkis_dml.sql | 18 ++++++
db/module/linkis_datasource.sql | 127 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 255 insertions(+), 1 deletion(-)
diff --git a/db/linkis_ddl.sql b/db/linkis_ddl.sql
index bac8527..9f4d8f3 100644
--- a/db/linkis_ddl.sql
+++ b/db/linkis_ddl.sql
@@ -777,4 +777,113 @@ CREATE TABLE `linkis_cg_engine_conn_plugin_bml_resources` (
`create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'created time',
`last_update_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'updated time',
PRIMARY KEY (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4;
\ No newline at end of file
+) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4;
+
+-- ----------------------------
+-- Table structure for linkis_datasource
+-- ----------------------------
+DROP TABLE IF EXISTS `linkis_datasource`;
+CREATE TABLE `linkis_datasource`
+(
+ `id` int(11) NOT NULL AUTO_INCREMENT,
+ `datasource_name` varchar(255) COLLATE utf8_bin NOT NULL,
+ `datasource_desc` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `datasource_type_id` int(11) NOT NULL,
+ `create_identify` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `create_system` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `parameter` varchar(255) COLLATE utf8_bin NULL DEFAULT NULL,
+ `create_time` datetime NULL DEFAULT CURRENT_TIMESTAMP,
+ `modify_time` datetime NULL DEFAULT CURRENT_TIMESTAMP,
+ `create_user` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `modify_user` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `labels` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `version_id` int(11) DEFAULT NULL COMMENT 'current version id',
+ `expire` tinyint(1) DEFAULT 0,
+ `published_version_id` int(11) DEFAULT NULL,
+ PRIMARY KEY (`id`)
+) ENGINE = InnoDB
+ CHARACTER SET = utf8
+ COLLATE = utf8_bin;
+
+-- ----------------------------
+-- Table structure for linkis_datasource_env
+-- ----------------------------
+DROP TABLE IF EXISTS `linkis_datasource_env`;
+CREATE TABLE `linkis_datasource_env`
+(
+ `id` int(11) NOT NULL AUTO_INCREMENT,
+ `env_name` varchar(32) COLLATE utf8_bin NOT NULL,
+ `env_desc` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `datasource_type_id` int(11) NOT NULL,
+ `parameter` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ `create_user` varchar(255) COLLATE utf8_bin NULL DEFAULT NULL,
+ `modify_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ `modify_user` varchar(255) COLLATE utf8_bin NULL DEFAULT NULL,
+ PRIMARY KEY (`id`)
+) ENGINE = InnoDB
+ CHARACTER SET = utf8
+ COLLATE = utf8_bin;
+
+
+-- ----------------------------
+-- Table structure for linkis_datasource_type
+-- ----------------------------
+DROP TABLE IF EXISTS `linkis_datasource_type`;
+CREATE TABLE `linkis_datasource_type`
+(
+ `id` int(11) NOT NULL AUTO_INCREMENT,
+ `name` varchar(32) COLLATE utf8_bin NOT NULL,
+ `description` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `option` varchar(32) COLLATE utf8_bin DEFAULT NULL,
+ `classifier` varchar(32) COLLATE utf8_bin NOT NULL,
+ `icon` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `layers` int(3) NOT NULL,
+ PRIMARY KEY (`id`)
+) ENGINE = InnoDB
+ CHARACTER SET = utf8
+ COLLATE = utf8_bin;
+
+-- ----------------------------
+-- Table structure for linkis_datasource_type_key
+-- ----------------------------
+DROP TABLE IF EXISTS `linkis_datasource_type_key`;
+CREATE TABLE `linkis_datasource_type_key`
+(
+ `id` int(11) NOT NULL AUTO_INCREMENT,
+ `data_source_type_id` int(11) NOT NULL,
+ `key` varchar(32) COLLATE utf8_bin NOT NULL,
+ `name` varchar(32) COLLATE utf8_bin NOT NULL,
+ `default_value` varchar(50) COLLATE utf8_bin NULL DEFAULT NULL,
+ `value_type` varchar(50) COLLATE utf8_bin NOT NULL,
+ `scope` varchar(50) COLLATE utf8_bin NULL DEFAULT NULL,
+ `require` tinyint(1) NULL DEFAULT 0,
+ `description` varchar(200) COLLATE utf8_bin NULL DEFAULT NULL,
+ `value_regex` varchar(200) COLLATE utf8_bin NULL DEFAULT NULL,
+ `ref_id` bigint(20) NULL DEFAULT NULL,
+ `ref_value` varchar(50) COLLATE utf8_bin NULL DEFAULT NULL,
+ `data_source` varchar(200) COLLATE utf8_bin NULL DEFAULT NULL,
+ `update_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ `create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ PRIMARY KEY (`id`)
+) ENGINE = InnoDB
+ CHARACTER SET = utf8
+ COLLATE = utf8_bin;
+
+-- ----------------------------
+-- Table structure for linkis_datasource_version
+-- ----------------------------
+DROP TABLE IF EXISTS `linkis_datasource_version`;
+CREATE TABLE `linkis_datasource_version`
+(
+ `version_id` int(11) NOT NULL AUTO_INCREMENT,
+ `datasource_id` int(11) NOT NULL,
+ `parameter` varchar(2048) COLLATE utf8_bin NULL DEFAULT NULL,
+ `comment` varchar(255) COLLATE utf8_bin NULL DEFAULT NULL,
+ `create_time` datetime(0) NULL DEFAULT CURRENT_TIMESTAMP,
+ `create_user` varchar(255) COLLATE utf8_bin NULL DEFAULT NULL,
+ PRIMARY KEY (`version_id`, `datasource_id`) USING BTREE
+) ENGINE = InnoDB
+ AUTO_INCREMENT = 21
+ CHARACTER SET = utf8
+ COLLATE = utf8_bin;
\ No newline at end of file
diff --git a/db/linkis_dml.sql b/db/linkis_dml.sql
index a4024a2..780534c 100644
--- a/db/linkis_dml.sql
+++ b/db/linkis_dml.sql
@@ -317,3 +317,21 @@ INSERT INTO linkis_ps_error_code (error_code,error_desc,error_regex,error_type)
INSERT INTO linkis_ps_error_code (error_code,error_desc,error_regex,error_type) VALUES ('95005','未配置代理用户，请在ITSM走WTSS用户变更单，为你的用户授权改代理用户','请联系系统管理员为您的用户添加该代理用户',0);
INSERT INTO linkis_ps_error_code (error_code,error_desc,error_regex,error_type) VALUES ('60079','所查库表无权限','Authorization failed:No privilege',0);
INSERT INTO linkis_ps_error_code (error_code,error_desc,error_regex,error_type) VALUES ('30002','启动引擎超时，您可以进行任务重试','wait for DefaultEngineConn',0);
+INSERT INTO `linkis_datasource_type` VALUES
+(1,'mysql','mysql数据库','mysql数据库','关系型数据库','',3),
+(2,'kafka','kafka','kafka','消息队列','',2),
+(3,'presto','presto SQL','presto','大数据存储','',3),
+(4,'hive','hive数据库','hive','大数据存储','',3),
+(6,'mongodb','default','default','DEFAULT',NULL,3);
+
+INSERT INTO `linkis_datasource_type_key` VALUES
+(1,1,'host','Host',NULL,'TEXT',NULL,1,'mysql Host ',NULL,NULL,NULL,NULL,'2021-04-08 03:13:36','2021-04-08 03:13:36'),
+(2,1,'port','端口',NULL,'TEXT',NULL,1,'端口',NULL,NULL,NULL,NULL,'2021-04-17 03:10:28','2021-04-17 03:10:28'),
+(3,1,'params','连接参数',NULL,'TEXT',NULL,0,'输入JSON格式: {\"param\":\"value\"}',NULL,NULL,NULL,NULL,'2021-12-06 10:35:00','2021-12-06 10:35:00'),
+(9,1,'username','用户名',NULL,'TEXT',NULL,1,'用户名','^[0-9A-Za-z_-]+$',NULL,NULL,NULL,'2021-04-12 01:54:39','2021-04-12 01:54:39'),
+(10,1,'password','密码',NULL,'PASSWORD',NULL,1,'密码','',NULL,NULL,NULL,'2021-04-12 01:54:39','2021-04-12 01:54:39'),
+(11,4,'envId','集群环境',NULL,'SELECT',NULL,1,'集群环境',NULL,NULL,NULL,'/api/rest_j/v1/data-source-manager/env_list/all/type/4','2021-05-12 03:03:34','2021-05-12 03:03:34');
+
+INSERT INTO `linkis_datasource_env` VALUES
+(1,'测试环境SIT','测试环境SIT',4,'{\"uris\":\"thrift://localhost:9083\", \"hadoopConf\":{\"hive.metastore.execute.setugi\":\"true\"}}','2021-10-26 14:57:29',NULL,'2021-10-26 14:57:29',NULL),
+(2,'测试环境UAT','测试环境UAT',4,'{\"uris\":\"thrift://localhost:9083\", \"hadoopConf\":{\"hive.metastore.execute.setugi\":\"true\"}}','2022-01-04 17:24:20',NULL,'2022-01-04 17:24:20',NULL);
diff --git a/db/module/linkis_datasource.sql b/db/module/linkis_datasource.sql
new file mode 100644
index 0000000..ae53a6e
--- /dev/null
+++ b/db/module/linkis_datasource.sql
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+SET FOREIGN_KEY_CHECKS = 0;
+
+-- ----------------------------
+-- Table structure for linkis_datasource
+-- ----------------------------
+DROP TABLE IF EXISTS `linkis_datasource`;
+CREATE TABLE `linkis_datasource`
+(
+ `id` int(11) NOT NULL AUTO_INCREMENT,
+ `datasource_name` varchar(255) COLLATE utf8_bin NOT NULL,
+ `datasource_desc` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `datasource_type_id` int(11) NOT NULL,
+ `create_identify` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `create_system` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `parameter` varchar(255) COLLATE utf8_bin NULL DEFAULT NULL,
+ `create_time` datetime NULL DEFAULT CURRENT_TIMESTAMP,
+ `modify_time` datetime NULL DEFAULT CURRENT_TIMESTAMP,
+ `create_user` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `modify_user` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `labels` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `version_id` int(11) DEFAULT NULL COMMENT 'current version id',
+ `expire` tinyint(1) DEFAULT 0,
+ `published_version_id` int(11) DEFAULT NULL,
+ PRIMARY KEY (`id`)
+) ENGINE = InnoDB
+ CHARACTER SET = utf8
+ COLLATE = utf8_bin;
+
+-- ----------------------------
+-- Table structure for linkis_datasource_env
+-- ----------------------------
+DROP TABLE IF EXISTS `linkis_datasource_env`;
+CREATE TABLE `linkis_datasource_env`
+(
+ `id` int(11) NOT NULL AUTO_INCREMENT,
+ `env_name` varchar(32) COLLATE utf8_bin NOT NULL,
+ `env_desc` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `datasource_type_id` int(11) NOT NULL,
+ `parameter` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ `create_user` varchar(255) COLLATE utf8_bin NULL DEFAULT NULL,
+ `modify_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ `modify_user` varchar(255) COLLATE utf8_bin NULL DEFAULT NULL,
+ PRIMARY KEY (`id`)
+) ENGINE = InnoDB
+ CHARACTER SET = utf8
+ COLLATE = utf8_bin;
+
+
+-- ----------------------------
+-- Table structure for linkis_datasource_type
+-- ----------------------------
+DROP TABLE IF EXISTS `linkis_datasource_type`;
+CREATE TABLE `linkis_datasource_type`
+(
+ `id` int(11) NOT NULL AUTO_INCREMENT,
+ `name` varchar(32) COLLATE utf8_bin NOT NULL,
+ `description` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `option` varchar(32) COLLATE utf8_bin DEFAULT NULL,
+ `classifier` varchar(32) COLLATE utf8_bin NOT NULL,
+ `icon` varchar(255) COLLATE utf8_bin DEFAULT NULL,
+ `layers` int(3) NOT NULL,
+ PRIMARY KEY (`id`)
+) ENGINE = InnoDB
+ CHARACTER SET = utf8
+ COLLATE = utf8_bin;
+
+-- ----------------------------
+-- Table structure for linkis_datasource_type_key
+-- ----------------------------
+DROP TABLE IF EXISTS `linkis_datasource_type_key`;
+CREATE TABLE `linkis_datasource_type_key`
+(
+ `id` int(11) NOT NULL AUTO_INCREMENT,
+ `data_source_type_id` int(11) NOT NULL,
+ `key` varchar(32) COLLATE utf8_bin NOT NULL,
+ `name` varchar(32) COLLATE utf8_bin NOT NULL,
+ `default_value` varchar(50) COLLATE utf8_bin NULL DEFAULT NULL,
+ `value_type` varchar(50) COLLATE utf8_bin NOT NULL,
+ `scope` varchar(50) COLLATE utf8_bin NULL DEFAULT NULL,
+ `require` tinyint(1) NULL DEFAULT 0,
+ `description` varchar(200) COLLATE utf8_bin NULL DEFAULT NULL,
+ `value_regex` varchar(200) COLLATE utf8_bin NULL DEFAULT NULL,
+ `ref_id` bigint(20) NULL DEFAULT NULL,
+ `ref_value` varchar(50) COLLATE utf8_bin NULL DEFAULT NULL,
+ `data_source` varchar(200) COLLATE utf8_bin NULL DEFAULT NULL,
+ `update_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ `create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ PRIMARY KEY (`id`)
+) ENGINE = InnoDB
+ CHARACTER SET = utf8
+ COLLATE = utf8_bin;
+
+-- ----------------------------
+-- Table structure for linkis_datasource_version
+-- ----------------------------
+DROP TABLE IF EXISTS `linkis_datasource_version`;
+CREATE TABLE `linkis_datasource_version`
+(
+ `version_id` int(11) NOT NULL AUTO_INCREMENT,
+ `datasource_id` int(11) NOT NULL,
+ `parameter` varchar(2048) COLLATE utf8_bin NULL DEFAULT NULL,
+ `comment` varchar(255) COLLATE utf8_bin NULL DEFAULT NULL,
+ `create_time` datetime(0) NULL DEFAULT CURRENT_TIMESTAMP,
+ `create_user` varchar(255) COLLATE utf8_bin NULL DEFAULT NULL,
+ PRIMARY KEY (`version_id`, `datasource_id`) USING BTREE
+) ENGINE = InnoDB
+ AUTO_INCREMENT = 21
+ CHARACTER SET = utf8
+ COLLATE = utf8_bin;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[incubator-linkis] 01/11: clear Redundant code for datasource
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git
commit e2d9fee1cf25343b975ff5b1b9101e94bc2354bc
Author: xiaojie19852006 <xi...@163.com>
AuthorDate: Wed Feb 23 14:55:59 2022 +0800
clear Redundant code for datasource
---
.../datasourcemanager/common/pom.xml | 73 ----
.../datasourcemanager/common/DsmConfiguration.java | 31 --
.../datasourcemanager/common/ServiceErrorCode.java | 38 ---
.../common/domain/DataSource.java | 245 --------------
.../common/domain/DataSourceEnv.java | 189 -----------
.../domain/DataSourceParamKeyDefinition.java | 233 -------------
.../common/domain/DataSourceType.java | 78 -----
.../datasourcemanager/common/util/CryptoUtils.java | 104 ------
.../common/util/PatternInjectUtils.java | 177 ----------
.../datasourcemanager/common/util/json/Json.java | 114 -------
.../common/protocol/DsmQueryProtocol.scala | 40 ---
.../datasourcemanager/server/pom.xml | 134 --------
.../datasourcemanager/core/dao/DataSourceDao.java | 77 -----
.../core/dao/DataSourceEnvDao.java | 69 ----
.../core/dao/DataSourceParamKeyDao.java | 42 ---
.../core/dao/DataSourceTypeDao.java | 37 ---
.../core/dao/DataSourceTypeEnvDao.java | 38 ---
.../core/dao/mapper/DataSouceMapper.xml | 182 ----------
.../core/dao/mapper/DataSourceEnvMapper.xml | 101 ------
.../core/dao/mapper/DataSourceParamKeyMapper.xml | 50 ---
.../core/dao/mapper/DataSourceTypeEnvMapper.xml | 33 --
.../core/dao/mapper/DataSourceTypeMapper.xml | 36 --
.../CustomMultiPartFormDataTransformer.java | 315 ------------------
.../core/formdata/FormDataTransformerFactory.java | 38 ---
.../core/formdata/FormStreamContent.java | 86 -----
.../formdata/MultiPartFormDataTransformer.java | 43 ---
.../core/restful/DataSourceAdminRestfulApi.java | 220 ------------
.../core/restful/DataSourceCoreRestfulApi.java | 257 --------------
.../core/restful/DataSourceOperateRestfulApi.java | 118 -------
.../core/restful/RestfulApiHelper.java | 115 -------
.../exception/BeanValidationExceptionMapper.java | 41 ---
.../core/service/BmlAppService.java | 50 ---
.../core/service/DataSourceInfoService.java | 120 -------
.../core/service/DataSourceOpService.java | 21 --
.../core/service/DataSourceRelateService.java | 54 ---
.../core/service/MetadataOperateService.java | 38 ---
.../core/service/impl/BmlAppServiceImpl.java | 104 ------
.../service/impl/DataSourceInfoServiceImpl.java | 369 ---------------------
.../service/impl/DataSourceRelateServiceImpl.java | 58 ----
.../service/impl/MetadataOperateServiceImpl.java | 110 ------
.../validate/DataSourceParameterValidator.java | 82 -----
.../core/validate/ParameterValidateException.java | 32 --
.../core/validate/ParameterValidateStrategy.java | 41 ---
.../core/validate/ParameterValidator.java | 45 ---
.../strategy/RegExpParameterValidateStrategy.java | 64 ----
.../strategy/TypeParameterValidateStrategy.java | 71 ----
.../datasourcemanager/core/vo/DataSourceEnvVo.java | 51 ---
.../datasourcemanager/core/vo/DataSourceVo.java | 82 -----
.../datasourcemanager/core/vo/PageViewVo.java | 39 ---
.../server/src/main/resources/application.yml | 39 ---
.../server/src/main/resources/linkis.properties | 33 --
.../server/src/main/resources/log4j2.xml | 39 ---
.../core/receivers/DsmReceiver.scala | 78 -----
.../metadatamanager/common/pom.xml | 77 -----
.../apache/linkis/metadatamanager/common/Json.java | 114 -------
.../metadatamanager/common/MdmConfiguration.java | 29 --
.../common/cache/CacheConfiguration.java | 29 --
.../metadatamanager/common/cache/CacheManager.java | 43 ---
.../common/cache/ConnCacheManager.java | 57 ----
.../common/domain/MetaColumnInfo.java | 67 ----
.../common/domain/MetaPartitionInfo.java | 96 ------
.../common/exception/MetaRuntimeException.java | 32 --
.../common/service/AbstractMetaService.java | 214 ------------
.../common/service/BaseMetadataService.java | 32 --
.../common/service/MetadataConnection.java | 58 ----
.../common/service/MetadataDbService.java | 68 ----
.../common/service/MetadataService.java | 21 --
.../common/protocol/MetadataOperateProtocol.scala | 31 --
.../common/protocol/MetadataProtocol.scala | 24 --
.../common/protocol/MetadataQueryProtocol.scala | 61 ----
.../common/receiver/BaseMetaReceiver.scala | 64 ----
.../metadatamanager/server/pom.xml | 94 ------
.../server/restful/MetadataCoreRestful.java | 134 --------
.../server/service/MetadataAppService.java | 69 ----
.../service/impl/MetadataAppServiceImpl.java | 153 ---------
.../server/src/main/resources/application.yml | 44 ---
.../server/src/main/resources/linkis.properties | 25 --
.../server/src/main/resources/log4j2.xml | 39 ---
.../metadatamanager/service/elasticsearch/pom.xml | 81 -----
.../metadatamanager/service/ElasticConnection.java | 140 --------
.../service/ElasticMetaService.java | 87 -----
.../service/ElasticParamsMapper.java | 34 --
.../src/main/resources/application.yml | 38 ---
.../src/main/resources/linkis.properties | 22 --
.../elasticsearch/src/main/resources/log4j2.xml | 39 ---
.../service/receiver/ElasticReceiver.scala | 33 --
.../metadatamanager/service/hive/pom.xml | 161 ---------
.../metadatamanager/service/HiveConnection.java | 139 --------
.../metadatamanager/service/HiveMetaService.java | 197 -----------
.../metadatamanager/service/HiveParamsMapper.java | 33 --
.../hive/src/main/resources/application.yml | 38 ---
.../hive/src/main/resources/linkis.properties | 25 --
.../service/hive/src/main/resources/log4j2.xml | 39 ---
.../service/receiver/HiveReceiver.scala | 33 --
.../metadatamanager/service/mysql/pom.xml | 86 -----
.../metadatamanager/service/SqlConnection.java | 210 ------------
.../metadatamanager/service/SqlMetaService.java | 79 -----
.../metadatamanager/service/SqlParamsMapper.java | 38 ---
.../mysql/src/main/resources/application.yml | 38 ---
.../mysql/src/main/resources/linkis.properties | 22 --
.../service/mysql/src/main/resources/log4j2.xml | 39 ---
.../service/receiver/SqlReceiver.scala | 33 --
102 files changed, 8333 deletions(-)
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/pom.xml b/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/pom.xml
deleted file mode 100644
index de14fdc..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/pom.xml
+++ /dev/null
@@ -1,73 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
- <parent>
- <artifactId>linkis</artifactId>
- <groupId>org.apache.linkis</groupId>
- <version>1.0.3</version>
- </parent>
- <modelVersion>4.0.0</modelVersion>
-
- <artifactId>linkis-datasourcemanager-common</artifactId>
-
-
- <properties>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
- <um.version>1.0.12</um.version>
- </properties>
-
- <dependencies>
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-module</artifactId>
- <version>${linkis.version}</version>
- <exclusions>
- <exclusion>
- <artifactId>asm</artifactId>
- <groupId>org.ow2.asm</groupId>
- </exclusion>
- </exclusions>
- </dependency>
- </dependencies>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-deploy-plugin</artifactId>
- </plugin>
-
- <plugin>
- <groupId>net.alchim31.maven</groupId>
- <artifactId>scala-maven-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- </plugin>
- </plugins>
- <resources>
- <resource>
- <directory>src/main/resources</directory>
- </resource>
- </resources>
- <finalName>${project.artifactId}-${project.version}</finalName>
- </build>
-</project>
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/DsmConfiguration.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/DsmConfiguration.java
deleted file mode 100644
index 547cc6b..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/DsmConfiguration.java
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.common;
-
-import org.apache.linkis.common.conf.CommonVars;
-
-/**
- * Configuration
- */
-public class DsmConfiguration {
-
- public static CommonVars<String> DSM_ADMIN_USER_LIST =
- CommonVars.apply("wds.linkis.server.dsm.admin.users", "");
-
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/ServiceErrorCode.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/ServiceErrorCode.java
deleted file mode 100644
index 0418c3b..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/ServiceErrorCode.java
+++ /dev/null
@@ -1,38 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.common;
-
-import org.apache.linkis.common.conf.CommonVars;
-
-/**
- * Store error code map
- */
-public class ServiceErrorCode {
-
- public static CommonVars<Integer> TRANSFORM_FORM_ERROR =
- CommonVars.apply("wds.linkis.server.dsm.error-code.transform", 99987);
-
- public static CommonVars<Integer> BML_SERVICE_ERROR =
- CommonVars.apply("wds.linkis.server.dsm.error-code.bml", 99982);
-
- public static CommonVars<Integer> REMOTE_METADATA_SERVICE_ERROR =
- CommonVars.apply("wds.linkis.server.dsm.error-code.metadata", 99983);
-
- public static CommonVars<Integer> PARAM_VALIDATE_ERROR =
- CommonVars.apply("wds.linkis.server.dsm.error-code.param-validate", 99986);
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java
deleted file mode 100644
index 900bbe4..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSource.java
+++ /dev/null
@@ -1,245 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.common.domain;
-
-import org.apache.linkis.datasourcemanager.common.util.json.Json;
-import org.apache.commons.lang.StringUtils;
-import org.codehaus.jackson.annotate.JsonIgnore;
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-
-import javax.validation.constraints.NotNull;
-import javax.validation.constraints.Size;
-import java.util.*;
-
-/**
- * Store the data source information
- */
-@JsonSerialize(include= JsonSerialize.Inclusion.NON_EMPTY)
-@JsonIgnoreProperties(value = {"hibernateLazyInitializer", "handler"}, ignoreUnknown = true)
-public class DataSource {
-
- private Long id;
-
- /**
- * Data source name
- */
- @NotNull
- private String dataSourceName;
-
- /**
- * Data source description
- */
- @Size(min = 0, max = 200)
- private String dataSourceDesc;
-
- /**
- * ID of data source type
- */
- @NotNull
- private Long dataSourceTypeId;
-
- /**
- * Identify from creator
- */
- private String createIdentify;
-
- /**
- * System name from creator
- */
- @NotNull
- private String createSystem;
- /**
- * Connection parameters
- */
- private Map<String, Object> connectParams = new HashMap<>();
- /**
- * Parameter JSON string
- */
- @JsonIgnore
- private String parameter;
-
- /**
- * ID of data source environment
- */
- private Long dataSourceEnvId;
-
- /**
- * Create time
- */
- private Date createTime;
-
- /**
- * Modify time
- */
- private Date modifyTime;
-
- /**
- * Modify user
- */
- private String modifyUser;
-
- private String createUser;
-
- /**
- * Data source type entity
- */
- private DataSourceType dataSourceType;
-
- /**
- * Data source env
- */
- private DataSourceEnv dataSourceEnv;
-
- @JsonIgnore
- private List<DataSourceParamKeyDefinition> keyDefinitions = new ArrayList<>();
-
- public DataSource(){
- this.createTime = Calendar.getInstance().getTime();
- }
- public Long getId() {
- return id;
- }
-
- public void setId(Long id) {
- this.id = id;
- }
-
- public String getDataSourceName() {
- return dataSourceName;
- }
-
- public void setDataSourceName(String dataSourceName) {
- this.dataSourceName = dataSourceName;
- }
-
- public String getDataSourceDesc() {
- return dataSourceDesc;
- }
-
- public void setDataSourceDesc(String dataSourceDesc) {
- this.dataSourceDesc = dataSourceDesc;
- }
-
- public Long getDataSourceTypeId() {
- return dataSourceTypeId;
- }
-
- public void setDataSourceTypeId(Long dataSourceTypeId) {
- this.dataSourceTypeId = dataSourceTypeId;
- }
-
- public String getCreateIdentify() {
- return createIdentify;
- }
-
- public void setCreateIdentify(String createIdentify) {
- this.createIdentify = createIdentify;
- }
-
- public String getCreateSystem() {
- return createSystem;
- }
-
- public void setCreateSystem(String createSystem) {
- this.createSystem = createSystem;
- }
-
- public String getParameter() {
- return parameter;
- }
-
- public void setParameter(String parameter) {
- this.parameter = parameter;
- }
-
- public Long getDataSourceEnvId() {
- return dataSourceEnvId;
- }
-
- public void setDataSourceEnvId(Long dataSourceEnvId) {
- this.dataSourceEnvId = dataSourceEnvId;
- }
-
- public Date getCreateTime() {
- return createTime;
- }
-
- public void setCreateTime(Date createTime) {
- this.createTime = createTime;
- }
-
- public Date getModifyTime() {
- return modifyTime;
- }
-
- public void setModifyTime(Date modifyTime) {
- this.modifyTime = modifyTime;
- }
-
- public Map<String, Object> getConnectParams() {
- if(connectParams.isEmpty() && StringUtils.isNotBlank(parameter)){
- connectParams.putAll(Objects.requireNonNull(Json.fromJson(parameter, Map.class)));
- }
- return connectParams;
- }
-
- public void setConnectParams(Map<String, Object> connectParams) {
- this.connectParams = connectParams;
- }
-
- public String getCreateUser() {
- return createUser;
- }
-
- public void setCreateUser(String createUser) {
- this.createUser = createUser;
- }
-
- public String getModifyUser() {
- return modifyUser;
- }
-
- public void setModifyUser(String modifyUser) {
- this.modifyUser = modifyUser;
- }
-
- public DataSourceType getDataSourceType() {
- return dataSourceType;
- }
-
- public void setDataSourceType(DataSourceType dataSourceType) {
- this.dataSourceType = dataSourceType;
- }
-
- public DataSourceEnv getDataSourceEnv() {
- return dataSourceEnv;
- }
-
- public void setDataSourceEnv(DataSourceEnv dataSourceEnv) {
- this.dataSourceEnv = dataSourceEnv;
- }
-
- public List<DataSourceParamKeyDefinition> getKeyDefinitions() {
- return keyDefinitions;
- }
-
- public void setKeyDefinitions(List<DataSourceParamKeyDefinition> keyDefinitions) {
- this.keyDefinitions = keyDefinitions;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java
deleted file mode 100644
index 3e3aa03..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceEnv.java
+++ /dev/null
@@ -1,189 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.common.domain;
-
-import org.apache.linkis.datasourcemanager.common.util.json.Json;
-import org.apache.commons.lang.StringUtils;
-import org.codehaus.jackson.annotate.JsonIgnore;
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-
-import javax.validation.constraints.NotNull;
-import javax.validation.constraints.Size;
-import java.util.*;
-
-/**
- * Store the data source environment information
- */
-@JsonSerialize(include= JsonSerialize.Inclusion.NON_EMPTY)
-@JsonIgnoreProperties(value = {"hibernateLazyInitializer", "handler"}, ignoreUnknown = true)
-public class DataSourceEnv {
-
- private Long id;
- /**
- * Environment name
- */
- @NotNull
- private String envName;
-
- /**
- * Environment description
- */
- @Size(min = 0, max = 200)
- private String envDesc;
-
- /**
- * ID of data source type
- */
- @NotNull
- private Long dataSourceTypeId;
-
- private DataSourceType dataSourceType;
- /**
- * Connection parameters for environment
- */
- private Map<String, Object> connectParams = new HashMap<>();
-
- /**
- * Parameter JSON string
- */
- @JsonIgnore
- private String parameter;
-
- /**
- * Create time
- */
- private Date createTime;
-
- /**
- * Creator
- */
- private String createUser;
-
- /**
- * Modify time
- */
- private Date modifyTime;
-
- /**
- * Modify user
- */
- private String modifyUser;
-
- @JsonIgnore
- private List<DataSourceParamKeyDefinition> keyDefinitions = new ArrayList<>();
-
- public Long getId() {
- return id;
- }
-
- public void setId(Long id) {
- this.id = id;
- }
-
- public String getEnvName() {
- return envName;
- }
-
- public void setEnvName(String envName) {
- this.envName = envName;
- }
-
- public String getEnvDesc() {
- return envDesc;
- }
-
- public void setEnvDesc(String envDesc) {
- this.envDesc = envDesc;
- }
-
- public Map<String, Object> getConnectParams() {
- if(connectParams.isEmpty() && StringUtils.isNotBlank(parameter)){
- connectParams.putAll(Objects.requireNonNull(Json.fromJson(parameter, Map.class)));
- }
- return connectParams;
- }
-
- public void setConnectParams(Map<String, Object> connectParams) {
- this.connectParams = connectParams;
- }
-
- public String getParameter() {
- return parameter;
- }
-
- public void setParameter(String parameter) {
- this.parameter = parameter;
- }
-
- public Date getCreateTime() {
- return createTime;
- }
-
- public void setCreateTime(Date createTime) {
- this.createTime = createTime;
- }
-
- public String getCreateUser() {
- return createUser;
- }
-
- public void setCreateUser(String createUser) {
- this.createUser = createUser;
- }
-
- public Date getModifyTime() {
- return modifyTime;
- }
-
- public void setModifyTime(Date modifyTime) {
- this.modifyTime = modifyTime;
- }
-
- public String getModifyUser() {
- return modifyUser;
- }
-
- public void setModifyUser(String modifyUser) {
- this.modifyUser = modifyUser;
- }
-
- public Long getDataSourceTypeId() {
- return dataSourceTypeId;
- }
-
- public void setDataSourceTypeId(Long dataSourceTypeId) {
- this.dataSourceTypeId = dataSourceTypeId;
- }
-
- public List<DataSourceParamKeyDefinition> getKeyDefinitions() {
- return keyDefinitions;
- }
-
- public void setKeyDefinitions(List<DataSourceParamKeyDefinition> keyDefinitions) {
- this.keyDefinitions = keyDefinitions;
- }
-
- public DataSourceType getDataSourceType() {
- return dataSourceType;
- }
-
- public void setDataSourceType(DataSourceType dataSourceType) {
- this.dataSourceType = dataSourceType;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParamKeyDefinition.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParamKeyDefinition.java
deleted file mode 100644
index 7e688ff..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceParamKeyDefinition.java
+++ /dev/null
@@ -1,233 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.common.domain;
-
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-
-import java.io.InputStream;
-import java.util.Date;
-import java.util.List;
-import java.util.Map;
-
-/**
- * Parameter key definition for data source type
- */
-@JsonSerialize(include= JsonSerialize.Inclusion.NON_EMPTY)
-@JsonIgnoreProperties(value = {"hibernateLazyInitializer", "handler"}, ignoreUnknown = true)
-public class DataSourceParamKeyDefinition {
- /**
- * Key-value type
- */
- public enum ValueType{
- /**
- * Email format
- */
- EMAIL(String.class),
- /**
- * String
- */
- TEXT(String.class),
- /**
- * Long
- */
- NUMBER(Long.class),
- /**
- * SELECT
- */
- SELECT(String.class),
- /**
- * List
- */
- LIST(List.class),
- /**
- * Map
- */
- MAP(Map.class),
- /**
- * RADIO
- */
- RADIO(String.class),
- /**
- * Password
- */
- PASSWORD(String.class),
- /**
- * DateTime
- */
- DATE(Date.class),
- /**
- * File
- */
- FILE(InputStream .class);
- Class<?> type;
- ValueType(Class<?> type){
- this.type = type;
- }
- public Class<?> getJavaType(){
- return this.type;
- }
- }
-
- public enum Scope{
- /**
- * Env SCOPE
- */
- ENV,
- }
- /**
- * Definition id
- */
- private Long id;
-
- /**
- * Key name
- */
- private String key;
-
- /**
- * Definition description
- */
- private String description;
-
- /**
- * Option name
- */
- private String name;
-
- /**
- * Default value
- */
- private String defaultValue;
-
- /**
- * Value type
- */
- private ValueType valueType;
- /**
- * Scope
- */
- private Scope scope;
- /**
- * If the definition is required
- */
- private boolean require;
- /**
- * Value regex
- */
- private String valueRegex;
-
- /**
- * Reference id
- */
- private Long refId;
-
- /**
- * Reference value
- */
- private String refValue;
-
- public Long getId() {
- return id;
- }
-
- public void setId(Long id) {
- this.id = id;
- }
-
- public String getKey() {
- return key;
- }
-
- public void setKey(String key) {
- this.key = key;
- }
-
- public String getDescription() {
- return description;
- }
-
- public void setDescription(String description) {
- this.description = description;
- }
-
- public String getName() {
- return name;
- }
-
- public void setName(String name) {
- this.name = name;
- }
-
- public String getDefaultValue() {
- return defaultValue;
- }
-
- public void setDefaultValue(String defaultValue) {
- this.defaultValue = defaultValue;
- }
-
- public ValueType getValueType() {
- return valueType;
- }
-
- public void setValueType(ValueType valueType) {
- this.valueType = valueType;
- }
-
- public String getValueRegex() {
- return valueRegex;
- }
-
- public void setValueRegex(String valueRegex) {
- this.valueRegex = valueRegex;
- }
-
- public boolean isRequire() {
- return require;
- }
-
- public void setRequire(boolean require) {
- this.require = require;
- }
-
- public Scope getScope() {
- return scope;
- }
-
- public void setScope(Scope scope) {
- this.scope = scope;
- }
-
- public Long getRefId() {
- return refId;
- }
-
- public void setRefId(Long refId) {
- this.refId = refId;
- }
-
- public String getRefValue() {
- return refValue;
- }
-
- public void setRefValue(String refValue) {
- this.refValue = refValue;
- }
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceType.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceType.java
deleted file mode 100644
index f4255c5..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/domain/DataSourceType.java
+++ /dev/null
@@ -1,78 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.common.domain;
-
-
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-
-/**
- * Data source type entity
- */
-@JsonSerialize(include= JsonSerialize.Inclusion.NON_EMPTY)
-@JsonIgnoreProperties(value = {"hibernateLazyInitializer", "handler"}, ignoreUnknown = true)
-public class DataSourceType {
-
- private String id;
- /**
- * Icon class
- */
- private String icon;
-
- /**
- * Description
- */
- private String description;
-
- /**
- * Name
- */
- private String name;
-
- public String getId() {
- return id;
- }
-
- public void setId(String id) {
- this.id = id;
- }
-
- public String getIcon() {
- return icon;
- }
-
- public void setIcon(String icon) {
- this.icon = icon;
- }
-
- public String getDescription() {
- return description;
- }
-
- public void setDescription(String description) {
- this.description = description;
- }
-
- public String getName() {
- return name;
- }
-
- public void setName(String name) {
- this.name = name;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/CryptoUtils.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/CryptoUtils.java
deleted file mode 100644
index 8f689e4..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/CryptoUtils.java
+++ /dev/null
@@ -1,104 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.common.util;
-
-import org.apache.commons.codec.binary.Base64;
-import org.apache.commons.lang.StringUtils;
-
-import java.io.*;
-import java.security.MessageDigest;
-
-public class CryptoUtils {
- private CryptoUtils(){
- }
- /**
- * 序列化对象为String字符
- *
- * @param o Object
- * @return String
- * @throws Exception
- */
- public static String object2String(Serializable o) {
- try {
- ByteArrayOutputStream bos = new ByteArrayOutputStream();
- ObjectOutputStream oos = new ObjectOutputStream(bos);
- oos.writeObject(o);
- oos.flush();
- oos.close();
- bos.close();
- return new String(new Base64().encode(bos.toByteArray()));
- }catch(Exception e){
- throw new RuntimeException(e);
- }
- }
-
- /**
- * 反序列化字符串为对象
- *
- * @param str String
- * @return Object
- * @throws IOException
- * @throws ClassNotFoundException
- */
- public static Object string2Object(String str) {
- try {
- ByteArrayInputStream bis = new ByteArrayInputStream(new Base64().decode(str.getBytes()));
- ObjectInputStream ois = new ObjectInputStream(bis);
- Object o = ois.readObject();
- bis.close();
- ois.close();
- return o;
- }catch(Exception e){
- throw new RuntimeException(e);
- }
-
- }
-
- /**
- * MD5 encrypt
- * @param source
- * @param salt
- * @param iterator
- * @return
- */
- public static String md5(String source, String salt, int iterator){
- StringBuilder token = new StringBuilder();
- try{
- MessageDigest digest = MessageDigest.getInstance("md5");
- if(StringUtils.isNotEmpty(salt)){
- digest.update(salt.getBytes("UTF-8"));
- }
- byte[] result = digest.digest(source.getBytes());
- for(int i = 0; i < iterator - 1; i++){
- digest.reset();
- result = digest.digest(result);
- }
- for (byte aResult : result) {
- int temp = aResult & 0xFF;
- if (temp <= 0xF) {
- token.append("0");
- }
- token.append(Integer.toHexString(temp));
- }
- }catch(Exception e){
- throw new RuntimeException(e.getMessage());
- }
- return token.toString();
- }
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/PatternInjectUtils.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/PatternInjectUtils.java
deleted file mode 100644
index e295827..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/PatternInjectUtils.java
+++ /dev/null
@@ -1,177 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.common.util;
-
-import org.apache.linkis.datasourcemanager.common.util.json.Json;
-import org.apache.commons.lang.StringEscapeUtils;
-import org.apache.commons.lang.StringUtils;
-
-import java.util.Map;
-import java.util.regex.Matcher;
-import java.util.regex.Pattern;
-
-public class PatternInjectUtils {
-
- private PatternInjectUtils(){
-
- }
-
- private static final String PARAMETER_PREFIX = "[#|$]";
-
- private static final String ASSIGN_SYMBOL = "=";
-
- private static final Pattern REGEX = Pattern.compile(
- "(" + ASSIGN_SYMBOL + "?)" +
- "("+ PARAMETER_PREFIX + ")" +
- "\\{([\\w-]+)[|]?([^}]*)}?"
- );
-
- /**
- * inject pattern
- * @param template
- * @param params
- * @param useDefault use default Value
- * @return
- */
- public static String inject(String template, Object[] params, boolean useDefault, boolean escape, boolean placeholder){
- Matcher matcher = REGEX.matcher(template);
- StringBuffer sb = new StringBuffer();
- int offset = 0;
- while(matcher.find()){
- String value = "";
- String extra = "";
- if( offset < params.length && null != params[offset] ){
- Object paramsV = params[offset];
- if( paramsV instanceof String || paramsV instanceof Enum ||
- paramsV.getClass().isPrimitive() ||
- isWrapClass(paramsV.getClass())){
- value = escape? StringEscapeUtils.escapeJava(String.valueOf(paramsV))
- : String.valueOf(paramsV);
- }else{
- value = Json.toJson(paramsV, null);
- value = escape? StringEscapeUtils.escapeJava(value) : value;
- }
- if( null != matcher.group(1)
- && !"".equals(matcher.group(1))){
- extra = matcher.group(1);
- }
- offset ++;
- }else if(null != matcher.group(4) && useDefault){
- value = escape? StringEscapeUtils.escapeJava(String.valueOf(matcher.group(4)))
- : matcher.group(4);
- }
- if(StringUtils.isBlank(value) && !useDefault){
- value = "\"" + (escape?StringEscapeUtils.escapeJava(matcher.group(3))
- : matcher.group(3));
- }else if(!"$".equals(matcher.group(2)) && placeholder){
- value = "\"" + StringEscapeUtils.escapeJava(value) + "\"";
- }
- String result = (extra + value).replace("$", "\\$");
- matcher.appendReplacement(sb, result);
- }
- matcher.appendTail(sb);
- return sb.toString().replace("\\$","$");
- }
- public static String inject(String pattern, Object[] params){
- return inject(pattern, params, true, true, true);
- }
-
- /**
- * inject pattern
- * @param template
- * @param params
- * @param useDefault
- * @return
- */
- public static String inject(String template, Map<String, Object> params, boolean useDefault, boolean escape, boolean placeholder){
- Matcher matcher = REGEX.matcher(template);
- StringBuffer sb = new StringBuffer();
- //will be more faster?
- while(matcher.find()){
- String injected = matcher.group(3);
- if(null != injected && !"".equals(injected)){
- int flag = 0;
- String value = "";
- String extra = "";
- for(Map.Entry<String, Object> entry : params.entrySet()){
- if(injected.equals(entry.getKey()) && null != entry.getValue()){
- Object entryV = entry.getValue();
- if(entryV instanceof String || entryV instanceof Enum ||
- entryV.getClass().isPrimitive() ||
- isWrapClass(entryV.getClass())){
- value = escape? StringEscapeUtils.escapeJava(String.valueOf(entryV))
- : String.valueOf(entryV);
- }else{
- value = Json.toJson(entryV, null);
- value = escape? StringEscapeUtils.escapeJava(value) : value;
- }
- if(null != matcher.group(1)
- || !"".equals(matcher.group(1))){
- extra = matcher.group(1);
- }
- flag = 1;
- break;
- }
- }
- if(flag == 0 && null != matcher.group(4) && useDefault){
- value = escape? StringEscapeUtils.escapeJava(String.valueOf(matcher.group(4)))
- : matcher.group(4);
- }
- if(StringUtils.isBlank(value) && !useDefault){
- value = "\"*#{" + (escape?StringEscapeUtils.escapeJava(matcher.group(3))
- : matcher.group(3)) + "}*\"";
- } else if(!"$".equals(matcher.group(2)) && placeholder){
- value = "\"" + StringEscapeUtils.escapeJava(value) + "\"";
- }
- String result = (extra + value).replace("$", "\\$");
- matcher.appendReplacement(sb, result);
- }
- }
- matcher.appendTail(sb);
- String print = sb.toString();
- return print.replace("\\$","$").replace("","");
- }
-
- public static String injectPattern(String template, String valuePattern){
- Matcher matcher = REGEX.matcher(template);
- StringBuffer sb = new StringBuffer();
- while(matcher.find()){
- String extra = matcher.group(1);
- String value = StringEscapeUtils.escapeJava(matcher.group(3));
- value = (extra + value.replaceAll("[\\s\\S]+", valuePattern))
- .replace("$", "\\$");
- matcher.appendReplacement(sb, value);
- }
- matcher.appendTail(sb);
- return sb.toString().replace("\\$", "$");
- }
-
- public static String inject(String template, Map<String, Object> params){
- return inject(template, params, true, true, true);
- }
-
-
- private static boolean isWrapClass(Class clz){
- try{
- return ((Class)clz.getField("TYPE").get(null)).isPrimitive();
- }catch(Exception e){
- return false;
- }
- }
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/json/Json.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/json/Json.java
deleted file mode 100644
index ca0ca87..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/java/org/apache/linkis/datasourcemanager/common/util/json/Json.java
+++ /dev/null
@@ -1,114 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.common.util.json;
-
-import com.fasterxml.jackson.core.JsonGenerator;
-import com.fasterxml.jackson.core.JsonParser;
-import com.fasterxml.jackson.core.JsonProcessingException;
-import com.fasterxml.jackson.databind.*;
-import org.apache.commons.lang.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.io.BufferedReader;
-import java.io.InputStream;
-import java.io.InputStreamReader;
-import java.util.ArrayList;
-
-/**
- * Json utils
- */
-public class Json {
- private static final String PREFIX = "[";
- private static final String SUFFIX = "]";
- private static final Logger logger = LoggerFactory.getLogger(Json.class);
-
- private static ObjectMapper mapper;
-
- static{
- mapper = new ObjectMapper();
- mapper.configure(JsonParser.Feature.ALLOW_COMMENTS, true);
- mapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true);
- mapper.configure(JsonParser.Feature.ALLOW_SINGLE_QUOTES, true);
- mapper.configure(DeserializationFeature.READ_ENUMS_USING_TO_STRING, true);
- mapper.configure(SerializationFeature.WRITE_ENUMS_USING_TO_STRING, true);
- mapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_CONTROL_CHARS, true);
- //empty beans allowed
- mapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
- //ignore unknown properties
- mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
- //cancel to scape non ascii
- mapper.configure(JsonGenerator.Feature.ESCAPE_NON_ASCII, false);
- }
- private Json(){}
-
- @SuppressWarnings("unchecked")
- public static <T> T fromJson(String json, Class<?> clazz, Class<?>... parameters){
- if(StringUtils.isNotBlank(json)){
- try{
- if(parameters.length > 0){
- return (T)mapper.readValue(json, mapper.getTypeFactory().constructParametricType(clazz, parameters));
- }
- if(json.startsWith(PREFIX)
- && json.endsWith(SUFFIX)){
- JavaType javaType = mapper.getTypeFactory()
- .constructParametricType(ArrayList.class, clazz);
- return mapper.readValue(json, javaType);
- }
- return (T)mapper.readValue(json, clazz);
- } catch (Exception e) {
- logger.info(e.getLocalizedMessage());
- throw new RuntimeException(e);
- }
- }
- return null;
- }
-
- public static <T> T fromJson(InputStream stream, Class<?> clazz, Class<?>... parameters){
- StringBuilder builder = new StringBuilder();
- String jsonStr = null;
- try{
- BufferedReader reader = new BufferedReader(new InputStreamReader(stream, "UTF-8"));
- while((jsonStr = reader.readLine()) != null){
- builder.append(jsonStr);
- }
- reader.close();
- }catch(Exception e){
- logger.info(e.getLocalizedMessage());
- throw new RuntimeException(e);
- }
- return fromJson(builder.toString(), clazz, parameters);
- }
-
- public static String toJson(Object obj, Class<?> model){
- ObjectWriter writer = mapper.writer();
- if(null != obj){
- try{
- if(null != model){
- writer = writer.withView(model);
- }
- return writer.writeValueAsString(obj);
- } catch (JsonProcessingException e) {
- logger.info(e.getLocalizedMessage());
- throw new RuntimeException(e);
- }
- }
- return null;
- }
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/scala/org/apache/linkis/datasourcemanager/common/protocol/DsmQueryProtocol.scala b/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/scala/org/apache/linkis/datasourcemanager/common/protocol/DsmQueryProtocol.scala
deleted file mode 100644
index 14366db..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/common/src/main/scala/org/apache/linkis/datasourcemanager/common/protocol/DsmQueryProtocol.scala
+++ /dev/null
@@ -1,40 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.common.protocol
-
-import java.util
-
-/**
- * Store error code map
- */
-trait DsmQueryProtocol {
-
-}
-
-/**
- * Query request of Data Source Information
- * @param id
- */
-case class DsInfoQueryRequest(id: String, system: String) extends DsmQueryProtocol
-
-/**
- * Response of parameter map
- * @param params
- */
-case class DsInfoResponse(status: Boolean, dsType: String, params : util.Map[String, Object], creator: String) extends DsmQueryProtocol
-
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/pom.xml b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/pom.xml
deleted file mode 100644
index 5b59e9a..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/pom.xml
+++ /dev/null
@@ -1,134 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
- <parent>
- <artifactId>linkis</artifactId>
- <groupId>org.apache.linkis</groupId>
- <version>1.0.3</version>
- </parent>
- <modelVersion>4.0.0</modelVersion>
-
- <artifactId>linkis-datasourcemanager-server</artifactId>
-
-
- <properties>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
- </properties>
-
- <dependencies>
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-module</artifactId>
- <version>${linkis.version}</version>
- <exclusions>
- <exclusion>
- <artifactId>asm</artifactId>
- <groupId>org.ow2.asm</groupId>
- </exclusion>
- </exclusions>
- </dependency>
- <!-- data source manager common dependency-->
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-datasourcemanager-common</artifactId>
- <version>${linkis.version}</version>
- <exclusions>
- <exclusion>
- <artifactId>asm</artifactId>
- <groupId>org.ow2.asm</groupId>
- </exclusion>
- </exclusions>
- </dependency>
- <!--jersey bean validation-->
- <dependency>
- <groupId>org.glassfish.jersey.ext</groupId>
- <artifactId>jersey-bean-validation</artifactId>
- <version>${jersey.version}</version>
- <exclusions>
- <exclusion>
- <artifactId>javax.ws.rs-api</artifactId>
- <groupId>javax.ws.rs</groupId>
- </exclusion>
- <exclusion>
- <artifactId>hk2-locator</artifactId>
- <groupId>org.glassfish.hk2</groupId>
- </exclusion>
- <exclusion>
- <artifactId>hk2-api</artifactId>
- <groupId>org.glassfish.hk2</groupId>
- </exclusion>
- </exclusions>
- </dependency>
- <!--bml client-->
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-bml-client</artifactId>
- <version>${linkis.version}</version>
- </dependency>
-
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-mybatis</artifactId>
- <version>${linkis.version}</version>
- </dependency>
- <!--rpc-->
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-rpc</artifactId>
- <version>${linkis.version}</version>
- </dependency>
- <!--Metadata common-->
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-metadatamanager-common</artifactId>
- <version>${linkis.version}</version>
- </dependency>
- </dependencies>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-deploy-plugin</artifactId>
- </plugin>
-
- <plugin>
- <groupId>net.alchim31.maven</groupId>
- <artifactId>scala-maven-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- </plugin>
- </plugins>
- <resources>
- <resource>
- <directory>src/main/resources</directory>
- </resource>
- <resource>
- <directory>src/main/java</directory>
- <includes>
- <include>**/*.xml</include>
- </includes>
- </resource>
- </resources>
- <finalName>${project.artifactId}-${project.version}</finalName>
- </build>
-</project>
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceDao.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceDao.java
deleted file mode 100644
index 2629217..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceDao.java
+++ /dev/null
@@ -1,77 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.dao;
-
-
-import org.apache.linkis.datasourcemanager.common.domain.DataSource;
-import org.apache.linkis.datasourcemanager.core.vo.DataSourceVo;
-import org.apache.ibatis.annotations.Param;
-
-import java.util.List;
-
-/**
- * Data source dao
- */
-public interface DataSourceDao {
-
- /**
- * Insert
- * @param dataSource data source
- */
- void insertOne(DataSource dataSource);
-
- /**
- * View detail
- * @param dataSourceId data source id
- * @param createSystem system name
- * @return data source entity
- */
- DataSource selectOneDetail(@Param("dataSourceId") Long dataSourceId,
- @Param("createSystem") String createSystem);
-
- /**
- * View normal
- * @param dataSourceId data source id
- * @param createSystem system name
- * @return data source entity
- */
- DataSource selectOne(@Param("dataSourceId") Long dataSourceId,
- @Param("createSystem") String createSystem);
- /**
- * Delete One
- * @param dataSourceId data source id
- * @param createSystem create system
- * @return affect row
- */
- int removeOne(@Param("dataSourceId")Long dataSourceId,
- @Param("createSystem")String createSystem);
-
-
- /**
- * Update one
- * @param updatedOne updated one
- */
- void updateOne(DataSource updatedOne);
-
- /**
- * Page of query
- * @param dataSourceVo data source view entity
- * @return query list
- */
- List<DataSource> selectByPageVo(DataSourceVo dataSourceVo);
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceEnvDao.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceEnvDao.java
deleted file mode 100644
index 1349f1c..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceEnvDao.java
+++ /dev/null
@@ -1,69 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.dao;
-
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceEnv;
-import org.apache.linkis.datasourcemanager.core.vo.DataSourceEnvVo;
-
-import java.util.List;
-
-/**
- * Data source dao
- */
-public interface DataSourceEnvDao {
-
- /**
- * View details of data source environment information
- * @param dataSourceEnvId env id
- * @return
- */
- DataSourceEnv selectOneDetail(Long dataSourceEnvId);
-
- /**
- * Insert one
- * @param dataSourceEnv environment
- */
- void insertOne(DataSourceEnv dataSourceEnv);
-
- /**
- * List all by type id
- * @param dataSourceTypeId type id
- * @return
- */
- List<DataSourceEnv> listByTypeId(Long dataSourceTypeId);
-
- /**
- * Remove one
- * @param envId env id
- * @return
- */
- int removeOne(Long envId);
-
- /**
- * Update one
- * @param updatedOne
- */
- void updateOne(DataSourceEnv updatedOne);
-
- /**
- * Page query
- * @param dataSourceEnvVo environment view entity
- * @return
- */
- List<DataSourceEnv> selectByPageVo(DataSourceEnvVo dataSourceEnvVo);
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceParamKeyDao.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceParamKeyDao.java
deleted file mode 100644
index 48a901a..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceParamKeyDao.java
+++ /dev/null
@@ -1,42 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.dao;
-
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-import org.apache.ibatis.annotations.Param;
-
-import java.util.List;
-
-public interface DataSourceParamKeyDao {
-
- /**
- * List by data source type id
- * @param dataSourceTypeId type id
- * @return
- */
- List<DataSourceParamKeyDefinition> listByDataSourceType(Long dataSourceTypeId);
-
- /**
- * List by data source type id and scope
- * @param dataSourceTypeId type id
- * @param scope scope
- * @return
- */
- List<DataSourceParamKeyDefinition> listByDataSourceTypeAndScope(@Param("dataSourceTypeId")Long dataSourceTypeId,
- @Param("scope")DataSourceParamKeyDefinition.Scope scope);
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceTypeDao.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceTypeDao.java
deleted file mode 100644
index c84fecb..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceTypeDao.java
+++ /dev/null
@@ -1,37 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.dao;
-
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceType;
-
-import java.util.List;
-
-public interface DataSourceTypeDao {
- /**
- * Get all types
- * @return type entity list
- */
- List<DataSourceType> getAllTypes();
-
- /**
- * View
- * @param typeId
- * @return
- */
- DataSourceType selectOne(Long typeId);
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceTypeEnvDao.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceTypeEnvDao.java
deleted file mode 100644
index 5f2f88b..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/DataSourceTypeEnvDao.java
+++ /dev/null
@@ -1,38 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.dao;
-
-import org.apache.ibatis.annotations.Param;
-
-public interface DataSourceTypeEnvDao {
-
- /**
- * Insert relation between type and environment
- * @param dataSourceTypeId data source type
- * @param dataSourceEnvId data source env
- */
- void insertRelation(@Param("dataSourceTypeId") Long dataSourceTypeId,
- @Param("dataSourceEnvId") Long dataSourceEnvId);
-
- /**
- * Remove relations by environment id
- * @param envId
- * @return
- */
- int removeRelationsByEnvId(Long envId);
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSouceMapper.xml b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSouceMapper.xml
deleted file mode 100644
index 4b90cb1..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSouceMapper.xml
+++ /dev/null
@@ -1,182 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
-
-<mapper namespace="org.apache.linkis.datasourcemanager.core.dao.DataSourceDao">
- <resultMap id="dataSourceDetailMap" type="DataSource">
- <result property="id" column="id"/>
- <result property="dataSourceName" column="datasource_name"/>
- <result property="dataSourceDesc" column="datasource_desc"/>
- <result property="dataSourceTypeId" column="datasource_type_id"/>
- <result property="createIdentify" column="create_identity"/>
- <result property="createSystem" column="create_system"/>
- <result property="parameter" column="parameter"/>
- <result property="dataSourceEnvId" column="datasource_env_id"/>
- <result property="createTime" column="create_time"/>
- <result property="modifyTime" column="modify_time"/>
- <result property="modifyUser" column="modify_user"/>
- <result property="createUser" column="create_user"/>
- <association property="dataSourceType" javaType="DataSourceType">
- <result property="icon" column="icon"/>
- <result property="name" column="name"/>
- </association>
- <association property="dataSourceEnv" javaType="DataSourceEnv" column="datasource_env_id" select="selectEnvById">
- <result property="envName" column="env_name"/>
- <result property="parameter" column="env_parameter"/>
- </association>
- </resultMap>
-
- <resultMap id="dataSourceMap" type="DataSource">
- <result property="id" column="id"/>
- <result property="dataSourceName" column="datasource_name"/>
- <result property="dataSourceDesc" column="datasource_desc"/>
- <result property="dataSourceTypeId" column="datasource_type_id"/>
- <result property="createIdentify" column="create_identity"/>
- <result property="createSystem" column="create_system"/>
- <result property="parameter" column="parameter"/>
- <result property="dataSourceEnvId" column="datasource_env_id"/>
- <result property="createTime" column="create_time"/>
- <result property="modifyTime" column="modify_time"/>
- <result property="modifyUser" column="modify_user"/>
- <result property="createUser" column="create_user"/>
- </resultMap>
-
- <sql id="data_source_insert_columns">
- `datasource_name`, `datasource_type_id`, `datasource_desc`,
- `create_identify`, `create_system`, `create_user`, `parameter`, `create_time`,
- `modify_user`, `modify_time`, `datasource_env_id`
- </sql>
-
- <sql id="data_source_query_columns">
- `id`, `datasource_name`, `datasource_type_id`, `datasource_desc`,
- `create_identify`, `create_system`, `create_user`, `parameter`, `create_time`,
- `modify_user`, `modify_time`, `datasource_env_id`
- </sql>
-
- <sql id="data_source_query_page">
- `id`, `datasource_name`, `datasource_type_id`, `datasource_desc`,
- `create_identify`, `create_system`, `create_user`, `create_time`,
- `modify_user`, `modify_time`, `datasource_env_id`
- </sql>
- <sql id="data_source_join_type">
- t.`name`, t.`icon`,
- d.`id`, d.`datasource_name`, d.`datasource_type_id`, d.`datasource_desc`,
- d.`create_identify`, d.`create_system`, d.`create_user`, d.`parameter`, d.`create_time`,
- d.`modify_user`, d.`modify_time`, d.`datasource_env_id`
- </sql>
-
- <insert id="insertOne" useGeneratedKeys="true" keyProperty="id" parameterType="DataSource">
- <![CDATA[INSERT INTO `linkis_datasource`(]]>
- <include refid="data_source_insert_columns"/>
- <![CDATA[) VALUES(#{dataSourceName}, #{dataSourceTypeId},
- #{dataSourceDesc,jdbcType=VARCHAR}, #{createIdentify}, #{createSystem}, #{createUser},
- #{parameter}, #{createTime}, #{modifyUser,jdbcType=VARCHAR}, #{modifyTime,jdbcType=DATE},
- #{dataSourceEnvId,jdbcType=INTEGER})]]>
- </insert>
-
- <select id="selectOneDetail" resultMap="dataSourceDetailMap">
- <![CDATA[SELECT ]]>
- <include refid="data_source_join_type"/>
- <![CDATA[ FROM `linkis_datasource_type` t
- INNER JOIN `linkis_datasource` d ON t.`id` = d.`datasource_type_id`]]>
- <where>
- d.`id` = #{dataSourceId}
- <if test="createSystem != null and createSystem != ''">
- AND d.`create_system` = #{createSystem}
- </if>
- </where>
- </select>
-
- <select id="selectOne" resultMap="dataSourceMap">
- <![CDATA[SELECT ]]>
- <include refid="data_source_query_columns"/>
- <![CDATA[ FROM `linkis_datasource` ]]>
- <where>
- `id` = #{dataSourceId}
- <if test="createSystem != null and createSystem != ''">
- AND `create_system` = #{createSystem}
- </if>
- </where>
- </select>
-
- <select id="selectEnvById" resultType="DataSourceEnv">
- <![CDATA[SELECT `env_name`, `parameter` AS `env_parameter`
- FROM `linkis_datasource_env` WHERE `id` = #{id};]]>
- </select>
-
- <delete id="removeOne">
- <![CDATA[DELETE FROM `linkis_datasource`]]>
- <where>
- `id` = #{dataSourceId}
- <if test="createSystem != null and createSystem != ''">
- AND `create_system` = #{createSystem}
- </if>
- </where>
- </delete>
-
- <update id="updateOne">
- <![CDATA[UPDATE `linkis_datasource`]]>
- <set>
- <![CDATA[`datasource_name` = #{dataSourceName},]]>
- <![CDATA[`datasource_desc` = #{dataSourceDesc,jdbcType=VARCHAR},]]>
- <if test="dataSourceTypeId != null and dataSourceTypeId > 0">
- <![CDATA[`datasource_type_id` = #{dataSourceTypeId},]]>
- </if>
- <![CDATA[`create_identify` = #{createIdentify,jdbcType=VARCHAR},]]>
- <![CDATA[`create_system` = #{createSystem,jdbcType=VARCHAR},]]>
- <![CDATA[`parameter` = #{parameter,jdbcType=VARCHAR},]]>
- <![CDATA[`datasource_env_id` = #{dataSourceEnvId,jdbcType=INTEGER},]]>
- <![CDATA[`modify_time` = #{modifyTime,jdbcType=DATE},]]>
- <![CDATA[`modify_user` = #{modifyUser,jdbcType=VARCHAR}]]>
- </set>
- <where>
- `id` = #{id}
- </where>
- </update>
-
- <select id="selectByPageVo" resultMap="dataSourceMap">
- <![CDATA[SELECT ]]>
- <include refid="data_source_query_page"/>
- <![CDATA[ FROM `linkis_datasource` ]]>
- <where>
- <![CDATA[`create_system` = #{createSystem}]]>
- <if test="dataSourceName != null and dataSourceName != ''">
- <![CDATA[AND `datasource_name` like concat('%', #{dataSourceName}, '%')]]>
- </if>
- <if test="dataSourceTypeId != null and dataSourceTypeId > 0">
- <![CDATA[AND `datasource_type_id` = #{dataSourceTypeId}]]>
- </if>
- <if test="createIdentifyList.size() > 0">
- <choose>
- <when test="createIdentifyList.size() == 1">
- <foreach collection="createIdentifyList" separator="" item="item" open="AND " close="">
- <![CDATA[`create_identity` = #{item}]]>
- </foreach>
- </when>
- <otherwise>
- <foreach collection="createIdentifyList" separator="," item="item"
- open="AND `create_identity` IN(" close=")">
- <![CDATA[#{item}]]>
- </foreach>
- </otherwise>
- </choose>
- </if>
- </where>
- </select>
-</mapper>
\ No newline at end of file
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceEnvMapper.xml b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceEnvMapper.xml
deleted file mode 100644
index d64d6d1..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceEnvMapper.xml
+++ /dev/null
@@ -1,101 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
-
-<mapper namespace="org.apache.linkis.datasourcemanager.core.dao.DataSourceEnvDao">
- <resultMap id="dataSourceEnvMap" type="DataSourceEnv">
- <result property="id" column="id"/>
- <result property="envName" column="env_name"/>
- <result property="envDesc" column="env_desc"/>
- <result property="parameter" column="parameter"/>
- <result property="createTime" column="create_time"/>
- <result property="createUser" column="create_user"/>
- <result property="modifyTime" column="modify_time"/>
- <result property="modifyUser" column="modify_user"/>
- <result property="dataSourceTypeId" column="data_source_type_id"/>
- <association property="dataSourceType" javaType="DataSourceType">
- <result property="icon" column="icon"/>
- <result property="name" column="name"/>
- </association>
- </resultMap>
-
- <sql id="data_source_env_insert_columns">
- `env_name`, `env_desc`, `parameter`, `create_time`,
- `create_user`, `modify_time`, `modify_user`
- </sql>
-
- <sql id="data_source_env_detail">
- `id`, `env_name`, `env_desc`, `parameter`, `create_time`,
- `create_user`, `modify_time`, `modify_user`
- </sql>
-
- <select id="selectOneDetail" resultMap="dataSourceEnvMap">
- <![CDATA[SELECT e.`id`, e.`env_name`, e.`env_desc`, e.`parameter`, e.`create_time`,
- e.`create_user`, e.`modify_time`, e.`modify_user`
- FROM `linkis_datasource_env` e WHERE id = #{dataSourceEnvId};]]>
- </select>
-
- <insert id="insertOne" useGeneratedKeys="true" keyProperty="id" parameterType="DataSourceEnv">
- <![CDATA[INSERT INTO `linkis_datasource_env`(]]>
- <include refid="data_source_env_insert_columns"/>
- <![CDATA[) VALUES(#{envName}, #{envDesc,jdbcType=VARCHAR},
- #{parameter}, #{createTime}, #{createUser},
- #{modifyTime,jdbcType=DATE}, #{modifyUser,jdbcType=VARCHAR});]]>
- </insert>
-
- <select id="listByTypeId" resultMap="dataSourceEnvMap">
- <![CDATA[SELECT `id`, `env_name` FROM `linkis_datasource_env`
- WHERE `id` IN (SELECT `env_id` FROM `linkis_datasource_type_env`
- WHERE `data_source_type_id` = #{id})]]>
- </select>
-
- <select id="selectByPageVo" resultMap="dataSourceEnvMap">
- <![CDATA[SELECT e.`id`, e.`env_name`, e.`env_desc`, e.`parameter`, e.`create_time`,
- e.`create_user`, e.`modify_time`, e.`modify_user`, te.`data_source_type_id`
- FROM `linkis_datasource_env` e INNER JOIN `linkis_datasource_type_env` te
- ON e.`id` = te.`env_id`]]>
- <where>
- <if test="envName != null and envName != ''">
- <![CDATA[e.`env_name` like concat('%', #{envName}, '%')]]>
- </if>
- <if test="dataSourceTypeId != null and dataSourceTypeId > 0">
- <![CDATA[AND te.`data_source_type_id` = #{dataSourceTypeId}]]>
- </if>
- </where>
- </select>
-
- <delete id="removeOne">
- <![CDATA[DELETE FROM `linkis_datasource_env` WHERE `id` = #{id};]]>
- </delete>
-
- <update id="updateOne">
- <![CDATA[UPDATE `linkis_datasource_env` ]]>
- <set>
- <![CDATA[`env_name` = #{envName},]]>
- <![CDATA[`env_desc` = #{envDesc,jdbcType=VARCHAR},]]>
- <![CDATA[`parameter` = #{parameter},]]>
- <![CDATA[`parameter` = #{parameter},]]>
- <![CDATA[`modify_time` = #{modifyTime,jdbcType=DATE},]]>
- <![CDATA[`modify_user` = #{modifyUser,jdbcType=VARCHAR}]]>
- </set>
- <where>
- `id` = #{id}
- </where>
- </update>
-</mapper>
\ No newline at end of file
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceParamKeyMapper.xml b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceParamKeyMapper.xml
deleted file mode 100644
index 5c03f8c..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceParamKeyMapper.xml
+++ /dev/null
@@ -1,50 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
-
-<mapper namespace="org.apache.linkis.datasourcemanager.core.dao.DataSourceParamKeyDao">
- <resultMap id="paramKeyMap" type="DataSourceParamKeyDefinition">
- <result property="id" column="id"/>
- <result property="key" column="key"/>
- <result property="description" column="description"/>
- <result property="name" column="name"/>
- <result property="defaultValue" column="default_value"/>
- <result property="valueType" column="value_type"/>
- <result property="scope" column="scope"/>
- <result property="require" column="require"/>
- <result property="valueRegex" column="value_regex"/>
- <result property="refId" column="ref_id"/>
- <result property="refValue" column="ref_value"/>
- </resultMap>
- <sql id="param_key_query">
- `id`, `key`, `description`, `name`, `require`, `scope`,
- `default_value`, `value_type`, `value_regex`, `ref_id`, `ref_value`
- </sql>
- <select id="listByDataSourceType" resultMap="paramKeyMap" parameterType="Long">
- <![CDATA[SELECT ]]>
- <include refid="param_key_query"/>
- <![CDATA[ FROM `linkis_datasource_type_key` WHERE `data_source_type_id` = #{dataSourceTypeId};]]>
- </select>
- <select id="listByDataSourceTypeAndScope" resultMap="paramKeyMap">
- <![CDATA[SELECT ]]>
- <include refid="param_key_query"/>
- <![CDATA[ FROM `linkis_datasource_type_key`
- WHERE `data_source_type_id` = #{dataSourceTypeId} AND `scope` = #{scope};]]>
- </select>
-</mapper>
\ No newline at end of file
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceTypeEnvMapper.xml b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceTypeEnvMapper.xml
deleted file mode 100644
index 20eb38f..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceTypeEnvMapper.xml
+++ /dev/null
@@ -1,33 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
-
-<mapper namespace="org.apache.linkis.datasourcemanager.core.dao.DataSourceTypeEnvDao">
-
- <insert id="insertRelation" useGeneratedKeys="true" keyProperty="id">
- <![CDATA[INSERT INTO `linkis_datasource_type_env`
- (`data_source_type_id`, `env_id`) VALUES
- (#{dataSourceTypeId}, #{dataSourceEnvId});]]>
- </insert>
-
- <delete id="removeRelationsByEnvId">
- <![CDATA[DELETE FROM `linkis_datasource_type_env` WHERE
- `env_id` = #{envId};]]>
- </delete>
-</mapper>
\ No newline at end of file
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceTypeMapper.xml b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceTypeMapper.xml
deleted file mode 100644
index 1712690..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/dao/mapper/DataSourceTypeMapper.xml
+++ /dev/null
@@ -1,36 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
-
-<mapper namespace="org.apache.linkis.datasourcemanager.core.dao.DataSourceTypeDao">
- <resultMap id="dataSourceTypeMap" type="dataSourceType">
- <result property="id" column="id"/>
- <result property="icon" column="icon"/>
- <result property="description" column="description"/>
- <result property="name" column="name"/>
- </resultMap>
-
- <select id="getAllTypes" resultMap="dataSourceTypeMap">
- <![CDATA[SELECT * FROM `linkis_datasource_type`;]]>
- </select>
-
- <select id="selectOne" resultMap="dataSourceTypeMap">
- <![CDATA[SELECT * FROM `linkis_datasource_type` WHERE id = #{id};]]>
- </select>
-</mapper>
\ No newline at end of file
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/CustomMultiPartFormDataTransformer.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/CustomMultiPartFormDataTransformer.java
deleted file mode 100644
index e49d149..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/CustomMultiPartFormDataTransformer.java
+++ /dev/null
@@ -1,315 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.formdata;
-
-import org.apache.linkis.common.exception.ErrorException;
-import org.apache.linkis.common.exception.WarnException;
-import org.apache.linkis.datasourcemanager.common.ServiceErrorCode;
-import org.apache.commons.lang.StringUtils;
-import org.glassfish.jersey.media.multipart.FormDataBodyPart;
-import org.glassfish.jersey.media.multipart.FormDataContentDisposition;
-import org.glassfish.jersey.media.multipart.FormDataMultiPart;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import javax.validation.ConstraintViolation;
-import javax.validation.ConstraintViolationException;
-import javax.validation.ValidationException;
-import javax.validation.Validator;
-import javax.validation.groups.Default;
-import javax.ws.rs.core.MediaType;
-import java.io.ByteArrayInputStream;
-import java.lang.reflect.Field;
-import java.lang.reflect.ParameterizedType;
-import java.lang.reflect.Type;
-import java.util.*;
-
-/**
- * Custom Transformer of multipart form
- */
-public class CustomMultiPartFormDataTransformer implements MultiPartFormDataTransformer{
- private static final Logger LOG = LoggerFactory.getLogger(CustomMultiPartFormDataTransformer.class);
- @Override
- public <T> T transformToObject(FormDataMultiPart formData, Class<?> clazz,
- Validator beanValidator)
- throws ValidationException, ErrorException {
- Field[] fields = clazz.getDeclaredFields();
- Map<String, Field> objectFieldMap = new HashMap<>();
- Object object;
- try {
- object = clazz.getConstructor().newInstance();
- //Put the bean fields information into map
- for (Field field : fields) {
- objectFieldMap.put(field.getName(), field);
- }
- Map<String, List<FormDataBodyPart>> formDataParts = formData.getFields();
- formDataParts.forEach((partName, bodyParts) -> {
- List<String> stepFieldNames = Arrays.asList(StringUtils.split(partName.replace("[", ".["), "."));
- String rootFieldName = stepFieldNames.get(0);
- Field field = objectFieldMap.get(rootFieldName);
- if (null != field) {
- try {
- injectToObject(object, field, stepFieldNames, 0, transformBodyPartToObject(bodyParts));
- } catch (Exception e) {
- throw new WarnException(ServiceErrorCode.TRANSFORM_FORM_ERROR.getValue(), "Fail to inject request parameter to object, message:[" + e.getMessage() +"]");
- }
- }
- });
- }catch(Exception e){
- LOG.error("Error in transform multipart from: " + e.getMessage());
- throw new ErrorException(ServiceErrorCode.TRANSFORM_FORM_ERROR.getValue(), "Error in transform multipart from: " + e.getMessage());
- }
- if(null != beanValidator){
- Set<ConstraintViolation<T>> result = beanValidator.validate((T)object, Default.class);
- if(result.size() > 0){
- throw new ConstraintViolationException(result);
- }
- }
- return (T)object;
- }
-
-
- /**
- * Transform body parts
- * @param bodyParts body parts
- * @return
- */
- private Object transformBodyPartToObject(List<FormDataBodyPart> bodyParts){
- if(bodyParts.size() > 0){
- Object[] result = new Object[bodyParts.size()];
- for(int i = 0 ; i < bodyParts.size(); i++){
- //Get media type
- FormDataBodyPart bodyPart = bodyParts.get(i);
- MediaType mediaType = bodyPart.getMediaType();
- if(mediaType.getType().equals(MediaType.TEXT_PLAIN_TYPE.getType())){
- result[i] = bodyPart.getValue();
- }else {
- //Think others as file
- FormStreamContent streamContent = new FormStreamContent();
- FormDataContentDisposition disposition = bodyPart.getFormDataContentDisposition();
- streamContent.setStream(new ByteArrayInputStream(bodyPart.getValueAs(byte[].class)));
- streamContent.setFileName(disposition.getFileName());
- streamContent.setCreateDate(disposition.getCreationDate());
- streamContent.setModifyDate(disposition.getModificationDate());
- streamContent.setParameters(disposition.getParameters());
- result[i] = streamContent;
- }
- }
- return result.length > 1 ? result : result[0];
- }
- return null;
- }
-
- /**
- * Inject value to object's field
- * @param object object injected
- * @param field field entity
- * @param stepFieldNames step field name list
- * @param step step index
- * @param value actual value
- * @throws Exception
- */
- private void injectToObject(Object object, Field field,
- List<String> stepFieldNames, int step, Object value) throws Exception {
- Class<?> fieldType = field.getType();
- String fieldName = field.getName();
- if(step + 1 >= stepFieldNames.size()){
- if(fieldType.equals(String.class) || PrimitiveUtils.isPrimitive(fieldType)){
- setObjectField(object, field, PrimitiveUtils.primitiveTypeConverse(value, fieldType));
- } else if (fieldType.equals(Object.class)) {
- setObjectField(object, field, value);
- } else {
- throw new IllegalAccessException("Cannot set value: " + value + " to object field: " + fieldName);
- }
- }else if(!PrimitiveUtils.isPrimitive(fieldType)){
- Object subObject = getObjectField(object, field);
- if(null == subObject){
- if(fieldType.equals(Map.class)){
- subObject = HashMap.class.getConstructor().newInstance();
- }else if(fieldType.equals(List.class)){
- subObject = ArrayList.class.getConstructor().newInstance();
- }else {
- subObject = fieldType.getConstructor().newInstance();
- }
- setObjectField(object, field, subObject);
- }
- injectRecurse(subObject, field, stepFieldNames, step + 1, value);
- }
-
- }
-
- /**
- * Inject value to map structure
- * @param mapObject map
- * @param valueType value type
- * @param stepFieldNames step field name list
- * @param step step index
- * @param value actual value
- * @throws Exception
- */
- private void injectToMap(Map mapObject, Class<?> valueType,
- List<String> stepFieldNames, int step, Object value) throws Exception{
- String fieldName = stepFieldNames.get(step);
- if(step + 1 >= stepFieldNames.size()){
- if(valueType.equals(String.class) || PrimitiveUtils.isPrimitive(valueType)){
- mapObject.put(fieldName, PrimitiveUtils.primitiveTypeConverse(value, valueType));
- }else if (valueType.equals(Object.class)){
- mapObject.put(fieldName, value);
- }else{
- throw new IllegalAccessException("Cannot set value: " + value + " to map: " + stepFieldNames.get(step - 1));
- }
- }
- }
-
- /**
- * Inject value to list structure
- * @param listObject list object
- * @param elementType element type
- * @param stepFieldNames step field name list
- * @param step step index
- * @param value value
- * @throws Exception
- */
- private void injectToList(List listObject, Class<?> elementType,
- List<String> stepFieldNames, int step, Object value) throws Exception{
- String fieldName = stepFieldNames.get(step);
- if(step + 1 >= stepFieldNames.size() && fieldName.matches("\\[\\d+]")){
- int index = Integer.parseInt(fieldName.substring(1, fieldName.length() - 1));
- //expand list
- int expand = index + 1 - listObject.size();
- while (expand-- > 0) {
- listObject.add(null);
- }
- if(elementType.equals(String.class) || PrimitiveUtils.isPrimitive(elementType)){
- listObject.set(index, PrimitiveUtils.primitiveTypeConverse(value, elementType));
- }else if (elementType.equals(Object.class)){
- listObject.set(index, value);
- }else{
- throw new IllegalAccessException("Cannot set value: " + value + " to array: " + stepFieldNames.get(step - 1));
- }
- }
- }
-
- /**
- * Inject recursively
- * @param subObject sub object
- * @param field sub object's field
- * @param stepFieldNames step field name list
- * @param step step index
- * @param value actual value
- * @throws Exception
- */
- private void injectRecurse(Object subObject, Field field,
- List<String> stepFieldNames, int step, Object value) throws Exception{
- Class<?> fieldType = field.getType();
- if(fieldType.equals(Map.class)){
- Class<?>[] generic = getGenericTypes(field);
- if(null == generic || generic[0].equals(String.class)){
- Class<?> valueType = null == generic ? String.class : generic[1];
- injectToMap((Map) subObject, valueType, stepFieldNames, step, value);
- }
- }else if(fieldType.equals(List.class)){
- Class<?>[] generic = getGenericTypes(field);
- injectToList((List)subObject, generic == null ? String.class : generic[0], stepFieldNames,
- step, value );
- }else{
- String nextFieldName = stepFieldNames.get(step);
- Field nextField = subObject.getClass().getField(nextFieldName);
- injectToObject(subObject, nextField, stepFieldNames, step, value);
- }
- }
-
- /**
- * Get generic types
- * @param field
- * @return
- */
- private Class<?>[] getGenericTypes(Field field){
- Type fc = field.getGenericType();
- if(fc instanceof ParameterizedType){
- Type[] types = ((ParameterizedType)fc).getActualTypeArguments();
- if(null != types && types.length > 0){
- Class<?>[] genericClazz = new Class<?>[types.length];
- for(int i = 0 ; i < genericClazz.length ; i++){
- genericClazz[i] = (Class<?>)types[i];
- }
- return genericClazz;
- }
- }
- return null;
- }
-
- private void setObjectField(Object object, Field field, Object value) throws Exception{
- field.setAccessible(true);
- field.set(object, value);
- }
-
- private Object getObjectField(Object object, Field field) throws Exception{
- field.setAccessible(true);
- return field.get(object);
- }
-
-
- /**
- * Tool of primitive
- */
- public static class PrimitiveUtils{
- public static Object primitiveTypeConverse(Object objValue, Class<?> type){
- if(type.equals(String.class) || null == objValue){
- return objValue;
- }
- String value = String.valueOf(objValue);
- if(!type.isPrimitive()){
- try {
- type = ((Class)type.getField("TYPE").get(null));
- } catch (Exception e) {
- //ignore
- }
- }
- switch (type.getSimpleName()) {
- case "int":
- return Integer.valueOf(value);
- case "long":
- return Long.valueOf(value);
- case "short":
- return Short.valueOf(value);
- case "char":
- return value.toCharArray()[0];
- case "float":
- return Float.valueOf(value);
- case "double":
- return Double.valueOf(value);
- case "byte":
- return Byte.valueOf(value);
- case "boolean":
- return Boolean.valueOf(value);
- default:
- throw new RuntimeException("Type: " + type.getSimpleName() + " is not primitive");
- }
- }
-
- public static boolean isPrimitive(Class<?> type){
- try {
- return type.isPrimitive() || ((Class) type.getField("TYPE").get(null)).isPrimitive();
- }catch(Exception e){
- return false;
- }
- }
- }
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/FormDataTransformerFactory.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/FormDataTransformerFactory.java
deleted file mode 100644
index d9661ed..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/FormDataTransformerFactory.java
+++ /dev/null
@@ -1,38 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.formdata;
-
-import java.util.concurrent.ConcurrentHashMap;
-
-/**
- * Simple static factory
- *
- */
-public class FormDataTransformerFactory {
-
- private static final ConcurrentHashMap<String, MultiPartFormDataTransformer>
- formDataTransformerStored = new ConcurrentHashMap<>();
-
- /**
- * Build custom implement
- * @return
- */
- public static MultiPartFormDataTransformer buildCustom(){
- return formDataTransformerStored.computeIfAbsent("custom", key -> new CustomMultiPartFormDataTransformer());
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/FormStreamContent.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/FormStreamContent.java
deleted file mode 100644
index fde2bc2..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/FormStreamContent.java
+++ /dev/null
@@ -1,86 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.formdata;
-
-import java.io.InputStream;
-import java.util.Date;
-import java.util.Map;
-
-/**
- * Stream content
- */
-public class FormStreamContent {
- private InputStream stream;
- private String fileName;
-
- private long size;
-
- private Date createDate;
-
- private Date modifyDate;
-
- private Map<String, String> parameters;
-
- public InputStream getStream() {
- return stream;
- }
-
- public void setStream(InputStream stream) {
- this.stream = stream;
- }
-
- public String getFileName() {
- return fileName;
- }
-
- public void setFileName(String fileName) {
- this.fileName = fileName;
- }
-
- public long getSize() {
- return size;
- }
-
- public void setSize(long size) {
- this.size = size;
- }
-
- public Date getCreateDate() {
- return createDate;
- }
-
- public void setCreateDate(Date createDate) {
- this.createDate = createDate;
- }
-
- public Date getModifyDate() {
- return modifyDate;
- }
-
- public void setModifyDate(Date modifyDate) {
- this.modifyDate = modifyDate;
- }
-
- public Map<String, String> getParameters() {
- return parameters;
- }
-
- public void setParameters(Map<String, String> parameters) {
- this.parameters = parameters;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/MultiPartFormDataTransformer.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/MultiPartFormDataTransformer.java
deleted file mode 100644
index 2b8061f..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/formdata/MultiPartFormDataTransformer.java
+++ /dev/null
@@ -1,43 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.formdata;
-
-import org.apache.linkis.common.exception.ErrorException;
-import org.glassfish.jersey.media.multipart.FormDataMultiPart;
-
-import javax.validation.ValidationException;
-import javax.validation.Validator;
-
-/**
- * Transformer of multipart form
- */
-public interface MultiPartFormDataTransformer {
- /**
- * Transform the form data to object and validate its fields
- * @param formData form data
- * @param clazz clazz
- * @param beanValidator validator
- * @param <T>
- * @return
- * @throws ValidationException
- * @throws ErrorException
- */
- <T>T transformToObject(FormDataMultiPart formData, Class<?> clazz, Validator beanValidator)
- throws ValidationException, ErrorException;
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceAdminRestfulApi.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceAdminRestfulApi.java
deleted file mode 100644
index 5235dab..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceAdminRestfulApi.java
+++ /dev/null
@@ -1,220 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.restful;
-
-import org.apache.linkis.common.exception.ErrorException;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceEnv;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-import org.apache.linkis.datasourcemanager.core.formdata.FormDataTransformerFactory;
-import org.apache.linkis.datasourcemanager.core.formdata.MultiPartFormDataTransformer;
-import org.apache.linkis.datasourcemanager.core.service.DataSourceInfoService;
-import org.apache.linkis.datasourcemanager.core.service.DataSourceRelateService;
-import org.apache.linkis.datasourcemanager.core.validate.ParameterValidator;
-import org.apache.linkis.datasourcemanager.core.vo.DataSourceEnvVo;
-import org.apache.linkis.server.Message;
-import org.apache.linkis.server.security.SecurityFilter;
-import org.glassfish.jersey.media.multipart.FormDataMultiPart;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.http.MediaType;
-import org.springframework.web.bind.annotation.*;
-
-import javax.annotation.PostConstruct;
-import javax.servlet.http.HttpServletRequest;
-import javax.validation.ConstraintViolation;
-import javax.validation.ConstraintViolationException;
-import javax.validation.Validator;
-import javax.validation.groups.Default;
-import java.util.Calendar;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-
-@RestController
-@RequestMapping(value = "/data_source",produces = {"application/json"})
-public class DataSourceAdminRestfulApi {
-
- @Autowired
- private DataSourceInfoService dataSourceInfoService;
-
- @Autowired
- private DataSourceRelateService dataSourceRelateService;
-
- @Autowired
- private ParameterValidator parameterValidator;
-
- @Autowired
- private Validator beanValidator;
-
- private MultiPartFormDataTransformer formDataTransformer;
-
- @PostConstruct
- public void initRestful(){
- this.formDataTransformer = FormDataTransformerFactory.buildCustom();
- }
-
- @RequestMapping(value = "/env/json",method = RequestMethod.POST)
- public Message insertJsonEnv(DataSourceEnv dataSourceEnv, HttpServletRequest req) throws ErrorException {
- String userName = SecurityFilter.getLoginUsername(req);
- if(!RestfulApiHelper.isAdminUser(userName)){
- return Message.error("User '" + userName + "' is not admin user[非管理员用户]");
- }
- //Bean validation
- Set<ConstraintViolation<DataSourceEnv>> result = beanValidator.validate(dataSourceEnv, Default.class);
- if(result.size() > 0){
- throw new ConstraintViolationException(result);
- }
- dataSourceEnv.setCreateUser(userName);
- insertDataSourceEnv(dataSourceEnv);
- return Message.ok().data("insert_id", dataSourceEnv.getId());
- }
-
- @RequestMapping(value = "/env/form",method = RequestMethod.POST)
- public Message insertFormEnv(FormDataMultiPart multiPartForm, HttpServletRequest request) throws ErrorException {
- String userName = SecurityFilter.getLoginUsername(request);
- if(!RestfulApiHelper.isAdminUser(userName)){
- return Message.error("User '" + userName + "' is not admin user[非管理员用户]");
- }
- DataSourceEnv dataSourceEnv = formDataTransformer.transformToObject(multiPartForm, DataSourceEnv.class, beanValidator);
- dataSourceEnv.setCreateUser(userName);
- insertDataSourceEnv(dataSourceEnv);
- return Message.ok().data("insert_id", dataSourceEnv.getId());
- }
-
- @RequestMapping(value = "/env_list/all/type/{type_id}",method = RequestMethod.GET)
- public Message getAllEnvListByDataSourceType(@PathVariable("type_id")Long typeId){
- List<DataSourceEnv> envList = dataSourceInfoService.listDataSourceEnvByType(typeId);
- return Message.ok().data("env_list", envList);
- }
-
- @RequestMapping (value = "/env/{env_id}",method = RequestMethod.GET)
- public Message getEnvEntityById(@PathVariable("env_id")Long envId){
- DataSourceEnv dataSourceEnv = dataSourceInfoService.getDataSourceEnv(envId);
- return Message.ok().data("env", dataSourceEnv);
- }
-
- @RequestMapping(value = "/env/{env_id}",method = RequestMethod.DELETE)
- public Message removeEnvEntity(@PathVariable("env_id")Long envId,
- HttpServletRequest request){
- String userName = SecurityFilter.getLoginUsername(request);
- if(!RestfulApiHelper.isAdminUser(userName)){
- return Message.error("User '" + userName + "' is not admin user[非管理员用户]");
- }
- Long removeId = dataSourceInfoService.removeDataSourceEnv(envId);
- if(removeId < 0){
- return Message.error("Fail to remove data source environment[删除数据源环境信息失败], [id:" +
- envId + "]");
- }
- return Message.ok().data("remove_id", removeId);
- }
-
- @RequestMapping(value = "/env/{env_id}/json",method = RequestMethod.PUT)
- public Message updateJsonEnv(DataSourceEnv dataSourceEnv,
- @PathVariable("env_id")Long envId,
- HttpServletRequest request) throws ErrorException {
- String userName = SecurityFilter.getLoginUsername(request);
- if(!RestfulApiHelper.isAdminUser(userName)){
- return Message.error("User '" + userName + "' is not admin user[非管理员用户]");
- }
- //Bean validation
- Set<ConstraintViolation<DataSourceEnv>> result = beanValidator.validate(dataSourceEnv, Default.class);
- if(result.size() > 0){
- throw new ConstraintViolationException(result);
- }
- dataSourceEnv.setId(envId);
- dataSourceEnv.setModifyUser(userName);
- dataSourceEnv.setModifyTime(Calendar.getInstance().getTime());
- DataSourceEnv storedDataSourceEnv = dataSourceInfoService.getDataSourceEnv(envId);
- if(null == storedDataSourceEnv){
- return Message.error("Fail to update data source environment[更新数据源环境失败], " + "[Please check the id:'"
- + envId + " is correct ']");
- }
- dataSourceEnv.setCreateUser(storedDataSourceEnv.getCreateUser());
- updateDataSourceEnv(dataSourceEnv, storedDataSourceEnv);
- return Message.ok().data("update_id", envId);
- }
-
- @RequestMapping(value = "/env/{env_id}/form",method = RequestMethod.PUT,consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
- public Message updateFormEnv(FormDataMultiPart multiPartForm,
- @PathVariable("env_id")Long envId,
- HttpServletRequest request) throws ErrorException {
- if(null != multiPartForm) {
- String userName = SecurityFilter.getLoginUsername(request);
- if (!RestfulApiHelper.isAdminUser(userName)) {
- return Message.error("User '" + userName + "' is not admin user[非管理员用户]");
- }
- DataSourceEnv dataSourceEnv = formDataTransformer.transformToObject(multiPartForm, DataSourceEnv.class, beanValidator);
- dataSourceEnv.setId(envId);
- dataSourceEnv.setModifyUser(userName);
- dataSourceEnv.setModifyTime(Calendar.getInstance().getTime());
- DataSourceEnv storedDataSourceEnv = dataSourceInfoService.getDataSourceEnv(envId);
- if (null == storedDataSourceEnv) {
- return Message.error("Fail to update data source environment[更新数据源环境失败], " + "[Please check the id:'"
- + envId + " is correct ']");
- }
- dataSourceEnv.setCreateUser(storedDataSourceEnv.getCreateUser());
- updateDataSourceEnv(dataSourceEnv, storedDataSourceEnv);
- return Message.ok().data("update_id", envId);
- }
- return Message.error("Empty request");
- }
-
- @RequestMapping(value = "/env",method = RequestMethod.GET)
- public Message queryDataSourceEnv(@RequestParam(value = "name",required = false)String envName,
- @RequestParam(value = "typeId",required = false)Long dataSourceTypeId,
- @RequestParam(value = "currentPage",required = false)Integer currentPage,
- @RequestParam(value = "pageSize",required = false)Integer pageSize){
- DataSourceEnvVo dataSourceEnvVo = new DataSourceEnvVo(envName, dataSourceTypeId);
- dataSourceEnvVo.setCurrentPage(null != currentPage ? currentPage : 1);
- dataSourceEnvVo.setPageSize(null != pageSize? pageSize : 10);
- List<DataSourceEnv> queryList = dataSourceInfoService.queryDataSourceEnvPage(dataSourceEnvVo);
- return Message.ok().data("query_list", queryList);
- }
-
- /**
- * Inner method to insert data source environment
- * @param dataSourceEnv data source environment entity
- * @throws ErrorException
- */
- private void insertDataSourceEnv(DataSourceEnv dataSourceEnv) throws ErrorException{
- //Get key definitions in environment scope
- List<DataSourceParamKeyDefinition> keyDefinitionList = dataSourceRelateService
- .getKeyDefinitionsByType(dataSourceEnv.getDataSourceTypeId(), DataSourceParamKeyDefinition.Scope.ENV);
- dataSourceEnv.setKeyDefinitions(keyDefinitionList);
- Map<String, Object> connectParams = dataSourceEnv.getConnectParams();
- //Validate connect parameters
- parameterValidator.validate(keyDefinitionList, connectParams);
- dataSourceInfoService.saveDataSourceEnv(dataSourceEnv);
- }
-
- /**
- * Inner method to update data source environment
- * @param updatedOne new entity
- * @param storedOne old entity
- * @throws ErrorException
- */
- private void updateDataSourceEnv(DataSourceEnv updatedOne, DataSourceEnv storedOne) throws ErrorException{
- //Get key definitions in environment scope
- List<DataSourceParamKeyDefinition> keyDefinitionList = dataSourceRelateService
- .getKeyDefinitionsByType(updatedOne.getDataSourceTypeId(), DataSourceParamKeyDefinition.Scope.ENV);
- updatedOne.setKeyDefinitions(keyDefinitionList);
- Map<String, Object> connectParams = updatedOne.getConnectParams();
- //Validate connect parameters
- parameterValidator.validate(keyDefinitionList, connectParams);
- dataSourceInfoService.updateDataSourceEnv(updatedOne, storedOne);
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java
deleted file mode 100644
index 91337e9..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java
+++ /dev/null
@@ -1,257 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.restful;
-
-import org.apache.linkis.common.exception.ErrorException;
-import org.apache.linkis.datasourcemanager.common.domain.DataSource;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceType;
-import org.apache.linkis.datasourcemanager.core.formdata.FormDataTransformerFactory;
-import org.apache.linkis.datasourcemanager.core.formdata.MultiPartFormDataTransformer;
-import org.apache.linkis.datasourcemanager.core.service.DataSourceInfoService;
-import org.apache.linkis.datasourcemanager.core.service.DataSourceRelateService;
-import org.apache.linkis.datasourcemanager.core.validate.ParameterValidator;
-import org.apache.linkis.datasourcemanager.core.vo.DataSourceVo;
-import org.apache.linkis.server.Message;
-import org.apache.linkis.server.security.SecurityFilter;
-import org.apache.commons.lang.StringUtils;
-import org.glassfish.jersey.media.multipart.FormDataMultiPart;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.http.MediaType;
-import org.springframework.web.bind.annotation.*;
-
-import javax.annotation.PostConstruct;
-import javax.servlet.http.HttpServletRequest;
-import javax.validation.ConstraintViolation;
-import javax.validation.ConstraintViolationException;
-import javax.validation.Validator;
-import javax.validation.groups.Default;
-import java.util.Calendar;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-
-
-@RestController
-@RequestMapping(value = "/data_source",produces = {"application/json"})
-public class DataSourceCoreRestfulApi {
-
- private static final Logger LOG = LoggerFactory.getLogger(DataSourceCoreRestfulApi.class);
- @Autowired
- private DataSourceInfoService dataSourceInfoService;
-
- @Autowired
- private DataSourceRelateService dataSourceRelateService;
-
- @Autowired
- private ParameterValidator parameterValidator;
-
- @Autowired
- private Validator beanValidator;
-
- private MultiPartFormDataTransformer formDataTransformer;
-
-
- @PostConstruct
- public void initRestful(){
- this.formDataTransformer = FormDataTransformerFactory.buildCustom();
- }
-
- @RequestMapping(value = "/info/json" ,method = RequestMethod.POST)
- public Message insertJsonInfo(DataSource dataSource, HttpServletRequest req) throws ErrorException {
- String userName = SecurityFilter.getLoginUsername(req);
- //Bean validation
- Set<ConstraintViolation<DataSource>> result = beanValidator.validate(dataSource, Default.class);
- if(result.size() > 0){
- throw new ConstraintViolationException(result);
- }
- dataSource.setCreateUser(userName);
- insertDataSourceConfig(dataSource);
- return Message.ok().data("insert_id", dataSource.getId());
- }
-
- @RequestMapping(value = "/info/form",consumes = MediaType.MULTIPART_FORM_DATA_VALUE,method = RequestMethod.POST)
- public Message insertFormConfig(FormDataMultiPart multiPartForm, HttpServletRequest request) throws ErrorException {
- if(null != multiPartForm) {
- String userName = SecurityFilter.getLoginUsername(request);
- DataSource dataSource = formDataTransformer.transformToObject(multiPartForm, DataSource.class, beanValidator);
- dataSource.setCreateUser(userName);
- insertDataSourceConfig(dataSource);
- return Message.ok().data("insert_id", dataSource.getId());
- }
- return Message.error("Empty request");
- }
-
- @RequestMapping(value = "/info/{data_source_id}",method = RequestMethod.GET)
- public Message getInfoByDataSourceId(@RequestParam(value = "system",required = false)String createSystem,
- @PathVariable("data_source_id")Long dataSourceId,
- HttpServletRequest request){
- if(StringUtils.isBlank(createSystem)){
- return Message.error("'create system' is missing[缺少系统名]");
- }
- DataSource dataSource = dataSourceInfoService.getDataSourceInfo(dataSourceId, createSystem);
- //Decrypt
- if(null != dataSource) {
- RestfulApiHelper.decryptPasswordKey(dataSourceRelateService.getKeyDefinitionsByType(dataSource.getDataSourceTypeId())
- , dataSource.getConnectParams());
- }
- return Message.ok().data("info", dataSource);
- }
-
-
- @RequestMapping(value = "/info/{data_source_id}",method = RequestMethod.DELETE)
- public Message removeDataSource(@RequestParam(value = "system",required = false)String createSystem,
- @PathVariable("data_source_id")Long dataSourceId){
- if(StringUtils.isBlank(createSystem)){
- return Message.error("'create system' is missing[缺少系统名]");
- }
- Long removeId = dataSourceInfoService.removeDataSourceInfo(dataSourceId, createSystem);
- if(removeId < 0){
- return Message.error("Fail to remove data source[删除数据源信息失败], [id:" + dataSourceId +"]");
- }
- return Message.ok().data("remove_id", removeId);
- }
-
- @RequestMapping(value = "/info/{data_source_id}/json",method = RequestMethod.PUT)
- public Message updateDataSourceInJson(DataSource dataSource,
- @PathVariable("data_source_id")Long dataSourceId,
- HttpServletRequest req) throws ErrorException {
- String userName = SecurityFilter.getLoginUsername(req);
- //Bean validation
- Set<ConstraintViolation<DataSource>> result = beanValidator.validate(dataSource, Default.class);
- if(result.size() > 0){
- throw new ConstraintViolationException(result);
- }
- dataSource.setId(dataSourceId);
- dataSource.setModifyUser(userName);
- dataSource.setModifyTime(Calendar.getInstance().getTime());
- DataSource storedDataSource = dataSourceInfoService.getDataSourceInfoBrief(dataSourceId, dataSource.getCreateSystem());
- if(null == storedDataSource){
- return Message.error("Fail to update data source[更新数据源失败], " + "[Please check the id:'"
- + dataSourceId +"' and create system: '"+dataSource.getCreateSystem()+" is correct ']");
- }
- dataSource.setCreateUser(storedDataSource.getCreateUser());
- updateDataSourceConfig(dataSource, storedDataSource);
- return Message.ok().data("update_id", dataSourceId);
- }
-
- @RequestMapping(value = "/info/{data_source_id}/form",method = RequestMethod.PUT,consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
- public Message updateDataSourceInForm(FormDataMultiPart multiPartForm,
- @PathVariable("data_source_id") Long dataSourceId,
- HttpServletRequest req) throws ErrorException {
- String userName = SecurityFilter.getLoginUsername(req);
- DataSource dataSource = formDataTransformer.transformToObject(multiPartForm, DataSource.class, beanValidator);
- dataSource.setId(dataSourceId);
- dataSource.setModifyUser(userName);
- dataSource.setModifyTime(Calendar.getInstance().getTime());
- DataSource storedDataSource = dataSourceInfoService.getDataSourceInfoBrief(dataSourceId, dataSource.getCreateSystem());
- if(null == storedDataSource){
- return Message.error("Fail to update data source[更新数据源失败], " +
- "[Please check the id:'" + dataSourceId +"' and create system: '"+dataSource.getCreateSystem()+" is correct ']");
- }
- dataSource.setCreateUser(storedDataSource.getCreateUser());
- updateDataSourceConfig(dataSource, storedDataSource);
- return Message.ok().data("update_id", dataSourceId);
- }
-
- @RequestMapping(value = "/info",method = RequestMethod.GET)
- public Message queryDataSource(@RequestParam(value = "system",required = false)String createSystem,
- @RequestParam(value = "name",required = false)String dataSourceName,
- @RequestParam(value = "typeId",required = false)Long dataSourceTypeId,
- @RequestParam(value = "identifies",required = false)String identifies,
- @RequestParam(value = "currentPage",required = false)Integer currentPage,
- @RequestParam(value = "pageSize",required = false)Integer pageSize){
- if(StringUtils.isBlank(createSystem)){
- return Message.error("'create system' is missing[缺少系统名]");
- }
- DataSourceVo dataSourceVo = new DataSourceVo(dataSourceName, dataSourceTypeId,
- identifies, createSystem);
- dataSourceVo.setCurrentPage(null != currentPage ? currentPage : 1);
- dataSourceVo.setPageSize(null != pageSize? pageSize : 10);
- List<DataSource> queryList = dataSourceInfoService.queryDataSourceInfoPage(dataSourceVo);
- return Message.ok().data("query_list", queryList);
- }
-
- @RequestMapping(value = "/key_define/type/{type_id}",method = RequestMethod.GET)
- public Message getKeyDefinitionsByType(@PathVariable("type_id") Long dataSourceTypeId){
- List<DataSourceParamKeyDefinition> keyDefinitions = dataSourceRelateService.getKeyDefinitionsByType(dataSourceTypeId);
- return Message.ok().data("key_define", keyDefinitions);
- }
-
- @RequestMapping(value = "/key_define/type/{type_id}/{scope}",method = RequestMethod.GET)
- public Message getKeyDefinitionsByTypeAndScope(@PathVariable("type_id") Long dataSourceTypeId,
- @PathVariable("scope") String scopeValue){
- DataSourceParamKeyDefinition.Scope scope = DataSourceParamKeyDefinition.Scope.valueOf(scopeValue.toUpperCase());
- List<DataSourceParamKeyDefinition> keyDefinitions = dataSourceRelateService
- .getKeyDefinitionsByType(dataSourceTypeId, scope);
- return Message.ok().data("key_define", keyDefinitions);
- }
-
- @RequestMapping(value = "/type/all",method = RequestMethod.GET)
- public Message getAllDataSourceTypes(){
- List<DataSourceType> dataSourceTypes = dataSourceRelateService.getAllDataSourceTypes();
- return Message.ok().data("type_list", dataSourceTypes);
- }
-
- /**
- * Inner method to insert data source
- * @param dataSource data source entity
- * @throws ErrorException
- */
- private void insertDataSourceConfig(DataSource dataSource) throws ErrorException {
- if(null != dataSource.getDataSourceEnvId()){
- //Merge parameters
- dataSourceInfoService.addEnvParamsToDataSource(dataSource.getDataSourceEnvId(), dataSource);
- }
- //Validate connect parameters
- List<DataSourceParamKeyDefinition> keyDefinitionList = dataSourceRelateService
- .getKeyDefinitionsByType(dataSource.getDataSourceTypeId());
- dataSource.setKeyDefinitions(keyDefinitionList);
- Map<String,Object> connectParams = dataSource.getConnectParams();
- parameterValidator.validate(keyDefinitionList, connectParams);
- //Encrypt password value type
- RestfulApiHelper.encryptPasswordKey(keyDefinitionList, connectParams);
- dataSourceInfoService.saveDataSourceInfo(dataSource);
- }
-
- /**
- * Inner method to update data source
- * @param updatedOne new entity
- * @param storedOne old entity
- * @throws ErrorException
- */
- private void updateDataSourceConfig(DataSource updatedOne, DataSource storedOne) throws ErrorException{
- if(null != updatedOne.getDataSourceEnvId()){
- //Merge parameters
- dataSourceInfoService.addEnvParamsToDataSource(updatedOne.getDataSourceEnvId(), updatedOne);
- }
- //Validate connect parameters
- List<DataSourceParamKeyDefinition> keyDefinitionList = dataSourceRelateService
- .getKeyDefinitionsByType(updatedOne.getDataSourceTypeId());
- updatedOne.setKeyDefinitions(keyDefinitionList);
- Map<String, Object> connectParams = updatedOne.getConnectParams();
- parameterValidator.validate(keyDefinitionList, connectParams);
- RestfulApiHelper.encryptPasswordKey(keyDefinitionList, connectParams);
- dataSourceInfoService.updateDataSourceInfo(updatedOne, storedOne);
- }
-
-
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceOperateRestfulApi.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceOperateRestfulApi.java
deleted file mode 100644
index ea5f679..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceOperateRestfulApi.java
+++ /dev/null
@@ -1,118 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.restful;
-
-import org.apache.linkis.common.exception.ErrorException;
-import org.apache.linkis.datasourcemanager.common.domain.DataSource;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceType;
-import org.apache.linkis.datasourcemanager.core.formdata.FormDataTransformerFactory;
-import org.apache.linkis.datasourcemanager.core.formdata.MultiPartFormDataTransformer;
-import org.apache.linkis.datasourcemanager.core.service.DataSourceInfoService;
-import org.apache.linkis.datasourcemanager.core.service.DataSourceRelateService;
-import org.apache.linkis.datasourcemanager.core.service.MetadataOperateService;
-import org.apache.linkis.datasourcemanager.core.validate.ParameterValidateException;
-import org.apache.linkis.datasourcemanager.core.validate.ParameterValidator;
-import org.apache.linkis.metadatamanager.common.MdmConfiguration;
-import org.apache.linkis.server.Message;
-import org.apache.linkis.server.security.SecurityFilter;
-import org.apache.commons.lang.StringUtils;
-import org.glassfish.jersey.media.multipart.FormDataMultiPart;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.web.bind.annotation.RequestMapping;
-import org.springframework.web.bind.annotation.RequestMethod;
-import org.springframework.web.bind.annotation.RestController;
-
-import javax.annotation.PostConstruct;
-import javax.servlet.http.HttpServletRequest;
-import javax.validation.ConstraintViolation;
-import javax.validation.ConstraintViolationException;
-import javax.validation.Validator;
-import javax.validation.groups.Default;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-
-@RestController
-@RequestMapping(value = "/data_source/op",produces = {"application/json"})
-public class DataSourceOperateRestfulApi {
-
- @Autowired
- private MetadataOperateService metadataOperateService;
-
- @Autowired
- private DataSourceRelateService dataSourceRelateService;
-
- @Autowired
- private DataSourceInfoService dataSourceInfoService;
-
- @Autowired
- private ParameterValidator parameterValidator;
-
- @Autowired
- private Validator beanValidator;
-
- private MultiPartFormDataTransformer formDataTransformer;
-
- @PostConstruct
- public void initRestful(){
- this.formDataTransformer = FormDataTransformerFactory.buildCustom();
- }
-
- @RequestMapping(value = "/connect/json",method = RequestMethod.POST)
- public Message connect(DataSource dataSource,
- HttpServletRequest request) throws ParameterValidateException {
- String operator = SecurityFilter.getLoginUsername(request);
- //Bean validation
- Set<ConstraintViolation<DataSource>> result = beanValidator.validate(dataSource, Default.class);
- if(result.size() > 0){
- throw new ConstraintViolationException(result);
- }
- doConnect(operator, dataSource);
- return Message.ok().data("ok", true);
- }
-
- @RequestMapping(value = "/connect/form",method = RequestMethod.POST)
- public Message connect(FormDataMultiPart multiPartForm,
- HttpServletRequest request) throws ErrorException {
- String operator = SecurityFilter.getLoginUsername(request);
- DataSource dataSource = formDataTransformer.transformToObject(multiPartForm, DataSource.class, beanValidator);
- doConnect(operator, dataSource);
- return Message.ok().data("ok", true);
- }
-
- /**
- * Build a connection
- * @param dataSource
- */
- protected void doConnect(String operator, DataSource dataSource) throws ParameterValidateException {
- if(null != dataSource.getDataSourceEnvId()){
- dataSourceInfoService.addEnvParamsToDataSource(dataSource.getDataSourceEnvId(), dataSource);
- }
- //Validate connect parameters
- List<DataSourceParamKeyDefinition> keyDefinitionList = dataSourceRelateService
- .getKeyDefinitionsByType(dataSource.getDataSourceTypeId());
- dataSource.setKeyDefinitions(keyDefinitionList);
- Map<String,Object> connectParams = dataSource.getConnectParams();
- parameterValidator.validate(keyDefinitionList, connectParams);
- DataSourceType dataSourceType = dataSourceRelateService.getDataSourceType(dataSource.getDataSourceTypeId());
- metadataOperateService.doRemoteConnect(MdmConfiguration.METADATA_SERVICE_APPLICATION.getValue()
- + (StringUtils.isNotBlank(dataSourceType.getName())?("-" +dataSourceType.getName().toLowerCase()) : ""),
- operator, dataSource.getConnectParams());
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/RestfulApiHelper.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/RestfulApiHelper.java
deleted file mode 100644
index 67db2db..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/RestfulApiHelper.java
+++ /dev/null
@@ -1,115 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.restful;
-
-import org.apache.linkis.common.exception.WarnException;
-import org.apache.linkis.datasourcemanager.common.DsmConfiguration;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-import org.apache.linkis.datasourcemanager.common.util.CryptoUtils;
-import org.apache.linkis.datasourcemanager.core.restful.exception.BeanValidationExceptionMapper;
-import org.apache.linkis.datasourcemanager.core.validate.ParameterValidateException;
-import org.apache.linkis.server.Message;
-
-import javax.validation.ConstraintViolationException;
-import javax.ws.rs.core.Response;
-import java.util.Arrays;
-import java.util.List;
-import java.util.Map;
-
-/**
- * Helper of restful api entrance
- */
-public class RestfulApiHelper {
- /**
- * If is administrator
- * @param userName user name
- * @return
- */
- public static boolean isAdminUser(String userName){
- List<String> userList = Arrays.asList(DsmConfiguration.DSM_ADMIN_USER_LIST.getValue().split(","));
- return userList.contains(userName);
- }
-
- /**
- * Encrypt key of password type
- * @param keyDefinitionList definition list
- * @param connectParams connection parameters
- */
- public static void encryptPasswordKey(List<DataSourceParamKeyDefinition> keyDefinitionList,
- Map<String, Object> connectParams){
- keyDefinitionList.forEach(keyDefinition -> {
- if(keyDefinition.getValueType() == DataSourceParamKeyDefinition.ValueType.PASSWORD){
- Object password = connectParams.get(keyDefinition.getKey());
- if(null != password){
- connectParams.put(keyDefinition.getKey(), CryptoUtils.object2String(String.valueOf(password)));
- }
- }
- });
- }
-
- /**
- * Encrypt key of password type
- * @param keyDefinitionList definition list
- * @param connectParams connection parameters
- */
- public static void decryptPasswordKey(List<DataSourceParamKeyDefinition> keyDefinitionList,
- Map<String, Object> connectParams){
- keyDefinitionList.forEach(keyDefinition -> {
- if(keyDefinition.getValueType() == DataSourceParamKeyDefinition.ValueType.PASSWORD){
- Object password = connectParams.get(keyDefinition.getKey());
- if(null != password){
- connectParams.put(keyDefinition.getKey(), CryptoUtils.string2Object(String.valueOf(password)));
- }
- }
- });
- }
-
- /**
- *
- * @param tryOperation operate function
- * @param failMessage message
- */
- public static Response doAndResponse(TryOperation tryOperation, String method, String failMessage){
- try{
- Message message = tryOperation.operateAndGetMessage();
- return Message.messageToResponse(setMethod(message, method));
- }catch(ParameterValidateException e){
- return Message.messageToResponse(setMethod(Message.error(e.getMessage()), method));
- }catch(ConstraintViolationException e){
- return new BeanValidationExceptionMapper().toResponse(e);
- }catch(WarnException e){
- return Message.messageToResponse(setMethod(Message.warn(e.getMessage()), method));
- }catch(Exception e){
- return Message.messageToResponse(setMethod(Message.error(failMessage, e), method));
- }
- }
- private static Message setMethod(Message message, String method){
- message.setMethod(method);
- return message;
- }
-
-
- @FunctionalInterface
- public interface TryOperation {
-
- /**
- * Operate method
- */
- Message operateAndGetMessage() throws Exception;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/exception/BeanValidationExceptionMapper.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/exception/BeanValidationExceptionMapper.java
deleted file mode 100644
index fd61a9b..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/exception/BeanValidationExceptionMapper.java
+++ /dev/null
@@ -1,41 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.restful.exception;
-
-import org.apache.linkis.server.Message;
-
-import javax.validation.ConstraintViolationException;
-import javax.validation.ValidationException;
-import javax.ws.rs.core.Response;
-import javax.ws.rs.ext.ExceptionMapper;
-import javax.ws.rs.ext.Provider;
-
-/**
- * Map bean validation exception to response
- */
-@Provider
-public class BeanValidationExceptionMapper implements ExceptionMapper<ValidationException> {
- @Override
- public Response toResponse(ValidationException exception) {
- StringBuilder stringBuilder = new StringBuilder();
- ((ConstraintViolationException)exception)
- .getConstraintViolations().forEach(constraintViolation -> stringBuilder.append(constraintViolation.getMessage()).append(";"));
- Message message = Message.error("Bean validation error[实例校验出错], detail:" + stringBuilder.toString());
- return Message.messageToResponse(message);
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/BmlAppService.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/BmlAppService.java
deleted file mode 100644
index 7adcd86..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/BmlAppService.java
+++ /dev/null
@@ -1,50 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.service;
-
-import org.apache.linkis.common.exception.ErrorException;
-
-import java.io.InputStream;
-
-/**
- * BML application service
- */
-public interface BmlAppService {
-
- /**
- * Upload resource by bml client
- * @param fileName file name
- * @param inputStream inputStream
- * @return resource id
- */
- String clientUploadResource(String userName, String fileName, InputStream inputStream) throws ErrorException;
-
- /**
- * Remove resource by bml client
- * @param resourceId resource id
- */
- void clientRemoveResource(String userName, String resourceId) throws ErrorException;
-
- /**
- * Update resource by bml client
- * @param resourceId resource id
- * @param inputStream input stream
- * @return version
- */
- String clientUpdateResource(String userName, String resourceId, InputStream inputStream) throws ErrorException;
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceInfoService.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceInfoService.java
deleted file mode 100644
index c60fa0a..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceInfoService.java
+++ /dev/null
@@ -1,120 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.service;
-
-import org.apache.linkis.common.exception.ErrorException;
-import org.apache.linkis.datasourcemanager.common.domain.DataSource;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceEnv;
-import org.apache.linkis.datasourcemanager.core.vo.DataSourceEnvVo;
-import org.apache.linkis.datasourcemanager.core.vo.DataSourceVo;
-
-import java.util.List;
-
-public interface DataSourceInfoService {
-
- /**
- * Save data source information
- * @param dataSource data source
- */
- void saveDataSourceInfo(DataSource dataSource) throws ErrorException;
-
- /**
- * Add parameters of data source environment
- * @param dataSourceEnvId data source environment
- * @param dataSource data source
- */
- void addEnvParamsToDataSource(Long dataSourceEnvId, DataSource dataSource);
-
- /**
- * Get data source
- * @param dataSourceId id
- * @param createSystem system name
- * @return data source entity
- */
- DataSource getDataSourceInfo(Long dataSourceId, String createSystem);
-
- /**
- * Get data source brief information
- * @param dataSourceId data source id
- * @param createSystem system
- * @return
- */
- DataSource getDataSourceInfoBrief(Long dataSourceId, String createSystem);
- /**
- * Remove data source
- * @param dataSourceId id
- * @param createSystem system name
- * @return
- */
- Long removeDataSourceInfo(Long dataSourceId, String createSystem);
-
- /**
- * Update data source
- * @param updatedOne updated data source
- * @param storedOne stored data source
- */
- void updateDataSourceInfo(DataSource updatedOne, DataSource storedOne) throws ErrorException;
-
- /**
- * Page query of data source
- * @param dataSourceVo data source view entity
- * @return
- */
- List<DataSource> queryDataSourceInfoPage(DataSourceVo dataSourceVo);
-
- /**
- * Save data source environment
- * @param dataSourceEnv data source environment
- */
- void saveDataSourceEnv(DataSourceEnv dataSourceEnv) throws ErrorException;
-
- /**
- * List data source environments
- * @param dataSourceTypeId type id
- * @return
- */
- List<DataSourceEnv> listDataSourceEnvByType(Long dataSourceTypeId);
-
- /**
- * Get data source environment
- * @param envId environment id
- * @return
- */
- DataSourceEnv getDataSourceEnv(Long envId);
-
- /**
- * Remove data source environment
- * @param envId environment id
- * @return
- */
- Long removeDataSourceEnv(Long envId);
-
- /**
- * Update data source environment
- * @param updatedOne
- * @param storedOne
- */
- void updateDataSourceEnv(DataSourceEnv updatedOne, DataSourceEnv storedOne) throws ErrorException;
-
- /**
- * Page query of data source environment
- * @param dataSourceEnvVo
- * @return
- */
- List<DataSourceEnv> queryDataSourceEnvPage(DataSourceEnvVo dataSourceEnvVo);
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceOpService.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceOpService.java
deleted file mode 100644
index aa442e8..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceOpService.java
+++ /dev/null
@@ -1,21 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.service;
-
-public interface DataSourceOpService {
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceRelateService.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceRelateService.java
deleted file mode 100644
index 50e04b5..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/DataSourceRelateService.java
+++ /dev/null
@@ -1,54 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.service;
-
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceType;
-
-import java.util.List;
-
-public interface DataSourceRelateService {
- /**
- * Get key definitions by data source type and scope
- * @param dataSourceTypeId data source type id
- * @param scope scope
- * @return
- */
- List<DataSourceParamKeyDefinition> getKeyDefinitionsByType(Long dataSourceTypeId,
- DataSourceParamKeyDefinition.Scope scope);
-
- /**
- * Get key definitions by data source type and scope
- * @param dataSourceTypeId data source type id
- * @return
- */
- List<DataSourceParamKeyDefinition> getKeyDefinitionsByType(Long dataSourceTypeId);
-
- /**
- * Get all data source types
- * @return
- */
- List<DataSourceType> getAllDataSourceTypes();
-
- /**
- * Get data source type
- * @param typeId
- * @return
- */
- DataSourceType getDataSourceType(Long typeId);
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/MetadataOperateService.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/MetadataOperateService.java
deleted file mode 100644
index d29842b..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/MetadataOperateService.java
+++ /dev/null
@@ -1,38 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.service;
-
-import org.apache.linkis.common.exception.ErrorException;
-import org.apache.linkis.common.exception.WarnException;
-
-import java.util.Map;
-
-/**
- * Metadata service
- */
-public interface MetadataOperateService {
-
- /**
- * Build connection with parameters in request
- * @param mdRemoteServiceName metadata remote service
- * @param operator operate user
- * @param connectParams parameters
- * @throws ErrorException
- */
- void doRemoteConnect(String mdRemoteServiceName, String operator, Map<String, Object> connectParams) throws WarnException;
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/BmlAppServiceImpl.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/BmlAppServiceImpl.java
deleted file mode 100644
index fc18028..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/BmlAppServiceImpl.java
+++ /dev/null
@@ -1,104 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.service.impl;
-
-import org.apache.linkis.bml.client.BmlClient;
-import org.apache.linkis.bml.client.BmlClientFactory;
-import org.apache.linkis.bml.protocol.BmlDeleteResponse;
-import org.apache.linkis.bml.protocol.BmlUpdateResponse;
-import org.apache.linkis.bml.protocol.BmlUploadResponse;
-import org.apache.linkis.common.exception.ErrorException;
-import org.apache.linkis.datasourcemanager.common.ServiceErrorCode;
-import org.apache.linkis.datasourcemanager.core.service.BmlAppService;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.cloud.context.config.annotation.RefreshScope;
-import org.springframework.stereotype.Service;
-
-import javax.annotation.PostConstruct;
-import java.io.InputStream;
-
-/**
- * Wrap the communication between Bml service
- */
-@Service
-@RefreshScope
-public class BmlAppServiceImpl implements BmlAppService {
- private static final Logger LOG = LoggerFactory.getLogger(BmlAppService.class);
- /**
- * Bml client
- */
- private BmlClient client;
-
- @PostConstruct
- public void buildClient(){
- client = BmlClientFactory.createBmlClient();
- }
- @Override
- public String clientUploadResource(String userName, String fileName,
- InputStream inputStream) throws ErrorException{
- LOG.info("Upload resource to bml server: [ proxy_to_user: " + userName +
- ", file name:" + fileName + " ]");
- try{
- BmlUploadResponse response = client.uploadResource(userName, fileName, inputStream);
- if(!response.isSuccess()){
- throw new ErrorException(ServiceErrorCode.BML_SERVICE_ERROR.getValue(), "");
- }
- return response.resourceId();
- }catch(Exception e){
- LOG.error("Failed to upload resource to bml server[上传资源文件失败], [ proxy_to_user: " + userName +
- ", file name:" + fileName + " ]", e);
- throw e;
- }
- }
-
- @Override
- public void clientRemoveResource(String userName, String resourceId) throws ErrorException{
- LOG.info("Remove resource to bml server: [ proxy_to_user: " + userName +
- ", resource id:" + resourceId + " ]");
- try{
- BmlDeleteResponse response = client.deleteResource(userName, resourceId);
- if(!response.isSuccess()){
- throw new ErrorException(ServiceErrorCode.BML_SERVICE_ERROR.getValue(), "");
- }
- }catch(Exception e){
- LOG.error("Fail to remove resource to bml server[删除资源文件失败], [ proxy_to_user: " + userName +
- ", resource id:" + resourceId + " ]");
- throw e;
- }
- }
-
- @Override
- public String clientUpdateResource(String userName, String resourceId,
- InputStream inputStream) throws ErrorException{
- LOG.info("Update resource to bml server: [ proxy_to_user: " + userName +
- ", resource id:" + resourceId + " ]");
- try{
- //File name is invalid;
- BmlUpdateResponse response = client.updateResource(userName, resourceId, "filename", inputStream);
- if(!response.isSuccess()){
- throw new ErrorException(ServiceErrorCode.BML_SERVICE_ERROR.getValue(), "");
- }
- return response.version();
- }catch(Exception e){
- LOG.error("Fail to update resource to bml server[更新资源文件失败], [ proxy_to_user: " + userName +
- ", resource id:" + resourceId + " ]");
- throw e;
- }
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/DataSourceInfoServiceImpl.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/DataSourceInfoServiceImpl.java
deleted file mode 100644
index 5bc2b6f..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/DataSourceInfoServiceImpl.java
+++ /dev/null
@@ -1,369 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.service.impl;
-
-import com.github.pagehelper.PageHelper;
-import com.github.pagehelper.PageInfo;
-import org.apache.linkis.common.exception.ErrorException;
-import org.apache.linkis.datasourcemanager.core.dao.DataSourceParamKeyDao;
-import org.apache.linkis.datasourcemanager.core.dao.DataSourceTypeEnvDao;
-import org.apache.linkis.datasourcemanager.core.formdata.FormStreamContent;
-import org.apache.linkis.datasourcemanager.common.util.json.Json;
-import org.apache.linkis.datasourcemanager.core.dao.DataSourceDao;
-import org.apache.linkis.datasourcemanager.core.dao.DataSourceEnvDao;
-import org.apache.linkis.datasourcemanager.common.domain.DataSource;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceEnv;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-import org.apache.linkis.datasourcemanager.core.service.BmlAppService;
-import org.apache.linkis.datasourcemanager.core.service.DataSourceInfoService;
-import org.apache.linkis.datasourcemanager.core.vo.DataSourceEnvVo;
-import org.apache.linkis.datasourcemanager.core.vo.DataSourceVo;
-import org.apache.commons.lang.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Service;
-import org.springframework.transaction.annotation.Transactional;
-
-import java.io.InputStream;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
-import java.util.function.Consumer;
-import java.util.stream.Collectors;
-
-@Service
-public class DataSourceInfoServiceImpl implements DataSourceInfoService {
-
- private static final Logger LOG = LoggerFactory.getLogger(DataSourceInfoService.class);
- @Autowired
- private BmlAppService bmlAppService;
-
- @Autowired
- private DataSourceTypeEnvDao dataSourceTypeEnvDao;
-
- @Autowired
- private DataSourceDao dataSourceDao;
-
- @Autowired
- private DataSourceEnvDao dataSourceEnvDao;
-
- @Autowired
- private DataSourceParamKeyDao dataSourceParamKeyDao;
-
- @Override
- @Transactional(rollbackFor = Exception.class)
- public void saveDataSourceInfo(DataSource dataSource) throws ErrorException {
- storeConnectParams(dataSource.getCreateUser(), dataSource.getKeyDefinitions(),
- dataSource.getConnectParams(), parameter ->{
- dataSource.setParameter(parameter);
- //Save information into database
- dataSourceDao.insertOne(dataSource);
- });
- }
-
- @Override
- public void addEnvParamsToDataSource(Long dataSourceEnvId, DataSource dataSource) {
- DataSourceEnv dataSourceEnv = dataSourceEnvDao.selectOneDetail(dataSourceEnvId);
- if(null != dataSourceEnv){
- Map<String, Object> envParamMap = dataSourceEnv.getConnectParams();
- envParamMap.putAll(dataSource.getConnectParams());
- dataSource.setConnectParams(envParamMap);
- }
- }
-
- @Override
- public DataSource getDataSourceInfo(Long dataSourceId, String createSystem) {
- return dataSourceDao.selectOneDetail(dataSourceId, createSystem);
- }
-
- @Override
- public DataSource getDataSourceInfoBrief(Long dataSourceId, String createSystem) {
- return dataSourceDao.selectOne(dataSourceId, createSystem);
- }
-
- @Override
- @Transactional(rollbackFor = Exception.class)
- public Long removeDataSourceInfo(Long dataSourceId, String createSystem) {
- DataSource dataSource = dataSourceDao.selectOne(dataSourceId, createSystem);
- if(null != dataSource){
- //First to delete record in db
- int affect = dataSourceDao.removeOne(dataSourceId, createSystem);
- if(affect > 0){
- //Remove resource
- Map<String, Object> connectParams = dataSource.getConnectParams();
- List<DataSourceParamKeyDefinition> keyDefinitions = dataSourceParamKeyDao
- .listByDataSourceType(dataSource.getDataSourceTypeId());
- keyDefinitions.forEach(keyDefinition -> {
- if(keyDefinition.getValueType() == DataSourceParamKeyDefinition.ValueType.FILE
- && keyDefinition.getScope() != DataSourceParamKeyDefinition.Scope.ENV
- && connectParams.containsKey(keyDefinition.getKey())){
- try {
- //Proxy creator to delete resource
- bmlAppService.clientRemoveResource(dataSource.getCreateUser(), String
- .valueOf(connectParams.get(keyDefinition.getKey())));
- }catch(Exception e){
- //Ignore remove error
- }
- }
- });
- return dataSourceId;
- }
- }
- return -1L;
- }
-
- @Override
- @Transactional(rollbackFor = Exception.class)
- public void updateDataSourceInfo(DataSource updatedOne, DataSource storedOne) throws ErrorException{
- updateConnectParams(updatedOne.getCreateUser(), updatedOne.getKeyDefinitions(),
- updatedOne.getConnectParams(), storedOne.getConnectParams(),
- parameter -> {
- updatedOne.setParameter(parameter);
- //Save information into database
- dataSourceDao.updateOne(updatedOne);
- });
- }
-
- @Override
- public List<DataSource> queryDataSourceInfoPage(DataSourceVo dataSourceVo) {
- PageHelper.startPage(dataSourceVo.getCurrentPage(), dataSourceVo.getPageSize());
- List<DataSource> queryList = dataSourceDao.selectByPageVo(dataSourceVo);
- PageInfo<DataSource> pageInfo = new PageInfo<>(queryList);
- return pageInfo.getList();
- }
-
- @Override
- @Transactional(rollbackFor = Exception.class)
- public void saveDataSourceEnv(DataSourceEnv dataSourceEnv) throws ErrorException {
- storeConnectParams(dataSourceEnv.getCreateUser(), dataSourceEnv.getKeyDefinitions(),
- dataSourceEnv.getConnectParams(),
- parameter ->{
- dataSourceEnv.setParameter(parameter);
- //Save environment into database
- dataSourceEnvDao.insertOne(dataSourceEnv);
- //Store relation
- dataSourceTypeEnvDao.insertRelation(dataSourceEnv.getDataSourceTypeId(), dataSourceEnv.getId());
- });
- }
-
- @Override
- public List<DataSourceEnv> listDataSourceEnvByType(Long dataSourceTypeId) {
- return dataSourceEnvDao.listByTypeId(dataSourceTypeId);
- }
-
- @Override
- public DataSourceEnv getDataSourceEnv(Long envId) {
- return dataSourceEnvDao.selectOneDetail(envId);
- }
-
- @Override
- @Transactional(rollbackFor = Exception.class)
- public Long removeDataSourceEnv(Long envId) {
- DataSourceEnv dataSourceEnv = dataSourceEnvDao.selectOneDetail(envId);
- if(null != dataSourceEnv){
- //First to delete record in db
- int affect = dataSourceEnvDao.removeOne(envId);
- if(affect > 0){
- //Remove relations
- dataSourceTypeEnvDao.removeRelationsByEnvId(envId);
- //Remove resource
- Map<String, Object> connectParams = dataSourceEnv.getConnectParams();
- List<DataSourceParamKeyDefinition> keyDefinitions = dataSourceParamKeyDao
- .listByDataSourceTypeAndScope(dataSourceEnv.getDataSourceTypeId(), DataSourceParamKeyDefinition.Scope.ENV);
- keyDefinitions.forEach(keyDefinition -> {
- if(keyDefinition.getValueType() == DataSourceParamKeyDefinition.ValueType.FILE
- && connectParams.containsKey(keyDefinition.getKey())){
- try{
- //Proxy creator to delete resource
- bmlAppService.clientRemoveResource(dataSourceEnv.getCreateUser(), String
- .valueOf(connectParams.get(keyDefinition.getKey())));
- }catch(Exception e){
- //Ignore remove error
- }
- }
- });
- return envId;
- }
- }
- return -1L;
- }
-
- @Override
- @Transactional(rollbackFor = Exception.class)
- public void updateDataSourceEnv(DataSourceEnv updatedOne, DataSourceEnv storedOne) throws ErrorException {
- updateConnectParams(updatedOne.getCreateUser(), updatedOne.getKeyDefinitions(),
- updatedOne.getConnectParams(), storedOne.getConnectParams(),
- parameter ->{
- updatedOne.setParameter(parameter);
- //Update environment into database
- dataSourceEnvDao.updateOne(updatedOne);
- if(!updatedOne.getDataSourceTypeId().equals(storedOne.getDataSourceTypeId())){
- //Remove old relation and add new relation
- dataSourceTypeEnvDao.removeRelationsByEnvId(updatedOne.getId());
- dataSourceTypeEnvDao.insertRelation(updatedOne.getDataSourceTypeId(), updatedOne.getId());
- }
- });
- }
-
- @Override
- public List<DataSourceEnv> queryDataSourceEnvPage(DataSourceEnvVo dataSourceEnvVo) {
- PageHelper.startPage(dataSourceEnvVo.getCurrentPage(), dataSourceEnvVo.getPageSize());
- List<DataSourceEnv> queryList = dataSourceEnvDao.selectByPageVo(dataSourceEnvVo);
- PageInfo<DataSourceEnv> pageInfo = new PageInfo<>(queryList);
- return pageInfo.getList();
- }
-
- /**
- *
- * @param userName
- * @param keyDefinitionList
- * @param updatedParams
- * @param storedParams
- * @param parameterCallback
- * @throws ErrorException
- */
- private void updateConnectParams(String userName, List<DataSourceParamKeyDefinition> keyDefinitionList,
- Map<String, Object> updatedParams, Map<String, Object> storedParams,
- Consumer<String> parameterCallback) throws ErrorException {
- List<String> definedKeyNames = keyDefinitionList.stream().map(DataSourceParamKeyDefinition::getKey)
- .collect(Collectors.toList());
- List<String> uploadedResources = new ArrayList<>();
- try {
- updatedParams.entrySet().removeIf(entry -> {
- if (!definedKeyNames.contains(entry.getKey())) {
- return true;
- }
- Object paramValue = entry.getValue();
- if (paramValue instanceof FormStreamContent) {
- String resourceId = String.valueOf(storedParams.getOrDefault(entry.getKey(), ""));
- if (StringUtils.isNotBlank(resourceId)) {
- uploadFormStream(userName, (FormStreamContent) paramValue, resourceId);
- } else {
- resourceId = uploadFormStream(userName, (FormStreamContent) paramValue, "");
- }
- if (null == resourceId) {
- return true;
- }
- uploadedResources.add(resourceId);
- entry.setValue(resourceId);
- }
- storedParams.remove(entry.getKey());
- return false;
- });
- //Found the duplicate File
- List<String> duplicateResources = new ArrayList<>();
- keyDefinitionList.forEach( definedKey-> {
- if(definedKey.getValueType() == DataSourceParamKeyDefinition.ValueType.FILE
- && storedParams.containsKey(definedKey.getKey())){
- duplicateResources.add(String.valueOf(storedParams.get(definedKey.getKey())));
- }
- });
- parameterCallback.accept(Json.toJson(updatedParams, null));
- deleteResources(userName, duplicateResources);
- }catch(Exception e){
- deleteResources(userName, uploadedResources);
- if(e.getCause() instanceof ErrorException){
- throw (ErrorException)e.getCause();
- }
- throw e;
- }
- }
-
- /**
- * Upload the form stream context in connect parameters,
- * and serialize parameters
- * @param keyDefinitionList
- * @param connectParams
- * @param parameterCallback
- */
- private void storeConnectParams(String userName, List<DataSourceParamKeyDefinition> keyDefinitionList,
- Map<String, Object> connectParams,
- Consumer<String> parameterCallback) throws ErrorException{
- List<String> definedKeyNames = keyDefinitionList.stream().map(DataSourceParamKeyDefinition::getKey)
- .collect(Collectors.toList());
- List<String> uploadedResources = new ArrayList<>();
- try{
- connectParams.entrySet().removeIf(entry -> {
- if(!definedKeyNames.contains(entry.getKey())){
- return true;
- }
- Object paramValue = entry.getValue();
- //Upload stream resource in connection params
- if (paramValue instanceof FormStreamContent) {
- String resourceId = uploadFormStream(userName, (FormStreamContent)paramValue, "");
- if(null == resourceId) {
- return true;
- }
- uploadedResources.add(resourceId);
- entry.setValue(resourceId);
- }
- return false;
- });
- parameterCallback.accept(Json.toJson(connectParams, null));
- }catch(Exception e){
- deleteResources(userName, uploadedResources);
- if(e.getCause() instanceof ErrorException){
- throw (ErrorException)e.getCause();
- }
- throw e;
- }
- }
-
- /**
- * Upload form stream
- * @param userName user name
- * @param streamContent stream content
- * @param resourceId resource id
- * @return resource id or version tab
- */
- private String uploadFormStream(String userName, FormStreamContent streamContent, String resourceId){
- String fileName = streamContent.getFileName();
- InputStream inputStream = streamContent.getStream();
- if (null != inputStream){
- //Proxy creator to upload resource
- try{
- return StringUtils.isBlank(resourceId)? bmlAppService.clientUploadResource(userName, fileName, inputStream)
- : bmlAppService.clientUpdateResource(userName, resourceId, inputStream);
- }catch(Exception e){
- //Wrap with runtime exception
- throw new RuntimeException(e);
- }
- }
- return null;
- }
-
- /**
- * Delete uploaded resources
- * @param userName user name
- * @param uploadedResources resource id list
- */
- private void deleteResources(String userName, List<String> uploadedResources){
- if(!uploadedResources.isEmpty()){
- //Remove duplicated resource
- for (String resourceId : uploadedResources) {
- try{
- //Proxy to delete resource
- bmlAppService.clientRemoveResource(userName, resourceId);
- }catch(Exception ie){
- //ignore
- }
- }
- }
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/DataSourceRelateServiceImpl.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/DataSourceRelateServiceImpl.java
deleted file mode 100644
index d607b73..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/DataSourceRelateServiceImpl.java
+++ /dev/null
@@ -1,58 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.service.impl;
-
-import org.apache.linkis.datasourcemanager.core.dao.DataSourceTypeDao;
-import org.apache.linkis.datasourcemanager.core.dao.DataSourceParamKeyDao;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceType;
-import org.apache.linkis.datasourcemanager.core.service.DataSourceRelateService;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Service;
-
-import java.util.List;
-
-@Service
-public class DataSourceRelateServiceImpl implements DataSourceRelateService {
-
- @Autowired
- private DataSourceParamKeyDao paramKeyDao;
-
- @Autowired
- private DataSourceTypeDao dataSourceTypeDao;
-
- @Override
- public List<DataSourceParamKeyDefinition> getKeyDefinitionsByType(Long dataSourceTypeId, DataSourceParamKeyDefinition.Scope scope) {
- return paramKeyDao.listByDataSourceTypeAndScope(dataSourceTypeId, scope);
- }
-
- @Override
- public List<DataSourceParamKeyDefinition> getKeyDefinitionsByType(Long dataSourceTypeId) {
- return paramKeyDao.listByDataSourceType(dataSourceTypeId);
- }
-
- @Override
- public List<DataSourceType> getAllDataSourceTypes() {
- return dataSourceTypeDao.getAllTypes();
- }
-
- @Override
- public DataSourceType getDataSourceType(Long typeId) {
- return dataSourceTypeDao.selectOne(typeId);
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/MetadataOperateServiceImpl.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/MetadataOperateServiceImpl.java
deleted file mode 100644
index e2263b7..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/service/impl/MetadataOperateServiceImpl.java
+++ /dev/null
@@ -1,110 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.service.impl;
-
-import org.apache.linkis.common.exception.ErrorException;
-import org.apache.linkis.common.exception.WarnException;
-import org.apache.linkis.datasourcemanager.core.formdata.FormStreamContent;
-import org.apache.linkis.datasourcemanager.core.service.BmlAppService;
-import org.apache.linkis.datasourcemanager.core.service.MetadataOperateService;
-import org.apache.linkis.metadatamanager.common.protocol.MetadataConnect;
-import org.apache.linkis.metadatamanager.common.protocol.MetadataResponse;
-import org.apache.linkis.rpc.Sender;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Service;
-
-import java.io.InputStream;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
-
-import static org.apache.linkis.datasourcemanager.common.ServiceErrorCode.BML_SERVICE_ERROR;
-import static org.apache.linkis.datasourcemanager.common.ServiceErrorCode.REMOTE_METADATA_SERVICE_ERROR;
-
-@Service
-public class MetadataOperateServiceImpl implements MetadataOperateService {
-
- private static final Logger LOG = LoggerFactory.getLogger(MetadataOperateService.class);
- @Autowired
- private BmlAppService bmlAppService;
- @Override
- public void doRemoteConnect(String mdRemoteServiceName ,
- String operator, Map<String, Object> connectParams) throws WarnException {
- List<String> uploadedResources = new ArrayList<>();
- try{
- connectParams.entrySet().removeIf(entry -> {
- Object paramValue = entry.getValue();
- //Upload stream resource in connection parameters
- if(paramValue instanceof FormStreamContent){
- FormStreamContent streamContent = (FormStreamContent)paramValue;
- String fileName = streamContent.getFileName();
- InputStream inputStream = streamContent.getStream();
- if (null != inputStream){
- try {
- String resourceId = bmlAppService.clientUploadResource(operator,
- fileName, inputStream);
- if(null == resourceId){
- return true;
- }
- uploadedResources.add(resourceId);
- entry.setValue(resourceId);
- } catch (ErrorException e) {
- throw new WarnException(BML_SERVICE_ERROR.getValue(), "Fail to operate file in request[上传文件处理失败]");
- }
- }
- }
- return false;
- });
- LOG.info("Send request to metadata service:[" + mdRemoteServiceName + "] for building a connection");
- //Get a sender
- Sender sender = Sender.getSender(mdRemoteServiceName);
- try {
- Object object = sender.ask(new MetadataConnect(operator, connectParams, ""));
- if (object instanceof MetadataResponse) {
- MetadataResponse response = (MetadataResponse) object;
- if (!response.status()) {
- throw new WarnException(REMOTE_METADATA_SERVICE_ERROR.getValue(),
- "Connection Failed[连接失败], Msg[" + response.data() + "]");
- }
- } else {
- throw new WarnException(REMOTE_METADATA_SERVICE_ERROR.getValue(),
- "Remote Service Error[远端服务出错, 联系运维处理]");
- }
- }catch(Throwable t){
- if(!(t instanceof WarnException)) {
- throw new WarnException(REMOTE_METADATA_SERVICE_ERROR.getValue(),
- "Remote Service Error[远端服务出错, 联系运维处理]");
- }
- throw t;
- }
- }finally{
- if(!uploadedResources.isEmpty()){
- uploadedResources.forEach( resourceId ->{
- try{
- //Proxy to delete resource
- bmlAppService.clientRemoveResource(operator, resourceId);
- }catch(Exception e){
- //ignore
- }
- });
- }
- }
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/DataSourceParameterValidator.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/DataSourceParameterValidator.java
deleted file mode 100644
index ffb725f..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/DataSourceParameterValidator.java
+++ /dev/null
@@ -1,82 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.validate;
-
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-import org.apache.linkis.datasourcemanager.core.validate.strategy.RegExpParameterValidateStrategy;
-import org.apache.linkis.datasourcemanager.core.validate.strategy.TypeParameterValidateStrategy;
-import org.apache.commons.lang.StringUtils;
-import org.springframework.stereotype.Component;
-
-import javax.annotation.PostConstruct;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
-
-@Component
-public class DataSourceParameterValidator implements ParameterValidator {
- @PostConstruct
- public void initToRegister(){
- registerStrategy(new TypeParameterValidateStrategy());
- registerStrategy(new RegExpParameterValidateStrategy());
- }
- /**
- * strategies list
- */
- private List<ParameterValidateStrategy> strategies = new ArrayList<>();
-
- @Override
- public void registerStrategy(ParameterValidateStrategy strategy) {
- strategies.add(strategy);
- }
-
- @Override
- public void validate(List<DataSourceParamKeyDefinition> paramKeyDefinitions,
- Map<String, Object> parameters) throws ParameterValidateException{
- for(DataSourceParamKeyDefinition paramKeyDefinition : paramKeyDefinitions){
- String keyName = paramKeyDefinition.getKey();
- Object keyValue = parameters.get(keyName);
- DataSourceParamKeyDefinition.ValueType valueType = paramKeyDefinition.getValueType();
- if(null == keyValue){
- String defaultValue = paramKeyDefinition.getDefaultValue();
- if(StringUtils.isNotBlank(defaultValue) &&
- valueType == DataSourceParamKeyDefinition.ValueType.SELECT){
- defaultValue = defaultValue.split(",")[0].trim();
- }
- keyValue = defaultValue;
- }
- if(null == keyValue || StringUtils.isBlank(String.valueOf(keyValue))){
- if(paramKeyDefinition.isRequire()) {
- throw new ParameterValidateException("Param Validate Failed[参数校验出错], [the value of key: '"
- + keyName + " cannot be blank']");
- }
- continue;
- }
- for(ParameterValidateStrategy validateStrategy : strategies){
- if(validateStrategy.accept(valueType)) {
- validateStrategy.validate(paramKeyDefinition, keyValue);
- }
- }
- }
- }
-
- @Override
- public List<ParameterValidateStrategy> getStrategies() {
- return strategies;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidateException.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidateException.java
deleted file mode 100644
index 8a7359a..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidateException.java
+++ /dev/null
@@ -1,32 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.validate;
-
-import org.apache.linkis.common.exception.ErrorException;
-
-import static org.apache.linkis.datasourcemanager.common.ServiceErrorCode.PARAM_VALIDATE_ERROR;
-
-public class ParameterValidateException extends ErrorException {
- public ParameterValidateException(String desc) {
- super(PARAM_VALIDATE_ERROR.getValue(), desc);
- }
-
- public ParameterValidateException(String desc, String ip, int port, String serviceKind) {
- super(PARAM_VALIDATE_ERROR.getValue(), desc, ip, port, serviceKind);
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidateStrategy.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidateStrategy.java
deleted file mode 100644
index cd33dcb..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidateStrategy.java
+++ /dev/null
@@ -1,41 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.validate;
-
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-
-/**
- * Parameter validate strategy
- */
-public interface ParameterValidateStrategy {
- /**
- * If accept value type
- * @param valueType validate type
- * @return result
- */
- boolean accept(DataSourceParamKeyDefinition.ValueType valueType);
-
- /**
- * Validate actual value by key definition
- * @param keyDefinition key definition
- * @param actualValue actual value
- * @return new value
- */
- Object validate(DataSourceParamKeyDefinition keyDefinition,
- Object actualValue) throws ParameterValidateException;
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidator.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidator.java
deleted file mode 100644
index f27086f..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/ParameterValidator.java
+++ /dev/null
@@ -1,45 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.validate;
-
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-
-import java.util.List;
-import java.util.Map;
-
-public interface ParameterValidator {
- /**
- * Register validate strategy
- * @param strategy strategy
- */
- void registerStrategy(ParameterValidateStrategy strategy);
-
- /**
- * Validate parameter dictionary
- * @param paramKeyDefinitions definitions
- * @param parameters parameters
- */
- void validate(List<DataSourceParamKeyDefinition> paramKeyDefinitions,
- Map<String, Object> parameters) throws ParameterValidateException;
-
- /**
- * Get all strategies
- * @return
- */
- List<ParameterValidateStrategy> getStrategies();
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/strategy/RegExpParameterValidateStrategy.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/strategy/RegExpParameterValidateStrategy.java
deleted file mode 100644
index 0366873..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/strategy/RegExpParameterValidateStrategy.java
+++ /dev/null
@@ -1,64 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.validate.strategy;
-
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-import org.apache.linkis.datasourcemanager.core.validate.ParameterValidateException;
-import org.apache.linkis.datasourcemanager.core.validate.ParameterValidateStrategy;
-import org.apache.commons.lang.StringUtils;
-
-import java.util.List;
-
-/**
- * RegExpression validate strategy
- */
-public class RegExpParameterValidateStrategy implements ParameterValidateStrategy {
-
-
- @Override
- public boolean accept(DataSourceParamKeyDefinition.ValueType valueType) {
- return valueType == DataSourceParamKeyDefinition.ValueType.EMAIL || valueType == DataSourceParamKeyDefinition.ValueType.TEXT || valueType == DataSourceParamKeyDefinition.ValueType.LIST;
- }
-
- @Override
- public Object validate(DataSourceParamKeyDefinition keyDefinition,
- Object actualValue) throws ParameterValidateException {
- String valueRegex = keyDefinition.getValueRegex();
- if(StringUtils.isNotBlank(valueRegex)){
- if(actualValue instanceof List){
- List valueList = ((List)actualValue);
- for(Object value : valueList){
- match(keyDefinition.getKey(), keyDefinition.getName(), String.valueOf(value), valueRegex);
- }
- }else{
- match(keyDefinition.getKey(), keyDefinition.getName(), String.valueOf(actualValue), valueRegex);
- }
-
- }
- return actualValue;
- }
-
- private void match(String key, String name, String value, String valueRegex) throws ParameterValidateException {
- boolean match = String.valueOf(value).matches(valueRegex);
- if(!match){
- throw new ParameterValidateException("Param Validate Failed[参数校验出错], [the value: '"
- + String.valueOf(value) + "' to key: '"
- + key + "(" + name +")' doesn't match]");
- }
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/strategy/TypeParameterValidateStrategy.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/strategy/TypeParameterValidateStrategy.java
deleted file mode 100644
index 27cda7e..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/validate/strategy/TypeParameterValidateStrategy.java
+++ /dev/null
@@ -1,71 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.validate.strategy;
-
-import org.apache.linkis.datasourcemanager.common.domain.DataSourceParamKeyDefinition;
-import org.apache.linkis.datasourcemanager.core.formdata.FormStreamContent;
-import org.apache.linkis.datasourcemanager.core.validate.ParameterValidateException;
-import org.apache.linkis.datasourcemanager.core.validate.ParameterValidateStrategy;
-import org.apache.linkis.metadatamanager.common.Json;
-
-import java.util.List;
-import java.util.Map;
-
-import static org.apache.linkis.datasourcemanager.core.formdata.CustomMultiPartFormDataTransformer.*;
-
-/**
- * Type validate strategy
- */
-public class TypeParameterValidateStrategy implements ParameterValidateStrategy {
- @Override
- public boolean accept(DataSourceParamKeyDefinition.ValueType valueType) {
- //Accept all value
- return true;
- }
-
- @Override
- public Object validate(DataSourceParamKeyDefinition keyDefinition,
- Object actualValue) throws ParameterValidateException {
- DataSourceParamKeyDefinition.ValueType valueType = keyDefinition.getValueType();
- Class<?> javaType = valueType.getJavaType();
- if(valueType == DataSourceParamKeyDefinition.ValueType.FILE ){
- if(!actualValue.getClass().equals(FormStreamContent.class)){
- throw new ParameterValidateException("Param Validate Failed[参数校验出错], [the value of '"
- + keyDefinition.getKey() + "' must be 'File']");
- }
- return actualValue;
- }
- if(!javaType.isAssignableFrom(actualValue.getClass())){
- try {
- if(javaType.equals(List.class)){
- return Json.fromJson(String.valueOf(actualValue), List.class, String.class);
- }else if(javaType.equals(Map.class)){
- return Json.fromJson(String.valueOf(actualValue), Map.class, String.class, String.class);
- }else if(PrimitiveUtils.isPrimitive(javaType)){
- return PrimitiveUtils.primitiveTypeConverse(actualValue, javaType);
- }
- }catch(Exception e){
- throw new ParameterValidateException("Param Validate Failed[参数校验出错], [type of value: '"
- + actualValue + "' is not '" + javaType.getSimpleName() + "']");
- }
- throw new ParameterValidateException("Param Validate Failed[参数校验出错], [type of value: '"
- + actualValue + "' is not '" + javaType.getSimpleName() + "']");
- }
- return actualValue;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/DataSourceEnvVo.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/DataSourceEnvVo.java
deleted file mode 100644
index 4da875d..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/DataSourceEnvVo.java
+++ /dev/null
@@ -1,51 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.vo;
-
-
-public class DataSourceEnvVo extends PageViewVo{
-
- private String envName;
-
- private Long dataSourceTypeId;
-
- public DataSourceEnvVo(){
-
- }
-
- public DataSourceEnvVo(String envName, Long dataSourceTypeId){
- this.envName = envName;
- this.dataSourceTypeId = dataSourceTypeId;
- }
-
- public String getEnvName() {
- return envName;
- }
-
- public void setEnvName(String envName) {
- this.envName = envName;
- }
-
- public Long getDataSourceTypeId() {
- return dataSourceTypeId;
- }
-
- public void setDataSourceTypeId(Long dataSourceTypeId) {
- this.dataSourceTypeId = dataSourceTypeId;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/DataSourceVo.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/DataSourceVo.java
deleted file mode 100644
index 6f72681..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/DataSourceVo.java
+++ /dev/null
@@ -1,82 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.vo;
-
-
-import org.apache.commons.lang.StringUtils;
-
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.List;
-
-public class DataSourceVo extends PageViewVo{
-
- private String dataSourceName;
-
- private Long dataSourceTypeId;
-
- private List<String> createIdentifyList = new ArrayList<>();
-
- private String createSystem;
-
- public DataSourceVo(){
-
- }
-
- public DataSourceVo(String dataSourceName, Long dataSourceTypeId,
- String createIdentifies, String createSystem){
- this.dataSourceName = dataSourceName;
- this.dataSourceTypeId = dataSourceTypeId;
- if(StringUtils.isNotBlank(createIdentifies)){
- createIdentifyList.addAll(Arrays.asList(createIdentifies.split(",")));
- }
- this.createSystem = createSystem;
- }
- public String getDataSourceName() {
- return dataSourceName;
- }
-
- public void setDataSourceName(String dataSourceName) {
- this.dataSourceName = dataSourceName;
- }
-
- public Long getDataSourceTypeId() {
- return dataSourceTypeId;
- }
-
- public void setDataSourceTypeId(Long dataSourceTypeId) {
- this.dataSourceTypeId = dataSourceTypeId;
- }
-
-
- public String getCreateSystem() {
- return createSystem;
- }
-
- public void setCreateSystem(String createSystem) {
- this.createSystem = createSystem;
- }
-
- public List<String> getCreateIdentifyList() {
- return createIdentifyList;
- }
-
- public void setCreateIdentifyList(List<String> createIdentifyList) {
- this.createIdentifyList = createIdentifyList;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/PageViewVo.java b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/PageViewVo.java
deleted file mode 100644
index 5082861..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/java/org/apache/linkis/datasourcemanager/core/vo/PageViewVo.java
+++ /dev/null
@@ -1,39 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.vo;
-
-public class PageViewVo {
- private int currentPage;
- private int pageSize;
-
- public int getCurrentPage() {
- return currentPage;
- }
-
- public void setCurrentPage(int currentPage) {
- this.currentPage = currentPage;
- }
-
- public int getPageSize() {
- return pageSize;
- }
-
- public void setPageSize(int pageSize) {
- this.pageSize = pageSize;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/application.yml b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/application.yml
deleted file mode 100644
index 3943738..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/application.yml
+++ /dev/null
@@ -1,39 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-# http://www.apache.org/licenses/LICENSE-2.0
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-server:
- port: 8196
-spring:
- application:
- name: linkis-ps-datasourcemanager
-
-eureka:
- client:
- serviceUrl:
- defaultZone: http://${ip}:${port}/eureka/
- instance:
- metadata-map:
- test: linkis
-
-management:
- endpoints:
- web:
- exposure:
- include: refresh,info
-logging:
- config: classpath:log4j2.xml
-
-
-
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/linkis.properties b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/linkis.properties
deleted file mode 100644
index 7760812..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/linkis.properties
+++ /dev/null
@@ -1,33 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-# http://www.apache.org/licenses/LICENSE-2.0
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-#
-wds.linkis.server.mybatis.mapperLocations=classpath:org/apache/linkis/datasourcemanager/core/dao/mapper/*.xml
-wds.linkis.server.mybatis.typeAliasesPackage=org.apache.linkis.datasourcemanager.common.domain
-wds.linkis.server.mybatis.BasePackage=org.apache.linkis.datasourcemanager.core.dao
-wds.linkis.server.restful.scan.packages=org.apache.linkis.datasourcemanager.core.restful
-#sit
-wds.linkis.server.version=v1
-#test
-wds.linkis.test.mode=true
-wds.linkis.test.user=
-wds.linkis.server.mybatis.datasource.url=
-wds.linkis.server.mybatis.datasource.username=
-wds.linkis.server.mybatis.datasource.password=
-#dsm
-wds.linkis.server.dsm.admin.users=
-#bml
-wds.linkis.gateway.ip=
-wds.linkis.gateway.port=
\ No newline at end of file
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/log4j2.xml b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/log4j2.xml
deleted file mode 100644
index 4acff50..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/resources/log4j2.xml
+++ /dev/null
@@ -1,39 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<configuration status="error" monitorInterval="30">
- <appenders>
- <Console name="Console" target="SYSTEM_OUT">
- <ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
- <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%t] %logger{36} %L %M - %msg%xEx%n"/>
- </Console>
- <RollingFile name="RollingFile" fileName="logs/linkis.log"
- filePattern="logs/$${date:yyyy-MM}/linkis-log-%d{yyyy-MM-dd}-%i.log">
- <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%-5level] [%-40t] %c{1.} (%L) [%M] - %msg%xEx%n"/>
- <SizeBasedTriggeringPolicy size="100MB"/>
- <DefaultRolloverStrategy max="20"/>
- </RollingFile>
- </appenders>
- <loggers>
- <root level="INFO">
- <appender-ref ref="RollingFile"/>
- <appender-ref ref="Console"/>
- </root>
- </loggers>
-</configuration>
-
diff --git a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/scala/org/apache/linkis/datasourcemanager/core/receivers/DsmReceiver.scala b/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/scala/org/apache/linkis/datasourcemanager/core/receivers/DsmReceiver.scala
deleted file mode 100644
index 70fe6e7..0000000
--- a/linkis-public-enhancements/linkis-datasource/datasourcemanager/server/src/main/scala/org/apache/linkis/datasourcemanager/core/receivers/DsmReceiver.scala
+++ /dev/null
@@ -1,78 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.datasourcemanager.core.receivers
-
-import java.util
-
-@Component
-class DsmReceiver extends Receiver with Logging{
-
- @Autowired
- private var dataSourceInfoService: DataSourceInfoService = _
-
- @Autowired
- private var dataSourceRelateService: DataSourceRelateService = _
-
- override def receive(message: Any, sender: Sender): Unit = ???
-
- override def receiveAndReply(message: Any, sender: Sender): Any = message match {
- case DsInfoQueryRequest(id, system) =>
- if ( id.toLong > 0 && Some(system).isDefined ) {
- Utils.tryCatch {
- val dataSource: DataSource = dataSourceInfoService.getDataSourceInfo(id.toLong, system)
- if ( null != dataSource ) {
- RestfulApiHelper.decryptPasswordKey(dataSourceRelateService.getKeyDefinitionsByType(dataSource.getDataSourceTypeId),
- dataSource.getConnectParams)
- DsInfoResponse(status = true, dataSource.getDataSourceType.getName,
- dataSource.getConnectParams, dataSource.getCreateUser)
- }else DsInfoResponse(status = true, "", new util.HashMap[String, Object](), "")
- }{
- case e: Exception => logger.error(s"Fail to get data source information, id:$id system:$system", e)
- DsInfoResponse(status = false, "", new util.HashMap[String, Object](), "")
- case t: Throwable => logger.error(s"Fail to get data source information, id:$id system:$system", t)
- DsInfoResponse(status = false, "", new util.HashMap[String, Object](), "")
- }
- } else {
- DsInfoResponse(status = true, "", new util.HashMap[String, Object](), "")
- }
- case _ => new Object()
- }
-
- override def receiveAndReply(message: Any, duration: Duration, sender: Sender): Any = message match {
- case DsInfoQueryRequest(id, system) =>
- if ( id.toLong > 0 && Some(system).isDefined ) {
- Utils.tryCatch {
- val dataSource: DataSource = dataSourceInfoService.getDataSourceInfo(id.toLong, system)
- if ( null != dataSource ) {
- RestfulApiHelper.decryptPasswordKey(dataSourceRelateService.getKeyDefinitionsByType(dataSource.getDataSourceTypeId),
- dataSource.getConnectParams)
- DsInfoResponse(status = true, dataSource.getDataSourceType.getName,
- dataSource.getConnectParams, dataSource.getCreateUser)
- }else DsInfoResponse(status = true, "", new util.HashMap[String, Object](), "")
- }{
- case e: Exception => logger.error(s"Fail to get data source information, id:$id system:$system", e)
- DsInfoResponse(status = false, "", new util.HashMap[String, Object](), "")
- case t: Throwable => logger.error(s"Fail to get data source information, id:$id system:$system", t)
- DsInfoResponse(status = false, "", new util.HashMap[String, Object](), "")
- }
- } else {
- DsInfoResponse(status = true, "", new util.HashMap[String, Object](), "")
- }
- case _ => new Object()
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/pom.xml b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/pom.xml
deleted file mode 100644
index 6a07f00..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/pom.xml
+++ /dev/null
@@ -1,77 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
- <parent>
- <artifactId>linkis</artifactId>
- <groupId>org.apache.linkis</groupId>
- <version>1.0.3</version>
- </parent>
- <modelVersion>4.0.0</modelVersion>
-
- <artifactId>linkis-metadatamanager-common</artifactId>
-
- <properties>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
- </properties>
-
- <dependencies>
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-common</artifactId>
- <version>${linkis.version}</version>
- <exclusions>
- <exclusion>
- <artifactId>asm</artifactId>
- <groupId>org.ow2.asm</groupId>
- </exclusion>
- </exclusions>
- </dependency>
- <!--rpc-->
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-rpc</artifactId>
- <version>${linkis.version}</version>
- </dependency>
- </dependencies>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-deploy-plugin</artifactId>
- </plugin>
-
- <plugin>
- <groupId>net.alchim31.maven</groupId>
- <artifactId>scala-maven-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- </plugin>
- </plugins>
- <resources>
- <resource>
- <directory>src/main/resources</directory>
- </resource>
- </resources>
- <finalName>${project.artifactId}-${project.version}</finalName>
- </build>
-</project>
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/Json.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/Json.java
deleted file mode 100644
index 3840444..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/Json.java
+++ /dev/null
@@ -1,114 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common;
-
-import com.fasterxml.jackson.core.JsonGenerator;
-import com.fasterxml.jackson.core.JsonParser;
-import com.fasterxml.jackson.core.JsonProcessingException;
-import com.fasterxml.jackson.databind.*;
-import org.apache.commons.lang.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.io.BufferedReader;
-import java.io.InputStream;
-import java.io.InputStreamReader;
-import java.util.ArrayList;
-
-/**
- * Json utils
- */
-public class Json {
- private static final String PREFIX = "[";
- private static final String SUFFIX = "]";
- private static final Logger logger = LoggerFactory.getLogger(Json.class);
-
- private static ObjectMapper mapper;
-
- static{
- mapper = new ObjectMapper();
- mapper.configure(JsonParser.Feature.ALLOW_COMMENTS, true);
- mapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true);
- mapper.configure(JsonParser.Feature.ALLOW_SINGLE_QUOTES, true);
- mapper.configure(DeserializationFeature.READ_ENUMS_USING_TO_STRING, true);
- mapper.configure(SerializationFeature.WRITE_ENUMS_USING_TO_STRING, true);
- mapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_CONTROL_CHARS, true);
- //empty beans allowed
- mapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
- //ignore unknown properties
- mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
- //cancel to scape non ascii
- mapper.configure(JsonGenerator.Feature.ESCAPE_NON_ASCII, false);
- }
- private Json(){}
-
- @SuppressWarnings("unchecked")
- public static <T> T fromJson(String json, Class<?> clazz, Class<?>... parameters){
- if(StringUtils.isNotBlank(json)){
- try{
- if(parameters.length > 0){
- return (T)mapper.readValue(json, mapper.getTypeFactory().constructParametricType(clazz, parameters));
- }
- if(json.startsWith(PREFIX)
- && json.endsWith(SUFFIX)){
- JavaType javaType = mapper.getTypeFactory()
- .constructParametricType(ArrayList.class, clazz);
- return mapper.readValue(json, javaType);
- }
- return (T)mapper.readValue(json, clazz);
- } catch (Exception e) {
- logger.info(e.getLocalizedMessage());
- throw new RuntimeException(e);
- }
- }
- return null;
- }
-
- public static <T> T fromJson(InputStream stream, Class<?> clazz, Class<?>... parameters){
- StringBuilder builder = new StringBuilder();
- String jsonStr = null;
- try{
- BufferedReader reader = new BufferedReader(new InputStreamReader(stream, "UTF-8"));
- while((jsonStr = reader.readLine()) != null){
- builder.append(jsonStr);
- }
- reader.close();
- }catch(Exception e){
- logger.info(e.getLocalizedMessage());
- throw new RuntimeException(e);
- }
- return fromJson(builder.toString(), clazz, parameters);
- }
-
- public static String toJson(Object obj, Class<?> model){
- ObjectWriter writer = mapper.writer();
- if(null != obj){
- try{
- if(null != model){
- writer = writer.withView(model);
- }
- return writer.writeValueAsString(obj);
- } catch (JsonProcessingException e) {
- logger.info(e.getLocalizedMessage());
- throw new RuntimeException(e);
- }
- }
- return null;
- }
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/MdmConfiguration.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/MdmConfiguration.java
deleted file mode 100644
index 124054f..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/MdmConfiguration.java
+++ /dev/null
@@ -1,29 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common;
-
-import org.apache.linkis.common.conf.CommonVars;
-
-public class MdmConfiguration {
-
- public static CommonVars<String> METADATA_SERVICE_APPLICATION =
- CommonVars.apply("wds.linkis.server.mdm.service.app.name", "linkis-ps-metadatamanagerservice");
-
- public static CommonVars<String> DATA_SOURCE_SERVICE_APPLICATION =
- CommonVars.apply("wds.linkis.server.dsm.app.name", "linkis-ps-datasourcemanager");
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/CacheConfiguration.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/CacheConfiguration.java
deleted file mode 100644
index 59ab207..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/CacheConfiguration.java
+++ /dev/null
@@ -1,29 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.cache;
-
-import org.apache.linkis.common.conf.CommonVars;
-
-public class CacheConfiguration {
-
- public static CommonVars<Long> CACHE_MAX_SIZE =
- CommonVars.apply("wds.linkis.server.mdm.service.cache.size", 1000L);
-
- public static CommonVars<Long> CACHE_EXPIRE_TIME =
- CommonVars.apply("wds.linkis.server.mdm.service.cache.expire", 600L);
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/CacheManager.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/CacheManager.java
deleted file mode 100644
index 0285438..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/CacheManager.java
+++ /dev/null
@@ -1,43 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.cache;
-
-
-import com.google.common.cache.Cache;
-import com.google.common.cache.CacheLoader;
-import com.google.common.cache.LoadingCache;
-import com.google.common.cache.RemovalListener;
-
-public interface CacheManager {
- /**
- * build simple cache
- * @param cacheId
- * @param removalListener
- * @return
- */
- <V> Cache<String, V> buildCache(String cacheId, RemovalListener<String, V> removalListener);
-
- /**
- * build loading cache
- * @param cacheId
- * @param loader
- * @param removalListener
- * @return
- */
- <V> LoadingCache<String, V> buildCache(String cacheId, CacheLoader<String, V> loader, RemovalListener<String, V> removalListener);
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/ConnCacheManager.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/ConnCacheManager.java
deleted file mode 100644
index 7628588..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/cache/ConnCacheManager.java
+++ /dev/null
@@ -1,57 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.cache;
-
-import com.google.common.cache.*;
-
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.TimeUnit;
-
-public class ConnCacheManager implements CacheManager {
- private ConcurrentHashMap<String, Cache> cacheStore = new ConcurrentHashMap<>();
- private ConnCacheManager(){
-
- }
-
- public static CacheManager custom(){
- return new ConnCacheManager();
- }
-
- @Override
- public <V> Cache<String, V> buildCache(String cacheId, RemovalListener<String, V> removalListener) {
- Cache<String, V> vCache = CacheBuilder.newBuilder()
- .maximumSize(CacheConfiguration.CACHE_MAX_SIZE.getValue())
- .expireAfterWrite(CacheConfiguration.CACHE_EXPIRE_TIME.getValue(), TimeUnit.SECONDS)
- .removalListener(removalListener)
- .build();
- cacheStore.putIfAbsent(cacheId, vCache);
- return vCache;
- }
-
- @Override
- public <V> LoadingCache<String, V> buildCache(String cacheId, CacheLoader<String, V> loader,
- RemovalListener<String, V> removalListener) {
- LoadingCache<String, V> vCache = CacheBuilder.newBuilder()
- .maximumSize(CacheConfiguration.CACHE_MAX_SIZE.getValue())
- .expireAfterWrite(CacheConfiguration.CACHE_EXPIRE_TIME.getValue(), TimeUnit.SECONDS)
- .removalListener(removalListener)
- .build(loader);
- cacheStore.putIfAbsent(cacheId, vCache);
- return vCache;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/domain/MetaColumnInfo.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/domain/MetaColumnInfo.java
deleted file mode 100644
index ac18e70..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/domain/MetaColumnInfo.java
+++ /dev/null
@@ -1,67 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.domain;
-
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-
-import java.io.Serializable;
-
-/**
- * The meta information of field
- */
-@JsonSerialize(include= JsonSerialize.Inclusion.NON_EMPTY)
-@JsonIgnoreProperties(ignoreUnknown = true)
-public class MetaColumnInfo implements Serializable {
- private int index = -1;
- private boolean isPrimaryKey;
- private String name;
- private String type;
-
- public int getIndex() {
- return index;
- }
-
- public void setIndex(int index) {
- this.index = index;
- }
-
- public String getName() {
- return name;
- }
-
- public void setName(String name) {
- this.name = name;
- }
-
- public String getType() {
- return type;
- }
-
- public void setType(String type) {
- this.type = type;
- }
-
- public boolean isPrimaryKey() {
- return isPrimaryKey;
- }
-
- public void setPrimaryKey(boolean primaryKey) {
- isPrimaryKey = primaryKey;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/domain/MetaPartitionInfo.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/domain/MetaPartitionInfo.java
deleted file mode 100644
index 56d83f3..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/domain/MetaPartitionInfo.java
+++ /dev/null
@@ -1,96 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.domain;
-
-import org.codehaus.jackson.annotate.JsonIgnoreProperties;
-import org.codehaus.jackson.map.annotate.JsonSerialize;
-
-import java.io.Serializable;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
-/**
- * The meta information of partition
- */
-@JsonSerialize(include= JsonSerialize.Inclusion.NON_EMPTY)
-@JsonIgnoreProperties(ignoreUnknown = true)
-public class MetaPartitionInfo implements Serializable {
- private List<String> partKeys = new ArrayList<>();
-
- private String name;
- /**
- * Partition tree
- */
- private PartitionNode root;
-
- @JsonSerialize(include= JsonSerialize.Inclusion.NON_EMPTY)
- @JsonIgnoreProperties(ignoreUnknown = true)
- public static class PartitionNode{
- /**
- * Node name
- */
- private String name;
- /**
- * Key: partition value
- * Value: child partition node
- */
- private Map<String, PartitionNode> partitions = new HashMap<>();
-
- public String getName() {
- return name;
- }
-
- public void setName(String name) {
- this.name = name;
- }
-
- public Map<String, PartitionNode> getPartitions() {
- return partitions;
- }
-
- public void setPartitions(Map<String, PartitionNode> partitions) {
- this.partitions = partitions;
- }
- }
-
- public List<String> getPartKeys() {
- return partKeys;
- }
-
- public void setPartKeys(List<String> partKeys) {
- this.partKeys = partKeys;
- }
-
- public String getName() {
- return name;
- }
-
- public void setName(String name) {
- this.name = name;
- }
-
- public PartitionNode getRoot() {
- return root;
- }
-
- public void setRoot(PartitionNode root) {
- this.root = root;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/exception/MetaRuntimeException.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/exception/MetaRuntimeException.java
deleted file mode 100644
index c723455..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/exception/MetaRuntimeException.java
+++ /dev/null
@@ -1,32 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.exception;
-
-import org.apache.linkis.common.exception.WarnException;
-
-public class MetaRuntimeException extends WarnException {
- private static final int ERROR_CODE = 99900;
- public MetaRuntimeException(String desc) {
- super(ERROR_CODE, desc);
- }
-
- public MetaRuntimeException(String desc, String ip, int port, String serviceKind) {
- super(ERROR_CODE, desc, ip, port, serviceKind);
- }
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/AbstractMetaService.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/AbstractMetaService.java
deleted file mode 100644
index 8b0a03f..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/AbstractMetaService.java
+++ /dev/null
@@ -1,214 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.service;
-
-import com.google.common.cache.Cache;
-import org.apache.linkis.common.exception.WarnException;
-import org.apache.linkis.metadatamanager.common.Json;
-import org.apache.linkis.metadatamanager.common.cache.CacheManager;
-import org.apache.linkis.metadatamanager.common.cache.ConnCacheManager;
-import org.apache.linkis.metadatamanager.common.domain.MetaColumnInfo;
-import org.apache.linkis.metadatamanager.common.domain.MetaPartitionInfo;
-import org.apache.linkis.metadatamanager.common.exception.MetaRuntimeException;
-import org.apache.commons.lang.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import javax.annotation.PostConstruct;
-import java.io.Closeable;
-import java.io.IOException;
-import java.nio.charset.StandardCharsets;
-import java.security.MessageDigest;
-import java.util.List;
-import java.util.Map;
-import java.util.function.Function;
-
-public abstract class AbstractMetaService<C extends Closeable> implements MetadataService {
- private static final Logger LOG = LoggerFactory.getLogger(AbstractMetaService.class);
- private static final String CONN_CACHE_REQ = "_STORED";
-
-
- private CacheManager connCacheManager;
-
- /**
- * Caching connections which built by connect parameters requested with
- */
- protected Cache<String, MetadataConnection<C>> reqCache;
-
- @PostConstruct
- public void init(){
- connCacheManager = ConnCacheManager.custom();
- initCache(connCacheManager);
- }
- /**
- * If want to use cache component, you should invoke this in constructor method
- * @param cacheManager
- */
- protected void initCache(CacheManager cacheManager){
- String prefix = this.getClass().getSimpleName();
- reqCache = cacheManager.buildCache(prefix + CONN_CACHE_REQ, notification ->{
- assert notification.getValue() != null;
- close(notification.getValue().getConnection());
- });
- }
-
- @Override
- public abstract MetadataConnection<C> getConnection(String operator, Map<String, Object> params) throws Exception;
-
- @Override
- public List<String> getDatabases(String operator, Map<String, Object> params) {
- return this.getConnAndRun(operator, params, this::queryDatabases);
- }
-
- @Override
- public List<String> getTables(String operator, Map<String, Object> params, String database) {
- return this.getConnAndRun(operator, params, conn -> this.queryTables(conn, database));
- }
-
- @Override
- public Map<String, String> getTableProps(String operator, Map<String, Object> params, String database, String table) {
- return this.getConnAndRun(operator, params, conn -> this.queryTableProps(conn, database, table));
- }
-
- @Override
- public MetaPartitionInfo getPartitions(String operator, Map<String, Object> params, String database, String table) {
- return this.getConnAndRun(operator, params, conn -> this.queryPartitions(conn, database, table));
- }
-
- @Override
- public List<MetaColumnInfo> getColumns(String operator, Map<String, Object> params, String database, String table) {
- return this.getConnAndRun(operator, params, conn -> this.queryColumns(conn, database, table));
- }
-
- /**
- * Get database list by connection
- * @param connection metadata connection
- * @return
- */
- public List<String> queryDatabases(C connection){
- throw new WarnException(-1, "This method is no supported");
- }
-
- /**
- * Get table list by connection and database
- * @param connection metadata connection
- * @param database database
- * @return
- */
- public List<String> queryTables(C connection, String database){
- throw new WarnException(-1, "This method is no supported");
- }
-
- /**
- * Get partitions by connection, database and table
- * @param connection metadata connection
- * @param database database
- * @param table table
- * @return
- */
- public MetaPartitionInfo queryPartitions(C connection, String database, String table){
- throw new WarnException(-1, "This method is no supported");
- }
-
- /**
- * Get columns by connection, database and table
- * @param connection metadata connection
- * @param database database
- * @param table table
- * @return
- */
- public List<MetaColumnInfo> queryColumns(C connection, String database, String table){
- throw new WarnException(-1, "This method is no supported");
- }
-
- /**
- * Get table properties
- * @param connection metadata connection
- * @param database database
- * @param table table
- * @return
- */
- public Map<String, String> queryTableProps(C connection, String database, String table){
- throw new WarnException(-1, "This method is no supported");
- }
-
- protected void close(C connection){
- try {
- connection.close();
- } catch (IOException e) {
- throw new MetaRuntimeException("Fail to close connection[关闭连接失败], [" + e.getMessage() + "]");
- }
- }
-
- protected <R>R getConnAndRun(String operator, Map<String, Object> params, Function<C, R> action) {
- String cacheKey = "";
- try{
- cacheKey = md5String(Json.toJson(params, null), "", 2);
- MetadataConnection<C> connection;
- if(null != reqCache) {
- connection = reqCache.get(cacheKey, () -> getConnection(operator, params));
- }else{
- connection = getConnection(operator, params);
- }
- return run(connection, action);
- }catch(Exception e){
- LOG.error("Error to invoke meta service", e);
- if(StringUtils.isNotBlank(cacheKey)){
- reqCache.invalidate(cacheKey);
- }
- throw new MetaRuntimeException(e.getMessage());
- }
- }
- private <R>R run(MetadataConnection<C> connection, Function<C, R> action){
- if(connection.isLock()){
- connection.getLock().lock();
- try{
- return action.apply(connection.getConnection());
- }finally{
- connection.getLock().unlock();
- }
- }else{
- return action.apply(connection.getConnection());
- }
- }
-
- private String md5String(String source, String salt, int iterator){
- StringBuilder token = new StringBuilder();
- try{
- MessageDigest digest = MessageDigest.getInstance("md5");
- if(StringUtils.isNotEmpty(salt)){
- digest.update(salt.getBytes(StandardCharsets.UTF_8));
- }
- byte[] result = digest.digest(source.getBytes());
- for(int i = 0; i < iterator - 1; i++){
- digest.reset();
- result = digest.digest(result);
- }
- for (byte aResult : result) {
- int temp = aResult & 0xFF;
- if (temp <= 0xF) {
- token.append("0");
- }
- token.append(Integer.toHexString(temp));
- }
- }catch(Exception e){
- throw new RuntimeException(e.getMessage());
- }
- return token.toString();
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/BaseMetadataService.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/BaseMetadataService.java
deleted file mode 100644
index 4f48042..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/BaseMetadataService.java
+++ /dev/null
@@ -1,32 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.service;
-
-
-import java.io.Closeable;
-import java.util.Map;
-
-public interface BaseMetadataService {
-
- /**
- * Get connection
- * @param params connect params
- * @return
- */
- MetadataConnection<? extends Closeable> getConnection(String operator, Map<String, Object> params) throws Exception;
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataConnection.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataConnection.java
deleted file mode 100644
index 21c36d3..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataConnection.java
+++ /dev/null
@@ -1,58 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.service;
-
-import java.util.concurrent.locks.ReentrantLock;
-
-/**
- * Connection for metadata
- */
-public class MetadataConnection<C> {
- /***
- * Avoid complication when different threads calling connection API
- */
- private ReentrantLock lock = new ReentrantLock();
-
- private boolean isLock = true;
-
- private C connection;
-
- public MetadataConnection(C connection){
- this.connection = connection;
- }
-
- public MetadataConnection(C connection, boolean isLock){
- this.connection = connection;
- this.isLock = isLock;
- }
- public ReentrantLock getLock() {
- return lock;
- }
-
- public C getConnection() {
- return connection;
- }
-
- public boolean isLock() {
- return isLock;
- }
-
- public void setLock(boolean lock) {
- isLock = lock;
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataDbService.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataDbService.java
deleted file mode 100644
index 46b463f..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataDbService.java
+++ /dev/null
@@ -1,68 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.service;
-
-import org.apache.linkis.metadatamanager.common.domain.MetaColumnInfo;
-import org.apache.linkis.metadatamanager.common.domain.MetaPartitionInfo;
-
-import java.util.List;
-import java.util.Map;
-
-public interface MetadataDbService extends BaseMetadataService {
-
- /**
- * Get all databases
- * @param params connect params
- * @return
- */
- List<String> getDatabases(String operator, Map<String, Object> params);
-
- /**
- * Get all tables from database specified
- * @param params params
- * @param database database name
- * @return
- */
- List<String> getTables(String operator, Map<String, Object> params, String database);
-
- /**
- * Get table properties from database specified
- * @param params params
- * @param database database name
- * @return
- */
- Map<String, String> getTableProps(String operator, Map<String, Object> params, String database, String table);
- /**
- * Get all partitions from table specified
- * @param params params
- * @param database
- * @param table
- * @return
- */
- MetaPartitionInfo getPartitions(String operator, Map<String, Object> params, String database, String table);
-
- /**
- * Get all field information from table specified
- * @param params
- * @param database
- * @param table
- * @return
- */
- List<MetaColumnInfo> getColumns(String operator, Map<String, Object> params, String database, String table);
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataService.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataService.java
deleted file mode 100644
index 4a7f443..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/java/org/apache/linkis/metadatamanager/common/service/MetadataService.java
+++ /dev/null
@@ -1,21 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.service;
-
-public interface MetadataService extends MetadataDbService {
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataOperateProtocol.scala b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataOperateProtocol.scala
deleted file mode 100644
index 2a78964..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataOperateProtocol.scala
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.protocol
-
-import java.util
-
-trait MetadataOperateProtocol {
-
-}
-
-/**
- * Request to do connect
- * @param version
- * @param params
- */
-case class MetadataConnect(operator: String, params: util.Map[String, Object], version: String) extends MetadataOperateProtocol
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataProtocol.scala b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataProtocol.scala
deleted file mode 100644
index 34d9cb8..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataProtocol.scala
+++ /dev/null
@@ -1,24 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.protocol
-
-trait MetadataProtocol {
-
-}
-
-case class MetadataResponse(status: Boolean, data: String) extends MetadataProtocol
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataQueryProtocol.scala b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataQueryProtocol.scala
deleted file mode 100644
index f9a3d5a..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/protocol/MetadataQueryProtocol.scala
+++ /dev/null
@@ -1,61 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.protocol
-
-import java.util
-
-trait MetadataQueryProtocol {
-
-}
-
-/**
- * Request to get database list
- * @param params
- */
-case class MetaGetDatabases(params: util.Map[String, Object], operator: String) extends MetadataQueryProtocol
-
-/**
- * Request to get table list
- * @param params
- */
-case class MetaGetTables(params: util.Map[String, Object], database: String, operator: String) extends MetadataQueryProtocol
-
-/**
- * Request to get table properties
- * @param params
- * @param database
- * @param table
- */
-case class MetaGetTableProps(params: util.Map[String, Object], database: String, table: String, operator: String) extends MetadataQueryProtocol
-
-/**
- * Request to get partition list
- * @param params
- * @param database
- * @param table
- */
-case class MetaGetPartitions(params: util.Map[String, Object], database: String, table: String, operator: String) extends MetadataQueryProtocol
-
-/**
- * Request to get column list
- * @param params
- * @param database
- * @param table
- */
-case class MetaGetColumns(params: util.Map[String, Object], database: String, table: String, operator: String) extends MetadataQueryProtocol
-
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/receiver/BaseMetaReceiver.scala b/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/receiver/BaseMetaReceiver.scala
deleted file mode 100644
index c767fc4..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/common/src/main/scala/org/apache/linkis/metadatamanager/common/receiver/BaseMetaReceiver.scala
+++ /dev/null
@@ -1,64 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.common.receiver
-
-import org.apache.linkis.common.exception.WarnException
-import org.apache.linkis.common.utils.{Logging, Utils}
-import org.apache.linkis.metadatamanager.common.protocol.{MetaGetColumns, MetaGetDatabases, MetaGetPartitions, MetaGetTableProps, MetaGetTables, MetadataConnect, MetadataResponse}
-import org.apache.linkis.metadatamanager.common.service.MetadataService
-import org.apache.linkis.rpc.{Receiver, Sender}
-import org.apache.linkis.server.BDPJettyServerHelper
-
-import scala.concurrent.duration.Duration
-
-class BaseMetaReceiver extends Receiver with Logging{
- protected var metadataService: MetadataService = _
-
- override def receive(message: Any, sender: Sender): Unit = ???
-
- override def receiveAndReply(message: Any, sender: Sender): Any = invoke(metadataService, message)
-
- override def receiveAndReply(message: Any, duration: Duration, sender: Sender): Any = invoke(metadataService, message)
-
-
- def invoke(service: MetadataService, message: Any): Any = Utils.tryCatch{
- val data = message match {
- case MetaGetDatabases(params, operator) => service.getDatabases(operator, params)
- case MetaGetTableProps(params, database, table, operator) => service.getTableProps(operator, params, database, table)
- case MetaGetTables(params, database, operator) => service.getTables(operator, params, database)
- case MetaGetPartitions(params, database, table, operator) => service.getPartitions(operator, params, database, table)
- case MetaGetColumns(params, database, table, operator) => service.getColumns(operator, params, database, table)
- case MetadataConnect(operator, params, version) =>
- service.getConnection(operator, params)
- //MetadataConnection is not scala class
- null
- case _ => new Object()
- }
- MetadataResponse(status = true, BDPJettyServerHelper.gson.toJson(data))
- }{
- case e:WarnException => val errorMsg = e.getMessage
- info(s"Fail to invoke meta service: [$message],[$errorMsg]")
- MetadataResponse(status = false, errorMsg)
- case t:Throwable =>
- val errorMsg = t.getMessage
- if (message.isInstanceOf[MetadataConnect])
- info(s"Fail to invoke meta service: [$message], [$errorMsg]")
- else error(s"Fail to invoke meta service", t)
- MetadataResponse(status = false, t.getMessage)
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/pom.xml b/linkis-public-enhancements/linkis-datasource/metadatamanager/server/pom.xml
deleted file mode 100644
index ddb2eeb..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/pom.xml
+++ /dev/null
@@ -1,94 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
- <parent>
- <artifactId>linkis</artifactId>
- <groupId>org.apache.linkis</groupId>
- <version>1.0.3</version>
- </parent>
- <modelVersion>4.0.0</modelVersion>
- <artifactId>linkis-metadatamanager-server</artifactId>
-
- <properties>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
- </properties>
-
- <dependencies>
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-module</artifactId>
- <version>${linkis.version}</version>
- <exclusions>
- <exclusion>
- <artifactId>asm</artifactId>
- <groupId>org.ow2.asm</groupId>
- </exclusion>
- </exclusions>
- </dependency>
- <!--Metadata common-->
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-metadatamanager-common</artifactId>
- <version>${linkis.version}</version>
- </dependency>
- <!-- data source manager common dependency-->
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-datasourcemanager-common</artifactId>
- <version>${linkis.version}</version>
- <exclusions>
- <exclusion>
- <artifactId>asm</artifactId>
- <groupId>org.ow2.asm</groupId>
- </exclusion>
- </exclusions>
- </dependency>
- <!--rpc-->
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-rpc</artifactId>
- <version>${linkis.version}</version>
- </dependency>
- </dependencies>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-deploy-plugin</artifactId>
- </plugin>
-
- <plugin>
- <groupId>net.alchim31.maven</groupId>
- <artifactId>scala-maven-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- </plugin>
- </plugins>
- <resources>
- <resource>
- <directory>src/main/resources</directory>
- </resource>
- </resources>
- <finalName>${project.artifactId}-${project.version}</finalName>
- </build>
-</project>
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/restful/MetadataCoreRestful.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/restful/MetadataCoreRestful.java
deleted file mode 100644
index ddbb12c..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/restful/MetadataCoreRestful.java
+++ /dev/null
@@ -1,134 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.server.restful;
-
-import org.apache.linkis.metadatamanager.common.domain.MetaColumnInfo;
-import org.apache.linkis.metadatamanager.common.domain.MetaPartitionInfo;
-import org.apache.linkis.metadatamanager.server.service.MetadataAppService;
-import org.apache.linkis.server.Message;
-import org.apache.commons.lang.StringUtils;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Component;
-
-import javax.ws.rs.*;
-import javax.ws.rs.core.MediaType;
-import javax.ws.rs.core.Response;
-import java.util.List;
-import java.util.Map;
-
-@Path("/metadata")
-@Consumes(MediaType.APPLICATION_JSON)
-@Produces(MediaType.APPLICATION_JSON)
-@Component
-public class MetadataCoreRestful {
-
- @Autowired
- private MetadataAppService metadataAppService;
-
- @GET
- @Path("/dbs/{data_source_id}")
- public Response getDatabases(@PathParam("data_source_id")String dataSourceId,
- @QueryParam("system")String system){
- try{
- if(StringUtils.isBlank(system)){
- return Message.messageToResponse(Message.error("'system' is missing[缺少系统名]"));
- }
- List<String> databases = metadataAppService.getDatabasesByDsId(dataSourceId, system);
- return Message.messageToResponse(Message.ok().data("dbs", databases));
- }catch(Exception e){
- return Message.messageToResponse(
- Message.error("Fail to get database list[获取库信息失败], id:[" + dataSourceId +"], system:[" + system + "]", e));
- }
- }
-
- @GET
- @Path("/tables/{data_source_id}/db/{database}")
- public Response getTables(@PathParam("data_source_id")String dataSourceId,
- @PathParam("database")String database,
- @QueryParam("system")String system){
- try{
- if(StringUtils.isBlank(system)){
- return Message.messageToResponse(Message.error("'system' is missing[缺少系统名]"));
- }
- List<String> tables = metadataAppService.getTablesByDsId(dataSourceId, database, system);
- return Message.messageToResponse(Message.ok().data("tables", tables));
- }catch(Exception e){
- return Message.messageToResponse(
- Message.error("Fail to get table list[获取表信息失败], id:[" + dataSourceId +"]" +
- ", system:[" + system + "], database:[" +database +"]", e));
- }
- }
-
- @GET
- @Path("/props/{data_source_id}/db/{database}/table/{table}")
- public Response getTableProps(@PathParam("data_source_id")String dataSourceId,
- @PathParam("database")String database,
- @PathParam("table") String table,
- @QueryParam("system")String system){
- try{
- if(StringUtils.isBlank(system)){
- return Message.messageToResponse(Message.error("'system' is missing[缺少系统名]"));
- }
- Map<String, String> tableProps = metadataAppService.getTablePropsByDsId(dataSourceId, database, table, system);
- return Message.messageToResponse(Message.ok().data("props", tableProps));
- }catch(Exception e){
- return Message.messageToResponse(
- Message.error("Fail to get table properties[获取表参数信息失败], id:[" + dataSourceId +"]" +
- ", system:[" + system + "], database:[" +database +"], table:[" + table +"]", e));
- }
- }
-
- @GET
- @Path("/partitions/{data_source_id}/db/{database}/table/{table}")
- public Response getPartitions(@PathParam("data_source_id")String dataSourceId,
- @PathParam("database")String database,
- @PathParam("table") String table,
- @QueryParam("system")String system){
- try{
- if(StringUtils.isBlank(system)){
- return Message.messageToResponse(Message.error("'system' is missing[缺少系统名]"));
- }
- MetaPartitionInfo partitionInfo = metadataAppService.getPartitionsByDsId(dataSourceId, database, table, system);
- return Message.messageToResponse(Message.ok().data("props", partitionInfo));
- }catch(Exception e){
- return Message.messageToResponse(
- Message.error("Fail to get partitions[获取表分区信息失败], id:[" + dataSourceId +"]" +
- ", system:[" + system + "], database:[" +database +"], table:[" + table +"]"));
- }
- }
-
- @GET
- @Path("/columns/{data_source_id}/db/{database}/table/{table}")
- public Response getColumns(@PathParam("data_source_id")String dataSourceId,
- @PathParam("database")String database,
- @PathParam("table") String table,
- @QueryParam("system")String system){
- try{
- if(StringUtils.isBlank(system)){
- return Message.messageToResponse(Message.error("'system' is missing[缺少系统名]"));
- }
- List<MetaColumnInfo> columns = metadataAppService.getColumns(dataSourceId, database, table, system);
- return Message.messageToResponse(Message.ok().data("columns", columns));
- }catch(Exception e){
- return Message.messageToResponse(
- Message.error("Fail to get column list[获取表字段信息失败], id:[" + dataSourceId +"]" +
- ", system:[" + system + "], database:[" +database +"], table:[" + table +"]", e));
- }
- }
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/service/MetadataAppService.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/service/MetadataAppService.java
deleted file mode 100644
index df259bc..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/service/MetadataAppService.java
+++ /dev/null
@@ -1,69 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.server.service;
-
-import org.apache.linkis.common.exception.ErrorException;
-import org.apache.linkis.metadatamanager.common.domain.MetaColumnInfo;
-import org.apache.linkis.metadatamanager.common.domain.MetaPartitionInfo;
-
-import java.util.List;
-import java.util.Map;
-
-public interface MetadataAppService {
-
- /**
- * @param dataSourceId data source id
- * @param system system
- * @return
- */
- List<String> getDatabasesByDsId(String dataSourceId, String system) throws ErrorException;
-
- /**
- * @param dataSourceId data source id
- * @param system system
- * @param database database
- * @return
- */
- List<String> getTablesByDsId(String dataSourceId, String database, String system) throws ErrorException;
-
- /**
- * @param dataSourceId data source id
- * @param database database
- * @param table table
- * @param system system
- * @return
- */
- Map<String, String> getTablePropsByDsId(String dataSourceId, String database, String table, String system) throws ErrorException;
- /**
- * @param dataSourceId data source i
- * @param database database
- * @param table table
- * @param system system
- * @return
- */
- MetaPartitionInfo getPartitionsByDsId(String dataSourceId, String database, String table, String system) throws ErrorException;
-
- /**
- * @param dataSourceId data source id
- * @param database database
- * @param table table
- * @param system system
- * @return
- */
- List<MetaColumnInfo> getColumns(String dataSourceId, String database, String table, String system) throws ErrorException;
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/service/impl/MetadataAppServiceImpl.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/service/impl/MetadataAppServiceImpl.java
deleted file mode 100644
index 84915b9..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/java/org/apache/linkis/metadatamanager/server/service/impl/MetadataAppServiceImpl.java
+++ /dev/null
@@ -1,153 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.server.service.impl;
-
-import org.apache.linkis.common.exception.ErrorException;
-import org.apache.linkis.datasourcemanager.common.util.json.Json;
-import org.apache.linkis.datasourcemanager.common.protocol.DsInfoQueryRequest;
-import org.apache.linkis.datasourcemanager.common.protocol.DsInfoResponse;
-import org.apache.linkis.metadatamanager.common.MdmConfiguration;
-import org.apache.linkis.metadatamanager.common.domain.MetaColumnInfo;
-import org.apache.linkis.metadatamanager.common.domain.MetaPartitionInfo;
-import org.apache.linkis.metadatamanager.server.service.MetadataAppService;
-import org.apache.linkis.metadatamanager.common.protocol.*;
-import org.apache.linkis.rpc.Sender;
-import org.apache.commons.lang.StringUtils;
-import org.springframework.stereotype.Service;
-
-import javax.annotation.PostConstruct;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
-@Service
-public class MetadataAppServiceImpl implements MetadataAppService {
- private Sender dataSourceRpcSender;
-
- @PostConstruct
- public void init(){
- dataSourceRpcSender = Sender.getSender(MdmConfiguration.DATA_SOURCE_SERVICE_APPLICATION.getValue());
- }
- @Override
- public List<String> getDatabasesByDsId(String dataSourceId, String system) throws ErrorException {
- DsInfoResponse dsInfoResponse = reqToGetDataSourceInfo(dataSourceId, system);
- if(StringUtils.isNotBlank(dsInfoResponse.dsType())){
- MetadataResponse metaQueryResponse = doAndGetMetaInfo(dsInfoResponse.dsType(),
- new MetaGetDatabases(dsInfoResponse.params(), dsInfoResponse.creator()));
- return Json.fromJson(metaQueryResponse.data(), List.class, String.class);
- }
- return new ArrayList<>();
- }
-
- @Override
- public List<String> getTablesByDsId(String dataSourceId, String database, String system) throws ErrorException {
- DsInfoResponse dsInfoResponse = reqToGetDataSourceInfo(dataSourceId, system);
- if(StringUtils.isNotBlank(dsInfoResponse.dsType())){
- MetadataResponse metaQueryResponse = doAndGetMetaInfo(dsInfoResponse.dsType(),
- new MetaGetTables(dsInfoResponse.params(), database, dsInfoResponse.creator()));
- return Json.fromJson(metaQueryResponse.data(), List.class, String.class);
- }
- return new ArrayList<>();
- }
-
- @Override
- public Map<String, String> getTablePropsByDsId(String dataSourceId, String database, String table, String system) throws ErrorException {
- DsInfoResponse dsInfoResponse = reqToGetDataSourceInfo(dataSourceId, system);
- if(StringUtils.isNotBlank(dsInfoResponse.dsType())){
- MetadataResponse metaQueryResponse = doAndGetMetaInfo(dsInfoResponse.dsType(),
- new MetaGetTableProps(dsInfoResponse.params(), database, table, dsInfoResponse.creator()));
- return Json.fromJson(metaQueryResponse.data(), Map.class, String.class, String.class);
- }
- return new HashMap<>();
- }
-
- @Override
- public MetaPartitionInfo getPartitionsByDsId(String dataSourceId, String database, String table, String system) throws ErrorException {
- DsInfoResponse dsInfoResponse = reqToGetDataSourceInfo(dataSourceId, system);
- if(StringUtils.isNotBlank(dsInfoResponse.dsType())){
- MetadataResponse metaQueryResponse = doAndGetMetaInfo(dsInfoResponse.dsType(),
- new MetaGetPartitions(dsInfoResponse.params(), database, table, dsInfoResponse.creator()));
- return Json.fromJson(metaQueryResponse.data(), MetaPartitionInfo.class);
- }
- return new MetaPartitionInfo();
- }
-
- @Override
- public List<MetaColumnInfo> getColumns(String dataSourceId, String database, String table, String system) throws ErrorException {
- DsInfoResponse dsInfoResponse = reqToGetDataSourceInfo(dataSourceId, system);
- if(StringUtils.isNotBlank(dsInfoResponse.dsType())){
- MetadataResponse metaQueryResponse = doAndGetMetaInfo(dsInfoResponse.dsType(),
- new MetaGetColumns(dsInfoResponse.params(), database, table, dsInfoResponse.creator()));
- return Json.fromJson(metaQueryResponse.data(), List.class, MetaColumnInfo.class);
- }
- return new ArrayList<>();
- }
-
- /**
- * Request to get data source information
- * (type and connection parameters)
- * @param dataSourceId data source id
- * @param system system
- * @return
- * @throws ErrorException
- */
- public DsInfoResponse reqToGetDataSourceInfo(String dataSourceId, String system) throws ErrorException{
- Object rpcResult = null;
- try {
- rpcResult = dataSourceRpcSender.ask(new DsInfoQueryRequest(dataSourceId, system));
- }catch(Exception e){
- throw new ErrorException(-1, "Remote Service Error[远端服务出错, 联系运维处理]");
- }
- if(rpcResult instanceof DsInfoResponse){
- DsInfoResponse response = (DsInfoResponse)rpcResult;
- if(!response.status()){
- throw new ErrorException(-1, "Error in Data Source Manager Server[数据源服务出错]");
- }
- return response;
- }else{
- throw new ErrorException(-1, "Remote Service Error[远端服务出错, 联系运维处理]");
- }
- }
-
- /**
- * Request to get meta information
- * @param dataSourceType
- * @param request
- * @return
- */
- public MetadataResponse doAndGetMetaInfo(String dataSourceType, MetadataQueryProtocol request) throws ErrorException {
- Sender sender = Sender.getSender(MdmConfiguration.METADATA_SERVICE_APPLICATION.getValue() + "-" + dataSourceType.toLowerCase());
- Object rpcResult = null;
- try{
- rpcResult = sender.ask(request);
- }catch(Exception e){
- throw new ErrorException(-1, "Remote Service Error[远端服务出错, 联系运维处理]");
- }
- if(rpcResult instanceof MetadataResponse){
- MetadataResponse response = (MetadataResponse)rpcResult;
- if(!response.status()){
- throw new ErrorException(-1, "Error in ["+dataSourceType.toUpperCase()+"] Metadata Service Server[元数据服务出错], " +
- "[" +response.data() + "]");
- }
- return response;
- }else{
- throw new ErrorException(-1, "Remote Service Error[远端服务出错, 联系运维处理]");
- }
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/application.yml b/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/application.yml
deleted file mode 100644
index a6a4769..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/application.yml
+++ /dev/null
@@ -1,44 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-# http://www.apache.org/licenses/LICENSE-2.0
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-server:
- port: 8296
-spring:
- application:
- name: linkis-ps-metadatamanager
-
-eureka:
- client:
- serviceUrl:
- defaultZone: http://${ip}:${port}/eureka/
- instance:
- metadata-map:
- test: linkis
-
-management:
- endpoints:
- web:
- exposure:
- include: refresh,info
-logging:
- config: classpath:log4j2.xml
-
-
-pagehelper:
- helper-dialect: mysql
- reasonable: true
- support-methods-arguments: true
- params: countSql
-
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/linkis.properties b/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/linkis.properties
deleted file mode 100644
index 9298ce3..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/linkis.properties
+++ /dev/null
@@ -1,25 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-# http://www.apache.org/licenses/LICENSE-2.0
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-#
-wds.linkis.server.mybatis.mapperLocations=
-wds.linkis.server.mybatis.typeAliasesPackage=
-wds.linkis.server.mybatis.BasePackage=
-wds.linkis.server.restful.scan.packages=org.apache.linkis.metadatamanager.server.restful
-#sit
-wds.linkis.server.version=v1
-#test
-wds.linkis.test.mode=true
-wds.linkis.test.user=hadoop
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/log4j2.xml b/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/log4j2.xml
deleted file mode 100644
index 4acff50..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/server/src/main/resources/log4j2.xml
+++ /dev/null
@@ -1,39 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<configuration status="error" monitorInterval="30">
- <appenders>
- <Console name="Console" target="SYSTEM_OUT">
- <ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
- <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%t] %logger{36} %L %M - %msg%xEx%n"/>
- </Console>
- <RollingFile name="RollingFile" fileName="logs/linkis.log"
- filePattern="logs/$${date:yyyy-MM}/linkis-log-%d{yyyy-MM-dd}-%i.log">
- <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%-5level] [%-40t] %c{1.} (%L) [%M] - %msg%xEx%n"/>
- <SizeBasedTriggeringPolicy size="100MB"/>
- <DefaultRolloverStrategy max="20"/>
- </RollingFile>
- </appenders>
- <loggers>
- <root level="INFO">
- <appender-ref ref="RollingFile"/>
- <appender-ref ref="Console"/>
- </root>
- </loggers>
-</configuration>
-
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/pom.xml b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/pom.xml
deleted file mode 100644
index ed6c851..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/pom.xml
+++ /dev/null
@@ -1,81 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
- <parent>
- <artifactId>linkis</artifactId>
- <groupId>org.apache.linkis</groupId>
- <version>1.0.3</version>
- </parent>
- <modelVersion>4.0.0</modelVersion>
-
- <artifactId>linkis-metadatamanager-service-es</artifactId>
- <properties>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
- <es.version>6.7.1</es.version>
- </properties>
-
- <dependencies>
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-metadatamanager-common</artifactId>
- <version>${linkis.version}</version>
- </dependency>
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-module</artifactId>
- <version>${linkis.version}</version>
- <exclusions>
- <exclusion>
- <artifactId>asm</artifactId>
- <groupId>org.ow2.asm</groupId>
- </exclusion>
- </exclusions>
- </dependency>
- <dependency>
- <groupId>org.elasticsearch.client</groupId>
- <artifactId>elasticsearch-rest-client</artifactId>
- <version>${es.version}</version>
- </dependency>
- </dependencies>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-deploy-plugin</artifactId>
- </plugin>
-
- <plugin>
- <groupId>net.alchim31.maven</groupId>
- <artifactId>scala-maven-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- </plugin>
- </plugins>
- <resources>
- <resource>
- <directory>src/main/resources</directory>
- </resource>
- </resources>
- <finalName>${project.artifactId}-${project.version}</finalName>
- </build>
-</project>
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticConnection.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticConnection.java
deleted file mode 100644
index ca65d68..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticConnection.java
+++ /dev/null
@@ -1,140 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service;
-
-import org.apache.linkis.metadatamanager.common.Json;
-import org.apache.commons.lang.StringUtils;
-import org.apache.http.HttpHost;
-import org.apache.http.auth.AuthScope;
-import org.apache.http.auth.UsernamePasswordCredentials;
-import org.apache.http.client.CredentialsProvider;
-import org.apache.http.impl.client.BasicCredentialsProvider;
-import org.apache.http.impl.nio.reactor.IOReactorConfig;
-import org.elasticsearch.client.Request;
-import org.elasticsearch.client.Response;
-import org.elasticsearch.client.RestClient;
-import org.elasticsearch.client.RestClientBuilder;
-
-import java.io.Closeable;
-import java.io.IOException;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-
-public class ElasticConnection implements Closeable {
-
- public static final String DEFAULT_TYPE_NAME = "type";
-
- private static final String DEFAULT_MAPPING_NAME = "mappings";
- private static final String DEFAULT_INDEX_NAME = "index";
- private static final String FIELD_PROPS = "properties";
-
- private RestClient restClient;
-
- public ElasticConnection(String[] endPoints, String username, String password) throws IOException {
- HttpHost[] httpHosts = new HttpHost[endPoints.length];
- for(int i = 0; i < endPoints.length; i++){
- httpHosts[i] = HttpHost.create(endPoints[i]);
- }
- RestClientBuilder restClientBuilder = RestClient.builder(httpHosts);
- CredentialsProvider credentialsProvider = null;
- if(StringUtils.isNotBlank(username) && StringUtils.isNotBlank(password)){
- credentialsProvider = new BasicCredentialsProvider();
- credentialsProvider.setCredentials(AuthScope.ANY,
- new UsernamePasswordCredentials(username, password));
- }
- //set only one thread
- CredentialsProvider finalCredentialsProvider = credentialsProvider;
- restClientBuilder.setHttpClientConfigCallback(httpClientBuilder -> {
- if(null != finalCredentialsProvider){
- httpClientBuilder
- .setDefaultCredentialsProvider(finalCredentialsProvider);
- }
- return httpClientBuilder.setDefaultIOReactorConfig(IOReactorConfig.custom().setIoThreadCount(1).build());
- });
- this.restClient = restClientBuilder.build();
- //Try to test connection
- ping();
- }
-
- public List<String> getAllIndices() throws IOException {
- List<String> indices = new ArrayList<>();
- Request request = new Request("GET", "_cat/indices");
- request.addParameter("format", "JSON");
- Response response = restClient.performRequest(request);
- List<Map<String, Object>> list = Json.fromJson(response.getEntity().getContent(), Map.class);
- list.forEach( v ->{
- String index = String.valueOf(v.getOrDefault(DEFAULT_INDEX_NAME, ""));
- if(StringUtils.isNotBlank(index) && !index.startsWith(".")){
- indices.add(index);
- }
- });
- return indices;
- }
-
- public List<String> getTypes(String index) throws IOException{
- List<String> types = new ArrayList<>();
- Request request = new Request("GET", index +"/_mappings");
- Response response = restClient.performRequest(request);
- Map<String, Map<String, Object>> result =
- Json.fromJson(response.getEntity().getContent(), Map.class);
- Map<String, Object> indexMap = result.get(index);
- Object props = indexMap.get(DEFAULT_MAPPING_NAME);
- if(props instanceof Map){
- Set keySet = ((Map)props).keySet();
- for(Object v : keySet){
- types.add(String.valueOf(v));
- }
- }
- return types;
- }
-
- public Map<Object, Object> getProps(String index, String type) throws IOException{
- Request request = new Request("GET", index + "/_mappings/" + type);
- Response response = restClient.performRequest(request);
- Map<String, Map<String, Object>> result =
- Json.fromJson(response.getEntity().getContent(), Map.class);
- Map mappings = (Map)result.get(index).get("mappings");
- Map propsMap = mappings;
- if(mappings.containsKey(type)){
- Object typeMap = mappings.get(type);
- if(typeMap instanceof Map){
- propsMap = (Map)typeMap;
- }
- }
- Object props = propsMap.get(FIELD_PROPS);
- if(props instanceof Map){
- return (Map)props;
- }
- return null;
- }
- public void ping() throws IOException{
- Response response = restClient.performRequest(new Request("GET", "/"));
- int code = response.getStatusLine().getStatusCode();
- int successCode = 200;
- if(code != successCode){
- throw new RuntimeException("Ping to ElasticSearch ERROR, response code: " + code);
- }
- }
-
- @Override
- public void close() throws IOException {
- this.restClient.close();
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticMetaService.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticMetaService.java
deleted file mode 100644
index 0941aa3..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticMetaService.java
+++ /dev/null
@@ -1,87 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service;
-
-import org.apache.linkis.metadatamanager.common.Json;
-import org.apache.linkis.metadatamanager.common.domain.MetaColumnInfo;
-import org.apache.linkis.metadatamanager.common.service.AbstractMetaService;
-import org.apache.linkis.metadatamanager.common.service.MetadataConnection;
-import org.springframework.stereotype.Component;
-
-import java.util.List;
-import java.util.Map;
-import java.util.stream.Collectors;
-
-@Component
-public class ElasticMetaService extends AbstractMetaService<ElasticConnection> {
- @Override
- public MetadataConnection<ElasticConnection> getConnection(String operator, Map<String, Object> params)
- throws Exception{
- String[] endPoints = new String[]{};
- Object urls = params.get(ElasticParamsMapper.PARAM_ES_URLS.getValue());
- if(!(urls instanceof List)){
- List<String> urlList = Json.fromJson(String.valueOf(urls), List.class, String.class);
- assert urlList != null;
- endPoints = urlList.toArray(endPoints);
- }else{
- endPoints = ((List<String>)urls).toArray(endPoints);
- }
- ElasticConnection conn = new ElasticConnection(endPoints, String.valueOf(params.getOrDefault(ElasticParamsMapper.PARAM_ES_USERNAME.getValue(), "")),
- String.valueOf(params.getOrDefault(ElasticParamsMapper.PARAM_ES_PASSWORD.getValue(), "")));
- return new MetadataConnection<>(conn, false);
- }
-
- @Override
- public List<String> queryDatabases(ElasticConnection connection) {
- //Get indices
- try{
- return connection.getAllIndices();
- }catch (Exception e){
- throw new RuntimeException("Fail to get ElasticSearch indices(获取索引列表失败)", e);
- }
- }
-
- @Override
- public List<String> queryTables(ElasticConnection connection, String database) {
- //Get types
- try{
- return connection.getTypes(database);
- }catch (Exception e){
- throw new RuntimeException("Fail to get ElasticSearch types(获取索引类型失败)", e);
- }
- }
-
- @Override
- public List<MetaColumnInfo> queryColumns(ElasticConnection connection, String database, String table) {
- try {
- Map<Object, Object> props = connection.getProps(database, table);
- return props.entrySet().stream().map(entry -> {
- MetaColumnInfo info = new MetaColumnInfo();
- info.setName(String.valueOf(entry.getKey()));
- Object value = entry.getValue();
- if(value instanceof Map){
- info.setType(String.valueOf(((Map)value)
- .getOrDefault(ElasticConnection.DEFAULT_TYPE_NAME, "")));
- }
- return info;
- }).collect(Collectors.toList());
- } catch (Exception e) {
- throw new RuntimeException("Fail to get ElasticSearch columns(获取索引字段失败)", e);
- }
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticParamsMapper.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticParamsMapper.java
deleted file mode 100644
index f091024..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/java/org/apache/linkis/metadatamanager/service/ElasticParamsMapper.java
+++ /dev/null
@@ -1,34 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service;
-
-import org.apache.linkis.common.conf.CommonVars;
-
-/**
- * Configuration
- */
-public class ElasticParamsMapper {
- public static final CommonVars<String> PARAM_ES_URLS =
- CommonVars.apply("wds.linkis.server.mdm.service.es.urls", "elasticUrls");
-
- public static final CommonVars<String> PARAM_ES_USERNAME =
- CommonVars.apply("wds.linkis.server.mdm.service.es.username", "username");
-
- public static final CommonVars<String> PARAM_ES_PASSWORD =
- CommonVars.apply("wds.linkis.server.mdm.service.es.password", "password");
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/application.yml b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/application.yml
deleted file mode 100644
index 6eacbb9..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/application.yml
+++ /dev/null
@@ -1,38 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-# http://www.apache.org/licenses/LICENSE-2.0
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-server:
- port: 8295
-spring:
- application:
- name: linkis-ps-metadatamanager-elasticsearch
-
-eureka:
- client:
- serviceUrl:
- defaultZone: http://${ip}:${port}/eureka/
- instance:
- metadata-map:
- test: linkis
-
-management:
- endpoints:
- web:
- exposure:
- include: refresh,info
-logging:
- config: classpath:log4j2.xml
-
-
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/linkis.properties b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/linkis.properties
deleted file mode 100644
index 95c63f9..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/linkis.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-# http://www.apache.org/licenses/LICENSE-2.0
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-#
-wds.linkis.server.mybatis.mapperLocations=
-wds.linkis.server.mybatis.typeAliasesPackage=
-wds.linkis.server.mybatis.BasePackage=
-wds.linkis.server.restful.scan.packages=
-#sit
-wds.linkis.server.version=v1
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/log4j2.xml b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/log4j2.xml
deleted file mode 100644
index 4acff50..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/resources/log4j2.xml
+++ /dev/null
@@ -1,39 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<configuration status="error" monitorInterval="30">
- <appenders>
- <Console name="Console" target="SYSTEM_OUT">
- <ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
- <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%t] %logger{36} %L %M - %msg%xEx%n"/>
- </Console>
- <RollingFile name="RollingFile" fileName="logs/linkis.log"
- filePattern="logs/$${date:yyyy-MM}/linkis-log-%d{yyyy-MM-dd}-%i.log">
- <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%-5level] [%-40t] %c{1.} (%L) [%M] - %msg%xEx%n"/>
- <SizeBasedTriggeringPolicy size="100MB"/>
- <DefaultRolloverStrategy max="20"/>
- </RollingFile>
- </appenders>
- <loggers>
- <root level="INFO">
- <appender-ref ref="RollingFile"/>
- <appender-ref ref="Console"/>
- </root>
- </loggers>
-</configuration>
-
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/ElasticReceiver.scala b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/ElasticReceiver.scala
deleted file mode 100644
index d346732..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/elasticsearch/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/ElasticReceiver.scala
+++ /dev/null
@@ -1,33 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service.receiver
-
-import org.apache.linkis.DataWorkCloudApplication
-import org.apache.linkis.metadatamanager.common.receiver.BaseMetaReceiver
-import org.apache.linkis.metadatamanager.common.service.MetadataService
-import org.springframework.stereotype.Component
-
-import javax.annotation.PostConstruct
-
-@Component
-class ElasticReceiver extends BaseMetaReceiver{
- @PostConstruct
- def init(): Unit = {
- metadataService = DataWorkCloudApplication.getApplicationContext.getBean(classOf[MetadataService])
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/pom.xml b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/pom.xml
deleted file mode 100644
index 6157a1b..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/pom.xml
+++ /dev/null
@@ -1,161 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
- <parent>
- <artifactId>linkis</artifactId>
- <groupId>org.apache.linkis</groupId>
- <version>1.0.3</version>
- </parent>
- <modelVersion>4.0.0</modelVersion>
- <artifactId>linkis-metadatamanager-service-hive</artifactId>
-
- <properties>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
- <hive.version>1.2.1</hive.version>
- <hadoop.version>2.7.2</hadoop.version>
- </properties>
-
- <dependencies>
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-metadatamanager-common</artifactId>
- <version>${linkis.version}</version>
- </dependency>
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-module</artifactId>
- <version>${linkis.version}</version>
- </dependency>
- <!--Hive dependencies-->
- <dependency>
- <groupId>org.apache.hadoop</groupId>
- <artifactId>hadoop-client</artifactId>
- <version>${hadoop.version}</version>
- <exclusions>
- <exclusion>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
- </exclusion>
- <exclusion>
- <artifactId>servlet-api</artifactId>
- <groupId>javax.servlet</groupId>
- </exclusion>
- <exclusion>
- <artifactId>guava</artifactId>
- <groupId>com.google.guava</groupId>
- </exclusion>
- </exclusions>
- </dependency>
- <dependency>
- <groupId>org.apache.hive</groupId>
- <artifactId>hive-exec</artifactId>
- <version>${hive.version}</version>
- <exclusions>
- <exclusion>
- <artifactId>slf4j-log4j12</artifactId>
- <groupId>org.slf4j</groupId>
- </exclusion>
- <exclusion>
- <groupId>org.apache.ivy</groupId>
- <artifactId>ivy</artifactId>
- </exclusion>
- <exclusion>
- <artifactId>guava</artifactId>
- <groupId>com.google.guava</groupId>
- </exclusion>
- <exclusion>
- <artifactId>commons-lang3</artifactId>
- <groupId>org.apache.commons</groupId>
- </exclusion>
- </exclusions>
- </dependency>
- <!--bml client-->
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-bml-client</artifactId>
- <version>${linkis.version}</version>
- </dependency>
- </dependencies>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-deploy-plugin</artifactId>
- </plugin>
-
- <plugin>
- <groupId>net.alchim31.maven</groupId>
- <artifactId>scala-maven-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-assembly-plugin</artifactId>
- <version>3.2.0</version>
- <inherited>false</inherited>
- <executions>
- <execution>
- <id>make-assembly</id>
- <phase>package</phase>
- <goals>
- <goal>single</goal>
- </goals>
- <configuration>
- <descriptors>
- <descriptor>src/main/assembly/distribution.xml</descriptor>
- </descriptors>
- </configuration>
- </execution>
- </executions>
- <configuration>
- <skipAssembly>false</skipAssembly>
- <finalName>linkis-mdm-service-hive</finalName>
- <appendAssemblyId>false</appendAssemblyId>
- <attach>false</attach>
- <descriptors>
- <descriptor>src/main/assembly/distribution.xml</descriptor>
- </descriptors>
- </configuration>
- </plugin>
- </plugins>
- <resources>
- <resource>
- <directory>src/main/java</directory>
- <includes>
- <include>**/*.xml</include>
- </includes>
- </resource>
- <resource>
- <directory>src/main/resources</directory>
- <excludes>
- <exclude>**/*.properties</exclude>
- <exclude>**/application.yml</exclude>
- <exclude>**/bootstrap.yml</exclude>
- <exclude>**/log4j2.xml</exclude>
- </excludes>
- </resource>
- </resources>
- <finalName>${project.artifactId}-${project.version}</finalName>
- </build>
-</project>
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveConnection.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveConnection.java
deleted file mode 100644
index 6ba9ced..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveConnection.java
+++ /dev/null
@@ -1,139 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service;
-
-import org.apache.linkis.common.conf.CommonVars;
-import org.apache.commons.lang.StringUtils;
-import org.apache.hadoop.conf.Configuration;
-import org.apache.hadoop.hive.conf.HiveConf;
-import org.apache.hadoop.hive.metastore.IMetaStoreClient;
-import org.apache.hadoop.hive.ql.metadata.Hive;
-import org.apache.hadoop.security.UserGroupInformation;
-
-import java.io.Closeable;
-import java.io.IOException;
-import java.net.URI;
-import java.security.PrivilegedExceptionAction;
-import java.util.concurrent.locks.ReentrantLock;
-
-import static org.apache.hadoop.fs.FileSystem.FS_DEFAULT_NAME_KEY;
-
-public class HiveConnection implements Closeable {
-
- private Hive hiveClient;
-
- private IMetaStoreClient metaStoreClient;
-
- private static final CommonVars<String> KERBEROS_DEFAULT_PRINCIPLE =
- CommonVars.apply("wds.linkis.server.mdm.service.kerberos.principle", "hadoop/_HOST@EXAMPLE.COM");
-
- private static final CommonVars<String> DEFAULT_SERVICE_USER =
- CommonVars.apply("wds.linkis.server.mdm.service.user", "hadoop");
-
- private static final CommonVars<String> KERBEROS_KRB5_CONF_PATH =
- CommonVars.apply("wds.linkis.server.mdm.service.kerberos.krb5.path", "");
-
- static{
- if(StringUtils.isNotBlank(KERBEROS_KRB5_CONF_PATH.getValue())){
- System.setProperty("java.security.krb5.conf", KERBEROS_KRB5_CONF_PATH.getValue());
- }
- }
-
- public HiveConnection(String uris, String principle, String keytabFilePath) throws Exception {
- final HiveConf conf = new HiveConf();
- conf.setVar(HiveConf.ConfVars.METASTOREURIS, uris);
- conf.setVar(HiveConf.ConfVars.METASTORE_USE_THRIFT_SASL, "true");
- conf.setVar(HiveConf.ConfVars.METASTORE_KERBEROS_PRINCIPAL, KERBEROS_DEFAULT_PRINCIPLE.getValue());
- //Disable the cache in FileSystem
- conf.setBoolean(String.format("fs.%s.impl.disable.cache", URI.create(conf.get(FS_DEFAULT_NAME_KEY, "")).getScheme()), true);
- conf.set("hadoop.security.authentication", "kerberos");
- principle = principle.substring(0, principle.indexOf("@"));
- UserGroupInformation ugi = UserGroupInformationWrapper.loginUserFromKeytab(conf,
- principle, keytabFilePath);
- hiveClient = getHive(ugi, conf);
- }
-
- public HiveConnection(String uris) throws Exception{
- final HiveConf conf = new HiveConf();
- conf.setVar(HiveConf.ConfVars.METASTOREURIS, uris);
- //Disable the cache in FileSystem
- conf.setBoolean(String.format("fs.%s.impl.disable.cache", URI.create(conf.get(FS_DEFAULT_NAME_KEY, "")).getScheme()), true);
- //TODO choose an authentication strategy for hive, and then use createProxyUser
- UserGroupInformation ugi = UserGroupInformation.createRemoteUser(DEFAULT_SERVICE_USER.getValue());
- hiveClient = getHive(ugi, conf);
- }
- /**
- * Get Hive client(Hive object)
- * @return hive
- */
- public Hive getClient(){
- return hiveClient;
- }
-
- private Hive getHive(UserGroupInformation ugi, HiveConf conf) throws IOException, InterruptedException {
- return ugi.doAs((PrivilegedExceptionAction<Hive>) () -> {
- Hive hive = Hive.get(conf);
- metaStoreClient = hive.getMSC();
- //To remove thread Local vars
- Hive.set(null);
- return hive;
- });
- }
- @Override
- public void close() throws IOException {
- //Close meta store client
- metaStoreClient.close();
- }
-
- /**
- * Wrapper class of UserGroupInformation
- */
- private static class UserGroupInformationWrapper{
- private static ReentrantLock globalLock = new ReentrantLock();
-
- public static UserGroupInformation loginUserFromKeytab(final Configuration conf, String user, String path) throws Exception {
- globalLock.lock();
- try{
- UserGroupInformation.setConfiguration(conf);
- return UserGroupInformation.loginUserFromKeytabAndReturnUGI(user, path);
- }finally{
- globalLock.unlock();
- }
- }
- public static UserGroupInformation createProxyUser(final Configuration conf, String user) throws Exception{
- globalLock.lock();
- try{
- UserGroupInformation.setLoginUser(null);
- UserGroupInformation.setConfiguration(conf);
- return UserGroupInformation.createProxyUser(user, UserGroupInformation.getLoginUser());
- }finally{
- globalLock.unlock();
- }
- }
-
- public static UserGroupInformation getLoginUser() throws Exception{
- globalLock.lock();
- try{
- UserGroupInformation.setLoginUser(null);
- return UserGroupInformation.getLoginUser();
- }finally{
- globalLock.unlock();
- }
- }
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveMetaService.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveMetaService.java
deleted file mode 100644
index 1d22f0d..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveMetaService.java
+++ /dev/null
@@ -1,197 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service;
-
-import org.apache.linkis.bml.client.BmlClient;
-import org.apache.linkis.bml.client.BmlClientFactory;
-import org.apache.linkis.bml.protocol.BmlDownloadResponse;
-import org.apache.linkis.common.conf.CommonVars;
-import org.apache.linkis.metadatamanager.common.domain.MetaColumnInfo;
-import org.apache.linkis.metadatamanager.common.domain.MetaPartitionInfo;
-import org.apache.linkis.metadatamanager.common.exception.MetaRuntimeException;
-import org.apache.linkis.metadatamanager.common.service.AbstractMetaService;
-import org.apache.linkis.metadatamanager.common.service.MetadataConnection;
-import org.apache.commons.io.IOUtils;
-import org.apache.commons.lang.StringUtils;
-import org.apache.hadoop.hive.metastore.api.FieldSchema;
-import org.apache.hadoop.hive.ql.metadata.HiveException;
-import org.apache.hadoop.hive.ql.metadata.Partition;
-import org.apache.hadoop.hive.ql.metadata.Table;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.core.io.Resource;
-import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
-import org.springframework.stereotype.Component;
-
-import javax.annotation.PostConstruct;
-import java.io.FileOutputStream;
-import java.io.IOException;
-import java.util.*;
-
-@Component
-public class HiveMetaService extends AbstractMetaService<HiveConnection> {
-
- private static final Logger LOG = LoggerFactory.getLogger(HiveMetaService.class);
- private static final CommonVars<String> TMP_FILE_STORE_LOCATION =
- CommonVars.apply("wds.linkis.server.mdm.service.temp.location", "classpath:/tmp");
-
- private BmlClient client;
-
- @PostConstruct
- public void buildClient(){
- client = BmlClientFactory.createBmlClient();
- }
- @Override
- public MetadataConnection<HiveConnection> getConnection(String operator, Map<String, Object> params) throws Exception {
- Resource resource = new PathMatchingResourcePatternResolver().getResource(TMP_FILE_STORE_LOCATION.getValue());
- String uris = String.valueOf(params.getOrDefault(HiveParamsMapper.PARAM_HIVE_URIS.getValue(), ""));
- String principle = String.valueOf(params.getOrDefault(HiveParamsMapper.PARAM_HIVE_PRINCIPLE.getValue(), ""));
- HiveConnection conn = null;
- if(StringUtils.isNotBlank(principle)){
- LOG.info("Try to connect Hive MetaStore in kerberos with principle:[" + principle +"]");
- String keytabResourceId = String.valueOf(params.getOrDefault(HiveParamsMapper.PARAM_HIVE_KEYTAB.getValue(), ""));
- if(StringUtils.isNotBlank(keytabResourceId)){
- LOG.info("Start to download resource id:[" + keytabResourceId +"]");
- String keytabFilePath = resource.getFile().getAbsolutePath() + "/" + UUID.randomUUID().toString().replace("-", "");
- if(!downloadResource(keytabResourceId, operator, keytabFilePath)){
- throw new MetaRuntimeException("Fail to download resource i:[" + keytabResourceId +"]");
- }
- conn = new HiveConnection(uris, principle, keytabFilePath);
- }else{
- throw new MetaRuntimeException("Cannot find the keytab file in connect parameters");
- }
- }else{
- conn = new HiveConnection(uris);
- }
- return new MetadataConnection<>(conn, true);
- }
-
- /**
- * Download resource to path by id
- * @param resourceId resource id
- * @param user user
- * @param absolutePath absolute path
- * @return
- * @throws IOException
- */
- private boolean downloadResource(String resourceId, String user, String absolutePath) throws IOException {
- BmlDownloadResponse downloadResponse = client.downloadResource(user, resourceId, absolutePath);
- if(downloadResponse.isSuccess()){
- IOUtils.copy(downloadResponse.inputStream(), new FileOutputStream(absolutePath));
- return true;
- }
- return false;
- }
-
- @Override
- public List<String> queryDatabases(HiveConnection connection) {
- try {
- return connection.getClient().getAllDatabases();
- } catch (HiveException e) {
- throw new RuntimeException("Fail to get Hive databases(获取数据库列表失败)", e);
- }
- }
-
- @Override
- public List<String> queryTables(HiveConnection connection, String database) {
- try {
- return connection.getClient().getAllTables(database);
- } catch (HiveException e) {
- throw new RuntimeException("Fail to get Hive tables(获取表列表失败)", e);
- }
- }
-
- @Override
- public MetaPartitionInfo queryPartitions(HiveConnection connection, String database, String table) {
- List<Partition> partitions;
- Table rawTable;
- try {
- rawTable = connection.getClient().getTable(database, table);
- partitions = connection.getClient().getPartitions(rawTable);
- } catch (HiveException e) {
- throw new RuntimeException("Fail to get Hive partitions(获取分区信息失败)", e);
- }
- MetaPartitionInfo info = new MetaPartitionInfo();
- List<FieldSchema> partitionKeys = rawTable.getPartitionKeys();
- List<String> partKeys = new ArrayList<>();
- partitionKeys.forEach(e -> partKeys.add(e.getName()));
- info.setPartKeys(partKeys);
- //Static partitions
- Map<String, MetaPartitionInfo.PartitionNode> pMap = new HashMap<>(20);
- MetaPartitionInfo.PartitionNode root = new MetaPartitionInfo.PartitionNode();
- info.setRoot(root);
- long t = System.currentTimeMillis();
- for(Partition p : partitions){
- try {
- List<String> values = p.getValues();
- if(!partitionKeys.isEmpty()){
- String parentNameValue = "";
- for(int i = 0; i < values.size(); i++){
- FieldSchema fieldSchema = partitionKeys.get(i);
- String name = fieldSchema.getName();
- String value = values.get(i);
- String nameValue= name + "=" + value;
- MetaPartitionInfo.PartitionNode node = new MetaPartitionInfo.PartitionNode();
- if(i > 0){
- MetaPartitionInfo.PartitionNode parent = pMap.get(parentNameValue);
- parent.setName(name);
- parent.getPartitions().putIfAbsent(value, node);
- }else{
- root.setName(name);
- root.getPartitions().putIfAbsent(value, node);
- }
- parentNameValue += "/" + nameValue;
- pMap.putIfAbsent(parentNameValue, node);
- }
- }
- }catch(Exception e){
- LOG.warn(e.getMessage(), e);
- }
- }
- return info;
- }
-
- @Override
- public List<MetaColumnInfo> queryColumns(HiveConnection connection, String database, String table) {
- List<MetaColumnInfo> columns = new ArrayList<>();
- Table tb;
- try {
- tb = connection.getClient().getTable(database, table);
- } catch (HiveException e) {
- throw new RuntimeException("Fail to get Hive columns(获得表字段信息失败)", e);
- }
- tb.getFields().forEach( field ->{
- MetaColumnInfo metaColumnInfo = new MetaColumnInfo();
- metaColumnInfo.setIndex(field.getFieldID());
- metaColumnInfo.setName(field.getFieldName());
- metaColumnInfo.setType(field.getFieldObjectInspector().getTypeName());
- columns.add(metaColumnInfo);
- });
- return columns;
- }
-
- @Override
- public Map<String, String> queryTableProps(HiveConnection connection, String database, String table) {
- try {
- Table rawTable = connection.getClient().getTable(database, table);
- return new HashMap<>((Map)rawTable.getMetadata());
- }catch(Exception e){
- throw new RuntimeException("Fail to get Hive table properties(获取表参数信息失败)", e);
- }
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveParamsMapper.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveParamsMapper.java
deleted file mode 100644
index 4a04fe2..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/java/org/apache/linkis/metadatamanager/service/HiveParamsMapper.java
+++ /dev/null
@@ -1,33 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service;
-
-import org.apache.linkis.common.conf.CommonVars;
-
-public class HiveParamsMapper {
-
- public static final CommonVars<String> PARAM_HIVE_PRINCIPLE =
- CommonVars.apply("wds.linkis.server.mdm.service.hive.principle", "principle");
-
- public static final CommonVars<String> PARAM_HIVE_URIS =
- CommonVars.apply("wds.linkis.server.mdm.service.hive.uris", "uris");
-
- public static final CommonVars<String> PARAM_HIVE_KEYTAB =
- CommonVars.apply("wds.linkis.server.mdm.service.hive.keytab", "keytab");
-
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/application.yml b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/application.yml
deleted file mode 100644
index c845f93..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/application.yml
+++ /dev/null
@@ -1,38 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-# http://www.apache.org/licenses/LICENSE-2.0
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-server:
- port: 8293
-spring:
- application:
- name: linkis-ps-metadatamanager-hive
-
-eureka:
- client:
- serviceUrl:
- defaultZone: http://${ip}:${port}/eureka/
- instance:
- metadata-map:
- test: linkis
-
-management:
- endpoints:
- web:
- exposure:
- include: refresh,info
-logging:
- config: classpath:log4j2.xml
-
-
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/linkis.properties b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/linkis.properties
deleted file mode 100644
index c73f777..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/linkis.properties
+++ /dev/null
@@ -1,25 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-# http://www.apache.org/licenses/LICENSE-2.0
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-#
-wds.linkis.server.mybatis.mapperLocations=
-wds.linkis.server.mybatis.typeAliasesPackage=
-wds.linkis.server.mybatis.BasePackage=
-wds.linkis.server.restful.scan.packages=
-#sit
-wds.linkis.server.version=v1
-#bml
-wds.linkis.gateway.ip=
-wds.linkis.gateway.port=
\ No newline at end of file
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/log4j2.xml b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/log4j2.xml
deleted file mode 100644
index 4acff50..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/resources/log4j2.xml
+++ /dev/null
@@ -1,39 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<configuration status="error" monitorInterval="30">
- <appenders>
- <Console name="Console" target="SYSTEM_OUT">
- <ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
- <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%t] %logger{36} %L %M - %msg%xEx%n"/>
- </Console>
- <RollingFile name="RollingFile" fileName="logs/linkis.log"
- filePattern="logs/$${date:yyyy-MM}/linkis-log-%d{yyyy-MM-dd}-%i.log">
- <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%-5level] [%-40t] %c{1.} (%L) [%M] - %msg%xEx%n"/>
- <SizeBasedTriggeringPolicy size="100MB"/>
- <DefaultRolloverStrategy max="20"/>
- </RollingFile>
- </appenders>
- <loggers>
- <root level="INFO">
- <appender-ref ref="RollingFile"/>
- <appender-ref ref="Console"/>
- </root>
- </loggers>
-</configuration>
-
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/HiveReceiver.scala b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/HiveReceiver.scala
deleted file mode 100644
index 6649156..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/hive/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/HiveReceiver.scala
+++ /dev/null
@@ -1,33 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service.receiver
-
-import org.apache.linkis.DataWorkCloudApplication
-import org.apache.linkis.metadatamanager.common.receiver.BaseMetaReceiver
-import org.apache.linkis.metadatamanager.common.service.MetadataService
-import org.springframework.stereotype.Component
-
-import javax.annotation.PostConstruct
-
-@Component
-class HiveReceiver extends BaseMetaReceiver{
- @PostConstruct
- def init(): Unit = {
- metadataService = DataWorkCloudApplication.getApplicationContext.getBean(classOf[MetadataService])
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/pom.xml b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/pom.xml
deleted file mode 100644
index efabb1e..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/pom.xml
+++ /dev/null
@@ -1,86 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
- <parent>
- <artifactId>linkis</artifactId>
- <groupId>org.apache.linkis</groupId>
- <version>1.0.3</version>
- </parent>
- <modelVersion>4.0.0</modelVersion>
-
- <artifactId>linkis-metadatamanager-service-mysql</artifactId>
-
- <properties>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
- <mysql.version>5.1.34</mysql.version>
- </properties>
-
- <dependencies>
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-metadatamanager-common</artifactId>
- <version>${linkis.version}</version>
- </dependency>
- <dependency>
- <groupId>org.apache.linkis</groupId>
- <artifactId>linkis-module</artifactId>
- <version>${linkis.version}</version>
- <exclusions>
- <exclusion>
- <artifactId>asm</artifactId>
- <groupId>org.ow2.asm</groupId>
- </exclusion>
- <exclusion>
- <groupId>mysql</groupId>
- <artifactId>mysql-connector-java</artifactId>
- </exclusion>
- </exclusions>
- </dependency>
- <dependency>
- <groupId>mysql</groupId>
- <artifactId>mysql-connector-java</artifactId>
- <version>${mysql.version}</version>
- </dependency>
- </dependencies>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-deploy-plugin</artifactId>
- </plugin>
-
- <plugin>
- <groupId>net.alchim31.maven</groupId>
- <artifactId>scala-maven-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- </plugin>
- </plugins>
- <resources>
- <resource>
- <directory>src/main/resources</directory>
- </resource>
- </resources>
- <finalName>${project.artifactId}-${project.version}</finalName>
- </build>
-</project>
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlConnection.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlConnection.java
deleted file mode 100644
index 21a5d84..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlConnection.java
+++ /dev/null
@@ -1,210 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service;
-
-import org.apache.linkis.common.conf.CommonVars;
-import org.apache.linkis.metadatamanager.common.domain.MetaColumnInfo;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.io.Closeable;
-import java.io.IOException;
-import java.sql.*;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
-import java.util.stream.Collectors;
-
-public class SqlConnection implements Closeable {
-
- private static final Logger LOG = LoggerFactory.getLogger(SqlConnection.class);
-
- private static final CommonVars<String> SQL_DRIVER_CLASS =
- CommonVars.apply("wds.linkis.server.mdm.service.sql.driver", "com.mysql.jdbc.Driver");
-
- private static final CommonVars<String> SQL_CONNECT_URL =
- CommonVars.apply("wds.linkis.server.mdm.service.sql.url", "jdbc:mysql://%s:%s/%s");
-
- private Connection conn;
-
- private ConnectMessage connectMessage;
-
- public SqlConnection(String host, Integer port,
- String username, String password,
- Map<String, Object> extraParams ) throws ClassNotFoundException, SQLException {
- connectMessage = new ConnectMessage(host, port, username, password, extraParams);
- conn = getDBConnection(connectMessage, "");
- //Try to create statement
- Statement statement = conn.createStatement();
- statement.close();
- }
-
- public List<String> getAllDatabases() throws SQLException {
- java.util.List<java.lang.String> dataBaseName = new ArrayList<>();
- Statement stmt = null;
- ResultSet rs = null;
- try{
- stmt = conn.createStatement();
- rs = stmt.executeQuery("SHOW DATABASES");
- while (rs.next()){
- dataBaseName.add(rs.getString(1));
- }
- } finally {
- closeResource(null, stmt, rs);
- }
- return dataBaseName;
- }
-
- public List<String> getAllTables(String database) throws SQLException {
- List<String> tableNames = new ArrayList<>();
- Statement stmt = null;
- ResultSet rs = null;
- try {
- stmt = conn.createStatement();
- rs = stmt.executeQuery("SHOW TABLES FROM `" + database + "`");
- while (rs.next()) {
- tableNames.add(rs.getString(1));
- }
- return tableNames;
- } finally{
- closeResource(null, stmt, rs);
- }
- }
-
- public List<MetaColumnInfo> getColumns(String database, String table) throws SQLException, ClassNotFoundException {
- List<MetaColumnInfo> columns = new ArrayList<>();
- String columnSql = "SELECT * FROM `" + database +"`.`" + table + "` WHERE 1 = 2";
- PreparedStatement ps = null;
- ResultSet rs = null;
- ResultSetMetaData meta = null;
- try {
- List<String> primaryKeys = getPrimaryKeys(getDBConnection(connectMessage, database), table);
- ps = conn.prepareStatement(columnSql);
- rs = ps.executeQuery();
- meta = rs.getMetaData();
- int columnCount = meta.getColumnCount();
- for (int i = 1; i < columnCount + 1; i++) {
- MetaColumnInfo info = new MetaColumnInfo();
- info.setIndex(i);
- info.setName(meta.getColumnName(i));
- info.setType(meta.getColumnTypeName(i));
- if(primaryKeys.contains(meta.getColumnName(i))){
- info.setPrimaryKey(true);
- }
- columns.add(info);
- }
- }finally {
- closeResource(null, ps, rs);
- }
- return columns;
- }
-
- /**
- * Get primary keys
- * @param connection connection
- * @param table table name
- * @return
- * @throws SQLException
- */
- private List<String> getPrimaryKeys(Connection connection, String table) throws SQLException {
- ResultSet rs = null;
- List<String> primaryKeys = new ArrayList<>();
- try {
- DatabaseMetaData dbMeta = connection.getMetaData();
- rs = dbMeta.getPrimaryKeys(null, null, table);
- while(rs.next()){
- primaryKeys.add(rs.getString("column_name"));
- }
- return primaryKeys;
- }finally{
- if(null != rs){
- closeResource(connection, null, rs);
- }
- }
- }
-
- /**
- * close database resource
- * @param connection connection
- * @param statement statement
- * @param resultSet result set
- */
- private void closeResource(Connection connection, Statement statement, ResultSet resultSet){
- try {
- if(null != resultSet && !resultSet.isClosed()) {
- resultSet.close();
- }
- if(null != statement && !statement.isClosed()){
- statement.close();
- }
- if(null != connection && !connection.isClosed()){
- connection.close();
- }
- }catch (SQLException e){
- LOG.warn("Fail to release resource [" + e.getMessage() +"]", e);
- }
- }
-
- @Override
- public void close() throws IOException {
- closeResource(conn, null, null);
- }
-
- /**
- * @param connectMessage
- * @param database
- * @return
- * @throws ClassNotFoundException
- */
- private Connection getDBConnection(ConnectMessage connectMessage, String database) throws ClassNotFoundException, SQLException {
- String extraParamString = connectMessage.extraParams.entrySet()
- .stream().map(e -> String.join("=", e.getKey(), String.valueOf(e.getValue())))
- .collect(Collectors.joining("&"));
- Class.forName(SQL_DRIVER_CLASS.getValue());
- String url = String.format(SQL_CONNECT_URL.getValue(), connectMessage.host, connectMessage.port, database);
- if(!connectMessage.extraParams.isEmpty()) {
- url += "?" + extraParamString;
- }
- return DriverManager.getConnection(url, connectMessage.username, connectMessage.password);
- }
-
- /**
- * Connect message
- */
- private static class ConnectMessage{
- private String host;
-
- private Integer port;
-
- private String username;
-
- private String password;
-
- private Map<String, Object> extraParams;
-
- public ConnectMessage(String host, Integer port,
- String username, String password,
- Map<String, Object> extraParams){
- this.host = host;
- this.port = port;
- this.username = username;
- this.password = password;
- this.extraParams = extraParams;
- }
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlMetaService.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlMetaService.java
deleted file mode 100644
index 3b1ba0a..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlMetaService.java
+++ /dev/null
@@ -1,79 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service;
-
-import org.apache.linkis.metadatamanager.common.Json;
-import org.apache.linkis.metadatamanager.common.domain.MetaColumnInfo;
-import org.apache.linkis.metadatamanager.common.service.AbstractMetaService;
-import org.apache.linkis.metadatamanager.common.service.MetadataConnection;
-import org.springframework.stereotype.Component;
-
-import java.sql.SQLException;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
-@Component
-public class SqlMetaService extends AbstractMetaService<SqlConnection> {
- @Override
- public MetadataConnection<SqlConnection> getConnection(String operator, Map<String, Object> params) throws Exception {
- String host = String.valueOf(params.getOrDefault(SqlParamsMapper.PARAM_SQL_HOST.getValue(), ""));
- //After deserialize, Integer will be Double, Why?
- Integer port = (Double.valueOf(String.valueOf(params.getOrDefault(SqlParamsMapper.PARAM_SQL_PORT.getValue(), 0)))).intValue();
- String username = String.valueOf(params.getOrDefault(SqlParamsMapper.PARAM_SQL_USERNAME.getValue(), ""));
- String password = String.valueOf(params.getOrDefault(SqlParamsMapper.PARAM_SQL_PASSWORD.getValue(), ""));
- Map<String, Object> extraParams = new HashMap<>();
- Object sqlParamObj = params.get(SqlParamsMapper.PARAM_SQL_EXTRA_PARAMS.getValue());
- if(null != sqlParamObj){
- if(!(sqlParamObj instanceof Map)){
- extraParams = Json.fromJson(String.valueOf(sqlParamObj), Map.class, String.class, Object.class);
- }else{
- extraParams = (Map<String, Object>)sqlParamObj;
- }
- }
- assert extraParams != null;
- return new MetadataConnection<>(new SqlConnection(host, port, username, password, extraParams));
- }
-
- @Override
- public List<String> queryDatabases(SqlConnection connection) {
- try {
- return connection.getAllDatabases();
- } catch (SQLException e) {
- throw new RuntimeException("Fail to get Sql databases(获取数据库列表失败)", e);
- }
- }
-
- @Override
- public List<String> queryTables(SqlConnection connection, String database) {
- try {
- return connection.getAllTables(database);
- } catch (SQLException e) {
- throw new RuntimeException("Fail to get Sql tables(获取表列表失败)", e);
- }
- }
-
- @Override
- public List<MetaColumnInfo> queryColumns(SqlConnection connection, String database, String table) {
- try {
- return connection.getColumns(database, table);
- } catch (SQLException | ClassNotFoundException e) {
- throw new RuntimeException("Fail to get Sql columns(获取字段列表失败)", e);
- }
- }
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlParamsMapper.java b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlParamsMapper.java
deleted file mode 100644
index cf0c282..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/java/org/apache/linkis/metadatamanager/service/SqlParamsMapper.java
+++ /dev/null
@@ -1,38 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service;
-
-import org.apache.linkis.common.conf.CommonVars;
-
-public class SqlParamsMapper {
-
- public static final CommonVars<String> PARAM_SQL_HOST =
- CommonVars.apply("wds.linkis.server.mdm.service.sql.host", "host");
-
- public static final CommonVars<String> PARAM_SQL_PORT =
- CommonVars.apply("wds.linkis.server.mdm.service.sql.port", "port");
-
- public static final CommonVars<String> PARAM_SQL_USERNAME =
- CommonVars.apply("wds.linkis.server.mdm.service.sql.username", "username");
-
- public static final CommonVars<String> PARAM_SQL_PASSWORD =
- CommonVars.apply("wds.linkis.server.mdm.service.sql.password", "password");
-
- public static final CommonVars<String> PARAM_SQL_EXTRA_PARAMS =
- CommonVars.apply("wds.linkis.server.mdm.service.sql.params", "params");
-}
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/application.yml b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/application.yml
deleted file mode 100644
index 44dd117..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/application.yml
+++ /dev/null
@@ -1,38 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-# http://www.apache.org/licenses/LICENSE-2.0
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-server:
- port: 8294
-spring:
- application:
- name: linkis-ps-metadatamanager-mysql
-
-eureka:
- client:
- serviceUrl:
- defaultZone: http://${ip}:${port}/eureka/
- instance:
- metadata-map:
- test: linkis
-
-management:
- endpoints:
- web:
- exposure:
- include: refresh,info
-logging:
- config: classpath:log4j2.xml
-
-
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/linkis.properties b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/linkis.properties
deleted file mode 100644
index 95c63f9..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/linkis.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-# http://www.apache.org/licenses/LICENSE-2.0
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-#
-wds.linkis.server.mybatis.mapperLocations=
-wds.linkis.server.mybatis.typeAliasesPackage=
-wds.linkis.server.mybatis.BasePackage=
-wds.linkis.server.restful.scan.packages=
-#sit
-wds.linkis.server.version=v1
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/log4j2.xml b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/log4j2.xml
deleted file mode 100644
index 4acff50..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/resources/log4j2.xml
+++ /dev/null
@@ -1,39 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<configuration status="error" monitorInterval="30">
- <appenders>
- <Console name="Console" target="SYSTEM_OUT">
- <ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
- <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%t] %logger{36} %L %M - %msg%xEx%n"/>
- </Console>
- <RollingFile name="RollingFile" fileName="logs/linkis.log"
- filePattern="logs/$${date:yyyy-MM}/linkis-log-%d{yyyy-MM-dd}-%i.log">
- <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%-5level] [%-40t] %c{1.} (%L) [%M] - %msg%xEx%n"/>
- <SizeBasedTriggeringPolicy size="100MB"/>
- <DefaultRolloverStrategy max="20"/>
- </RollingFile>
- </appenders>
- <loggers>
- <root level="INFO">
- <appender-ref ref="RollingFile"/>
- <appender-ref ref="Console"/>
- </root>
- </loggers>
-</configuration>
-
diff --git a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/SqlReceiver.scala b/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/SqlReceiver.scala
deleted file mode 100644
index f572793..0000000
--- a/linkis-public-enhancements/linkis-datasource/metadatamanager/service/mysql/src/main/scala/org/apache/linkis/metadatamanager/service/receiver/SqlReceiver.scala
+++ /dev/null
@@ -1,33 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.linkis.metadatamanager.service.receiver
-
-import org.apache.linkis.DataWorkCloudApplication
-import org.apache.linkis.metadatamanager.common.receiver.BaseMetaReceiver
-import org.apache.linkis.metadatamanager.common.service.MetadataService
-import org.springframework.stereotype.Component
-
-import javax.annotation.PostConstruct
-
-@Component
-class SqlReceiver extends BaseMetaReceiver{
- @PostConstruct
- def init(): Unit = {
- metadataService = DataWorkCloudApplication.getApplicationContext.getBean(classOf[MetadataService])
- }
-}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[incubator-linkis] 03/11: add datasource related properties
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git
commit 4554c8a122cb8eff70a1d49a20d5041d55a57cf8
Author: xiaojie19852006 <xi...@163.com>
AuthorDate: Wed Feb 23 14:58:23 2022 +0800
add datasource related properties
---
.../conf/linkis-ps-data-source-manager.properties | 37 ++++++++++++++++++++++
.../conf/linkis-ps-metadatamanager.properties | 21 ++++++++++++
2 files changed, 58 insertions(+)
diff --git a/assembly-combined-package/assembly-combined/conf/linkis-ps-data-source-manager.properties b/assembly-combined-package/assembly-combined/conf/linkis-ps-data-source-manager.properties
new file mode 100644
index 0000000..0d374b0
--- /dev/null
+++ b/assembly-combined-package/assembly-combined/conf/linkis-ps-data-source-manager.properties
@@ -0,0 +1,37 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+# http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+##restful
+wds.linkis.server.restful.scan.packages=org.apache.linkis.datasourcemanager.core.restful
+
+##mybatis
+wds.linkis.server.mybatis.mapperLocations=classpath:org/apache/linkis/datasourcemanager/core/dao/mapper/*.xml
+
+wds.linkis.server.mybatis.typeAliasesPackage=org.apache.linkis.datasourcemanager.common.domain,org.apache.linkis.datasourcemanager.core.vo
+
+wds.linkis.server.mybatis.BasePackage=org.apache.linkis.datasourcemanager.core.dao
+
+
+##hive meta
+hive.meta.url=
+hive.meta.user=
+hive.meta.password=
+wds.linkis.metadata.hive.encode.enabled=false
+
+##Spring
+spring.server.port=9106
+spring.spring.main.allow-bean-definition-overriding=true
+spring.spring.jackson.serialization.FAIL_ON_EMPTY_BEANS=false
+spring.jackson.serialization.FAIL_ON_EMPTY_BEANS=false
diff --git a/assembly-combined-package/assembly-combined/conf/linkis-ps-metadatamanager.properties b/assembly-combined-package/assembly-combined/conf/linkis-ps-metadatamanager.properties
new file mode 100644
index 0000000..1336c86
--- /dev/null
+++ b/assembly-combined-package/assembly-combined/conf/linkis-ps-metadatamanager.properties
@@ -0,0 +1,21 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+# http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+wds.linkis.server.mdm.service.instance.expire-in-seconds=1800
+wds.linkis.server.restful.scan.packages=org.apache.linkis.metadatamanager.server.restful
+
+wds.linkis.server.dsm.app.name=linkis-ps-data-source-manager
+##Spring
+spring.server.port=8399
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[incubator-linkis] 07/11: fix datasource web visit failure
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git
commit f53e04e46a2fad8422172a83de1e9c4997a307c4
Author: xiaojie19852006 <xi...@163.com>
AuthorDate: Wed Feb 23 15:08:51 2022 +0800
fix datasource web visit failure
---
.../core/restful/DataSourceCoreRestfulApi.java | 19 ++++++++-----------
1 file changed, 8 insertions(+), 11 deletions(-)
diff --git a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java
index 6e92828..7cdc008 100644
--- a/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java
+++ b/linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceCoreRestfulApi.java
@@ -17,6 +17,7 @@
package org.apache.linkis.datasourcemanager.core.restful;
+import com.github.pagehelper.PageInfo;
import org.apache.linkis.common.exception.ErrorException;
import org.apache.linkis.datasourcemanager.common.ServiceErrorCode;
import org.apache.linkis.datasourcemanager.common.auth.AuthContext;
@@ -35,7 +36,8 @@ import org.apache.linkis.datasourcemanager.core.vo.DataSourceVo;
import org.apache.linkis.metadatamanager.common.MdmConfiguration;
import org.apache.linkis.server.Message;
import org.apache.linkis.server.security.SecurityFilter;
-
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
@@ -45,18 +47,13 @@ import javax.validation.ConstraintViolation;
import javax.validation.ConstraintViolationException;
import javax.validation.Validator;
import javax.validation.groups.Default;
-
-import com.github.pagehelper.PageInfo;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.*;
@RestController
@RequestMapping(
- value = "/data-source-manager-manager",
+ value = "/data-source-manager",
produces = {"application/json"})
public class DataSourceCoreRestfulApi {
@@ -513,10 +510,10 @@ public class DataSourceCoreRestfulApi {
@RequestMapping(value = "/info", method = RequestMethod.GET)
public Message queryDataSource(
- @RequestParam("system") String createSystem,
- @RequestParam("name") String dataSourceName,
- @RequestParam("typeId") Long dataSourceTypeId,
- @RequestParam("identifies") String identifies,
+ @RequestParam(value = "system", required = false) String createSystem,
+ @RequestParam(value = "name", required = false) String dataSourceName,
+ @RequestParam(value = "typeId", required = false) Long dataSourceTypeId,
+ @RequestParam(value = "identifies", required = false) String identifies,
@RequestParam("currentPage") Integer currentPage,
@RequestParam("pageSize") Integer pageSize,
HttpServletRequest req) {
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org
[incubator-linkis] 02/11: add datasource and metadata manager to assembly package
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev-1.1.0-datasource
in repository https://gitbox.apache.org/repos/asf/incubator-linkis.git
commit acace79d874caf5108f6fe84dfba374d08878dfd
Author: xiaojie19852006 <xi...@163.com>
AuthorDate: Wed Feb 23 14:57:12 2022 +0800
add datasource and metadata manager to assembly package
---
.../src/main/assembly/assembly.xml | 25 ++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/assembly-combined-package/assembly-combined/src/main/assembly/assembly.xml b/assembly-combined-package/assembly-combined/src/main/assembly/assembly.xml
index 69d95b7..9f2d250 100644
--- a/assembly-combined-package/assembly-combined/src/main/assembly/assembly.xml
+++ b/assembly-combined-package/assembly-combined/src/main/assembly/assembly.xml
@@ -338,6 +338,31 @@
</includes>
</fileSet>
+ <!-- datasource manager -->
+ <fileSet>
+ <directory>
+ ../../linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/target/out/lib/
+ </directory>
+ <outputDirectory>
+ lib/linkis-public-enhancements/linkis-ps-data-source-manager
+ </outputDirectory>
+ <includes>
+ <include>*.jar</include>
+ </includes>
+ </fileSet>
+ <!-- metadata manager -->
+ <fileSet>
+ <directory>
+ ../../linkis-public-enhancements/linkis-datasource/linkis-metadata-manager/server/target/out/lib/
+ </directory>
+ <outputDirectory>
+ lib/linkis-public-enhancements/linkis-ps-metadatamanager
+ </outputDirectory>
+ <includes>
+ <include>**/*</include>
+ </includes>
+ </fileSet>
+
</fileSets>
<dependencySets>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@linkis.apache.org
For additional commands, e-mail: commits-help@linkis.apache.org