You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "zhaohehuhu (via GitHub)" <gi...@apache.org> on 2023/02/28 03:30:08 UTC
[GitHub] [dolphinscheduler] zhaohehuhu opened a new pull request, #13642: [Feature][plugin] Support Kyuubi datasource
zhaohehuhu opened a new pull request, #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642
<!--Thanks very much for contributing to Apache DolphinScheduler. Please review https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html before opening a pull request.-->
## Purpose of the pull request
<!--(For example: This pull request adds checkstyle plugin).-->
## Brief change log
<!--*(for example:)*
- *Add maven-checkstyle-plugin to root pom.xml*
-->
## Verify this pull request
<!--*(Please pick either of the following options)*-->
This pull request is code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
<!--*(example:)*
- *Added dolphinscheduler-dao tests for end-to-end.*
- *Added CronUtilsTest to verify the change.*
- *Manually verified the change by testing locally.* -->
(or)
If your pull request contain incompatible change, you should also add it to `docs/docs/en/guide/upgrede/incompatible.md`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1502736946
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [33 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [58.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1502889743
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [33 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [58.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1159335007
##########
dolphinscheduler-bom/pom.xml:
##########
@@ -536,7 +537,17 @@
</exclusion>
</exclusions>
</dependency>
-
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-hive-jdbc-shaded</artifactId>
+ <version>${kyuubi-jdbc.version}</version>
+ <exclusions>
Review Comment:
OK
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/pom.xml:
##########
@@ -0,0 +1,310 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-datasource-kyuubi</artifactId>
+ <packaging>jar</packaging>
+ <name>${project.artifactId}</name>
+ <dependencies>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-spi</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-api</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hbase.thirdparty</groupId>
+ <artifactId>hbase-noop-htrace</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-client</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>javax.servlet</groupId>
+ <artifactId>servlet-api</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.codehaus.jackson</groupId>
+ <artifactId>jackson-jaxrs</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.codehaus.jackson</groupId>
+ <artifactId>jackson-xc</artifactId>
+ </exclusion>
+
+ <exclusion>
+ <groupId>org.fusesource.leveldbjni</groupId>
+ <artifactId>leveldbjni-all</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.zookeeper</groupId>
+ <artifactId>zookeeper</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-mapreduce-client-shuffle</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.sun.jersey</groupId>
+ <artifactId>jersey-client</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.sun.jersey</groupId>
+ <artifactId>jersey-core</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>javax.xml.bind</groupId>
+ <artifactId>jaxb-api</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>log4j</groupId>
+ <artifactId>log4j</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-hive-jdbc-shaded</artifactId>
+ <exclusions>
Review Comment:
Yes, Sir.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1159334508
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/pom.xml:
##########
@@ -0,0 +1,310 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-datasource-kyuubi</artifactId>
+ <packaging>jar</packaging>
+ <name>${project.artifactId}</name>
+ <dependencies>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-spi</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-api</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hbase.thirdparty</groupId>
+ <artifactId>hbase-noop-htrace</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-client</artifactId>
Review Comment:
got it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1493539674
> @zhongjiajie 17 is used by ssh now
In this case, you should rebase the latest code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] fuchanghai commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "fuchanghai (via GitHub)" <gi...@apache.org>.
fuchanghai commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457432477
>
> > @zhaohehuhu You can refer to this #12379 
>
> should be fixed. Please hep check it @fuchanghai
maybe you need to add kyuubi's license in this file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1491397163
@SbloodyS plz help run CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161390871
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/pom.xml:
##########
@@ -0,0 +1,61 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-datasource-kyuubi</artifactId>
+ <packaging>jar</packaging>
+ <name>${project.artifactId}</name>
+ <dependencies>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-spi</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-api</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hbase.thirdparty</groupId>
+ <artifactId>hbase-noop-htrace</artifactId>
Review Comment:
why this dep is required?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161422081
##########
docs/docs/en/guide/datasource/kyuubi.md:
##########
@@ -18,25 +18,17 @@
| Database name | Enter the database name of the KYUUBI connection. |
| Jdbc connection parameters | Parameter settings for KYUUBI connection, in JSON format. |
-## Use HiveServer2 HA ZooKeeper
+```Kerberos Authentication
+just simply configure them in the connection parameters if kerberos authentication is required.
-NOTICE: If Kerberos is disabled, ensure the parameter `hadoop.security.authentication.startup.state` is false, and parameter `java.security.krb5.conf.path` value sets null.
-If **Kerberos** is enabled, needs to set the following parameters in `common.properties`:
+clientKeytab: path of Kerberos keytab file for client authentication
-```conf
-# whether to startup kerberos
-hadoop.security.authentication.startup.state=true
+clientPrincipal: Kerberos principal for client authentication
-# java.security.krb5.conf path
-java.security.krb5.conf.path=/opt/krb5.conf
-
-# login user from keytab username
-login.user.keytab.username=hdfs-mycluster@ESZ.COM
-
-# login user from keytab path
-login.user.keytab.path=/opt/hdfs.headless.keytab
+serverPrincipal: Kerberos principal configured by kyuubi.kinit.principal at the server side. serverPrincipal is available since 1.7.0, for previous versions, use principal instead.
Review Comment:
> `serverPrincipal is available since 1.7.0, for previous versions, use principal instead.`
Please remove this part, since we start from 1.7.0, the legacy message is not needed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1506269064
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [28 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [58.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu closed pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu closed pull request #13642: [Feature][plugin] Support Kyuubi datasource
URL: https://github.com/apache/dolphinscheduler/pull/13642
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457610378
>  loss denpendies
<img width="1268" alt="Screenshot 2023-03-07 at 14 20 59" src="https://user-images.githubusercontent.com/32693629/223337382-5fa1a776-28b7-4c41-b2c6-717d2fb8ff54.png">
I can't see the method as you said
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1501407672
@SbloodyS plz help run CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161422485
##########
docs/docs/zh/guide/datasource/kyuubi.md:
##########
@@ -14,23 +14,14 @@
- 数据库名：输入连接 KYUUBI 的数据库名称
- Jdbc 连接参数：用于 KYUUBI 连接的参数设置，以 JSON 形式填写
-## 使用 HiveServer2 HA Zookeeper
+```Kerberos 验证
+如需Kerberos验证，请直接配置相应参数在jdbc连接参数中
-注意：如果没有开启 kerberos,请保证参数 `hadoop.security.authentication.startup.state` 值为 `false`,
-参数 `java.security.krb5.conf.path` 值为空. 开启了 **kerberos**，则需要在 `common.properties` 配置以下参数
+clientKeytab: 用户客户端验证的keytab文件路径
-```conf
-# whether to startup kerberos
-hadoop.security.authentication.startup.state=true
+clientPrincipal: 用户客户端验证的Kerberos principal
-# java.security.krb5.conf path
-java.security.krb5.conf.path=/opt/krb5.conf
-
-# login user from keytab username
-login.user.keytab.username=hdfs-mycluster@ESZ.COM
-
-# login user from keytab path
-login.user.keytab.path=/opt/hdfs.headless.keytab
+serverPrincipal: 在服务端通过kyuubi.kinit.principal配置的Kerberos principal. serverPrincipal在kyuubi 1.7.0后启用，针对老版本请使用principal
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1504488847
Please retry analysis of this Pull-Request directly on [SonarCloud](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1506247951
Hi All, what should I do next ? @EricGao888 @fuchanghai @Radeity
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1463171937
> > > Agreed with @fuchanghai , we should set 16 for KYUUBI. Please fix it.
> >
> >
> > I got another snowflake datasource that need to contribute to community later. The snowflake datasource can be set to anything you prefer. Could we keep using 16 for Kyuubi? @fuchanghai @ @SbloodyS
>
> @zhaohehuhu That's a bit wired to leave 17 empty. Since you plan to contribute snowflake datasource to the community, what about submitting a PR and setting the snowflake datasource with id 17 first? Once the snowflake PR gets merged, I think it is ok to set kyuubi with id 18.
OK. I'll commit snowflake datasource at the same time. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1455613678
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [11.7% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457625364
> > >  loss denpendies
> >
> >
> > <img alt="Screenshot 2023-03-07 at 14 20 59" width="1268" src="https://user-images.githubusercontent.com/32693629/223337382-5fa1a776-28b7-4c41-b2c6-717d2fb8ff54.png">
> > I can't see the method as you said
>
> sorry，my fault ,my local version is 3.0.x。so ignore it
NP
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1155429209
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/pom.xml:
##########
@@ -0,0 +1,310 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-datasource-kyuubi</artifactId>
+ <packaging>jar</packaging>
+ <name>${project.artifactId}</name>
+ <dependencies>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-spi</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-api</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hbase.thirdparty</groupId>
+ <artifactId>hbase-noop-htrace</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-client</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>javax.servlet</groupId>
+ <artifactId>servlet-api</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.codehaus.jackson</groupId>
+ <artifactId>jackson-jaxrs</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.codehaus.jackson</groupId>
+ <artifactId>jackson-xc</artifactId>
+ </exclusion>
+
+ <exclusion>
+ <groupId>org.fusesource.leveldbjni</groupId>
+ <artifactId>leveldbjni-all</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.zookeeper</groupId>
+ <artifactId>zookeeper</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-mapreduce-client-shuffle</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.sun.jersey</groupId>
+ <artifactId>jersey-client</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>com.sun.jersey</groupId>
+ <artifactId>jersey-core</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>javax.xml.bind</groupId>
+ <artifactId>jaxb-api</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>log4j</groupId>
+ <artifactId>log4j</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-hive-jdbc-shaded</artifactId>
+ <exclusions>
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1154075027
##########
dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/enums/DbType.java:
##########
@@ -43,7 +43,8 @@ public enum DbType {
TRINO(12, "trino"),
STARROCKS(13, "starrocks"),
AZURESQL(14, "azuresql"),
- DAMENG(15, "dameng");
+ DAMENG(15, "dameng"),
+ KYUUBI(18, "kyuubi");
Review Comment:
17 is used by ssh now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161406185
##########
tools/dependencies/known-dependencies.txt:
##########
@@ -431,6 +431,48 @@ azure-storage-file-share-12.16.1.jar
nimbus-jose-jwt-9.22.jar
woodstox-core-6.4.0.jar
azure-core-management-1.9.0.jar
+kyuubi-hive-jdbc-1.7.0.jar
+jcl-over-slf4j-1.7.36.jar
+api-common-2.6.0.jar
+auto-value-1.10.1.jar
+auto-value-annotations-1.10.1.jar
+bcpkix-jdk15on-1.67.jar
+bcprov-jdk15on-1.67.jar
+conscrypt-openjdk-uber-2.5.2.jar
+gapic-google-cloud-storage-v2-2.18.0-alpha.jar
+gax-2.23.0.jar
+gax-grpc-2.23.0.jar
+gax-httpjson-0.108.0.jar
+google-api-client-2.2.0.jar
+google-api-services-storage-v1-rev20220705-2.0.0.jar
+google-auth-library-credentials-1.15.0.jar
+google-auth-library-oauth2-http-1.15.0.jar
+google-cloud-core-2.10.0.jar
+google-cloud-core-grpc-2.10.0.jar
+google-cloud-core-http-2.10.0.jar
+google-cloud-storage-2.18.0.jar
+google-http-client-1.42.3.jar
+google-http-client-apache-v2-1.42.3.jar
+google-http-client-appengine-1.42.3.jar
+google-http-client-gson-1.42.3.jar
+google-http-client-jackson2-1.42.3.jar
+google-oauth-client-1.34.1.jar
+grpc-alts-1.41.0.jar
+grpc-auth-1.41.0.jar
+grpc-google-cloud-storage-v2-2.18.0-alpha.jar
+grpc-googleapis-1.52.1.jar
+grpc-netty-shaded-1.41.0.jar
+grpc-services-1.41.0.jar
+grpc-xds-1.41.0.jar
+opencensus-api-0.31.1.jar
+opencensus-contrib-http-util-0.31.1.jar
+opencensus-proto-0.2.0.jar
+proto-google-cloud-storage-v2-2.18.0-alpha.jar
+proto-google-iam-v1-1.9.0.jar
+re2j-1.6.jar
+threetenbp-1.6.5.jar
+kyuubi-hive-jdbc-1.6.0-incubating.jar
+jcl-over-slf4j-1.7.36.jar
Review Comment:
update kyuubi version, but I keep the others as I'm not sure if these are needed by other people.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu closed pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu closed pull request #13642: [Feature][plugin] Support Kyuubi datasource
URL: https://github.com/apache/dolphinscheduler/pull/13642
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1125961121
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.*;
Review Comment:
got it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-code-scanning[bot] commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "github-code-scanning[bot] (via GitHub)" <gi...@apache.org>.
github-code-scanning[bot] commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1126035011
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF_PATH;
+
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.UserGroupInformationFactory;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import sun.security.krb5.Config;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+
+import java.lang.reflect.Field;
+import java.sql.Connection;
+import java.sql.SQLException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.jdbc.core.JdbcTemplate;
+
+public class KyuubiDataSourceClient extends CommonDataSourceClient {
+
+ private static final Logger logger = LoggerFactory.getLogger(KyuubiDataSourceClient.class);
+
+ public KyuubiDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
+ super(baseConnectionParam, dbType);
+ }
+
+ @Override
+ protected void preInit() {
+ logger.info("PreInit in {}", getClass().getName());
+ }
+
+ @Override
+ protected void initClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
+ logger.info("Create UserGroupInformation.");
+ UserGroupInformationFactory.login(baseConnectionParam.getUser());
+ logger.info("Create ugi success.");
+ this.dataSource = JDBCDataSourceProvider.createOneSessionJdbcDataSource(baseConnectionParam, dbType);
+ this.jdbcTemplate = new JdbcTemplate(dataSource);
+ logger.info("Init {} success.", getClass().getName());
+ }
+
+ @Override
+ protected void checkEnv(BaseConnectionParam baseConnectionParam) {
+ super.checkEnv(baseConnectionParam);
+ checkKerberosEnv();
+ }
+
+ private void checkKerberosEnv() {
+ String krb5File = PropertyUtils.getString(JAVA_SECURITY_KRB5_CONF_PATH);
+ Boolean kerberosStartupState = PropertyUtils.getBoolean(HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE, false);
+ if (kerberosStartupState && StringUtils.isNotBlank(krb5File)) {
+ System.setProperty(JAVA_SECURITY_KRB5_CONF, krb5File);
+ try {
+ Config.refresh();
+ Class<?> kerberosName = Class.forName("org.apache.hadoop.security.authentication.util.KerberosName");
+ Field field = kerberosName.getDeclaredField("defaultRealm");
+ field.setAccessible(true);
+ field.set(null, Config.getInstance().getDefaultRealm());
Review Comment:
## Access to unsupported JDK-internal API
Access to unsupported JDK-internal API 'sun.security.krb5.Config'. (Use com.sun.security.jgss)
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/2702)
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF_PATH;
+
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.UserGroupInformationFactory;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import sun.security.krb5.Config;
Review Comment:
## Access to unsupported JDK-internal API
Access to unsupported JDK-internal API 'sun.security.krb5.Config'. (Use com.sun.security.jgss)
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/2700)
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/param/KyuubiDataSourceProcessor.java:
##########
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi.param;
+
+import org.apache.dolphinscheduler.common.constants.Constants;
+import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
+import org.apache.dolphinscheduler.common.utils.JSONUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.datasource.AbstractDataSourceProcessor;
+import org.apache.dolphinscheduler.plugin.datasource.api.datasource.BaseDataSourceParamDTO;
+import org.apache.dolphinscheduler.plugin.datasource.api.datasource.DataSourceProcessor;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.CommonUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import org.apache.commons.collections4.MapUtils;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+
+import com.google.auto.service.AutoService;
+
+@AutoService(DataSourceProcessor.class)
+public class KyuubiDataSourceProcessor extends AbstractDataSourceProcessor {
+
+ @Override
+ public BaseDataSourceParamDTO castDatasourceParamDTO(String paramJson) {
+ return JSONUtils.parseObject(paramJson, KyuubiDataSourceParamDTO.class);
+ }
+
+ @Override
+ public BaseDataSourceParamDTO createDatasourceParamDTO(String connectionJson) {
+ KyuubiDataSourceParamDTO kyuubiDataSourceParamDTO = new KyuubiDataSourceParamDTO();
+ KyuubiConnectionParam kyuubiConnectionParam = (KyuubiConnectionParam) createConnectionParams(connectionJson);
+ kyuubiDataSourceParamDTO.setDatabase(kyuubiConnectionParam.getDatabase());
+ kyuubiDataSourceParamDTO.setUserName(kyuubiConnectionParam.getUser());
+ kyuubiDataSourceParamDTO.setOther(kyuubiConnectionParam.getOther());
+ kyuubiDataSourceParamDTO.setLoginUserKeytabUsername(kyuubiConnectionParam.getLoginUserKeytabUsername());
+ kyuubiDataSourceParamDTO.setLoginUserKeytabPath(kyuubiConnectionParam.getLoginUserKeytabPath());
+ kyuubiDataSourceParamDTO.setJavaSecurityKrb5Conf(kyuubiConnectionParam.getJavaSecurityKrb5Conf());
+
+ String[] tmpArray = kyuubiConnectionParam.getAddress().split(Constants.DOUBLE_SLASH);
+ StringBuilder hosts = new StringBuilder();
+ String[] hostPortArray = tmpArray[tmpArray.length - 1].split(Constants.COMMA);
+ for (String hostPort : hostPortArray) {
+ hosts.append(hostPort.split(Constants.COLON)[0]).append(Constants.COMMA);
+ }
+ hosts.deleteCharAt(hosts.length() - 1);
+ kyuubiDataSourceParamDTO.setHost(hosts.toString());
+ kyuubiDataSourceParamDTO.setPort(Integer.parseInt(hostPortArray[0].split(Constants.COLON)[1]));
Review Comment:
## Missing catch of NumberFormatException
Potential uncaught 'java.lang.NumberFormatException'.
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/2703)
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF_PATH;
+
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.UserGroupInformationFactory;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import sun.security.krb5.Config;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+
+import java.lang.reflect.Field;
+import java.sql.Connection;
+import java.sql.SQLException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.jdbc.core.JdbcTemplate;
+
+public class KyuubiDataSourceClient extends CommonDataSourceClient {
+
+ private static final Logger logger = LoggerFactory.getLogger(KyuubiDataSourceClient.class);
+
+ public KyuubiDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
+ super(baseConnectionParam, dbType);
+ }
+
+ @Override
+ protected void preInit() {
+ logger.info("PreInit in {}", getClass().getName());
+ }
+
+ @Override
+ protected void initClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
+ logger.info("Create UserGroupInformation.");
+ UserGroupInformationFactory.login(baseConnectionParam.getUser());
+ logger.info("Create ugi success.");
+ this.dataSource = JDBCDataSourceProvider.createOneSessionJdbcDataSource(baseConnectionParam, dbType);
+ this.jdbcTemplate = new JdbcTemplate(dataSource);
+ logger.info("Init {} success.", getClass().getName());
+ }
+
+ @Override
+ protected void checkEnv(BaseConnectionParam baseConnectionParam) {
+ super.checkEnv(baseConnectionParam);
+ checkKerberosEnv();
+ }
+
+ private void checkKerberosEnv() {
+ String krb5File = PropertyUtils.getString(JAVA_SECURITY_KRB5_CONF_PATH);
+ Boolean kerberosStartupState = PropertyUtils.getBoolean(HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE, false);
+ if (kerberosStartupState && StringUtils.isNotBlank(krb5File)) {
+ System.setProperty(JAVA_SECURITY_KRB5_CONF, krb5File);
+ try {
+ Config.refresh();
Review Comment:
## Access to unsupported JDK-internal API
Access to unsupported JDK-internal API 'sun.security.krb5.Config'. (Use com.sun.security.jgss)
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/2701)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1471366928
> > > > > > > Agreed with @fuchanghai , we should set 16 for KYUUBI. Please fix it.
> > > > > >
> > > > > >
> > > > > > I got another snowflake datasource that need to contribute to community later. The snowflake datasource can be set to anything you prefer. Could we keep using 16 for Kyuubi? @fuchanghai @ @SbloodyS
> > > > >
> > > > >
> > > > > @zhaohehuhu That's a bit wired to leave 17 empty. Since you plan to contribute snowflake datasource to the community, what about submitting a PR and setting the snowflake datasource with id 17 first? Once the snowflake PR gets merged, I think it is ok to set kyuubi with id 18.
> > > >
> > > >
> > > > OK. I'll commit snowflake datasource at the same time. Thanks.
> > >
> > >
> > > @EricGao888 @SbloodyS already requested #13729. Is it ok to set kyuubi to 18 now. Thanks.
> >
> >
> > Hi All, could you guide me for the work left ? @EricGao888 @SbloodyS @fuchanghai
>
> Could u plz resolve the conflicts? 
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1477337492
Hi all, I added a few UT inside. Please help check if I need to do something else.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1147178313
##########
dolphinscheduler-bom/pom.xml:
##########
@@ -67,6 +67,7 @@
<guava.version>31.1-jre</guava.version>
<postgresql.version>42.4.1</postgresql.version>
<hive-jdbc.version>2.3.9</hive-jdbc.version>
+ <kyuubi-jdbc.version>1.6.0-incubating</kyuubi-jdbc.version>
Review Comment:
Kyuubi 1.7.0 is out a few weeks ago
```suggestion
<kyuubi-jdbc.version>1.7.0</kyuubi-jdbc.version>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1147177959
##########
dolphinscheduler-bom/pom.xml:
##########
@@ -535,7 +536,17 @@
</exclusion>
</exclusions>
</dependency>
-
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-hive-jdbc</artifactId>
Review Comment:
The shaded JDBC client is recommended
```suggestion
<artifactId>kyuubi-hive-jdbc-shaded</artifactId>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161391855
##########
tools/dependencies/known-dependencies.txt:
##########
@@ -431,6 +431,48 @@ azure-storage-file-share-12.16.1.jar
nimbus-jose-jwt-9.22.jar
woodstox-core-6.4.0.jar
azure-core-management-1.9.0.jar
+kyuubi-hive-jdbc-1.7.0.jar
+jcl-over-slf4j-1.7.36.jar
+api-common-2.6.0.jar
+auto-value-1.10.1.jar
+auto-value-annotations-1.10.1.jar
+bcpkix-jdk15on-1.67.jar
+bcprov-jdk15on-1.67.jar
+conscrypt-openjdk-uber-2.5.2.jar
+gapic-google-cloud-storage-v2-2.18.0-alpha.jar
+gax-2.23.0.jar
+gax-grpc-2.23.0.jar
+gax-httpjson-0.108.0.jar
+google-api-client-2.2.0.jar
+google-api-services-storage-v1-rev20220705-2.0.0.jar
+google-auth-library-credentials-1.15.0.jar
+google-auth-library-oauth2-http-1.15.0.jar
+google-cloud-core-2.10.0.jar
+google-cloud-core-grpc-2.10.0.jar
+google-cloud-core-http-2.10.0.jar
+google-cloud-storage-2.18.0.jar
+google-http-client-1.42.3.jar
+google-http-client-apache-v2-1.42.3.jar
+google-http-client-appengine-1.42.3.jar
+google-http-client-gson-1.42.3.jar
+google-http-client-jackson2-1.42.3.jar
+google-oauth-client-1.34.1.jar
+grpc-alts-1.41.0.jar
+grpc-auth-1.41.0.jar
+grpc-google-cloud-storage-v2-2.18.0-alpha.jar
+grpc-googleapis-1.52.1.jar
+grpc-netty-shaded-1.41.0.jar
+grpc-services-1.41.0.jar
+grpc-xds-1.41.0.jar
+opencensus-api-0.31.1.jar
+opencensus-contrib-http-util-0.31.1.jar
+opencensus-proto-0.2.0.jar
+proto-google-cloud-storage-v2-2.18.0-alpha.jar
+proto-google-iam-v1-1.9.0.jar
+re2j-1.6.jar
+threetenbp-1.6.5.jar
+kyuubi-hive-jdbc-1.6.0-incubating.jar
+jcl-over-slf4j-1.7.36.jar
Review Comment:
outdated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1501406968
> Thanks @zhaohehuhu, LGTM on my side
Thanks. Buddy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1501415471
> @SbloodyS plz help run CI
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1502737238
Please retry analysis of this Pull-Request directly on [SonarCloud](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457579415
> @zhaohehuhu plz reslove the conflicts
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] fuchanghai commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "fuchanghai (via GitHub)" <gi...@apache.org>.
fuchanghai commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457622076
> >  loss denpendies
>
> <img alt="Screenshot 2023-03-07 at 14 20 59" width="1268" src="https://user-images.githubusercontent.com/32693629/223337382-5fa1a776-28b7-4c41-b2c6-717d2fb8ff54.png">
>
> I can't see the method as you said
sorry，my fault ,my local version is 3.0.x。so ignore it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Radeity commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "Radeity (via GitHub)" <gi...@apache.org>.
Radeity commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1127561719
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF_PATH;
+
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.UserGroupInformationFactory;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import sun.security.krb5.Config;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+
+import java.lang.reflect.Field;
+import java.sql.Connection;
+import java.sql.SQLException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.jdbc.core.JdbcTemplate;
+
+public class KyuubiDataSourceClient extends CommonDataSourceClient {
+
+ private static final Logger logger = LoggerFactory.getLogger(KyuubiDataSourceClient.class);
Review Comment:
BTW, can we directly reuse `HiveDataSourceClient` and `UserGroupInformationFactory` in ds hive plugin, instead of creating new one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1473056420
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [13.5% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1455615408
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [11.7% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1455525047
> where did you use this parameter `datawarehouse` ?
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457600545
> getDatasourceProcessor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457642816
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [11.7% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1504810114
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [28 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [58.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161420218
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/test/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/param/KyuubiDataSourceProcessorTest.java:
##########
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi.param;
+
+import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
+import org.apache.dolphinscheduler.common.utils.JSONUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.CommonUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.DataSourceUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
+import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+import org.mockito.junit.jupiter.MockitoExtension;
+
+@ExtendWith(MockitoExtension.class)
+public class KyuubiDataSourceProcessorTest {
+
+ private KyuubiDataSourceProcessor kyuubiDatasourceProcessor = new KyuubiDataSourceProcessor();
+ @Test
+ public void testCheckDatasourceParam() {
+ KyuubiDataSourceParamDTO kyuubiDatasourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDatasourceParamDTO.setHost("localhost");
+ kyuubiDatasourceParamDTO.setDatabase("default");
+ Map<String, String> other = new HashMap<>();
+ other.put("serverTimezone", "Asia/Shanghai");
+ kyuubiDatasourceParamDTO.setOther(other);
+ DataSourceUtils.checkDatasourceParam(kyuubiDatasourceParamDTO);
+ Assertions.assertTrue(true);
+ }
+
+ @Test
+ public void testBuildConnectionParams() {
+ KyuubiDataSourceParamDTO kyuubiDataSourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDataSourceParamDTO.setHost("localhost");
+ kyuubiDataSourceParamDTO.setDatabase("default");
+ kyuubiDataSourceParamDTO.setUserName("root");
+ kyuubiDataSourceParamDTO.setPort(3306);
+ kyuubiDataSourceParamDTO.setPassword("123456");
+
+ try (
+ MockedStatic<PasswordUtils> mockedStaticPasswordUtils = Mockito.mockStatic(PasswordUtils.class);
+ MockedStatic<CommonUtils> mockedStaticCommonUtils = Mockito.mockStatic(CommonUtils.class)) {
+ mockedStaticPasswordUtils.when(() -> PasswordUtils.encodePassword(Mockito.anyString()))
+ .thenReturn("123456");
+ mockedStaticCommonUtils.when(CommonUtils::getKerberosStartupState).thenReturn(false);
+ ConnectionParam connectionParam = DataSourceUtils.buildConnectionParams(kyuubiDataSourceParamDTO);
+ Assertions.assertNotNull(connectionParam);
+ }
+ }
+
+ @Test
+ public void testBuildConnectionParams2() {
+ KyuubiDataSourceParamDTO kyuubiDataSourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDataSourceParamDTO.setHost("localhost");
+ kyuubiDataSourceParamDTO.setDatabase("default");
+ kyuubiDataSourceParamDTO.setUserName("root");
+ kyuubiDataSourceParamDTO.setPort(3306);
+ kyuubiDataSourceParamDTO.setPassword("123456");
+ ConnectionParam connectionParam =
+ DataSourceUtils.buildConnectionParams(DbType.KYUUBI, JSONUtils.toJsonString(kyuubiDataSourceParamDTO));
+ Assertions.assertNotNull(connectionParam);
+ }
+ @Test
+ public void testCreateConnectionParams() {
+ Map<String, String> props = new HashMap<>();
+ props.put("serverTimezone", "utc");
+ KyuubiDataSourceParamDTO kyuubiDataSourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDataSourceParamDTO.setHost("localhost1,localhost2");
+ kyuubiDataSourceParamDTO.setPort(5142);
+ kyuubiDataSourceParamDTO.setUserName("default");
+ kyuubiDataSourceParamDTO.setDatabase("default");
+ kyuubiDataSourceParamDTO.setOther(props);
+
+ try (
+ MockedStatic<PasswordUtils> mockedStaticPasswordUtils = Mockito.mockStatic(PasswordUtils.class);
+ MockedStatic<CommonUtils> mockedStaticCommonUtils = Mockito.mockStatic(CommonUtils.class)) {
Review Comment:
nit: indention
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/test/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/param/KyuubiDataSourceProcessorTest.java:
##########
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi.param;
+
+import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
+import org.apache.dolphinscheduler.common.utils.JSONUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.CommonUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.DataSourceUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
+import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+import org.mockito.junit.jupiter.MockitoExtension;
+
+@ExtendWith(MockitoExtension.class)
+public class KyuubiDataSourceProcessorTest {
+
+ private KyuubiDataSourceProcessor kyuubiDatasourceProcessor = new KyuubiDataSourceProcessor();
+ @Test
+ public void testCheckDatasourceParam() {
+ KyuubiDataSourceParamDTO kyuubiDatasourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDatasourceParamDTO.setHost("localhost");
+ kyuubiDatasourceParamDTO.setDatabase("default");
+ Map<String, String> other = new HashMap<>();
+ other.put("serverTimezone", "Asia/Shanghai");
+ kyuubiDatasourceParamDTO.setOther(other);
+ DataSourceUtils.checkDatasourceParam(kyuubiDatasourceParamDTO);
+ Assertions.assertTrue(true);
+ }
+
+ @Test
+ public void testBuildConnectionParams() {
+ KyuubiDataSourceParamDTO kyuubiDataSourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDataSourceParamDTO.setHost("localhost");
+ kyuubiDataSourceParamDTO.setDatabase("default");
+ kyuubiDataSourceParamDTO.setUserName("root");
+ kyuubiDataSourceParamDTO.setPort(3306);
+ kyuubiDataSourceParamDTO.setPassword("123456");
+
+ try (
+ MockedStatic<PasswordUtils> mockedStaticPasswordUtils = Mockito.mockStatic(PasswordUtils.class);
+ MockedStatic<CommonUtils> mockedStaticCommonUtils = Mockito.mockStatic(CommonUtils.class)) {
Review Comment:
nit: indention
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161420668
##########
tools/dependencies/known-dependencies.txt:
##########
@@ -431,6 +431,48 @@ azure-storage-file-share-12.16.1.jar
nimbus-jose-jwt-9.22.jar
woodstox-core-6.4.0.jar
azure-core-management-1.9.0.jar
+kyuubi-hive-jdbc-1.7.0.jar
+jcl-over-slf4j-1.7.36.jar
+api-common-2.6.0.jar
+auto-value-1.10.1.jar
+auto-value-annotations-1.10.1.jar
+bcpkix-jdk15on-1.67.jar
+bcprov-jdk15on-1.67.jar
+conscrypt-openjdk-uber-2.5.2.jar
+gapic-google-cloud-storage-v2-2.18.0-alpha.jar
+gax-2.23.0.jar
+gax-grpc-2.23.0.jar
+gax-httpjson-0.108.0.jar
+google-api-client-2.2.0.jar
+google-api-services-storage-v1-rev20220705-2.0.0.jar
+google-auth-library-credentials-1.15.0.jar
+google-auth-library-oauth2-http-1.15.0.jar
+google-cloud-core-2.10.0.jar
+google-cloud-core-grpc-2.10.0.jar
+google-cloud-core-http-2.10.0.jar
+google-cloud-storage-2.18.0.jar
+google-http-client-1.42.3.jar
+google-http-client-apache-v2-1.42.3.jar
+google-http-client-appengine-1.42.3.jar
+google-http-client-gson-1.42.3.jar
+google-http-client-jackson2-1.42.3.jar
+google-oauth-client-1.34.1.jar
+grpc-alts-1.41.0.jar
+grpc-auth-1.41.0.jar
+grpc-google-cloud-storage-v2-2.18.0-alpha.jar
+grpc-googleapis-1.52.1.jar
+grpc-netty-shaded-1.41.0.jar
+grpc-services-1.41.0.jar
+grpc-xds-1.41.0.jar
+opencensus-api-0.31.1.jar
+opencensus-contrib-http-util-0.31.1.jar
+opencensus-proto-0.2.0.jar
+proto-google-cloud-storage-v2-2.18.0-alpha.jar
+proto-google-iam-v1-1.9.0.jar
+re2j-1.6.jar
+threetenbp-1.6.5.jar
+kyuubi-hive-jdbc-1.6.0-incubating.jar
+jcl-over-slf4j-1.7.36.jar
Review Comment:
this PR actually only introduces `kyuubi-hive-jdbc-shaded-1.7.0.jar`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Radeity commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "Radeity (via GitHub)" <gi...@apache.org>.
Radeity commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1473024147
> > Please retry analysis of this Pull-Request directly on [SonarCloud](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642).
>
> Hi All, what should I do for this ? @EricGao888 @SbloodyS @fuchanghai
Hi, @zhaohehuhu , you can close this pull request and open again, all checks will rerun.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1511476561
Hi @EricGao888 @fuchanghai @Radeity, it seems there is no error anymore for this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1493540794
I re-trigger the CI run
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1155825465
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/pom.xml:
##########
@@ -0,0 +1,310 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-datasource-kyuubi</artifactId>
+ <packaging>jar</packaging>
+ <name>${project.artifactId}</name>
+ <dependencies>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-spi</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-api</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hbase.thirdparty</groupId>
+ <artifactId>hbase-noop-htrace</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-client</artifactId>
Review Comment:
The doc is ready now https://kyuubi.readthedocs.io/en/master/client/jdbc/kyuubi_jdbc.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1455644590
Hi All, what should I do next ? @EricGao888 @fuchanghai
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457366898
> @zhaohehuhu You can refer to this #12379 
should be fixed. Please hep check it @fuchanghai
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] fuchanghai commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "fuchanghai (via GitHub)" <gi...@apache.org>.
fuchanghai commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1126189851
##########
dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/enums/DbType.java:
##########
@@ -43,7 +43,8 @@ public enum DbType {
TRINO(12, "trino"),
STARROCKS(13, "starrocks"),
AZURESQL(14, "azuresql"),
- DAMENG(15, "dameng");
+ DAMENG(15, "dameng"),
+ KYUUBI(18, "kyuubi");
Review Comment:
why not set ``` 16 ```?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457573643
> >
>
> > > @zhaohehuhu You can refer to this #12379 
> >
> >
> > should be fixed. Please hep check it @fuchanghai
> > maybe you need to add kyuubi in this file
> > 
> > 
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] fuchanghai commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "fuchanghai (via GitHub)" <gi...@apache.org>.
fuchanghai commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1455657115
> Hi All, what should I do next ? @EricGao888 @fuchanghai
can you add related license?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] fuchanghai commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "fuchanghai (via GitHub)" <gi...@apache.org>.
fuchanghai commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1461453990
> > Agreed with @fuchanghai , we should set 16 for KYUUBI. Please fix it.
>
> I got another snowflake datasource that need to contribute to community later. The snowflake datasource can be set to anything you prefer. Could we keep using 16 for Kyuubi? @fuchanghai @ @SbloodyS
yes ,and @SbloodyS WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] EricGao888 commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "EricGao888 (via GitHub)" <gi...@apache.org>.
EricGao888 commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1461689043
> > Agreed with @fuchanghai , we should set 16 for KYUUBI. Please fix it.
>
> I got another snowflake datasource that need to contribute to community later. The snowflake datasource can be set to anything you prefer. Could we keep using 16 for Kyuubi? @fuchanghai @ @SbloodyS
@zhaohehuhu That's a bit wired to leave 17 empty. Since you plan to contribute snowflake datasource to the community, what about submitting a PR and setting the snowflake datasource with id 17 first? Once the snowflake PR gets merged, I think it is ok to set kyuubi with id 18.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1148716343
##########
dolphinscheduler-bom/pom.xml:
##########
@@ -535,7 +536,17 @@
</exclusion>
</exclusions>
</dependency>
-
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-hive-jdbc</artifactId>
Review Comment:
got it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161435278
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/test/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/param/KyuubiDataSourceProcessorTest.java:
##########
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi.param;
+
+import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
+import org.apache.dolphinscheduler.common.utils.JSONUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.CommonUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.DataSourceUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
+import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+import org.mockito.junit.jupiter.MockitoExtension;
+
+@ExtendWith(MockitoExtension.class)
+public class KyuubiDataSourceProcessorTest {
+
+ private KyuubiDataSourceProcessor kyuubiDatasourceProcessor = new KyuubiDataSourceProcessor();
+ @Test
+ public void testCheckDatasourceParam() {
+ KyuubiDataSourceParamDTO kyuubiDatasourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDatasourceParamDTO.setHost("localhost");
+ kyuubiDatasourceParamDTO.setDatabase("default");
+ Map<String, String> other = new HashMap<>();
+ other.put("serverTimezone", "Asia/Shanghai");
+ kyuubiDatasourceParamDTO.setOther(other);
+ DataSourceUtils.checkDatasourceParam(kyuubiDatasourceParamDTO);
+ Assertions.assertTrue(true);
+ }
+
+ @Test
+ public void testBuildConnectionParams() {
+ KyuubiDataSourceParamDTO kyuubiDataSourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDataSourceParamDTO.setHost("localhost");
+ kyuubiDataSourceParamDTO.setDatabase("default");
+ kyuubiDataSourceParamDTO.setUserName("root");
+ kyuubiDataSourceParamDTO.setPort(3306);
+ kyuubiDataSourceParamDTO.setPassword("123456");
+
+ try (
+ MockedStatic<PasswordUtils> mockedStaticPasswordUtils = Mockito.mockStatic(PasswordUtils.class);
+ MockedStatic<CommonUtils> mockedStaticCommonUtils = Mockito.mockStatic(CommonUtils.class)) {
+ mockedStaticPasswordUtils.when(() -> PasswordUtils.encodePassword(Mockito.anyString()))
+ .thenReturn("123456");
+ mockedStaticCommonUtils.when(CommonUtils::getKerberosStartupState).thenReturn(false);
+ ConnectionParam connectionParam = DataSourceUtils.buildConnectionParams(kyuubiDataSourceParamDTO);
+ Assertions.assertNotNull(connectionParam);
+ }
+ }
+
+ @Test
+ public void testBuildConnectionParams2() {
+ KyuubiDataSourceParamDTO kyuubiDataSourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDataSourceParamDTO.setHost("localhost");
+ kyuubiDataSourceParamDTO.setDatabase("default");
+ kyuubiDataSourceParamDTO.setUserName("root");
+ kyuubiDataSourceParamDTO.setPort(3306);
+ kyuubiDataSourceParamDTO.setPassword("123456");
+ ConnectionParam connectionParam =
+ DataSourceUtils.buildConnectionParams(DbType.KYUUBI, JSONUtils.toJsonString(kyuubiDataSourceParamDTO));
+ Assertions.assertNotNull(connectionParam);
+ }
+ @Test
+ public void testCreateConnectionParams() {
+ Map<String, String> props = new HashMap<>();
+ props.put("serverTimezone", "utc");
+ KyuubiDataSourceParamDTO kyuubiDataSourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDataSourceParamDTO.setHost("localhost1,localhost2");
+ kyuubiDataSourceParamDTO.setPort(5142);
+ kyuubiDataSourceParamDTO.setUserName("default");
+ kyuubiDataSourceParamDTO.setDatabase("default");
+ kyuubiDataSourceParamDTO.setOther(props);
+
+ try (
+ MockedStatic<PasswordUtils> mockedStaticPasswordUtils = Mockito.mockStatic(PasswordUtils.class);
+ MockedStatic<CommonUtils> mockedStaticCommonUtils = Mockito.mockStatic(CommonUtils.class)) {
Review Comment:
done
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/test/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/param/KyuubiDataSourceProcessorTest.java:
##########
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi.param;
+
+import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
+import org.apache.dolphinscheduler.common.utils.JSONUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.CommonUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.DataSourceUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
+import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+import org.mockito.junit.jupiter.MockitoExtension;
+
+@ExtendWith(MockitoExtension.class)
+public class KyuubiDataSourceProcessorTest {
+
+ private KyuubiDataSourceProcessor kyuubiDatasourceProcessor = new KyuubiDataSourceProcessor();
+ @Test
+ public void testCheckDatasourceParam() {
+ KyuubiDataSourceParamDTO kyuubiDatasourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDatasourceParamDTO.setHost("localhost");
+ kyuubiDatasourceParamDTO.setDatabase("default");
+ Map<String, String> other = new HashMap<>();
+ other.put("serverTimezone", "Asia/Shanghai");
+ kyuubiDatasourceParamDTO.setOther(other);
+ DataSourceUtils.checkDatasourceParam(kyuubiDatasourceParamDTO);
+ Assertions.assertTrue(true);
+ }
+
+ @Test
+ public void testBuildConnectionParams() {
+ KyuubiDataSourceParamDTO kyuubiDataSourceParamDTO = new KyuubiDataSourceParamDTO();
+ kyuubiDataSourceParamDTO.setHost("localhost");
+ kyuubiDataSourceParamDTO.setDatabase("default");
+ kyuubiDataSourceParamDTO.setUserName("root");
+ kyuubiDataSourceParamDTO.setPort(3306);
+ kyuubiDataSourceParamDTO.setPassword("123456");
+
+ try (
+ MockedStatic<PasswordUtils> mockedStaticPasswordUtils = Mockito.mockStatic(PasswordUtils.class);
+ MockedStatic<CommonUtils> mockedStaticCommonUtils = Mockito.mockStatic(CommonUtils.class)) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161435369
##########
docs/docs/en/guide/datasource/kyuubi.md:
##########
@@ -18,25 +18,17 @@
| Database name | Enter the database name of the KYUUBI connection. |
| Jdbc connection parameters | Parameter settings for KYUUBI connection, in JSON format. |
-## Use HiveServer2 HA ZooKeeper
+```Kerberos Authentication
+just simply configure them in the connection parameters if kerberos authentication is required.
-NOTICE: If Kerberos is disabled, ensure the parameter `hadoop.security.authentication.startup.state` is false, and parameter `java.security.krb5.conf.path` value sets null.
-If **Kerberos** is enabled, needs to set the following parameters in `common.properties`:
+clientKeytab: path of Kerberos keytab file for client authentication
-```conf
-# whether to startup kerberos
-hadoop.security.authentication.startup.state=true
+clientPrincipal: Kerberos principal for client authentication
-# java.security.krb5.conf path
-java.security.krb5.conf.path=/opt/krb5.conf
-
-# login user from keytab username
-login.user.keytab.username=hdfs-mycluster@ESZ.COM
-
-# login user from keytab path
-login.user.keytab.path=/opt/hdfs.headless.keytab
+serverPrincipal: Kerberos principal configured by kyuubi.kinit.principal at the server side. serverPrincipal is available since 1.7.0, for previous versions, use principal instead.
Review Comment:
done
##########
docs/docs/zh/guide/datasource/kyuubi.md:
##########
@@ -14,23 +14,14 @@
- 数据库名：输入连接 KYUUBI 的数据库名称
- Jdbc 连接参数：用于 KYUUBI 连接的参数设置，以 JSON 形式填写
-## 使用 HiveServer2 HA Zookeeper
+```Kerberos 验证
+如需Kerberos验证，请直接配置相应参数在jdbc连接参数中
-注意：如果没有开启 kerberos,请保证参数 `hadoop.security.authentication.startup.state` 值为 `false`,
-参数 `java.security.krb5.conf.path` 值为空. 开启了 **kerberos**，则需要在 `common.properties` 配置以下参数
+clientKeytab: 用户客户端验证的keytab文件路径
-```conf
-# whether to startup kerberos
-hadoop.security.authentication.startup.state=true
+clientPrincipal: 用户客户端验证的Kerberos principal
-# java.security.krb5.conf path
-java.security.krb5.conf.path=/opt/krb5.conf
-
-# login user from keytab username
-login.user.keytab.username=hdfs-mycluster@ESZ.COM
-
-# login user from keytab path
-login.user.keytab.path=/opt/hdfs.headless.keytab
+serverPrincipal: 在服务端通过kyuubi.kinit.principal配置的Kerberos principal. serverPrincipal在kyuubi 1.7.0后启用，针对老版本请使用principal
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1506444908
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [28 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [64.2% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161405495
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/param/KyuubiDataSourceProcessor.java:
##########
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi.param;
+
+import org.apache.dolphinscheduler.common.constants.Constants;
+import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
+import org.apache.dolphinscheduler.common.utils.JSONUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.datasource.AbstractDataSourceProcessor;
+import org.apache.dolphinscheduler.plugin.datasource.api.datasource.BaseDataSourceParamDTO;
+import org.apache.dolphinscheduler.plugin.datasource.api.datasource.DataSourceProcessor;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import org.apache.commons.collections4.MapUtils;
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+
+import com.google.auto.service.AutoService;
+
+@AutoService(DataSourceProcessor.class)
+public class KyuubiDataSourceProcessor extends AbstractDataSourceProcessor {
+
+ @Override
+ public BaseDataSourceParamDTO castDatasourceParamDTO(String paramJson) {
+ return JSONUtils.parseObject(paramJson, KyuubiDataSourceParamDTO.class);
+ }
+
+ @Override
+ public BaseDataSourceParamDTO createDatasourceParamDTO(String connectionJson) {
+ KyuubiDataSourceParamDTO kyuubiDataSourceParamDTO = new KyuubiDataSourceParamDTO();
+ KyuubiConnectionParam kyuubiConnectionParam = (KyuubiConnectionParam) createConnectionParams(connectionJson);
+ kyuubiDataSourceParamDTO.setDatabase(kyuubiConnectionParam.getDatabase());
+ kyuubiDataSourceParamDTO.setUserName(kyuubiConnectionParam.getUser());
+ kyuubiDataSourceParamDTO.setOther(kyuubiConnectionParam.getOther());
+
+ String[] tmpArray = kyuubiConnectionParam.getAddress().split(Constants.DOUBLE_SLASH);
+ StringBuilder hosts = new StringBuilder();
+ String[] hostPortArray = tmpArray[tmpArray.length - 1].split(Constants.COMMA);
+ for (String hostPort : hostPortArray) {
+ hosts.append(hostPort.split(Constants.COLON)[0]).append(Constants.COMMA);
+ }
+ hosts.deleteCharAt(hosts.length() - 1);
+ kyuubiDataSourceParamDTO.setHost(hosts.toString());
+ kyuubiDataSourceParamDTO.setPort(Integer.parseInt(hostPortArray[0].split(Constants.COLON)[1]));
+
+ return kyuubiDataSourceParamDTO;
+ }
+
+ @Override
+ public BaseConnectionParam createConnectionParams(BaseDataSourceParamDTO datasourceParam) {
+ KyuubiDataSourceParamDTO kyuubiParam = (KyuubiDataSourceParamDTO) datasourceParam;
+ StringBuilder address = new StringBuilder();
+ address.append(DataSourceConstants.JDBC_HIVE_2);
Review Comment:
changed to jdbc:kyuubi://
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1506268552
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [28 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [58.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161405419
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/pom.xml:
##########
@@ -0,0 +1,61 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-datasource-kyuubi</artifactId>
+ <packaging>jar</packaging>
+ <name>${project.artifactId}</name>
+ <dependencies>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-spi</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-api</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hbase.thirdparty</groupId>
+ <artifactId>hbase-noop-htrace</artifactId>
Review Comment:
Deleted this dep.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1148716431
##########
dolphinscheduler-bom/pom.xml:
##########
@@ -67,6 +67,7 @@
<guava.version>31.1-jre</guava.version>
<postgresql.version>42.4.1</postgresql.version>
<hive-jdbc.version>2.3.9</hive-jdbc.version>
+ <kyuubi-jdbc.version>1.6.0-incubating</kyuubi-jdbc.version>
Review Comment:
OK. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1455867745
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [11.7% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161394395
##########
docs/docs/en/guide/datasource/kyuubi.md:
##########
@@ -0,0 +1,42 @@
+# KYUUBI
+
+## Use HiveServer2
+
+
+
+## Datasource Parameters
+
+| **Datasource** | **Description** |
+|----------------------------|-----------------------------------------------------------|
+| Datasource | Select KYUUBI. |
+| Datasource name | Enter the name of the DataSource. |
+| Description | Enter a description of the DataSource. |
+| IP/Host Name | Enter the KYUUBI service IP. |
+| Port | Enter the KYUUBI service port. |
+| Username | Set the username for KYUUBI connection. |
+| Password | Set the password for KYUUBI connection. |
+| Database name | Enter the database name of the KYUUBI connection. |
+| Jdbc connection parameters | Parameter settings for KYUUBI connection, in JSON format. |
+
+## Use HiveServer2 HA ZooKeeper
+
+NOTICE: If Kerberos is disabled, ensure the parameter `hadoop.security.authentication.startup.state` is false, and parameter `java.security.krb5.conf.path` value sets null.
+If **Kerberos** is enabled, needs to set the following parameters in `common.properties`:
+
+```conf
+# whether to startup kerberos
+hadoop.security.authentication.startup.state=true
+
+# java.security.krb5.conf path
+java.security.krb5.conf.path=/opt/krb5.conf
+
+# login user from keytab username
+login.user.keytab.username=hdfs-mycluster@ESZ.COM
+
+# login user from keytab path
+login.user.keytab.path=/opt/hdfs.headless.keytab
Review Comment:
the code does not match the doc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1459939985
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [13.5% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1459549715
> basic LGTM of the backend part, But i think `DBType` of `Kyuubi ` is better to be set `16`
>
> WDYT @EricGao888 @Radeity
It need some effort for us to change the id of DBType, cause we need to change the data stored in database.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457604999
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [11.7% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] fuchanghai commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "fuchanghai (via GitHub)" <gi...@apache.org>.
fuchanghai commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1455662198
@zhaohehuhu You can refer to this #12379
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] fuchanghai commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "fuchanghai (via GitHub)" <gi...@apache.org>.
fuchanghai commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457575538
@zhaohehuhu plz reslove the conflicts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1473173246
@EricGao888 @SbloodyS @fuchanghai Hi all, is there any improvements I need to do?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1482153315
@SbloodyS plz run CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1455512685
> @zhaohehuhu CI failed by style check. Could u plz run `mvn spotless:apply`? `Spotless` will automatically fix those errors.
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] EricGao888 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "EricGao888 (via GitHub)" <gi...@apache.org>.
EricGao888 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1124557462
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.*;
Review Comment:
Please avoid using `wildcard imports` and fix it before running spotless cmd, or spotless will directly remove it and lead to compilation failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457649837
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [11.7% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] fuchanghai commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "fuchanghai (via GitHub)" <gi...@apache.org>.
fuchanghai commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457590181
``` DataSourceUtils#getDatasourceProcessor``` should be add logic about kyuubi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1125970727
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.*;
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1449685158
> where did you use this parameter `datawarehouse` ?
will check later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1152802567
##########
docs/docs/en/guide/datasource/kyuubi.md:
##########
@@ -0,0 +1,42 @@
+# KYUUBI
Review Comment:
you should also add new docs to https://github.com/apache/dolphinscheduler/blob/9df89cf0cf6eef7875e1fc476aab38e48bd8bb33/docs/configs/docsdev.js#L363-L365 to show in our website
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF_PATH;
+
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.UserGroupInformationFactory;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import sun.security.krb5.Config;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+
+import java.lang.reflect.Field;
+import java.sql.Connection;
+import java.sql.SQLException;
+
+import lombok.extern.slf4j.Slf4j;
+
+import org.springframework.jdbc.core.JdbcTemplate;
+@Slf4j
Review Comment:
```suggestion
import org.springframework.jdbc.core.JdbcTemplate;
@Slf4j
```
##########
dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/enums/DbType.java:
##########
@@ -43,7 +43,8 @@ public enum DbType {
TRINO(12, "trino"),
STARROCKS(13, "starrocks"),
AZURESQL(14, "azuresql"),
- DAMENG(15, "dameng");
+ DAMENG(15, "dameng"),
+ KYUUBI(18, "kyuubi");
Review Comment:
how about use `17`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1154074157
##########
docs/docs/en/guide/datasource/kyuubi.md:
##########
@@ -0,0 +1,42 @@
+# KYUUBI
Review Comment:
done
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF_PATH;
+
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.UserGroupInformationFactory;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import sun.security.krb5.Config;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+
+import java.lang.reflect.Field;
+import java.sql.Connection;
+import java.sql.SQLException;
+
+import lombok.extern.slf4j.Slf4j;
+
+import org.springframework.jdbc.core.JdbcTemplate;
+@Slf4j
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1471390410
Please retry analysis of this Pull-Request directly on [SonarCloud](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1473060027
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [13.5% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1502561445
approval to run CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1504488167
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [33 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [58.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1502891110
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [33 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [58.9% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1501436415
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [19 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [33.6% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1501641399
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [18 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [40.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1155429112
##########
dolphinscheduler-bom/pom.xml:
##########
@@ -536,7 +537,17 @@
</exclusion>
</exclusions>
</dependency>
-
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-hive-jdbc-shaded</artifactId>
+ <version>${kyuubi-jdbc.version}</version>
+ <exclusions>
Review Comment:
The `kyuubi-hive-jdbc-shaded` has NO transitive deps, please remove all exclusions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] fuchanghai commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "fuchanghai (via GitHub)" <gi...@apache.org>.
fuchanghai commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457594239
loss denpendies
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu closed pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu closed pull request #13642: [Feature][plugin] Support Kyuubi datasource
URL: https://github.com/apache/dolphinscheduler/pull/13642
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1128918026
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF_PATH;
+
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.UserGroupInformationFactory;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import sun.security.krb5.Config;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+
+import java.lang.reflect.Field;
+import java.sql.Connection;
+import java.sql.SQLException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.jdbc.core.JdbcTemplate;
+
+public class KyuubiDataSourceClient extends CommonDataSourceClient {
+
+ private static final Logger logger = LoggerFactory.getLogger(KyuubiDataSourceClient.class);
Review Comment:
> BTW, can we directly reuse `HiveDataSourceClient` and `UserGroupInformationFactory` in ds hive plugin, instead of creating new one?
It's a bit duplicated. But I'm not sure if it's reasonable to make Kyuubi plugin have the dependency of Hive plugin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1459615818
> > DBType
>
> 16 was used for OCEANBASE
17 conflicts with our internal datasource. really painful for me to reset the id
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1459933145
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [13.5% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1461148788
> Agreed with @fuchanghai , we should set 16 for KYUUBI. Please fix it.
I got another snowflake datasource that need to contribute to community later. The snowflake datasource can be set to anything you prefer. Could we keep using 16 for Kyuubi? @fuchanghai @ @SbloodyS
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1465641467
> > > > Agreed with @fuchanghai , we should set 16 for KYUUBI. Please fix it.
> > >
> > >
> > > I got another snowflake datasource that need to contribute to community later. The snowflake datasource can be set to anything you prefer. Could we keep using 16 for Kyuubi? @fuchanghai @ @SbloodyS
> >
> >
> > @zhaohehuhu That's a bit wired to leave 17 empty. Since you plan to contribute snowflake datasource to the community, what about submitting a PR and setting the snowflake datasource with id 17 first? Once the snowflake PR gets merged, I think it is ok to set kyuubi with id 18.
>
> OK. I'll commit snowflake datasource at the same time. Thanks.
@EricGao888 @SbloodyS already requested #13729
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1502667543
> approval to run CI
plz help run
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1506442520
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [28 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [64.2% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1504810259
Please retry analysis of this Pull-Request directly on [SonarCloud](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1471389917
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [13.5% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1471203680
> > > > > Agreed with @fuchanghai , we should set 16 for KYUUBI. Please fix it.
> > > >
> > > >
> > > > I got another snowflake datasource that need to contribute to community later. The snowflake datasource can be set to anything you prefer. Could we keep using 16 for Kyuubi? @fuchanghai @ @SbloodyS
> > >
> > >
> > > @zhaohehuhu That's a bit wired to leave 17 empty. Since you plan to contribute snowflake datasource to the community, what about submitting a PR and setting the snowflake datasource with id 17 first? Once the snowflake PR gets merged, I think it is ok to set kyuubi with id 18.
> >
> >
> > OK. I'll commit snowflake datasource at the same time. Thanks.
>
> @EricGao888 @SbloodyS already requested #13729. Is it ok to set kyuubi to 18 now. Thanks.
Hi All, could you guide me for the work left ? @EricGao888 @SbloodyS @fuchanghai
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] EricGao888 commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "EricGao888 (via GitHub)" <gi...@apache.org>.
EricGao888 commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1471230282
> > > > > > Agreed with @fuchanghai , we should set 16 for KYUUBI. Please fix it.
> > > > >
> > > > >
> > > > > I got another snowflake datasource that need to contribute to community later. The snowflake datasource can be set to anything you prefer. Could we keep using 16 for Kyuubi? @fuchanghai @ @SbloodyS
> > > >
> > > >
> > > > @zhaohehuhu That's a bit wired to leave 17 empty. Since you plan to contribute snowflake datasource to the community, what about submitting a PR and setting the snowflake datasource with id 17 first? Once the snowflake PR gets merged, I think it is ok to set kyuubi with id 18.
> > >
> > >
> > > OK. I'll commit snowflake datasource at the same time. Thanks.
> >
> >
> > @EricGao888 @SbloodyS already requested #13729. Is it ok to set kyuubi to 18 now. Thanks.
>
> Hi All, could you guide me for the work left ? @EricGao888 @SbloodyS @fuchanghai
Could u plz resolve the conflicts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1155429559
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/pom.xml:
##########
@@ -0,0 +1,310 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-datasource-kyuubi</artifactId>
+ <packaging>jar</packaging>
+ <name>${project.artifactId}</name>
+ <dependencies>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-spi</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-api</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hbase.thirdparty</groupId>
+ <artifactId>hbase-noop-htrace</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-client</artifactId>
Review Comment:
is it for Kerberos authentication? Kyuubi implements the Kerberos authentication based on JDK JAAS framework, it does not requires Hadoop classes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1155429778
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/pom.xml:
##########
@@ -0,0 +1,310 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-datasource-kyuubi</artifactId>
+ <packaging>jar</packaging>
+ <name>${project.artifactId}</name>
+ <dependencies>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-spi</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-api</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hbase.thirdparty</groupId>
+ <artifactId>hbase-noop-htrace</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-client</artifactId>
Review Comment:
The related docs is under review
https://github.com/apache/kyuubi/pull/4649
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161391366
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/param/KyuubiDataSourceProcessor.java:
##########
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi.param;
+
+import org.apache.dolphinscheduler.common.constants.Constants;
+import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
+import org.apache.dolphinscheduler.common.utils.JSONUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.datasource.AbstractDataSourceProcessor;
+import org.apache.dolphinscheduler.plugin.datasource.api.datasource.BaseDataSourceParamDTO;
+import org.apache.dolphinscheduler.plugin.datasource.api.datasource.DataSourceProcessor;
+import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import org.apache.commons.collections4.MapUtils;
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+
+import com.google.auto.service.AutoService;
+
+@AutoService(DataSourceProcessor.class)
+public class KyuubiDataSourceProcessor extends AbstractDataSourceProcessor {
+
+ @Override
+ public BaseDataSourceParamDTO castDatasourceParamDTO(String paramJson) {
+ return JSONUtils.parseObject(paramJson, KyuubiDataSourceParamDTO.class);
+ }
+
+ @Override
+ public BaseDataSourceParamDTO createDatasourceParamDTO(String connectionJson) {
+ KyuubiDataSourceParamDTO kyuubiDataSourceParamDTO = new KyuubiDataSourceParamDTO();
+ KyuubiConnectionParam kyuubiConnectionParam = (KyuubiConnectionParam) createConnectionParams(connectionJson);
+ kyuubiDataSourceParamDTO.setDatabase(kyuubiConnectionParam.getDatabase());
+ kyuubiDataSourceParamDTO.setUserName(kyuubiConnectionParam.getUser());
+ kyuubiDataSourceParamDTO.setOther(kyuubiConnectionParam.getOther());
+
+ String[] tmpArray = kyuubiConnectionParam.getAddress().split(Constants.DOUBLE_SLASH);
+ StringBuilder hosts = new StringBuilder();
+ String[] hostPortArray = tmpArray[tmpArray.length - 1].split(Constants.COMMA);
+ for (String hostPort : hostPortArray) {
+ hosts.append(hostPort.split(Constants.COLON)[0]).append(Constants.COMMA);
+ }
+ hosts.deleteCharAt(hosts.length() - 1);
+ kyuubiDataSourceParamDTO.setHost(hosts.toString());
+ kyuubiDataSourceParamDTO.setPort(Integer.parseInt(hostPortArray[0].split(Constants.COLON)[1]));
+
+ return kyuubiDataSourceParamDTO;
+ }
+
+ @Override
+ public BaseConnectionParam createConnectionParams(BaseDataSourceParamDTO datasourceParam) {
+ KyuubiDataSourceParamDTO kyuubiParam = (KyuubiDataSourceParamDTO) datasourceParam;
+ StringBuilder address = new StringBuilder();
+ address.append(DataSourceConstants.JDBC_HIVE_2);
Review Comment:
`jdbc:kyuubi://` is recommended, of course, `jdbc:hive2://` is supported as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu closed pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu closed pull request #13642: [Feature][plugin] Support Kyuubi datasource
URL: https://github.com/apache/dolphinscheduler/pull/13642
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161406220
##########
docs/docs/en/guide/datasource/kyuubi.md:
##########
@@ -0,0 +1,42 @@
+# KYUUBI
+
+## Use HiveServer2
+
+
+
+## Datasource Parameters
+
+| **Datasource** | **Description** |
+|----------------------------|-----------------------------------------------------------|
+| Datasource | Select KYUUBI. |
+| Datasource name | Enter the name of the DataSource. |
+| Description | Enter a description of the DataSource. |
+| IP/Host Name | Enter the KYUUBI service IP. |
+| Port | Enter the KYUUBI service port. |
+| Username | Set the username for KYUUBI connection. |
+| Password | Set the password for KYUUBI connection. |
+| Database name | Enter the database name of the KYUUBI connection. |
+| Jdbc connection parameters | Parameter settings for KYUUBI connection, in JSON format. |
+
+## Use HiveServer2 HA ZooKeeper
+
+NOTICE: If Kerberos is disabled, ensure the parameter `hadoop.security.authentication.startup.state` is false, and parameter `java.security.krb5.conf.path` value sets null.
+If **Kerberos** is enabled, needs to set the following parameters in `common.properties`:
+
+```conf
+# whether to startup kerberos
+hadoop.security.authentication.startup.state=true
+
+# java.security.krb5.conf path
+java.security.krb5.conf.path=/opt/krb5.conf
+
+# login user from keytab username
+login.user.keytab.username=hdfs-mycluster@ESZ.COM
+
+# login user from keytab path
+login.user.keytab.path=/opt/hdfs.headless.keytab
Review Comment:
updated doc. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] fuchanghai commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "fuchanghai (via GitHub)" <gi...@apache.org>.
fuchanghai commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457717943
i found above error, but i remind that i fixed it #13599 .so make sure the lastest codes in you branch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1128918026
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF_PATH;
+
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.UserGroupInformationFactory;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import sun.security.krb5.Config;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+
+import java.lang.reflect.Field;
+import java.sql.Connection;
+import java.sql.SQLException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.jdbc.core.JdbcTemplate;
+
+public class KyuubiDataSourceClient extends CommonDataSourceClient {
+
+ private static final Logger logger = LoggerFactory.getLogger(KyuubiDataSourceClient.class);
Review Comment:
> BTW, can we directly reuse `HiveDataSourceClient` and `UserGroupInformationFactory` in ds hive plugin, instead of creating new one?
It's a bit duplicated. But I'm not sure if it's reasonable to make Kyuubi plugin have the dependency with Hive plugin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1127258533
##########
dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/enums/DbType.java:
##########
@@ -43,7 +43,8 @@ public enum DbType {
TRINO(12, "trino"),
STARROCKS(13, "starrocks"),
AZURESQL(14, "azuresql"),
- DAMENG(15, "dameng");
+ DAMENG(15, "dameng"),
+ KYUUBI(18, "kyuubi");
Review Comment:
> why not set `16`?
cause 16 is occupied by other datasources implemented by ourselves
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457604197
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [11.7% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1457601072
>  loss denpendies
weird. I never use this before. Anyway, I will add this dependency as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Radeity commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "Radeity (via GitHub)" <gi...@apache.org>.
Radeity commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1127531723
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF_PATH;
+
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.UserGroupInformationFactory;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import sun.security.krb5.Config;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+
+import java.lang.reflect.Field;
+import java.sql.Connection;
+import java.sql.SQLException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.jdbc.core.JdbcTemplate;
+
+public class KyuubiDataSourceClient extends CommonDataSourceClient {
+
+ private static final Logger logger = LoggerFactory.getLogger(KyuubiDataSourceClient.class);
Review Comment:
Can we use @Slf4j here? (ref https://github.com/apache/dolphinscheduler/pull/13509)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1128932028
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/src/main/java/org/apache/dolphinscheduler/plugin/datasource/kyuubi/KyuubiDataSourceClient.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.datasource.kyuubi;
+
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF;
+import static org.apache.dolphinscheduler.plugin.task.api.TaskConstants.JAVA_SECURITY_KRB5_CONF_PATH;
+
+import org.apache.dolphinscheduler.common.utils.PropertyUtils;
+import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
+import org.apache.dolphinscheduler.plugin.datasource.api.provider.JDBCDataSourceProvider;
+import org.apache.dolphinscheduler.plugin.datasource.kyuubi.security.UserGroupInformationFactory;
+import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
+import org.apache.dolphinscheduler.spi.enums.DbType;
+
+import sun.security.krb5.Config;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+
+import java.lang.reflect.Field;
+import java.sql.Connection;
+import java.sql.SQLException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.jdbc.core.JdbcTemplate;
+
+public class KyuubiDataSourceClient extends CommonDataSourceClient {
+
+ private static final Logger logger = LoggerFactory.getLogger(KyuubiDataSourceClient.class);
Review Comment:
> Can we use @slf4j here? (ref #13509)
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1455882280
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [21 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [17.7% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [11.7% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1455602380
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#13642](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdcf6c3) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/9668c84d30b2c885cbf20ccf9e76f64a41f77096?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9668c84) will **decrease** coverage by `0.11%`.
> The diff coverage is `28.87%`.
> :exclamation: Current head fdcf6c3 differs from pull request most recent head 32f79ed. Consider uploading reports for the commit 32f79ed to get more accurate results
```diff
@@ Coverage Diff @@
## dev #13642 +/- ##
============================================
- Coverage 39.08% 38.97% -0.11%
+ Complexity 4425 4416 -9
============================================
Files 1127 1136 +9
Lines 42096 42258 +162
Branches 4771 4768 -3
============================================
+ Hits 16452 16470 +18
- Misses 23826 23973 +147
+ Partials 1818 1815 -3
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...heduler/api/service/impl/ResourcesServiceImpl.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL3NlcnZpY2UvaW1wbC9SZXNvdXJjZXNTZXJ2aWNlSW1wbC5qYXZh) | `38.49% <ø> (-0.03%)` | :arrow_down: |
| [...cheduler/common/constants/DataSourceConstants.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2NvbnN0YW50cy9EYXRhU291cmNlQ29uc3RhbnRzLmphdmE=) | `0.00% <ø> (ø)` | |
| [...e/kyuubi/security/UserGroupInformationFactory.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYXRhc291cmNlLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLWRhdGFzb3VyY2Uta3l1dWJpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3BsdWdpbi9kYXRhc291cmNlL2t5dXViaS9zZWN1cml0eS9Vc2VyR3JvdXBJbmZvcm1hdGlvbkZhY3RvcnkuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [.../org/apache/dolphinscheduler/spi/enums/DbType.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc3BpL2VudW1zL0RiVHlwZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...duler/plugin/task/api/AbstractCommandExecutor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci10YXNrLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLXRhc2stYXBpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3BsdWdpbi90YXNrL2FwaS9BYnN0cmFjdENvbW1hbmRFeGVjdXRvci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...ugin/task/api/am/KubernetesApplicationManager.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci10YXNrLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLXRhc2stYXBpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3BsdWdpbi90YXNrL2FwaS9hbS9LdWJlcm5ldGVzQXBwbGljYXRpb25NYW5hZ2VyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...e/dolphinscheduler/server/worker/WorkerServer.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci13b3JrZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci9Xb3JrZXJTZXJ2ZXIuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...duler/server/worker/log/TaskInstanceLogHeader.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci13b3JrZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci9sb2cvVGFza0luc3RhbmNlTG9nSGVhZGVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...server/worker/processor/TaskDispatchProcessor.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci13b3JrZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci9wcm9jZXNzb3IvVGFza0Rpc3BhdGNoUHJvY2Vzc29yLmphdmE=) | `64.10% <0.00%> (ø)` | |
| [...heduler/server/worker/runner/TaskCallbackImpl.java](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci13b3JrZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci9ydW5uZXIvVGFza0NhbGxiYWNrSW1wbC5qYXZh) | `0.00% <ø> (ø)` | |
| ... and [44 more](https://codecov.io/gh/apache/dolphinscheduler/pull/13642?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1455797607
> > Hi All, what should I do next ? @EricGao888 @fuchanghai
> > 
>
> can you add related license?
Sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1459614244
> DBType
16 was used for OCEANBASE
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1512536188
Thanks all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie merged pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie merged PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1491397986
@zhongjiajie 17 is used by ssh now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1501430886
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [19 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [33.6% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1161384643
##########
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-kyuubi/pom.xml:
##########
@@ -0,0 +1,310 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-plugin</artifactId>
+ <version>dev-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>dolphinscheduler-datasource-kyuubi</artifactId>
+ <packaging>jar</packaging>
+ <name>${project.artifactId}</name>
+ <dependencies>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-spi</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-datasource-api</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <artifactId>dolphinscheduler-task-api</artifactId>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hbase.thirdparty</groupId>
+ <artifactId>hbase-noop-htrace</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-client</artifactId>
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1501632330
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=13642)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL) [18 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list) [40.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=13642&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] EricGao888 commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "EricGao888 (via GitHub)" <gi...@apache.org>.
EricGao888 commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1453636415
@zhaohehuhu CI failed by style check. Could u plz run `mvn spotless:apply`? `Spotless` will automatically fix those errors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhaohehuhu commented on pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "zhaohehuhu (via GitHub)" <gi...@apache.org>.
zhaohehuhu commented on PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#issuecomment-1473012129
> Please retry analysis of this Pull-Request directly on [SonarCloud](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=13642).
Hi All, what should I do for this ? @EricGao888 @SbloodyS @fuchanghai
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pan3793 commented on a diff in pull request #13642: [Feature][plugin] Support Kyuubi datasource
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #13642:
URL: https://github.com/apache/dolphinscheduler/pull/13642#discussion_r1147177959
##########
dolphinscheduler-bom/pom.xml:
##########
@@ -535,7 +536,17 @@
</exclusion>
</exclusions>
</dependency>
-
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-hive-jdbc</artifactId>
Review Comment:
The shaded JDBC client is recommended, and the exclusions part can be removed
```suggestion
<artifactId>kyuubi-hive-jdbc-shaded</artifactId>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org