You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "njnu-seafish (via GitHub)" <gi...@apache.org> on 2025/08/13 01:51:32 UTC

[I] [Bug] [DataSource] Hive and Spark not work [dolphinscheduler]

njnu-seafish opened a new issue, #17413:
URL: https://github.com/apache/dolphinscheduler/issues/17413

   ### Search before asking
   
   - [x] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What happened
   
   1, First, create a Hive data source
   
   <img width="475" height="641" alt="Image" src="https://github.com/user-attachments/assets/b0340696-fdcc-4eb1-8862-2394b1e15f25" />
   
   
   2, Edit the data source and find that the principal field is not displayed.
   
   <img width="429" height="643" alt="Image" src="https://github.com/user-attachments/assets/dd13c435-cc9e-46b6-860b-54b1503a0da8" />
   
   
   3,The connection test shows the following error message:
   
   > 
   2025-08-12 10:04:52.808 WARN  [qtp1869116781-5208] o.a.h.j.HiveConnection:[237] - Failed to connect to 192.168.11.231:10000
   2025-08-12 10:04:52.809 WARN  [qtp1869116781-5208] o.a.k.j.h.KyuubiConnection:[198] - Failed to connect to 192.168.11.231:10000
   2025-08-12 10:04:52.810 WARN  [qtp1869116781-5208] n.s.c.j.SnowflakeConnectString:[136] - Connect strings must start with jdbc:snowflake://
   2025-08-12 10:04:52.810 ERROR [qtp1869116781-5208] o.a.d.p.d.a.d.AbstractDataSourceProcessor:[130] - Check datasource connectivity for: HIVE error
   java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://192.168.11.231:10000/default: Peer indicated failure: Unsupported mechanism type PLAIN
   	at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:256)
   	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
   	at java.sql.DriverManager.getConnection(DriverManager.java:664)
   	at java.sql.DriverManager.getConnection(DriverManager.java:247)
   	at org.apache.dolphinscheduler.plugin.datasource.hive.param.HiveDataSourceProcessor.getConnection(HiveDataSourceProcessor.java:139)
   	at org.apache.dolphinscheduler.plugin.datasource.api.datasource.AbstractDataSourceProcessor.checkDataSourceConnectivity(AbstractDataSourceProcessor.java:127)
   	at org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl.checkConnection(DataSourceServiceImpl.java:326)
   
   
   To resolve the above issue, The principal field needs to appear in the JDBC connection parameter input box.
   <img width="430" height="643" alt="Image" src="https://github.com/user-attachments/assets/483496ae-46d2-4857-a43c-8c8251bd7365" />
   
   
   
   4, Click the connection test again, and the following error appears:
   
   > 
   Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/thrift/TFilterTransport
   	at java.lang.ClassLoader.defineClass1(Native Method)
   	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
   	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
   	at java.net.URLClassLoader.defineClass(URLClassLoader.java:493)
   	at java.net.URLClassLoader.access$100(URLClassLoader.java:75)
   	at java.net.URLClassLoader$1.run(URLClassLoader.java:389)
   	at java.net.URLClassLoader$1.run(URLClassLoader.java:383)
   	at java.security.AccessController.doPrivileged(Native Method)
   	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
   	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
   	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
   	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
   	at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:601)
   	at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:341)
   	at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:228)
   	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
   	at java.sql.DriverManager.getConnection(DriverManager.java:664)
   	at java.sql.DriverManager.getConnection(DriverManager.java:247)
   	at org.apache.dolphinscheduler.plugin.datasource.hive.param.HiveDataSourceProcessor.getConnection(HiveDataSourceProcessor.java:139)
   	at org.apache.dolphinscheduler.plugin.datasource.api.datasource.AbstractDataSourceProcessor.checkDataSourceConnectivity(AbstractDataSourceProcessor.java:127)
   	at org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl.checkConnection(DataSourceServiceImpl.java:326)
   
   ### What you expected to happen
   
   The Hive and Spark data source information can be displayed normally, and the tests pass.
   
   ### How to reproduce
   
   1, First, create a Hive data source
   2, Edit the data source and find that the principal field is not displayed.
   3, Click the connection test 
   
   ### Anything else
   
   _No response_
   
   ### Version
   
   3.3.0-alpha
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [DataSource] Hive and Spark not work properly (Principal field is not displayed and not used correctly) [dolphinscheduler]

Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on issue #17413:
URL: https://github.com/apache/dolphinscheduler/issues/17413#issuecomment-3248992649

   Right now we need to open kerberos in common.properties and set storage type to HDFS, then will use principal.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [DataSource] Hive and Spark not work properly in kerberos env(Principal field is not displayed and not used correctly) [dolphinscheduler]

Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS closed issue #17413: [Bug] [DataSource] Hive and Spark not work properly in kerberos env(Principal field is not displayed and not used correctly)
URL: https://github.com/apache/dolphinscheduler/issues/17413


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [DataSource] Hive and Spark not work properly (Principal field is not displayed and not used correctly) [dolphinscheduler]

Posted by "njnu-seafish (via GitHub)" <gi...@apache.org>.
njnu-seafish commented on issue #17413:
URL: https://github.com/apache/dolphinscheduler/issues/17413#issuecomment-3251304145

   > Right now we need to open kerberos in common.properties and set storage type to HDFS, then will use principal.
   
   I used it exactly as described, and then I discovered this bug.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [DataSource] Hive and Spark not work [dolphinscheduler]

Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on issue #17413:
URL: https://github.com/apache/dolphinscheduler/issues/17413#issuecomment-3182453472

   Did you install hive datasource plugin? @njnu-seafish 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [DataSource] Hive and Spark not work [dolphinscheduler]

Posted by "njnu-seafish (via GitHub)" <gi...@apache.org>.
njnu-seafish commented on issue #17413:
URL: https://github.com/apache/dolphinscheduler/issues/17413#issuecomment-3186346817

   > Did you install hive datasource plugin? [@njnu-seafish](https://github.com/njnu-seafish)
   
   Yes, the build command is as follows
   mvn clean package -Pstaging -D'maven.test.skip=true' -D'checkstyle.skip=true'
   
   The deployment directory already contains the plugin.  Otherwise, the service startup will fail.
   
   $ ls -l apache-dolphinscheduler-dev-SNAPSHOT-bin/plugins/
   total 52
   drwxr-xr-x 1 Administrator 197121 0 Jun 11 10:11 alert-plugins/
   drwxr-xr-x 1 Administrator 197121 0 Jun 11 10:11 datasource-plugins/
   drwxr-xr-x 1 Administrator 197121 0 Jun 11 10:11 storage-plugins/
   drwxr-xr-x 1 Administrator 197121 0 Jun 11 10:11 task-plugins/
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [DataSource] Hive and Spark not work [dolphinscheduler]

Posted by "njnu-seafish (via GitHub)" <gi...@apache.org>.
njnu-seafish commented on issue #17413:
URL: https://github.com/apache/dolphinscheduler/issues/17413#issuecomment-3196240496

   > Did you install hive datasource plugin? [@njnu-seafish](https://github.com/njnu-seafish)
   
   Hey, do you have this kind of error in your environment?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org