You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/12/18 12:43:38 UTC
[GitHub] [dolphinscheduler] wangbowen1024 opened a new issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
wangbowen1024 opened a new issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
i get a error when try to add a hive datasource by zookeeper way:
### What you expected to happen
create datasource success
### How to reproduce
Like the picture in question in 2.0.1（This version option is not available when you submit the question, so it is noted ）
### Anything else
_No response_
### Version
2.0.0
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangbowen1024 commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
wangbowen1024 commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997216414
> This issue didn't reproduce in my environment.Would you please paste the exception info of dolphinscheduler-api.log?
and i change 3 files #7487 ,then i can connect hive:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangbowen1024 commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
wangbowen1024 commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997215890
> This issue didn't reproduce in my environment.Would you please paste the exception info of dolphinscheduler-api.log?
The Error show in web:
---------
the log:
```
[INFO] 2021-12-18 23:08:41.180 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[76] - REQUEST TRANCE_ID:bd1960db-779b-427e-ab3d-1a54603716f6, LOGIN_USER:admin, URI:/dolphinscheduler/datasources/connect, METHOD:POST, HANDLER:org.apache.dolphinscheduler.api.controller.DataSourceController.connectDataSource, ARGS:{dataSourceParam=HiveDataSourceParamDTO{host='oss.slave01,oss.slave02', port=2181, database='test', principal='null', userName='test', password='null', other='{serviceDiscoveryMode=zooKeeper, zooKeeperNamespace=hiveserver2}', javaSecurityKrb5Conf='null', loginUserKeytabUsername='null', loginUserKeytabPath='null'}}
[INFO] 2021-12-18 23:08:41.182 org.apache.dolphinscheduler.plugin.datasource.api.plugin.DataSourceClientProvider:[56] - getConnection datasourceUniqueId hive@test@jdbc:hive2://oss.slave01:2181,oss.slave02:2181/test
[INFO] 2021-12-18 23:08:41.182 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[67] - PreInit in org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient
[INFO] 2021-12-18 23:08:41.182 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[73] - Create Configuration for hive configuration.
[INFO] 2021-12-18 23:08:41.194 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[75] - Create Configuration success.
[INFO] 2021-12-18 23:08:41.195 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[77] - Create UserGroupInformation.
[INFO] 2021-12-18 23:08:41.195 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[79] - Create ugi success.
[INFO] 2021-12-18 23:08:41.196 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[43] - Creating HikariDataSource pool for maxActive:50
[INFO] 2021-12-18 23:08:41.196 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[63] - Creating HikariDataSource pool success.
[INFO] 2021-12-18 23:08:41.196 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[71] - Creating OneSession HikariDataSource pool for maxActive:50
[INFO] 2021-12-18 23:08:41.197 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[88] - Creating OneSession HikariDataSource pool success.
[INFO] 2021-12-18 23:08:41.197 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[83] - Init org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient success.
[INFO] 2021-12-18 23:08:41.197 com.zaxxer.hikari.HikariDataSource:[110] - HikariPool-5 - Starting...
[INFO] 2021-12-18 23:08:41.198 org.apache.hive.jdbc.Utils:[318] - Supplied authorities: oss.slave01:2181,oss.slave02:2181
[INFO] 2021-12-18 23:08:41.199 org.apache.hive.jdbc.Utils:[437] - Resolved authority: oss.slave01:2181
[WARN] 2021-12-18 23:08:41.224 org.apache.hive.jdbc.HiveConnection:[200] - Failed to connect to oss.slave01:2181
[ERROR] 2021-12-18 23:08:42.227 com.zaxxer.hikari.pool.HikariPool:[594] - HikariPool-5 - Exception during pool initialization.
java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://oss.slave01:2181/test: null
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:219)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:157)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:159)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:117)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:376)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:431)
at org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient.checkClient(CommonDataSourceClient.java:104)
at org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient.<init>(CommonDataSourceClient.java:55)
at org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient.<init>(HiveDataSourceClient.java:62)
at org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceChannel.createDataSourceClient(HiveDataSourceChannel.java:29)
at org.apache.dolphinscheduler.plugin.datasource.api.plugin.DataSourceClientProvider.lambda$getConnection$0(DataSourceClientProvider.java:64)
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
at org.apache.dolphinscheduler.plugin.datasource.api.plugin.DataSourceClientProvider.getConnection(DataSourceClientProvider.java:58)
at org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl.checkConnection(DataSourceServiceImpl.java:320)
at org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl$$FastClassBySpringCGLIB$$a86d54aa.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl$$EnhancerBySpringCGLIB$$49be0dfc.checkConnection(<generated>)
at org.apache.dolphinscheduler.api.controller.DataSourceController.connectDataSource(DataSourceController.java:215)
at org.apache.dolphinscheduler.api.controller.DataSourceController$$FastClassBySpringCGLIB$$835fdd04.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at org.apache.dolphinscheduler.api.aspect.AccessLogAspect.doAround(AccessLogAspect.java:87)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at org.apache.dolphinscheduler.api.controller.DataSourceController$$EnhancerBySpringCGLIB$$46939b4f.connectDataSource(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:517)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)
at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException: null
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:307)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:195)
... 121 common frames omitted
[INFO] 2021-12-18 23:08:42.229 org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient:[108] - Time to execute check jdbc client with sql select 1 for 1031 ms
[ERROR] 2021-12-18 23:08:42.230 org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl:[328] - datasource test connection error, dbType:HIVE, connectionParam:HiveConnectionParam{user='test', password='123456', address='jdbc:hive2://oss.slave01:2181,oss.slave02:2181', database='test', jdbcUrl='jdbc:hive2://oss.slave01:2181,oss.slave02:2181/test', driverLocation='null', driverClassName='org.apache.hive.jdbc.HiveDriver', validationQuery='select 1', other='serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;', principal='null', javaSecurityKrb5Conf='null', loginUserKeytabUsername='null', loginUserKeytabPath='null'}, message:JDBC connect failed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie closed issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
zhongjiajie closed issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997228373
Yes,you are right!
Please submit PR from your dev branch to apache dolphinscheduler's dev branch.
Thanks.^_^
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-999272041
BTW, thanks @SbloodyS for the clarification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangbowen1024 commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
wangbowen1024 commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997208215
> 它将在 2.0.2-release 中修复。 解决此问题的另一种方法是删除 conf/common.properties 中的 java.security.krb5.conf.path。
>
> 与#7392重复[](https://github.com/apache/dolphinscheduler/issues/7392)
i think it's different，i debug find create datasource use url like `jdbc:hive2://test.master:2181,test.slave01:2181,test.slave02:2181/test`, but in fact we need `jdbc:hive2://test.master:2181,test.slave01:2181,test.slave02:2181/test;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangbowen1024 commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
wangbowen1024 commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997237995
> Yes,you are right!
>
> Please submit PR from your dev branch to apache dolphinscheduler's dev branch.
>
> Thanks.^_^
OK, I closed the old PR (because from 2.0.1-release bransh) and committed a new PR to dev branch. #7489
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] nana7miAva commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
nana7miAva commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997678535
大佬,试过jdbc链接hive吗? 是不是有updata kerberos environment failed的Error
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997206411
It will be fixed in 2.0.2-release.
An alternative way to solve this problem is remove java.security.krb5.conf.path in conf/common.properties.
Duplicated with #7392
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-999271894
Duplicate to https://github.com/apache/dolphinscheduler/issues/7392 and already fix in https://github.com/apache/dolphinscheduler/pull/7393. Will release in 2.0.2 and also fix in dev. You could reopen this issue when you have additional problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997197384
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangbowen1024 commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
wangbowen1024 commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997215807
The Error show in web:
---------
the log:
```
[INFO] 2021-12-18 23:08:41.180 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[76] - REQUEST TRANCE_ID:bd1960db-779b-427e-ab3d-1a54603716f6, LOGIN_USER:admin, URI:/dolphinscheduler/datasources/connect, METHOD:POST, HANDLER:org.apache.dolphinscheduler.api.controller.DataSourceController.connectDataSource, ARGS:{dataSourceParam=HiveDataSourceParamDTO{host='oss.slave01,oss.slave02', port=2181, database='test', principal='null', userName='test', password='null', other='{serviceDiscoveryMode=zooKeeper, zooKeeperNamespace=hiveserver2}', javaSecurityKrb5Conf='null', loginUserKeytabUsername='null', loginUserKeytabPath='null'}}
[INFO] 2021-12-18 23:08:41.182 org.apache.dolphinscheduler.plugin.datasource.api.plugin.DataSourceClientProvider:[56] - getConnection datasourceUniqueId hive@test@jdbc:hive2://oss.slave01:2181,oss.slave02:2181/test
[INFO] 2021-12-18 23:08:41.182 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[67] - PreInit in org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient
[INFO] 2021-12-18 23:08:41.182 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[73] - Create Configuration for hive configuration.
[INFO] 2021-12-18 23:08:41.194 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[75] - Create Configuration success.
[INFO] 2021-12-18 23:08:41.195 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[77] - Create UserGroupInformation.
[INFO] 2021-12-18 23:08:41.195 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[79] - Create ugi success.
[INFO] 2021-12-18 23:08:41.196 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[43] - Creating HikariDataSource pool for maxActive:50
[INFO] 2021-12-18 23:08:41.196 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[63] - Creating HikariDataSource pool success.
[INFO] 2021-12-18 23:08:41.196 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[71] - Creating OneSession HikariDataSource pool for maxActive:50
[INFO] 2021-12-18 23:08:41.197 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[88] - Creating OneSession HikariDataSource pool success.
[INFO] 2021-12-18 23:08:41.197 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[83] - Init org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient success.
[INFO] 2021-12-18 23:08:41.197 com.zaxxer.hikari.HikariDataSource:[110] - HikariPool-5 - Starting...
[INFO] 2021-12-18 23:08:41.198 org.apache.hive.jdbc.Utils:[318] - Supplied authorities: oss.slave01:2181,oss.slave02:2181
[INFO] 2021-12-18 23:08:41.199 org.apache.hive.jdbc.Utils:[437] - Resolved authority: oss.slave01:2181
[WARN] 2021-12-18 23:08:41.224 org.apache.hive.jdbc.HiveConnection:[200] - Failed to connect to oss.slave01:2181
[ERROR] 2021-12-18 23:08:42.227 com.zaxxer.hikari.pool.HikariPool:[594] - HikariPool-5 - Exception during pool initialization.
java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://oss.slave01:2181/test: null
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:219)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:157)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:159)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:117)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:376)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:431)
at org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient.checkClient(CommonDataSourceClient.java:104)
at org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient.<init>(CommonDataSourceClient.java:55)
at org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient.<init>(HiveDataSourceClient.java:62)
at org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceChannel.createDataSourceClient(HiveDataSourceChannel.java:29)
at org.apache.dolphinscheduler.plugin.datasource.api.plugin.DataSourceClientProvider.lambda$getConnection$0(DataSourceClientProvider.java:64)
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
at org.apache.dolphinscheduler.plugin.datasource.api.plugin.DataSourceClientProvider.getConnection(DataSourceClientProvider.java:58)
at org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl.checkConnection(DataSourceServiceImpl.java:320)
at org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl$$FastClassBySpringCGLIB$$a86d54aa.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl$$EnhancerBySpringCGLIB$$49be0dfc.checkConnection(<generated>)
at org.apache.dolphinscheduler.api.controller.DataSourceController.connectDataSource(DataSourceController.java:215)
at org.apache.dolphinscheduler.api.controller.DataSourceController$$FastClassBySpringCGLIB$$835fdd04.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at org.apache.dolphinscheduler.api.aspect.AccessLogAspect.doAround(AccessLogAspect.java:87)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at org.apache.dolphinscheduler.api.controller.DataSourceController$$EnhancerBySpringCGLIB$$46939b4f.connectDataSource(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:517)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)
at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException: null
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:307)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:195)
... 121 common frames omitted
[INFO] 2021-12-18 23:08:42.229 org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient:[108] - Time to execute check jdbc client with sql select 1 for 1031 ms
[ERROR] 2021-12-18 23:08:42.230 org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl:[328] - datasource test connection error, dbType:HIVE, connectionParam:HiveConnectionParam{user='test', password='123456', address='jdbc:hive2://oss.slave01:2181,oss.slave02:2181', database='test', jdbcUrl='jdbc:hive2://oss.slave01:2181,oss.slave02:2181/test', driverLocation='null', driverClassName='org.apache.hive.jdbc.HiveDriver', validationQuery='select 1', other='serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;', principal='null', javaSecurityKrb5Conf='null', loginUserKeytabUsername='null', loginUserKeytabPath='null'}, message:JDBC connect failed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997685257
> 大佬,试过jdbc链接hive吗? 是不是有updata kerberos environment failed的Error
It will be fixed in 2.0.2-release.
An alternative way to solve this problem is remove java.security.krb5.conf.path in conf/common.properties.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangbowen1024 commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
wangbowen1024 commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997208491
and i submit a pr #7487 ,I don't know if that's right
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangbowen1024 removed a comment on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
wangbowen1024 removed a comment on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997215807
The Error show in web:
---------
the log:
```
[INFO] 2021-12-18 23:08:41.180 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[76] - REQUEST TRANCE_ID:bd1960db-779b-427e-ab3d-1a54603716f6, LOGIN_USER:admin, URI:/dolphinscheduler/datasources/connect, METHOD:POST, HANDLER:org.apache.dolphinscheduler.api.controller.DataSourceController.connectDataSource, ARGS:{dataSourceParam=HiveDataSourceParamDTO{host='oss.slave01,oss.slave02', port=2181, database='test', principal='null', userName='test', password='null', other='{serviceDiscoveryMode=zooKeeper, zooKeeperNamespace=hiveserver2}', javaSecurityKrb5Conf='null', loginUserKeytabUsername='null', loginUserKeytabPath='null'}}
[INFO] 2021-12-18 23:08:41.182 org.apache.dolphinscheduler.plugin.datasource.api.plugin.DataSourceClientProvider:[56] - getConnection datasourceUniqueId hive@test@jdbc:hive2://oss.slave01:2181,oss.slave02:2181/test
[INFO] 2021-12-18 23:08:41.182 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[67] - PreInit in org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient
[INFO] 2021-12-18 23:08:41.182 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[73] - Create Configuration for hive configuration.
[INFO] 2021-12-18 23:08:41.194 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[75] - Create Configuration success.
[INFO] 2021-12-18 23:08:41.195 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[77] - Create UserGroupInformation.
[INFO] 2021-12-18 23:08:41.195 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[79] - Create ugi success.
[INFO] 2021-12-18 23:08:41.196 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[43] - Creating HikariDataSource pool for maxActive:50
[INFO] 2021-12-18 23:08:41.196 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[63] - Creating HikariDataSource pool success.
[INFO] 2021-12-18 23:08:41.196 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[71] - Creating OneSession HikariDataSource pool for maxActive:50
[INFO] 2021-12-18 23:08:41.197 org.apache.dolphinscheduler.plugin.datasource.api.provider.JdbcDataSourceProvider:[88] - Creating OneSession HikariDataSource pool success.
[INFO] 2021-12-18 23:08:41.197 org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient:[83] - Init org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient success.
[INFO] 2021-12-18 23:08:41.197 com.zaxxer.hikari.HikariDataSource:[110] - HikariPool-5 - Starting...
[INFO] 2021-12-18 23:08:41.198 org.apache.hive.jdbc.Utils:[318] - Supplied authorities: oss.slave01:2181,oss.slave02:2181
[INFO] 2021-12-18 23:08:41.199 org.apache.hive.jdbc.Utils:[437] - Resolved authority: oss.slave01:2181
[WARN] 2021-12-18 23:08:41.224 org.apache.hive.jdbc.HiveConnection:[200] - Failed to connect to oss.slave01:2181
[ERROR] 2021-12-18 23:08:42.227 com.zaxxer.hikari.pool.HikariPool:[594] - HikariPool-5 - Exception during pool initialization.
java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://oss.slave01:2181/test: null
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:219)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:157)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:159)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:117)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:376)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:431)
at org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient.checkClient(CommonDataSourceClient.java:104)
at org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient.<init>(CommonDataSourceClient.java:55)
at org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceClient.<init>(HiveDataSourceClient.java:62)
at org.apache.dolphinscheduler.plugin.datasource.hive.HiveDataSourceChannel.createDataSourceClient(HiveDataSourceChannel.java:29)
at org.apache.dolphinscheduler.plugin.datasource.api.plugin.DataSourceClientProvider.lambda$getConnection$0(DataSourceClientProvider.java:64)
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
at org.apache.dolphinscheduler.plugin.datasource.api.plugin.DataSourceClientProvider.getConnection(DataSourceClientProvider.java:58)
at org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl.checkConnection(DataSourceServiceImpl.java:320)
at org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl$$FastClassBySpringCGLIB$$a86d54aa.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl$$EnhancerBySpringCGLIB$$49be0dfc.checkConnection(<generated>)
at org.apache.dolphinscheduler.api.controller.DataSourceController.connectDataSource(DataSourceController.java:215)
at org.apache.dolphinscheduler.api.controller.DataSourceController$$FastClassBySpringCGLIB$$835fdd04.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at org.apache.dolphinscheduler.api.aspect.AccessLogAspect.doAround(AccessLogAspect.java:87)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at org.apache.dolphinscheduler.api.controller.DataSourceController$$EnhancerBySpringCGLIB$$46939b4f.connectDataSource(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:517)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)
at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException: null
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:307)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:195)
... 121 common frames omitted
[INFO] 2021-12-18 23:08:42.229 org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient:[108] - Time to execute check jdbc client with sql select 1 for 1031 ms
[ERROR] 2021-12-18 23:08:42.230 org.apache.dolphinscheduler.api.service.impl.DataSourceServiceImpl:[328] - datasource test connection error, dbType:HIVE, connectionParam:HiveConnectionParam{user='test', password='123456', address='jdbc:hive2://oss.slave01:2181,oss.slave02:2181', database='test', jdbcUrl='jdbc:hive2://oss.slave01:2181,oss.slave02:2181/test', driverLocation='null', driverClassName='org.apache.hive.jdbc.HiveDriver', validationQuery='select 1', other='serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;', principal='null', javaSecurityKrb5Conf='null', loginUserKeytabUsername='null', loginUserKeytabPath='null'}, message:JDBC connect failed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #7485: [Bug] [dolphinscheduler-datasource-api] Failed to create hive datasource using ZooKeeper way in 2.0.1
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #7485:
URL: https://github.com/apache/dolphinscheduler/issues/7485#issuecomment-997213166
This issue didn't reproduce in my environment.Would you please paste the exception info of dolphinscheduler-api.log?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org