You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/10/28 15:39:45 UTC
[GitHub] [druid] Fryuni opened a new issue, #13277: Kubernetes leader election fails with "Zookeeper is disabled, Can't create CuratorFramework."
Fryuni opened a new issue, #13277:
URL: https://github.com/apache/druid/issues/13277
### Affected Version
24.0.0
### Description
This is the exact same problem as #12083, but the cause mentioned in the comments does not apply to us.
- Cluster size: 5 coordinators+overlords, 4 brokers, 4 routers, 20 middle-managers, 6 historicals
- Configurations in use:
The configuration is exactly as described by the [docs](https://druid.apache.org/docs/latest/development/extensions-core/kubernetes.html):
```
druid.zk.service.enabled=false
druid.serverview.type=http
druid.coordinator.loadqueuepeon.type=http
druid.indexer.runner.type=httpRemote
druid.discovery.type=k8s
```
- Steps to reproduce the problem: Just follow the docs
- The error message or stack traces encountered: The logs are getting truncated in our version, but debugging gets us the same stack as #12083
- Any debugging that you have already done: It seems the CuratorModule should not be called when `druid.zk.service.enabled=false`, but it is being called by the huge stack of dynamic dependency injection. I could not figure out why, or even how it wouldn't be.
If anyone got this extension working, I'd appreciate the help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] Fryuni commented on issue #13277: Kubernetes leader election fails with "Zookeeper is disabled, Can't create CuratorFramework."
Posted by GitBox <gi...@apache.org>.
Fryuni commented on issue #13277:
URL: https://github.com/apache/druid/issues/13277#issuecomment-1295171122
That issue mentions that this extension is being used by some people. Might I ask how?
Following the docs without a single extra step gives this error. And the error is not incorrect... ZooKeeper truly is disabled.
I have no clue what even to try.
Exploring the points mentioned on the stack trace, it seems the `RemoteTaskRunnerFactory` is being loaded as configured [here](https://github.com/apache/druid/blob/41e51b21c3396a78c7a6b736bcb87903876991b2/services/src/main/java/org/apache/druid/cli/CliOverlord.java#L299-L322), even though `druid.indexer.runner.type` is set to `httpRemote`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] Fryuni closed issue #13277: Kubernetes leader election fails with "Zookeeper is disabled, Can't create CuratorFramework."
Posted by GitBox <gi...@apache.org>.
Fryuni closed issue #13277: Kubernetes leader election fails with "Zookeeper is disabled, Can't create CuratorFramework."
URL: https://github.com/apache/druid/issues/13277
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm commented on issue #13277: Kubernetes leader election fails with "Zookeeper is disabled, Can't create CuratorFramework."
Posted by GitBox <gi...@apache.org>.
gianm commented on issue #13277:
URL: https://github.com/apache/druid/issues/13277#issuecomment-1295164039
Hi @Fryuni, I linked this issue at https://github.com/apache/druid/issues/12904 where we are tracking any problems people run into with this extension.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] Fryuni commented on issue #13277: Kubernetes leader election fails with "Zookeeper is disabled, Can't create CuratorFramework."
Posted by GitBox <gi...@apache.org>.
Fryuni commented on issue #13277:
URL: https://github.com/apache/druid/issues/13277#issuecomment-1295285274
Okay, the issue was a combination of factors. Multiple layers of configurations modified the configuration. At one point in the chain, `druid.indexer.runner.type=remote` was being appended to one of the files loaded by Druid to ensure it was not using the local runner when running as a cluster. This should be either prepended or conditionally appended, so we fixed that.
It would be nice to have in the docs a troubleshooting step mentioning that this error is related to the `druid.indexer.runner.type` configuration so others won't have to look into the source :)
I'm now facing some problems regarding authentication between coordinators and historicals when using K8s discovery, but I'll do some more digging before opening another issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org