You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "findingrish (via GitHub)" <gi...@apache.org> on 2024/03/28 09:11:42 UTC
[PR] Use DruidLeaderSelector instead of DruidCoordinator in CliCoordinator.HeartbeatSupplier (druid)
findingrish opened a new pull request, #16215:
URL: https://github.com/apache/druid/pull/16215
While working on this change https://github.com/apache/druid/pull/15817, I encountered Guice injection issues in Coordinator after introducing a dependency on `CoordinatorSegmentMetadataCache` within `DruidCoordinator`. This cache depends on `QueryLifecycleFactory` , which in turn depends on `AuthorizerMapper` . It appears a cyclic dependency between `AuthorizerMapper` and `BasicAuthorizerCacheManager` (likely pre-existing) is causing the problem when using authentication.
Interestingly, running Coordinator and Overlord as a single service locally avoids the issue.
The circular dependency chain is (this is pre-existing).
```
AuthorizerMapper -> BasicRoleBasedAuthroizer -> BasicAuthorizerCacheManager-> MetadataStoragePollingBasicAuthorizerCacheManager -> CoordinatorBasicAuthorizerMetadataStorageUpdater -> AuthorizerMapper
```
This dependency chain `CoordinatorSegmentMetadataCache` -> `QueryLifecycleFactory` -> `AuthorizerMapper` had existed and worked fine.
The problem began with `DruidCoordinator` -> `CoordinatorSegmentMetadataCache` dependency and basic authentication, without authentication there are no issues in building the classes.
stack trace
```
1) Problem parsing object at prefix[druid.auth.authorizer.basic]: Cannot construct instance of `org.apache.druid.security.basic.authorization.BasicRoleBasedAuthorizer`, problem: Unable to provision, see the following errors:
1) Tried proxying org.apache.druid.server.security.AuthorizerMapper to support a circular dependency, but it is not an interface.
at org.apache.druid.server.initialization.AuthorizerMapperModule.configure(AuthorizerMapperModule.java:54) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.initialization.AuthorizerMapperModule)
while locating org.apache.druid.server.security.AuthorizerMapper
for the 1st parameter of org.apache.druid.security.basic.authorization.db.updater.CoordinatorBasicAuthorizerMetadataStorageUpdater.<init>(CoordinatorBasicAuthorizerMetadataStorageUpdater.java:122)
at org.apache.druid.security.basic.authorization.db.updater.CoordinatorBasicAuthorizerMetadataStorageUpdater.class(CoordinatorBasicAuthorizerMetadataStorageUpdater.java:79)
while locating org.apache.druid.security.basic.authorization.db.updater.CoordinatorBasicAuthorizerMetadataStorageUpdater
at org.apache.druid.security.basic.BasicSecurityDruidModule.createAuthorizerStorageUpdater(BasicSecurityDruidModule.java:158) (via modules: com.google.inject.util.Modules$OverrideModule -> org.apache.druid.security.basic.BasicSecurityDruidModule)
at org.apache.druid.security.basic.BasicSecurityDruidModule.createAuthorizerStorageUpdater(BasicSecurityDruidModule.java:158) (via modules: com.google.inject.util.Modules$OverrideModule -> org.apache.druid.security.basic.BasicSecurityDruidModule)
while locating org.apache.druid.security.basic.authorization.db.updater.BasicAuthorizerMetadataStorageUpdater
for the 1st parameter of org.apache.druid.security.basic.authorization.db.cache.MetadataStoragePollingBasicAuthorizerCacheManager.<init>(MetadataStoragePollingBasicAuthorizerCacheManager.java:38)
while locating org.apache.druid.security.basic.authorization.db.cache.MetadataStoragePollingBasicAuthorizerCacheManager
at org.apache.druid.security.basic.BasicSecurityDruidModule.createAuthorizerCacheManager(BasicSecurityDruidModule.java:173) (via modules: com.google.inject.util.Modules$OverrideModule -> org.apache.druid.security.basic.BasicSecurityDruidModule)
at org.apache.druid.security.basic.BasicSecurityDruidModule.createAuthorizerCacheManager(BasicSecurityDruidModule.java:173) (via modules: com.google.inject.util.Modules$OverrideModule -> org.apache.druid.security.basic.BasicSecurityDruidModule)
while locating org.apache.druid.security.basic.authorization.db.cache.BasicAuthorizerCacheManager
2) Tried proxying org.apache.druid.server.security.AuthorizerMapper to support a circular dependency, but it is not an interface.
at org.apache.druid.server.initialization.AuthorizerMapperModule.configure(AuthorizerMapperModule.java:54) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.initialization.AuthorizerMapperModule)
while locating org.apache.druid.server.security.AuthorizerMapper
for the 1st parameter of org.apache.druid.security.basic.authorization.db.cache.CoordinatorBasicAuthorizerCacheNotifier.<init>(CoordinatorBasicAuthorizerCacheNotifier.java:56)
at org.apache.druid.security.basic.authorization.db.cache.CoordinatorBasicAuthorizerCacheNotifier.class(CoordinatorBasicAuthorizerCacheNotifier.java:46)
while locating org.apache.druid.security.basic.authorization.db.cache.CoordinatorBasicAuthorizerCacheNotifier
at org.apache.druid.security.basic.BasicSecurityDruidModule.createAuthorizerCacheNotifier(BasicSecurityDruidModule.java:203) (via modules: com.google.inject.util.Modules$OverrideModule -> org.apache.druid.security.basic.BasicSecurityDruidModule)
at org.apache.druid.security.basic.BasicSecurityDruidModule.createAuthorizerCacheNotifier(BasicSecurityDruidModule.java:203) (via modules: com.google.inject.util.Modules$OverrideModule -> org.apache.druid.security.basic.BasicSecurityDruidModule)
while locating org.apache.druid.security.basic.authorization.db.cache.BasicAuthorizerCacheNotifier
for the 6th parameter of org.apache.druid.security.basic.authorization.db.updater.CoordinatorBasicAuthorizerMetadataStorageUpdater.<init>(CoordinatorBasicAuthorizerMetadataStorageUpdater.java:122)
at org.apache.druid.security.basic.authorization.db.updater.CoordinatorBasicAuthorizerMetadataStorageUpdater.class(CoordinatorBasicAuthorizerMetadataStorageUpdater.java:79)
while locating org.apache.druid.security.basic.authorization.db.updater.CoordinatorBasicAuthorizerMetadataStorageUpdater
at org.apache.druid.security.basic.BasicSecurityDruidModule.createAuthorizerStorageUpdater(BasicSecurityDruidModule.java:158) (via modules: com.google.inject.util.Modules$OverrideModule -> org.apache.druid.security.basic.BasicSecurityDruidModule)
at org.apache.druid.security.basic.BasicSecurityDruidModule.createAuthorizerStorageUpdater(BasicSecurityDruidModule.java:158) (via modules: com.google.inject.util.Modules$OverrideModule -> org.apache.druid.security.basic.BasicSecurityDruidModule)
while locating org.apache.druid.security.basic.authorization.db.updater.BasicAuthorizerMetadataStorageUpdater
for the 1st parameter of org.apache.druid.security.basic.authorization.db.cache.MetadataStoragePollingBasicAuthorizerCacheManager.<init>(MetadataStoragePollingBasicAuthorizerCacheManager.java:38)
while locating org.apache.druid.security.basic.authorization.db.cache.MetadataStoragePollingBasicAuthorizerCacheManager
at org.apache.druid.security.basic.BasicSecurityDruidModule.createAuthorizerCacheManager(BasicSecurityDruidModule.java:173) (via modules: com.google.inject.util.Modules$OverrideModule -> org.apache.druid.security.basic.BasicSecurityDruidModule)
at org.apache.druid.security.basic.BasicSecurityDruidModule.createAuthorizerCacheManager(BasicSecurityDruidModule.java:173) (via modules: com.google.inject.util.Modules$OverrideModule -> org.apache.druid.security.basic.BasicSecurityDruidModule)
while locating org.apache.druid.security.basic.authorization.db.cache.BasicAuthorizerCacheManager
2 errors
at [Source: UNKNOWN; line: -1, column: -1].
at org.apache.druid.server.initialization.AuthorizerMapperModule.configure(AuthorizerMapperModule.java:54) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.initialization.AuthorizerMapperModule)
at org.apache.druid.server.initialization.AuthorizerMapperModule.configure(AuthorizerMapperModule.java:54) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.initialization.AuthorizerMapperModule)
while locating org.apache.druid.server.security.AuthorizerMapper
for the 7th parameter of org.apache.druid.server.QueryLifecycleFactory.<init>(QueryLifecycleFactory.java:57)
at org.apache.druid.server.QueryLifecycleFactory.class(QueryLifecycleFactory.java:57)
while locating org.apache.druid.server.QueryLifecycleFactory
for the 1st parameter of org.apache.druid.segment.metadata.CoordinatorSegmentMetadataCache.<init>(CoordinatorSegmentMetadataCache.java:97)
at org.apache.druid.segment.metadata.CoordinatorSegmentMetadataCache.class(CoordinatorSegmentMetadataCache.java:77)
while locating org.apache.druid.segment.metadata.CoordinatorSegmentMetadataCache
for the 17th parameter of org.apache.druid.server.coordinator.DruidCoordinator.<init>(DruidCoordinator.java:201)
at org.apache.druid.cli.CliCoordinator$1.configure(CliCoordinator.java:278) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.cli.CliCoordinator$1)
while locating org.apache.druid.server.coordinator.DruidCoordinator
for the 1st parameter of org.apache.druid.cli.CliCoordinator$HeartbeatSupplier.<init>(CliCoordinator.java:477)
while locating org.apache.druid.cli.CliCoordinator$HeartbeatSupplier
while locating com.google.common.base.Supplier<java.util.Map<java.lang.String, java.lang.Object>> annotated with @com.google.inject.name.Named(value="heartbeat")
for field at org.apache.druid.server.metrics.ServiceStatusMonitor.heartbeatTagsSupplier(ServiceStatusMonitor.java:41)
while locating org.apache.druid.server.metrics.ServiceStatusMonitor
at org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:113) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.metrics.MetricsModule)
at org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:113) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.metrics.MetricsModule)
while locating org.apache.druid.java.util.metrics.MonitorScheduler
at org.apache.druid.server.metrics.MetricsModule.configure(MetricsModule.java:98) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.server.metrics.MetricsModule)
while locating org.apache.druid.java.util.metrics.MonitorScheduler annotated with @com.google.inject.name.Named(value=ForTheEagerness)
1 error
at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:470)
at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:184)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110)
at com.google.inject.Guice.createInjector(Guice.java:99)
at com.google.inject.Guice.createInjector(Guice.java:73)
at com.google.inject.Guice.createInjector(Guice.java:62)
at org.apache.druid.initialization.ExtensionInjectorBuilder.build(ExtensionInjectorBuilder.java:49)
at org.apache.druid.initialization.ServerInjectorBuilder.build(ServerInjectorBuilder.java:118)
at org.apache.druid.initialization.ServerInjectorBuilder.makeServerInjector(ServerInjectorBuilder.java:73)
at org.apache.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:85)
... 2 more
Caused by: java.lang.IllegalArgumentException: Cannot construct instance of `org.apache.druid.security.basic.authorization.BasicRoleBasedAuthorizer`, problem: Unable to provision, see the following errors:
```
The problem seems to get resolved by removing `DruidCoordinator` dependency from `CliCoordinator.HeartbeatSupplier`.
The exact reason for the above issue is yet to be uncovered, meanwhile this change doesn't seem to affect any other thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Use DruidLeaderSelector instead of DruidCoordinator in CliCoordinator.HeartbeatSupplier (druid)
Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz merged PR #16215:
URL: https://github.com/apache/druid/pull/16215
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org