You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@druid.apache.org by Jeet Patel <jp...@gmail.com> on 2021/08/20 21:56:14 UTC
Get Druid Service details in runtime (via extension)
Hi all,
Is there a way to to know what druid services are running in a DruidNode (Not
talking about the HTTP APIs)?
I went through druid-server module, class
DruidNodeDiscoveryProvider.getForNodeRole which accepts a NodeRole and
returns a DruidNodeDiscovery instance after which we can use
getAllNodes() method
which returns Collection<DiscoveryDruidNode>. And for each item in the
Collection<DiscoveryDruidNode> we can use getServiceName() method to get
the service name.
The question is, how can we get the instance of NodeRole running in the
druid process. For example, if we have a host running broker service, is
there a way to get NodeRole for broker process dynamically?
For now I'm doing something like this. Adding all NodeRole in every host,
since our extension runs in every host.:
List<DruidNodeDiscovery> druidNodeDiscoveryList = ImmutableList.of(
druidNodeDiscoveryProvider.getForNodeRole(NodeRole.COORDINATOR),
druidNodeDiscoveryProvider.getForNodeRole(NodeRole.OVERLORD),
druidNodeDiscoveryProvider.getForNodeRole(NodeRole.HISTORICAL),
druidNodeDiscoveryProvider.getForNodeRole(NodeRole.MIDDLE_MANAGER),
druidNodeDiscoveryProvider.getForNodeRole(NodeRole.INDEXER),
druidNodeDiscoveryProvider.getForNodeRole(NodeRole.BROKER),
druidNodeDiscoveryProvider.getForNodeRole(NodeRole.ROUTER)
);
I'm trying to build an extension. So this extension will run in every hosts
in our druid cluster. After getting the service details we wanted to some
further procession from our side.
Will really appreciate some pointers on this.
Thank you :)
Re: Get Druid Service details in runtime (via extension)
Posted by Jeet Patel <jp...@gmail.com>.
Awesome, it worked. Thank you Gian. Added @Self like below and getting the set of NodeRole
@Provides
@ManageLifecycle
@Named(EMITTER_TYPE)
public Emitter getEmitter(MyEmitterConfig myEmitterConfig, ObjectMapper mapper, @Self Set<NodeRole> nodeRoleSet)
{
return new MyEmitter(myConfig, mapper, nodeRoleSet);
}
Here is the log from Historical process:
2021-08-23T15:39:41,261 INFO [main] com.custom.MyEmitter - NodeRoleSet: [HISTORICAL]
Thank you again :)
On 2021/08/23 14:44:06, Gian Merlino <gi...@apache.org> wrote:
> Ah, if you're wanting to get the roles of the node that your own code is
> running on, try adding Set<NodeRole>. You might need the @Self annotation
> also.
>
> On Mon, Aug 23, 2021 at 6:27 AM Jeet Patel <jp...@gmail.com> wrote:
>
> > Hi Gian,
> >
> > Thank you for pointing that out. Although I'm getting following error when
> > using DiscoveryDruidNode
> >
> > // Declared a variable in MyEmitter
> > private final DiscoveryDruidNode discoveryDruidNode;
> >
> > // Added in Constructor
> > public MyEmitter(
> > MyEmitterConfig myEmitterConfig,
> > ObjectMapper mapper,
> > DiscoveryDruidNode discoveryDruidNode
> > )
> > {
> > this.mapper = mapper;
> > this.myEmitterConfig = myEmitterConfig;
> > this.whiteListMetricsMap =
> > readMap(myEmitterConfig.getWhiteListMetricsMapPath());
> > this.discoveryDruidNode = discoveryDruidNode;
> > log.info("Constructed MyEmitter");
> > }
> >
> > // Added a log.info in emit() method just to check if it works
> > log.info("NodeRole: " + discoveryDruidNode.getNodeRole().getJsonName());
> >
> > Below is the error I'm getting:
> > 1) Could not find a suitable constructor in
> > org.apache.druid.discovery.DiscoveryDruidNode. Classes must have either one
> > (and only one) constructor annotated with @Inject
> > or a zero-argument constructor that is not private.
> > at
> > org.apache.druid.discovery.DiscoveryDruidNode.class(DiscoveryDruidNode.java:47)
> > while locating org.apache.druid.discovery.DiscoveryDruidNode
> > for the 3rd parameter of
> > com.custom.MyEmitterModule.getEmitter(MyEmitterModule.java:39)
> >
> > According to the error, it looks like I cannot add DiscoveryDruidNode
> > because it does not have @Inject or a zero-argument constructor. But I'm
> > able to ad my MyEmitterConfig class which does not have zero-argument
> > constructor.
> >
> > On 2021/08/22 23:40:08, Gian Merlino <gi...@apache.org> wrote:
> > > Does the "getNodeRole()" method on DiscoveryDruidNode do what you want?
> > >
> > > On Fri, Aug 20, 2021 at 3:07 PM Jeet Patel <jp...@gmail.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > Is there a way to to know what druid services are running in a
> > DruidNode
> > > > (Not
> > > > talking about the HTTP APIs)?
> > > > I went through druid-server module, class
> > > > DruidNodeDiscoveryProvider.getForNodeRole which accepts a NodeRole and
> > > > returns a DruidNodeDiscovery instance after which we can use
> > > > getAllNodes() method
> > > > which returns Collection<DiscoveryDruidNode>. And for each item in the
> > > > Collection<DiscoveryDruidNode> we can use getServiceName() method to
> > get
> > > > the service name.
> > > >
> > > > The question is, how can we get the instance of NodeRole running in the
> > > > druid process. For example, if we have a host running broker service,
> > is
> > > > there a way to get NodeRole for broker process dynamically?
> > > >
> > > > For now I'm doing something like this. Adding all NodeRole in every
> > host,
> > > > since our extension runs in every host.:
> > > >
> > > > List<DruidNodeDiscovery> druidNodeDiscoveryList = ImmutableList.of(
> > > >
> > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.COORDINATOR),
> > > > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.OVERLORD),
> > > > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.HISTORICAL),
> > > >
> > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.MIDDLE_MANAGER),
> > > > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.INDEXER),
> > > > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.BROKER),
> > > > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.ROUTER)
> > > > );
> > > >
> > > > I'm trying to build an extension. So this extension will run in every
> > hosts
> > > > in our druid cluster. After getting the service details we wanted to
> > some
> > > > further procession from our side.
> > > >
> > > > Will really appreciate some pointers on this.
> > > >
> > > > Thank you :)
> > > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > For additional commands, e-mail: dev-help@druid.apache.org
> >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
For additional commands, e-mail: dev-help@druid.apache.org
Re: Get Druid Service details in runtime (via extension)
Posted by Gian Merlino <gi...@apache.org>.
Ah, if you're wanting to get the roles of the node that your own code is
running on, try adding Set<NodeRole>. You might need the @Self annotation
also.
On Mon, Aug 23, 2021 at 6:27 AM Jeet Patel <jp...@gmail.com> wrote:
> Hi Gian,
>
> Thank you for pointing that out. Although I'm getting following error when
> using DiscoveryDruidNode
>
> // Declared a variable in MyEmitter
> private final DiscoveryDruidNode discoveryDruidNode;
>
> // Added in Constructor
> public MyEmitter(
> MyEmitterConfig myEmitterConfig,
> ObjectMapper mapper,
> DiscoveryDruidNode discoveryDruidNode
> )
> {
> this.mapper = mapper;
> this.myEmitterConfig = myEmitterConfig;
> this.whiteListMetricsMap =
> readMap(myEmitterConfig.getWhiteListMetricsMapPath());
> this.discoveryDruidNode = discoveryDruidNode;
> log.info("Constructed MyEmitter");
> }
>
> // Added a log.info in emit() method just to check if it works
> log.info("NodeRole: " + discoveryDruidNode.getNodeRole().getJsonName());
>
> Below is the error I'm getting:
> 1) Could not find a suitable constructor in
> org.apache.druid.discovery.DiscoveryDruidNode. Classes must have either one
> (and only one) constructor annotated with @Inject
> or a zero-argument constructor that is not private.
> at
> org.apache.druid.discovery.DiscoveryDruidNode.class(DiscoveryDruidNode.java:47)
> while locating org.apache.druid.discovery.DiscoveryDruidNode
> for the 3rd parameter of
> com.custom.MyEmitterModule.getEmitter(MyEmitterModule.java:39)
>
> According to the error, it looks like I cannot add DiscoveryDruidNode
> because it does not have @Inject or a zero-argument constructor. But I'm
> able to ad my MyEmitterConfig class which does not have zero-argument
> constructor.
>
> On 2021/08/22 23:40:08, Gian Merlino <gi...@apache.org> wrote:
> > Does the "getNodeRole()" method on DiscoveryDruidNode do what you want?
> >
> > On Fri, Aug 20, 2021 at 3:07 PM Jeet Patel <jp...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > Is there a way to to know what druid services are running in a
> DruidNode
> > > (Not
> > > talking about the HTTP APIs)?
> > > I went through druid-server module, class
> > > DruidNodeDiscoveryProvider.getForNodeRole which accepts a NodeRole and
> > > returns a DruidNodeDiscovery instance after which we can use
> > > getAllNodes() method
> > > which returns Collection<DiscoveryDruidNode>. And for each item in the
> > > Collection<DiscoveryDruidNode> we can use getServiceName() method to
> get
> > > the service name.
> > >
> > > The question is, how can we get the instance of NodeRole running in the
> > > druid process. For example, if we have a host running broker service,
> is
> > > there a way to get NodeRole for broker process dynamically?
> > >
> > > For now I'm doing something like this. Adding all NodeRole in every
> host,
> > > since our extension runs in every host.:
> > >
> > > List<DruidNodeDiscovery> druidNodeDiscoveryList = ImmutableList.of(
> > >
> druidNodeDiscoveryProvider.getForNodeRole(NodeRole.COORDINATOR),
> > > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.OVERLORD),
> > > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.HISTORICAL),
> > >
> druidNodeDiscoveryProvider.getForNodeRole(NodeRole.MIDDLE_MANAGER),
> > > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.INDEXER),
> > > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.BROKER),
> > > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.ROUTER)
> > > );
> > >
> > > I'm trying to build an extension. So this extension will run in every
> hosts
> > > in our druid cluster. After getting the service details we wanted to
> some
> > > further procession from our side.
> > >
> > > Will really appreciate some pointers on this.
> > >
> > > Thank you :)
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> For additional commands, e-mail: dev-help@druid.apache.org
>
>
Re: Get Druid Service details in runtime (via extension)
Posted by Jeet Patel <jp...@gmail.com>.
Hi Gian,
Thank you for pointing that out. Although I'm getting following error when using DiscoveryDruidNode
// Declared a variable in MyEmitter
private final DiscoveryDruidNode discoveryDruidNode;
// Added in Constructor
public MyEmitter(
MyEmitterConfig myEmitterConfig,
ObjectMapper mapper,
DiscoveryDruidNode discoveryDruidNode
)
{
this.mapper = mapper;
this.myEmitterConfig = myEmitterConfig;
this.whiteListMetricsMap = readMap(myEmitterConfig.getWhiteListMetricsMapPath());
this.discoveryDruidNode = discoveryDruidNode;
log.info("Constructed MyEmitter");
}
// Added a log.info in emit() method just to check if it works
log.info("NodeRole: " + discoveryDruidNode.getNodeRole().getJsonName());
Below is the error I'm getting:
1) Could not find a suitable constructor in org.apache.druid.discovery.DiscoveryDruidNode. Classes must have either one (and only one) constructor annotated with @Inject
or a zero-argument constructor that is not private.
at org.apache.druid.discovery.DiscoveryDruidNode.class(DiscoveryDruidNode.java:47)
while locating org.apache.druid.discovery.DiscoveryDruidNode
for the 3rd parameter of com.custom.MyEmitterModule.getEmitter(MyEmitterModule.java:39)
According to the error, it looks like I cannot add DiscoveryDruidNode because it does not have @Inject or a zero-argument constructor. But I'm able to ad my MyEmitterConfig class which does not have zero-argument constructor.
On 2021/08/22 23:40:08, Gian Merlino <gi...@apache.org> wrote:
> Does the "getNodeRole()" method on DiscoveryDruidNode do what you want?
>
> On Fri, Aug 20, 2021 at 3:07 PM Jeet Patel <jp...@gmail.com> wrote:
>
> > Hi all,
> >
> > Is there a way to to know what druid services are running in a DruidNode
> > (Not
> > talking about the HTTP APIs)?
> > I went through druid-server module, class
> > DruidNodeDiscoveryProvider.getForNodeRole which accepts a NodeRole and
> > returns a DruidNodeDiscovery instance after which we can use
> > getAllNodes() method
> > which returns Collection<DiscoveryDruidNode>. And for each item in the
> > Collection<DiscoveryDruidNode> we can use getServiceName() method to get
> > the service name.
> >
> > The question is, how can we get the instance of NodeRole running in the
> > druid process. For example, if we have a host running broker service, is
> > there a way to get NodeRole for broker process dynamically?
> >
> > For now I'm doing something like this. Adding all NodeRole in every host,
> > since our extension runs in every host.:
> >
> > List<DruidNodeDiscovery> druidNodeDiscoveryList = ImmutableList.of(
> > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.COORDINATOR),
> > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.OVERLORD),
> > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.HISTORICAL),
> > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.MIDDLE_MANAGER),
> > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.INDEXER),
> > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.BROKER),
> > druidNodeDiscoveryProvider.getForNodeRole(NodeRole.ROUTER)
> > );
> >
> > I'm trying to build an extension. So this extension will run in every hosts
> > in our druid cluster. After getting the service details we wanted to some
> > further procession from our side.
> >
> > Will really appreciate some pointers on this.
> >
> > Thank you :)
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
For additional commands, e-mail: dev-help@druid.apache.org
Re: Get Druid Service details in runtime (via extension)
Posted by Gian Merlino <gi...@apache.org>.
Does the "getNodeRole()" method on DiscoveryDruidNode do what you want?
On Fri, Aug 20, 2021 at 3:07 PM Jeet Patel <jp...@gmail.com> wrote:
> Hi all,
>
> Is there a way to to know what druid services are running in a DruidNode
> (Not
> talking about the HTTP APIs)?
> I went through druid-server module, class
> DruidNodeDiscoveryProvider.getForNodeRole which accepts a NodeRole and
> returns a DruidNodeDiscovery instance after which we can use
> getAllNodes() method
> which returns Collection<DiscoveryDruidNode>. And for each item in the
> Collection<DiscoveryDruidNode> we can use getServiceName() method to get
> the service name.
>
> The question is, how can we get the instance of NodeRole running in the
> druid process. For example, if we have a host running broker service, is
> there a way to get NodeRole for broker process dynamically?
>
> For now I'm doing something like this. Adding all NodeRole in every host,
> since our extension runs in every host.:
>
> List<DruidNodeDiscovery> druidNodeDiscoveryList = ImmutableList.of(
> druidNodeDiscoveryProvider.getForNodeRole(NodeRole.COORDINATOR),
> druidNodeDiscoveryProvider.getForNodeRole(NodeRole.OVERLORD),
> druidNodeDiscoveryProvider.getForNodeRole(NodeRole.HISTORICAL),
> druidNodeDiscoveryProvider.getForNodeRole(NodeRole.MIDDLE_MANAGER),
> druidNodeDiscoveryProvider.getForNodeRole(NodeRole.INDEXER),
> druidNodeDiscoveryProvider.getForNodeRole(NodeRole.BROKER),
> druidNodeDiscoveryProvider.getForNodeRole(NodeRole.ROUTER)
> );
>
> I'm trying to build an extension. So this extension will run in every hosts
> in our druid cluster. After getting the service details we wanted to some
> further procession from our side.
>
> Will really appreciate some pointers on this.
>
> Thank you :)
>