You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by Kristopher Kane <kr...@gmail.com> on 2015/06/16 17:04:12 UTC
Re: New service definitions with non-standard topology role hostnames.
Kevin,
ServiceRegistryFunctionProcessorBase.lookupServiceUrl() and subclasses -
Just to verify, these are added at Knox startup to register the end point
URL with the role name. Is lookupServiceUrl() only called at startup and
the HA Provider there is simply to provide a starting point URL for HA
services?
I was thinking to add the Solr provider there but want to ensure that these
checks are not happening with each call. I don't believe they are as the
WebHDFS HA provider is doing work elsewhere.
Kris
On Thu, May 28, 2015 at 10:21 PM, Kristopher Kane <kristopher.kane@gmail.com
> wrote:
> Thanks for the detailed response Kevin. I will give it a shot tonight.
>
> NowŠ The outstanding question for me is the list is Zookeeper URLs. We
>> may want to treat them like we do the NAMENODE and JOBTRACKER services
>> today which are in the topology (and therefore in the service registry)
>> but not really exposed.
>>
>
> That was my intention.
>
> Thanks,
> Kris
>
>
>
Re: New service definitions with non-standard topology role hostnames.
Posted by Kristopher Kane <kr...@gmail.com>.
Thanks Sumit. I will take the HAProvider as a template and create a new
one.
Kris
On Tue, Jun 16, 2015 at 2:49 PM, Sumit Gupta <su...@hortonworks.com>
wrote:
> I believe it is these two lines:
>
> inboundRequest.setAttribute(AbstractGatewayFilter.TARGET_REQUEST_URL_ATTRIB
> UTE_NAME, null);
> URI uri = getDispatchUrl(inboundRequest);
>
>
> But it has been a while and I remember spending some time in the debugger
> to figure out how to connect the dots.
>
> Sumit
>
> On 6/16/15, 12:51 PM, "Kristopher Kane" <kk...@hortonworks.com> wrote:
>
> >Sumit,
> >
> >Haven't determined if the HA provider will do what I need yet but it
> >isn't looking that way.
> >
> >I am unable to figure out what in failoverRequest links back to the
> >service registry lookupServiceUrl() could you point it out to me?
> >
> >Thanks,
> >
> >Kris
> >
> >From: Sumit Gupta
> ><su...@hortonworks.com>>
> >Reply-To: "dev@knox.apache.org<ma...@knox.apache.org>"
> ><de...@knox.apache.org>>
> >Date: Tuesday, June 16, 2015 at 10:43 AM
> >To: "dev@knox.apache.org<ma...@knox.apache.org>"
> ><de...@knox.apache.org>>
> >Subject: Re: New service definitions with non-standard topology role
> >hostnames.
> >
> >Hey Kris,
> >
> >The class WebHdfsHaDispatch and method failoverRequest has some code in
> >there that causes the method lookupServiceUrl() to be called again.
> >
> >The HA Provider mainly manages the services that have HA capabilities
> >using the HA configuration and the service registry's URL information.
> >There are thoughts I have around enhancing the service registry to have
> >some more the multiple url management capabilities, but for now hopefully
> >the HA provider can do what you need. Are you looking to add a special
> >dispatch and use the HA provider or add a provider as well for Solr?
> >
> >Sumit.
> >
> >On 6/16/15, 11:04 AM, "Kristopher Kane"
> ><kr...@gmail.com>> wrote:
> >
> >Kevin,
> >
> >ServiceRegistryFunctionProcessorBase.lookupServiceUrl() and subclasses -
> >Just to verify, these are added at Knox startup to register the end point
> >URL with the role name. Is lookupServiceUrl() only called at startup and
> >the HA Provider there is simply to provide a starting point URL for HA
> >services?
> >
> >I was thinking to add the Solr provider there but want to ensure that
> >these
> >checks are not happening with each call. I don't believe they are as the
> >WebHDFS HA provider is doing work elsewhere.
> >
> >Kris
> >
> >On Thu, May 28, 2015 at 10:21 PM, Kristopher Kane
> ><kr...@gmail.com>
> >wrote:
> >
> >Thanks for the detailed response Kevin. I will give it a shot tonight.
> >
> >NowŠ The outstanding question for me is the list is Zookeeper URLs. We
> >may want to treat them like we do the NAMENODE and JOBTRACKER services
> >today which are in the topology (and therefore in the service registry)
> >but not really exposed.
> >
> >
> >That was my intention.
> >
> >Thanks,
> >Kris
> >
> >
> >
> >
> >
>
>
Re: New service definitions with non-standard topology role
hostnames.
Posted by Sumit Gupta <su...@hortonworks.com>.
I believe it is these two lines:
inboundRequest.setAttribute(AbstractGatewayFilter.TARGET_REQUEST_URL_ATTRIB
UTE_NAME, null);
URI uri = getDispatchUrl(inboundRequest);
But it has been a while and I remember spending some time in the debugger
to figure out how to connect the dots.
Sumit
On 6/16/15, 12:51 PM, "Kristopher Kane" <kk...@hortonworks.com> wrote:
>Sumit,
>
>Haven't determined if the HA provider will do what I need yet but it
>isn't looking that way.
>
>I am unable to figure out what in failoverRequest links back to the
>service registry lookupServiceUrl() could you point it out to me?
>
>Thanks,
>
>Kris
>
>From: Sumit Gupta
><su...@hortonworks.com>>
>Reply-To: "dev@knox.apache.org<ma...@knox.apache.org>"
><de...@knox.apache.org>>
>Date: Tuesday, June 16, 2015 at 10:43 AM
>To: "dev@knox.apache.org<ma...@knox.apache.org>"
><de...@knox.apache.org>>
>Subject: Re: New service definitions with non-standard topology role
>hostnames.
>
>Hey Kris,
>
>The class WebHdfsHaDispatch and method failoverRequest has some code in
>there that causes the method lookupServiceUrl() to be called again.
>
>The HA Provider mainly manages the services that have HA capabilities
>using the HA configuration and the service registry's URL information.
>There are thoughts I have around enhancing the service registry to have
>some more the multiple url management capabilities, but for now hopefully
>the HA provider can do what you need. Are you looking to add a special
>dispatch and use the HA provider or add a provider as well for Solr?
>
>Sumit.
>
>On 6/16/15, 11:04 AM, "Kristopher Kane"
><kr...@gmail.com>> wrote:
>
>Kevin,
>
>ServiceRegistryFunctionProcessorBase.lookupServiceUrl() and subclasses -
>Just to verify, these are added at Knox startup to register the end point
>URL with the role name. Is lookupServiceUrl() only called at startup and
>the HA Provider there is simply to provide a starting point URL for HA
>services?
>
>I was thinking to add the Solr provider there but want to ensure that
>these
>checks are not happening with each call. I don't believe they are as the
>WebHDFS HA provider is doing work elsewhere.
>
>Kris
>
>On Thu, May 28, 2015 at 10:21 PM, Kristopher Kane
><kr...@gmail.com>
>wrote:
>
>Thanks for the detailed response Kevin. I will give it a shot tonight.
>
>NowŠ The outstanding question for me is the list is Zookeeper URLs. We
>may want to treat them like we do the NAMENODE and JOBTRACKER services
>today which are in the topology (and therefore in the service registry)
>but not really exposed.
>
>
>That was my intention.
>
>Thanks,
>Kris
>
>
>
>
>
Re: New service definitions with non-standard topology role
hostnames.
Posted by Kristopher Kane <kk...@hortonworks.com>.
Sumit,
Haven't determined if the HA provider will do what I need yet but it isn't looking that way.
I am unable to figure out what in failoverRequest links back to the service registry lookupServiceUrl() could you point it out to me?
Thanks,
Kris
From: Sumit Gupta <su...@hortonworks.com>>
Reply-To: "dev@knox.apache.org<ma...@knox.apache.org>" <de...@knox.apache.org>>
Date: Tuesday, June 16, 2015 at 10:43 AM
To: "dev@knox.apache.org<ma...@knox.apache.org>" <de...@knox.apache.org>>
Subject: Re: New service definitions with non-standard topology role hostnames.
Hey Kris,
The class WebHdfsHaDispatch and method failoverRequest has some code in
there that causes the method lookupServiceUrl() to be called again.
The HA Provider mainly manages the services that have HA capabilities
using the HA configuration and the service registry's URL information.
There are thoughts I have around enhancing the service registry to have
some more the multiple url management capabilities, but for now hopefully
the HA provider can do what you need. Are you looking to add a special
dispatch and use the HA provider or add a provider as well for Solr?
Sumit.
On 6/16/15, 11:04 AM, "Kristopher Kane" <kr...@gmail.com>> wrote:
Kevin,
ServiceRegistryFunctionProcessorBase.lookupServiceUrl() and subclasses -
Just to verify, these are added at Knox startup to register the end point
URL with the role name. Is lookupServiceUrl() only called at startup and
the HA Provider there is simply to provide a starting point URL for HA
services?
I was thinking to add the Solr provider there but want to ensure that
these
checks are not happening with each call. I don't believe they are as the
WebHDFS HA provider is doing work elsewhere.
Kris
On Thu, May 28, 2015 at 10:21 PM, Kristopher Kane
<kr...@gmail.com>
wrote:
Thanks for the detailed response Kevin. I will give it a shot tonight.
NowŠ The outstanding question for me is the list is Zookeeper URLs. We
may want to treat them like we do the NAMENODE and JOBTRACKER services
today which are in the topology (and therefore in the service registry)
but not really exposed.
That was my intention.
Thanks,
Kris
Re: New service definitions with non-standard topology role
hostnames.
Posted by Sumit Gupta <su...@hortonworks.com>.
Hey Kris,
The class WebHdfsHaDispatch and method failoverRequest has some code in
there that causes the method lookupServiceUrl() to be called again.
The HA Provider mainly manages the services that have HA capabilities
using the HA configuration and the service registry's URL information.
There are thoughts I have around enhancing the service registry to have
some more the multiple url management capabilities, but for now hopefully
the HA provider can do what you need. Are you looking to add a special
dispatch and use the HA provider or add a provider as well for Solr?
Sumit.
On 6/16/15, 11:04 AM, "Kristopher Kane" <kr...@gmail.com> wrote:
>Kevin,
>
>ServiceRegistryFunctionProcessorBase.lookupServiceUrl() and subclasses -
>Just to verify, these are added at Knox startup to register the end point
>URL with the role name. Is lookupServiceUrl() only called at startup and
>the HA Provider there is simply to provide a starting point URL for HA
>services?
>
>I was thinking to add the Solr provider there but want to ensure that
>these
>checks are not happening with each call. I don't believe they are as the
>WebHDFS HA provider is doing work elsewhere.
>
>Kris
>
>On Thu, May 28, 2015 at 10:21 PM, Kristopher Kane
><kristopher.kane@gmail.com
>> wrote:
>
>> Thanks for the detailed response Kevin. I will give it a shot tonight.
>>
>> NowŠ The outstanding question for me is the list is Zookeeper URLs. We
>>> may want to treat them like we do the NAMENODE and JOBTRACKER services
>>> today which are in the topology (and therefore in the service registry)
>>> but not really exposed.
>>>
>>
>> That was my intention.
>>
>> Thanks,
>> Kris
>>
>>
>>