You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by Yong Feng <fe...@gmail.com> on 2015/01/02 04:13:22 UTC
Re: How Sliders configure configuration file of regionserver of HBase
Jon, Ted
Happy new year.
After read more, I understand knowing the host list or single host of
master role in Slider script or template is not supported by either Slider
or YARN.
So what I can do is
a) pre-define a group of hosts with label of master
b) specify label of master for master role in app resource specfication
c) get host list with label of master in template/scripts when deploying
slave role
However I did not find doc how to get host list with label of master in
Slider template/scripts like rs_hosts as follows
{% for host in rs_hosts %}{{host}}
{% endfor %}
Any suggestion?
Thanks,
Yong
On Wed, Dec 31, 2014 at 5:18 PM, Yong Feng <fe...@gmail.com> wrote:
> Hi John
>
> I do expect the hosts allocated to master role of my slider application
> are labeled as appname_rolename, but the question is who will add such
> label and how? and how to get the hosts with the label?
>
> Thanks,
>
> Yong
>
> On Wed, Dec 31, 2014 at 3:57 PM, Jon Maron <jm...@hortonworks.com> wrote:
>
>> I’m confused - if certain hosts were labeled as MASTER, wouldn’t you know
>> a priori the list of master hosts?
>>
>>
>> On Dec 31, 2014, at 12:31 PM, Yong Feng <fe...@gmail.com> wrote:
>>
>> > Thanks Ted.
>> >
>> >>> knowing the host list of master role in Slider script or template is
>> not
>> > supported by either Slider or YARN.
>> > [Yong] If there is only one instance of master role, at least I could
>> know
>> > its host name by COMPONENT_HOST in slave hosts, right? If so, I could
>> work
>> > it around to just configure one master.
>> >
>> > Yong
>> >
>> > On Tue, Dec 30, 2014 at 10:59 PM, Ted Yu <yu...@gmail.com> wrote:
>> >
>> >> What's the timeframe for this migration ?
>> >>
>> >> If time permits, you should consider rewriting master failover logic in
>> >> your application.
>> >> Using zookeeper is a common technique. HBase, e.g., registers active
>> master
>> >> under a well known zookeeper znode.
>> >>
>> >> This is much more flexible than the way you described.
>> >>
>> >> Again, knowing the host list of master role in Slider script or
>> template is
>> >> not supported by either Slider or YARN.
>> >>
>> >> Cheers
>> >>
>> >> On Tue, Dec 30, 2014 at 7:50 PM, Yong Feng <fe...@gmail.com>
>> wrote:
>> >>
>> >>> ah, rs_hosts means all the nodes managed by YARN.
>> >>>
>> >>> I know a little about labels feature of YARN which adds a tag to host
>> and
>> >>> allow yarn API client to filter hosts according to the tag. I don't
>> think
>> >>> it helps my case.
>> >>>
>> >>> Taking a typical master/slave cluster as an example. To support
>> >>> active-passive fail over, we need define multiple instances for master
>> >> role
>> >>> (unfortunately it does not use Zookeeper as the cluster is implemented
>> >> 10+
>> >>> years ago before Zookeeper). In slave node, a list of mast host are
>> >>> specified in conf file, so that slave node could contact master.
>> >>>
>> >>> To deploy such cluster, we need know host list of master and configure
>> >> them
>> >>> in the conf file of slave. Changing membership of master need restart
>> >> whole
>> >>> cluster so we will not support flex up/down role of master.
>> >>>
>> >>> How can I know the host list of master role in Slider script or
>> template?
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Yong
>> >>>
>> >>> On Tue, Dec 30, 2014 at 9:08 PM, Ted Yu <yu...@gmail.com> wrote:
>> >>>
>> >>>> bq. host list of all the instances of a component
>> >>>>
>> >>>> Are you using node labels feature of YARN ?
>> >>>> If not, the list would comprise of all the nodes in your YARN
>> cluster.
>> >>>>
>> >>>> Cheers
>> >>>>
>> >>>> On Tue, Dec 30, 2014 at 4:43 PM, Yong Feng <fe...@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>>> Hi Team,
>> >>>>>
>> >>>>> The template of regionserver of HBase application uses following
>> >>> scripts
>> >>>> to
>> >>>>> configure configuraiton file of regionserver of HBase. I did not
>> find
>> >>>> where
>> >>>>> the rs_hosts is defined in any file in Slider HBase package. How a
>> >>>> template
>> >>>>> is designed is mentioned a little at the end of
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> http://slider.incubator.apache.org/docs/slider_specs/application_package.html
>> >>>>> ,
>> >>>>> but cannot help much.
>> >>>>>
>> >>>>> {% for host in rs_hosts %}{{host}}
>> >>>>> {% endfor %}
>> >>>>>
>> >>>>> Does anyone know where rs_hosts is defined? I am now writing a
>> >> package
>> >>>> for
>> >>>>> a HPC applicaiton and need to know host list of all the instances of
>> >> a
>> >>>>> component to prepare configuration file of the application.
>> >>>>>
>> >>>>> Thanks,
>> >>>>>
>> >>>>> Yong
>> >>>>>
>> >>>>
>> >>>
>> >>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>
Re: How Sliders configure configuration file of regionserver of HBase
Posted by Yong Feng <fe...@gmail.com>.
Thanks Ted.
I will check if I could get some resources to contribute.
Yong
On Sat, Jan 3, 2015 at 2:13 PM, Ted Yu <yu...@gmail.com> wrote:
> YARN-3003 was logged.
> But it is not known when it will be implemented.
>
> Yong:
> Please consider using zookeeper for recoding master nodes. This makes your
> solution workable on Slider 0.60
>
> Cheers
>
> On Fri, Jan 2, 2015 at 10:33 AM, Wangda Tan <wt...@hortonworks.com> wrote:
>
> > Currently, YARN doesn't have direct method to do that, but you can get a
> > label-to-nodes mapping using YarnClient.getNodeToLabels().
> >
> > We can consider to add a method label-to-nodes mapping in future release
> > if necessary.
> >
> > Thanks,
> > Wangda
> >
> > On Fri, Jan 2, 2015 at 9:11 AM, Ted Yu <yu...@gmail.com> wrote:
> >
> >> Thanks for the background information.
> >>
> >> Adding Wangda who worked on node labels feature.
> >>
> >> On Fri, Jan 2, 2015 at 9:04 AM, Yong Feng <fe...@gmail.com>
> wrote:
> >>
> >>> Thanks Ted.
> >>>
> >>> Basically what we are doing is to ask HPC workload and BigData workload
> >>> share the same infrastructure by Slider. The current way to fit Hadoop
> >>> into
> >>> HPC community is either on demand provisioning Hadoop cluster by HPC
> >>> scheduler (for example HOD or myHadoop) or enhancing Hadoop as HPC
> >>> workload
> >>> scheduler directly (quite a lot papers).
> >>>
> >>> Besides them, we actually could ask Yarn/Slider as resource manager to
> >>> manage both HPC wlm and BigData wlm. Mesos/Marathon is another choice
> >>> however I prefer Yarn/Slider due to its BigData DNA.
> >>>
> >>> Looking forward to getting more support from Slider and Yarn team.
> >>>
> >>> Thanks,
> >>>
> >>> Yong
> >>>
> >>>
> >>> On Thu, Jan 1, 2015 at 10:31 PM, Ted Yu <yu...@gmail.com> wrote:
> >>>
> >>> > Yong:
> >>> > Let me discuss with slider / YARN developers and get back to you.
> >>> >
> >>> > Happy new year.
> >>> >
> >>> > On Thu, Jan 1, 2015 at 7:13 PM, Yong Feng <fe...@gmail.com>
> >>> wrote:
> >>> >
> >>> > > Jon, Ted
> >>> > >
> >>> > > Happy new year.
> >>> > >
> >>> > > After read more, I understand knowing the host list or single host
> of
> >>> > > master role in Slider script or template is not supported by either
> >>> > Slider
> >>> > > or YARN.
> >>> > >
> >>> > > So what I can do is
> >>> > >
> >>> > > a) pre-define a group of hosts with label of master
> >>> > > b) specify label of master for master role in app resource
> >>> specfication
> >>> > > c) get host list with label of master in template/scripts when
> >>> deploying
> >>> > > slave role
> >>> > >
> >>> > > However I did not find doc how to get host list with label of
> master
> >>> in
> >>> > > Slider template/scripts like rs_hosts as follows
> >>> > >
> >>> > > {% for host in rs_hosts %}{{host}}
> >>> > > {% endfor %}
> >>> > >
> >>> > > Any suggestion?
> >>> > >
> >>> > > Thanks,
> >>> > >
> >>> > > Yong
> >>> > >
> >>> > >
> >>> > >
> >>> > >
> >>> > >
> >>> > >
> >>> > > On Wed, Dec 31, 2014 at 5:18 PM, Yong Feng <fe...@gmail.com>
> >>> > wrote:
> >>> > >
> >>> > > > Hi John
> >>> > > >
> >>> > > > I do expect the hosts allocated to master role of my slider
> >>> application
> >>> > > > are labeled as appname_rolename, but the question is who will add
> >>> such
> >>> > > > label and how? and how to get the hosts with the label?
> >>> > > >
> >>> > > > Thanks,
> >>> > > >
> >>> > > > Yong
> >>> > > >
> >>> > > > On Wed, Dec 31, 2014 at 3:57 PM, Jon Maron <
> jmaron@hortonworks.com
> >>> >
> >>> > > wrote:
> >>> > > >
> >>> > > >> I’m confused - if certain hosts were labeled as MASTER, wouldn’t
> >>> you
> >>> > > know
> >>> > > >> a priori the list of master hosts?
> >>> > > >>
> >>> > > >>
> >>> > > >> On Dec 31, 2014, at 12:31 PM, Yong Feng <fe...@gmail.com>
> >>> > wrote:
> >>> > > >>
> >>> > > >> > Thanks Ted.
> >>> > > >> >
> >>> > > >> >>> knowing the host list of master role in Slider script or
> >>> template
> >>> > is
> >>> > > >> not
> >>> > > >> > supported by either Slider or YARN.
> >>> > > >> > [Yong] If there is only one instance of master role, at least
> I
> >>> > could
> >>> > > >> know
> >>> > > >> > its host name by COMPONENT_HOST in slave hosts, right? If so,
> I
> >>> > could
> >>> > > >> work
> >>> > > >> > it around to just configure one master.
> >>> > > >> >
> >>> > > >> > Yong
> >>> > > >> >
> >>> > > >> > On Tue, Dec 30, 2014 at 10:59 PM, Ted Yu <yuzhihong@gmail.com
> >
> >>> > wrote:
> >>> > > >> >
> >>> > > >> >> What's the timeframe for this migration ?
> >>> > > >> >>
> >>> > > >> >> If time permits, you should consider rewriting master
> failover
> >>> > logic
> >>> > > in
> >>> > > >> >> your application.
> >>> > > >> >> Using zookeeper is a common technique. HBase, e.g., registers
> >>> > active
> >>> > > >> master
> >>> > > >> >> under a well known zookeeper znode.
> >>> > > >> >>
> >>> > > >> >> This is much more flexible than the way you described.
> >>> > > >> >>
> >>> > > >> >> Again, knowing the host list of master role in Slider script
> or
> >>> > > >> template is
> >>> > > >> >> not supported by either Slider or YARN.
> >>> > > >> >>
> >>> > > >> >> Cheers
> >>> > > >> >>
> >>> > > >> >> On Tue, Dec 30, 2014 at 7:50 PM, Yong Feng <
> >>> fengyong.gm@gmail.com>
> >>> > > >> wrote:
> >>> > > >> >>
> >>> > > >> >>> ah, rs_hosts means all the nodes managed by YARN.
> >>> > > >> >>>
> >>> > > >> >>> I know a little about labels feature of YARN which adds a
> tag
> >>> to
> >>> > > host
> >>> > > >> and
> >>> > > >> >>> allow yarn API client to filter hosts according to the tag.
> I
> >>> > don't
> >>> > > >> think
> >>> > > >> >>> it helps my case.
> >>> > > >> >>>
> >>> > > >> >>> Taking a typical master/slave cluster as an example. To
> >>> support
> >>> > > >> >>> active-passive fail over, we need define multiple instances
> >>> for
> >>> > > master
> >>> > > >> >> role
> >>> > > >> >>> (unfortunately it does not use Zookeeper as the cluster is
> >>> > > implemented
> >>> > > >> >> 10+
> >>> > > >> >>> years ago before Zookeeper). In slave node, a list of mast
> >>> host
> >>> > are
> >>> > > >> >>> specified in conf file, so that slave node could contact
> >>> master.
> >>> > > >> >>>
> >>> > > >> >>> To deploy such cluster, we need know host list of master and
> >>> > > configure
> >>> > > >> >> them
> >>> > > >> >>> in the conf file of slave. Changing membership of master
> need
> >>> > > restart
> >>> > > >> >> whole
> >>> > > >> >>> cluster so we will not support flex up/down role of master.
> >>> > > >> >>>
> >>> > > >> >>> How can I know the host list of master role in Slider script
> >>> or
> >>> > > >> template?
> >>> > > >> >>>
> >>> > > >> >>> Thanks,
> >>> > > >> >>>
> >>> > > >> >>> Yong
> >>> > > >> >>>
> >>> > > >> >>> On Tue, Dec 30, 2014 at 9:08 PM, Ted Yu <
> yuzhihong@gmail.com>
> >>> > > wrote:
> >>> > > >> >>>
> >>> > > >> >>>> bq. host list of all the instances of a component
> >>> > > >> >>>>
> >>> > > >> >>>> Are you using node labels feature of YARN ?
> >>> > > >> >>>> If not, the list would comprise of all the nodes in your
> YARN
> >>> > > >> cluster.
> >>> > > >> >>>>
> >>> > > >> >>>> Cheers
> >>> > > >> >>>>
> >>> > > >> >>>> On Tue, Dec 30, 2014 at 4:43 PM, Yong Feng <
> >>> > fengyong.gm@gmail.com>
> >>> > > >> >>> wrote:
> >>> > > >> >>>>
> >>> > > >> >>>>> Hi Team,
> >>> > > >> >>>>>
> >>> > > >> >>>>> The template of regionserver of HBase application uses
> >>> following
> >>> > > >> >>> scripts
> >>> > > >> >>>> to
> >>> > > >> >>>>> configure configuraiton file of regionserver of HBase. I
> >>> did not
> >>> > > >> find
> >>> > > >> >>>> where
> >>> > > >> >>>>> the rs_hosts is defined in any file in Slider HBase
> package.
> >>> > How a
> >>> > > >> >>>> template
> >>> > > >> >>>>> is designed is mentioned a little at the end of
> >>> > > >> >>>>>
> >>> > > >> >>>>>
> >>> > > >> >>>>
> >>> > > >> >>>
> >>> > > >> >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
> http://slider.incubator.apache.org/docs/slider_specs/application_package.html
> >>> > > >> >>>>> ,
> >>> > > >> >>>>> but cannot help much.
> >>> > > >> >>>>>
> >>> > > >> >>>>> {% for host in rs_hosts %}{{host}}
> >>> > > >> >>>>> {% endfor %}
> >>> > > >> >>>>>
> >>> > > >> >>>>> Does anyone know where rs_hosts is defined? I am now
> >>> writing a
> >>> > > >> >> package
> >>> > > >> >>>> for
> >>> > > >> >>>>> a HPC applicaiton and need to know host list of all the
> >>> > instances
> >>> > > of
> >>> > > >> >> a
> >>> > > >> >>>>> component to prepare configuration file of the
> application.
> >>> > > >> >>>>>
> >>> > > >> >>>>> Thanks,
> >>> > > >> >>>>>
> >>> > > >> >>>>> Yong
> >>> > > >> >>>>>
> >>> > > >> >>>>
> >>> > > >> >>>
> >>> > > >> >>
> >>> > > >>
> >>> > > >>
> >>> > > >> --
> >>> > > >> CONFIDENTIALITY NOTICE
> >>> > > >> NOTICE: This message is intended for the use of the individual
> or
> >>> > entity
> >>> > > >> to
> >>> > > >> which it is addressed and may contain information that is
> >>> > confidential,
> >>> > > >> privileged and exempt from disclosure under applicable law. If
> the
> >>> > > reader
> >>> > > >> of this message is not the intended recipient, you are hereby
> >>> notified
> >>> > > >> that
> >>> > > >> any printing, copying, dissemination, distribution, disclosure
> or
> >>> > > >> forwarding of this communication is strictly prohibited. If you
> >>> have
> >>> > > >> received this communication in error, please contact the sender
> >>> > > >> immediately
> >>> > > >> and delete it from your system. Thank You.
> >>> > > >>
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> > to which it is addressed and may contain information that is
> confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
Re: How Sliders configure configuration file of regionserver of HBase
Posted by Ted Yu <yu...@gmail.com>.
YARN-3003 was logged.
But it is not known when it will be implemented.
Yong:
Please consider using zookeeper for recoding master nodes. This makes your
solution workable on Slider 0.60
Cheers
On Fri, Jan 2, 2015 at 10:33 AM, Wangda Tan <wt...@hortonworks.com> wrote:
> Currently, YARN doesn't have direct method to do that, but you can get a
> label-to-nodes mapping using YarnClient.getNodeToLabels().
>
> We can consider to add a method label-to-nodes mapping in future release
> if necessary.
>
> Thanks,
> Wangda
>
> On Fri, Jan 2, 2015 at 9:11 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Thanks for the background information.
>>
>> Adding Wangda who worked on node labels feature.
>>
>> On Fri, Jan 2, 2015 at 9:04 AM, Yong Feng <fe...@gmail.com> wrote:
>>
>>> Thanks Ted.
>>>
>>> Basically what we are doing is to ask HPC workload and BigData workload
>>> share the same infrastructure by Slider. The current way to fit Hadoop
>>> into
>>> HPC community is either on demand provisioning Hadoop cluster by HPC
>>> scheduler (for example HOD or myHadoop) or enhancing Hadoop as HPC
>>> workload
>>> scheduler directly (quite a lot papers).
>>>
>>> Besides them, we actually could ask Yarn/Slider as resource manager to
>>> manage both HPC wlm and BigData wlm. Mesos/Marathon is another choice
>>> however I prefer Yarn/Slider due to its BigData DNA.
>>>
>>> Looking forward to getting more support from Slider and Yarn team.
>>>
>>> Thanks,
>>>
>>> Yong
>>>
>>>
>>> On Thu, Jan 1, 2015 at 10:31 PM, Ted Yu <yu...@gmail.com> wrote:
>>>
>>> > Yong:
>>> > Let me discuss with slider / YARN developers and get back to you.
>>> >
>>> > Happy new year.
>>> >
>>> > On Thu, Jan 1, 2015 at 7:13 PM, Yong Feng <fe...@gmail.com>
>>> wrote:
>>> >
>>> > > Jon, Ted
>>> > >
>>> > > Happy new year.
>>> > >
>>> > > After read more, I understand knowing the host list or single host of
>>> > > master role in Slider script or template is not supported by either
>>> > Slider
>>> > > or YARN.
>>> > >
>>> > > So what I can do is
>>> > >
>>> > > a) pre-define a group of hosts with label of master
>>> > > b) specify label of master for master role in app resource
>>> specfication
>>> > > c) get host list with label of master in template/scripts when
>>> deploying
>>> > > slave role
>>> > >
>>> > > However I did not find doc how to get host list with label of master
>>> in
>>> > > Slider template/scripts like rs_hosts as follows
>>> > >
>>> > > {% for host in rs_hosts %}{{host}}
>>> > > {% endfor %}
>>> > >
>>> > > Any suggestion?
>>> > >
>>> > > Thanks,
>>> > >
>>> > > Yong
>>> > >
>>> > >
>>> > >
>>> > >
>>> > >
>>> > >
>>> > > On Wed, Dec 31, 2014 at 5:18 PM, Yong Feng <fe...@gmail.com>
>>> > wrote:
>>> > >
>>> > > > Hi John
>>> > > >
>>> > > > I do expect the hosts allocated to master role of my slider
>>> application
>>> > > > are labeled as appname_rolename, but the question is who will add
>>> such
>>> > > > label and how? and how to get the hosts with the label?
>>> > > >
>>> > > > Thanks,
>>> > > >
>>> > > > Yong
>>> > > >
>>> > > > On Wed, Dec 31, 2014 at 3:57 PM, Jon Maron <jmaron@hortonworks.com
>>> >
>>> > > wrote:
>>> > > >
>>> > > >> I’m confused - if certain hosts were labeled as MASTER, wouldn’t
>>> you
>>> > > know
>>> > > >> a priori the list of master hosts?
>>> > > >>
>>> > > >>
>>> > > >> On Dec 31, 2014, at 12:31 PM, Yong Feng <fe...@gmail.com>
>>> > wrote:
>>> > > >>
>>> > > >> > Thanks Ted.
>>> > > >> >
>>> > > >> >>> knowing the host list of master role in Slider script or
>>> template
>>> > is
>>> > > >> not
>>> > > >> > supported by either Slider or YARN.
>>> > > >> > [Yong] If there is only one instance of master role, at least I
>>> > could
>>> > > >> know
>>> > > >> > its host name by COMPONENT_HOST in slave hosts, right? If so, I
>>> > could
>>> > > >> work
>>> > > >> > it around to just configure one master.
>>> > > >> >
>>> > > >> > Yong
>>> > > >> >
>>> > > >> > On Tue, Dec 30, 2014 at 10:59 PM, Ted Yu <yu...@gmail.com>
>>> > wrote:
>>> > > >> >
>>> > > >> >> What's the timeframe for this migration ?
>>> > > >> >>
>>> > > >> >> If time permits, you should consider rewriting master failover
>>> > logic
>>> > > in
>>> > > >> >> your application.
>>> > > >> >> Using zookeeper is a common technique. HBase, e.g., registers
>>> > active
>>> > > >> master
>>> > > >> >> under a well known zookeeper znode.
>>> > > >> >>
>>> > > >> >> This is much more flexible than the way you described.
>>> > > >> >>
>>> > > >> >> Again, knowing the host list of master role in Slider script or
>>> > > >> template is
>>> > > >> >> not supported by either Slider or YARN.
>>> > > >> >>
>>> > > >> >> Cheers
>>> > > >> >>
>>> > > >> >> On Tue, Dec 30, 2014 at 7:50 PM, Yong Feng <
>>> fengyong.gm@gmail.com>
>>> > > >> wrote:
>>> > > >> >>
>>> > > >> >>> ah, rs_hosts means all the nodes managed by YARN.
>>> > > >> >>>
>>> > > >> >>> I know a little about labels feature of YARN which adds a tag
>>> to
>>> > > host
>>> > > >> and
>>> > > >> >>> allow yarn API client to filter hosts according to the tag. I
>>> > don't
>>> > > >> think
>>> > > >> >>> it helps my case.
>>> > > >> >>>
>>> > > >> >>> Taking a typical master/slave cluster as an example. To
>>> support
>>> > > >> >>> active-passive fail over, we need define multiple instances
>>> for
>>> > > master
>>> > > >> >> role
>>> > > >> >>> (unfortunately it does not use Zookeeper as the cluster is
>>> > > implemented
>>> > > >> >> 10+
>>> > > >> >>> years ago before Zookeeper). In slave node, a list of mast
>>> host
>>> > are
>>> > > >> >>> specified in conf file, so that slave node could contact
>>> master.
>>> > > >> >>>
>>> > > >> >>> To deploy such cluster, we need know host list of master and
>>> > > configure
>>> > > >> >> them
>>> > > >> >>> in the conf file of slave. Changing membership of master need
>>> > > restart
>>> > > >> >> whole
>>> > > >> >>> cluster so we will not support flex up/down role of master.
>>> > > >> >>>
>>> > > >> >>> How can I know the host list of master role in Slider script
>>> or
>>> > > >> template?
>>> > > >> >>>
>>> > > >> >>> Thanks,
>>> > > >> >>>
>>> > > >> >>> Yong
>>> > > >> >>>
>>> > > >> >>> On Tue, Dec 30, 2014 at 9:08 PM, Ted Yu <yu...@gmail.com>
>>> > > wrote:
>>> > > >> >>>
>>> > > >> >>>> bq. host list of all the instances of a component
>>> > > >> >>>>
>>> > > >> >>>> Are you using node labels feature of YARN ?
>>> > > >> >>>> If not, the list would comprise of all the nodes in your YARN
>>> > > >> cluster.
>>> > > >> >>>>
>>> > > >> >>>> Cheers
>>> > > >> >>>>
>>> > > >> >>>> On Tue, Dec 30, 2014 at 4:43 PM, Yong Feng <
>>> > fengyong.gm@gmail.com>
>>> > > >> >>> wrote:
>>> > > >> >>>>
>>> > > >> >>>>> Hi Team,
>>> > > >> >>>>>
>>> > > >> >>>>> The template of regionserver of HBase application uses
>>> following
>>> > > >> >>> scripts
>>> > > >> >>>> to
>>> > > >> >>>>> configure configuraiton file of regionserver of HBase. I
>>> did not
>>> > > >> find
>>> > > >> >>>> where
>>> > > >> >>>>> the rs_hosts is defined in any file in Slider HBase package.
>>> > How a
>>> > > >> >>>> template
>>> > > >> >>>>> is designed is mentioned a little at the end of
>>> > > >> >>>>>
>>> > > >> >>>>>
>>> > > >> >>>>
>>> > > >> >>>
>>> > > >> >>
>>> > > >>
>>> > >
>>> >
>>> http://slider.incubator.apache.org/docs/slider_specs/application_package.html
>>> > > >> >>>>> ,
>>> > > >> >>>>> but cannot help much.
>>> > > >> >>>>>
>>> > > >> >>>>> {% for host in rs_hosts %}{{host}}
>>> > > >> >>>>> {% endfor %}
>>> > > >> >>>>>
>>> > > >> >>>>> Does anyone know where rs_hosts is defined? I am now
>>> writing a
>>> > > >> >> package
>>> > > >> >>>> for
>>> > > >> >>>>> a HPC applicaiton and need to know host list of all the
>>> > instances
>>> > > of
>>> > > >> >> a
>>> > > >> >>>>> component to prepare configuration file of the application.
>>> > > >> >>>>>
>>> > > >> >>>>> Thanks,
>>> > > >> >>>>>
>>> > > >> >>>>> Yong
>>> > > >> >>>>>
>>> > > >> >>>>
>>> > > >> >>>
>>> > > >> >>
>>> > > >>
>>> > > >>
>>> > > >> --
>>> > > >> CONFIDENTIALITY NOTICE
>>> > > >> NOTICE: This message is intended for the use of the individual or
>>> > entity
>>> > > >> to
>>> > > >> which it is addressed and may contain information that is
>>> > confidential,
>>> > > >> privileged and exempt from disclosure under applicable law. If the
>>> > > reader
>>> > > >> of this message is not the intended recipient, you are hereby
>>> notified
>>> > > >> that
>>> > > >> any printing, copying, dissemination, distribution, disclosure or
>>> > > >> forwarding of this communication is strictly prohibited. If you
>>> have
>>> > > >> received this communication in error, please contact the sender
>>> > > >> immediately
>>> > > >> and delete it from your system. Thank You.
>>> > > >>
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: How Sliders configure configuration file of regionserver of HBase
Posted by Ted Yu <yu...@gmail.com>.
Thanks for the background information.
Adding Wangda who worked on node labels feature.
On Fri, Jan 2, 2015 at 9:04 AM, Yong Feng <fe...@gmail.com> wrote:
> Thanks Ted.
>
> Basically what we are doing is to ask HPC workload and BigData workload
> share the same infrastructure by Slider. The current way to fit Hadoop into
> HPC community is either on demand provisioning Hadoop cluster by HPC
> scheduler (for example HOD or myHadoop) or enhancing Hadoop as HPC workload
> scheduler directly (quite a lot papers).
>
> Besides them, we actually could ask Yarn/Slider as resource manager to
> manage both HPC wlm and BigData wlm. Mesos/Marathon is another choice
> however I prefer Yarn/Slider due to its BigData DNA.
>
> Looking forward to getting more support from Slider and Yarn team.
>
> Thanks,
>
> Yong
>
>
> On Thu, Jan 1, 2015 at 10:31 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Yong:
> > Let me discuss with slider / YARN developers and get back to you.
> >
> > Happy new year.
> >
> > On Thu, Jan 1, 2015 at 7:13 PM, Yong Feng <fe...@gmail.com> wrote:
> >
> > > Jon, Ted
> > >
> > > Happy new year.
> > >
> > > After read more, I understand knowing the host list or single host of
> > > master role in Slider script or template is not supported by either
> > Slider
> > > or YARN.
> > >
> > > So what I can do is
> > >
> > > a) pre-define a group of hosts with label of master
> > > b) specify label of master for master role in app resource specfication
> > > c) get host list with label of master in template/scripts when
> deploying
> > > slave role
> > >
> > > However I did not find doc how to get host list with label of master in
> > > Slider template/scripts like rs_hosts as follows
> > >
> > > {% for host in rs_hosts %}{{host}}
> > > {% endfor %}
> > >
> > > Any suggestion?
> > >
> > > Thanks,
> > >
> > > Yong
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Dec 31, 2014 at 5:18 PM, Yong Feng <fe...@gmail.com>
> > wrote:
> > >
> > > > Hi John
> > > >
> > > > I do expect the hosts allocated to master role of my slider
> application
> > > > are labeled as appname_rolename, but the question is who will add
> such
> > > > label and how? and how to get the hosts with the label?
> > > >
> > > > Thanks,
> > > >
> > > > Yong
> > > >
> > > > On Wed, Dec 31, 2014 at 3:57 PM, Jon Maron <jm...@hortonworks.com>
> > > wrote:
> > > >
> > > >> I’m confused - if certain hosts were labeled as MASTER, wouldn’t you
> > > know
> > > >> a priori the list of master hosts?
> > > >>
> > > >>
> > > >> On Dec 31, 2014, at 12:31 PM, Yong Feng <fe...@gmail.com>
> > wrote:
> > > >>
> > > >> > Thanks Ted.
> > > >> >
> > > >> >>> knowing the host list of master role in Slider script or
> template
> > is
> > > >> not
> > > >> > supported by either Slider or YARN.
> > > >> > [Yong] If there is only one instance of master role, at least I
> > could
> > > >> know
> > > >> > its host name by COMPONENT_HOST in slave hosts, right? If so, I
> > could
> > > >> work
> > > >> > it around to just configure one master.
> > > >> >
> > > >> > Yong
> > > >> >
> > > >> > On Tue, Dec 30, 2014 at 10:59 PM, Ted Yu <yu...@gmail.com>
> > wrote:
> > > >> >
> > > >> >> What's the timeframe for this migration ?
> > > >> >>
> > > >> >> If time permits, you should consider rewriting master failover
> > logic
> > > in
> > > >> >> your application.
> > > >> >> Using zookeeper is a common technique. HBase, e.g., registers
> > active
> > > >> master
> > > >> >> under a well known zookeeper znode.
> > > >> >>
> > > >> >> This is much more flexible than the way you described.
> > > >> >>
> > > >> >> Again, knowing the host list of master role in Slider script or
> > > >> template is
> > > >> >> not supported by either Slider or YARN.
> > > >> >>
> > > >> >> Cheers
> > > >> >>
> > > >> >> On Tue, Dec 30, 2014 at 7:50 PM, Yong Feng <
> fengyong.gm@gmail.com>
> > > >> wrote:
> > > >> >>
> > > >> >>> ah, rs_hosts means all the nodes managed by YARN.
> > > >> >>>
> > > >> >>> I know a little about labels feature of YARN which adds a tag to
> > > host
> > > >> and
> > > >> >>> allow yarn API client to filter hosts according to the tag. I
> > don't
> > > >> think
> > > >> >>> it helps my case.
> > > >> >>>
> > > >> >>> Taking a typical master/slave cluster as an example. To support
> > > >> >>> active-passive fail over, we need define multiple instances for
> > > master
> > > >> >> role
> > > >> >>> (unfortunately it does not use Zookeeper as the cluster is
> > > implemented
> > > >> >> 10+
> > > >> >>> years ago before Zookeeper). In slave node, a list of mast host
> > are
> > > >> >>> specified in conf file, so that slave node could contact master.
> > > >> >>>
> > > >> >>> To deploy such cluster, we need know host list of master and
> > > configure
> > > >> >> them
> > > >> >>> in the conf file of slave. Changing membership of master need
> > > restart
> > > >> >> whole
> > > >> >>> cluster so we will not support flex up/down role of master.
> > > >> >>>
> > > >> >>> How can I know the host list of master role in Slider script or
> > > >> template?
> > > >> >>>
> > > >> >>> Thanks,
> > > >> >>>
> > > >> >>> Yong
> > > >> >>>
> > > >> >>> On Tue, Dec 30, 2014 at 9:08 PM, Ted Yu <yu...@gmail.com>
> > > wrote:
> > > >> >>>
> > > >> >>>> bq. host list of all the instances of a component
> > > >> >>>>
> > > >> >>>> Are you using node labels feature of YARN ?
> > > >> >>>> If not, the list would comprise of all the nodes in your YARN
> > > >> cluster.
> > > >> >>>>
> > > >> >>>> Cheers
> > > >> >>>>
> > > >> >>>> On Tue, Dec 30, 2014 at 4:43 PM, Yong Feng <
> > fengyong.gm@gmail.com>
> > > >> >>> wrote:
> > > >> >>>>
> > > >> >>>>> Hi Team,
> > > >> >>>>>
> > > >> >>>>> The template of regionserver of HBase application uses
> following
> > > >> >>> scripts
> > > >> >>>> to
> > > >> >>>>> configure configuraiton file of regionserver of HBase. I did
> not
> > > >> find
> > > >> >>>> where
> > > >> >>>>> the rs_hosts is defined in any file in Slider HBase package.
> > How a
> > > >> >>>> template
> > > >> >>>>> is designed is mentioned a little at the end of
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > >
> >
> http://slider.incubator.apache.org/docs/slider_specs/application_package.html
> > > >> >>>>> ,
> > > >> >>>>> but cannot help much.
> > > >> >>>>>
> > > >> >>>>> {% for host in rs_hosts %}{{host}}
> > > >> >>>>> {% endfor %}
> > > >> >>>>>
> > > >> >>>>> Does anyone know where rs_hosts is defined? I am now writing a
> > > >> >> package
> > > >> >>>> for
> > > >> >>>>> a HPC applicaiton and need to know host list of all the
> > instances
> > > of
> > > >> >> a
> > > >> >>>>> component to prepare configuration file of the application.
> > > >> >>>>>
> > > >> >>>>> Thanks,
> > > >> >>>>>
> > > >> >>>>> Yong
> > > >> >>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >>
> > > >>
> > > >> --
> > > >> CONFIDENTIALITY NOTICE
> > > >> NOTICE: This message is intended for the use of the individual or
> > entity
> > > >> to
> > > >> which it is addressed and may contain information that is
> > confidential,
> > > >> privileged and exempt from disclosure under applicable law. If the
> > > reader
> > > >> of this message is not the intended recipient, you are hereby
> notified
> > > >> that
> > > >> any printing, copying, dissemination, distribution, disclosure or
> > > >> forwarding of this communication is strictly prohibited. If you have
> > > >> received this communication in error, please contact the sender
> > > >> immediately
> > > >> and delete it from your system. Thank You.
> > > >>
> > > >
> > > >
> > >
> >
>
Re: How Sliders configure configuration file of regionserver of HBase
Posted by Yong Feng <fe...@gmail.com>.
Thanks Ted.
Basically what we are doing is to ask HPC workload and BigData workload
share the same infrastructure by Slider. The current way to fit Hadoop into
HPC community is either on demand provisioning Hadoop cluster by HPC
scheduler (for example HOD or myHadoop) or enhancing Hadoop as HPC workload
scheduler directly (quite a lot papers).
Besides them, we actually could ask Yarn/Slider as resource manager to
manage both HPC wlm and BigData wlm. Mesos/Marathon is another choice
however I prefer Yarn/Slider due to its BigData DNA.
Looking forward to getting more support from Slider and Yarn team.
Thanks,
Yong
On Thu, Jan 1, 2015 at 10:31 PM, Ted Yu <yu...@gmail.com> wrote:
> Yong:
> Let me discuss with slider / YARN developers and get back to you.
>
> Happy new year.
>
> On Thu, Jan 1, 2015 at 7:13 PM, Yong Feng <fe...@gmail.com> wrote:
>
> > Jon, Ted
> >
> > Happy new year.
> >
> > After read more, I understand knowing the host list or single host of
> > master role in Slider script or template is not supported by either
> Slider
> > or YARN.
> >
> > So what I can do is
> >
> > a) pre-define a group of hosts with label of master
> > b) specify label of master for master role in app resource specfication
> > c) get host list with label of master in template/scripts when deploying
> > slave role
> >
> > However I did not find doc how to get host list with label of master in
> > Slider template/scripts like rs_hosts as follows
> >
> > {% for host in rs_hosts %}{{host}}
> > {% endfor %}
> >
> > Any suggestion?
> >
> > Thanks,
> >
> > Yong
> >
> >
> >
> >
> >
> >
> > On Wed, Dec 31, 2014 at 5:18 PM, Yong Feng <fe...@gmail.com>
> wrote:
> >
> > > Hi John
> > >
> > > I do expect the hosts allocated to master role of my slider application
> > > are labeled as appname_rolename, but the question is who will add such
> > > label and how? and how to get the hosts with the label?
> > >
> > > Thanks,
> > >
> > > Yong
> > >
> > > On Wed, Dec 31, 2014 at 3:57 PM, Jon Maron <jm...@hortonworks.com>
> > wrote:
> > >
> > >> I’m confused - if certain hosts were labeled as MASTER, wouldn’t you
> > know
> > >> a priori the list of master hosts?
> > >>
> > >>
> > >> On Dec 31, 2014, at 12:31 PM, Yong Feng <fe...@gmail.com>
> wrote:
> > >>
> > >> > Thanks Ted.
> > >> >
> > >> >>> knowing the host list of master role in Slider script or template
> is
> > >> not
> > >> > supported by either Slider or YARN.
> > >> > [Yong] If there is only one instance of master role, at least I
> could
> > >> know
> > >> > its host name by COMPONENT_HOST in slave hosts, right? If so, I
> could
> > >> work
> > >> > it around to just configure one master.
> > >> >
> > >> > Yong
> > >> >
> > >> > On Tue, Dec 30, 2014 at 10:59 PM, Ted Yu <yu...@gmail.com>
> wrote:
> > >> >
> > >> >> What's the timeframe for this migration ?
> > >> >>
> > >> >> If time permits, you should consider rewriting master failover
> logic
> > in
> > >> >> your application.
> > >> >> Using zookeeper is a common technique. HBase, e.g., registers
> active
> > >> master
> > >> >> under a well known zookeeper znode.
> > >> >>
> > >> >> This is much more flexible than the way you described.
> > >> >>
> > >> >> Again, knowing the host list of master role in Slider script or
> > >> template is
> > >> >> not supported by either Slider or YARN.
> > >> >>
> > >> >> Cheers
> > >> >>
> > >> >> On Tue, Dec 30, 2014 at 7:50 PM, Yong Feng <fe...@gmail.com>
> > >> wrote:
> > >> >>
> > >> >>> ah, rs_hosts means all the nodes managed by YARN.
> > >> >>>
> > >> >>> I know a little about labels feature of YARN which adds a tag to
> > host
> > >> and
> > >> >>> allow yarn API client to filter hosts according to the tag. I
> don't
> > >> think
> > >> >>> it helps my case.
> > >> >>>
> > >> >>> Taking a typical master/slave cluster as an example. To support
> > >> >>> active-passive fail over, we need define multiple instances for
> > master
> > >> >> role
> > >> >>> (unfortunately it does not use Zookeeper as the cluster is
> > implemented
> > >> >> 10+
> > >> >>> years ago before Zookeeper). In slave node, a list of mast host
> are
> > >> >>> specified in conf file, so that slave node could contact master.
> > >> >>>
> > >> >>> To deploy such cluster, we need know host list of master and
> > configure
> > >> >> them
> > >> >>> in the conf file of slave. Changing membership of master need
> > restart
> > >> >> whole
> > >> >>> cluster so we will not support flex up/down role of master.
> > >> >>>
> > >> >>> How can I know the host list of master role in Slider script or
> > >> template?
> > >> >>>
> > >> >>> Thanks,
> > >> >>>
> > >> >>> Yong
> > >> >>>
> > >> >>> On Tue, Dec 30, 2014 at 9:08 PM, Ted Yu <yu...@gmail.com>
> > wrote:
> > >> >>>
> > >> >>>> bq. host list of all the instances of a component
> > >> >>>>
> > >> >>>> Are you using node labels feature of YARN ?
> > >> >>>> If not, the list would comprise of all the nodes in your YARN
> > >> cluster.
> > >> >>>>
> > >> >>>> Cheers
> > >> >>>>
> > >> >>>> On Tue, Dec 30, 2014 at 4:43 PM, Yong Feng <
> fengyong.gm@gmail.com>
> > >> >>> wrote:
> > >> >>>>
> > >> >>>>> Hi Team,
> > >> >>>>>
> > >> >>>>> The template of regionserver of HBase application uses following
> > >> >>> scripts
> > >> >>>> to
> > >> >>>>> configure configuraiton file of regionserver of HBase. I did not
> > >> find
> > >> >>>> where
> > >> >>>>> the rs_hosts is defined in any file in Slider HBase package.
> How a
> > >> >>>> template
> > >> >>>>> is designed is mentioned a little at the end of
> > >> >>>>>
> > >> >>>>>
> > >> >>>>
> > >> >>>
> > >> >>
> > >>
> >
> http://slider.incubator.apache.org/docs/slider_specs/application_package.html
> > >> >>>>> ,
> > >> >>>>> but cannot help much.
> > >> >>>>>
> > >> >>>>> {% for host in rs_hosts %}{{host}}
> > >> >>>>> {% endfor %}
> > >> >>>>>
> > >> >>>>> Does anyone know where rs_hosts is defined? I am now writing a
> > >> >> package
> > >> >>>> for
> > >> >>>>> a HPC applicaiton and need to know host list of all the
> instances
> > of
> > >> >> a
> > >> >>>>> component to prepare configuration file of the application.
> > >> >>>>>
> > >> >>>>> Thanks,
> > >> >>>>>
> > >> >>>>> Yong
> > >> >>>>>
> > >> >>>>
> > >> >>>
> > >> >>
> > >>
> > >>
> > >> --
> > >> CONFIDENTIALITY NOTICE
> > >> NOTICE: This message is intended for the use of the individual or
> entity
> > >> to
> > >> which it is addressed and may contain information that is
> confidential,
> > >> privileged and exempt from disclosure under applicable law. If the
> > reader
> > >> of this message is not the intended recipient, you are hereby notified
> > >> that
> > >> any printing, copying, dissemination, distribution, disclosure or
> > >> forwarding of this communication is strictly prohibited. If you have
> > >> received this communication in error, please contact the sender
> > >> immediately
> > >> and delete it from your system. Thank You.
> > >>
> > >
> > >
> >
>
Re: How Sliders configure configuration file of regionserver of HBase
Posted by Ted Yu <yu...@gmail.com>.
Yong:
Let me discuss with slider / YARN developers and get back to you.
Happy new year.
On Thu, Jan 1, 2015 at 7:13 PM, Yong Feng <fe...@gmail.com> wrote:
> Jon, Ted
>
> Happy new year.
>
> After read more, I understand knowing the host list or single host of
> master role in Slider script or template is not supported by either Slider
> or YARN.
>
> So what I can do is
>
> a) pre-define a group of hosts with label of master
> b) specify label of master for master role in app resource specfication
> c) get host list with label of master in template/scripts when deploying
> slave role
>
> However I did not find doc how to get host list with label of master in
> Slider template/scripts like rs_hosts as follows
>
> {% for host in rs_hosts %}{{host}}
> {% endfor %}
>
> Any suggestion?
>
> Thanks,
>
> Yong
>
>
>
>
>
>
> On Wed, Dec 31, 2014 at 5:18 PM, Yong Feng <fe...@gmail.com> wrote:
>
> > Hi John
> >
> > I do expect the hosts allocated to master role of my slider application
> > are labeled as appname_rolename, but the question is who will add such
> > label and how? and how to get the hosts with the label?
> >
> > Thanks,
> >
> > Yong
> >
> > On Wed, Dec 31, 2014 at 3:57 PM, Jon Maron <jm...@hortonworks.com>
> wrote:
> >
> >> I’m confused - if certain hosts were labeled as MASTER, wouldn’t you
> know
> >> a priori the list of master hosts?
> >>
> >>
> >> On Dec 31, 2014, at 12:31 PM, Yong Feng <fe...@gmail.com> wrote:
> >>
> >> > Thanks Ted.
> >> >
> >> >>> knowing the host list of master role in Slider script or template is
> >> not
> >> > supported by either Slider or YARN.
> >> > [Yong] If there is only one instance of master role, at least I could
> >> know
> >> > its host name by COMPONENT_HOST in slave hosts, right? If so, I could
> >> work
> >> > it around to just configure one master.
> >> >
> >> > Yong
> >> >
> >> > On Tue, Dec 30, 2014 at 10:59 PM, Ted Yu <yu...@gmail.com> wrote:
> >> >
> >> >> What's the timeframe for this migration ?
> >> >>
> >> >> If time permits, you should consider rewriting master failover logic
> in
> >> >> your application.
> >> >> Using zookeeper is a common technique. HBase, e.g., registers active
> >> master
> >> >> under a well known zookeeper znode.
> >> >>
> >> >> This is much more flexible than the way you described.
> >> >>
> >> >> Again, knowing the host list of master role in Slider script or
> >> template is
> >> >> not supported by either Slider or YARN.
> >> >>
> >> >> Cheers
> >> >>
> >> >> On Tue, Dec 30, 2014 at 7:50 PM, Yong Feng <fe...@gmail.com>
> >> wrote:
> >> >>
> >> >>> ah, rs_hosts means all the nodes managed by YARN.
> >> >>>
> >> >>> I know a little about labels feature of YARN which adds a tag to
> host
> >> and
> >> >>> allow yarn API client to filter hosts according to the tag. I don't
> >> think
> >> >>> it helps my case.
> >> >>>
> >> >>> Taking a typical master/slave cluster as an example. To support
> >> >>> active-passive fail over, we need define multiple instances for
> master
> >> >> role
> >> >>> (unfortunately it does not use Zookeeper as the cluster is
> implemented
> >> >> 10+
> >> >>> years ago before Zookeeper). In slave node, a list of mast host are
> >> >>> specified in conf file, so that slave node could contact master.
> >> >>>
> >> >>> To deploy such cluster, we need know host list of master and
> configure
> >> >> them
> >> >>> in the conf file of slave. Changing membership of master need
> restart
> >> >> whole
> >> >>> cluster so we will not support flex up/down role of master.
> >> >>>
> >> >>> How can I know the host list of master role in Slider script or
> >> template?
> >> >>>
> >> >>> Thanks,
> >> >>>
> >> >>> Yong
> >> >>>
> >> >>> On Tue, Dec 30, 2014 at 9:08 PM, Ted Yu <yu...@gmail.com>
> wrote:
> >> >>>
> >> >>>> bq. host list of all the instances of a component
> >> >>>>
> >> >>>> Are you using node labels feature of YARN ?
> >> >>>> If not, the list would comprise of all the nodes in your YARN
> >> cluster.
> >> >>>>
> >> >>>> Cheers
> >> >>>>
> >> >>>> On Tue, Dec 30, 2014 at 4:43 PM, Yong Feng <fe...@gmail.com>
> >> >>> wrote:
> >> >>>>
> >> >>>>> Hi Team,
> >> >>>>>
> >> >>>>> The template of regionserver of HBase application uses following
> >> >>> scripts
> >> >>>> to
> >> >>>>> configure configuraiton file of regionserver of HBase. I did not
> >> find
> >> >>>> where
> >> >>>>> the rs_hosts is defined in any file in Slider HBase package. How a
> >> >>>> template
> >> >>>>> is designed is mentioned a little at the end of
> >> >>>>>
> >> >>>>>
> >> >>>>
> >> >>>
> >> >>
> >>
> http://slider.incubator.apache.org/docs/slider_specs/application_package.html
> >> >>>>> ,
> >> >>>>> but cannot help much.
> >> >>>>>
> >> >>>>> {% for host in rs_hosts %}{{host}}
> >> >>>>> {% endfor %}
> >> >>>>>
> >> >>>>> Does anyone know where rs_hosts is defined? I am now writing a
> >> >> package
> >> >>>> for
> >> >>>>> a HPC applicaiton and need to know host list of all the instances
> of
> >> >> a
> >> >>>>> component to prepare configuration file of the application.
> >> >>>>>
> >> >>>>> Thanks,
> >> >>>>>
> >> >>>>> Yong
> >> >>>>>
> >> >>>>
> >> >>>
> >> >>
> >>
> >>
> >> --
> >> CONFIDENTIALITY NOTICE
> >> NOTICE: This message is intended for the use of the individual or entity
> >> to
> >> which it is addressed and may contain information that is confidential,
> >> privileged and exempt from disclosure under applicable law. If the
> reader
> >> of this message is not the intended recipient, you are hereby notified
> >> that
> >> any printing, copying, dissemination, distribution, disclosure or
> >> forwarding of this communication is strictly prohibited. If you have
> >> received this communication in error, please contact the sender
> >> immediately
> >> and delete it from your system. Thank You.
> >>
> >
> >
>