You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by Steve Loughran <st...@hortonworks.com> on 2012/09/24 16:37:54 UTC
how to do preflight checking
I'm writing a service that is having some preflight checking of the cluster
spec done up front:
#of master nodes == 1
#master-service and worker service must not co-exist
>From what I see, most services (implicitly) check this before the configure
operation, as they look for where the NN is, JT is, or similar, usually
implicitly and failing with a NoSuchElementException (e.g. WHIRR-652).
I'm trying to be stricter and want to verify the template's validity before
any VMs have been requested. This would fail-fast and save money.
Is there a way to do this at present -and if so, how?
-steve
Re: how to do preflight checking
Posted by Steve Loughran <st...@hortonworks.com>.
It's low priority, as I've got my validation checks going in later, its
just they don't kick in until after the cluster is in the air, at which
point failures are more dramatic.
If there was some spec validation utils I'd add the code (& tests) to the
Hadoop services to ensure that the HDFS and MR layers were valid if any of
their roles were specified.
On 25 September 2012 13:52, Andrei Savu <sa...@gmail.com> wrote:
> On Tue, Sep 25, 2012 at 3:50 PM, Steve Loughran <stevel@hortonworks.com
> >wrote:
>
> > cluster spec doesn't have a flat list of instances to search on -the
> data's
> > there, just less obviously accessible
> >
>
> I see. Maybe we need a set of utilities methods for making assertions on
> ClusterSpec.
>
Re: how to do preflight checking
Posted by Andrei Savu <sa...@gmail.com>.
On Tue, Sep 25, 2012 at 3:50 PM, Steve Loughran <st...@hortonworks.com>wrote:
> cluster spec doesn't have a flat list of instances to search on -the data's
> there, just less obviously accessible
>
I see. Maybe we need a set of utilities methods for making assertions on
ClusterSpec.
Re: how to do preflight checking
Posted by Steve Loughran <st...@hortonworks.com>.
On 25 September 2012 13:27, Andrei Savu <sa...@gmail.com> wrote:
> Why not just check the ClusterSpec? Why do you need another structure?
>
cluster spec doesn't have a flat list of instances to search on -the data's
there, just less obviously accessible
>
> The role assignments after bootstrap will match whirr.instance-templates.
>
> -- Andrei Savu
>
> On Tue, Sep 25, 2012 at 3:23 PM, Steve Loughran <stevel@hortonworks.com
> >wrote:
>
> > On 24 September 2012 16:03, Andrei Savu <sa...@gmail.com> wrote:
> >
> > > No VMs are started during beforeBootstrap().
> > >
> > > You can throw validation exceptions in that method for each role.
> > >
> >
> > Of course, as the set of instances in a cluster is null at that point,
> you
> > can't do a lookup and assert that there is exactly one namenode in the
> > cluster.
> >
> > It looks like I could it by going through the instance templates and
> > creating a structure that modelled what is going to be instantiated,
> that I
> > could then preflight over.
> >
> > Has anyone doe anything like that yet?
> >
>
Re: how to do preflight checking
Posted by Andrei Savu <sa...@gmail.com>.
Why not just check the ClusterSpec? Why do you need another structure?
The role assignments after bootstrap will match whirr.instance-templates.
-- Andrei Savu
On Tue, Sep 25, 2012 at 3:23 PM, Steve Loughran <st...@hortonworks.com>wrote:
> On 24 September 2012 16:03, Andrei Savu <sa...@gmail.com> wrote:
>
> > No VMs are started during beforeBootstrap().
> >
> > You can throw validation exceptions in that method for each role.
> >
>
> Of course, as the set of instances in a cluster is null at that point, you
> can't do a lookup and assert that there is exactly one namenode in the
> cluster.
>
> It looks like I could it by going through the instance templates and
> creating a structure that modelled what is going to be instantiated, that I
> could then preflight over.
>
> Has anyone doe anything like that yet?
>
Re: how to do preflight checking
Posted by Steve Loughran <st...@hortonworks.com>.
On 24 September 2012 16:03, Andrei Savu <sa...@gmail.com> wrote:
> No VMs are started during beforeBootstrap().
>
> You can throw validation exceptions in that method for each role.
>
Of course, as the set of instances in a cluster is null at that point, you
can't do a lookup and assert that there is exactly one namenode in the
cluster.
It looks like I could it by going through the instance templates and
creating a structure that modelled what is going to be instantiated, that I
could then preflight over.
Has anyone doe anything like that yet?
Re: how to do preflight checking
Posted by Steve Loughran <st...@hortonworks.com>.
thanks. Better start writing those tests then...
On 24 September 2012 16:03, Andrei Savu <sa...@gmail.com> wrote:
> No VMs are started during beforeBootstrap().
>
> You can throw validation exceptions in that method for each role.
>
> -- Andrei Savu
>
> On Mon, Sep 24, 2012 at 5:37 PM, Steve Loughran <stevel@hortonworks.com
> >wrote:
>
> > I'm writing a service that is having some preflight checking of the
> cluster
> > spec done up front:
> > #of master nodes == 1
> > #master-service and worker service must not co-exist
> >
> > From what I see, most services (implicitly) check this before the
> configure
> > operation, as they look for where the NN is, JT is, or similar, usually
> > implicitly and failing with a NoSuchElementException (e.g. WHIRR-652).
> >
> > I'm trying to be stricter and want to verify the template's validity
> before
> > any VMs have been requested. This would fail-fast and save money.
> >
> > Is there a way to do this at present -and if so, how?
> >
> > -steve
> >
>
Re: how to do preflight checking
Posted by Andrei Savu <sa...@gmail.com>.
No VMs are started during beforeBootstrap().
You can throw validation exceptions in that method for each role.
-- Andrei Savu
On Mon, Sep 24, 2012 at 5:37 PM, Steve Loughran <st...@hortonworks.com>wrote:
> I'm writing a service that is having some preflight checking of the cluster
> spec done up front:
> #of master nodes == 1
> #master-service and worker service must not co-exist
>
> From what I see, most services (implicitly) check this before the configure
> operation, as they look for where the NN is, JT is, or similar, usually
> implicitly and failing with a NoSuchElementException (e.g. WHIRR-652).
>
> I'm trying to be stricter and want to verify the template's validity before
> any VMs have been requested. This would fail-fast and save money.
>
> Is there a way to do this at present -and if so, how?
>
> -steve
>