Any good URL/write-ups on the various bin/*.sh files & conf files?

[Aaron <aa...@gmail.com>]

Was curious if anyone knew of some decent write-ups on various shell
scripts? Also, curious about master, slaves, gc, tracers, etc conf files.
Main reason I ask is about what .sh to run "where."

For example, have "head" system that is running master/gc/monitor/tracer,
then we have other various nodes running tserver/logger.

1a.  Does every node, including the "head" node need all the conf files the
same...masters/slaves/tracers, etc?

1b. Or does the "master" node only need to know about where the slaves are,
so it uses the slaves file.  And conversely, do the tserver nodes only need
to have a master conf file to put themselves to the master?

2.  I saw someplace not to use tup.sh on the compute/storage nodes...do I
just use start-server.sh to only bring up the logger/tserver..and NOT the
other services?

Basically, there are more questions like this, but, wanted to ask if anyone
out there has some write-ups to I can RTFM first.  Basically, looking for a
little more into the weeds that the Administration/Installation section of
the manual.

Thanks in advance,
Aaron

Re: Any good URL/write-ups on the various bin/*.sh files & conf files?

[John Vines <vi...@apache.org>]

It is worth noting that the self identification is not perfect. It gathers
a few different identities for itself; including localhost, the output of
hostname, and a non loopback output of ipconfig. However it is possible to
have a machine configured where that is not sufficient so start-all works
but start-here does not.

Sent from my phone, please pardon the typos and brevity.
On May 6, 2013 8:04 AM, "Aaron" <aa...@gmail.com> wrote:

> Ahhh cool...so start-all.sh & start-here.sh are "smart" and by that I
> mean, will only start the certain processes based upon which machine they
> are on.  And if you do a start-here.sh on a box, it will look through every
> file, and only start the processes for which it "finds itself."
>
> So, if a machine is in master & gc, but not in tracer....and when I call
> start-here.sh on that box, it will only start the master & gc
> processes..and NOT tracer?  Assuming the answer is yes, but just confirming.
>
> For slave, does that assume 2 processes: tserver & logger?  so, if a
> machine is in the slave file, it will start both of those?
>
>
>
> On Fri, May 3, 2013 at 4:06 PM, John Vines <vi...@apache.org> wrote:
>
>> Those files are used for 2 purposes.
>> 1. Start-all.sh, start-here.sh, and tup.sh read all of the files (tup
>> only uses slaves) to start up the processes. Start-all will go through each
>> file for each process, ssh to that server (no ssh if the current host) and
>> start the process. Start-all uses tup for the tserver and logger processes.
>> Start-here reads through each file, and if the machines match, it will
>> start that process locally.
>>
>> 2. The files are used in attempts to bind to a port. This is for when
>> your on a machine with multiple ports, to help it bind on the port you
>> want. This MAY only actually occur if you're using the above mentioned
>> scripts.
>>
>> So that said, as long as the configuration of those files is correct for
>> the node you're on, you can run them there. And on that same note, the
>> reason you want to keep those various server list files synced is so you
>> can run start-all/tup from any node and have the same effect.
>>
>>
>> On Fri, May 3, 2013 at 3:47 PM, Aaron <aa...@gmail.com> wrote:
>>
>>> Was curious if anyone knew of some decent write-ups on various shell
>>> scripts? Also, curious about master, slaves, gc, tracers, etc conf files.
>>> Main reason I ask is about what .sh to run "where."
>>>
>>> For example, have "head" system that is running
>>> master/gc/monitor/tracer, then we have other various nodes running
>>> tserver/logger.
>>>
>>> 1a.  Does every node, including the "head" node need all the conf files
>>> the same...masters/slaves/tracers, etc?
>>>
>>> 1b. Or does the "master" node only need to know about where the slaves
>>> are, so it uses the slaves file.  And conversely, do the tserver nodes only
>>> need to have a master conf file to put themselves to the master?
>>>
>>> 2.  I saw someplace not to use tup.sh on the compute/storage nodes...do
>>> I just use start-server.sh to only bring up the logger/tserver..and NOT the
>>> other services?
>>>
>>> Basically, there are more questions like this, but, wanted to ask if
>>> anyone out there has some write-ups to I can RTFM first.  Basically,
>>> looking for a little more into the weeds that the
>>> Administration/Installation section of the manual.
>>>
>>> Thanks in advance,
>>> Aaron
>>>
>>
>>
>

Re: Any good URL/write-ups on the various bin/*.sh files & conf files?

[Christopher <ct...@apache.org>]

Yes on both counts.

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii


On Mon, May 6, 2013 at 8:03 AM, Aaron <aa...@gmail.com> wrote:
> Ahhh cool...so start-all.sh & start-here.sh are "smart" and by that I mean,
> will only start the certain processes based upon which machine they are on.
> And if you do a start-here.sh on a box, it will look through every file, and
> only start the processes for which it "finds itself."
>
> So, if a machine is in master & gc, but not in tracer....and when I call
> start-here.sh on that box, it will only start the master & gc processes..and
> NOT tracer?  Assuming the answer is yes, but just confirming.
>
> For slave, does that assume 2 processes: tserver & logger?  so, if a machine
> is in the slave file, it will start both of those?
>
>
>
> On Fri, May 3, 2013 at 4:06 PM, John Vines <vi...@apache.org> wrote:
>>
>> Those files are used for 2 purposes.
>> 1. Start-all.sh, start-here.sh, and tup.sh read all of the files (tup only
>> uses slaves) to start up the processes. Start-all will go through each file
>> for each process, ssh to that server (no ssh if the current host) and start
>> the process. Start-all uses tup for the tserver and logger processes.
>> Start-here reads through each file, and if the machines match, it will start
>> that process locally.
>>
>> 2. The files are used in attempts to bind to a port. This is for when your
>> on a machine with multiple ports, to help it bind on the port you want. This
>> MAY only actually occur if you're using the above mentioned scripts.
>>
>> So that said, as long as the configuration of those files is correct for
>> the node you're on, you can run them there. And on that same note, the
>> reason you want to keep those various server list files synced is so you can
>> run start-all/tup from any node and have the same effect.
>>
>>
>> On Fri, May 3, 2013 at 3:47 PM, Aaron <aa...@gmail.com> wrote:
>>>
>>> Was curious if anyone knew of some decent write-ups on various shell
>>> scripts? Also, curious about master, slaves, gc, tracers, etc conf files.
>>> Main reason I ask is about what .sh to run "where."
>>>
>>> For example, have "head" system that is running master/gc/monitor/tracer,
>>> then we have other various nodes running tserver/logger.
>>>
>>> 1a.  Does every node, including the "head" node need all the conf files
>>> the same...masters/slaves/tracers, etc?
>>>
>>> 1b. Or does the "master" node only need to know about where the slaves
>>> are, so it uses the slaves file.  And conversely, do the tserver nodes only
>>> need to have a master conf file to put themselves to the master?
>>>
>>> 2.  I saw someplace not to use tup.sh on the compute/storage nodes...do I
>>> just use start-server.sh to only bring up the logger/tserver..and NOT the
>>> other services?
>>>
>>> Basically, there are more questions like this, but, wanted to ask if
>>> anyone out there has some write-ups to I can RTFM first.  Basically, looking
>>> for a little more into the weeds that the Administration/Installation
>>> section of the manual.
>>>
>>> Thanks in advance,
>>> Aaron
>>
>>
>

Re: Any good URL/write-ups on the various bin/*.sh files & conf files?

[Aaron <aa...@gmail.com>]

Ahhh cool...so start-all.sh & start-here.sh are "smart" and by that I mean,
will only start the certain processes based upon which machine they are on.
 And if you do a start-here.sh on a box, it will look through every file,
and only start the processes for which it "finds itself."

So, if a machine is in master & gc, but not in tracer....and when I call
start-here.sh on that box, it will only start the master & gc
processes..and NOT tracer?  Assuming the answer is yes, but just confirming.

For slave, does that assume 2 processes: tserver & logger?  so, if a
machine is in the slave file, it will start both of those?



On Fri, May 3, 2013 at 4:06 PM, John Vines <vi...@apache.org> wrote:

> Those files are used for 2 purposes.
> 1. Start-all.sh, start-here.sh, and tup.sh read all of the files (tup only
> uses slaves) to start up the processes. Start-all will go through each file
> for each process, ssh to that server (no ssh if the current host) and start
> the process. Start-all uses tup for the tserver and logger processes.
> Start-here reads through each file, and if the machines match, it will
> start that process locally.
>
> 2. The files are used in attempts to bind to a port. This is for when your
> on a machine with multiple ports, to help it bind on the port you want.
> This MAY only actually occur if you're using the above mentioned scripts.
>
> So that said, as long as the configuration of those files is correct for
> the node you're on, you can run them there. And on that same note, the
> reason you want to keep those various server list files synced is so you
> can run start-all/tup from any node and have the same effect.
>
>
> On Fri, May 3, 2013 at 3:47 PM, Aaron <aa...@gmail.com> wrote:
>
>> Was curious if anyone knew of some decent write-ups on various shell
>> scripts? Also, curious about master, slaves, gc, tracers, etc conf files.
>> Main reason I ask is about what .sh to run "where."
>>
>> For example, have "head" system that is running master/gc/monitor/tracer,
>> then we have other various nodes running tserver/logger.
>>
>> 1a.  Does every node, including the "head" node need all the conf files
>> the same...masters/slaves/tracers, etc?
>>
>> 1b. Or does the "master" node only need to know about where the slaves
>> are, so it uses the slaves file.  And conversely, do the tserver nodes only
>> need to have a master conf file to put themselves to the master?
>>
>> 2.  I saw someplace not to use tup.sh on the compute/storage nodes...do I
>> just use start-server.sh to only bring up the logger/tserver..and NOT the
>> other services?
>>
>> Basically, there are more questions like this, but, wanted to ask if
>> anyone out there has some write-ups to I can RTFM first.  Basically,
>> looking for a little more into the weeds that the
>> Administration/Installation section of the manual.
>>
>> Thanks in advance,
>> Aaron
>>
>
>

Re: Any good URL/write-ups on the various bin/*.sh files & conf files?

[John Vines <vi...@apache.org>]

Those files are used for 2 purposes.
1. Start-all.sh, start-here.sh, and tup.sh read all of the files (tup only
uses slaves) to start up the processes. Start-all will go through each file
for each process, ssh to that server (no ssh if the current host) and start
the process. Start-all uses tup for the tserver and logger processes.
Start-here reads through each file, and if the machines match, it will
start that process locally.

2. The files are used in attempts to bind to a port. This is for when your
on a machine with multiple ports, to help it bind on the port you want.
This MAY only actually occur if you're using the above mentioned scripts.

So that said, as long as the configuration of those files is correct for
the node you're on, you can run them there. And on that same note, the
reason you want to keep those various server list files synced is so you
can run start-all/tup from any node and have the same effect.


On Fri, May 3, 2013 at 3:47 PM, Aaron <aa...@gmail.com> wrote:

> Was curious if anyone knew of some decent write-ups on various shell
> scripts? Also, curious about master, slaves, gc, tracers, etc conf files.
> Main reason I ask is about what .sh to run "where."
>
> For example, have "head" system that is running master/gc/monitor/tracer,
> then we have other various nodes running tserver/logger.
>
> 1a.  Does every node, including the "head" node need all the conf files
> the same...masters/slaves/tracers, etc?
>
> 1b. Or does the "master" node only need to know about where the slaves
> are, so it uses the slaves file.  And conversely, do the tserver nodes only
> need to have a master conf file to put themselves to the master?
>
> 2.  I saw someplace not to use tup.sh on the compute/storage nodes...do I
> just use start-server.sh to only bring up the logger/tserver..and NOT the
> other services?
>
> Basically, there are more questions like this, but, wanted to ask if
> anyone out there has some write-ups to I can RTFM first.  Basically,
> looking for a little more into the weeds that the
> Administration/Installation section of the manual.
>
> Thanks in advance,
> Aaron
>