You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by "Terry P." <te...@gmail.com> on 2013/11/05 21:13:03 UTC
Accumulo Standby Master question
I put my secondary namenode in the masters file for the first time with
this latest Accumulo 1.4.2 cluster deployment so it would run as a Standby
Accumulo Master which I read about in the User Guide. Recently had the
Master lose its Zookeeper lock (network glitch, being researched), and was
glad to see the secondary namenode Master process took over as it should.
But if my Namenode / Accumulo Master server goes down, I also lose the gc,
monitor, and tracer processes as well. *Can I configure the secondary
namenode in gc, tracer, and monitor files as well, or should they run on
only one host at a time*? In this case, the GC also lost its Zookeeper
lock, which resulted in a cluster with no GC running at all until I caught
it.
Thanks in advance.
Re: Accumulo Standby Master question
Posted by "Terry P." <te...@gmail.com>.
Thanks Josh, I really appreciate your response. That gives me the
confidence to go ahead and configure the Secondary Namenode with GC,
Monitor, and Tracer processes as backups.
On Tue, Nov 5, 2013 at 2:51 PM, Josh Elser <jo...@gmail.com> wrote:
> It looks like you can configure multiple hosts for GC and they'll use
> ZooKeeper to perform failover (like the master).
>
> Tracers -- You can run multiple tracer processes. You likely don't need
> 1:1 as you run tservers, but you can run a few if you're concerned about
> it. They're not required for Accumulo operation.
>
> Same for the monitor. If you need to have multiple running for failover
> purposes, it looks like you can specify multiple and it will just launch a
> monitor on each host you specified. There's no centralize URL you can
> always hit here. You would have to check each one to find one that was,
> unless you want to run some sort of reverse-proxy in front of them all.
>
> _obligatory I just looked to source code to come to this conclusion_
>
>
> On 11/5/13, 3:13 PM, Terry P. wrote:
>
>> I put my secondary namenode in the masters file for the first time with
>> this latest Accumulo 1.4.2 cluster deployment so it would run as a
>> Standby Accumulo Master which I read about in the User Guide. Recently
>> had the Master lose its Zookeeper lock (network glitch, being
>> researched), and was glad to see the secondary namenode Master process
>> took over as it should.
>>
>> But if my Namenode / Accumulo Master server goes down, I also lose the
>> gc, monitor, and tracer processes as well. *Can I configure the
>>
>> secondary namenode in gc, tracer, and monitor files as well, or should
>> they run on only one host at a time*? In this case, the GC also lost
>>
>> its Zookeeper lock, which resulted in a cluster with no GC running at
>> all until I caught it.
>>
>> Thanks in advance.
>>
>
Re: Accumulo Standby Master question
Posted by Josh Elser <jo...@gmail.com>.
No worries -- we're always happy to help people along. It's not hard, it
just requires some patience to get used to the process.
Let us know if you have any problems/questions as you move forward, and
I'll keep an eye out for a ticket from you.
On 11/13/13, 4:49 PM, Terry P. wrote:
> Hi Josh,
> This is definitely an easy one that I can handle! Assuming I can find
> where the missing shutdown hook is for the gc process ... (startup fix
> is done though at least).
>
> It will be my first time contributing to any Apache project (or any open
> source) so I am a noob from that perspective -- I'll read up on the
> Developer specifics on the web site and how to submit a patch and will
> work on it on my own time this weekend.
>
> Happy to give back, you all have helped me so much.
>
>
> On Wed, Nov 13, 2013 at 6:18 PM, Josh Elser <josh.elser@gmail.com
> <ma...@gmail.com>> wrote:
>
> Sounds like that would be a good ticket to create that affects 1.4.4
> and 1.5.0 something we should try to fix for 1.4.5, 1.5.1 and 1.6.0.
>
> Would be great to get a patch too!
>
>
Re: Accumulo Standby Master question
Posted by "Terry P." <te...@gmail.com>.
Hi Josh,
This is definitely an easy one that I can handle! Assuming I can find
where the missing shutdown hook is for the gc process ... (startup fix is
done though at least).
It will be my first time contributing to any Apache project (or any open
source) so I am a noob from that perspective -- I'll read up on the
Developer specifics on the web site and how to submit a patch and will work
on it on my own time this weekend.
Happy to give back, you all have helped me so much.
On Wed, Nov 13, 2013 at 6:18 PM, Josh Elser <jo...@gmail.com> wrote:
> Sounds like that would be a good ticket to create that affects 1.4.4 and
> 1.5.0 something we should try to fix for 1.4.5, 1.5.1 and 1.6.0.
>
> Would be great to get a patch too!
>
>
Re: Accumulo Standby Master question
Posted by Josh Elser <jo...@gmail.com>.
Sounds like that would be a good ticket to create that affects 1.4.4 and
1.5.0 something we should try to fix for 1.4.5, 1.5.1 and 1.6.0.
Would be great to get a patch too!
On 11/13/13, 3:43 PM, Terry P. wrote:
> Correction: just did an Accumulo cluster shutdown and the gc on the
> secondary namenode did not shutdown along with the rest of the cluster.
> No errors on the Master when stop-all.sh was called.
>
> No time to research right now though, but wanted to let everyone know.
>
>
> On Wed, Nov 13, 2013 at 4:29 PM, Terry P. <texpilot@gmail.com
> <ma...@gmail.com>> wrote:
>
> I was able to get to the 1.5.0 source, and start-here.sh has the
> same problem as described above for 1.4.2.
>
>
> On Wed, Nov 13, 2013 at 4:12 PM, Terry P. <texpilot@gmail.com
> <ma...@gmail.com>> wrote:
>
> Revisiting this, looks like I found even when I have my
> secondary namenode hostname in the gc file on the master and
> secondary namenode (my standby master), the gc process would not
> start on any node but the master.
>
> Looking at the config.sh script on lines 102-104:
>
> if [ -f "$ACCUMULO_HOME/conf/gc" ]; then
> GC=`grep -v '^#' "$ACCUMULO_HOME/conf/gc" | head -1`
> fi
>
> That only puts the first node found in the GC variable, similar
> to what is done with MASTER.
>
> Then in start-here.sh, whereas the masters file is checked to
> see if the current host is in the masters file (starting at line
> 38):
>
> for host in $HOSTS
> do
> *if grep -q "^${host}\$" $ACCUMULO_HOME/conf/masters*
> then
> ${bin}/accumulo
> org.apache.accumulo.server.master.state.SetGoalState NORMAL
> ${bin}/start-server.sh $host master
> break
> fi
> done
>
> *For GC, only the ${GC} variable is checked (starting on line
> 48), and start of the gc is only attempted on the one host in
> the GC variable*:
>
> for host in $HOSTS
> do
> *if [ ${host} = ${GC} ]*
> then
> ${bin}/start-server.sh *$GC* gc "garbage collector"
> break
> fi
> done
>
> *I've modified my start-here.sh script's GC section to*:
>
> for host in $HOSTS
> do
> *if grep -q "^${host}\$" $ACCUMULO_HOME/conf/gc*
> then
> ${bin}/start-server.sh *${host}* gc "garbage collector"
> break
> fi
> done
>
> And that did the trick. I didn't see any changes needed in the
> stop-here.sh script, as it already iterates over all possible
> processes to stop them if they are found.
>
> Can anyone think of anything I might have missed?
>
> I searched for any JIRAs on this but did not find any.
>
> I tried to download the 1.5.0 binaries, but download link
> doesn't work for me (has happened before with my work's overly
> restrictive internet gateway) -- but FYI I asked my brother to
> download it, and the "mirrors" link under GENERIC BINARIES
> didn't work for his as well.
>
> If I can get my hands on the 1.5.0 binaries, I'll check to see
> if this is already OBE in 1.5.0 today else it will have to wait
> until tonight.
>
>
>
> On Wed, Nov 6, 2013 at 12:12 PM, Terry P. <texpilot@gmail.com
> <ma...@gmail.com>> wrote:
>
> Ahh thanks Billie, I'll stick with just one monitor then.
> Thanks!
>
>
> On Wed, Nov 6, 2013 at 10:32 AM, Billie Rinaldi
> <billie.rinaldi@gmail.com <ma...@gmail.com>>
> wrote:
>
> On Tue, Nov 5, 2013 at 12:51 PM, Josh Elser
> <josh.elser@gmail.com <ma...@gmail.com>> wrote:
>
> It looks like you can configure multiple hosts for
> GC and they'll use ZooKeeper to perform failover
> (like the master).
>
> Tracers -- You can run multiple tracer processes.
> You likely don't need 1:1 as you run tservers, but
> you can run a few if you're concerned about it.
> They're not required for Accumulo operation.
>
> Same for the monitor. If you need to have multiple
> running for failover purposes, it looks like you can
> specify multiple and it will just launch a monitor
> on each host you specified. There's no centralize
> URL you can always hit here. You would have to check
> each one to find one that was, unless you want to
> run some sort of reverse-proxy in front of them all.
>
>
> I think additional monitors won't work entirely as
> expected. The log forwarding from the other processes
> is set up when the processes are started, and the logs
> are only sent to the first host in the monitor file.
>
>
> _obligatory I just looked to source code to come to
> this conclusion_
>
>
> On 11/5/13, 3:13 PM, Terry P. wrote:
>
> I put my secondary namenode in the masters file
> for the first time with
> this latest Accumulo 1.4.2 cluster deployment so
> it would run as a
> Standby Accumulo Master which I read about in
> the User Guide. Recently
> had the Master lose its Zookeeper lock (network
> glitch, being
> researched), and was glad to see the secondary
> namenode Master process
> took over as it should.
>
> But if my Namenode / Accumulo Master server goes
> down, I also lose the
> gc, monitor, and tracer processes as well. *Can
> I configure the
>
> secondary namenode in gc, tracer, and monitor
> files as well, or should
> they run on only one host at a time*? In this
> case, the GC also lost
>
> its Zookeeper lock, which resulted in a cluster
> with no GC running at
> all until I caught it.
>
> Thanks in advance.
>
>
>
>
>
>
Re: Accumulo Standby Master question
Posted by "Terry P." <te...@gmail.com>.
Correction: just did an Accumulo cluster shutdown and the gc on the
secondary namenode did not shutdown along with the rest of the cluster. No
errors on the Master when stop-all.sh was called.
No time to research right now though, but wanted to let everyone know.
On Wed, Nov 13, 2013 at 4:29 PM, Terry P. <te...@gmail.com> wrote:
> I was able to get to the 1.5.0 source, and start-here.sh has the same
> problem as described above for 1.4.2.
>
>
> On Wed, Nov 13, 2013 at 4:12 PM, Terry P. <te...@gmail.com> wrote:
>
>> Revisiting this, looks like I found even when I have my secondary
>> namenode hostname in the gc file on the master and secondary namenode (my
>> standby master), the gc process would not start on any node but the master.
>>
>> Looking at the config.sh script on lines 102-104:
>>
>> if [ -f "$ACCUMULO_HOME/conf/gc" ]; then
>> GC=`grep -v '^#' "$ACCUMULO_HOME/conf/gc" | head -1`
>> fi
>>
>> That only puts the first node found in the GC variable, similar to what
>> is done with MASTER.
>>
>> Then in start-here.sh, whereas the masters file is checked to see if the
>> current host is in the masters file (starting at line 38):
>>
>> for host in $HOSTS
>> do
>> *if grep -q "^${host}\$" $ACCUMULO_HOME/conf/masters*
>> then
>> ${bin}/accumulo
>> org.apache.accumulo.server.master.state.SetGoalState NORMAL
>> ${bin}/start-server.sh $host master
>> break
>> fi
>> done
>>
>> *For GC, only the ${GC} variable is checked (starting on line 48), and
>> start of the gc is only attempted on the one host in the GC variable*:
>>
>> for host in $HOSTS
>> do
>> *if [ ${host} = ${GC} ]*
>> then
>> ${bin}/start-server.sh *$GC* gc "garbage collector"
>> break
>> fi
>> done
>>
>> *I've modified my start-here.sh script's GC section to*:
>>
>> for host in $HOSTS
>> do
>> *if grep -q "^${host}\$" $ACCUMULO_HOME/conf/gc*
>> then
>> ${bin}/start-server.sh *${host}* gc "garbage collector"
>> break
>> fi
>> done
>>
>> And that did the trick. I didn't see any changes needed in the
>> stop-here.sh script, as it already iterates over all possible processes to
>> stop them if they are found.
>>
>> Can anyone think of anything I might have missed?
>>
>> I searched for any JIRAs on this but did not find any.
>>
>> I tried to download the 1.5.0 binaries, but download link doesn't work
>> for me (has happened before with my work's overly restrictive internet
>> gateway) -- but FYI I asked my brother to download it, and the "mirrors"
>> link under GENERIC BINARIES didn't work for his as well.
>>
>> If I can get my hands on the 1.5.0 binaries, I'll check to see if this is
>> already OBE in 1.5.0 today else it will have to wait until tonight.
>>
>>
>>
>> On Wed, Nov 6, 2013 at 12:12 PM, Terry P. <te...@gmail.com> wrote:
>>
>>> Ahh thanks Billie, I'll stick with just one monitor then. Thanks!
>>>
>>>
>>> On Wed, Nov 6, 2013 at 10:32 AM, Billie Rinaldi <
>>> billie.rinaldi@gmail.com> wrote:
>>>
>>>> On Tue, Nov 5, 2013 at 12:51 PM, Josh Elser <jo...@gmail.com>wrote:
>>>>
>>>>> It looks like you can configure multiple hosts for GC and they'll use
>>>>> ZooKeeper to perform failover (like the master).
>>>>>
>>>>> Tracers -- You can run multiple tracer processes. You likely don't
>>>>> need 1:1 as you run tservers, but you can run a few if you're concerned
>>>>> about it. They're not required for Accumulo operation.
>>>>>
>>>>> Same for the monitor. If you need to have multiple running for
>>>>> failover purposes, it looks like you can specify multiple and it will just
>>>>> launch a monitor on each host you specified. There's no centralize URL you
>>>>> can always hit here. You would have to check each one to find one that was,
>>>>> unless you want to run some sort of reverse-proxy in front of them all.
>>>>>
>>>>
>>>> I think additional monitors won't work entirely as expected. The log
>>>> forwarding from the other processes is set up when the processes are
>>>> started, and the logs are only sent to the first host in the monitor file.
>>>>
>>>>
>>>>> _obligatory I just looked to source code to come to this conclusion_
>>>>>
>>>>>
>>>>> On 11/5/13, 3:13 PM, Terry P. wrote:
>>>>>
>>>>>> I put my secondary namenode in the masters file for the first time
>>>>>> with
>>>>>> this latest Accumulo 1.4.2 cluster deployment so it would run as a
>>>>>> Standby Accumulo Master which I read about in the User Guide.
>>>>>> Recently
>>>>>> had the Master lose its Zookeeper lock (network glitch, being
>>>>>> researched), and was glad to see the secondary namenode Master process
>>>>>> took over as it should.
>>>>>>
>>>>>> But if my Namenode / Accumulo Master server goes down, I also lose the
>>>>>> gc, monitor, and tracer processes as well. *Can I configure the
>>>>>>
>>>>>> secondary namenode in gc, tracer, and monitor files as well, or should
>>>>>> they run on only one host at a time*? In this case, the GC also lost
>>>>>>
>>>>>> its Zookeeper lock, which resulted in a cluster with no GC running at
>>>>>> all until I caught it.
>>>>>>
>>>>>> Thanks in advance.
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Accumulo Standby Master question
Posted by "Terry P." <te...@gmail.com>.
I was able to get to the 1.5.0 source, and start-here.sh has the same
problem as described above for 1.4.2.
On Wed, Nov 13, 2013 at 4:12 PM, Terry P. <te...@gmail.com> wrote:
> Revisiting this, looks like I found even when I have my secondary namenode
> hostname in the gc file on the master and secondary namenode (my standby
> master), the gc process would not start on any node but the master.
>
> Looking at the config.sh script on lines 102-104:
>
> if [ -f "$ACCUMULO_HOME/conf/gc" ]; then
> GC=`grep -v '^#' "$ACCUMULO_HOME/conf/gc" | head -1`
> fi
>
> That only puts the first node found in the GC variable, similar to what is
> done with MASTER.
>
> Then in start-here.sh, whereas the masters file is checked to see if the
> current host is in the masters file (starting at line 38):
>
> for host in $HOSTS
> do
> *if grep -q "^${host}\$" $ACCUMULO_HOME/conf/masters*
> then
> ${bin}/accumulo
> org.apache.accumulo.server.master.state.SetGoalState NORMAL
> ${bin}/start-server.sh $host master
> break
> fi
> done
>
> *For GC, only the ${GC} variable is checked (starting on line 48), and
> start of the gc is only attempted on the one host in the GC variable*:
>
> for host in $HOSTS
> do
> *if [ ${host} = ${GC} ]*
> then
> ${bin}/start-server.sh *$GC* gc "garbage collector"
> break
> fi
> done
>
> *I've modified my start-here.sh script's GC section to*:
>
> for host in $HOSTS
> do
> *if grep -q "^${host}\$" $ACCUMULO_HOME/conf/gc*
> then
> ${bin}/start-server.sh *${host}* gc "garbage collector"
> break
> fi
> done
>
> And that did the trick. I didn't see any changes needed in the
> stop-here.sh script, as it already iterates over all possible processes to
> stop them if they are found.
>
> Can anyone think of anything I might have missed?
>
> I searched for any JIRAs on this but did not find any.
>
> I tried to download the 1.5.0 binaries, but download link doesn't work for
> me (has happened before with my work's overly restrictive internet gateway)
> -- but FYI I asked my brother to download it, and the "mirrors" link under
> GENERIC BINARIES didn't work for his as well.
>
> If I can get my hands on the 1.5.0 binaries, I'll check to see if this is
> already OBE in 1.5.0 today else it will have to wait until tonight.
>
>
>
> On Wed, Nov 6, 2013 at 12:12 PM, Terry P. <te...@gmail.com> wrote:
>
>> Ahh thanks Billie, I'll stick with just one monitor then. Thanks!
>>
>>
>> On Wed, Nov 6, 2013 at 10:32 AM, Billie Rinaldi <billie.rinaldi@gmail.com
>> > wrote:
>>
>>> On Tue, Nov 5, 2013 at 12:51 PM, Josh Elser <jo...@gmail.com>wrote:
>>>
>>>> It looks like you can configure multiple hosts for GC and they'll use
>>>> ZooKeeper to perform failover (like the master).
>>>>
>>>> Tracers -- You can run multiple tracer processes. You likely don't need
>>>> 1:1 as you run tservers, but you can run a few if you're concerned about
>>>> it. They're not required for Accumulo operation.
>>>>
>>>> Same for the monitor. If you need to have multiple running for failover
>>>> purposes, it looks like you can specify multiple and it will just launch a
>>>> monitor on each host you specified. There's no centralize URL you can
>>>> always hit here. You would have to check each one to find one that was,
>>>> unless you want to run some sort of reverse-proxy in front of them all.
>>>>
>>>
>>> I think additional monitors won't work entirely as expected. The log
>>> forwarding from the other processes is set up when the processes are
>>> started, and the logs are only sent to the first host in the monitor file.
>>>
>>>
>>>> _obligatory I just looked to source code to come to this conclusion_
>>>>
>>>>
>>>> On 11/5/13, 3:13 PM, Terry P. wrote:
>>>>
>>>>> I put my secondary namenode in the masters file for the first time with
>>>>> this latest Accumulo 1.4.2 cluster deployment so it would run as a
>>>>> Standby Accumulo Master which I read about in the User Guide. Recently
>>>>> had the Master lose its Zookeeper lock (network glitch, being
>>>>> researched), and was glad to see the secondary namenode Master process
>>>>> took over as it should.
>>>>>
>>>>> But if my Namenode / Accumulo Master server goes down, I also lose the
>>>>> gc, monitor, and tracer processes as well. *Can I configure the
>>>>>
>>>>> secondary namenode in gc, tracer, and monitor files as well, or should
>>>>> they run on only one host at a time*? In this case, the GC also lost
>>>>>
>>>>> its Zookeeper lock, which resulted in a cluster with no GC running at
>>>>> all until I caught it.
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>
>>>
>>
>
Re: Accumulo Standby Master question
Posted by "Terry P." <te...@gmail.com>.
Revisiting this, looks like I found even when I have my secondary namenode
hostname in the gc file on the master and secondary namenode (my standby
master), the gc process would not start on any node but the master.
Looking at the config.sh script on lines 102-104:
if [ -f "$ACCUMULO_HOME/conf/gc" ]; then
GC=`grep -v '^#' "$ACCUMULO_HOME/conf/gc" | head -1`
fi
That only puts the first node found in the GC variable, similar to what is
done with MASTER.
Then in start-here.sh, whereas the masters file is checked to see if the
current host is in the masters file (starting at line 38):
for host in $HOSTS
do
*if grep -q "^${host}\$" $ACCUMULO_HOME/conf/masters*
then
${bin}/accumulo org.apache.accumulo.server.master.state.SetGoalState
NORMAL
${bin}/start-server.sh $host master
break
fi
done
*For GC, only the ${GC} variable is checked (starting on line 48), and
start of the gc is only attempted on the one host in the GC variable*:
for host in $HOSTS
do
*if [ ${host} = ${GC} ]*
then
${bin}/start-server.sh *$GC* gc "garbage collector"
break
fi
done
*I've modified my start-here.sh script's GC section to*:
for host in $HOSTS
do
*if grep -q "^${host}\$" $ACCUMULO_HOME/conf/gc*
then
${bin}/start-server.sh *${host}* gc "garbage collector"
break
fi
done
And that did the trick. I didn't see any changes needed in the
stop-here.sh script, as it already iterates over all possible processes to
stop them if they are found.
Can anyone think of anything I might have missed?
I searched for any JIRAs on this but did not find any.
I tried to download the 1.5.0 binaries, but download link doesn't work for
me (has happened before with my work's overly restrictive internet gateway)
-- but FYI I asked my brother to download it, and the "mirrors" link under
GENERIC BINARIES didn't work for his as well.
If I can get my hands on the 1.5.0 binaries, I'll check to see if this is
already OBE in 1.5.0 today else it will have to wait until tonight.
On Wed, Nov 6, 2013 at 12:12 PM, Terry P. <te...@gmail.com> wrote:
> Ahh thanks Billie, I'll stick with just one monitor then. Thanks!
>
>
> On Wed, Nov 6, 2013 at 10:32 AM, Billie Rinaldi <bi...@gmail.com>wrote:
>
>> On Tue, Nov 5, 2013 at 12:51 PM, Josh Elser <jo...@gmail.com> wrote:
>>
>>> It looks like you can configure multiple hosts for GC and they'll use
>>> ZooKeeper to perform failover (like the master).
>>>
>>> Tracers -- You can run multiple tracer processes. You likely don't need
>>> 1:1 as you run tservers, but you can run a few if you're concerned about
>>> it. They're not required for Accumulo operation.
>>>
>>> Same for the monitor. If you need to have multiple running for failover
>>> purposes, it looks like you can specify multiple and it will just launch a
>>> monitor on each host you specified. There's no centralize URL you can
>>> always hit here. You would have to check each one to find one that was,
>>> unless you want to run some sort of reverse-proxy in front of them all.
>>>
>>
>> I think additional monitors won't work entirely as expected. The log
>> forwarding from the other processes is set up when the processes are
>> started, and the logs are only sent to the first host in the monitor file.
>>
>>
>>> _obligatory I just looked to source code to come to this conclusion_
>>>
>>>
>>> On 11/5/13, 3:13 PM, Terry P. wrote:
>>>
>>>> I put my secondary namenode in the masters file for the first time with
>>>> this latest Accumulo 1.4.2 cluster deployment so it would run as a
>>>> Standby Accumulo Master which I read about in the User Guide. Recently
>>>> had the Master lose its Zookeeper lock (network glitch, being
>>>> researched), and was glad to see the secondary namenode Master process
>>>> took over as it should.
>>>>
>>>> But if my Namenode / Accumulo Master server goes down, I also lose the
>>>> gc, monitor, and tracer processes as well. *Can I configure the
>>>>
>>>> secondary namenode in gc, tracer, and monitor files as well, or should
>>>> they run on only one host at a time*? In this case, the GC also lost
>>>>
>>>> its Zookeeper lock, which resulted in a cluster with no GC running at
>>>> all until I caught it.
>>>>
>>>> Thanks in advance.
>>>>
>>>
>>
>
Re: Accumulo Standby Master question
Posted by "Terry P." <te...@gmail.com>.
Ahh thanks Billie, I'll stick with just one monitor then. Thanks!
On Wed, Nov 6, 2013 at 10:32 AM, Billie Rinaldi <bi...@gmail.com>wrote:
> On Tue, Nov 5, 2013 at 12:51 PM, Josh Elser <jo...@gmail.com> wrote:
>
>> It looks like you can configure multiple hosts for GC and they'll use
>> ZooKeeper to perform failover (like the master).
>>
>> Tracers -- You can run multiple tracer processes. You likely don't need
>> 1:1 as you run tservers, but you can run a few if you're concerned about
>> it. They're not required for Accumulo operation.
>>
>> Same for the monitor. If you need to have multiple running for failover
>> purposes, it looks like you can specify multiple and it will just launch a
>> monitor on each host you specified. There's no centralize URL you can
>> always hit here. You would have to check each one to find one that was,
>> unless you want to run some sort of reverse-proxy in front of them all.
>>
>
> I think additional monitors won't work entirely as expected. The log
> forwarding from the other processes is set up when the processes are
> started, and the logs are only sent to the first host in the monitor file.
>
>
>> _obligatory I just looked to source code to come to this conclusion_
>>
>>
>> On 11/5/13, 3:13 PM, Terry P. wrote:
>>
>>> I put my secondary namenode in the masters file for the first time with
>>> this latest Accumulo 1.4.2 cluster deployment so it would run as a
>>> Standby Accumulo Master which I read about in the User Guide. Recently
>>> had the Master lose its Zookeeper lock (network glitch, being
>>> researched), and was glad to see the secondary namenode Master process
>>> took over as it should.
>>>
>>> But if my Namenode / Accumulo Master server goes down, I also lose the
>>> gc, monitor, and tracer processes as well. *Can I configure the
>>>
>>> secondary namenode in gc, tracer, and monitor files as well, or should
>>> they run on only one host at a time*? In this case, the GC also lost
>>>
>>> its Zookeeper lock, which resulted in a cluster with no GC running at
>>> all until I caught it.
>>>
>>> Thanks in advance.
>>>
>>
>
Re: Accumulo Standby Master question
Posted by Billie Rinaldi <bi...@gmail.com>.
On Tue, Nov 5, 2013 at 12:51 PM, Josh Elser <jo...@gmail.com> wrote:
> It looks like you can configure multiple hosts for GC and they'll use
> ZooKeeper to perform failover (like the master).
>
> Tracers -- You can run multiple tracer processes. You likely don't need
> 1:1 as you run tservers, but you can run a few if you're concerned about
> it. They're not required for Accumulo operation.
>
> Same for the monitor. If you need to have multiple running for failover
> purposes, it looks like you can specify multiple and it will just launch a
> monitor on each host you specified. There's no centralize URL you can
> always hit here. You would have to check each one to find one that was,
> unless you want to run some sort of reverse-proxy in front of them all.
>
I think additional monitors won't work entirely as expected. The log
forwarding from the other processes is set up when the processes are
started, and the logs are only sent to the first host in the monitor file.
> _obligatory I just looked to source code to come to this conclusion_
>
>
> On 11/5/13, 3:13 PM, Terry P. wrote:
>
>> I put my secondary namenode in the masters file for the first time with
>> this latest Accumulo 1.4.2 cluster deployment so it would run as a
>> Standby Accumulo Master which I read about in the User Guide. Recently
>> had the Master lose its Zookeeper lock (network glitch, being
>> researched), and was glad to see the secondary namenode Master process
>> took over as it should.
>>
>> But if my Namenode / Accumulo Master server goes down, I also lose the
>> gc, monitor, and tracer processes as well. *Can I configure the
>>
>> secondary namenode in gc, tracer, and monitor files as well, or should
>> they run on only one host at a time*? In this case, the GC also lost
>>
>> its Zookeeper lock, which resulted in a cluster with no GC running at
>> all until I caught it.
>>
>> Thanks in advance.
>>
>
Re: Accumulo Standby Master question
Posted by Josh Elser <jo...@gmail.com>.
It looks like you can configure multiple hosts for GC and they'll use
ZooKeeper to perform failover (like the master).
Tracers -- You can run multiple tracer processes. You likely don't need
1:1 as you run tservers, but you can run a few if you're concerned about
it. They're not required for Accumulo operation.
Same for the monitor. If you need to have multiple running for failover
purposes, it looks like you can specify multiple and it will just launch
a monitor on each host you specified. There's no centralize URL you can
always hit here. You would have to check each one to find one that was,
unless you want to run some sort of reverse-proxy in front of them all.
_obligatory I just looked to source code to come to this conclusion_
On 11/5/13, 3:13 PM, Terry P. wrote:
> I put my secondary namenode in the masters file for the first time with
> this latest Accumulo 1.4.2 cluster deployment so it would run as a
> Standby Accumulo Master which I read about in the User Guide. Recently
> had the Master lose its Zookeeper lock (network glitch, being
> researched), and was glad to see the secondary namenode Master process
> took over as it should.
>
> But if my Namenode / Accumulo Master server goes down, I also lose the
> gc, monitor, and tracer processes as well. *Can I configure the
> secondary namenode in gc, tracer, and monitor files as well, or should
> they run on only one host at a time*? In this case, the GC also lost
> its Zookeeper lock, which resulted in a cluster with no GC running at
> all until I caught it.
>
> Thanks in advance.