You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by "Terry P." <te...@gmail.com> on 2013/06/06 20:44:48 UTC
No info for Accumulo Master on Monitor Overview page
Greetings all,
We have two Accumulo clusters built from essentially the same build
procedure. Everything is running well with the exception that on the
Accumulo Monitor Overview web page, the Accumulo Master box is blank -- the
"Accumulo Master" title/hyperlink works (takes you to the "Master Server"
page), but nothing is in the box. The NameNode, JobTracker, and Zookeeper
boxes are all populated.
On the Master Server page, I see the Master Status that shows 3 Online
Tablet Servers, Loggers, Last GC, etc. It also shows the Table List. The
Tablet Servers page looks good.
In these small clusters we have one Accumulo Master that runs on the
NameNode, 3 DataNodes, 3 Zookeepers. I'm in the process of firing up a
Secondary NameNode which will also run an Alternate Accumulo Master in case
the Master on the NameNode were to fail for some reason, but I have not
fired up the Accumulo processes on the Secondary NameNode server yet at all.
Any idea where I can look to try and determine why the Accumulo Master box
is empty? It's not really a big deal to me, but I can see an Operator
asking questions about it someday ;-)
Thanks and best regards,
Terry
Re: No info for Accumulo Master on Monitor Overview page
Posted by "Terry P." <te...@gmail.com>.
FYI everyone, turns out that changing the permissions at the hdfs level
does NOT persist across an HDFS restart, so after restarting your Accumulo
Monitor Overview page will have empty boxes again (at least with Accumulo
1.4.2).
The permanent fix is to add the following two properties to the
hdfs-site.xml config file:
<property><name>dfs.permissions</name><value>true</value></property>
<property><name>dfs.permissions.supergroup</name><value>hadoop</value></property>
In my case, I had already added the accumulo account to the hadoop group so
that's why I set the value to 'hadoop'. That's also why changing the perms
at the HDFS level worked until HDFS was restarted, but the datanode sets
the perms back to 700 during its startup routine (again, this is CDH3u5).
Cheers folks.
On Fri, Aug 2, 2013 at 4:41 PM, Terry P. <te...@gmail.com> wrote:
> Thanks Charles, I finally got around to looking at this on a new
> installation. I didn't have to set the monitor to DEBUG mode though to
> find the faulty path, as the default monitor_logger.xml config file creates
> a DEBUG log (though you may have disabled it).
>
> For those that are still on 1.4.x, take a look at the monitor DEBUG log
> file. Look for an entry with AccessControlException: Permission denied,
> and the path it points to. My mapred system directory is /mapred-system,
> so for me the fix was:
>
> sudo -u hdfs hadoop fs -chmod 775 /mapred-system
>
> With that, the Accumulo Master box on the Monitor's Overview page
> populates as it should. Not certain if the hadoop permission fix will
> persist across restarts though, as doesn't the daemon set the perms each
> time? (I'm running CDH3u5).
>
> -Terry
>
>
>
> On Fri, Jun 7, 2013 at 8:56 AM, Ott, Charles H. <CH...@saic.com>wrote:
>
>> I had the exact same issue a couple weeks back.****
>>
>> ** **
>>
>> I was able to find out what specific HDFS path had permission issues by
>> switching the Monitor logger to DEBUG mode. Then a chmod 775 fixed the
>> issue by giving the group write permissions.****
>>
>> ** **
>>
>> *From:* user-return-2668-CHARLES.H.OTT=saic.com@accumulo.apache.org[mailto:
>> user-return-2668-CHARLES.H.OTT=saic.com@accumulo.apache.org] *On Behalf
>> Of *Terry P.
>> *Sent:* Thursday, June 06, 2013 6:13 PM
>> *To:* user@accumulo.apache.org
>> *Subject:* Re: No info for Accumulo Master on Monitor Overview page****
>>
>> ** **
>>
>> In our installation the hdfs account has hdfs as his primary group and
>> hadoop as a secondary group. Our accumulo account has accumulo as his
>> primary group.****
>>
>> I added both hdfs and hadoop as secondary groups for the accumulo
>> account, but no joy -- still a blank Accumulo Master summary box on the
>> overview page.****
>>
>> So do both the accumulo and hdfs users have to have the same *primary *group
>> then?
>>
>> I put accumulo into a separate group in order to protect the shared
>> secret in accumulo-site.xml, but I guess the better approach is to have
>> both accumulo and hdfs in the same Linux group and just protect this file
>> by making it readable by owner only? If both accumulo and hdfs users must
>> have the same primary group, looks like that's about the only option.****
>>
>> ** **
>>
>> On Thu, Jun 6, 2013 at 4:57 PM, Eric Newton <er...@gmail.com>
>> wrote:****
>>
>> Yea, that's a 1.4.2 issue. You can work around it by putting the
>> accumulo and hdfs accounts into the same group, or just ignore the display
>> issue.****
>>
>> ** **
>>
>> ACCUMULO-1282****
>>
>> ** **
>>
>> -Eric****
>>
>> ** **
>>
>> ** **
>>
>> On Thu, Jun 6, 2013 at 5:48 PM, Terry P. <te...@gmail.com> wrote:****
>>
>> Hi Eric,
>> Sorry I should have thought of including a screenshot. I've attached one
>> to this email -- first one I've done, I hope it works just like a regular
>> attachment?****
>>
>> To ensure I had a clean monitor log, I shutdown Accumulo and started it
>> up again. First there are all the [server.Accumulo] entries, here's the
>> head of that which looks good:
>>
>> 2013-06-06 21:34:37,823 [server.Accumulo] INFO : monitor starting
>> 2013-06-06 21:34:37,824 [server.Accumulo] INFO : Instance
>> 22e13d96-d1ac-45c7-9d90-ae2e813e983c
>> 2013-06-06 21:34:37,841 [server.Accumulo] INFO : Data Version 4
>> 2013-06-06 21:34:37,841 [server.Accumulo] INFO : Attempting to talk to
>> zookeeper
>> 2013-06-06 21:34:38,026 [server.Accumulo] INFO : Zookeeper connected and
>> initialized, attemping to talk to HDFS
>> 2013-06-06 21:34:38,030 [server.Accumulo] INFO : Connected to HDFS
>> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.delay = 5m
>> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.start = 30s
>> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.port.client = 50091
>> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.threads.delete = 16
>> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : general.classpaths =
>> $ACCUMULO_HOME/src/server/target/classes/,
>> $ACCUMULO_HOME/src/core/target/classes/,
>> $ACCUMULO_HOME/src/start/target/classes/,
>> $ACCUMULO_HOME/src/examples/target/classes/,
>> $ACCUMULO_HOME/lib/[^.].$ACCUMULO_VERSION.jar,
>> $ACCUMULO_HOME/lib/[^.].*.jar,
>> $ZOOKEEPER_HOME/zookeeper[^.].*.jar,
>> $HADOOP_HOME/conf,
>> $HADOOP_HOME/[^.].*.jar,
>> $HADOOP_HOME/lib/[^.].*.jar,****
>>
>> Then at the very end I found (with only INFO messages in between):
>>
>> 2013-06-06 21:34:39,132 [monitor.Monitor] WARN :
>> java.lang.NullPointerException
>> at
>> org.apache.accumulo.server.monitor.Monitor.fetchData(Monitor.java:308)
>> at
>> org.apache.accumulo.server.monitor.Monitor$2.run(Monitor.java:476)
>> at
>> org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
>> at java.lang.Thread.run(Unknown Source)****
>>
>> That's the last entry in the monitor log.
>>
>> The second image I attached shows the Master Status page, showing there
>> is no problem with that display.****
>>
>> This is Accumulo 1.4.2 running as linux account 'accumulo' -- HDFS runs
>> as the hdfs account, which is why the NameNode section shows "permission
>> denied." NameNode and Accumulo Master reside on the same server. OS is
>> Red Hat Enterprise Linux 6.3 64-bit.****
>>
>>
>> Thanks in advance,
>> Terry****
>>
>> ** **
>>
>> On Thu, Jun 6, 2013 at 2:58 PM, Eric Newton <er...@gmail.com>
>> wrote:****
>>
>> Would you have a screenshot? Any messages in the monitor log?****
>>
>> ** **
>>
>> -Eric****
>>
>> ** **
>>
>> ** **
>>
>> On Thu, Jun 6, 2013 at 2:44 PM, Terry P. <te...@gmail.com> wrote:****
>>
>> Greetings all,
>> We have two Accumulo clusters built from essentially the same build
>> procedure. Everything is running well with the exception that on the
>> Accumulo Monitor Overview web page, the Accumulo Master box is blank -- the
>> "Accumulo Master" title/hyperlink works (takes you to the "Master Server"
>> page), but nothing is in the box. The NameNode, JobTracker, and Zookeeper
>> boxes are all populated.****
>>
>> On the Master Server page, I see the Master Status that shows 3 Online
>> Tablet Servers, Loggers, Last GC, etc. It also shows the Table List. The
>> Tablet Servers page looks good.****
>>
>> In these small clusters we have one Accumulo Master that runs on the
>> NameNode, 3 DataNodes, 3 Zookeepers. I'm in the process of firing up a
>> Secondary NameNode which will also run an Alternate Accumulo Master in case
>> the Master on the NameNode were to fail for some reason, but I have not
>> fired up the Accumulo processes on the Secondary NameNode server yet at all.
>> ****
>>
>> ** **
>>
>> Any idea where I can look to try and determine why the Accumulo Master
>> box is empty? It's not really a big deal to me, but I can see an Operator
>> asking questions about it someday ;-)****
>>
>> Thanks and best regards,
>> Terry****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>
>
Re: No info for Accumulo Master on Monitor Overview page
Posted by "Terry P." <te...@gmail.com>.
Thanks Charles, I finally got around to looking at this on a new
installation. I didn't have to set the monitor to DEBUG mode though to
find the faulty path, as the default monitor_logger.xml config file creates
a DEBUG log (though you may have disabled it).
For those that are still on 1.4.x, take a look at the monitor DEBUG log
file. Look for an entry with AccessControlException: Permission denied,
and the path it points to. My mapred system directory is /mapred-system,
so for me the fix was:
sudo -u hdfs hadoop fs -chmod 775 /mapred-system
With that, the Accumulo Master box on the Monitor's Overview page populates
as it should. Not certain if the hadoop permission fix will persist across
restarts though, as doesn't the daemon set the perms each time? (I'm
running CDH3u5).
-Terry
On Fri, Jun 7, 2013 at 8:56 AM, Ott, Charles H. <CH...@saic.com>wrote:
> I had the exact same issue a couple weeks back.****
>
> ** **
>
> I was able to find out what specific HDFS path had permission issues by
> switching the Monitor logger to DEBUG mode. Then a chmod 775 fixed the
> issue by giving the group write permissions.****
>
> ** **
>
> *From:* user-return-2668-CHARLES.H.OTT=saic.com@accumulo.apache.org[mailto:
> user-return-2668-CHARLES.H.OTT=saic.com@accumulo.apache.org] *On Behalf
> Of *Terry P.
> *Sent:* Thursday, June 06, 2013 6:13 PM
> *To:* user@accumulo.apache.org
> *Subject:* Re: No info for Accumulo Master on Monitor Overview page****
>
> ** **
>
> In our installation the hdfs account has hdfs as his primary group and
> hadoop as a secondary group. Our accumulo account has accumulo as his
> primary group.****
>
> I added both hdfs and hadoop as secondary groups for the accumulo account,
> but no joy -- still a blank Accumulo Master summary box on the overview
> page.****
>
> So do both the accumulo and hdfs users have to have the same *primary *group
> then?
>
> I put accumulo into a separate group in order to protect the shared secret
> in accumulo-site.xml, but I guess the better approach is to have both
> accumulo and hdfs in the same Linux group and just protect this file by
> making it readable by owner only? If both accumulo and hdfs users must
> have the same primary group, looks like that's about the only option.****
>
> ** **
>
> On Thu, Jun 6, 2013 at 4:57 PM, Eric Newton <er...@gmail.com> wrote:
> ****
>
> Yea, that's a 1.4.2 issue. You can work around it by putting the accumulo
> and hdfs accounts into the same group, or just ignore the display issue.**
> **
>
> ** **
>
> ACCUMULO-1282****
>
> ** **
>
> -Eric****
>
> ** **
>
> ** **
>
> On Thu, Jun 6, 2013 at 5:48 PM, Terry P. <te...@gmail.com> wrote:****
>
> Hi Eric,
> Sorry I should have thought of including a screenshot. I've attached one
> to this email -- first one I've done, I hope it works just like a regular
> attachment?****
>
> To ensure I had a clean monitor log, I shutdown Accumulo and started it up
> again. First there are all the [server.Accumulo] entries, here's the head
> of that which looks good:
>
> 2013-06-06 21:34:37,823 [server.Accumulo] INFO : monitor starting
> 2013-06-06 21:34:37,824 [server.Accumulo] INFO : Instance
> 22e13d96-d1ac-45c7-9d90-ae2e813e983c
> 2013-06-06 21:34:37,841 [server.Accumulo] INFO : Data Version 4
> 2013-06-06 21:34:37,841 [server.Accumulo] INFO : Attempting to talk to
> zookeeper
> 2013-06-06 21:34:38,026 [server.Accumulo] INFO : Zookeeper connected and
> initialized, attemping to talk to HDFS
> 2013-06-06 21:34:38,030 [server.Accumulo] INFO : Connected to HDFS
> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.delay = 5m
> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.start = 30s
> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.port.client = 50091
> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.threads.delete = 16
> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : general.classpaths =
> $ACCUMULO_HOME/src/server/target/classes/,
> $ACCUMULO_HOME/src/core/target/classes/,
> $ACCUMULO_HOME/src/start/target/classes/,
> $ACCUMULO_HOME/src/examples/target/classes/,
> $ACCUMULO_HOME/lib/[^.].$ACCUMULO_VERSION.jar,
> $ACCUMULO_HOME/lib/[^.].*.jar,
> $ZOOKEEPER_HOME/zookeeper[^.].*.jar,
> $HADOOP_HOME/conf,
> $HADOOP_HOME/[^.].*.jar,
> $HADOOP_HOME/lib/[^.].*.jar,****
>
> Then at the very end I found (with only INFO messages in between):
>
> 2013-06-06 21:34:39,132 [monitor.Monitor] WARN :
> java.lang.NullPointerException
> at
> org.apache.accumulo.server.monitor.Monitor.fetchData(Monitor.java:308)
> at
> org.apache.accumulo.server.monitor.Monitor$2.run(Monitor.java:476)
> at
> org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
> at java.lang.Thread.run(Unknown Source)****
>
> That's the last entry in the monitor log.
>
> The second image I attached shows the Master Status page, showing there is
> no problem with that display.****
>
> This is Accumulo 1.4.2 running as linux account 'accumulo' -- HDFS runs as
> the hdfs account, which is why the NameNode section shows "permission
> denied." NameNode and Accumulo Master reside on the same server. OS is
> Red Hat Enterprise Linux 6.3 64-bit.****
>
>
> Thanks in advance,
> Terry****
>
> ** **
>
> On Thu, Jun 6, 2013 at 2:58 PM, Eric Newton <er...@gmail.com> wrote:
> ****
>
> Would you have a screenshot? Any messages in the monitor log?****
>
> ** **
>
> -Eric****
>
> ** **
>
> ** **
>
> On Thu, Jun 6, 2013 at 2:44 PM, Terry P. <te...@gmail.com> wrote:****
>
> Greetings all,
> We have two Accumulo clusters built from essentially the same build
> procedure. Everything is running well with the exception that on the
> Accumulo Monitor Overview web page, the Accumulo Master box is blank -- the
> "Accumulo Master" title/hyperlink works (takes you to the "Master Server"
> page), but nothing is in the box. The NameNode, JobTracker, and Zookeeper
> boxes are all populated.****
>
> On the Master Server page, I see the Master Status that shows 3 Online
> Tablet Servers, Loggers, Last GC, etc. It also shows the Table List. The
> Tablet Servers page looks good.****
>
> In these small clusters we have one Accumulo Master that runs on the
> NameNode, 3 DataNodes, 3 Zookeepers. I'm in the process of firing up a
> Secondary NameNode which will also run an Alternate Accumulo Master in case
> the Master on the NameNode were to fail for some reason, but I have not
> fired up the Accumulo processes on the Secondary NameNode server yet at all.
> ****
>
> ** **
>
> Any idea where I can look to try and determine why the Accumulo Master box
> is empty? It's not really a big deal to me, but I can see an Operator
> asking questions about it someday ;-)****
>
> Thanks and best regards,
> Terry****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
RE: No info for Accumulo Master on Monitor Overview page
Posted by "Ott, Charles H." <CH...@saic.com>.
I had the exact same issue a couple weeks back.
I was able to find out what specific HDFS path had permission issues by switching the Monitor logger to DEBUG mode. Then a chmod 775 fixed the issue by giving the group write permissions.
From: user-return-2668-CHARLES.H.OTT=saic.com@accumulo.apache.org [mailto:user-return-2668-CHARLES.H.OTT=saic.com@accumulo.apache.org] On Behalf Of Terry P.
Sent: Thursday, June 06, 2013 6:13 PM
To: user@accumulo.apache.org
Subject: Re: No info for Accumulo Master on Monitor Overview page
In our installation the hdfs account has hdfs as his primary group and hadoop as a secondary group. Our accumulo account has accumulo as his primary group.
I added both hdfs and hadoop as secondary groups for the accumulo account, but no joy -- still a blank Accumulo Master summary box on the overview page.
So do both the accumulo and hdfs users have to have the same primary group then?
I put accumulo into a separate group in order to protect the shared secret in accumulo-site.xml, but I guess the better approach is to have both accumulo and hdfs in the same Linux group and just protect this file by making it readable by owner only? If both accumulo and hdfs users must have the same primary group, looks like that's about the only option.
On Thu, Jun 6, 2013 at 4:57 PM, Eric Newton <er...@gmail.com> wrote:
Yea, that's a 1.4.2 issue. You can work around it by putting the accumulo and hdfs accounts into the same group, or just ignore the display issue.
ACCUMULO-1282
-Eric
On Thu, Jun 6, 2013 at 5:48 PM, Terry P. <te...@gmail.com> wrote:
Hi Eric,
Sorry I should have thought of including a screenshot. I've attached one to this email -- first one I've done, I hope it works just like a regular attachment?
To ensure I had a clean monitor log, I shutdown Accumulo and started it up again. First there are all the [server.Accumulo] entries, here's the head of that which looks good:
2013-06-06 21:34:37,823 [server.Accumulo] INFO : monitor starting
2013-06-06 21:34:37,824 [server.Accumulo] INFO : Instance 22e13d96-d1ac-45c7-9d90-ae2e813e983c
2013-06-06 21:34:37,841 [server.Accumulo] INFO : Data Version 4
2013-06-06 21:34:37,841 [server.Accumulo] INFO : Attempting to talk to zookeeper
2013-06-06 21:34:38,026 [server.Accumulo] INFO : Zookeeper connected and initialized, attemping to talk to HDFS
2013-06-06 21:34:38,030 [server.Accumulo] INFO : Connected to HDFS
2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.delay = 5m
2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.start = 30s
2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.port.client = 50091
2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.threads.delete = 16
2013-06-06 21:34:38,034 [server.Accumulo] INFO : general.classpaths =
$ACCUMULO_HOME/src/server/target/classes/,
$ACCUMULO_HOME/src/core/target/classes/,
$ACCUMULO_HOME/src/start/target/classes/,
$ACCUMULO_HOME/src/examples/target/classes/,
$ACCUMULO_HOME/lib/[^.].$ACCUMULO_VERSION.jar,
$ACCUMULO_HOME/lib/[^.].*.jar,
$ZOOKEEPER_HOME/zookeeper[^.].*.jar,
$HADOOP_HOME/conf,
$HADOOP_HOME/[^.].*.jar,
$HADOOP_HOME/lib/[^.].*.jar,
Then at the very end I found (with only INFO messages in between):
2013-06-06 21:34:39,132 [monitor.Monitor] WARN :
java.lang.NullPointerException
at org.apache.accumulo.server.monitor.Monitor.fetchData(Monitor.java:308)
at org.apache.accumulo.server.monitor.Monitor$2.run(Monitor.java:476)
at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
at java.lang.Thread.run(Unknown Source)
That's the last entry in the monitor log.
The second image I attached shows the Master Status page, showing there is no problem with that display.
This is Accumulo 1.4.2 running as linux account 'accumulo' -- HDFS runs as the hdfs account, which is why the NameNode section shows "permission denied." NameNode and Accumulo Master reside on the same server. OS is Red Hat Enterprise Linux 6.3 64-bit.
Thanks in advance,
Terry
On Thu, Jun 6, 2013 at 2:58 PM, Eric Newton <er...@gmail.com> wrote:
Would you have a screenshot? Any messages in the monitor log?
-Eric
On Thu, Jun 6, 2013 at 2:44 PM, Terry P. <te...@gmail.com> wrote:
Greetings all,
We have two Accumulo clusters built from essentially the same build procedure. Everything is running well with the exception that on the Accumulo Monitor Overview web page, the Accumulo Master box is blank -- the "Accumulo Master" title/hyperlink works (takes you to the "Master Server" page), but nothing is in the box. The NameNode, JobTracker, and Zookeeper boxes are all populated.
On the Master Server page, I see the Master Status that shows 3 Online Tablet Servers, Loggers, Last GC, etc. It also shows the Table List. The Tablet Servers page looks good.
In these small clusters we have one Accumulo Master that runs on the NameNode, 3 DataNodes, 3 Zookeepers. I'm in the process of firing up a Secondary NameNode which will also run an Alternate Accumulo Master in case the Master on the NameNode were to fail for some reason, but I have not fired up the Accumulo processes on the Secondary NameNode server yet at all.
Any idea where I can look to try and determine why the Accumulo Master box is empty? It's not really a big deal to me, but I can see an Operator asking questions about it someday ;-)
Thanks and best regards,
Terry
Re: No info for Accumulo Master on Monitor Overview page
Posted by "Terry P." <te...@gmail.com>.
In our installation the hdfs account has hdfs as his primary group and
hadoop as a secondary group. Our accumulo account has accumulo as his
primary group.
I added both hdfs and hadoop as secondary groups for the accumulo account,
but no joy -- still a blank Accumulo Master summary box on the overview
page.
So do both the accumulo and hdfs users have to have the same *primary *group
then?
I put accumulo into a separate group in order to protect the shared secret
in accumulo-site.xml, but I guess the better approach is to have both
accumulo and hdfs in the same Linux group and just protect this file by
making it readable by owner only? If both accumulo and hdfs users must
have the same primary group, looks like that's about the only option.
On Thu, Jun 6, 2013 at 4:57 PM, Eric Newton <er...@gmail.com> wrote:
> Yea, that's a 1.4.2 issue. You can work around it by putting the accumulo
> and hdfs accounts into the same group, or just ignore the display issue.
>
> ACCUMULO-1282
>
> -Eric
>
>
>
> On Thu, Jun 6, 2013 at 5:48 PM, Terry P. <te...@gmail.com> wrote:
>
>> Hi Eric,
>> Sorry I should have thought of including a screenshot. I've attached one
>> to this email -- first one I've done, I hope it works just like a regular
>> attachment?
>>
>> To ensure I had a clean monitor log, I shutdown Accumulo and started it
>> up again. First there are all the [server.Accumulo] entries, here's the
>> head of that which looks good:
>>
>> 2013-06-06 21:34:37,823 [server.Accumulo] INFO : monitor starting
>> 2013-06-06 21:34:37,824 [server.Accumulo] INFO : Instance
>> 22e13d96-d1ac-45c7-9d90-ae2e813e983c
>> 2013-06-06 21:34:37,841 [server.Accumulo] INFO : Data Version 4
>> 2013-06-06 21:34:37,841 [server.Accumulo] INFO : Attempting to talk to
>> zookeeper
>> 2013-06-06 21:34:38,026 [server.Accumulo] INFO : Zookeeper connected and
>> initialized, attemping to talk to HDFS
>> 2013-06-06 21:34:38,030 [server.Accumulo] INFO : Connected to HDFS
>> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.delay = 5m
>> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.start = 30s
>> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.port.client = 50091
>> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.threads.delete = 16
>> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : general.classpaths =
>> $ACCUMULO_HOME/src/server/target/classes/,
>> $ACCUMULO_HOME/src/core/target/classes/,
>> $ACCUMULO_HOME/src/start/target/classes/,
>> $ACCUMULO_HOME/src/examples/target/classes/,
>> $ACCUMULO_HOME/lib/[^.].$ACCUMULO_VERSION.jar,
>> $ACCUMULO_HOME/lib/[^.].*.jar,
>> $ZOOKEEPER_HOME/zookeeper[^.].*.jar,
>> $HADOOP_HOME/conf,
>> $HADOOP_HOME/[^.].*.jar,
>> $HADOOP_HOME/lib/[^.].*.jar,
>>
>> Then at the very end I found (with only INFO messages in between):
>>
>> 2013-06-06 21:34:39,132 [monitor.Monitor] WARN :
>> java.lang.NullPointerException
>> at
>> org.apache.accumulo.server.monitor.Monitor.fetchData(Monitor.java:308)
>> at
>> org.apache.accumulo.server.monitor.Monitor$2.run(Monitor.java:476)
>> at
>> org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
>> at java.lang.Thread.run(Unknown Source)
>>
>> That's the last entry in the monitor log.
>>
>> The second image I attached shows the Master Status page, showing there
>> is no problem with that display.
>>
>> This is Accumulo 1.4.2 running as linux account 'accumulo' -- HDFS runs
>> as the hdfs account, which is why the NameNode section shows "permission
>> denied." NameNode and Accumulo Master reside on the same server. OS is
>> Red Hat Enterprise Linux 6.3 64-bit.
>>
>> Thanks in advance,
>> Terry
>>
>>
>>
>> On Thu, Jun 6, 2013 at 2:58 PM, Eric Newton <er...@gmail.com>wrote:
>>
>>> Would you have a screenshot? Any messages in the monitor log?
>>>
>>> -Eric
>>>
>>>
>>>
>>> On Thu, Jun 6, 2013 at 2:44 PM, Terry P. <te...@gmail.com> wrote:
>>>
>>>> Greetings all,
>>>> We have two Accumulo clusters built from essentially the same build
>>>> procedure. Everything is running well with the exception that on the
>>>> Accumulo Monitor Overview web page, the Accumulo Master box is blank -- the
>>>> "Accumulo Master" title/hyperlink works (takes you to the "Master Server"
>>>> page), but nothing is in the box. The NameNode, JobTracker, and Zookeeper
>>>> boxes are all populated.
>>>>
>>>> On the Master Server page, I see the Master Status that shows 3 Online
>>>> Tablet Servers, Loggers, Last GC, etc. It also shows the Table List. The
>>>> Tablet Servers page looks good.
>>>>
>>>> In these small clusters we have one Accumulo Master that runs on the
>>>> NameNode, 3 DataNodes, 3 Zookeepers. I'm in the process of firing up a
>>>> Secondary NameNode which will also run an Alternate Accumulo Master in case
>>>> the Master on the NameNode were to fail for some reason, but I have not
>>>> fired up the Accumulo processes on the Secondary NameNode server yet at all.
>>>>
>>>> Any idea where I can look to try and determine why the Accumulo Master
>>>> box is empty? It's not really a big deal to me, but I can see an Operator
>>>> asking questions about it someday ;-)
>>>>
>>>> Thanks and best regards,
>>>> Terry
>>>>
>>>>
>>>
>>
>
Re: No info for Accumulo Master on Monitor Overview page
Posted by Eric Newton <er...@gmail.com>.
Yea, that's a 1.4.2 issue. You can work around it by putting the accumulo
and hdfs accounts into the same group, or just ignore the display issue.
ACCUMULO-1282
-Eric
On Thu, Jun 6, 2013 at 5:48 PM, Terry P. <te...@gmail.com> wrote:
> Hi Eric,
> Sorry I should have thought of including a screenshot. I've attached one
> to this email -- first one I've done, I hope it works just like a regular
> attachment?
>
> To ensure I had a clean monitor log, I shutdown Accumulo and started it up
> again. First there are all the [server.Accumulo] entries, here's the head
> of that which looks good:
>
> 2013-06-06 21:34:37,823 [server.Accumulo] INFO : monitor starting
> 2013-06-06 21:34:37,824 [server.Accumulo] INFO : Instance
> 22e13d96-d1ac-45c7-9d90-ae2e813e983c
> 2013-06-06 21:34:37,841 [server.Accumulo] INFO : Data Version 4
> 2013-06-06 21:34:37,841 [server.Accumulo] INFO : Attempting to talk to
> zookeeper
> 2013-06-06 21:34:38,026 [server.Accumulo] INFO : Zookeeper connected and
> initialized, attemping to talk to HDFS
> 2013-06-06 21:34:38,030 [server.Accumulo] INFO : Connected to HDFS
> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.delay = 5m
> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.start = 30s
> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.port.client = 50091
> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.threads.delete = 16
> 2013-06-06 21:34:38,034 [server.Accumulo] INFO : general.classpaths =
> $ACCUMULO_HOME/src/server/target/classes/,
> $ACCUMULO_HOME/src/core/target/classes/,
> $ACCUMULO_HOME/src/start/target/classes/,
> $ACCUMULO_HOME/src/examples/target/classes/,
> $ACCUMULO_HOME/lib/[^.].$ACCUMULO_VERSION.jar,
> $ACCUMULO_HOME/lib/[^.].*.jar,
> $ZOOKEEPER_HOME/zookeeper[^.].*.jar,
> $HADOOP_HOME/conf,
> $HADOOP_HOME/[^.].*.jar,
> $HADOOP_HOME/lib/[^.].*.jar,
>
> Then at the very end I found (with only INFO messages in between):
>
> 2013-06-06 21:34:39,132 [monitor.Monitor] WARN :
> java.lang.NullPointerException
> at
> org.apache.accumulo.server.monitor.Monitor.fetchData(Monitor.java:308)
> at
> org.apache.accumulo.server.monitor.Monitor$2.run(Monitor.java:476)
> at
> org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
> at java.lang.Thread.run(Unknown Source)
>
> That's the last entry in the monitor log.
>
> The second image I attached shows the Master Status page, showing there is
> no problem with that display.
>
> This is Accumulo 1.4.2 running as linux account 'accumulo' -- HDFS runs as
> the hdfs account, which is why the NameNode section shows "permission
> denied." NameNode and Accumulo Master reside on the same server. OS is
> Red Hat Enterprise Linux 6.3 64-bit.
>
> Thanks in advance,
> Terry
>
>
>
> On Thu, Jun 6, 2013 at 2:58 PM, Eric Newton <er...@gmail.com> wrote:
>
>> Would you have a screenshot? Any messages in the monitor log?
>>
>> -Eric
>>
>>
>>
>> On Thu, Jun 6, 2013 at 2:44 PM, Terry P. <te...@gmail.com> wrote:
>>
>>> Greetings all,
>>> We have two Accumulo clusters built from essentially the same build
>>> procedure. Everything is running well with the exception that on the
>>> Accumulo Monitor Overview web page, the Accumulo Master box is blank -- the
>>> "Accumulo Master" title/hyperlink works (takes you to the "Master Server"
>>> page), but nothing is in the box. The NameNode, JobTracker, and Zookeeper
>>> boxes are all populated.
>>>
>>> On the Master Server page, I see the Master Status that shows 3 Online
>>> Tablet Servers, Loggers, Last GC, etc. It also shows the Table List. The
>>> Tablet Servers page looks good.
>>>
>>> In these small clusters we have one Accumulo Master that runs on the
>>> NameNode, 3 DataNodes, 3 Zookeepers. I'm in the process of firing up a
>>> Secondary NameNode which will also run an Alternate Accumulo Master in case
>>> the Master on the NameNode were to fail for some reason, but I have not
>>> fired up the Accumulo processes on the Secondary NameNode server yet at all.
>>>
>>> Any idea where I can look to try and determine why the Accumulo Master
>>> box is empty? It's not really a big deal to me, but I can see an Operator
>>> asking questions about it someday ;-)
>>>
>>> Thanks and best regards,
>>> Terry
>>>
>>>
>>
>
Re: No info for Accumulo Master on Monitor Overview page
Posted by "Terry P." <te...@gmail.com>.
Hi Eric,
Sorry I should have thought of including a screenshot. I've attached one
to this email -- first one I've done, I hope it works just like a regular
attachment?
To ensure I had a clean monitor log, I shutdown Accumulo and started it up
again. First there are all the [server.Accumulo] entries, here's the head
of that which looks good:
2013-06-06 21:34:37,823 [server.Accumulo] INFO : monitor starting
2013-06-06 21:34:37,824 [server.Accumulo] INFO : Instance
22e13d96-d1ac-45c7-9d90-ae2e813e983c
2013-06-06 21:34:37,841 [server.Accumulo] INFO : Data Version 4
2013-06-06 21:34:37,841 [server.Accumulo] INFO : Attempting to talk to
zookeeper
2013-06-06 21:34:38,026 [server.Accumulo] INFO : Zookeeper connected and
initialized, attemping to talk to HDFS
2013-06-06 21:34:38,030 [server.Accumulo] INFO : Connected to HDFS
2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.delay = 5m
2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.cycle.start = 30s
2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.port.client = 50091
2013-06-06 21:34:38,034 [server.Accumulo] INFO : gc.threads.delete = 16
2013-06-06 21:34:38,034 [server.Accumulo] INFO : general.classpaths =
$ACCUMULO_HOME/src/server/target/classes/,
$ACCUMULO_HOME/src/core/target/classes/,
$ACCUMULO_HOME/src/start/target/classes/,
$ACCUMULO_HOME/src/examples/target/classes/,
$ACCUMULO_HOME/lib/[^.].$ACCUMULO_VERSION.jar,
$ACCUMULO_HOME/lib/[^.].*.jar,
$ZOOKEEPER_HOME/zookeeper[^.].*.jar,
$HADOOP_HOME/conf,
$HADOOP_HOME/[^.].*.jar,
$HADOOP_HOME/lib/[^.].*.jar,
Then at the very end I found (with only INFO messages in between):
2013-06-06 21:34:39,132 [monitor.Monitor] WARN :
java.lang.NullPointerException
at
org.apache.accumulo.server.monitor.Monitor.fetchData(Monitor.java:308)
at
org.apache.accumulo.server.monitor.Monitor$2.run(Monitor.java:476)
at
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
at java.lang.Thread.run(Unknown Source)
That's the last entry in the monitor log.
The second image I attached shows the Master Status page, showing there is
no problem with that display.
This is Accumulo 1.4.2 running as linux account 'accumulo' -- HDFS runs as
the hdfs account, which is why the NameNode section shows "permission
denied." NameNode and Accumulo Master reside on the same server. OS is
Red Hat Enterprise Linux 6.3 64-bit.
Thanks in advance,
Terry
On Thu, Jun 6, 2013 at 2:58 PM, Eric Newton <er...@gmail.com> wrote:
> Would you have a screenshot? Any messages in the monitor log?
>
> -Eric
>
>
>
> On Thu, Jun 6, 2013 at 2:44 PM, Terry P. <te...@gmail.com> wrote:
>
>> Greetings all,
>> We have two Accumulo clusters built from essentially the same build
>> procedure. Everything is running well with the exception that on the
>> Accumulo Monitor Overview web page, the Accumulo Master box is blank -- the
>> "Accumulo Master" title/hyperlink works (takes you to the "Master Server"
>> page), but nothing is in the box. The NameNode, JobTracker, and Zookeeper
>> boxes are all populated.
>>
>> On the Master Server page, I see the Master Status that shows 3 Online
>> Tablet Servers, Loggers, Last GC, etc. It also shows the Table List. The
>> Tablet Servers page looks good.
>>
>> In these small clusters we have one Accumulo Master that runs on the
>> NameNode, 3 DataNodes, 3 Zookeepers. I'm in the process of firing up a
>> Secondary NameNode which will also run an Alternate Accumulo Master in case
>> the Master on the NameNode were to fail for some reason, but I have not
>> fired up the Accumulo processes on the Secondary NameNode server yet at all.
>>
>> Any idea where I can look to try and determine why the Accumulo Master
>> box is empty? It's not really a big deal to me, but I can see an Operator
>> asking questions about it someday ;-)
>>
>> Thanks and best regards,
>> Terry
>>
>>
>
Re: No info for Accumulo Master on Monitor Overview page
Posted by Eric Newton <er...@gmail.com>.
Would you have a screenshot? Any messages in the monitor log?
-Eric
On Thu, Jun 6, 2013 at 2:44 PM, Terry P. <te...@gmail.com> wrote:
> Greetings all,
> We have two Accumulo clusters built from essentially the same build
> procedure. Everything is running well with the exception that on the
> Accumulo Monitor Overview web page, the Accumulo Master box is blank -- the
> "Accumulo Master" title/hyperlink works (takes you to the "Master Server"
> page), but nothing is in the box. The NameNode, JobTracker, and Zookeeper
> boxes are all populated.
>
> On the Master Server page, I see the Master Status that shows 3 Online
> Tablet Servers, Loggers, Last GC, etc. It also shows the Table List. The
> Tablet Servers page looks good.
>
> In these small clusters we have one Accumulo Master that runs on the
> NameNode, 3 DataNodes, 3 Zookeepers. I'm in the process of firing up a
> Secondary NameNode which will also run an Alternate Accumulo Master in case
> the Master on the NameNode were to fail for some reason, but I have not
> fired up the Accumulo processes on the Secondary NameNode server yet at all.
>
> Any idea where I can look to try and determine why the Accumulo Master box
> is empty? It's not really a big deal to me, but I can see an Operator
> asking questions about it someday ;-)
>
> Thanks and best regards,
> Terry
>
>