You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by Jim Klucar <kl...@gmail.com> on 2012/07/16 04:52:02 UTC
getMasterStats problems
I'm trying to get the master stats in the same way the monitor server
does, but I keep having issues.
The code I'm using is from the GetMasterStats.java program:
MasterClientService.Iface client = null;
MasterMonitorInfo stats = null;
try {
client = MasterClient.getConnectionWithRetry(HdfsZooInstance.getInstance());
stats = client.getMasterStats(null,
SecurityConstants.getSystemCredentials());
} finally {
if (client != null)
MasterClient.close(client);
}
The problem is that the HdfsZooInstance can never find the instance
data, even though it's there. I can create a ZooKeeperInstance just
fine, but passing that into the MasterClient fails the same way.
Caused by: java.lang.RuntimeException: Accumulo not initialized, there
is no instance id at /accumulo/instance_id
at org.apache.accumulo.core.client.ZooKeeperInstance.getInstanceIDFromHdfs(ZooKeeperInstance.java:263)
at org.apache.accumulo.server.conf.ZooConfiguration.getInstance(ZooConfiguration.java:65)
at org.apache.accumulo.server.conf.ServerConfiguration.getZooConfiguration(ServerConfiguration.java:49)
at org.apache.accumulo.server.conf.ServerConfiguration.getSystemConfiguration(ServerConfiguration.java:58)
at org.apache.accumulo.server.client.HdfsZooInstance.<init>(HdfsZooInstance.java:62)
at org.apache.accumulo.server.client.HdfsZooInstance.getInstance(HdfsZooInstance.java:70)
at org.apache.accumulo.server.security.SecurityConstants.makeSystemPassword(SecurityConstants.java:58)
at org.apache.accumulo.server.security.SecurityConstants.<clinit>(SecurityConstants.java:43)
... 91 more
Any ideas as to what I'm missing?
Re: getMasterStats problems
Posted by Jim Klucar <kl...@gmail.com>.
Looks like I didn't have the hadoop configs on my classpath, but the
accumulo-site.xml was on the classpath. Once I added the hadoop files,
I was able to create a ZooKeeperInstance, get the system auths, and
the MasterStatus.
I did this in Clojure/Leiningen though and it looks like Clojure is
somehow evaluating static classes / methods etc at compile time, so I
have to have accumulo running to get it to work. This is probably more
of a quirk of Clojure than anything else, but some of the classes
referenced statically use the HdfsZooInstance in them, so that has to
be working for me to compile. weird.
I'll give Adam's route a try and report how it goes.
On Mon, Jul 16, 2012 at 8:08 AM, Eric Newton <er...@gmail.com> wrote:
> You also need to have the hadoop configuration in your classpath so
> the client can find HDFS, and not the local file system.
>
> -Eric
>
> On Sun, Jul 15, 2012 at 11:09 PM, Adam Fuchs <af...@apache.org> wrote:
>> Jim,
>>
>> The HdfsZooInstance looks for accumulo-site.xml on the classpath to find the
>> directory in HDFS to look for the instance ID. If accumulo-site.xml is not
>> on the classpath then it will default to /accumulo, which is probably
>> different from the directory you are using. accumulo-site.xml also includes
>> a shared secret which is used to construct the system credentials. This is
>> the standard way that servers communicate with each other.
>>
>> Clients, however, typically don't use HdfsZooInstance. You might try
>> replacing that with ZooKeeperInstance, and then construct your own AuthInfo
>> object (i.e. new
>> AuthInfo("user",ByteBuffer.wrap("password".getBytes()),zkInstance.getInstanceID())
>> ) instead of trying to use the system credentials.
>>
>> Cheers,
>> Adam
>>
>>
>> On Sun, Jul 15, 2012 at 10:52 PM, Jim Klucar <kl...@gmail.com> wrote:
>>>
>>> I'm trying to get the master stats in the same way the monitor server
>>> does, but I keep having issues.
>>>
>>> The code I'm using is from the GetMasterStats.java program:
>>>
>>> MasterClientService.Iface client = null;
>>> MasterMonitorInfo stats = null;
>>> try {
>>> client =
>>> MasterClient.getConnectionWithRetry(HdfsZooInstance.getInstance());
>>> stats = client.getMasterStats(null,
>>> SecurityConstants.getSystemCredentials());
>>> } finally {
>>> if (client != null)
>>> MasterClient.close(client);
>>> }
>>>
>>> The problem is that the HdfsZooInstance can never find the instance
>>> data, even though it's there. I can create a ZooKeeperInstance just
>>> fine, but passing that into the MasterClient fails the same way.
>>>
>>> Caused by: java.lang.RuntimeException: Accumulo not initialized, there
>>> is no instance id at /accumulo/instance_id
>>> at
>>> org.apache.accumulo.core.client.ZooKeeperInstance.getInstanceIDFromHdfs(ZooKeeperInstance.java:263)
>>> at
>>> org.apache.accumulo.server.conf.ZooConfiguration.getInstance(ZooConfiguration.java:65)
>>> at
>>> org.apache.accumulo.server.conf.ServerConfiguration.getZooConfiguration(ServerConfiguration.java:49)
>>> at
>>> org.apache.accumulo.server.conf.ServerConfiguration.getSystemConfiguration(ServerConfiguration.java:58)
>>> at
>>> org.apache.accumulo.server.client.HdfsZooInstance.<init>(HdfsZooInstance.java:62)
>>> at
>>> org.apache.accumulo.server.client.HdfsZooInstance.getInstance(HdfsZooInstance.java:70)
>>> at
>>> org.apache.accumulo.server.security.SecurityConstants.makeSystemPassword(SecurityConstants.java:58)
>>> at
>>> org.apache.accumulo.server.security.SecurityConstants.<clinit>(SecurityConstants.java:43)
>>> ... 91 more
>>>
>>> Any ideas as to what I'm missing?
>>
>>
Re: getMasterStats problems
Posted by Eric Newton <er...@gmail.com>.
You also need to have the hadoop configuration in your classpath so
the client can find HDFS, and not the local file system.
-Eric
On Sun, Jul 15, 2012 at 11:09 PM, Adam Fuchs <af...@apache.org> wrote:
> Jim,
>
> The HdfsZooInstance looks for accumulo-site.xml on the classpath to find the
> directory in HDFS to look for the instance ID. If accumulo-site.xml is not
> on the classpath then it will default to /accumulo, which is probably
> different from the directory you are using. accumulo-site.xml also includes
> a shared secret which is used to construct the system credentials. This is
> the standard way that servers communicate with each other.
>
> Clients, however, typically don't use HdfsZooInstance. You might try
> replacing that with ZooKeeperInstance, and then construct your own AuthInfo
> object (i.e. new
> AuthInfo("user",ByteBuffer.wrap("password".getBytes()),zkInstance.getInstanceID())
> ) instead of trying to use the system credentials.
>
> Cheers,
> Adam
>
>
> On Sun, Jul 15, 2012 at 10:52 PM, Jim Klucar <kl...@gmail.com> wrote:
>>
>> I'm trying to get the master stats in the same way the monitor server
>> does, but I keep having issues.
>>
>> The code I'm using is from the GetMasterStats.java program:
>>
>> MasterClientService.Iface client = null;
>> MasterMonitorInfo stats = null;
>> try {
>> client =
>> MasterClient.getConnectionWithRetry(HdfsZooInstance.getInstance());
>> stats = client.getMasterStats(null,
>> SecurityConstants.getSystemCredentials());
>> } finally {
>> if (client != null)
>> MasterClient.close(client);
>> }
>>
>> The problem is that the HdfsZooInstance can never find the instance
>> data, even though it's there. I can create a ZooKeeperInstance just
>> fine, but passing that into the MasterClient fails the same way.
>>
>> Caused by: java.lang.RuntimeException: Accumulo not initialized, there
>> is no instance id at /accumulo/instance_id
>> at
>> org.apache.accumulo.core.client.ZooKeeperInstance.getInstanceIDFromHdfs(ZooKeeperInstance.java:263)
>> at
>> org.apache.accumulo.server.conf.ZooConfiguration.getInstance(ZooConfiguration.java:65)
>> at
>> org.apache.accumulo.server.conf.ServerConfiguration.getZooConfiguration(ServerConfiguration.java:49)
>> at
>> org.apache.accumulo.server.conf.ServerConfiguration.getSystemConfiguration(ServerConfiguration.java:58)
>> at
>> org.apache.accumulo.server.client.HdfsZooInstance.<init>(HdfsZooInstance.java:62)
>> at
>> org.apache.accumulo.server.client.HdfsZooInstance.getInstance(HdfsZooInstance.java:70)
>> at
>> org.apache.accumulo.server.security.SecurityConstants.makeSystemPassword(SecurityConstants.java:58)
>> at
>> org.apache.accumulo.server.security.SecurityConstants.<clinit>(SecurityConstants.java:43)
>> ... 91 more
>>
>> Any ideas as to what I'm missing?
>
>
Re: getMasterStats problems
Posted by Adam Fuchs <af...@apache.org>.
Jim,
The HdfsZooInstance looks for accumulo-site.xml on the classpath to find
the directory in HDFS to look for the instance ID. If accumulo-site.xml is
not on the classpath then it will default to /accumulo, which is probably
different from the directory you are using. accumulo-site.xml also includes
a shared secret which is used to construct the system credentials. This is
the standard way that servers communicate with each other.
Clients, however, typically don't use HdfsZooInstance. You might try
replacing that with ZooKeeperInstance, and then construct your own AuthInfo
object (i.e. new
AuthInfo("user",ByteBuffer.wrap("password".getBytes()),zkInstance.getInstanceID())
) instead of trying to use the system credentials.
Cheers,
Adam
On Sun, Jul 15, 2012 at 10:52 PM, Jim Klucar <kl...@gmail.com> wrote:
> I'm trying to get the master stats in the same way the monitor server
> does, but I keep having issues.
>
> The code I'm using is from the GetMasterStats.java program:
>
> MasterClientService.Iface client = null;
> MasterMonitorInfo stats = null;
> try {
> client =
> MasterClient.getConnectionWithRetry(HdfsZooInstance.getInstance());
> stats = client.getMasterStats(null,
> SecurityConstants.getSystemCredentials());
> } finally {
> if (client != null)
> MasterClient.close(client);
> }
>
> The problem is that the HdfsZooInstance can never find the instance
> data, even though it's there. I can create a ZooKeeperInstance just
> fine, but passing that into the MasterClient fails the same way.
>
> Caused by: java.lang.RuntimeException: Accumulo not initialized, there
> is no instance id at /accumulo/instance_id
> at
> org.apache.accumulo.core.client.ZooKeeperInstance.getInstanceIDFromHdfs(ZooKeeperInstance.java:263)
> at
> org.apache.accumulo.server.conf.ZooConfiguration.getInstance(ZooConfiguration.java:65)
> at
> org.apache.accumulo.server.conf.ServerConfiguration.getZooConfiguration(ServerConfiguration.java:49)
> at
> org.apache.accumulo.server.conf.ServerConfiguration.getSystemConfiguration(ServerConfiguration.java:58)
> at
> org.apache.accumulo.server.client.HdfsZooInstance.<init>(HdfsZooInstance.java:62)
> at
> org.apache.accumulo.server.client.HdfsZooInstance.getInstance(HdfsZooInstance.java:70)
> at
> org.apache.accumulo.server.security.SecurityConstants.makeSystemPassword(SecurityConstants.java:58)
> at
> org.apache.accumulo.server.security.SecurityConstants.<clinit>(SecurityConstants.java:43)
> ... 91 more
>
> Any ideas as to what I'm missing?
>