Run multiple accumulo instances on one Hadoop cluster

[Kristopher Kane <kk...@gmail.com>]

Forgive me if the question has been asked/answered, but, I built 1.5
from source last night and wanted to try it out on my existing Hadoop
cluster without overwriting my current 1.4 set.  Is there a way to
specify the /accumulo directory in HDFS such that you can run multiple
instances?

I couldn't find an obvious way to search the mail archives from apache.org.

Thanks for the help,

-Kris Kane

Re: Run multiple accumulo instances on one Hadoop cluster

[Eric Newton <er...@gmail.com>]

Most of the configurable properties are not in the example files.  We limit
the properties in the examples to those that most need to be changed, so
that they are not hidden among those that can be left unchanged.  Most
people do not change the location in HDFS.

I mentioned the monitor documentation link, since it's guaranteed to
document the settings for *your* version of the software.  And it's how I
find the settings myself.

We can look into generating the configurable values in the user manual; but
since it can change with each version, I would prefer that we just note
that *all* configurable properties can be found in the docs directory, or
by browsing the online monitor.

-Eric

On Tue, Oct 2, 2012 at 9:01 PM, Kristopher Kane <kk...@gmail.com>wrote:

> > Configuration, see the first property:
> >
> > instance.dfs.dir
> >
>
> Thanks Eric.
>
> So, I have never read the documentation from the monitor as I've
> always used the manual from the site, which, doesn't include this
> property.  Shouldn't they be the same?  Also, the example
> configuration doesn't populate this property by default.  I can or
> cannot say that it should, but it seems to be a pretty important value
> and would help user understanding/adoption if it is included in the
> default configurations with its default value.  Is that feature
> request worthy?
>
> Thanks again,
>
> -Kris
>

Re: Run multiple accumulo instances on one Hadoop cluster

[Kristopher Kane <kk...@gmail.com>]

> Configuration, see the first property:
>
> instance.dfs.dir
>

Thanks Eric.

So, I have never read the documentation from the monitor as I've
always used the manual from the site, which, doesn't include this
property.  Shouldn't they be the same?  Also, the example
configuration doesn't populate this property by default.  I can or
cannot say that it should, but it seems to be a pretty important value
and would help user understanding/adoption if it is included in the
default configurations with its default value.  Is that feature
request worthy?

Thanks again,

-Kris

Re: Run multiple accumulo instances on one Hadoop cluster

[Eric Newton <er...@gmail.com>]

>From the monitoring user interface, see the Documentation link, then
Configuration, see the first property:

instance.dfs.dir

You'll also change all the port numbers from the defaults.  And there's a
port number in conf/generic_logger.xml that points to the logging port on
the monitor.

For example, here are some entries from my conf/accumulo-site.xml file:

    <property>
      <name>master.port.client</name>
      <value>10010</value>
    </property>

    <property>
      <name>tserver.port.client</name>
      <value>10011</value>
    </property>

    <property>
      <name>gc.port.client</name>
      <value>10101</value>
    </property>

    <property>
      <name>trace.port.client</name>
      <value>10111</value>
    </property>

    <property>
      <name>monitor.port.client</name>
      <value>11111</value>
    </property>

    <property>
      <name>monitor.port.log4j</name>
      <value>1560</value>
    </property>

And conf generic_logger.xml:

  <!-- Send all logging data to a centralized logger -->
  <appender name="N1" class="org.apache.log4j.net.SocketAppender">
     <param name="remoteHost"
value="${org.apache.accumulo.core.host.log}"/>
     <param name="port"           value="1560"/>
     <param name="application"
 value="${org.apache.accumulo.core.application}:${org.apache.accumulo.core.ip.localhost.hostname}"/>
     <param name="Threshold"      value="WARN"/>
  </appender>

-Eric

On Tue, Oct 2, 2012 at 10:40 AM, Kristopher Kane <kk...@gmail.com>
wrote:
>
> Forgive me if the question has been asked/answered, but, I built 1.5
> from source last night and wanted to try it out on my existing Hadoop
> cluster without overwriting my current 1.4 set.  Is there a way to
> specify the /accumulo directory in HDFS such that you can run multiple
> instances?
>
> I couldn't find an obvious way to search the mail archives from apache.org
.
>
> Thanks for the help,
>
> -Kris Kane