You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by "Brown, Scott M (IS)" <Sc...@ngc.com> on 2016/07/27 15:22:59 UTC
RE: EXT :Re: cannot get accumulo 1.6.2 and later to resolve
classpaths
Hi Sean, thanks for the fast response!
I'm downloading the files from https://archive.apache.org/dist/accumulo/ for example, this package name accumulo-1.6.2-bin.tar.gz
I'm hoping the pastebin location is http://pastebin.com/c2xSP5dL
Commons-io is under the following:
[root@cmtest ~]# find / -name commons-io-2.4.jar
/usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar
/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.4.jar
/usr/lib/hadoop/client/commons-io-2.4.jar
/usr/lib/hadoop/lib/commons-io-2.4.jar
/usr/lib/hadoop/commons-io-2.4.jar
/usr/lib/hadoop/client-0.20/commons-io-2.4.jar
/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar
/usr/lib/hadoop-mapreduce/commons-io-2.4.jar
/usr/lib/hadoop-yarn/lib/commons-io-2.4.jar
/usr/lib/parquet/lib/commons-io-2.4.jar
Commons-logging is under the same paths.
Im not sure what to set to resolve this, as when we install 1.6.0 we do not set CLASSPATH, or LD_LIBRARY_LD.. we also have contact with a site that is running 1.6.4, without those set, but they may get their code from apache directly (special).
-----Original Message-----
From: Sean Busbey [mailto:busbey@cloudera.com]
Sent: Wednesday, July 27, 2016 10:00 AM
To: Accumulo User List
Subject: EXT :Re: cannot get accumulo 1.6.2 and later to resolve classpaths
What packaging of Accumulo are you relying on? Is it the binary tarballs from the asf release site? Can you pastebin the accumulo-site.xml and accumulo-env.sh files from your conf/ directories (after redacting instance secret and any other sensitive information)?
It looks like you are missing commons-io and commons-logging. IIRC, in the example configs for the binary tarballs we expect those to come from the hadoop classpath. AFAIK, we did this in both 1.6.1 and 1.6.2+.
On Wed, Jul 27, 2016 at 9:20 AM, Brown, Scott M (IS) <Sc...@ngc.com> wrote:
>
>
>
> We are attempting to upgrade our application stack which includes accumulo.
> Currently we are running 1.6.0 but want to upgrade to at least 1.6.5.
> After 1.6.1 we are running into unresolved library, commons-io.
>
> The strange thing is, if I have $ACCUMULO_HOME set to point at version
> 1.6.1 or
> 1.6.0 I can resolve 'accumulo classpath' fine in versions 1.6.2 and
> later, and it will init. But if I have $ACCUMULO_HOME set to 1.6.2
> and later, I get the commons-io message
> below. Also I have ACCUMULO_CONF_DIR set to $ACCUMULO_HOME/conf.
>
> So again, 1..6.2 and beyond will not pass 'accumulo classpath' without
> $ACCUMULO HOME being set to 1.6.1 or 1.6.0.
>
> Here is the commons-io error we receive.
> Out setup is
> Centos 6.5 base install
> Hadoop version = 2.6.0-cdh5.7.1
> Zookeeper 3.4.5
> we do not have LD_LIBRARY_LD or CLASSPATH defined as we know a site
> that has 1.6.5 running without those defined.
>
> Appreciate any help as this is kinda strange and not finding much help
> on it on Google.
>
> [root@cmtest bin]# ./accumulo classpath Uncaught exception:
> java.lang.reflect.InvocationTargetException
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.accumulo.start.Main.main(Main.java:43)
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/commons/logging/LogFactory
> at
> org.apache.commons.vfs2.impl.DefaultFileSystemManager.<init>(DefaultFileSystemManager.java:120)
> at
> org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.generateVfs(AccumuloVFSClassLoader.java:245)
> at
> org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.getClassLoader(AccumuloVFSClassLoader.java:203)
> ... 5 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.logging.LogFactory
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at
> org.apache.accumulo.start.classloader.AccumuloClassLoader$2.loadClass(AccumuloClassLoader.java:270)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 8 more
> Exception in thread "Thread-0" java.lang.NoClassDefFoundError:
> org/apache/commons/io/FileUtils
> at
> org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.close(AccumuloVFSClassLoader.java:405)
> at
> org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader$AccumuloVFSClassLoaderShutdownThread.run(AccumuloVFSClassLoader.java:73)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.FileUtils
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at
> org.apache.accumulo.start.classloader.AccumuloClassLoader$2.loadClass(AccumuloClassLoader.java:270)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 3 more
>
>
--
busbey
Re: EXT :Re: cannot get accumulo 1.6.2 and later to resolve classpaths
Posted by Sean Busbey <bu...@cloudera.com>.
They're commented out because you should only include the paths that
correspond to a single hadoop installation. The earlier example was
likely to lead folks astray by including two.
Glad to hear you got past your immediate problem, but I'd strongly
recommend you look at using the CDH5-specific general classpaths value
if you're going to run on top of CDH5.
On Thu, Jul 28, 2016 at 10:39 AM, Brown, Scott M (IS)
<Sc...@ngc.com> wrote:
> Uncommenting the needed paths got me past that.. I have no idea why they would be commented, but that solved it, thanks!
>
> -----Original Message-----
> From: Sean Busbey [mailto:busbey@cloudera.com]
> Sent: Wednesday, July 27, 2016 3:17 PM
> To: Accumulo User List
> Subject: Re: EXT :Re: cannot get accumulo 1.6.2 and later to resolve classpaths
>
> On Wed, Jul 27, 2016 at 1:00 PM, Brown, Scott M (IS) <Sc...@ngc.com> wrote:
>> Yes I only sent the one set from 1.6.2 that doesn’t work.. I can paste the 2 files from 1.6.1 that will resolve if you want them, let me know, glad to do it.
>>
>
> Yes, please include the configs you are using in both cases.
>
>> I'm downloading the yum install packages from Cloudera now, I'll
>> install and look at the conf files and see how general_classpath is
>> set. The setting is looking in the correct paths for commons-io, as
>> both /usr/lib/Hadoop and /usr/lib/Hadoop/lib are defined in there, so
>> its like you referred to, the setting of classpath has changed from
>> 1.6.1 to 1.6.2+
>>
>
>
> The configs you posted from the 1.6.2 home do not include the /usr/lib/hadoop and /usr/lib/hadoop/lib directories. Those paths are in a commented out example for HDP2.0. The only un-commented out hadoop set is the Hadoop 2 based on the packaging that Apache Hadoop itself does for their convenience binary tarballs. Those paths don't work on CDH.
>
>
>> Could you clarify this " but I wouldn't expect a deployment with these configs to work."? If there is a bug in the way classpath works after 1.6.1, would it not have a workaround or fix?
>>
>
> As mentioned above, the config that was posted on pastebin doesn't contain a path to the jars Accumulo needs in a CDH installation. So even if a particular command like "classpath" works, I wouldn't expect a cluster deployment based on it to be production worthy.
>
>
>> Typically this has always been minimal setup, and init works, but
>> there seems to be a difference in 1.6.2+
>
>
> The example configs that ship with Accumulo are just examples. As mentioned in them, you have to customize them to your environment (such as the zookeeper quorum). The classpath config will vary greatly depending on what you're using to satisfy your runtime dependencies, since Hadoop locations on disk a different for different packaging options (and some folks would rather avoid getting them from Hadoop entirely).
>
> I believe in 1.6.1 the example configs contained both the hadoop 2 packaging from the Hadoop PMC and the HDP 2.0 layout. That's probably why it happened to work.
>
> --
> busbey
>
--
busbey
RE: EXT :Re: cannot get accumulo 1.6.2 and later to resolve
classpaths
Posted by "Brown, Scott M (IS)" <Sc...@ngc.com>.
Uncommenting the needed paths got me past that.. I have no idea why they would be commented, but that solved it, thanks!
-----Original Message-----
From: Sean Busbey [mailto:busbey@cloudera.com]
Sent: Wednesday, July 27, 2016 3:17 PM
To: Accumulo User List
Subject: Re: EXT :Re: cannot get accumulo 1.6.2 and later to resolve classpaths
On Wed, Jul 27, 2016 at 1:00 PM, Brown, Scott M (IS) <Sc...@ngc.com> wrote:
> Yes I only sent the one set from 1.6.2 that doesn’t work.. I can paste the 2 files from 1.6.1 that will resolve if you want them, let me know, glad to do it.
>
Yes, please include the configs you are using in both cases.
> I'm downloading the yum install packages from Cloudera now, I'll
> install and look at the conf files and see how general_classpath is
> set. The setting is looking in the correct paths for commons-io, as
> both /usr/lib/Hadoop and /usr/lib/Hadoop/lib are defined in there, so
> its like you referred to, the setting of classpath has changed from
> 1.6.1 to 1.6.2+
>
The configs you posted from the 1.6.2 home do not include the /usr/lib/hadoop and /usr/lib/hadoop/lib directories. Those paths are in a commented out example for HDP2.0. The only un-commented out hadoop set is the Hadoop 2 based on the packaging that Apache Hadoop itself does for their convenience binary tarballs. Those paths don't work on CDH.
> Could you clarify this " but I wouldn't expect a deployment with these configs to work."? If there is a bug in the way classpath works after 1.6.1, would it not have a workaround or fix?
>
As mentioned above, the config that was posted on pastebin doesn't contain a path to the jars Accumulo needs in a CDH installation. So even if a particular command like "classpath" works, I wouldn't expect a cluster deployment based on it to be production worthy.
> Typically this has always been minimal setup, and init works, but
> there seems to be a difference in 1.6.2+
The example configs that ship with Accumulo are just examples. As mentioned in them, you have to customize them to your environment (such as the zookeeper quorum). The classpath config will vary greatly depending on what you're using to satisfy your runtime dependencies, since Hadoop locations on disk a different for different packaging options (and some folks would rather avoid getting them from Hadoop entirely).
I believe in 1.6.1 the example configs contained both the hadoop 2 packaging from the Hadoop PMC and the HDP 2.0 layout. That's probably why it happened to work.
--
busbey
Re: EXT :Re: cannot get accumulo 1.6.2 and later to resolve classpaths
Posted by Sean Busbey <bu...@cloudera.com>.
On Wed, Jul 27, 2016 at 1:00 PM, Brown, Scott M (IS)
<Sc...@ngc.com> wrote:
> Yes I only sent the one set from 1.6.2 that doesn’t work.. I can paste the 2 files from 1.6.1 that will resolve if you want them, let me know, glad to do it.
>
Yes, please include the configs you are using in both cases.
> I'm downloading the yum install packages from Cloudera now, I'll install and look at the conf files and see how general_classpath is set. The setting is looking in the correct paths for commons-io, as both /usr/lib/Hadoop and /usr/lib/Hadoop/lib are defined in there, so its like you referred to, the setting of classpath has changed from 1.6.1 to 1.6.2+
>
The configs you posted from the 1.6.2 home do not include the
/usr/lib/hadoop and /usr/lib/hadoop/lib directories. Those paths are
in a commented out example for HDP2.0. The only un-commented out
hadoop set is the Hadoop 2 based on the packaging that Apache Hadoop
itself does for their convenience binary tarballs. Those paths don't
work on CDH.
> Could you clarify this " but I wouldn't expect a deployment with these configs to work."? If there is a bug in the way classpath works after 1.6.1, would it not have a workaround or fix?
>
As mentioned above, the config that was posted on pastebin doesn't
contain a path to the jars Accumulo needs in a CDH installation. So
even if a particular command like "classpath" works, I wouldn't expect
a cluster deployment based on it to be production worthy.
> Typically this has always been minimal setup, and init works, but there seems to be a difference in 1.6.2+
The example configs that ship with Accumulo are just examples. As
mentioned in them, you have to customize them to your environment
(such as the zookeeper quorum). The classpath config will vary greatly
depending on what you're using to satisfy your runtime dependencies,
since Hadoop locations on disk a different for different packaging
options (and some folks would rather avoid getting them from Hadoop
entirely).
I believe in 1.6.1 the example configs contained both the hadoop 2
packaging from the Hadoop PMC and the HDP 2.0 layout. That's probably
why it happened to work.
--
busbey
RE: EXT :Re: cannot get accumulo 1.6.2 and later to resolve
classpaths
Posted by "Brown, Scott M (IS)" <Sc...@ngc.com>.
Yes I only sent the one set from 1.6.2 that doesn’t work.. I can paste the 2 files from 1.6.1 that will resolve if you want them, let me know, glad to do it.
I'm downloading the yum install packages from Cloudera now, I'll install and look at the conf files and see how general_classpath is set. The setting is looking in the correct paths for commons-io, as both /usr/lib/Hadoop and /usr/lib/Hadoop/lib are defined in there, so its like you referred to, the setting of classpath has changed from 1.6.1 to 1.6.2+
Could you clarify this " but I wouldn't expect a deployment with these configs to work."? If there is a bug in the way classpath works after 1.6.1, would it not have a workaround or fix?
Typically this has always been minimal setup, and init works, but there seems to be a difference in 1.6.2+
-----Original Message-----
From: Sean Busbey [mailto:busbey@cloudera.com]
Sent: Wednesday, July 27, 2016 11:53 AM
To: Accumulo User List
Subject: Re: EXT :Re: cannot get accumulo 1.6.2 and later to resolve classpaths
On Wed, Jul 27, 2016 at 10:22 AM, Brown, Scott M (IS) <Sc...@ngc.com> wrote:
> Hi Sean, thanks for the fast response!
>
> I'm downloading the files from
> https://archive.apache.org/dist/accumulo/ for example, this package
> name accumulo-1.6.2-bin.tar.gz
>
> I'm hoping the pastebin location is http://pastebin.com/c2xSP5dL
>
This is only a single set of conf files. Does ACCUMULO_CONF point to these files under the 1.6.2 ACCUMULO_HOME in all cases? Or does it point to the conf/ dir within the 1.6.1 installation when you update ACCUMULO_HOME to point there?
> Commons-io is under the following:
> [root@cmtest ~]# find / -name commons-io-2.4.jar
> /usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar
> /usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.4.jar
> /usr/lib/hadoop/client/commons-io-2.4.jar
> /usr/lib/hadoop/lib/commons-io-2.4.jar
> /usr/lib/hadoop/commons-io-2.4.jar
> /usr/lib/hadoop/client-0.20/commons-io-2.4.jar
> /usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar
> /usr/lib/hadoop-mapreduce/commons-io-2.4.jar
> /usr/lib/hadoop-yarn/lib/commons-io-2.4.jar
> /usr/lib/parquet/lib/commons-io-2.4.jar
>
> Commons-logging is under the same paths.
>
> Im not sure what to set to resolve this, as when we install 1.6.0 we do not set CLASSPATH, or LD_LIBRARY_LD.. we also have contact with a site that is running 1.6.4, without those set, but they may get their code from apache directly (special).
>
The settings for the property "general.classpaths" in the pasted accumulo-site.xml are not correct for running on top of a packages installation of CDH5 (which is what I'm guessing you are doing based on the output of the find command and your stated Hadoop version).
You should start by looking at the example configuration files that Cloudera publishes for "CDH5_packages". You'll need to download their packaging of Accumulo to extract them. If you have difficulty using those configs, you should ask about them on the vendor's support forum (community.cloudera.com presuming you are looking for the "community"
edition). Keep in mind that you'll be asking about using them with a packaging of Accumulo that vendor doesn't support, so help might be limited.
I don't see what here would cause the move from 1.6.0 / 1.6.1 to 1.6.2 to make a difference in working. I haven't gotten to dig in to see if maybe we do different things when loading the classpath command, but I wouldn't expect a deployment with these configs to work.
--
busbey
Re: EXT :Re: cannot get accumulo 1.6.2 and later to resolve classpaths
Posted by Sean Busbey <bu...@cloudera.com>.
On Wed, Jul 27, 2016 at 10:22 AM, Brown, Scott M (IS)
<Sc...@ngc.com> wrote:
> Hi Sean, thanks for the fast response!
>
> I'm downloading the files from https://archive.apache.org/dist/accumulo/ for example, this package name accumulo-1.6.2-bin.tar.gz
>
> I'm hoping the pastebin location is http://pastebin.com/c2xSP5dL
>
This is only a single set of conf files. Does ACCUMULO_CONF point to
these files under the 1.6.2 ACCUMULO_HOME in all cases? Or does it
point to the conf/ dir within the 1.6.1 installation when you update
ACCUMULO_HOME to point there?
> Commons-io is under the following:
> [root@cmtest ~]# find / -name commons-io-2.4.jar
> /usr/lib/hadoop-hdfs/lib/commons-io-2.4.jar
> /usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.4.jar
> /usr/lib/hadoop/client/commons-io-2.4.jar
> /usr/lib/hadoop/lib/commons-io-2.4.jar
> /usr/lib/hadoop/commons-io-2.4.jar
> /usr/lib/hadoop/client-0.20/commons-io-2.4.jar
> /usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar
> /usr/lib/hadoop-mapreduce/commons-io-2.4.jar
> /usr/lib/hadoop-yarn/lib/commons-io-2.4.jar
> /usr/lib/parquet/lib/commons-io-2.4.jar
>
> Commons-logging is under the same paths.
>
> Im not sure what to set to resolve this, as when we install 1.6.0 we do not set CLASSPATH, or LD_LIBRARY_LD.. we also have contact with a site that is running 1.6.4, without those set, but they may get their code from apache directly (special).
>
The settings for the property "general.classpaths" in the pasted
accumulo-site.xml are not correct for running on top of a packages
installation of CDH5 (which is what I'm guessing you are doing based
on the output of the find command and your stated Hadoop version).
You should start by looking at the example configuration files that
Cloudera publishes for "CDH5_packages". You'll need to download their
packaging of Accumulo to extract them. If you have difficulty using
those configs, you should ask about them on the vendor's support forum
(community.cloudera.com presuming you are looking for the "community"
edition). Keep in mind that you'll be asking about using them with a
packaging of Accumulo that vendor doesn't support, so help might be
limited.
I don't see what here would cause the move from 1.6.0 / 1.6.1 to 1.6.2
to make a difference in working. I haven't gotten to dig in to see if
maybe we do different things when loading the classpath command, but I
wouldn't expect a deployment with these configs to work.
--
busbey