Using HDFS Java client in Karaf

[Stefan Meisner Larsen <st...@meisner-larsen.dk>]

I am trying to get Hadoop HDFS2 to work in Karaf 3. Below is what I have
tried...which doesn't work for me.
Can anyone point me to a description of a working solution? Or explain why
don't succeed doing the following?

I have the following camel bundles installed:
karaf@root>list|grep camel
102 | Active  |  50 | 2.14.0         |
camel-core
103 | Active  |  50 | 2.14.0         |
camel-karaf-commands
113 | Active  |  50 | 2.14.0         |
camel-aws
137 | Active  |  50 | 2.14.0         | camel-hdfs2

Since we are using Camel too, I tried following the recipe found here:
http://camel.apache.org/hdfs2.html


I used the last blueprint option, inserting the following

<bean id="hdfsOsgiHelper"
class="org.apache.camel.component.hdfs2.HdfsOsgiHelper">
   <argument>
      <map>
                <entry key="${hadoop.url}"
value="org.apache.hadoop.hdfs.DistributedFileSystem" />
      </map>
   </argument>
</bean>

<bean id="hdfs2" class="org.apache.camel.component.hdfs2.HdfsComponent"
depends-on="hdfsOsgiHelper" />

I am not using the Camel HDFS2 component, and could probably remove the
last bean.

My own bean definition uses 'depends-on' to ensure that it is created after
hdfsOsgiHelper

In my bean I do:
        configuration = new Configuration();
        fileSystem = FileSystem.get(URI.create(url),configuration,
username);

which fails with java.io.IOException: No FileSystem for scheme: hdfs

I then tried:
        configuration = new Configuration();
        configuration.setClass("fs.file.impl",
org.apache.hadoop.fs.LocalFileSystem.class, FileSystem.class);
        configuration.setClass("fs.hdfs.impl",
org.apache.hadoop.hdfs.DistributedFileSystem.class, FileSystem.class);
        fileSystem = FileSystem.get(URI.create(url),configuration,
username);

Which gives me:
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.hdfs.DistributedFileSystem not found
    at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1788)
    at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1880)

Executing classes|grep DistributedFileSystem  in karaf console proves that
this class is actually present...

In my bundle I have core-site.xml and hdfs-site.xml which proves to work
when testing outside Karaf

Cheers!
Stefan