You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Amit Sela <am...@infolinks.com> on 2013/04/21 19:00:53 UTC
Setting ClassLoader in HBaseConfiguration
Hi all,
I'm trying to run an HBase client from an OSGI environment and for that I
need to set the Configuration classLoader.
In Configuration (Hadoop) itself, there is a method for that but since
HBaseConfiguration.create() is static the only solution I found was:
Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
before calling HBaseConfiguration.create() .
Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
Thanks,
Amit.
Re: Setting ClassLoader in HBaseConfiguration
Posted by Ted Yu <yu...@gmail.com>.
The static block in HConnectionManager requires some consideration if we
are to support custom classloader in HBaseConfiguration.
Suppose you have a Configuration object with custom classloader, we need to
delay instantiating HBASE_INSTANCES so that you can pass Configuration
object to HConnectionManager.
For the moment, it looks like your bypass is the best choice.
Thanks for sharing your experiences.
On Mon, Apr 22, 2013 at 9:04 AM, Amit Sela <am...@infolinks.com> wrote:
> Well, not really... utilizing the method doesn't really work for me - Since
> the constructors in HBaseConfiguration are deprecated and the way to use it
> is by calling the static create methods, I can't really use the
> setClassLoader method from Configuration. In addition to that, I need to
> set the ClassLoader before create executes the new Configuration and
> addResources because all configurations to set are in the bundle context.
>
> Even when implementing my own HBaseConfiguration (loading the class loader
> before new Configuration() and addHBaseResources()), I get an exception
> when trying to create a new HTableInterface because HConnectionManager has
> a static block that calls HBaseConfiguration.create() (line: 148) and the
> configuration returned doesn't have the right class loader...
>
> The only way I found so far (without changing HBase code) is by
> calling:
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> before executing my code.
>
> Hadoop uses Configuration when submitting jobs for example, and it has
> clone constructor that can pass the class loader (and more), but from what
> I saw the HBase framework uses create() and even if using
> create(Configuration) the merge method merges only <String, String>
> properties map.
>
> Any ideas ? Should I stay with my bypass ? I saw a TODO to replace the
> constructor with a private one, and not extend Configuration, so maybe also
> allow setting of ClassLoader and use clone constructors in the framework
> (for instance HConnectionManager will use a cloned configuration when
> calling new HTable()).
>
> Thanks.
>
>
>
> On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Since HBaseConfiguration extends Configuration, can you utilize this
> method
> > from Configuration ?
> >
> > public void setClassLoader(ClassLoader classLoader) {
> > Thanks
> >
> > On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <am...@infolinks.com> wrote:
> >
> > > Hi all,
> > >
> > > I'm trying to run an HBase client from an OSGI environment and for
> that I
> > > need to set the Configuration classLoader.
> > > In Configuration (Hadoop) itself, there is a method for that but since
> > > HBaseConfiguration.create() is static the only solution I found was:
> > >
> > >
> >
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> > > before calling HBaseConfiguration.create() .
> > >
> > > Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
> > >
> > > Thanks,
> > >
> > > Amit.
> > >
> >
>
>
>
> On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Since HBaseConfiguration extends Configuration, can you utilize this
> method
> > from Configuration ?
> >
> > public void setClassLoader(ClassLoader classLoader) {
> > Thanks
> >
> > On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <am...@infolinks.com> wrote:
> >
> > > Hi all,
> > >
> > > I'm trying to run an HBase client from an OSGI environment and for
> that I
> > > need to set the Configuration classLoader.
> > > In Configuration (Hadoop) itself, there is a method for that but since
> > > HBaseConfiguration.create() is static the only solution I found was:
> > >
> > >
> >
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> > > before calling HBaseConfiguration.create() .
> > >
> > > Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
> > >
> > > Thanks,
> > >
> > > Amit.
> > >
> >
>
>
>
> On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Since HBaseConfiguration extends Configuration, can you utilize this
> method
> > from Configuration ?
> >
> > public void setClassLoader(ClassLoader classLoader) {
> > Thanks
> >
> > On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <am...@infolinks.com> wrote:
> >
> > > Hi all,
> > >
> > > I'm trying to run an HBase client from an OSGI environment and for
> that I
> > > need to set the Configuration classLoader.
> > > In Configuration (Hadoop) itself, there is a method for that but since
> > > HBaseConfiguration.create() is static the only solution I found was:
> > >
> > >
> >
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> > > before calling HBaseConfiguration.create() .
> > >
> > > Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
> > >
> > > Thanks,
> > >
> > > Amit.
> > >
> >
>
Re: Setting ClassLoader in HBaseConfiguration
Posted by Amit Sela <am...@infolinks.com>.
Well, not really... utilizing the method doesn't really work for me - Since
the constructors in HBaseConfiguration are deprecated and the way to use it
is by calling the static create methods, I can't really use the
setClassLoader method from Configuration. In addition to that, I need to
set the ClassLoader before create executes the new Configuration and
addResources because all configurations to set are in the bundle context.
Even when implementing my own HBaseConfiguration (loading the class loader
before new Configuration() and addHBaseResources()), I get an exception
when trying to create a new HTableInterface because HConnectionManager has
a static block that calls HBaseConfiguration.create() (line: 148) and the
configuration returned doesn't have the right class loader...
The only way I found so far (without changing HBase code) is by
calling: Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
before executing my code.
Hadoop uses Configuration when submitting jobs for example, and it has
clone constructor that can pass the class loader (and more), but from what
I saw the HBase framework uses create() and even if using
create(Configuration) the merge method merges only <String, String>
properties map.
Any ideas ? Should I stay with my bypass ? I saw a TODO to replace the
constructor with a private one, and not extend Configuration, so maybe also
allow setting of ClassLoader and use clone constructors in the framework
(for instance HConnectionManager will use a cloned configuration when
calling new HTable()).
Thanks.
On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <yu...@gmail.com> wrote:
> Since HBaseConfiguration extends Configuration, can you utilize this method
> from Configuration ?
>
> public void setClassLoader(ClassLoader classLoader) {
> Thanks
>
> On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <am...@infolinks.com> wrote:
>
> > Hi all,
> >
> > I'm trying to run an HBase client from an OSGI environment and for that I
> > need to set the Configuration classLoader.
> > In Configuration (Hadoop) itself, there is a method for that but since
> > HBaseConfiguration.create() is static the only solution I found was:
> >
> >
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> > before calling HBaseConfiguration.create() .
> >
> > Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
> >
> > Thanks,
> >
> > Amit.
> >
>
On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <yu...@gmail.com> wrote:
> Since HBaseConfiguration extends Configuration, can you utilize this method
> from Configuration ?
>
> public void setClassLoader(ClassLoader classLoader) {
> Thanks
>
> On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <am...@infolinks.com> wrote:
>
> > Hi all,
> >
> > I'm trying to run an HBase client from an OSGI environment and for that I
> > need to set the Configuration classLoader.
> > In Configuration (Hadoop) itself, there is a method for that but since
> > HBaseConfiguration.create() is static the only solution I found was:
> >
> >
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> > before calling HBaseConfiguration.create() .
> >
> > Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
> >
> > Thanks,
> >
> > Amit.
> >
>
On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <yu...@gmail.com> wrote:
> Since HBaseConfiguration extends Configuration, can you utilize this method
> from Configuration ?
>
> public void setClassLoader(ClassLoader classLoader) {
> Thanks
>
> On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <am...@infolinks.com> wrote:
>
> > Hi all,
> >
> > I'm trying to run an HBase client from an OSGI environment and for that I
> > need to set the Configuration classLoader.
> > In Configuration (Hadoop) itself, there is a method for that but since
> > HBaseConfiguration.create() is static the only solution I found was:
> >
> >
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> > before calling HBaseConfiguration.create() .
> >
> > Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
> >
> > Thanks,
> >
> > Amit.
> >
>
Re: Setting ClassLoader in HBaseConfiguration
Posted by Ted Yu <yu...@gmail.com>.
Since HBaseConfiguration extends Configuration, can you utilize this method
from Configuration ?
public void setClassLoader(ClassLoader classLoader) {
Thanks
On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <am...@infolinks.com> wrote:
> Hi all,
>
> I'm trying to run an HBase client from an OSGI environment and for that I
> need to set the Configuration classLoader.
> In Configuration (Hadoop) itself, there is a method for that but since
> HBaseConfiguration.create() is static the only solution I found was:
>
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> before calling HBaseConfiguration.create() .
>
> Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
>
> Thanks,
>
> Amit.
>