best practice for deploying server side iterator jars

[Mike Hugo <mi...@piragua.com>]

Hello,

I'm wondering if there's an easy way to make sure a jar we have with custom
server side iterators gets properly placed into the tserver classpath on
each tserver (in ACCUMULO_HOME/lib/ext).  We are about to integrate that
into our devops scripts so that the jar is put in the right place as part
of server install and configuration, but i was wondering - is there a way
to put the jar into HDFS or something so that it's more "automagically"
picked up by all the tservers?

Thanks,

Mike

RE: best practice for deploying server side iterator jars

[dl...@comcast.net]

You will also want to set the replication on the directory/jars in HDFS to
something high. I think in my testing I could not set it above 50.

-----Original Message-----
From: Josh Elser [mailto:josh.elser@gmail.com] 
Sent: Thursday, February 20, 2014 12:30 PM
To: user@accumulo.apache.org
Subject: Re: best practice for deploying server side iterator jars

A word of warning... https://issues.apache.org/jira/browse/VFS-487

Be careful about redeploying jars in "rapid" succession as it's possible
that you'll get hit with the above issue. The classloader is currently
configured to monitor the filesystem every second -- it might be prudent to
make sure you wait at least 5s to make sure each file monitor sees the
update.

On 2/20/14, 5:53 AM, Brian Loss wrote:
> Starting with Accumulo 1.5 there is a VFS class loader that is capable of
loading jars from HDFS.  If you have a look at the accumulo-site.xml in
conf/examples/vfs-classloader, there is example configuration in the "VFS
ClassLoader Settings" section.
>
> If you're on 1.4 or earlier, I don't believe there is any alternative to
just making sure the jars are distributed to all tservers.
>
> On Feb 20, 2014, at 7:54 AM, Mike Hugo <mi...@piragua.com> wrote:
>
>> Hello,
>>
>> I'm wondering if there's an easy way to make sure a jar we have with
custom server side iterators gets properly placed into the tserver classpath
on each tserver (in ACCUMULO_HOME/lib/ext).  We are about to integrate that
into our devops scripts so that the jar is put in the right place as part of
server install and configuration, but i was wondering - is there a way to
put the jar into HDFS or something so that it's more "automagically" picked
up by all the tservers?
>>
>> Thanks,
>>
>> Mike
>

Re: best practice for deploying server side iterator jars

[Josh Elser <jo...@gmail.com>]

A word of warning... https://issues.apache.org/jira/browse/VFS-487

Be careful about redeploying jars in "rapid" succession as it's possible 
that you'll get hit with the above issue. The classloader is currently 
configured to monitor the filesystem every second -- it might be prudent 
to make sure you wait at least 5s to make sure each file monitor sees 
the update.

On 2/20/14, 5:53 AM, Brian Loss wrote:
> Starting with Accumulo 1.5 there is a VFS class loader that is capable of loading jars from HDFS.  If you have a look at the accumulo-site.xml in conf/examples/vfs-classloader, there is example configuration in the "VFS ClassLoader Settings” section.
>
> If you’re on 1.4 or earlier, I don’t believe there is any alternative to just making sure the jars are distributed to all tservers.
>
> On Feb 20, 2014, at 7:54 AM, Mike Hugo <mi...@piragua.com> wrote:
>
>> Hello,
>>
>> I'm wondering if there's an easy way to make sure a jar we have with custom server side iterators gets properly placed into the tserver classpath on each tserver (in ACCUMULO_HOME/lib/ext).  We are about to integrate that into our devops scripts so that the jar is put in the right place as part of server install and configuration, but i was wondering - is there a way to put the jar into HDFS or something so that it's more "automagically" picked up by all the tservers?
>>
>> Thanks,
>>
>> Mike
>

Re: best practice for deploying server side iterator jars

[Mike Hugo <mi...@piragua.com>]

Great, thanks Brian!
Mike

> On Feb 20, 2014, at 7:53 AM, Brian Loss <bf...@praxiseng.com> wrote:
>
> Starting with Accumulo 1.5 there is a VFS class loader that is capable of loading jars from HDFS.  If you have a look at the accumulo-site.xml in conf/examples/vfs-classloader, there is example configuration in the "VFS ClassLoader Settings" section.
>
> If you're on 1.4 or earlier, I don't believe there is any alternative to just making sure the jars are distributed to all tservers.
>
>> On Feb 20, 2014, at 7:54 AM, Mike Hugo <mi...@piragua.com> wrote:
>>
>> Hello,
>>
>> I'm wondering if there's an easy way to make sure a jar we have with custom server side iterators gets properly placed into the tserver classpath on each tserver (in ACCUMULO_HOME/lib/ext).  We are about to integrate that into our devops scripts so that the jar is put in the right place as part of server install and configuration, but i was wondering - is there a way to put the jar into HDFS or something so that it's more "automagically" picked up by all the tservers?
>>
>> Thanks,
>>
>> Mike
>

Re: best practice for deploying server side iterator jars

[Brian Loss <bf...@praxiseng.com>]

Starting with Accumulo 1.5 there is a VFS class loader that is capable of loading jars from HDFS.  If you have a look at the accumulo-site.xml in conf/examples/vfs-classloader, there is example configuration in the "VFS ClassLoader Settings” section.

If you’re on 1.4 or earlier, I don’t believe there is any alternative to just making sure the jars are distributed to all tservers.

On Feb 20, 2014, at 7:54 AM, Mike Hugo <mi...@piragua.com> wrote:

> Hello,
> 
> I'm wondering if there's an easy way to make sure a jar we have with custom server side iterators gets properly placed into the tserver classpath on each tserver (in ACCUMULO_HOME/lib/ext).  We are about to integrate that into our devops scripts so that the jar is put in the right place as part of server install and configuration, but i was wondering - is there a way to put the jar into HDFS or something so that it's more "automagically" picked up by all the tservers?
> 
> Thanks,
> 
> Mike