Ruby scripting in HBase?

[Konstantin Boudnik <co...@apache.org>]

Guys,

I've been looking into some service scripting around HBase lifecycle
management, etc. and couldn't help but wonder why those were written in Ruby
of all JVM languages? Historical legacy aside, it seems that current HBase is
still using JRuby 1.6.5 vs the latest at 1.9+ or perhaps even later.

At any rate, I was wondering if replacing Ruby with a more Java-like scripting
extension (if the scripting-2-Java API bridge is what indeed desired) would be
of any interest here? An obvious choice would be Groovy
(http://groovy.codehaus.org/). One of the main reasons behind my proposal is
stack simplification: Bigtop is very actively using Groovy as a scripting
language of choice to do builds, develop smoke tests, etc. So, it is already
there and guaranteed to be installed as a part of any Bigtop-derived Hadoop
distro. There are other benefits, where, if desired, one can just write Java
code inside of a Groovy script, without a need to learn yet another language
like Ruby.

This is perhaps not of an immediate priority for the community, but if there's
enough interest, I can give it an initial shot to demo'ed what I am really
talking about.

Thoughts?
-- 
Regards,
  Cos

Re: Ruby scripting in HBase?

[Stack <st...@duboce.net>]

On Thu, Aug 28, 2014 at 4:06 PM, Konstantin Boudnik <co...@apache.org> wrote:

> Guys,
>
> I've been looking into some service scripting around HBase lifecycle
> management, etc. and couldn't help but wonder why those were written in
> Ruby
> of all JVM languages? Historical legacy aside, it seems that current HBase
> is
> still using JRuby 1.6.5 vs the latest at 1.9+ or perhaps even later.
>
>
HBASE-7028



> At any rate, I was wondering if replacing Ruby with a more Java-like
> scripting
> extension (if the scripting-2-Java API bridge is what indeed desired)
> would be
> of any interest here? An obvious choice would be Groovy
>


To be clear, our having jruby in place does not hinder being able to
groovy-script hbase?

St.Ack

Re: Ruby scripting in HBase?

[Stack <st...@duboce.net>]

On Thu, Aug 28, 2014 at 5:34 PM, Konstantin Boudnik <co...@apache.org> wrote:
...

> As for the 'pain vs pain' comment: I am not really sure why. Groovy is
> really
> just a Java with added benefits of real lambdas, dynamic bindings, etc.
>

I understand what JMS means.

We picked our poison long time ago.  Why would we abuse users swapping it
out for another especially when no big win.

Consult old JIRAs for why JRuby. Suffice to say it was the best alternative
at the time in spite of 'faults' (size, startup-time, etc.). But hey, times
change.

I like Sean's meta approach where he stands back and is trying to get away
with differentiating 'user' and 'dev' needs, and users get the, careful
kid-glove treatment. I think I see where he is going (accumulo-like custom
shell?).  If he succeeds in this distinction, I'd say come back then Cos w/
your groovy suggestion.  It might fly better if it is just for devs.

Also, if we are heading down this route, separating user and dev
shell/scripting needs, lets put phoenix in the running as the user tool

St.Ack

Re: Ruby scripting in HBase?

[Sean Busbey <bu...@cloudera.com>]

Hey Cos,

A couple of weeks ago I started a thread on what kind of compatibility
needs have to be met when updating the shell, which is the vast majority of
our jruby.

*http://s.apache.org/tFV <http://s.apache.org/tFV>*

Can we combine discussions into one place over there?

Ideally, I'd like to replace the jruby (at least for user facing
operational needs) with a pure-java implementation that can cut down on
size while getting faster. Unfortunately I got caught up in some work and
the thread went quite. I'm going to try to address Stack's concerns
tomorrow.

My hope is that we can have language appropriate client bindings for folks
who'd like to build their own tooling in a scripting language.


On Thu, Aug 28, 2014 at 7:34 PM, Konstantin Boudnik <co...@apache.org> wrote:

> On Thu, Aug 28, 2014 at 07:51PM, Jean-Marc Spaggiari wrote:
> > Was more thinking about the first. Having a way to automate groovy shell
> > commands testing in the build.
>
> I am pretty sure there is. After all, Gradle (a build DSL based, using
> Groovy
> as the underlying language) has unit testing in it ;)
>
> The issue with testing of these scripts, as far as I can see, is that they
> might need some sort of mocking involved to work around the fact that these
> command expect a running HBase cluster. Makes sense?
>
> As for the 'pain vs pain' comment: I am not really sure why. Groovy is
> really
> just a Java with added benefits of real lambdas, dynamic bindings, etc.
>
> Cos
>
> > Le 2014-08-28 19:45, "Mikhail Antonov" <ol...@gmail.com> a écrit :
> >
> > > JM - do you mean writing unit or integration tests for groovy commands
> > > themselves, or to be able to write HBase tests in Groovy? If later
> > > one, then I'd think HBase tests may benefit a lot in conciseness if
> > > written in Groovy.
> > >
> > > -Mikhail
> > >
> > > 2014-08-28 16:39 GMT-07:00, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org>:
> > > > Are we not just going to replace a pain by another pain?
> > > >
> > > > Can we build test suites for Groovy? I mean, not just use groovy to
> build
> > > > test, but build a test script which will test groovy? I think it's
> one of
> > > > the main issues today with JRuby shell.
> > > >
> > > > I prefer Groovy over JRuby but not sure if the move really worse it.
> > > >
> > > >
> > > > 2014-08-28 19:06 GMT-04:00 Konstantin Boudnik <co...@apache.org>:
> > > >
> > > >> Guys,
> > > >>
> > > >> I've been looking into some service scripting around HBase lifecycle
> > > >> management, etc. and couldn't help but wonder why those were
> written in
> > > >> Ruby
> > > >> of all JVM languages? Historical legacy aside, it seems that current
> > > >> HBase
> > > >> is
> > > >> still using JRuby 1.6.5 vs the latest at 1.9+ or perhaps even later.
> > > >>
> > > >> At any rate, I was wondering if replacing Ruby with a more Java-like
> > > >> scripting
> > > >> extension (if the scripting-2-Java API bridge is what indeed
> desired)
> > > >> would be
> > > >> of any interest here? An obvious choice would be Groovy
> > > >> (http://groovy.codehaus.org/). One of the main reasons behind my
> > > proposal
> > > >> is
> > > >> stack simplification: Bigtop is very actively using Groovy as a
> > > scripting
> > > >> language of choice to do builds, develop smoke tests, etc. So, it is
> > > >> already
> > > >> there and guaranteed to be installed as a part of any Bigtop-derived
> > > >> Hadoop
> > > >> distro. There are other benefits, where, if desired, one can just
> write
> > > >> Java
> > > >> code inside of a Groovy script, without a need to learn yet another
> > > >> language
> > > >> like Ruby.
> > > >>
> > > >> This is perhaps not of an immediate priority for the community, but
> if
> > > >> there's
> > > >> enough interest, I can give it an initial shot to demo'ed what I am
> > > >> really
> > > >> talking about.
> > > >>
> > > >> Thoughts?
> > > >> --
> > > >> Regards,
> > > >>   Cos
> > > >>
> > > >>
> > > >
> > >
> > >
> > > --
> > > Thanks,
> > > Michael Antonov
> > >
>



-- 
Sean

Re: Ruby scripting in HBase?

[Konstantin Boudnik <co...@apache.org>]

On Thu, Aug 28, 2014 at 07:51PM, Jean-Marc Spaggiari wrote:
> Was more thinking about the first. Having a way to automate groovy shell
> commands testing in the build.

I am pretty sure there is. After all, Gradle (a build DSL based, using Groovy
as the underlying language) has unit testing in it ;)

The issue with testing of these scripts, as far as I can see, is that they
might need some sort of mocking involved to work around the fact that these
command expect a running HBase cluster. Makes sense?

As for the 'pain vs pain' comment: I am not really sure why. Groovy is really
just a Java with added benefits of real lambdas, dynamic bindings, etc.

Cos

> Le 2014-08-28 19:45, "Mikhail Antonov" <ol...@gmail.com> a écrit :
> 
> > JM - do you mean writing unit or integration tests for groovy commands
> > themselves, or to be able to write HBase tests in Groovy? If later
> > one, then I'd think HBase tests may benefit a lot in conciseness if
> > written in Groovy.
> >
> > -Mikhail
> >
> > 2014-08-28 16:39 GMT-07:00, Jean-Marc Spaggiari <je...@spaggiari.org>:
> > > Are we not just going to replace a pain by another pain?
> > >
> > > Can we build test suites for Groovy? I mean, not just use groovy to build
> > > test, but build a test script which will test groovy? I think it's one of
> > > the main issues today with JRuby shell.
> > >
> > > I prefer Groovy over JRuby but not sure if the move really worse it.
> > >
> > >
> > > 2014-08-28 19:06 GMT-04:00 Konstantin Boudnik <co...@apache.org>:
> > >
> > >> Guys,
> > >>
> > >> I've been looking into some service scripting around HBase lifecycle
> > >> management, etc. and couldn't help but wonder why those were written in
> > >> Ruby
> > >> of all JVM languages? Historical legacy aside, it seems that current
> > >> HBase
> > >> is
> > >> still using JRuby 1.6.5 vs the latest at 1.9+ or perhaps even later.
> > >>
> > >> At any rate, I was wondering if replacing Ruby with a more Java-like
> > >> scripting
> > >> extension (if the scripting-2-Java API bridge is what indeed desired)
> > >> would be
> > >> of any interest here? An obvious choice would be Groovy
> > >> (http://groovy.codehaus.org/). One of the main reasons behind my
> > proposal
> > >> is
> > >> stack simplification: Bigtop is very actively using Groovy as a
> > scripting
> > >> language of choice to do builds, develop smoke tests, etc. So, it is
> > >> already
> > >> there and guaranteed to be installed as a part of any Bigtop-derived
> > >> Hadoop
> > >> distro. There are other benefits, where, if desired, one can just write
> > >> Java
> > >> code inside of a Groovy script, without a need to learn yet another
> > >> language
> > >> like Ruby.
> > >>
> > >> This is perhaps not of an immediate priority for the community, but if
> > >> there's
> > >> enough interest, I can give it an initial shot to demo'ed what I am
> > >> really
> > >> talking about.
> > >>
> > >> Thoughts?
> > >> --
> > >> Regards,
> > >>   Cos
> > >>
> > >>
> > >
> >
> >
> > --
> > Thanks,
> > Michael Antonov
> >

Re: Ruby scripting in HBase?

[Jean-Marc Spaggiari <je...@spaggiari.org>]

Was more thinking about the first. Having a way to automate groovy shell
commands testing in the build.
Le 2014-08-28 19:45, "Mikhail Antonov" <ol...@gmail.com> a écrit :

> JM - do you mean writing unit or integration tests for groovy commands
> themselves, or to be able to write HBase tests in Groovy? If later
> one, then I'd think HBase tests may benefit a lot in conciseness if
> written in Groovy.
>
> -Mikhail
>
> 2014-08-28 16:39 GMT-07:00, Jean-Marc Spaggiari <je...@spaggiari.org>:
> > Are we not just going to replace a pain by another pain?
> >
> > Can we build test suites for Groovy? I mean, not just use groovy to build
> > test, but build a test script which will test groovy? I think it's one of
> > the main issues today with JRuby shell.
> >
> > I prefer Groovy over JRuby but not sure if the move really worse it.
> >
> >
> > 2014-08-28 19:06 GMT-04:00 Konstantin Boudnik <co...@apache.org>:
> >
> >> Guys,
> >>
> >> I've been looking into some service scripting around HBase lifecycle
> >> management, etc. and couldn't help but wonder why those were written in
> >> Ruby
> >> of all JVM languages? Historical legacy aside, it seems that current
> >> HBase
> >> is
> >> still using JRuby 1.6.5 vs the latest at 1.9+ or perhaps even later.
> >>
> >> At any rate, I was wondering if replacing Ruby with a more Java-like
> >> scripting
> >> extension (if the scripting-2-Java API bridge is what indeed desired)
> >> would be
> >> of any interest here? An obvious choice would be Groovy
> >> (http://groovy.codehaus.org/). One of the main reasons behind my
> proposal
> >> is
> >> stack simplification: Bigtop is very actively using Groovy as a
> scripting
> >> language of choice to do builds, develop smoke tests, etc. So, it is
> >> already
> >> there and guaranteed to be installed as a part of any Bigtop-derived
> >> Hadoop
> >> distro. There are other benefits, where, if desired, one can just write
> >> Java
> >> code inside of a Groovy script, without a need to learn yet another
> >> language
> >> like Ruby.
> >>
> >> This is perhaps not of an immediate priority for the community, but if
> >> there's
> >> enough interest, I can give it an initial shot to demo'ed what I am
> >> really
> >> talking about.
> >>
> >> Thoughts?
> >> --
> >> Regards,
> >>   Cos
> >>
> >>
> >
>
>
> --
> Thanks,
> Michael Antonov
>

Re: Ruby scripting in HBase?

[Mikhail Antonov <ol...@gmail.com>]

JM - do you mean writing unit or integration tests for groovy commands
themselves, or to be able to write HBase tests in Groovy? If later
one, then I'd think HBase tests may benefit a lot in conciseness if
written in Groovy.

-Mikhail

2014-08-28 16:39 GMT-07:00, Jean-Marc Spaggiari <je...@spaggiari.org>:
> Are we not just going to replace a pain by another pain?
>
> Can we build test suites for Groovy? I mean, not just use groovy to build
> test, but build a test script which will test groovy? I think it's one of
> the main issues today with JRuby shell.
>
> I prefer Groovy over JRuby but not sure if the move really worse it.
>
>
> 2014-08-28 19:06 GMT-04:00 Konstantin Boudnik <co...@apache.org>:
>
>> Guys,
>>
>> I've been looking into some service scripting around HBase lifecycle
>> management, etc. and couldn't help but wonder why those were written in
>> Ruby
>> of all JVM languages? Historical legacy aside, it seems that current
>> HBase
>> is
>> still using JRuby 1.6.5 vs the latest at 1.9+ or perhaps even later.
>>
>> At any rate, I was wondering if replacing Ruby with a more Java-like
>> scripting
>> extension (if the scripting-2-Java API bridge is what indeed desired)
>> would be
>> of any interest here? An obvious choice would be Groovy
>> (http://groovy.codehaus.org/). One of the main reasons behind my proposal
>> is
>> stack simplification: Bigtop is very actively using Groovy as a scripting
>> language of choice to do builds, develop smoke tests, etc. So, it is
>> already
>> there and guaranteed to be installed as a part of any Bigtop-derived
>> Hadoop
>> distro. There are other benefits, where, if desired, one can just write
>> Java
>> code inside of a Groovy script, without a need to learn yet another
>> language
>> like Ruby.
>>
>> This is perhaps not of an immediate priority for the community, but if
>> there's
>> enough interest, I can give it an initial shot to demo'ed what I am
>> really
>> talking about.
>>
>> Thoughts?
>> --
>> Regards,
>>   Cos
>>
>>
>


-- 
Thanks,
Michael Antonov

Re: Ruby scripting in HBase?

[Jean-Marc Spaggiari <je...@spaggiari.org>]

Are we not just going to replace a pain by another pain?

Can we build test suites for Groovy? I mean, not just use groovy to build
test, but build a test script which will test groovy? I think it's one of
the main issues today with JRuby shell.

I prefer Groovy over JRuby but not sure if the move really worse it.


2014-08-28 19:06 GMT-04:00 Konstantin Boudnik <co...@apache.org>:

> Guys,
>
> I've been looking into some service scripting around HBase lifecycle
> management, etc. and couldn't help but wonder why those were written in
> Ruby
> of all JVM languages? Historical legacy aside, it seems that current HBase
> is
> still using JRuby 1.6.5 vs the latest at 1.9+ or perhaps even later.
>
> At any rate, I was wondering if replacing Ruby with a more Java-like
> scripting
> extension (if the scripting-2-Java API bridge is what indeed desired)
> would be
> of any interest here? An obvious choice would be Groovy
> (http://groovy.codehaus.org/). One of the main reasons behind my proposal
> is
> stack simplification: Bigtop is very actively using Groovy as a scripting
> language of choice to do builds, develop smoke tests, etc. So, it is
> already
> there and guaranteed to be installed as a part of any Bigtop-derived Hadoop
> distro. There are other benefits, where, if desired, one can just write
> Java
> code inside of a Groovy script, without a need to learn yet another
> language
> like Ruby.
>
> This is perhaps not of an immediate priority for the community, but if
> there's
> enough interest, I can give it an initial shot to demo'ed what I am really
> talking about.
>
> Thoughts?
> --
> Regards,
>   Cos
>
>