[DISCUSS] Interest in "client" and "server" tarballs?

[Josh Elser <el...@apache.org>]

Had an ask from some $dayjobs folks to split up the "monolithic" hbase 
binary tarball that we generate in the build into two pieces:

1. A minimal installation which would only contain things that "normal 
users" would need.
2. A server-side installation which would not contain things only used 
by clients (in reality, I think this would be essentially the same as 
our current tarball).

This would be pretty simple to do in >=2.0.0 with some extra assembly 
descriptors (with some of the extra break-out of classes previously in 
hbase-server), but I wasn't sure what kind of interest other folks would 
have. Obviously, I would only want to suggest such a change if there was 
buy-in from everyone else; otherwise, it will just cause fragmentation 
in the project. It's something easily (I think) achievable outside of 
the standard build process, so it's not something super-critical to be 
maintained upstream.

Any thoughts?

- Josh

Re: [DISCUSS] Interest in "client" and "server" tarballs?

[Josh Elser <el...@apache.org>]

On 1/5/18 10:43 AM, Stack wrote:
> On Thu, Jan 4, 2018 at 2:50 PM, Josh Elser<el...@apache.org>  wrote:
> 
>> Had an ask from some $dayjobs folks to split up the "monolithic" hbase
>> binary tarball that we generate in the build into two pieces:
>>
>> 1. A minimal installation which would only contain things that "normal
>> users" would need.
>> 2. A server-side installation which would not contain things only used by
>> clients (in reality, I think this would be essentially the same as our
>> current tarball).
>>
>> This would be pretty simple to do in >=2.0.0 with some extra assembly
>> descriptors (with some of the extra break-out of classes previously in
>> hbase-server), but I wasn't sure what kind of interest other folks would
>> have. Obviously, I would only want to suggest such a change if there was
>> buy-in from everyone else; otherwise, it will just cause fragmentation in
>> the project. It's something easily (I think) achievable outside of the
>> standard build process, so it's not something super-critical to be
>> maintained upstream.
>>
>> Any thoughts?
>>
>>
> We've been working toward this goal w/ a good while now (Appy's untangling
> mapreduce and zookeeper breaking them out as distinct modules, Duo making
> client depend on basic, read-only usage of zk, the CP refactoring project,
> and so on).
> 
> An assembly that did an hbase-client tgz is probably not far off (Might
> require some more messing untangling dependency knots and tangles...). As
> per Drob, it would be good to foreground our shaded access artifacts.
> 
> Thanks Josh,
> S

Just opened up https://issues.apache.org/jira/browse/HBASE-19735 and 
parked an initial patch on it. Happy to entertain more discussion here 
or move it up there.

Our dependency de-coupling that Appy, et al, have been doing helped, but 
trying to make a "minimal" tarball for clients definitely still has some 
pains that require manual exclusions. Will continue to poke.

Re: [DISCUSS] Interest in "client" and "server" tarballs?

[Josh Elser <el...@apache.org>]

On 1/5/18 10:43 AM, Stack wrote:
> On Thu, Jan 4, 2018 at 2:50 PM, Josh Elser <el...@apache.org> wrote:
> 
>> Had an ask from some $dayjobs folks to split up the "monolithic" hbase
>> binary tarball that we generate in the build into two pieces:
>>
>> 1. A minimal installation which would only contain things that "normal
>> users" would need.
>> 2. A server-side installation which would not contain things only used by
>> clients (in reality, I think this would be essentially the same as our
>> current tarball).
>>
>> This would be pretty simple to do in >=2.0.0 with some extra assembly
>> descriptors (with some of the extra break-out of classes previously in
>> hbase-server), but I wasn't sure what kind of interest other folks would
>> have. Obviously, I would only want to suggest such a change if there was
>> buy-in from everyone else; otherwise, it will just cause fragmentation in
>> the project. It's something easily (I think) achievable outside of the
>> standard build process, so it's not something super-critical to be
>> maintained upstream.
>>
>> Any thoughts?
>>
>>
> We've been working toward this goal w/ a good while now (Appy's untangling
> mapreduce and zookeeper breaking them out as distinct modules, Duo making
> client depend on basic, read-only usage of zk, the CP refactoring project,
> and so on).
> 
> An assembly that did an hbase-client tgz is probably not far off (Might
> require some more messing untangling dependency knots and tangles...). As
> per Drob, it would be good to foreground our shaded access artifacts.
> 
> Thanks Josh,
> S

Superb. Sounds like we're on the same page that something here would be 
aligned with the long-term vision. I'll try to see what I can whip up 
and we can iterate on that.

Thanks all.

Re: [DISCUSS] Interest in "client" and "server" tarballs?

[Stack <st...@duboce.net>]

On Thu, Jan 4, 2018 at 2:50 PM, Josh Elser <el...@apache.org> wrote:

> Had an ask from some $dayjobs folks to split up the "monolithic" hbase
> binary tarball that we generate in the build into two pieces:
>
> 1. A minimal installation which would only contain things that "normal
> users" would need.
> 2. A server-side installation which would not contain things only used by
> clients (in reality, I think this would be essentially the same as our
> current tarball).
>
> This would be pretty simple to do in >=2.0.0 with some extra assembly
> descriptors (with some of the extra break-out of classes previously in
> hbase-server), but I wasn't sure what kind of interest other folks would
> have. Obviously, I would only want to suggest such a change if there was
> buy-in from everyone else; otherwise, it will just cause fragmentation in
> the project. It's something easily (I think) achievable outside of the
> standard build process, so it's not something super-critical to be
> maintained upstream.
>
> Any thoughts?
>
>
We've been working toward this goal w/ a good while now (Appy's untangling
mapreduce and zookeeper breaking them out as distinct modules, Duo making
client depend on basic, read-only usage of zk, the CP refactoring project,
and so on).

An assembly that did an hbase-client tgz is probably not far off (Might
require some more messing untangling dependency knots and tangles...). As
per Drob, it would be good to foreground our shaded access artifacts.

Thanks Josh,
S





> - Josh
>

Re: [DISCUSS] Interest in "client" and "server" tarballs?

[Josh Elser <el...@apache.org>]

Depends on the definition of "covered", I think :)

I think that the shaded artifacts (client and mapreduce) would be 
contained in such as tarball as described by #1, but we would also have 
some more in there. A short list:

* hbase-env.sh and hbase-site.xml (or pared down versions, thereof)
* bin/hbase,hirb.rb,hbase-common.sh,hbase-config.sh
* Subset of jars and all of the ruby files in lib/

Essentially, whatever might be needed for a human who wants to interact 
with HBase as a "user". Things that an administrator would need would be 
in the #2 tarball.

On 1/4/18 3:54 PM, Mike Drob wrote:
> Isn't the first covered by the shaded-client and shaded-mapreduce artifacts?
> 
> On Thu, Jan 4, 2018 at 2:50 PM, Josh Elser <el...@apache.org> wrote:
> 
>> Had an ask from some $dayjobs folks to split up the "monolithic" hbase
>> binary tarball that we generate in the build into two pieces:
>>
>> 1. A minimal installation which would only contain things that "normal
>> users" would need.
>> 2. A server-side installation which would not contain things only used by
>> clients (in reality, I think this would be essentially the same as our
>> current tarball).
>>
>> This would be pretty simple to do in >=2.0.0 with some extra assembly
>> descriptors (with some of the extra break-out of classes previously in
>> hbase-server), but I wasn't sure what kind of interest other folks would
>> have. Obviously, I would only want to suggest such a change if there was
>> buy-in from everyone else; otherwise, it will just cause fragmentation in
>> the project. It's something easily (I think) achievable outside of the
>> standard build process, so it's not something super-critical to be
>> maintained upstream.
>>
>> Any thoughts?
>>
>> - Josh
>>
> 

Re: [DISCUSS] Interest in "client" and "server" tarballs?

[Mike Drob <md...@apache.org>]

Isn't the first covered by the shaded-client and shaded-mapreduce artifacts?

On Thu, Jan 4, 2018 at 2:50 PM, Josh Elser <el...@apache.org> wrote:

> Had an ask from some $dayjobs folks to split up the "monolithic" hbase
> binary tarball that we generate in the build into two pieces:
>
> 1. A minimal installation which would only contain things that "normal
> users" would need.
> 2. A server-side installation which would not contain things only used by
> clients (in reality, I think this would be essentially the same as our
> current tarball).
>
> This would be pretty simple to do in >=2.0.0 with some extra assembly
> descriptors (with some of the extra break-out of classes previously in
> hbase-server), but I wasn't sure what kind of interest other folks would
> have. Obviously, I would only want to suggest such a change if there was
> buy-in from everyone else; otherwise, it will just cause fragmentation in
> the project. It's something easily (I think) achievable outside of the
> standard build process, so it's not something super-critical to be
> maintained upstream.
>
> Any thoughts?
>
> - Josh
>