You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Lars Francke <la...@gmail.com> on 2010/05/14 10:54:00 UTC
Re: Decision/Discussion about HBASE-2170: Lightweight
client/Refactoring of source tree
I'm sorry I must have missed your answer somehow.
> The problem with using 1 jar "everywhere" is that the client
> dependencies are more than the server ones. So to use the client you
> only need zookeeper and hadoop, but the server also needs JSP, JspC,
> etc, etc, things we'd rather not have our clients have to pull into
> their classpath.
Exactly.
> Maybe we can publish a POM with a stripped down set of deps? Would
> that be a reasonable half solution? :-)
That would be possible too, yes. As you aptly named it: It's a
reasonable half solution :) It'd be great if folks could just add a
dependency on org.apache.hbase:hbase-client and be done with it. I
think users won't care that much about how it is done but I think it
would be a step in the right direction and make adoption of HBase even
easier.
The separation of code would make this distinction even clearer and
the dependencies easier to maintain but I'm good with any (agreed
upon) solution for this problem.
Cheers,
Lars
Re: Decision/Discussion about HBASE-2170: Lightweight client/Refactoring
of source tree
Posted by Al Lias <al...@gmx.de>.
Am 17.05.2010 20:28, Stack wrote:
> ...
>>
>
> So, if no modules, could we still produce an hbase-client via
> something like an assembly or, given that it would need a supporting
> pom, we could only do it if we keep up modules?
>
I think yes, otherwise you (basically) confuse the one-project->one
artifact logic of mvn. But the "module" is just one dir + the pom (which
is the artifact), perhaps you can even get away without a dir. But if
thats well maintained, it gives a much nicer (i.e. simpler) reference
for other projects building with mvn.
Dont forget about many deep dependencies that occationally lead to
conflicts on a nasty nested level (log4j is a nice one...).
>> If you then look further for a one-piece-jar (for the poor without
>> maven), the shade plugin would do the job.
>>
>
> This seems easy to do in mvn. The resultant jar though would be enormous.
>
Yes; around 23Mb today. And when you think about putting it into some
repo, one probably has to be clear about the licenses insight too.
A jar based on a cleaned pom as mentioned above does make it to around 5
Mb (based on hbase0.20.3 excersize).
But the one-jar thing is a solution for reasonable simple projects only:
the classes inside could conflict with classes of other jars in your
classpath, thus even increasing the confusion.
So a well maintained pom for a artifact "hbase-client" would be great...
Al
> ...
Re: Decision/Discussion about HBASE-2170: Lightweight
client/Refactoring of source tree
Posted by Stack <st...@duboce.net>.
On Fri, May 14, 2010 at 3:00 PM, Al Lias <al...@gmx.de> wrote:
> oh I was not in the trunk and haven't seen the modules...
>
Yeah, trunk has modules but I'm thinking we should get rid of them.
My reasoning is that we've just undone all contribs. They've all been
moved out to github or folded back up into core. Now we are mavenized
its easy (enough) for dependent projects pulling in the latest hbase
build. If no contribs, there is no need for an involved mvn build we
currently have with submodules of core, contrib, etc., so we should be
able to move to a cleaner, flat mvn model where there are no
submodules and the mvn product is a single hbase.jar.
> The best would certainly be a sub dir + a module "hbase-client".
>
> If that is too much work with moving files around, you could start a
> hbase-client module without any files in it, just a dumb pom.xml that
> excludes everything not needed (see the attached example; projects that
> include that dependency will not go for the org.eclipse.jdt dependency
> for instance).
>
So, if no modules, could we still produce an hbase-client via
something like an assembly or, given that it would need a supporting
pom, we could only do it if we keep up modules?
> If you then look further for a one-piece-jar (for the poor without
> maven), the shade plugin would do the job.
>
This seems easy to do in mvn. The resultant jar though would be enormous.
Thanks,
St.Ack
> Regards
>
> Al
>
> Am 14.05.2010 18:59, schrieb Lars Francke:
>> Sorry. I accidentally hit "send" too soon. Damn phone keyboards.
>>
>> HBase already uses sub modules. But to split this up even further we'd have
>> to refactor the code which won't be easy.
>>
>> So what we thought of doing is provide a second .pom for the same
>> code/artifact but with a stripped down set of dependencies. There are
>> multiple ways of doing this. Do you have any experience with this use-case
>> and provide insight into a good solution? That'd be great as I've never done
>> _this_ in particular.
>>
>> Any help is appreciated.
>>
>> Cheers,
>> Lars
>>
>>
>>>
>>> On May 14, 2010 6:10 PM, "Al Lias" <al...@gmx.de> wrote:
>>>
>>> ...Maven can have submodules, ea...
>>
>>>
>>>> I'm sorry I must have missed your answer somehow.
>>>>
>>>> The problem with using 1 jar "everywhere" ...
>>
>
>
Re: Decision/Discussion about HBASE-2170: Lightweight client/Refactoring
of source tree
Posted by Al Lias <al...@gmx.de>.
oh I was not in the trunk and haven't seen the modules...
The best would certainly be a sub dir + a module "hbase-client".
If that is too much work with moving files around, you could start a
hbase-client module without any files in it, just a dumb pom.xml that
excludes everything not needed (see the attached example; projects that
include that dependency will not go for the org.eclipse.jdt dependency
for instance).
But in any case you'll have to maintain a exclusion list that will be
quite big; and many libs have to be excluded from hbase:core AND from
hadoop:core
One can also argue, why these dependencies got in initially...still I
also think it makes sense (We actually also use a stripped down jar set,
reducing client sizes by 15 Mb)
If you then look further for a one-piece-jar (for the poor without
maven), the shade plugin would do the job.
Regards
Al
Am 14.05.2010 18:59, schrieb Lars Francke:
> Sorry. I accidentally hit "send" too soon. Damn phone keyboards.
>
> HBase already uses sub modules. But to split this up even further we'd have
> to refactor the code which won't be easy.
>
> So what we thought of doing is provide a second .pom for the same
> code/artifact but with a stripped down set of dependencies. There are
> multiple ways of doing this. Do you have any experience with this use-case
> and provide insight into a good solution? That'd be great as I've never done
> _this_ in particular.
>
> Any help is appreciated.
>
> Cheers,
> Lars
>
>
>>
>> On May 14, 2010 6:10 PM, "Al Lias" <al...@gmx.de> wrote:
>>
>> ...Maven can have submodules, ea...
>
>>
>>> I'm sorry I must have missed your answer somehow.
>>>
>>> The problem with using 1 jar "everywhere" ...
>
Re: Decision/Discussion about HBASE-2170: Lightweight
client/Refactoring of source tree
Posted by Lars Francke <la...@gmail.com>.
Sorry. I accidentally hit "send" too soon. Damn phone keyboards.
HBase already uses sub modules. But to split this up even further we'd have
to refactor the code which won't be easy.
So what we thought of doing is provide a second .pom for the same
code/artifact but with a stripped down set of dependencies. There are
multiple ways of doing this. Do you have any experience with this use-case
and provide insight into a good solution? That'd be great as I've never done
_this_ in particular.
Any help is appreciated.
Cheers,
Lars
>
> On May 14, 2010 6:10 PM, "Al Lias" <al...@gmx.de> wrote:
>
> ...Maven can have submodules, ea...
>
> > I'm sorry I must have missed your answer somehow.
> >
>> The problem with using 1 jar "everywhere" ...
Re: Decision/Discussion about HBASE-2170: Lightweight
client/Refactoring of source tree
Posted by Lars Francke <la...@gmail.com>.
Mr. Lias,
HBase currently uses aub
On May 14, 2010 6:10 PM, "Al Lias" <al...@gmx.de> wrote:
...Maven can have submodules, each having its own artifact and thus a
own pom. Its a simple setup. The client code seems to me a perfect
module....
Regards,
Al
Am 14.05.2010 10:54, schrieb Lars Francke:
> I'm sorry I must have missed your answer somehow.
>
>> The problem with using 1 jar "everywhere" ...
Re: Decision/Discussion about HBASE-2170: Lightweight client/Refactoring
of source tree
Posted by Al Lias <al...@gmx.de>.
...Maven can have submodules, each having its own artifact and thus a
own pom. Its a simple setup. The client code seems to me a perfect
module....
Regards,
Al
Am 14.05.2010 10:54, schrieb Lars Francke:
> I'm sorry I must have missed your answer somehow.
>
>> The problem with using 1 jar "everywhere" is that the client
>> dependencies are more than the server ones. So to use the client you
>> only need zookeeper and hadoop, but the server also needs JSP, JspC,
>> etc, etc, things we'd rather not have our clients have to pull into
>> their classpath.
>
> Exactly.
>
>> Maybe we can publish a POM with a stripped down set of deps? Would
>> that be a reasonable half solution? :-)
>
> That would be possible too, yes. As you aptly named it: It's a
> reasonable half solution :) It'd be great if folks could just add a
> dependency on org.apache.hbase:hbase-client and be done with it. I
> think users won't care that much about how it is done but I think it
> would be a step in the right direction and make adoption of HBase even
> easier.
>
> The separation of code would make this distinction even clearer and
> the dependencies easier to maintain but I'm good with any (agreed
> upon) solution for this problem.
>
> Cheers,
> Lars