You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@bigtop.apache.org by Ted Yu <yu...@gmail.com> on 2016/06/22 22:23:01 UTC

redundant zookeeper/lib directory

Hi,
In bigtop-packages/src/common/hbase/install_hbase.sh :

export
HBASE_CLASSPATH=\$HADOOP_CONF:\$HADOOP_HOME/*:\$HADOOP_HOME/lib/*:\$ZOOKEEPER_HOME/*:\$ZOOKEEPER_HOME/lib/*:\$HBASE_CLASSPATH

I am a bit curious why zookeeper/lib directory is needed in the classpath.
The zookeeper jar is in $HBASE_HOME/lib, right ?
What else is needed ?

Thanks

Re: redundant zookeeper/lib directory

Posted by Ted Yu <yu...@gmail.com>.

Thanks for responding, Roman.

I think symlinks are used (lib below is hbase/lib):

lrwxrwxrwx. 1 root root 45 Jan 27 18:33 lib/zookeeper.jar ->
<path>/zookeeper/zookeeper.jar


On Thu, Jun 23, 2016 at 10:17 AM, Roman Shaposhnik <ro...@shaposhnik.org>
wrote:

> Thanks for filing the JIRA Ted, let me take a more careful look at the
> symptopms.
>
> Let me try to explain the rationale though first: the original intent
> was to make sure
> that our packages never ship the same binary twice. IOW, while it is
> OK for a binary
> distribution of HBase to have zookeper.jar in its lib folder, it is
> NOT quite ok to do so
> in Linux packaging (for the reasons of upgrades, etc.). That is why
> HBase package
> depdends on zookeeper package -- to make sure that there's a single place
> on the
> system from where ZK jars come from.
>
> Hence the manipulation of the classpath.
>
> Now, since the original decision, I *think* (but I need to check) we
> went the symlinks
> route. If that's the case -- you're right -- classpath modificiations
> are no longer needed.
>
> Thanks,
> Roman.
>
> On Thu, Jun 23, 2016 at 2:53 AM, Ted Yu <yu...@gmail.com> wrote:
> > Thanks for responding.
> >
> > Logged BIGTOP-2488 with symptom which prompted my question.
> >
> > On Thu, Jun 23, 2016 at 12:56 AM, Konstantinos Tsakalozos <
> > kos.tsakalozos@canonical.com> wrote:
> >
> >> Hi Ted,
> >>
> >> The issue you report, could very well be related to ticket:
> >> https://issues.apache.org/jira/browse/BIGTOP-2466 where we identify
> that
> >> HBase puppet scripts also deploy Zookeeper. I would suggest you open a
> >> ticket so that we keep track of this.
> >>
> >> Is having the the zookeeper lib in the classpath causing a problem? How
> did
> >> you spot this potential defect?
> >>
> >> Thank you,
> >> Konstantinos
> >>
> >>
> >> On Thu, Jun 23, 2016 at 1:23 AM, Ted Yu <yu...@gmail.com> wrote:
> >>
> >> > Hi,
> >> > In bigtop-packages/src/common/hbase/install_hbase.sh :
> >> >
> >> > export
> >> >
> >> >
> >>
> HBASE_CLASSPATH=\$HADOOP_CONF:\$HADOOP_HOME/*:\$HADOOP_HOME/lib/*:\$ZOOKEEPER_HOME/*:\$ZOOKEEPER_HOME/lib/*:\$HBASE_CLASSPATH
> >> >
> >> > I am a bit curious why zookeeper/lib directory is needed in the
> >> classpath.
> >> > The zookeeper jar is in $HBASE_HOME/lib, right ?
> >> > What else is needed ?
> >> >
> >> > Thanks
> >> >
> >>
>

Re: redundant zookeeper/lib directory

Posted by Roman Shaposhnik <ro...@shaposhnik.org>.

Thanks for filing the JIRA Ted, let me take a more careful look at the
symptopms.

Let me try to explain the rationale though first: the original intent
was to make sure
that our packages never ship the same binary twice. IOW, while it is
OK for a binary
distribution of HBase to have zookeper.jar in its lib folder, it is
NOT quite ok to do so
in Linux packaging (for the reasons of upgrades, etc.). That is why
HBase package
depdends on zookeeper package -- to make sure that there's a single place on the
system from where ZK jars come from.

Hence the manipulation of the classpath.

Now, since the original decision, I *think* (but I need to check) we
went the symlinks
route. If that's the case -- you're right -- classpath modificiations
are no longer needed.

Thanks,
Roman.

On Thu, Jun 23, 2016 at 2:53 AM, Ted Yu <yu...@gmail.com> wrote:
> Thanks for responding.
>
> Logged BIGTOP-2488 with symptom which prompted my question.
>
> On Thu, Jun 23, 2016 at 12:56 AM, Konstantinos Tsakalozos <
> kos.tsakalozos@canonical.com> wrote:
>
>> Hi Ted,
>>
>> The issue you report, could very well be related to ticket:
>> https://issues.apache.org/jira/browse/BIGTOP-2466 where we identify that
>> HBase puppet scripts also deploy Zookeeper. I would suggest you open a
>> ticket so that we keep track of this.
>>
>> Is having the the zookeeper lib in the classpath causing a problem? How did
>> you spot this potential defect?
>>
>> Thank you,
>> Konstantinos
>>
>>
>> On Thu, Jun 23, 2016 at 1:23 AM, Ted Yu <yu...@gmail.com> wrote:
>>
>> > Hi,
>> > In bigtop-packages/src/common/hbase/install_hbase.sh :
>> >
>> > export
>> >
>> >
>> HBASE_CLASSPATH=\$HADOOP_CONF:\$HADOOP_HOME/*:\$HADOOP_HOME/lib/*:\$ZOOKEEPER_HOME/*:\$ZOOKEEPER_HOME/lib/*:\$HBASE_CLASSPATH
>> >
>> > I am a bit curious why zookeeper/lib directory is needed in the
>> classpath.
>> > The zookeeper jar is in $HBASE_HOME/lib, right ?
>> > What else is needed ?
>> >
>> > Thanks
>> >
>>

Re: redundant zookeeper/lib directory

Posted by Ted Yu <yu...@gmail.com>.

Thanks for responding.

Logged BIGTOP-2488 with symptom which prompted my question.

On Thu, Jun 23, 2016 at 12:56 AM, Konstantinos Tsakalozos <
kos.tsakalozos@canonical.com> wrote:

> Hi Ted,
>
> The issue you report, could very well be related to ticket:
> https://issues.apache.org/jira/browse/BIGTOP-2466 where we identify that
> HBase puppet scripts also deploy Zookeeper. I would suggest you open a
> ticket so that we keep track of this.
>
> Is having the the zookeeper lib in the classpath causing a problem? How did
> you spot this potential defect?
>
> Thank you,
> Konstantinos
>
>
> On Thu, Jun 23, 2016 at 1:23 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > Hi,
> > In bigtop-packages/src/common/hbase/install_hbase.sh :
> >
> > export
> >
> >
> HBASE_CLASSPATH=\$HADOOP_CONF:\$HADOOP_HOME/*:\$HADOOP_HOME/lib/*:\$ZOOKEEPER_HOME/*:\$ZOOKEEPER_HOME/lib/*:\$HBASE_CLASSPATH
> >
> > I am a bit curious why zookeeper/lib directory is needed in the
> classpath.
> > The zookeeper jar is in $HBASE_HOME/lib, right ?
> > What else is needed ?
> >
> > Thanks
> >
>

Re: redundant zookeeper/lib directory

Posted by Konstantinos Tsakalozos <ko...@canonical.com>.

Hi Ted,

The issue you report, could very well be related to ticket:
https://issues.apache.org/jira/browse/BIGTOP-2466 where we identify that
HBase puppet scripts also deploy Zookeeper. I would suggest you open a
ticket so that we keep track of this.

Is having the the zookeeper lib in the classpath causing a problem? How did
you spot this potential defect?

Thank you,
Konstantinos

On Thu, Jun 23, 2016 at 1:23 AM, Ted Yu <yu...@gmail.com> wrote:

> Hi,
> In bigtop-packages/src/common/hbase/install_hbase.sh :
>
> export
>
> HBASE_CLASSPATH=\$HADOOP_CONF:\$HADOOP_HOME/*:\$HADOOP_HOME/lib/*:\$ZOOKEEPER_HOME/*:\$ZOOKEEPER_HOME/lib/*:\$HBASE_CLASSPATH
>
> I am a bit curious why zookeeper/lib directory is needed in the classpath.
> The zookeeper jar is in $HBASE_HOME/lib, right ?
> What else is needed ?
>
> Thanks
>