You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Jason Wen <zh...@workday.com.INVALID> on 2021/10/25 19:23:16 UTC
Oozie 5.2.1 and Zookeeper 3.4.14 compatibility issue
Hi All,
We recently hit an issue when using oozie 5.2.1 with zookeeper 3.4.14 that was built from bigtop. The curator version 2.5.0 that oozie 5.2.1 is using seems not working with zookeeper 3.4.14. When oozie is started, it tries to connect to zookeeper but is always stuck at following log:
Connecting to ZooKeeper with SASL/Kerberos and using 'sasl' ACLs
Hadoop 3.2.2 is using curator 2.13.0 version, so we tried to replace curator libs in /usr/lib/oozie/lib dir with curator 2.13.0 version jars. After the curator jars are replaced with 2.13.0 version, the oozie service can be started successfully.
Has anyone tested Oozie 5.2.1 connection to Zookeeper 3.4.14 with Kerberos enabled? This looks like a bug in bigtop 3.0.0
Thanks,
Jason
Re: Oozie 5.2.1 and Zookeeper 3.4.14 compatibility issue
Posted by Evans Ye <ev...@apache.org>.
Hi Jason,
Good finding and look forward to your PR.
Evans
Jason Wen <zh...@workday.com>於 2021年10月29日 週五，下午9:46寫道：
> Hi All,
>
>
>
> We find the root cause of the issue.
>
> This issue happens when HA is enabled on oozie. oozie-5.2.1 is using
> curator-2.5.0 which depends on guava-11.0.2, but bigtop-3.0.0 apply the
> patch to change guava version to 27.0-jre in order to work with
> Hadoop-3.2.2:
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/oozie/patch0-hadoop322-hive312.diff#L103
>
> This guava version change breaks the curator-2.5.0 because it depends on
> some API that is not available anymore in guava-27.0-jre, so it will throw
> NoSuchMethod error when starting oozie.
>
> I will file a jira to bigtop and create the PR for fixing the issue.
>
>
>
> Thanks,
>
> Jason
>
>
>
> *From: *Jason Wen <zh...@workday.com>
> *Reply-To: *"user@bigtop.apache.org" <us...@bigtop.apache.org>
> *Date: *Monday, October 25, 2021 at 12:23 PM
> *To: *"user@bigtop.apache.org" <us...@bigtop.apache.org>, "
> dev@bigtop.apache.org" <de...@bigtop.apache.org>
> *Subject: *Oozie 5.2.1 and Zookeeper 3.4.14 compatibility issue
>
>
>
> Hi All,
>
>
>
> We recently hit an issue when using oozie 5.2.1 with zookeeper 3.4.14 that
> was built from bigtop. The curator version 2.5.0 that oozie 5.2.1 is using
> seems not working with zookeeper 3.4.14. When oozie is started, it tries to
> connect to zookeeper but is always stuck at following log:
> Connecting to ZooKeeper with SASL/Kerberos and using 'sasl' ACLs
>
>
>
> Hadoop 3.2.2 is using curator 2.13.0 version, so we tried to replace
> curator libs in /usr/lib/oozie/lib dir with curator 2.13.0 version jars.
> After the curator jars are replaced with 2.13.0 version, the oozie service
> can be started successfully.
>
>
>
> Has anyone tested Oozie 5.2.1 connection to Zookeeper 3.4.14 with Kerberos
> enabled? This looks like a bug in bigtop 3.0.0
>
>
>
> Thanks,
>
> Jason
>
>
>
Re: Oozie 5.2.1 and Zookeeper 3.4.14 compatibility issue
Posted by Evans Ye <ev...@apache.org>.
Hi Jason,
Good finding and look forward to your PR.
Evans
Jason Wen <zh...@workday.com>於 2021年10月29日 週五，下午9:46寫道：
> Hi All,
>
>
>
> We find the root cause of the issue.
>
> This issue happens when HA is enabled on oozie. oozie-5.2.1 is using
> curator-2.5.0 which depends on guava-11.0.2, but bigtop-3.0.0 apply the
> patch to change guava version to 27.0-jre in order to work with
> Hadoop-3.2.2:
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/oozie/patch0-hadoop322-hive312.diff#L103
>
> This guava version change breaks the curator-2.5.0 because it depends on
> some API that is not available anymore in guava-27.0-jre, so it will throw
> NoSuchMethod error when starting oozie.
>
> I will file a jira to bigtop and create the PR for fixing the issue.
>
>
>
> Thanks,
>
> Jason
>
>
>
> *From: *Jason Wen <zh...@workday.com>
> *Reply-To: *"user@bigtop.apache.org" <us...@bigtop.apache.org>
> *Date: *Monday, October 25, 2021 at 12:23 PM
> *To: *"user@bigtop.apache.org" <us...@bigtop.apache.org>, "
> dev@bigtop.apache.org" <de...@bigtop.apache.org>
> *Subject: *Oozie 5.2.1 and Zookeeper 3.4.14 compatibility issue
>
>
>
> Hi All,
>
>
>
> We recently hit an issue when using oozie 5.2.1 with zookeeper 3.4.14 that
> was built from bigtop. The curator version 2.5.0 that oozie 5.2.1 is using
> seems not working with zookeeper 3.4.14. When oozie is started, it tries to
> connect to zookeeper but is always stuck at following log:
> Connecting to ZooKeeper with SASL/Kerberos and using 'sasl' ACLs
>
>
>
> Hadoop 3.2.2 is using curator 2.13.0 version, so we tried to replace
> curator libs in /usr/lib/oozie/lib dir with curator 2.13.0 version jars.
> After the curator jars are replaced with 2.13.0 version, the oozie service
> can be started successfully.
>
>
>
> Has anyone tested Oozie 5.2.1 connection to Zookeeper 3.4.14 with Kerberos
> enabled? This looks like a bug in bigtop 3.0.0
>
>
>
> Thanks,
>
> Jason
>
>
>
Re: Oozie 5.2.1 and Zookeeper 3.4.14 compatibility issue
Posted by Jason Wen <zh...@workday.com.INVALID>.
Hi All,
We find the root cause of the issue.
This issue happens when HA is enabled on oozie. oozie-5.2.1 is using curator-2.5.0 which depends on guava-11.0.2, but bigtop-3.0.0 apply the patch to change guava version to 27.0-jre in order to work with Hadoop-3.2.2: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/oozie/patch0-hadoop322-hive312.diff#L103
This guava version change breaks the curator-2.5.0 because it depends on some API that is not available anymore in guava-27.0-jre, so it will throw NoSuchMethod error when starting oozie.
I will file a jira to bigtop and create the PR for fixing the issue.
Thanks,
Jason
From: Jason Wen <zh...@workday.com>
Reply-To: "user@bigtop.apache.org" <us...@bigtop.apache.org>
Date: Monday, October 25, 2021 at 12:23 PM
To: "user@bigtop.apache.org" <us...@bigtop.apache.org>, "dev@bigtop.apache.org" <de...@bigtop.apache.org>
Subject: Oozie 5.2.1 and Zookeeper 3.4.14 compatibility issue
Hi All,
We recently hit an issue when using oozie 5.2.1 with zookeeper 3.4.14 that was built from bigtop. The curator version 2.5.0 that oozie 5.2.1 is using seems not working with zookeeper 3.4.14. When oozie is started, it tries to connect to zookeeper but is always stuck at following log:
Connecting to ZooKeeper with SASL/Kerberos and using 'sasl' ACLs
Hadoop 3.2.2 is using curator 2.13.0 version, so we tried to replace curator libs in /usr/lib/oozie/lib dir with curator 2.13.0 version jars. After the curator jars are replaced with 2.13.0 version, the oozie service can be started successfully.
Has anyone tested Oozie 5.2.1 connection to Zookeeper 3.4.14 with Kerberos enabled? This looks like a bug in bigtop 3.0.0
Thanks,
Jason
Re: Oozie 5.2.1 and Zookeeper 3.4.14 compatibility issue
Posted by Jason Wen <zh...@workday.com>.
Hi All,
We find the root cause of the issue.
This issue happens when HA is enabled on oozie. oozie-5.2.1 is using curator-2.5.0 which depends on guava-11.0.2, but bigtop-3.0.0 apply the patch to change guava version to 27.0-jre in order to work with Hadoop-3.2.2: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/oozie/patch0-hadoop322-hive312.diff#L103
This guava version change breaks the curator-2.5.0 because it depends on some API that is not available anymore in guava-27.0-jre, so it will throw NoSuchMethod error when starting oozie.
I will file a jira to bigtop and create the PR for fixing the issue.
Thanks,
Jason
From: Jason Wen <zh...@workday.com>
Reply-To: "user@bigtop.apache.org" <us...@bigtop.apache.org>
Date: Monday, October 25, 2021 at 12:23 PM
To: "user@bigtop.apache.org" <us...@bigtop.apache.org>, "dev@bigtop.apache.org" <de...@bigtop.apache.org>
Subject: Oozie 5.2.1 and Zookeeper 3.4.14 compatibility issue
Hi All,
We recently hit an issue when using oozie 5.2.1 with zookeeper 3.4.14 that was built from bigtop. The curator version 2.5.0 that oozie 5.2.1 is using seems not working with zookeeper 3.4.14. When oozie is started, it tries to connect to zookeeper but is always stuck at following log:
Connecting to ZooKeeper with SASL/Kerberos and using 'sasl' ACLs
Hadoop 3.2.2 is using curator 2.13.0 version, so we tried to replace curator libs in /usr/lib/oozie/lib dir with curator 2.13.0 version jars. After the curator jars are replaced with 2.13.0 version, the oozie service can be started successfully.
Has anyone tested Oozie 5.2.1 connection to Zookeeper 3.4.14 with Kerberos enabled? This looks like a bug in bigtop 3.0.0
Thanks,
Jason