You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Masatake Iwasaki (Jira)" <ji...@apache.org> on 2022/09/26 04:20:00 UTC
[jira] [Updated] (BIGTOP-3695) Define Bigtop 3.2 release BOM
[ https://issues.apache.org/jira/browse/BIGTOP-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Masatake Iwasaki updated BIGTOP-3695:
-------------------------------------
Description:
Let's discuss the component stack of the next release.
I'd like to propose upgrading Hadoop to 3.3.x since we have put it off long time.
I'd also like to add Ranger to our stack, which is often required in the projects with high security requirements.
h3. Components
{noformat}
Components in v3.1 in v3.2
alluxio 2.4.1 => 2.8.0 [1]
bigtop-ambari-mpack 2.7.5 2.7.5
bigtop-groovy 2.5.4 2.5.4
bigtop-jsvc 1.2.4 1.2.4
bigtop-utils 3.1.0 3.1.0
elasticsearch 5.6.14 => dropped
flink 1.11.6 => 1.15.0
gpdb 5.28.5 5.28.5
hadoop 3.2.3 => 3.3.4
hbase 2.4.11 => 2.4.13 [2]
hive 3.1.2 => 3.1.3 [3]
kafka 2.8.1 2.8.1
kibana 5.4.1 => dropped
livy 0.7.1 0.7.1
logstash 5.4.1 => dropped
oozie 5.2.1 5.2.1
phoenix 5.1.2 5.1.2
ranger n/a => 2.2.0 [4]
solr 8.11.1 => 8.11.2
spark 3.1.2 => 3.2.1 [5]
sqoop 1.4.7 dropped
tez 0.10.1 0.10.1
ycsb 0.17.0 0.17.0
zeppelin 0.10.0 => 0.10.1 [6]
zookeeper 3.5.9 3.5.9
{noformat}
h3. Distros
{noformat}
- CentOS 7, Rocky Linux 8
- Debian 10, 11
- Fedora 35
- Ubuntu 18.04, 20.04
{noformat}
h3. Archs
{noformat}
- x86_64
- aarch64
- ppc64le
{noformat}
h3. JDK
{noformat}
JDK8
{noformat}
[1]: According to the following document, Alluxio 2.8.0 is supposed to be compatible with HDFS 3.3.x.
[https://docs.alluxio.io/os/user/2.8.0/en/ufs/HDFS.html#supported-hdfs-versions]
[2]: According to the following document, HBase 2.4.12 is supposed to be compatible with Hadoop 3.3.x.
[https://hbase.apache.org/book.html#hadoop]
[3]: According to the following document, Hive 3.1.3 is supposed to be compatible with Hadoop 3.x.y.
[https://hive.apache.org/downloads.html]
[4]: Ranger 2.2.0 is built with Hadoop 3.3.0 by default, so we can expect it's compatible with Hadoop 3.3.3.
[https://github.com/apache/ranger/blob/release-ranger-2.2.0/pom.xml#L121]
[5]: We can expect Spark 3.2.1 is compatible with Hadoop 3.3.x, since it's built for Hadoop 3.3.0 in the following example:
[https://spark.apache.org/docs/3.2.1/building-spark.html#specifying-the-hadoop-version-and-enabling-yarn]
[6]: Zeppelin 0.10.1, which is the latest release as of now, already has the "spark-3.2" build profile ([https://github.com/apache/zeppelin/blob/v0.10.1/spark/pom.xml#L172]), but doesn't the Flink shim for 1.15.x yet ([https://github.com/apache/zeppelin/tree/v0.10.1/flink]).
Probably we have to apply ZEPPELIN-5600.
was:
Let's discuss the component stack of the next release.
I'd like to propose upgrading Hadoop to 3.3.x since we have put it off long time.
I'd also like to add Ranger to our stack, which is often required in the projects with high security requirements.
h3. Components
{noformat}
Components in v3.1 in v3.2
alluxio 2.4.1 => 2.8.0 [1]
bigtop-ambari-mpack 2.7.5 2.7.5
bigtop-groovy 2.5.4 2.5.4
bigtop-jsvc 1.2.4 1.2.4
bigtop-utils 3.1.0 3.1.0
elasticsearch 5.6.14 => dropped
flink 1.11.6 => 1.15.0
gpdb 5.28.5 5.28.5
hadoop 3.2.3 => 3.3.4
hbase 2.4.11 => 2.4.13 [2]
hive 3.1.2 => 3.1.3 [3]
kafka 2.8.1 2.8.1
kibana 5.4.1 => dropped
livy 0.7.1 0.7.1
logstash 5.4.1 => dropped
oozie 5.2.1 5.2.1
phoenix 5.1.2 5.1.2
ranger n/a => 2.2.0 [4]
solr 8.11.1 => 8.11.2
spark 3.1.2 => 3.2.1 [5]
sqoop 1.4.7 1.4.7
tez 0.10.1 0.10.1
ycsb 0.17.0 0.17.0
zeppelin 0.10.0 => 0.10.1 [6]
zookeeper 3.5.9 3.5.9
{noformat}
h3. Distros
{noformat}
- CentOS 7, Rocky Linux 8
- Debian 10, 11
- Fedora 35
- Ubuntu 18.04, 20.04
{noformat}
h3. Archs
{noformat}
- x86_64
- aarch64
- ppc64le
{noformat}
h3. JDK
{noformat}
JDK8
{noformat}
[1]: According to the following document, Alluxio 2.8.0 is supposed to be compatible with HDFS 3.3.x.
[https://docs.alluxio.io/os/user/2.8.0/en/ufs/HDFS.html#supported-hdfs-versions]
[2]: According to the following document, HBase 2.4.12 is supposed to be compatible with Hadoop 3.3.x.
[https://hbase.apache.org/book.html#hadoop]
[3]: According to the following document, Hive 3.1.3 is supposed to be compatible with Hadoop 3.x.y.
[https://hive.apache.org/downloads.html]
[4]: Ranger 2.2.0 is built with Hadoop 3.3.0 by default, so we can expect it's compatible with Hadoop 3.3.3.
[https://github.com/apache/ranger/blob/release-ranger-2.2.0/pom.xml#L121]
[5]: We can expect Spark 3.2.1 is compatible with Hadoop 3.3.x, since it's built for Hadoop 3.3.0 in the following example:
[https://spark.apache.org/docs/3.2.1/building-spark.html#specifying-the-hadoop-version-and-enabling-yarn]
[6]: Zeppelin 0.10.1, which is the latest release as of now, already has the "spark-3.2" build profile ([https://github.com/apache/zeppelin/blob/v0.10.1/spark/pom.xml#L172]), but doesn't the Flink shim for 1.15.x yet ([https://github.com/apache/zeppelin/tree/v0.10.1/flink]).
Probably we have to apply ZEPPELIN-5600.
> Define Bigtop 3.2 release BOM
> -----------------------------
>
> Key: BIGTOP-3695
> URL: https://issues.apache.org/jira/browse/BIGTOP-3695
> Project: Bigtop
> Issue Type: Task
> Reporter: Kengo Seki
> Assignee: Yuqi Gu
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Let's discuss the component stack of the next release.
> I'd like to propose upgrading Hadoop to 3.3.x since we have put it off long time.
> I'd also like to add Ranger to our stack, which is often required in the projects with high security requirements.
> h3. Components
> {noformat}
> Components in v3.1 in v3.2
> alluxio 2.4.1 => 2.8.0 [1]
> bigtop-ambari-mpack 2.7.5 2.7.5
> bigtop-groovy 2.5.4 2.5.4
> bigtop-jsvc 1.2.4 1.2.4
> bigtop-utils 3.1.0 3.1.0
> elasticsearch 5.6.14 => dropped
> flink 1.11.6 => 1.15.0
> gpdb 5.28.5 5.28.5
> hadoop 3.2.3 => 3.3.4
> hbase 2.4.11 => 2.4.13 [2]
> hive 3.1.2 => 3.1.3 [3]
> kafka 2.8.1 2.8.1
> kibana 5.4.1 => dropped
> livy 0.7.1 0.7.1
> logstash 5.4.1 => dropped
> oozie 5.2.1 5.2.1
> phoenix 5.1.2 5.1.2
> ranger n/a => 2.2.0 [4]
> solr 8.11.1 => 8.11.2
> spark 3.1.2 => 3.2.1 [5]
> sqoop 1.4.7 dropped
> tez 0.10.1 0.10.1
> ycsb 0.17.0 0.17.0
> zeppelin 0.10.0 => 0.10.1 [6]
> zookeeper 3.5.9 3.5.9
> {noformat}
> h3. Distros
> {noformat}
> - CentOS 7, Rocky Linux 8
> - Debian 10, 11
> - Fedora 35
> - Ubuntu 18.04, 20.04
> {noformat}
> h3. Archs
> {noformat}
> - x86_64
> - aarch64
> - ppc64le
> {noformat}
> h3. JDK
> {noformat}
> JDK8
> {noformat}
> [1]: According to the following document, Alluxio 2.8.0 is supposed to be compatible with HDFS 3.3.x.
> [https://docs.alluxio.io/os/user/2.8.0/en/ufs/HDFS.html#supported-hdfs-versions]
> [2]: According to the following document, HBase 2.4.12 is supposed to be compatible with Hadoop 3.3.x.
> [https://hbase.apache.org/book.html#hadoop]
> [3]: According to the following document, Hive 3.1.3 is supposed to be compatible with Hadoop 3.x.y.
> [https://hive.apache.org/downloads.html]
> [4]: Ranger 2.2.0 is built with Hadoop 3.3.0 by default, so we can expect it's compatible with Hadoop 3.3.3.
> [https://github.com/apache/ranger/blob/release-ranger-2.2.0/pom.xml#L121]
> [5]: We can expect Spark 3.2.1 is compatible with Hadoop 3.3.x, since it's built for Hadoop 3.3.0 in the following example:
> [https://spark.apache.org/docs/3.2.1/building-spark.html#specifying-the-hadoop-version-and-enabling-yarn]
> [6]: Zeppelin 0.10.1, which is the latest release as of now, already has the "spark-3.2" build profile ([https://github.com/apache/zeppelin/blob/v0.10.1/spark/pom.xml#L172]), but doesn't the Flink shim for 1.15.x yet ([https://github.com/apache/zeppelin/tree/v0.10.1/flink]).
> Probably we have to apply ZEPPELIN-5600.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)