[jira] [Created] (BIGTOP-3695) Define Bigtop 3.2 release BOM

["Kengo Seki (Jira)" <ji...@apache.org>]

Kengo Seki created BIGTOP-3695:
----------------------------------

             Summary: Define Bigtop 3.2 release BOM
                 Key: BIGTOP-3695
                 URL: https://issues.apache.org/jira/browse/BIGTOP-3695
             Project: Bigtop
          Issue Type: Task
            Reporter: Kengo Seki


Let's discuss the component stack of the next release.
I'd like to propose upgrading Hadoop to 3.3.x since we have put it off long time.
I'd also like to add Ranger to our stack, which is often required in the projects with high security requirements.

h3. Components

{noformat}
Components                    in v3.1    in v3.2
alluxio                        2.4.1  =>  2.8.0 [1]
ambari                         2.7.5  =>  2.7.6
bigtop-ambari-mpack            2.7.5      2.7.5
bigtop-groovy                  2.5.4      2.5.4
bigtop-jsvc                    1.2.4      1.2.4
bigtop-utils                   3.1.0      3.1.0
elasticsearch                 5.6.14     5.6.14
flink                         1.11.6  => 1.15.0
gpdb                          5.28.5     5.28.5
hadoop                         3.2.3  =>  3.3.3
hbase                         2.4.11  => 2.4.12 [2]
hive                           3.1.2  =>  3.1.3 [3]
kafka                          2.8.1      2.8.1
kibana                         5.4.1      5.4.1
livy                           0.7.1      0.7.1
logstash                       5.4.1      5.4.1
oozie                          5.2.1      5.2.1
phoenix                        5.1.2      5.1.2
ranger                           -    =>  2.2.0 or greater [4]
solr                          8.11.1     8.11.1
spark                          3.1.2  =>  3.2.1 [5]
sqoop                          1.4.7      1.4.7
tez                           0.10.1     0.10.1
ycsb                          0.17.0     0.17.0
zeppelin                      0.10.0  => 0.10.1 [6]
zookeeper                      3.5.9      3.5.9
{noformat}

h3. Distros

{noformat}
- CentOS 7, Rocky Linux 8
- Debian 10, 11
- Fedora 35
- Ubuntu 18.04, 20.04
{noformat}

h3. Archs

{noformat}
- x86_64
- aarch64
- ppc64le
{noformat}

h3. JDK

{noformat}
JDK8
{noformat}

[1]: According to the following document, Alluxio 2.8.0 is supposed to be compatible with HDFS 3.3.x.
     https://docs.alluxio.io/os/user/2.8.0/en/ufs/HDFS.html#supported-hdfs-versions

[2]: According to the following document, HBase 2.4.12 is supposed to be compatible with Hadoop 3.3.x.
     https://hbase.apache.org/book.html#hadoop

[3]: According to the following document, Hive 3.1.3 is supposed to be compatible with Hadoop 3.x.y.
     https://hive.apache.org/downloads.html

[4]: Ranger 2.2.0, which is the latest release as of now, doesn't support Hadoop 3.3.x yet.
     So we probably have to apply RANGER-3047 with some modification or wait for the 2.3.0 release.

[5]: We can expect Spark 3.2.1 is compatible with Hadoop 3.3.x, since it's built for Hadoop 3.3.0 in the following example:
     https://spark.apache.org/docs/3.2.1/building-spark.html#specifying-the-hadoop-version-and-enabling-yarn

[6]: Zeppelin 0.10.1, which is the latest release as of now, already has the "spark-3.2" build profile (https://github.com/apache/zeppelin/blob/v0.10.1/spark/pom.xml#L172), but doesn't the Flink shim for 1.15.x yet (https://github.com/apache/zeppelin/tree/v0.10.1/flink).
     Probably we have to apply ZEPPELIN-5600.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)