You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "Kengo Seki (Jira)" <ji...@apache.org> on 2021/01/05 00:28:00 UTC
[jira] [Created] (BIGTOP-3471) Define Bigtop 3.0 release BOM
Kengo Seki created BIGTOP-3471:
----------------------------------
Summary: Define Bigtop 3.0 release BOM
Key: BIGTOP-3471
URL: https://issues.apache.org/jira/browse/BIGTOP-3471
Project: Bigtop
Issue Type: Task
Reporter: Kengo Seki
It's about time to discuss the next release BOM :)
Based on the past discussions [1][2], we're going to focus "core" components in the next release and drop the ones (at least temporarily) which are
* not so widely used by our users
* not actively developed recently
* incompatible (or not sure if compatible and no enough time/resource to investigate it) with Hadoop 3.x
so that we can publish our releases more frequently with limited resources.
To drop such components, I marked the ones that matches any of the above conditions as "to be removed" in the below list.
(Tez, Phoenix and Livy seem to be inactive recently, but they significantly improve performance or usability of Hadoop/Hive, HBase and Spark respectively, so I'd like to keep them as an exception, from my biased perspective)
So *please update the BOM if your required components are going to be removed!*
h3. Components
{code}
Components in v1.5 in v3.0
alluxio 1.8.2 => removed
ambari 2.6.1 => 2.7.5
bigtop-ambari-mpack 2.6.1 => 2.7.5
bigtop_groovy 2.5.4 => 2.5.4
bigtop_jsvc 1.0.15 => 1.0.15
bigtop_tomcat 8.5.57 => 8.5.57
bigtop_utils 1.5.0 => 3.0.0
elasticsearch 5.6.14 => removed
flink 1.6.4 => 1.11.3 [3]
flume 1.9.0 => removed
giraph 1.2.0 => removed
gpdb 5.10.0 => removed
hadoop 2.10.1 => 3.2.1 (or 3.3.x (BIGTOP-3403)? see also the discussions on BIGTOP-3280 and [4])
hbase 1.5.0 => 2.3.3 [5]
hive 2.3.6 => 3.1.2 [6]
ignite_hadoop 2.7.6 => removed
kafka 2.4.0 => 2.7.0
kibana 5.4.1 => removed
livy 0.7.0 => 0.7.0
logstash 5.4.1 => removed
mahout 0.13.0 => removed
oozie 4.3.0 => removed
phoenix 4.15.0-HBase-1.5 => 5.0.0-HBase-2.0 [7]
qfs 2.0.0 => removed
solr 6.6.6 => removed
spark 2.4.5 => 3.0.1
sqoop 1.4.6 => removed
sqoop2 1.99.4 => removed
tez 0.9.2 => 0.9.2
ycsb 0.12.0 => removed
zeppelin 0.8.2 => 0.9.0 [8]
zookeeper 3.4.13 => 3.5.7 (or newer) [9]
{code}
h3. Distros
{code}
- CentOS 7, 8
- Debian 9, 10
- Fedora 31
- Ubuntu 16.04, 18.04
{code}
Some points of discussion:
* Should we still support CentOS 8 which is supposed to be EOL'd within 2021 [10]?
* Should we update Fedora to 33, the latest version as of Jan 2021?
* Should we also support Ubuntu 20.04 additionally?
h3. Archs
{code}
- x86_64
- aarch64
- ppc64le (if the CI server is recovered)
{code}
Also, I tentatively assigned 3.0 as its version for the following reasons, but let's discuss it too if someone has another option.
* We're going to adopt Hadoop 3.x (and hopefully Spark 3.x), so Bigtop 3.0 is aligned with them [11].
* Bigtop 2.x is (unofficially) reserved for the "Cloud Native Bigtop" (BIGTOP-3225) release line [12].
* In my opinion, this major version leap is good to impress users backward incompatibility described above.
[1]: [https://lists.apache.org/thread.html/fe54be35856c8f6a23fb7d35b979e3dc90ad58baf94d15a206cdaf63%40%3Cdev.bigtop.apache.org%3E]
[2]: [https://lists.apache.org/thread.html/r71869111448bf5f8bc82e7031ff25d5455ab957e8e536d000026fe16%40%3Cdev.bigtop.apache.org%3E]
[3]: The latest release which is officially supported by Zeppelin 0.9.0. [https://twitter.com/ApacheZeppelin/status/1288855217895698433]
[4]: [https://github.com/apache/bigtop/pull/704]
[5]: The latest release which is officially compatible with Hadoop 3.2.x as of now. [https://hbase.apache.org/book.html#hadoop]
[6]: The latest release which is officially compatible with Hadoop 3.x.y as of now. [https://hive.apache.org/downloads.html#26-august-2019-release-312-available]
[7]: The latest release which is officially compatible with HBase 2.x as of now. [https://phoenix.apache.org/download.html]
[8]: The latest release which officially supports Spark 3.0 as of now. [https://twitter.com/ApacheZeppelin/status/1288855217895698433]
[9]: HBase 2.3.3 is built with Zookeeper 3.5.7 by default and doesn't seem to be built with Zookeeper 3.4.x as it is.
[10]: [https://blog.centos.org/2020/12/future-is-centos-stream/]
[11]: https://issues.apache.org/jira/browse/BIGTOP-3287?focusedCommentId=17018665&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17018665
[12]: [https://lists.apache.org/thread.html/e0af97d83c33bf370e25fca61aee95d0ed63193893c5f0ff7d187ebb%40%3Cdev.bigtop.apache.org%3E]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)