You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Kengo Seki (Jira)" <ji...@apache.org> on 2021/03/03 01:42:00 UTC
[jira] [Commented] (BIGTOP-3471) Define Bigtop 3.0 release BOM
[ https://issues.apache.org/jira/browse/BIGTOP-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17294218#comment-17294218 ]
Kengo Seki commented on BIGTOP-3471:
------------------------------------
[~telyukov] I'm going to upgrade GPDB in BIGTOP-3514. Would you test it on your environment and give us feedback if possible?
> Define Bigtop 3.0 release BOM
> -----------------------------
>
> Key: BIGTOP-3471
> URL: https://issues.apache.org/jira/browse/BIGTOP-3471
> Project: Bigtop
> Issue Type: Task
> Reporter: Kengo Seki
> Assignee: Yuqi Gu
> Priority: Major
>
> It's about time to discuss the next release BOM :)
> Based on the past discussions [1][2], we're going to focus "core" components in the next release and drop the ones (at least temporarily) which are
> * not so widely used by our users
> * not actively developed recently
> * incompatible (or not sure if compatible and no enough time/resource to investigate it) with Hadoop 3.x
> so that we can publish our releases more frequently with limited resources.
> To drop such components, I marked the ones that matches any of the above conditions as "to be removed" in the below list.
> (Tez, Phoenix and Livy seem to be inactive recently, but they significantly improve performance or usability of Hadoop/Hive, HBase and Spark respectively, so I'd like to keep them as an exception, from my biased perspective)
> So *please update the BOM if your required components are going to be removed!*
> h3. Components
> {code:java}
> Components in v1.5 in v3.0
> alluxio 1.8.2 => 2.4.1
> ambari 2.6.1 => 2.7.5
> bigtop-ambari-mpack 2.6.1 => 2.7.5
> bigtop_groovy 2.5.4 => 2.5.4
> bigtop_jsvc 1.0.15 => 1.0.15
> bigtop_tomcat 8.5.57 => 8.5.57
> bigtop_utils 1.5.0 => 3.0.0
> elasticsearch 5.6.14 => 5.6.14 (or greater, as far as its license is ASLv2-compliant)
> flink 1.6.4 => 1.11.3 [3]
> flume 1.9.0 => removed
> giraph 1.2.0 => removed
> gpdb 5.10.0 => removed
> hadoop 2.10.1 => 3.2.1 (or 3.3.x (BIGTOP-3403)? see also the discussions on BIGTOP-3280 and [4])
> hbase 1.5.0 => 2.2.6 (or 2.3.3 [5], if possible)
> hive 2.3.6 => 3.1.2 [6]
> ignite_hadoop 2.7.6 => removed
> kafka 2.4.0 => 2.4.1
> kibana 5.4.1 => 5.4.1 (or greater, as far as its license is ASLv2-compliant)
> livy 0.7.0 => 0.7.0
> logstash 5.4.1 => 5.4.1 (or greater, as far as its license is ASLv2-compliant)
> mahout 0.13.0 => removed
> oozie 4.3.0 => 5.2.0
> phoenix 4.15.0-HBase-1.5 => 5.1.0 [7]
> qfs 2.0.0 => removed
> solr 6.6.6 => 8.7.0 (or maybe 8.8.0?)
> spark 2.4.5 => 3.0.1
> sqoop 1.4.6 => 1.4.7
> sqoop2 1.99.4 => removed
> tez 0.9.2 => 0.10.0
> ycsb 0.12.0 => 0.17.0
> zeppelin 0.8.2 => 0.9.0 [8]
> zookeeper 3.4.13 => 3.4.14 (or 3.5+ [9] if possible)
> {code}
> h3. Distros
> {code:java}
> - CentOS 7, 8
> - Debian 10, 11 (if officially released before this release)
> - Fedora 31
> - Ubuntu 16.04, 18.04, 20.04
> {code}
> Some points of discussion:
> * Should we still support CentOS 8 which is supposed to be EOL'd within 2021 [10]?
> * Should we update Fedora to 33, the latest version as of Jan 2021?
> * Should we also support Ubuntu 20.04 additionally?
> h3. Archs
> {code:java}
> - x86_64
> - aarch64
> - ppc64le (if the CI server is recovered)
> {code}
> *JDK*
> {code:bash}
> JDK8
> {code}
> Also, I tentatively assigned 3.0 as its version for the following reasons, but let's discuss it too if someone has another option.
> * We're going to adopt Hadoop 3.x (and hopefully Spark 3.x), so Bigtop 3.0 is aligned with them [11].
> * Bigtop 2.x is (unofficially) reserved for the "Cloud Native Bigtop" (BIGTOP-3225) release line [12].
> * In my opinion, this major version leap is good to impress users backward incompatibility described above.
> [1]: [https://lists.apache.org/thread.html/fe54be35856c8f6a23fb7d35b979e3dc90ad58baf94d15a206cdaf63%40%3Cdev.bigtop.apache.org%3E]
> [2]: [https://lists.apache.org/thread.html/r71869111448bf5f8bc82e7031ff25d5455ab957e8e536d000026fe16%40%3Cdev.bigtop.apache.org%3E]
> [3]: The latest release which is officially supported by Zeppelin 0.9.0. [https://twitter.com/ApacheZeppelin/status/1288855217895698433]
> [4]: [https://github.com/apache/bigtop/pull/704]
> [5]: The latest release which is officially compatible with Hadoop 3.2.x as of now. [https://hbase.apache.org/book.html#hadoop]
> [6]: The latest release which is officially compatible with Hadoop 3.x.y as of now. [https://hive.apache.org/downloads.html#26-august-2019-release-312-available]
> [7]: The latest release which is officially compatible with HBase 2.x as of now. [https://phoenix.apache.org/download.html]
> [8]: The latest release which officially supports Spark 3.0 as of now. [https://twitter.com/ApacheZeppelin/status/1288855217895698433]
> [9]: HBase 2.3.3 is built with Zookeeper 3.5.7 by default and doesn't seem to be built with Zookeeper 3.4.x as it is.
> [10]: [https://blog.centos.org/2020/12/future-is-centos-stream/]
> [11]: https://issues.apache.org/jira/browse/BIGTOP-3287?focusedCommentId=17018665&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17018665
> [12]: [https://lists.apache.org/thread.html/e0af97d83c33bf370e25fca61aee95d0ed63193893c5f0ff7d187ebb%40%3Cdev.bigtop.apache.org%3E]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)