You are viewing a plain text version of this content. The canonical link for it is here.

Posted to general@incubator.apache.org by Dave Fisher <da...@comcast.net> on 2018/07/05 19:22:19 UTC

[VOTE] Accept Doris into the Apache Incubator

Hi All,

I would like to start a VOTE to bring the Doris project as an Apache incubator podling.

The ASF voting rules are described:

https://www.apache.org/foundation/voting.html <https://www.apache.org/foundation/voting.html>

A vote for accepting a new Apache Incubator podling is a majority vote for which only Incubator PMC member votes are binding.

This vote will run for at least 72 hours. Please VOTE as follows
[] +1 Accept Doris into the Apache Incubator
[] +0 Abstain.
[] -1 Do not accept Doris into the Apache Incubator because ...

The proposal is listed below, but you can also access it on the wiki:

https://wiki.apache.org/incubator/DorisProposal

Best regards,
Dave

= Apache Doris =

== Abstract ==

Doris is a MPP-based interactive SQL data warehousing for reporting and analysis.

== Proposal ==

We propose to contribute the Doris codebase and associated artifacts (e.g. documentation, web-site content etc.) to the Apache Software Foundation, and aim to build an open community around Doris’s continued development in the ‘Apache Way’.

=== Overview of Doris ===

Doris’s implementation consists of two daemons: Frontend (FE) and Backend (BE).

**Frontend daemon** consists of query coordinator and catalog manager. Query coordinator is responsible for receiving users’ sql queries, compiling queries and managing queries execution. Catalog manager is responsible for managing metadata such as databases, tables, partitions, replicas and etc. Several frontend daemons could be deployed to guarantee fault-tolerance, and load balancing.

**Backend daemon** stores the data and executes the query fragments. Many backend daemons could also be deployed to provide scalability and fault-tolerance.

A typical Doris cluster generally composes of several frontend daemons and dozens to hundreds of backend daemons.

Users can use MySQL client tools to connect any frontend daemon to submit SQL query. Frontend receives the query and compiles it into query plans executable by the Backend. Then Frontend sends the query plan fragments to Backend. Backend will build a query execution DAG. Data is fetched and pipelined into the DAG. The final result response is sent to client via Frontend. The distribution of query fragment execution takes minimizing data movement and maximizing scan locality as the main goal.

== Background ==

At Baidu, Prior to Doris, different tools were deployed to solve diverse requirements in many ways. And when a use case requires the simultaneous availability of capabilities that cannot all be provided by a single tool, users were forced to build hybrid architectures that stitch multiple tools together, but we believe that they shouldn’t need to accept such inherent complexity. A storage system built to provide great performance across a broad range of workloads provides a more elegant solution to the problems that hybrid architectures aim to solve. Doris is the solution.

Doris is designed to be a simple and single tightly coupled system, not depending on other systems. Doris provides high concurrent low latency point query performance, but also provides high throughput queries of ad-hoc analysis. Doris provides bulk-batch data loading, but also provides near real-time mini-batch data loading. Doris also provides high availability, reliability, fault tolerance, and scalability.

== Rationale ==

Doris mainly integrates the technology of Google Mesa and Apache Impala.

Mesa is a highly scalable analytic data storage system that stores critical measurement data related to Google's Internet advertising business. Mesa is designed to satisfy complex and challenging set of users’ and systems’ requirements, including near real-time data ingestion and query ability, as well as high availability, reliability, fault tolerance, and scalability for large data and query volumes.

Impala is a modern, open-source MPP SQL engine architected from the ground up for the Hadoop data processing environment. At present, by virtue of its superior performance and rich functionality， Impala has been comparable to many commercial MPP database query engine. Mesa can satisfy the needs of many of our storage requirements, however Mesa itself does not provide a SQL query engine; Impala is a very good MPP SQL query engine, but the lack of a perfect distributed storage engine. So in the end we chose the combination of these two technologies.

Learning from Mesa’s data model, we developed a distributed storage engine. Unlike Mesa, this storage engine does not rely on any distributed file system. Then we deeply integrate this storage engine with Impala query engine. Query compiling, query execution coordination and catalog management of storage engine are integrated to be frontend daemon; query execution and data storage are integrated to be backend daemon. With this integration, we implemented a single, full-featured, high performance state the art of MPP database, as well as maintaining the simplicity.

== Current Status ==

Doris has been an open source project on GitHub (https://github.com/baidu/palo).

=== Meritocracy ===

Doris has been deployed in production at Baidu and is applying more than 200 lines of business. It has demonstrated great performance benefits and has proved to be a better way for reporting and analysis based big data. Still We look forward to growing a rich user and developer community.

=== Community ===

Doris seeks to develop developer and user communities during incubation.

Doris makes use of Apache Impala. It was identified during early review of the proposal that the Doris community will need to work with Impala to define a suitable API.

=== Core Developers ===

 * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
 * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
 * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
 * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
 * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
 * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
 * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)

=== Alignment ===

Doris is related to several other Apache projects:

 * Doris can also read data stored in Apache Hadoop clusters powered by the HDFS filesystem.
 * Doris is closely integrated with Impala, which has graduated from Apache Incubator.
 * Doris uses Apache Thrift as its RPC and serialization framework of choice.

== Known Risks ==

=== Orphaned Products ===

The core developers of Doris team plan to work full time on this project. There is very little risk of Doris getting orphaned since at least one large company (Baidu) is extensively using it in their production. For example, currently there are more than 200 use cases using Doris in production. Furthermore, since Doris was open sourced at the beginning of October 2017, it has received more than 660 stars and been forked nearly 170 times. We plan to extend and diversify this community further through Apache.

=== Inexperience with Open Source ===

The core developers are all active users and followers of open source. They are already committers and contributors to the Doris Github project. All have been involved with the source code that has been released under an open source license, and several of them also have experience developing code in an open source environment. Though the core set of Developers do not have Apache Open Source experience, there are plans to onboard individuals with Apache open source experience on to the project.

=== Homogenous Developers ===

The most of core developers are from Baidu, but after Doris was open sourced, Doris received a lot of bug fixes and enhancements from other developers not working at Baidu.

=== Reliance on Salaried Developers ===

Baidu invested in Doris as the OLAP solution and some of its key engineers are working full time on the project. In addition, since there is a growing Big Data need for scalable OLAP solutions, we look forward to other Apache developers and researchers to contribute to the project. Also key to addressing the risk associated with relying on Salaried developers from a single entity is to increase the diversity of the contributors and actively lobby for Domain experts in the BI space to contribute. Apache Doris intends to do this.

=== An Excessive Fascination with the Apache Brand ===

Doris is proposing to enter incubation at Apache in order to help efforts to diversify the committer-base, not so much to capitalize on the Apache brand. The Doris project is in production use already inside Baidu, but is not expected to be an Baidu product for external customers. As such, the Doris project is not seeking to use the Apache brand as a marketing tool.

== Documentation ==

Information about Doris can be found at https://github.com/baidu/palo. The following links provide more information about Doris in open source:

 * Doris wiki site: https://github.com/baidu/palo/wiki
 * Codebase at Github: https://github.com/baidu/palo
 * Issue Tracking: https://github.com/baidu/palo/issues
 * Overview: https://github.com/baidu/Doris/wiki/palo-Overview
 * FAQ: https://github.com/baidu/palo/wiki/palo-FAQ

== Initial Source ==

Doris has been under development since 2017 by a team of engineers at Baidu Inc. It is currently hosted on Github.com under an Apache license at https://github.com/baidu/palo.

== External Dependencies ==

Doris has the following external dependencies.

 * Google gflags (BSD)
 * Google glog (BSD)
 * Apache Thrift (Apache Software License v2.0)
 * Apache Commons (Apache Software License v2.0)
 * Boost (Boost Software License)
 * rapidjson (Tencent)
 * Google RE2 (BSD-style)
 * lz4 (BSD)
 * snappy (BSD)
 * Twitter Bootstrap (Apache Software License v2.0)
 * d3 (BSD)
 * LLVM (BSD-like)

Build and test dependencies:

 * Apache Ant (Apache Software License v2.0)
 * Apache Maven (Apache Software License v2.0)
 * cmake (BSD)
 * clang (BSD)
 * Google gtest (Apache Software License v2.0)

== Required Resources ==

=== Mailing List ===

There are currently no mailing lists. The usual mailing lists are expected to be set up when entering incubation:

 * private@doris.incubator.apache.org
 * dev@doris.incubator.apache.org
 * commits@doris.incubator.apache.org

=== Subversion Directory ===

Upon entering incubation, we want to move (or copy) the existing repo from https://github.com/baidu/palo to Apache infrastructure at https://github.com/apache/incubator-doris.

=== Issue Tracking ===

Doris currently uses GitHub to track issues. Would like to continue to do so while we discuss migration possibilities with the ASF Infra committee.

=== Other Resources ===

The existing code already has unit tests so we will make use of existing Apache continuous testing infrastructure. The resulting load should not be very large.

== Initial Committers ==

 * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
 * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
 * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
 * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
 * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
 * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
 * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
 * Sijie Guo (guosijie@gmail dot com)
 * Zheng Shao (zshao@apache.org)

== Affiliations ==

The initial committers are employees of Baidu Inc..

== Sponsors ==

=== Champion ===

 * Dave Fisher, wave@apache.org

=== Nominated Mentors ===

 * Luke Han, lukehan@apache.org
 * Dave Fisher, wave@apache.org
 * Willem Jiang, ningjiang@apache.org

=== Sponsoring Entity ===

We are requesting the Incubator to sponsor this project.

Re: [RESULTS][VOTE] Accept Doris into the Apache Incubator

Posted by "Tan,Zhongyi" <ta...@baidu.com>.

thanks


在 2018年7月12日，15:18，Dave Fisher <da...@comcast.net>> 写道：

Hi -

Welcome Doris to the Apache Incubator!

The vote passes with 8 +1 (binding votes) and 6 +1 (non-binding votes)

Binding votes:
Dave Fisher
Bertrand Delacretaz
Timothy Chen
Matt Sicker
Willem Jiang
Jean-Baptiste Onofré
Kevin A. McGrail
Han Luke

Non-binding votes:
Sijie Guo
Zhongyi Tan
Charith Elvitigala
Xin Wang
吴晟 Sheng Wu
Jianru Tong

The VOTE thread archive.
https://lists.apache.org/thread.html/916a2249a78bd78b77ded01d42fe932db1b8c3a3c795038128e58ced@%3Cgeneral.incubator.apache.org%3E

Regards,
Dave

On Jul 11, 2018, at 4:31 PM, Han Luke <lu...@gmail.com>> wrote:

+1 binding

________________________________
From: Timothy Chen <tn...@gmail.com>>
Sent: Friday, July 6, 2018 5:19:43 PM
To: general@incubator.apache.org<ma...@incubator.apache.org>
Subject: Re: [VOTE] Accept Doris into the Apache Incubator

+1 (binding)

Been following the discussions and knowing the mentors I believe the team
can continue to improve and learn how the Apache way works.

Tim
On Fri, Jul 6, 2018 at 5:15 PM Bertrand Delacretaz <
bdelacretaz@codeconsult.ch<ma...@codeconsult.ch>> wrote:

On Fri, Jul 6, 2018 at 10:11 AM Li,De(BDG) <li...@baidu.com>> wrote:
...None of them are affiliated with Baidu Inc., just initial committers
are...

Ok, thanks for clarifying!

Here's my +1

-Bertrand

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org<ma...@incubator.apache.org>
For additional commands, e-mail: general-help@incubator.apache.org<ma...@incubator.apache.org>

Re: [RESULTS][VOTE] Accept Doris into the Apache Incubator

Posted by Dave Fisher <da...@comcast.net>.

Hi -

Some of the initial steps are completed and I submitted INFRA-16786 <https://issues.apache.org/jira/browse/INFRA-16786> Setup DNS, LDAP for the new podling Apache Doris

We are currently in this step: https://incubator.apache.org/guides/mentor.html#resources

Once Infra completes the JIRA I will request the mailing lists. Once those exist I will send out information to all of the initial committers and mentors.

Regards,
Dave


> On Jul 17, 2018, at 9:45 PM, Dave Fisher <da...@comcast.net> wrote:
> 
> Hi Li, De,
> 
> I have a couple of administrative steps to do including sending the initial committers a list.
> 
> I’ll do this during my morning in 12 hours. If Luke or Willem wish they can start. Tasks are here: https://incubator.apache.org/policy/incubation.html#setting_up_a_new_podling
> 
> The fourth bullet is special since this will be a github transfer.
> 
> Please review ICLA, but the process is easier if you wait to submit them to the secretary until the mentors complete initial setup.
> 
> Regards,
> Dave
> 
> Sent from my iPhone
> 
>> On Jul 17, 2018, at 7:42 PM, Li,De(BDG) <li...@baidu.com> wrote:
>> 
>> Hi Dave,
>> 
>> What should we do next? Can you give me a to do list or something?
>> 
>> Best Regards,
>> Reed
>> 
>> 
>> On 2018/7/13 上午10:07， "Li,De(BDG)" <li...@baidu.com>> wrote:
>> 
>> That’s great, thank you, Dave, thank everyone for your support.
>> 
>> On 2018/7/13 上午6:17， "Dave Fisher" <da...@comcast.net>> wrote:
>> 
>> Hi -
>> 
>> Welcome Doris to the Apache Incubator!
>> 
>> The vote passes with 8 +1 (binding votes) and 6 +1 (non-binding votes)
>> 
>> Binding votes:
>> Dave Fisher
>> Bertrand Delacretaz
>> Timothy Chen
>> Matt Sicker
>> Willem Jiang
>> Jean-Baptiste Onofré
>> Kevin A. McGrail
>> Han Luke
>> 
>> Non-binding votes:
>> Sijie Guo
>> Zhongyi Tan
>> Charith Elvitigala
>> Xin Wang
>> 吴晟 Sheng Wu
>> Jianru Tong
>> 
>> The VOTE thread archive.
>> https://lists.apache.org/thread.html/916a2249a78bd78b77ded01d42fe932db1b8c3a3c795038128e58ced@%3Cgeneral.incubator.apache.org%3E
>> 
>> Regards,
>> Dave
>> 
>> On Jul 11, 2018, at 4:31 PM, Han Luke <lu...@gmail.com>> wrote:
>> 
>> +1 binding
>> 
>> ________________________________
>> From: Timothy Chen <tn...@gmail.com>>
>> Sent: Friday, July 6, 2018 5:19:43 PM
>> To: general@incubator.apache.org<ma...@incubator.apache.org>
>> Subject: Re: [VOTE] Accept Doris into the Apache Incubator
>> 
>> +1 (binding)
>> 
>> Been following the discussions and knowing the mentors I believe the team
>> can continue to improve and learn how the Apache way works.
>> 
>> Tim
>> On Fri, Jul 6, 2018 at 5:15 PM Bertrand Delacretaz <
>> bdelacretaz@codeconsult.ch<ma...@codeconsult.ch>> wrote:
>> 
>> On Fri, Jul 6, 2018 at 10:11 AM Li,De(BDG) <li...@baidu.com>> wrote:
>> ...None of them are affiliated with Baidu Inc., just initial committers
>> are...
>> 
>> Ok, thanks for clarifying!
>> 
>> Here's my +1
>> 
>> -Bertrand
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org<ma...@incubator.apache.org>
>> For additional commands, e-mail: general-help@incubator.apache.org<ma...@incubator.apache.org>
>> 
>> 
>>

Re: [RESULTS][VOTE] Accept Doris into the Apache Incubator

Posted by Dave Fisher <da...@comcast.net>.

Hi Li, De,

I have a couple of administrative steps to do including sending the initial committers a list.

I’ll do this during my morning in 12 hours. If Luke or Willem wish they can start. Tasks are here: https://incubator.apache.org/policy/incubation.html#setting_up_a_new_podling

The fourth bullet is special since this will be a github transfer.

Please review ICLA, but the process is easier if you wait to submit them to the secretary until the mentors complete initial setup.

Regards,
Dave

Sent from my iPhone

> On Jul 17, 2018, at 7:42 PM, Li,De(BDG) <li...@baidu.com> wrote:
> 
> Hi Dave,
> 
> What should we do next? Can you give me a to do list or something?
> 
> Best Regards,
> Reed
> 
> 
> On 2018/7/13 上午10:07， "Li,De(BDG)" <li...@baidu.com>> wrote:
> 
> That’s great, thank you, Dave, thank everyone for your support.
> 
> On 2018/7/13 上午6:17， "Dave Fisher" <da...@comcast.net>> wrote:
> 
> Hi -
> 
> Welcome Doris to the Apache Incubator!
> 
> The vote passes with 8 +1 (binding votes) and 6 +1 (non-binding votes)
> 
> Binding votes:
> Dave Fisher
> Bertrand Delacretaz
> Timothy Chen
> Matt Sicker
> Willem Jiang
> Jean-Baptiste Onofré
> Kevin A. McGrail
> Han Luke
> 
> Non-binding votes:
> Sijie Guo
> Zhongyi Tan
> Charith Elvitigala
> Xin Wang
> 吴晟 Sheng Wu
> Jianru Tong
> 
> The VOTE thread archive.
> https://lists.apache.org/thread.html/916a2249a78bd78b77ded01d42fe932db1b8c3a3c795038128e58ced@%3Cgeneral.incubator.apache.org%3E
> 
> Regards,
> Dave
> 
> On Jul 11, 2018, at 4:31 PM, Han Luke <lu...@gmail.com>> wrote:
> 
> +1 binding
> 
> ________________________________
> From: Timothy Chen <tn...@gmail.com>>
> Sent: Friday, July 6, 2018 5:19:43 PM
> To: general@incubator.apache.org<ma...@incubator.apache.org>
> Subject: Re: [VOTE] Accept Doris into the Apache Incubator
> 
> +1 (binding)
> 
> Been following the discussions and knowing the mentors I believe the team
> can continue to improve and learn how the Apache way works.
> 
> Tim
> On Fri, Jul 6, 2018 at 5:15 PM Bertrand Delacretaz <
> bdelacretaz@codeconsult.ch<ma...@codeconsult.ch>> wrote:
> 
> On Fri, Jul 6, 2018 at 10:11 AM Li,De(BDG) <li...@baidu.com>> wrote:
> ...None of them are affiliated with Baidu Inc., just initial committers
> are...
> 
> Ok, thanks for clarifying!
> 
> Here's my +1
> 
> -Bertrand
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org<ma...@incubator.apache.org>
> For additional commands, e-mail: general-help@incubator.apache.org<ma...@incubator.apache.org>
> 
> 
>

Re: [RESULTS][VOTE] Accept Doris into the Apache Incubator

Posted by "Li,De(BDG)" <li...@baidu.com>.

Hi Dave,

What should we do next? Can you give me a to do list or something?

Best Regards,
Reed


On 2018/7/13 上午10:07， "Li,De(BDG)" <li...@baidu.com>> wrote:

That’s great, thank you, Dave, thank everyone for your support.

On 2018/7/13 上午6:17， "Dave Fisher" <da...@comcast.net>> wrote:

Hi -

Welcome Doris to the Apache Incubator!

The vote passes with 8 +1 (binding votes) and 6 +1 (non-binding votes)

Binding votes:
Dave Fisher
Bertrand Delacretaz
Timothy Chen
Matt Sicker
Willem Jiang
Jean-Baptiste Onofré
Kevin A. McGrail
Han Luke

Non-binding votes:
Sijie Guo
Zhongyi Tan
Charith Elvitigala
Xin Wang
吴晟 Sheng Wu
Jianru Tong

The VOTE thread archive.
https://lists.apache.org/thread.html/916a2249a78bd78b77ded01d42fe932db1b8c3a3c795038128e58ced@%3Cgeneral.incubator.apache.org%3E

Regards,
Dave

On Jul 11, 2018, at 4:31 PM, Han Luke <lu...@gmail.com>> wrote:

+1 binding

________________________________
From: Timothy Chen <tn...@gmail.com>>
Sent: Friday, July 6, 2018 5:19:43 PM
To: general@incubator.apache.org<ma...@incubator.apache.org>
Subject: Re: [VOTE] Accept Doris into the Apache Incubator

+1 (binding)

Been following the discussions and knowing the mentors I believe the team
can continue to improve and learn how the Apache way works.

Tim
On Fri, Jul 6, 2018 at 5:15 PM Bertrand Delacretaz <
bdelacretaz@codeconsult.ch<ma...@codeconsult.ch>> wrote:

On Fri, Jul 6, 2018 at 10:11 AM Li,De(BDG) <li...@baidu.com>> wrote:
...None of them are affiliated with Baidu Inc., just initial committers
are...

Ok, thanks for clarifying!

Here's my +1

-Bertrand

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org<ma...@incubator.apache.org>
For additional commands, e-mail: general-help@incubator.apache.org<ma...@incubator.apache.org>

Re: [RESULTS][VOTE] Accept Doris into the Apache Incubator

Posted by "Li,De(BDG)" <li...@baidu.com>.

That’s great, thank you, Dave, thank everyone for your support.

On 2018/7/13 上午6:17， "Dave Fisher" <da...@comcast.net>> wrote:

Hi -

Welcome Doris to the Apache Incubator!

The vote passes with 8 +1 (binding votes) and 6 +1 (non-binding votes)

Binding votes:
Dave Fisher
Bertrand Delacretaz
Timothy Chen
Matt Sicker
Willem Jiang
Jean-Baptiste Onofré
Kevin A. McGrail
Han Luke

Non-binding votes:
Sijie Guo
Zhongyi Tan
Charith Elvitigala
Xin Wang
吴晟 Sheng Wu
Jianru Tong

The VOTE thread archive.
https://lists.apache.org/thread.html/916a2249a78bd78b77ded01d42fe932db1b8c3a3c795038128e58ced@%3Cgeneral.incubator.apache.org%3E

Regards,
Dave

On Jul 11, 2018, at 4:31 PM, Han Luke <lu...@gmail.com>> wrote:

+1 binding

________________________________
From: Timothy Chen <tn...@gmail.com>>
Sent: Friday, July 6, 2018 5:19:43 PM
To: general@incubator.apache.org<ma...@incubator.apache.org>
Subject: Re: [VOTE] Accept Doris into the Apache Incubator

+1 (binding)

Been following the discussions and knowing the mentors I believe the team
can continue to improve and learn how the Apache way works.

Tim
On Fri, Jul 6, 2018 at 5:15 PM Bertrand Delacretaz <
bdelacretaz@codeconsult.ch<ma...@codeconsult.ch>> wrote:

On Fri, Jul 6, 2018 at 10:11 AM Li,De(BDG) <li...@baidu.com>> wrote:
...None of them are affiliated with Baidu Inc., just initial committers
are...

Ok, thanks for clarifying!

Here's my +1

-Bertrand

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org<ma...@incubator.apache.org>
For additional commands, e-mail: general-help@incubator.apache.org<ma...@incubator.apache.org>

[RESULTS][VOTE] Accept Doris into the Apache Incubator

Posted by Dave Fisher <da...@comcast.net>.

Hi -

Welcome Doris to the Apache Incubator!

The vote passes with 8 +1 (binding votes) and 6 +1 (non-binding votes)

Binding votes:
Dave Fisher
Bertrand Delacretaz
Timothy Chen
Matt Sicker
Willem Jiang
Jean-Baptiste Onofré
Kevin A. McGrail
Han Luke

Non-binding votes:
Sijie Guo
Zhongyi Tan
Charith Elvitigala
Xin Wang
吴晟 Sheng Wu
Jianru Tong

The VOTE thread archive.
https://lists.apache.org/thread.html/916a2249a78bd78b77ded01d42fe932db1b8c3a3c795038128e58ced@%3Cgeneral.incubator.apache.org%3E

Regards,
Dave

> On Jul 11, 2018, at 4:31 PM, Han Luke <lu...@gmail.com> wrote:
> 
> +1 binding
> 
> ________________________________
> From: Timothy Chen <tn...@gmail.com>
> Sent: Friday, July 6, 2018 5:19:43 PM
> To: general@incubator.apache.org
> Subject: Re: [VOTE] Accept Doris into the Apache Incubator
> 
> +1 (binding)
> 
> Been following the discussions and knowing the mentors I believe the team
> can continue to improve and learn how the Apache way works.
> 
> Tim
> On Fri, Jul 6, 2018 at 5:15 PM Bertrand Delacretaz <
> bdelacretaz@codeconsult.ch> wrote:
> 
>> On Fri, Jul 6, 2018 at 10:11 AM Li,De(BDG) <li...@baidu.com> wrote:
>>> ...None of them are affiliated with Baidu Inc., just initial committers
>> are...
>> 
>> Ok, thanks for clarifying!
>> 
>> Here's my +1
>> 
>> -Bertrand
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>> 
>>

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by Han Luke <lu...@gmail.com>.

+1 binding

________________________________
From: Timothy Chen <tn...@gmail.com>
Sent: Friday, July 6, 2018 5:19:43 PM
To: general@incubator.apache.org
Subject: Re: [VOTE] Accept Doris into the Apache Incubator

+1 (binding)

Been following the discussions and knowing the mentors I believe the team
can continue to improve and learn how the Apache way works.

Tim
On Fri, Jul 6, 2018 at 5:15 PM Bertrand Delacretaz <
bdelacretaz@codeconsult.ch> wrote:

> On Fri, Jul 6, 2018 at 10:11 AM Li,De(BDG) <li...@baidu.com> wrote:
> > ...None of them are affiliated with Baidu Inc., just initial committers
> are...
>
> Ok, thanks for clarifying!
>
> Here's my +1
>
> -Bertrand
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by 吴晟 Sheng Wu <wu...@foxmail.com>.

My +1 no-binding.


Welcome to join the community. Interesting project. 


------------------
Sheng Wu
Apache SkyWalking 


 




------------------ Original ------------------
From:  "Matt Sicker"<bo...@gmail.com>;
Date:  Sun, Jul 8, 2018 03:22 AM
To:  "general"<ge...@incubator.apache.org>;

Subject:  Re: [VOTE] Accept Doris into the Apache Incubator



+1 (binding)

Sounds like a really interesting project. I hope to see more PPMC members
from outside Baidu over time of course!

On Fri, 6 Jul 2018 at 04:20, Timothy Chen <tn...@gmail.com> wrote:

> +1 (binding)
>
> Been following the discussions and knowing the mentors I believe the team
> can continue to improve and learn how the Apache way works.
>
> Tim
> On Fri, Jul 6, 2018 at 5:15 PM Bertrand Delacretaz <
> bdelacretaz@codeconsult.ch> wrote:
>
> > On Fri, Jul 6, 2018 at 10:11 AM Li,De(BDG) <li...@baidu.com> wrote:
> > > ...None of them are affiliated with Baidu Inc., just initial committers
> > are...
> >
> > Ok, thanks for clarifying!
> >
> > Here's my +1
> >
> > -Bertrand
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: general-help@incubator.apache.org
> >
> >
>


-- 
Matt Sicker <bo...@gmail.com>

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by Matt Sicker <bo...@gmail.com>.

+1 (binding)

Sounds like a really interesting project. I hope to see more PPMC members
from outside Baidu over time of course!

On Fri, 6 Jul 2018 at 04:20, Timothy Chen <tn...@gmail.com> wrote:

> +1 (binding)
>
> Been following the discussions and knowing the mentors I believe the team
> can continue to improve and learn how the Apache way works.
>
> Tim
> On Fri, Jul 6, 2018 at 5:15 PM Bertrand Delacretaz <
> bdelacretaz@codeconsult.ch> wrote:
>
> > On Fri, Jul 6, 2018 at 10:11 AM Li,De(BDG) <li...@baidu.com> wrote:
> > > ...None of them are affiliated with Baidu Inc., just initial committers
> > are...
> >
> > Ok, thanks for clarifying!
> >
> > Here's my +1
> >
> > -Bertrand
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: general-help@incubator.apache.org
> >
> >
>


-- 
Matt Sicker <bo...@gmail.com>

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by Timothy Chen <tn...@gmail.com>.

+1 (binding)

Been following the discussions and knowing the mentors I believe the team
can continue to improve and learn how the Apache way works.

Tim
On Fri, Jul 6, 2018 at 5:15 PM Bertrand Delacretaz <
bdelacretaz@codeconsult.ch> wrote:

> On Fri, Jul 6, 2018 at 10:11 AM Li,De(BDG) <li...@baidu.com> wrote:
> > ...None of them are affiliated with Baidu Inc., just initial committers
> are...
>
> Ok, thanks for clarifying!
>
> Here's my +1
>
> -Bertrand
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by Bertrand Delacretaz <bd...@codeconsult.ch>.

On Fri, Jul 6, 2018 at 10:11 AM Li,De(BDG) <li...@baidu.com> wrote:
> ...None of them are affiliated with Baidu Inc., just initial committers are...

Ok, thanks for clarifying!

Here's my +1

-Bertrand

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by "Li,De(BDG)" <li...@baidu.com>.

None of them are affiliated with Baidu Inc., just initial committers are.

Best Regards,
Reed

On 2018/7/6 下午4:04， "Bertrand Delacretaz" <bd...@codeconsult.ch>
wrote:

>Hi,
>
>On Thu, Jul 5, 2018 at 9:22 PM Dave Fisher <da...@comcast.net> wrote:
>> The initial committers are employees of Baidu Inc..
>...
>> === Nominated Mentors ===
>>
>>  * Luke Han, lukehan@apache.org
>>  * Dave Fisher, wave@apache.org
>>  * Willem Jiang, ningjiang@apache.org ...
>
>Are the mentors also affiliated with Baidu Inc.. ?
>
>-Bertrand
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>For additional commands, e-mail: general-help@incubator.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by Bertrand Delacretaz <bd...@codeconsult.ch>.

Hi,

On Thu, Jul 5, 2018 at 9:22 PM Dave Fisher <da...@comcast.net> wrote:
> The initial committers are employees of Baidu Inc..
...
> === Nominated Mentors ===
>
>  * Luke Han, lukehan@apache.org
>  * Dave Fisher, wave@apache.org
>  * Willem Jiang, ningjiang@apache.org ...

Are the mentors also affiliated with Baidu Inc.. ?

-Bertrand

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by Willem Jiang <wi...@gmail.com>.

+1 (binding)



Willem Jiang

Twitter: willemjiang
Weibo: 姜宁willem

On Fri, Jul 6, 2018 at 3:22 AM, Dave Fisher <da...@comcast.net> wrote:

> Hi All,
>
> I would like to start a VOTE to bring the Doris project as an Apache
> incubator podling.
>
> The ASF voting rules are described:
>
> https://www.apache.org/foundation/voting.html
>
> A vote for accepting a new Apache Incubator podling is a majority vote for which
> only Incubator PMC member votes are binding.
>
> This vote will run for at least 72 hours. Please VOTE as follows
> [] +1 Accept Doris into the Apache Incubator
> [] +0 Abstain.
> [] -1 Do not accept Doris into the Apache Incubator because ...
>
> The proposal is listed below, but you can also access it on the wiki:
>
> https://wiki.apache.org/incubator/DorisProposal
>
> Best regards,
> Dave
>
> = Apache Doris =
>
> == Abstract ==
>
> Doris is a MPP-based interactive SQL data warehousing for reporting and
> analysis.
>
> == Proposal ==
>
> We propose to contribute the Doris codebase and associated artifacts (e.g.
> documentation, web-site content etc.) to the Apache Software Foundation,
> and aim to build an open community around Doris’s continued development in
> the ‘Apache Way’.
>
> === Overview of Doris ===
>
> Doris’s implementation consists of two daemons: Frontend (FE) and Backend
> (BE).
>
> **Frontend daemon** consists of query coordinator and catalog manager.
> Query coordinator is responsible for receiving users’ sql queries,
> compiling queries and managing queries execution. Catalog manager is
> responsible for managing metadata such as databases, tables, partitions,
> replicas and etc. Several frontend daemons could be deployed to guarantee
> fault-tolerance, and load balancing.
>
> **Backend daemon** stores the data and executes the query fragments. Many
> backend daemons could also be deployed to provide scalability and
> fault-tolerance.
>
> A typical Doris cluster generally composes of several frontend daemons and
> dozens to hundreds of backend daemons.
>
> Users can use MySQL client tools to connect any frontend daemon to submit
> SQL query. Frontend receives the query and compiles it into query plans
> executable by the Backend. Then Frontend sends the query plan fragments to
> Backend. Backend will build a query execution DAG. Data is fetched and
> pipelined into the DAG. The final result response is sent to client via
> Frontend. The distribution of query fragment execution takes minimizing
> data movement and maximizing scan locality as the main goal.
>
> == Background ==
>
> At Baidu, Prior to Doris, different tools were deployed to solve diverse
> requirements in many ways. And when a use case requires the simultaneous
> availability of capabilities that cannot all be provided by a single tool,
> users were forced to build hybrid architectures that stitch multiple tools
> together, but we believe that they shouldn’t need to accept such inherent
> complexity. A storage system built to provide great performance across a
> broad range of workloads provides a more elegant solution to the problems
> that hybrid architectures aim to solve. Doris is the solution.
>
> Doris is designed to be a simple and single tightly coupled system, not
> depending on other systems. Doris provides high concurrent low latency
> point query performance, but also provides high throughput queries of
> ad-hoc analysis. Doris provides bulk-batch data loading, but also provides
> near real-time mini-batch data loading. Doris also provides high
> availability, reliability, fault tolerance, and scalability.
>
> == Rationale ==
>
> Doris mainly integrates the technology of Google Mesa and Apache Impala.
>
> Mesa is a highly scalable analytic data storage system that stores
> critical measurement data related to Google's Internet advertising
> business. Mesa is designed to satisfy complex and challenging set of users’
> and systems’ requirements, including near real-time data ingestion and
> query ability, as well as high availability, reliability, fault tolerance,
> and scalability for large data and query volumes.
>
> Impala is a modern, open-source MPP SQL engine architected from the ground
> up for the Hadoop data processing environment. At present, by virtue of its
> superior performance and rich functionality， Impala has been comparable to
> many commercial MPP database query engine. Mesa can satisfy the needs of
> many of our storage requirements, however Mesa itself does not provide a
> SQL query engine; Impala is a very good MPP SQL query engine, but the lack
> of a perfect distributed storage engine. So in the end we chose the
> combination of these two technologies.
>
> Learning from Mesa’s data model, we developed a distributed storage
> engine. Unlike Mesa, this storage engine does not rely on any distributed
> file system. Then we deeply integrate this storage engine with Impala query
> engine. Query compiling, query execution coordination and catalog
> management of storage engine are integrated to be frontend daemon; query
> execution and data storage are integrated to be backend daemon. With this
> integration, we implemented a single, full-featured, high performance state
> the art of MPP database, as well as maintaining the simplicity.
>
> == Current Status ==
>
> Doris has been an open source project on GitHub (
> https://github.com/baidu/palo).
>
> === Meritocracy ===
>
> Doris has been deployed in production at Baidu and is applying more than
> 200 lines of business. It has demonstrated great performance benefits and
> has proved to be a better way for reporting and analysis based big data.
> Still We look forward to growing a rich user and developer community.
>
> === Community ===
>
> Doris seeks to develop developer and user communities during incubation.
>
> Doris makes use of Apache Impala. It was identified during early review of
> the proposal that the Doris community will need to work with Impala to
> define a suitable API.
>
> === Core Developers ===
>
>  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
>  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
>
> === Alignment ===
>
> Doris is related to several other Apache projects:
>
>  * Doris can also read data stored in Apache Hadoop clusters powered by
> the HDFS filesystem.
>  * Doris is closely integrated with Impala, which has graduated from
> Apache Incubator.
>  * Doris uses Apache Thrift as its RPC and serialization framework of
> choice.
>
> == Known Risks ==
>
> === Orphaned Products ===
>
> The core developers of Doris team plan to work full time on this project.
> There is very little risk of Doris getting orphaned since at least one
> large company (Baidu) is extensively using it in their production. For
> example, currently there are more than 200 use cases using Doris in
> production. Furthermore, since Doris was open sourced at the beginning of
> October 2017, it has received more than 660 stars and been forked nearly
> 170 times. We plan to extend and diversify this community further through
> Apache.
>
> === Inexperience with Open Source ===
>
> The core developers are all active users and followers of open source.
> They are already committers and contributors to the Doris Github project.
> All have been involved with the source code that has been released under an
> open source license, and several of them also have experience developing
> code in an open source environment. Though the core set of Developers do
> not have Apache Open Source experience, there are plans to onboard
> individuals with Apache open source experience on to the project.
>
> === Homogenous Developers ===
>
> The most of core developers are from Baidu, but after Doris was open
> sourced, Doris received a lot of bug fixes and enhancements from other
> developers not working at Baidu.
>
> === Reliance on Salaried Developers ===
>
> Baidu invested in Doris as the OLAP solution and some of its key engineers
> are working full time on the project. In addition, since there is a growing
> Big Data need for scalable OLAP solutions, we look forward to other Apache
> developers and researchers to contribute to the project. Also key to
> addressing the risk associated with relying on Salaried developers from a
> single entity is to increase the diversity of the contributors and actively
> lobby for Domain experts in the BI space to contribute. Apache Doris
> intends to do this.
>
> === An Excessive Fascination with the Apache Brand ===
>
> Doris is proposing to enter incubation at Apache in order to help efforts
> to diversify the committer-base, not so much to capitalize on the Apache
> brand. The Doris project is in production use already inside Baidu, but is
> not expected to be an Baidu product for external customers. As such, the
> Doris project is not seeking to use the Apache brand as a marketing tool.
>
> == Documentation ==
>
> Information about Doris can be found at https://github.com/baidu/palo.
> The following links provide more information about Doris in open source:
>
>  * Doris wiki site: https://github.com/baidu/palo/wiki
>  * Codebase at Github: https://github.com/baidu/palo
>  * Issue Tracking: https://github.com/baidu/palo/issues
>  * Overview: https://github.com/baidu/Doris/wiki/palo-Overview
>  * FAQ: https://github.com/baidu/palo/wiki/palo-FAQ
>
> == Initial Source ==
>
> Doris has been under development since 2017 by a team of engineers at
> Baidu Inc. It is currently hosted on Github.com under an Apache license
> at https://github.com/baidu/palo.
>
> == External Dependencies ==
>
> Doris has the following external dependencies.
>
>  * Google gflags (BSD)
>  * Google glog (BSD)
>  * Apache Thrift (Apache Software License v2.0)
>  * Apache Commons (Apache Software License v2.0)
>  * Boost (Boost Software License)
>  * rapidjson (Tencent)
>  * Google RE2 (BSD-style)
>  * lz4 (BSD)
>  * snappy (BSD)
>  * Twitter Bootstrap (Apache Software License v2.0)
>  * d3 (BSD)
>  * LLVM (BSD-like)
>
> Build and test dependencies:
>
>  * Apache Ant (Apache Software License v2.0)
>  * Apache Maven (Apache Software License v2.0)
>  * cmake (BSD)
>  * clang (BSD)
>  * Google gtest (Apache Software License v2.0)
>
> == Required Resources ==
>
> === Mailing List ===
>
> There are currently no mailing lists. The usual mailing lists are expected
> to be set up when entering incubation:
>
>  * private@doris.incubator.apache.org
>  * dev@doris.incubator.apache.org
>  * commits@doris.incubator.apache.org
>
> === Subversion Directory ===
>
> Upon entering incubation, we want to move (or copy) the existing repo from
> https://github.com/baidu/palo to Apache infrastructure at
> https://github.com/apache/incubator-doris.
>
> === Issue Tracking ===
>
> Doris currently uses GitHub to track issues. Would like to continue to do
> so while we discuss migration possibilities with the ASF Infra committee.
>
> === Other Resources ===
>
> The existing code already has unit tests so we will make use of existing
> Apache continuous testing infrastructure. The resulting load should not be
> very large.
>
> == Initial Committers ==
>
>  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
>  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
>  * Sijie Guo (guosijie@gmail dot com)
>  * Zheng Shao (zshao@apache.org)
>
> == Affiliations ==
>
> The initial committers are employees of Baidu Inc..
>
> == Sponsors ==
>
> === Champion ===
>
>  * Dave Fisher, wave@apache.org
>
> === Nominated Mentors ===
>
>  * Luke Han, lukehan@apache.org
>  * Dave Fisher, wave@apache.org
>  * Willem Jiang, ningjiang@apache.org
>
> === Sponsoring Entity ===
>
> We are requesting the Incubator to sponsor this project.
>
>

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.

+1 (binding)

Regards
JB

On 05/07/2018 21:22, Dave Fisher wrote:
> Hi All,
> 
> I would like to start a VOTE to bring the Doris project as an Apache
> incubator podling.
> 
> The ASF voting rules are described:
> 
> https://www.apache.org/foundation/voting.html
> 
> A vote for accepting a new Apache Incubator podling is a majority vote
> for which only Incubator PMC member votes are binding.
> 
> This vote will run for at least 72 hours. Please VOTE as follows
> [] +1 Accept Doris into the Apache Incubator
> [] +0 Abstain.
> [] -1 Do not accept Doris into the Apache Incubator because ...
> 
> The proposal is listed below, but you can also access it on the wiki:
> 
> https://wiki.apache.org/incubator/DorisProposal
> 
> Best regards,
> Dave
> 
> = Apache Doris =
> 
> == Abstract ==
> 
> Doris is a MPP-based interactive SQL data warehousing for reporting and
> analysis. 
> 
> == Proposal ==
> 
> We propose to contribute the Doris codebase and associated artifacts
> (e.g. documentation, web-site content etc.) to the Apache Software
> Foundation, and aim to build an open community around Doris’s continued
> development in the ‘Apache Way’.
> 
> === Overview of Doris ===
> 
> Doris’s implementation consists of two daemons: Frontend (FE) and
> Backend (BE).
> 
> **Frontend daemon** consists of query coordinator and catalog manager.
> Query coordinator is responsible for receiving users’ sql queries,
> compiling queries and managing queries execution. Catalog manager is
> responsible for managing metadata such as databases, tables, partitions,
> replicas and etc. Several frontend daemons could be deployed to
> guarantee fault-tolerance, and load balancing.
> 
> **Backend daemon** stores the data and executes the query fragments.
> Many backend daemons could also be deployed to provide scalability and
> fault-tolerance.
> 
> A typical Doris cluster generally composes of several frontend daemons
> and dozens to hundreds of backend daemons.
> 
> Users can use MySQL client tools to connect any frontend daemon to
> submit SQL query. Frontend receives the query and compiles it into query
> plans executable by the Backend. Then Frontend sends the query plan
> fragments to Backend. Backend will build a query execution DAG. Data is
> fetched and pipelined into the DAG. The final result response is sent to
> client via Frontend. The distribution of query fragment execution takes
> minimizing data movement and maximizing scan locality as the main goal. 
> 
> == Background ==
> 
> At Baidu, Prior to Doris, different tools were deployed to solve diverse
> requirements in many ways. And when a use case requires the simultaneous
> availability of capabilities that cannot all be provided by a single
> tool, users were forced to build hybrid architectures that stitch
> multiple tools together, but we believe that they shouldn’t need to
> accept such inherent complexity. A storage system built to provide great
> performance across a broad range of workloads provides a more elegant
> solution to the problems that hybrid architectures aim to solve. Doris
> is the solution. 
> 
> Doris is designed to be a simple and single tightly coupled system, not
> depending on other systems. Doris provides high concurrent low latency
> point query performance, but also provides high throughput queries of
> ad-hoc analysis. Doris provides bulk-batch data loading, but also
> provides near real-time mini-batch data loading. Doris also provides
> high availability, reliability, fault tolerance, and scalability.
> 
> == Rationale ==
> 
> Doris mainly integrates the technology of Google Mesa and Apache Impala. 
> 
> Mesa is a highly scalable analytic data storage system that stores
> critical measurement data related to Google's Internet advertising
> business. Mesa is designed to satisfy complex and challenging set of
> users’ and systems’ requirements, including near real-time data
> ingestion and query ability, as well as high availability, reliability,
> fault tolerance, and scalability for large data and query volumes. 
> 
> Impala is a modern, open-source MPP SQL engine architected from the
> ground up for the Hadoop data processing environment. At present, by
> virtue of its superior performance and rich functionality， Impala has
> been comparable to many commercial MPP database query engine. Mesa can
> satisfy the needs of many of our storage requirements, however Mesa
> itself does not provide a SQL query engine; Impala is a very good MPP
> SQL query engine, but the lack of a perfect distributed storage engine.
> So in the end we chose the combination of these two technologies.
> 
> Learning from Mesa’s data model, we developed a distributed storage
> engine. Unlike Mesa, this storage engine does not rely on any
> distributed file system. Then we deeply integrate this storage engine
> with Impala query engine. Query compiling, query execution coordination
> and catalog management of storage engine are integrated to be frontend
> daemon; query execution and data storage are integrated to be backend
> daemon. With this integration, we implemented a single, full-featured,
> high performance state the art of MPP database, as well as maintaining
> the simplicity.
> 
> == Current Status ==
> 
> Doris has been an open source project on GitHub
> (https://github.com/baidu/palo).
> 
> === Meritocracy ===
> 
> Doris has been deployed in production at Baidu and is applying more than
> 200 lines of business. It has demonstrated great performance benefits
> and has proved to be a better way for reporting and analysis based big
> data. Still We look forward to growing a rich user and developer community.
> 
> === Community ===
> 
> Doris seeks to develop developer and user communities during incubation.
> 
> Doris makes use of Apache Impala. It was identified during early review
> of the proposal that the Doris community will need to work with Impala
> to define a suitable API.
> 
> === Core Developers ===
> 
>  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
>  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
> 
> === Alignment ===
> 
> Doris is related to several other Apache projects:
> 
>  * Doris can also read data stored in Apache Hadoop clusters powered by
> the HDFS filesystem.
>  * Doris is closely integrated with Impala, which has graduated from
> Apache Incubator.
>  * Doris uses Apache Thrift as its RPC and serialization framework of
> choice.
> 
> == Known Risks ==
> 
> === Orphaned Products ===
> 
> The core developers of Doris team plan to work full time on this
> project. There is very little risk of Doris getting orphaned since at
> least one large company (Baidu) is extensively using it in their
> production. For example, currently there are more than 200 use cases
> using Doris in production. Furthermore, since Doris was open sourced at
> the beginning of October 2017, it has received more than 660 stars and
> been forked nearly 170 times. We plan to extend and diversify this
> community further through Apache.
> 
> === Inexperience with Open Source ===
> 
> The core developers are all active users and followers of open source.
> They are already committers and contributors to the Doris Github
> project. All have been involved with the source code that has been
> released under an open source license, and several of them also have
> experience developing code in an open source environment. Though the
> core set of Developers do not have Apache Open Source experience, there
> are plans to onboard individuals with Apache open source experience on
> to the project.
> 
> === Homogenous Developers ===
> 
> The most of core developers are from Baidu, but after Doris was open
> sourced, Doris received a lot of bug fixes and enhancements from other
> developers not working at Baidu.
> 
> === Reliance on Salaried Developers ===
> 
> Baidu invested in Doris as the OLAP solution and some of its key
> engineers are working full time on the project. In addition, since there
> is a growing Big Data need for scalable OLAP solutions, we look forward
> to other Apache developers and researchers to contribute to the project.
> Also key to addressing the risk associated with relying on Salaried
> developers from a single entity is to increase the diversity of the
> contributors and actively lobby for Domain experts in the BI space to
> contribute. Apache Doris intends to do this. 
> 
> === An Excessive Fascination with the Apache Brand ===
> 
> Doris is proposing to enter incubation at Apache in order to help
> efforts to diversify the committer-base, not so much to capitalize on
> the Apache brand. The Doris project is in production use already inside
> Baidu, but is not expected to be an Baidu product for external
> customers. As such, the Doris project is not seeking to use the Apache
> brand as a marketing tool.
> 
> == Documentation ==
> 
> Information about Doris can be found at https://github.com/baidu/palo.
> The following links provide more information about Doris in open source:
> 
>  * Doris wiki site: https://github.com/baidu/palo/wiki
>  * Codebase at Github: https://github.com/baidu/palo
>  * Issue Tracking: https://github.com/baidu/palo/issues
>  * Overview: https://github.com/baidu/Doris/wiki/palo-Overview
>  * FAQ: https://github.com/baidu/palo/wiki/palo-FAQ
> 
> == Initial Source ==
> 
> Doris has been under development since 2017 by a team of engineers at
> Baidu Inc. It is currently hosted on Github.com <http://Github.com>
> under an Apache license at https://github.com/baidu/palo.
> 
> == External Dependencies ==
> 
> Doris has the following external dependencies.
> 
>  * Google gflags (BSD)
>  * Google glog (BSD)
>  * Apache Thrift (Apache Software License v2.0)
>  * Apache Commons (Apache Software License v2.0)
>  * Boost (Boost Software License)
>  * rapidjson (Tencent)
>  * Google RE2 (BSD-style)
>  * lz4 (BSD)
>  * snappy (BSD)
>  * Twitter Bootstrap (Apache Software License v2.0)
>  * d3 (BSD)
>  * LLVM (BSD-like)
> 
> Build and test dependencies:
> 
>  * Apache Ant (Apache Software License v2.0)
>  * Apache Maven (Apache Software License v2.0)
>  * cmake (BSD)
>  * clang (BSD)
>  * Google gtest (Apache Software License v2.0)
> 
> == Required Resources ==
> 
> === Mailing List ===
> 
> There are currently no mailing lists. The usual mailing lists are
> expected to be set up when entering incubation:
> 
>  * private@doris.incubator.apache.org
> <ma...@doris.incubator.apache.org>
>  * dev@doris.incubator.apache.org <ma...@doris.incubator.apache.org>
>  * commits@doris.incubator.apache.org
> <ma...@doris.incubator.apache.org>
> 
> === Subversion Directory ===
> 
> Upon entering incubation, we want to move (or copy) the existing repo
> from https://github.com/baidu/palo to Apache infrastructure at
> https://github.com/apache/incubator-doris.
> 
> === Issue Tracking ===
> 
> Doris currently uses GitHub to track issues. Would like to continue to
> do so while we discuss migration possibilities with the ASF Infra committee.
> 
> === Other Resources ===
> 
> The existing code already has unit tests so we will make use of existing
> Apache continuous testing infrastructure. The resulting load should not
> be very large.
> 
> == Initial Committers ==
> 
>  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
>  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
>  * Sijie Guo (guosijie@gmail dot com)
>  * Zheng Shao (zshao@apache.org <ma...@apache.org>)
> 
> == Affiliations ==
> 
> The initial committers are employees of Baidu Inc.. 
>  
> == Sponsors ==
> 
> === Champion ===
> 
>  * Dave Fisher, wave@apache.org <ma...@apache.org>
> 
> === Nominated Mentors ===
> 
>  * Luke Han, lukehan@apache.org <ma...@apache.org>
>  * Dave Fisher, wave@apache.org <ma...@apache.org>
>  * Willem Jiang, ningjiang@apache.org <ma...@apache.org>
> 
> === Sponsoring Entity ===
> 
> We are requesting the Incubator to sponsor this project.
> 

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by Xin Wang <wa...@apache.org>.

+1

Charith Elvitigala <ch...@apache.org> 于2018年7月6日周五 上午9:37写道：

> +1
>
> On Fri, 6 Jul 2018 at 06:21, Tan,Zhongyi <ta...@baidu.com> wrote:
>
> > +1 （no binding）
> >
> > 发件人: Dave Fisher <da...@comcast.net>>
> > 答复: <ge...@incubator.apache.org>>
> > 日期: 2018年7月6日 星期五 上午3:22
> > 至: <ge...@incubator.apache.org>>
> > 主题: [VOTE] Accept Doris into the Apache Incubator
> >
> > Hi All,
> >
> > I would like to start a VOTE to bring the Doris project as an Apache
> > incubator podling.
> >
> > The ASF voting rules are described:
> >
> > https://www.apache.org/foundation/voting.html
> >
> > A vote for accepting a new Apache Incubator podling is a majority vote
> for
> > which only Incubator PMC member votes are binding.
> >
> > This vote will run for at least 72 hours. Please VOTE as follows
> > [] +1 Accept Doris into the Apache Incubator
> > [] +0 Abstain.
> > [] -1 Do not accept Doris into the Apache Incubator because ...
> >
> > The proposal is listed below, but you can also access it on the wiki:
> >
> > https://wiki.apache.org/incubator/DorisProposal
> >
> > Best regards,
> > Dave
> >
> > = Apache Doris =
> >
> > == Abstract ==
> >
> > Doris is a MPP-based interactive SQL data warehousing for reporting and
> > analysis.
> >
> > == Proposal ==
> >
> > We propose to contribute the Doris codebase and associated artifacts
> (e.g.
> > documentation, web-site content etc.) to the Apache Software Foundation,
> > and aim to build an open community around Doris’s continued development
> in
> > the ‘Apache Way’.
> >
> > === Overview of Doris ===
> >
> > Doris’s implementation consists of two daemons: Frontend (FE) and Backend
> > (BE).
> >
> > **Frontend daemon** consists of query coordinator and catalog manager.
> > Query coordinator is responsible for receiving users’ sql queries,
> > compiling queries and managing queries execution. Catalog manager is
> > responsible for managing metadata such as databases, tables, partitions,
> > replicas and etc. Several frontend daemons could be deployed to guarantee
> > fault-tolerance, and load balancing.
> >
> > **Backend daemon** stores the data and executes the query fragments. Many
> > backend daemons could also be deployed to provide scalability and
> > fault-tolerance.
> >
> > A typical Doris cluster generally composes of several frontend daemons
> and
> > dozens to hundreds of backend daemons.
> >
> > Users can use MySQL client tools to connect any frontend daemon to submit
> > SQL query. Frontend receives the query and compiles it into query plans
> > executable by the Backend. Then Frontend sends the query plan fragments
> to
> > Backend. Backend will build a query execution DAG. Data is fetched and
> > pipelined into the DAG. The final result response is sent to client via
> > Frontend. The distribution of query fragment execution takes minimizing
> > data movement and maximizing scan locality as the main goal.
> >
> > == Background ==
> >
> > At Baidu, Prior to Doris, different tools were deployed to solve diverse
> > requirements in many ways. And when a use case requires the simultaneous
> > availability of capabilities that cannot all be provided by a single
> tool,
> > users were forced to build hybrid architectures that stitch multiple
> tools
> > together, but we believe that they shouldn’t need to accept such inherent
> > complexity. A storage system built to provide great performance across a
> > broad range of workloads provides a more elegant solution to the problems
> > that hybrid architectures aim to solve. Doris is the solution.
> >
> > Doris is designed to be a simple and single tightly coupled system, not
> > depending on other systems. Doris provides high concurrent low latency
> > point query performance, but also provides high throughput queries of
> > ad-hoc analysis. Doris provides bulk-batch data loading, but also
> provides
> > near real-time mini-batch data loading. Doris also provides high
> > availability, reliability, fault tolerance, and scalability.
> >
> > == Rationale ==
> >
> > Doris mainly integrates the technology of Google Mesa and Apache Impala.
> >
> > Mesa is a highly scalable analytic data storage system that stores
> > critical measurement data related to Google's Internet advertising
> > business. Mesa is designed to satisfy complex and challenging set of
> users’
> > and systems’ requirements, including near real-time data ingestion and
> > query ability, as well as high availability, reliability, fault
> tolerance,
> > and scalability for large data and query volumes.
> >
> > Impala is a modern, open-source MPP SQL engine architected from the
> ground
> > up for the Hadoop data processing environment. At present, by virtue of
> its
> > superior performance and rich functionality， Impala has been comparable
> to
> > many commercial MPP database query engine. Mesa can satisfy the needs of
> > many of our storage requirements, however Mesa itself does not provide a
> > SQL query engine; Impala is a very good MPP SQL query engine, but the
> lack
> > of a perfect distributed storage engine. So in the end we chose the
> > combination of these two technologies.
> >
> > Learning from Mesa’s data model, we developed a distributed storage
> > engine. Unlike Mesa, this storage engine does not rely on any distributed
> > file system. Then we deeply integrate this storage engine with Impala
> query
> > engine. Query compiling, query execution coordination and catalog
> > management of storage engine are integrated to be frontend daemon; query
> > execution and data storage are integrated to be backend daemon. With this
> > integration, we implemented a single, full-featured, high performance
> state
> > the art of MPP database, as well as maintaining the simplicity.
> >
> > == Current Status ==
> >
> > Doris has been an open source project on GitHub (
> > https://github.com/baidu/palo).
> >
> > === Meritocracy ===
> >
> > Doris has been deployed in production at Baidu and is applying more than
> > 200 lines of business. It has demonstrated great performance benefits and
> > has proved to be a better way for reporting and analysis based big data.
> > Still We look forward to growing a rich user and developer community.
> >
> > === Community ===
> >
> > Doris seeks to develop developer and user communities during incubation.
> >
> > Doris makes use of Apache Impala. It was identified during early review
> of
> > the proposal that the Doris community will need to work with Impala to
> > define a suitable API.
> >
> > === Core Developers ===
> >
> >  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
> >  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
> >  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
> >  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
> >  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
> >  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
> >  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
> >
> > === Alignment ===
> >
> > Doris is related to several other Apache projects:
> >
> >  * Doris can also read data stored in Apache Hadoop clusters powered by
> > the HDFS filesystem.
> >  * Doris is closely integrated with Impala, which has graduated from
> > Apache Incubator.
> >  * Doris uses Apache Thrift as its RPC and serialization framework of
> > choice.
> >
> > == Known Risks ==
> >
> > === Orphaned Products ===
> >
> > The core developers of Doris team plan to work full time on this project.
> > There is very little risk of Doris getting orphaned since at least one
> > large company (Baidu) is extensively using it in their production. For
> > example, currently there are more than 200 use cases using Doris in
> > production. Furthermore, since Doris was open sourced at the beginning of
> > October 2017, it has received more than 660 stars and been forked nearly
> > 170 times. We plan to extend and diversify this community further through
> > Apache.
> >
> > === Inexperience with Open Source ===
> >
> > The core developers are all active users and followers of open source.
> > They are already committers and contributors to the Doris Github project.
> > All have been involved with the source code that has been released under
> an
> > open source license, and several of them also have experience developing
> > code in an open source environment. Though the core set of Developers do
> > not have Apache Open Source experience, there are plans to onboard
> > individuals with Apache open source experience on to the project.
> >
> > === Homogenous Developers ===
> >
> > The most of core developers are from Baidu, but after Doris was open
> > sourced, Doris received a lot of bug fixes and enhancements from other
> > developers not working at Baidu.
> >
> > === Reliance on Salaried Developers ===
> >
> > Baidu invested in Doris as the OLAP solution and some of its key
> engineers
> > are working full time on the project. In addition, since there is a
> growing
> > Big Data need for scalable OLAP solutions, we look forward to other
> Apache
> > developers and researchers to contribute to the project. Also key to
> > addressing the risk associated with relying on Salaried developers from a
> > single entity is to increase the diversity of the contributors and
> actively
> > lobby for Domain experts in the BI space to contribute. Apache Doris
> > intends to do this.
> >
> > === An Excessive Fascination with the Apache Brand ===
> >
> > Doris is proposing to enter incubation at Apache in order to help efforts
> > to diversify the committer-base, not so much to capitalize on the Apache
> > brand. The Doris project is in production use already inside Baidu, but
> is
> > not expected to be an Baidu product for external customers. As such, the
> > Doris project is not seeking to use the Apache brand as a marketing tool.
> >
> > == Documentation ==
> >
> > Information about Doris can be found at https://github.com/baidu/palo.
> > The following links provide more information about Doris in open source:
> >
> >  * Doris wiki site: https://github.com/baidu/palo/wiki
> >  * Codebase at Github: https://github.com/baidu/palo
> >  * Issue Tracking: https://github.com/baidu/palo/issues
> >  * Overview: https://github.com/baidu/Doris/wiki/palo-Overview
> >  * FAQ: https://github.com/baidu/palo/wiki/palo-FAQ
> >
> > == Initial Source ==
> >
> > Doris has been under development since 2017 by a team of engineers at
> > Baidu Inc. It is currently hosted on Github.com<http://Github.com> under
> > an Apache license at https://github.com/baidu/palo.
> >
> > == External Dependencies ==
> >
> > Doris has the following external dependencies.
> >
> >  * Google gflags (BSD)
> >  * Google glog (BSD)
> >  * Apache Thrift (Apache Software License v2.0)
> >  * Apache Commons (Apache Software License v2.0)
> >  * Boost (Boost Software License)
> >  * rapidjson (Tencent)
> >  * Google RE2 (BSD-style)
> >  * lz4 (BSD)
> >  * snappy (BSD)
> >  * Twitter Bootstrap (Apache Software License v2.0)
> >  * d3 (BSD)
> >  * LLVM (BSD-like)
> >
> > Build and test dependencies:
> >
> >  * Apache Ant (Apache Software License v2.0)
> >  * Apache Maven (Apache Software License v2.0)
> >  * cmake (BSD)
> >  * clang (BSD)
> >  * Google gtest (Apache Software License v2.0)
> >
> > == Required Resources ==
> >
> > === Mailing List ===
> >
> > There are currently no mailing lists. The usual mailing lists are
> expected
> > to be set up when entering incubation:
> >
> >  * private@doris.incubator.apache.org<mailto:
> > private@doris.incubator.apache.org>
> >  * dev@doris.incubator.apache.org<ma...@doris.incubator.apache.org>
> >  * commits@doris.incubator.apache.org<mailto:
> > commits@doris.incubator.apache.org>
> >
> > === Subversion Directory ===
> >
> > Upon entering incubation, we want to move (or copy) the existing repo
> from
> > https://github.com/baidu/palo to Apache infrastructure at
> > https://github.com/apache/incubator-doris.
> >
> > === Issue Tracking ===
> >
> > Doris currently uses GitHub to track issues. Would like to continue to do
> > so while we discuss migration possibilities with the ASF Infra committee.
> >
> > === Other Resources ===
> >
> > The existing code already has unit tests so we will make use of existing
> > Apache continuous testing infrastructure. The resulting load should not
> be
> > very large.
> >
> > == Initial Committers ==
> >
> >  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
> >  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
> >  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
> >  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
> >  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
> >  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
> >  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
> >  * Sijie Guo (guosijie@gmail dot com)
> >  * Zheng Shao (zshao@apache.org<ma...@apache.org>)
> >
> > == Affiliations ==
> >
> > The initial committers are employees of Baidu Inc..
> >
> > == Sponsors ==
> >
> > === Champion ===
> >
> >  * Dave Fisher, wave@apache.org<ma...@apache.org>
> >
> > === Nominated Mentors ===
> >
> >  * Luke Han, lukehan@apache.org<ma...@apache.org>
> >  * Dave Fisher, wave@apache.org<ma...@apache.org>
> >  * Willem Jiang, ningjiang@apache.org<ma...@apache.org>
> >
> > === Sponsoring Entity ===
> >
> > We are requesting the Incubator to sponsor this project.
> >
> > --
> > Charitha Elvitigala
> >
>

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by Charith Elvitigala <ch...@apache.org>.

+1

On Fri, 6 Jul 2018 at 06:21, Tan,Zhongyi <ta...@baidu.com> wrote:

> +1 （no binding）
>
> 发件人: Dave Fisher <da...@comcast.net>>
> 答复: <ge...@incubator.apache.org>>
> 日期: 2018年7月6日 星期五 上午3:22
> 至: <ge...@incubator.apache.org>>
> 主题: [VOTE] Accept Doris into the Apache Incubator
>
> Hi All,
>
> I would like to start a VOTE to bring the Doris project as an Apache
> incubator podling.
>
> The ASF voting rules are described:
>
> https://www.apache.org/foundation/voting.html
>
> A vote for accepting a new Apache Incubator podling is a majority vote for
> which only Incubator PMC member votes are binding.
>
> This vote will run for at least 72 hours. Please VOTE as follows
> [] +1 Accept Doris into the Apache Incubator
> [] +0 Abstain.
> [] -1 Do not accept Doris into the Apache Incubator because ...
>
> The proposal is listed below, but you can also access it on the wiki:
>
> https://wiki.apache.org/incubator/DorisProposal
>
> Best regards,
> Dave
>
> = Apache Doris =
>
> == Abstract ==
>
> Doris is a MPP-based interactive SQL data warehousing for reporting and
> analysis.
>
> == Proposal ==
>
> We propose to contribute the Doris codebase and associated artifacts (e.g.
> documentation, web-site content etc.) to the Apache Software Foundation,
> and aim to build an open community around Doris’s continued development in
> the ‘Apache Way’.
>
> === Overview of Doris ===
>
> Doris’s implementation consists of two daemons: Frontend (FE) and Backend
> (BE).
>
> **Frontend daemon** consists of query coordinator and catalog manager.
> Query coordinator is responsible for receiving users’ sql queries,
> compiling queries and managing queries execution. Catalog manager is
> responsible for managing metadata such as databases, tables, partitions,
> replicas and etc. Several frontend daemons could be deployed to guarantee
> fault-tolerance, and load balancing.
>
> **Backend daemon** stores the data and executes the query fragments. Many
> backend daemons could also be deployed to provide scalability and
> fault-tolerance.
>
> A typical Doris cluster generally composes of several frontend daemons and
> dozens to hundreds of backend daemons.
>
> Users can use MySQL client tools to connect any frontend daemon to submit
> SQL query. Frontend receives the query and compiles it into query plans
> executable by the Backend. Then Frontend sends the query plan fragments to
> Backend. Backend will build a query execution DAG. Data is fetched and
> pipelined into the DAG. The final result response is sent to client via
> Frontend. The distribution of query fragment execution takes minimizing
> data movement and maximizing scan locality as the main goal.
>
> == Background ==
>
> At Baidu, Prior to Doris, different tools were deployed to solve diverse
> requirements in many ways. And when a use case requires the simultaneous
> availability of capabilities that cannot all be provided by a single tool,
> users were forced to build hybrid architectures that stitch multiple tools
> together, but we believe that they shouldn’t need to accept such inherent
> complexity. A storage system built to provide great performance across a
> broad range of workloads provides a more elegant solution to the problems
> that hybrid architectures aim to solve. Doris is the solution.
>
> Doris is designed to be a simple and single tightly coupled system, not
> depending on other systems. Doris provides high concurrent low latency
> point query performance, but also provides high throughput queries of
> ad-hoc analysis. Doris provides bulk-batch data loading, but also provides
> near real-time mini-batch data loading. Doris also provides high
> availability, reliability, fault tolerance, and scalability.
>
> == Rationale ==
>
> Doris mainly integrates the technology of Google Mesa and Apache Impala.
>
> Mesa is a highly scalable analytic data storage system that stores
> critical measurement data related to Google's Internet advertising
> business. Mesa is designed to satisfy complex and challenging set of users’
> and systems’ requirements, including near real-time data ingestion and
> query ability, as well as high availability, reliability, fault tolerance,
> and scalability for large data and query volumes.
>
> Impala is a modern, open-source MPP SQL engine architected from the ground
> up for the Hadoop data processing environment. At present, by virtue of its
> superior performance and rich functionality， Impala has been comparable to
> many commercial MPP database query engine. Mesa can satisfy the needs of
> many of our storage requirements, however Mesa itself does not provide a
> SQL query engine; Impala is a very good MPP SQL query engine, but the lack
> of a perfect distributed storage engine. So in the end we chose the
> combination of these two technologies.
>
> Learning from Mesa’s data model, we developed a distributed storage
> engine. Unlike Mesa, this storage engine does not rely on any distributed
> file system. Then we deeply integrate this storage engine with Impala query
> engine. Query compiling, query execution coordination and catalog
> management of storage engine are integrated to be frontend daemon; query
> execution and data storage are integrated to be backend daemon. With this
> integration, we implemented a single, full-featured, high performance state
> the art of MPP database, as well as maintaining the simplicity.
>
> == Current Status ==
>
> Doris has been an open source project on GitHub (
> https://github.com/baidu/palo).
>
> === Meritocracy ===
>
> Doris has been deployed in production at Baidu and is applying more than
> 200 lines of business. It has demonstrated great performance benefits and
> has proved to be a better way for reporting and analysis based big data.
> Still We look forward to growing a rich user and developer community.
>
> === Community ===
>
> Doris seeks to develop developer and user communities during incubation.
>
> Doris makes use of Apache Impala. It was identified during early review of
> the proposal that the Doris community will need to work with Impala to
> define a suitable API.
>
> === Core Developers ===
>
>  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
>  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
>
> === Alignment ===
>
> Doris is related to several other Apache projects:
>
>  * Doris can also read data stored in Apache Hadoop clusters powered by
> the HDFS filesystem.
>  * Doris is closely integrated with Impala, which has graduated from
> Apache Incubator.
>  * Doris uses Apache Thrift as its RPC and serialization framework of
> choice.
>
> == Known Risks ==
>
> === Orphaned Products ===
>
> The core developers of Doris team plan to work full time on this project.
> There is very little risk of Doris getting orphaned since at least one
> large company (Baidu) is extensively using it in their production. For
> example, currently there are more than 200 use cases using Doris in
> production. Furthermore, since Doris was open sourced at the beginning of
> October 2017, it has received more than 660 stars and been forked nearly
> 170 times. We plan to extend and diversify this community further through
> Apache.
>
> === Inexperience with Open Source ===
>
> The core developers are all active users and followers of open source.
> They are already committers and contributors to the Doris Github project.
> All have been involved with the source code that has been released under an
> open source license, and several of them also have experience developing
> code in an open source environment. Though the core set of Developers do
> not have Apache Open Source experience, there are plans to onboard
> individuals with Apache open source experience on to the project.
>
> === Homogenous Developers ===
>
> The most of core developers are from Baidu, but after Doris was open
> sourced, Doris received a lot of bug fixes and enhancements from other
> developers not working at Baidu.
>
> === Reliance on Salaried Developers ===
>
> Baidu invested in Doris as the OLAP solution and some of its key engineers
> are working full time on the project. In addition, since there is a growing
> Big Data need for scalable OLAP solutions, we look forward to other Apache
> developers and researchers to contribute to the project. Also key to
> addressing the risk associated with relying on Salaried developers from a
> single entity is to increase the diversity of the contributors and actively
> lobby for Domain experts in the BI space to contribute. Apache Doris
> intends to do this.
>
> === An Excessive Fascination with the Apache Brand ===
>
> Doris is proposing to enter incubation at Apache in order to help efforts
> to diversify the committer-base, not so much to capitalize on the Apache
> brand. The Doris project is in production use already inside Baidu, but is
> not expected to be an Baidu product for external customers. As such, the
> Doris project is not seeking to use the Apache brand as a marketing tool.
>
> == Documentation ==
>
> Information about Doris can be found at https://github.com/baidu/palo.
> The following links provide more information about Doris in open source:
>
>  * Doris wiki site: https://github.com/baidu/palo/wiki
>  * Codebase at Github: https://github.com/baidu/palo
>  * Issue Tracking: https://github.com/baidu/palo/issues
>  * Overview: https://github.com/baidu/Doris/wiki/palo-Overview
>  * FAQ: https://github.com/baidu/palo/wiki/palo-FAQ
>
> == Initial Source ==
>
> Doris has been under development since 2017 by a team of engineers at
> Baidu Inc. It is currently hosted on Github.com<http://Github.com> under
> an Apache license at https://github.com/baidu/palo.
>
> == External Dependencies ==
>
> Doris has the following external dependencies.
>
>  * Google gflags (BSD)
>  * Google glog (BSD)
>  * Apache Thrift (Apache Software License v2.0)
>  * Apache Commons (Apache Software License v2.0)
>  * Boost (Boost Software License)
>  * rapidjson (Tencent)
>  * Google RE2 (BSD-style)
>  * lz4 (BSD)
>  * snappy (BSD)
>  * Twitter Bootstrap (Apache Software License v2.0)
>  * d3 (BSD)
>  * LLVM (BSD-like)
>
> Build and test dependencies:
>
>  * Apache Ant (Apache Software License v2.0)
>  * Apache Maven (Apache Software License v2.0)
>  * cmake (BSD)
>  * clang (BSD)
>  * Google gtest (Apache Software License v2.0)
>
> == Required Resources ==
>
> === Mailing List ===
>
> There are currently no mailing lists. The usual mailing lists are expected
> to be set up when entering incubation:
>
>  * private@doris.incubator.apache.org<mailto:
> private@doris.incubator.apache.org>
>  * dev@doris.incubator.apache.org<ma...@doris.incubator.apache.org>
>  * commits@doris.incubator.apache.org<mailto:
> commits@doris.incubator.apache.org>
>
> === Subversion Directory ===
>
> Upon entering incubation, we want to move (or copy) the existing repo from
> https://github.com/baidu/palo to Apache infrastructure at
> https://github.com/apache/incubator-doris.
>
> === Issue Tracking ===
>
> Doris currently uses GitHub to track issues. Would like to continue to do
> so while we discuss migration possibilities with the ASF Infra committee.
>
> === Other Resources ===
>
> The existing code already has unit tests so we will make use of existing
> Apache continuous testing infrastructure. The resulting load should not be
> very large.
>
> == Initial Committers ==
>
>  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
>  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
>  * Sijie Guo (guosijie@gmail dot com)
>  * Zheng Shao (zshao@apache.org<ma...@apache.org>)
>
> == Affiliations ==
>
> The initial committers are employees of Baidu Inc..
>
> == Sponsors ==
>
> === Champion ===
>
>  * Dave Fisher, wave@apache.org<ma...@apache.org>
>
> === Nominated Mentors ===
>
>  * Luke Han, lukehan@apache.org<ma...@apache.org>
>  * Dave Fisher, wave@apache.org<ma...@apache.org>
>  * Willem Jiang, ningjiang@apache.org<ma...@apache.org>
>
> === Sponsoring Entity ===
>
> We are requesting the Incubator to sponsor this project.
>
> --
> Charitha Elvitigala
>

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by "Tan,Zhongyi" <ta...@baidu.com>.

+1 （no binding）

发件人: Dave Fisher <da...@comcast.net>>
答复: <ge...@incubator.apache.org>>
日期: 2018年7月6日 星期五 上午3:22
至: <ge...@incubator.apache.org>>
主题: [VOTE] Accept Doris into the Apache Incubator

Hi All,

I would like to start a VOTE to bring the Doris project as an Apache incubator podling.

The ASF voting rules are described:

https://www.apache.org/foundation/voting.html

A vote for accepting a new Apache Incubator podling is a majority vote for which only Incubator PMC member votes are binding.

This vote will run for at least 72 hours. Please VOTE as follows
[] +1 Accept Doris into the Apache Incubator
[] +0 Abstain.
[] -1 Do not accept Doris into the Apache Incubator because ...

The proposal is listed below, but you can also access it on the wiki:

https://wiki.apache.org/incubator/DorisProposal

Best regards,
Dave

= Apache Doris =

== Abstract ==

Doris is a MPP-based interactive SQL data warehousing for reporting and analysis.

== Proposal ==

We propose to contribute the Doris codebase and associated artifacts (e.g. documentation, web-site content etc.) to the Apache Software Foundation, and aim to build an open community around Doris’s continued development in the ‘Apache Way’.

=== Overview of Doris ===

Doris’s implementation consists of two daemons: Frontend (FE) and Backend (BE).

**Frontend daemon** consists of query coordinator and catalog manager. Query coordinator is responsible for receiving users’ sql queries, compiling queries and managing queries execution. Catalog manager is responsible for managing metadata such as databases, tables, partitions, replicas and etc. Several frontend daemons could be deployed to guarantee fault-tolerance, and load balancing.

**Backend daemon** stores the data and executes the query fragments. Many backend daemons could also be deployed to provide scalability and fault-tolerance.

A typical Doris cluster generally composes of several frontend daemons and dozens to hundreds of backend daemons.

Users can use MySQL client tools to connect any frontend daemon to submit SQL query. Frontend receives the query and compiles it into query plans executable by the Backend. Then Frontend sends the query plan fragments to Backend. Backend will build a query execution DAG. Data is fetched and pipelined into the DAG. The final result response is sent to client via Frontend. The distribution of query fragment execution takes minimizing data movement and maximizing scan locality as the main goal.

== Background ==

At Baidu, Prior to Doris, different tools were deployed to solve diverse requirements in many ways. And when a use case requires the simultaneous availability of capabilities that cannot all be provided by a single tool, users were forced to build hybrid architectures that stitch multiple tools together, but we believe that they shouldn’t need to accept such inherent complexity. A storage system built to provide great performance across a broad range of workloads provides a more elegant solution to the problems that hybrid architectures aim to solve. Doris is the solution.

Doris is designed to be a simple and single tightly coupled system, not depending on other systems. Doris provides high concurrent low latency point query performance, but also provides high throughput queries of ad-hoc analysis. Doris provides bulk-batch data loading, but also provides near real-time mini-batch data loading. Doris also provides high availability, reliability, fault tolerance, and scalability.

== Rationale ==

Doris mainly integrates the technology of Google Mesa and Apache Impala.

Mesa is a highly scalable analytic data storage system that stores critical measurement data related to Google's Internet advertising business. Mesa is designed to satisfy complex and challenging set of users’ and systems’ requirements, including near real-time data ingestion and query ability, as well as high availability, reliability, fault tolerance, and scalability for large data and query volumes.

Impala is a modern, open-source MPP SQL engine architected from the ground up for the Hadoop data processing environment. At present, by virtue of its superior performance and rich functionality， Impala has been comparable to many commercial MPP database query engine. Mesa can satisfy the needs of many of our storage requirements, however Mesa itself does not provide a SQL query engine; Impala is a very good MPP SQL query engine, but the lack of a perfect distributed storage engine. So in the end we chose the combination of these two technologies.

Learning from Mesa’s data model, we developed a distributed storage engine. Unlike Mesa, this storage engine does not rely on any distributed file system. Then we deeply integrate this storage engine with Impala query engine. Query compiling, query execution coordination and catalog management of storage engine are integrated to be frontend daemon; query execution and data storage are integrated to be backend daemon. With this integration, we implemented a single, full-featured, high performance state the art of MPP database, as well as maintaining the simplicity.

== Current Status ==

Doris has been an open source project on GitHub (https://github.com/baidu/palo).

=== Meritocracy ===

Doris has been deployed in production at Baidu and is applying more than 200 lines of business. It has demonstrated great performance benefits and has proved to be a better way for reporting and analysis based big data. Still We look forward to growing a rich user and developer community.

=== Community ===

Doris seeks to develop developer and user communities during incubation.

Doris makes use of Apache Impala. It was identified during early review of the proposal that the Doris community will need to work with Impala to define a suitable API.

=== Core Developers ===

 * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
 * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
 * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
 * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
 * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
 * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
 * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)

=== Alignment ===

Doris is related to several other Apache projects:

 * Doris can also read data stored in Apache Hadoop clusters powered by the HDFS filesystem.
 * Doris is closely integrated with Impala, which has graduated from Apache Incubator.
 * Doris uses Apache Thrift as its RPC and serialization framework of choice.

== Known Risks ==

=== Orphaned Products ===

The core developers of Doris team plan to work full time on this project. There is very little risk of Doris getting orphaned since at least one large company (Baidu) is extensively using it in their production. For example, currently there are more than 200 use cases using Doris in production. Furthermore, since Doris was open sourced at the beginning of October 2017, it has received more than 660 stars and been forked nearly 170 times. We plan to extend and diversify this community further through Apache.

=== Inexperience with Open Source ===

The core developers are all active users and followers of open source. They are already committers and contributors to the Doris Github project. All have been involved with the source code that has been released under an open source license, and several of them also have experience developing code in an open source environment. Though the core set of Developers do not have Apache Open Source experience, there are plans to onboard individuals with Apache open source experience on to the project.

=== Homogenous Developers ===

The most of core developers are from Baidu, but after Doris was open sourced, Doris received a lot of bug fixes and enhancements from other developers not working at Baidu.

=== Reliance on Salaried Developers ===

Baidu invested in Doris as the OLAP solution and some of its key engineers are working full time on the project. In addition, since there is a growing Big Data need for scalable OLAP solutions, we look forward to other Apache developers and researchers to contribute to the project. Also key to addressing the risk associated with relying on Salaried developers from a single entity is to increase the diversity of the contributors and actively lobby for Domain experts in the BI space to contribute. Apache Doris intends to do this.

=== An Excessive Fascination with the Apache Brand ===

Doris is proposing to enter incubation at Apache in order to help efforts to diversify the committer-base, not so much to capitalize on the Apache brand. The Doris project is in production use already inside Baidu, but is not expected to be an Baidu product for external customers. As such, the Doris project is not seeking to use the Apache brand as a marketing tool.

== Documentation ==

Information about Doris can be found at https://github.com/baidu/palo. The following links provide more information about Doris in open source:

 * Doris wiki site: https://github.com/baidu/palo/wiki
 * Codebase at Github: https://github.com/baidu/palo
 * Issue Tracking: https://github.com/baidu/palo/issues
 * Overview: https://github.com/baidu/Doris/wiki/palo-Overview
 * FAQ: https://github.com/baidu/palo/wiki/palo-FAQ

== Initial Source ==

Doris has been under development since 2017 by a team of engineers at Baidu Inc. It is currently hosted on Github.com<http://Github.com> under an Apache license at https://github.com/baidu/palo.

== External Dependencies ==

Doris has the following external dependencies.

 * Google gflags (BSD)
 * Google glog (BSD)
 * Apache Thrift (Apache Software License v2.0)
 * Apache Commons (Apache Software License v2.0)
 * Boost (Boost Software License)
 * rapidjson (Tencent)
 * Google RE2 (BSD-style)
 * lz4 (BSD)
 * snappy (BSD)
 * Twitter Bootstrap (Apache Software License v2.0)
 * d3 (BSD)
 * LLVM (BSD-like)

Build and test dependencies:

 * Apache Ant (Apache Software License v2.0)
 * Apache Maven (Apache Software License v2.0)
 * cmake (BSD)
 * clang (BSD)
 * Google gtest (Apache Software License v2.0)

== Required Resources ==

=== Mailing List ===

There are currently no mailing lists. The usual mailing lists are expected to be set up when entering incubation:

 * private@doris.incubator.apache.org<ma...@doris.incubator.apache.org>
 * dev@doris.incubator.apache.org<ma...@doris.incubator.apache.org>
 * commits@doris.incubator.apache.org<ma...@doris.incubator.apache.org>

=== Subversion Directory ===

Upon entering incubation, we want to move (or copy) the existing repo from https://github.com/baidu/palo to Apache infrastructure at https://github.com/apache/incubator-doris.

=== Issue Tracking ===

Doris currently uses GitHub to track issues. Would like to continue to do so while we discuss migration possibilities with the ASF Infra committee.

=== Other Resources ===

The existing code already has unit tests so we will make use of existing Apache continuous testing infrastructure. The resulting load should not be very large.

== Initial Committers ==

 * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
 * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
 * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
 * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
 * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
 * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
 * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
 * Sijie Guo (guosijie@gmail dot com)
 * Zheng Shao (zshao@apache.org<ma...@apache.org>)

== Affiliations ==

The initial committers are employees of Baidu Inc..

== Sponsors ==

=== Champion ===

 * Dave Fisher, wave@apache.org<ma...@apache.org>

=== Nominated Mentors ===

 * Luke Han, lukehan@apache.org<ma...@apache.org>
 * Dave Fisher, wave@apache.org<ma...@apache.org>
 * Willem Jiang, ningjiang@apache.org<ma...@apache.org>

=== Sponsoring Entity ===

We are requesting the Incubator to sponsor this project.

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by Dave Fisher <da...@comcast.net>.

Here is my +1 (binding)

> On Jul 5, 2018, at 12:22 PM, Dave Fisher <da...@comcast.net> wrote:
> 
> Hi All,
> 
> I would like to start a VOTE to bring the Doris project as an Apache incubator podling.
> 
> The ASF voting rules are described:
> 
> https://www.apache.org/foundation/voting.html <https://www.apache.org/foundation/voting.html>
> 
> A vote for accepting a new Apache Incubator podling is a majority vote for which only Incubator PMC member votes are binding.
> 
> This vote will run for at least 72 hours. Please VOTE as follows
> [] +1 Accept Doris into the Apache Incubator
> [] +0 Abstain.
> [] -1 Do not accept Doris into the Apache Incubator because ...
> 
> The proposal is listed below, but you can also access it on the wiki:
> 
> https://wiki.apache.org/incubator/DorisProposal <https://wiki.apache.org/incubator/DorisProposal>
> 
> Best regards,
> Dave
> 
> = Apache Doris =
> 
> == Abstract ==
> 
> Doris is a MPP-based interactive SQL data warehousing for reporting and analysis.
> 
> == Proposal ==
> 
> We propose to contribute the Doris codebase and associated artifacts (e.g. documentation, web-site content etc.) to the Apache Software Foundation, and aim to build an open community around Doris’s continued development in the ‘Apache Way’.
> 
> === Overview of Doris ===
> 
> Doris’s implementation consists of two daemons: Frontend (FE) and Backend (BE).
> 
> **Frontend daemon** consists of query coordinator and catalog manager. Query coordinator is responsible for receiving users’ sql queries, compiling queries and managing queries execution. Catalog manager is responsible for managing metadata such as databases, tables, partitions, replicas and etc. Several frontend daemons could be deployed to guarantee fault-tolerance, and load balancing.
> 
> **Backend daemon** stores the data and executes the query fragments. Many backend daemons could also be deployed to provide scalability and fault-tolerance.
> 
> A typical Doris cluster generally composes of several frontend daemons and dozens to hundreds of backend daemons.
> 
> Users can use MySQL client tools to connect any frontend daemon to submit SQL query. Frontend receives the query and compiles it into query plans executable by the Backend. Then Frontend sends the query plan fragments to Backend. Backend will build a query execution DAG. Data is fetched and pipelined into the DAG. The final result response is sent to client via Frontend. The distribution of query fragment execution takes minimizing data movement and maximizing scan locality as the main goal.
> 
> == Background ==
> 
> At Baidu, Prior to Doris, different tools were deployed to solve diverse requirements in many ways. And when a use case requires the simultaneous availability of capabilities that cannot all be provided by a single tool, users were forced to build hybrid architectures that stitch multiple tools together, but we believe that they shouldn’t need to accept such inherent complexity. A storage system built to provide great performance across a broad range of workloads provides a more elegant solution to the problems that hybrid architectures aim to solve. Doris is the solution.
> 
> Doris is designed to be a simple and single tightly coupled system, not depending on other systems. Doris provides high concurrent low latency point query performance, but also provides high throughput queries of ad-hoc analysis. Doris provides bulk-batch data loading, but also provides near real-time mini-batch data loading. Doris also provides high availability, reliability, fault tolerance, and scalability.
> 
> == Rationale ==
> 
> Doris mainly integrates the technology of Google Mesa and Apache Impala.
> 
> Mesa is a highly scalable analytic data storage system that stores critical measurement data related to Google's Internet advertising business. Mesa is designed to satisfy complex and challenging set of users’ and systems’ requirements, including near real-time data ingestion and query ability, as well as high availability, reliability, fault tolerance, and scalability for large data and query volumes.
> 
> Impala is a modern, open-source MPP SQL engine architected from the ground up for the Hadoop data processing environment. At present, by virtue of its superior performance and rich functionality， Impala has been comparable to many commercial MPP database query engine. Mesa can satisfy the needs of many of our storage requirements, however Mesa itself does not provide a SQL query engine; Impala is a very good MPP SQL query engine, but the lack of a perfect distributed storage engine. So in the end we chose the combination of these two technologies.
> 
> Learning from Mesa’s data model, we developed a distributed storage engine. Unlike Mesa, this storage engine does not rely on any distributed file system. Then we deeply integrate this storage engine with Impala query engine. Query compiling, query execution coordination and catalog management of storage engine are integrated to be frontend daemon; query execution and data storage are integrated to be backend daemon. With this integration, we implemented a single, full-featured, high performance state the art of MPP database, as well as maintaining the simplicity.
> 
> == Current Status ==
> 
> Doris has been an open source project on GitHub (https://github.com/baidu/palo <https://github.com/baidu/palo>).
> 
> === Meritocracy ===
> 
> Doris has been deployed in production at Baidu and is applying more than 200 lines of business. It has demonstrated great performance benefits and has proved to be a better way for reporting and analysis based big data. Still We look forward to growing a rich user and developer community.
> 
> === Community ===
> 
> Doris seeks to develop developer and user communities during incubation.
> 
> Doris makes use of Apache Impala. It was identified during early review of the proposal that the Doris community will need to work with Impala to define a suitable API.
> 
> === Core Developers ===
> 
>  * Ruyue Ma (https://github.com/maruyue <https://github.com/maruyue>, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay <https://github.com/imay>, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu <ht...@baidu> dot com)
>  * De Li（https://github.com/lide-reed <https://github.com/lide-reed>, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886 <https://github.com/chenhao7253886>, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli <https://github.com/cyongli>, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin <https://github.com/lingbin>, lingbinlb@gmail dot com)
> 
> === Alignment ===
> 
> Doris is related to several other Apache projects:
> 
>  * Doris can also read data stored in Apache Hadoop clusters powered by the HDFS filesystem.
>  * Doris is closely integrated with Impala, which has graduated from Apache Incubator.
>  * Doris uses Apache Thrift as its RPC and serialization framework of choice.
> 
> == Known Risks ==
> 
> === Orphaned Products ===
> 
> The core developers of Doris team plan to work full time on this project. There is very little risk of Doris getting orphaned since at least one large company (Baidu) is extensively using it in their production. For example, currently there are more than 200 use cases using Doris in production. Furthermore, since Doris was open sourced at the beginning of October 2017, it has received more than 660 stars and been forked nearly 170 times. We plan to extend and diversify this community further through Apache.
> 
> === Inexperience with Open Source ===
> 
> The core developers are all active users and followers of open source. They are already committers and contributors to the Doris Github project. All have been involved with the source code that has been released under an open source license, and several of them also have experience developing code in an open source environment. Though the core set of Developers do not have Apache Open Source experience, there are plans to onboard individuals with Apache open source experience on to the project.
> 
> === Homogenous Developers ===
> 
> The most of core developers are from Baidu, but after Doris was open sourced, Doris received a lot of bug fixes and enhancements from other developers not working at Baidu.
> 
> === Reliance on Salaried Developers ===
> 
> Baidu invested in Doris as the OLAP solution and some of its key engineers are working full time on the project. In addition, since there is a growing Big Data need for scalable OLAP solutions, we look forward to other Apache developers and researchers to contribute to the project. Also key to addressing the risk associated with relying on Salaried developers from a single entity is to increase the diversity of the contributors and actively lobby for Domain experts in the BI space to contribute. Apache Doris intends to do this.
> 
> === An Excessive Fascination with the Apache Brand ===
> 
> Doris is proposing to enter incubation at Apache in order to help efforts to diversify the committer-base, not so much to capitalize on the Apache brand. The Doris project is in production use already inside Baidu, but is not expected to be an Baidu product for external customers. As such, the Doris project is not seeking to use the Apache brand as a marketing tool.
> 
> == Documentation ==
> 
> Information about Doris can be found at https://github.com/baidu/palo <https://github.com/baidu/palo>. The following links provide more information about Doris in open source:
> 
>  * Doris wiki site: https://github.com/baidu/palo/wiki <https://github.com/baidu/palo/wiki>
>  * Codebase at Github: https://github.com/baidu/palo <https://github.com/baidu/palo>
>  * Issue Tracking: https://github.com/baidu/palo/issues <https://github.com/baidu/palo/issues>
>  * Overview: https://github.com/baidu/Doris/wiki/palo-Overview <https://github.com/baidu/Doris/wiki/palo-Overview>
>  * FAQ: https://github.com/baidu/palo/wiki/palo-FAQ <https://github.com/baidu/palo/wiki/palo-FAQ>
> 
> == Initial Source ==
> 
> Doris has been under development since 2017 by a team of engineers at Baidu Inc. It is currently hosted on Github.com <http://github.com/> under an Apache license at https://github.com/baidu/palo <https://github.com/baidu/palo>.
> 
> == External Dependencies ==
> 
> Doris has the following external dependencies.
> 
>  * Google gflags (BSD)
>  * Google glog (BSD)
>  * Apache Thrift (Apache Software License v2.0)
>  * Apache Commons (Apache Software License v2.0)
>  * Boost (Boost Software License)
>  * rapidjson (Tencent)
>  * Google RE2 (BSD-style)
>  * lz4 (BSD)
>  * snappy (BSD)
>  * Twitter Bootstrap (Apache Software License v2.0)
>  * d3 (BSD)
>  * LLVM (BSD-like)
> 
> Build and test dependencies:
> 
>  * Apache Ant (Apache Software License v2.0)
>  * Apache Maven (Apache Software License v2.0)
>  * cmake (BSD)
>  * clang (BSD)
>  * Google gtest (Apache Software License v2.0)
> 
> == Required Resources ==
> 
> === Mailing List ===
> 
> There are currently no mailing lists. The usual mailing lists are expected to be set up when entering incubation:
> 
>  * private@doris.incubator.apache.org <ma...@doris.incubator.apache.org>
>  * dev@doris.incubator.apache.org <ma...@doris.incubator.apache.org>
>  * commits@doris.incubator.apache.org <ma...@doris.incubator.apache.org>
> 
> === Subversion Directory ===
> 
> Upon entering incubation, we want to move (or copy) the existing repo from https://github.com/baidu/palo <https://github.com/baidu/palo> to Apache infrastructure at https://github.com/apache/incubator-doris <https://github.com/apache/incubator-doris>.
> 
> === Issue Tracking ===
> 
> Doris currently uses GitHub to track issues. Would like to continue to do so while we discuss migration possibilities with the ASF Infra committee.
> 
> === Other Resources ===
> 
> The existing code already has unit tests so we will make use of existing Apache continuous testing infrastructure. The resulting load should not be very large.
> 
> == Initial Committers ==
> 
>  * Ruyue Ma (https://github.com/maruyue <https://github.com/maruyue>, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay <https://github.com/imay>, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu <ht...@baidu> dot com)
>  * De Li（https://github.com/lide-reed <https://github.com/lide-reed>, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886 <https://github.com/chenhao7253886>, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli <https://github.com/cyongli>, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin <https://github.com/lingbin>, lingbinlb@gmail dot com)
>  * Sijie Guo (guosijie@gmail dot com)
>  * Zheng Shao (zshao@apache.org <ma...@apache.org>)
> 
> == Affiliations ==
> 
> The initial committers are employees of Baidu Inc..
> 
> == Sponsors ==
> 
> === Champion ===
> 
>  * Dave Fisher, wave@apache.org <ma...@apache.org>
> 
> === Nominated Mentors ===
> 
>  * Luke Han, lukehan@apache.org <ma...@apache.org>
>  * Dave Fisher, wave@apache.org <ma...@apache.org>
>  * Willem Jiang, ningjiang@apache.org <ma...@apache.org>
> 
> === Sponsoring Entity ===
> 
> We are requesting the Incubator to sponsor this project.
>

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by Sijie Guo <gu...@gmail.com>.

+1 (non-binding)

It is a good project to incubate in the ASF. Doris (Palo) team was very
responsive to the questions raised in the discussion. I believe with the
help from Mentors, it can become a great project in the ASF.

- Sijie

On Thu, Jul 5, 2018 at 12:22 PM Dave Fisher <da...@comcast.net> wrote:

> Hi All,
>
> I would like to start a VOTE to bring the Doris project as an Apache
> incubator podling.
>
> The ASF voting rules are described:
>
> https://www.apache.org/foundation/voting.html
>
> A vote for accepting a new Apache Incubator podling is a majority vote for which
> only Incubator PMC member votes are binding.
>
> This vote will run for at least 72 hours. Please VOTE as follows
> [] +1 Accept Doris into the Apache Incubator
> [] +0 Abstain.
> [] -1 Do not accept Doris into the Apache Incubator because ...
>
> The proposal is listed below, but you can also access it on the wiki:
>
> https://wiki.apache.org/incubator/DorisProposal
>
> Best regards,
> Dave
>
> = Apache Doris =
>
> == Abstract ==
>
> Doris is a MPP-based interactive SQL data warehousing for reporting and
> analysis.
>
> == Proposal ==
>
> We propose to contribute the Doris codebase and associated artifacts (e.g.
> documentation, web-site content etc.) to the Apache Software Foundation,
> and aim to build an open community around Doris’s continued development in
> the ‘Apache Way’.
>
> === Overview of Doris ===
>
> Doris’s implementation consists of two daemons: Frontend (FE) and Backend
> (BE).
>
> **Frontend daemon** consists of query coordinator and catalog manager.
> Query coordinator is responsible for receiving users’ sql queries,
> compiling queries and managing queries execution. Catalog manager is
> responsible for managing metadata such as databases, tables, partitions,
> replicas and etc. Several frontend daemons could be deployed to guarantee
> fault-tolerance, and load balancing.
>
> **Backend daemon** stores the data and executes the query fragments. Many
> backend daemons could also be deployed to provide scalability and
> fault-tolerance.
>
> A typical Doris cluster generally composes of several frontend daemons and
> dozens to hundreds of backend daemons.
>
> Users can use MySQL client tools to connect any frontend daemon to submit
> SQL query. Frontend receives the query and compiles it into query plans
> executable by the Backend. Then Frontend sends the query plan fragments to
> Backend. Backend will build a query execution DAG. Data is fetched and
> pipelined into the DAG. The final result response is sent to client via
> Frontend. The distribution of query fragment execution takes minimizing
> data movement and maximizing scan locality as the main goal.
>
> == Background ==
>
> At Baidu, Prior to Doris, different tools were deployed to solve diverse
> requirements in many ways. And when a use case requires the simultaneous
> availability of capabilities that cannot all be provided by a single tool,
> users were forced to build hybrid architectures that stitch multiple tools
> together, but we believe that they shouldn’t need to accept such inherent
> complexity. A storage system built to provide great performance across a
> broad range of workloads provides a more elegant solution to the problems
> that hybrid architectures aim to solve. Doris is the solution.
>
> Doris is designed to be a simple and single tightly coupled system, not
> depending on other systems. Doris provides high concurrent low latency
> point query performance, but also provides high throughput queries of
> ad-hoc analysis. Doris provides bulk-batch data loading, but also provides
> near real-time mini-batch data loading. Doris also provides high
> availability, reliability, fault tolerance, and scalability.
>
> == Rationale ==
>
> Doris mainly integrates the technology of Google Mesa and Apache Impala.
>
> Mesa is a highly scalable analytic data storage system that stores
> critical measurement data related to Google's Internet advertising
> business. Mesa is designed to satisfy complex and challenging set of users’
> and systems’ requirements, including near real-time data ingestion and
> query ability, as well as high availability, reliability, fault tolerance,
> and scalability for large data and query volumes.
>
> Impala is a modern, open-source MPP SQL engine architected from the ground
> up for the Hadoop data processing environment. At present, by virtue of its
> superior performance and rich functionality， Impala has been comparable to
> many commercial MPP database query engine. Mesa can satisfy the needs of
> many of our storage requirements, however Mesa itself does not provide a
> SQL query engine; Impala is a very good MPP SQL query engine, but the lack
> of a perfect distributed storage engine. So in the end we chose the
> combination of these two technologies.
>
> Learning from Mesa’s data model, we developed a distributed storage
> engine. Unlike Mesa, this storage engine does not rely on any distributed
> file system. Then we deeply integrate this storage engine with Impala query
> engine. Query compiling, query execution coordination and catalog
> management of storage engine are integrated to be frontend daemon; query
> execution and data storage are integrated to be backend daemon. With this
> integration, we implemented a single, full-featured, high performance state
> the art of MPP database, as well as maintaining the simplicity.
>
> == Current Status ==
>
> Doris has been an open source project on GitHub (
> https://github.com/baidu/palo).
>
> === Meritocracy ===
>
> Doris has been deployed in production at Baidu and is applying more than
> 200 lines of business. It has demonstrated great performance benefits and
> has proved to be a better way for reporting and analysis based big data.
> Still We look forward to growing a rich user and developer community.
>
> === Community ===
>
> Doris seeks to develop developer and user communities during incubation.
>
> Doris makes use of Apache Impala. It was identified during early review of
> the proposal that the Doris community will need to work with Impala to
> define a suitable API.
>
> === Core Developers ===
>
>  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
>  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
>
> === Alignment ===
>
> Doris is related to several other Apache projects:
>
>  * Doris can also read data stored in Apache Hadoop clusters powered by
> the HDFS filesystem.
>  * Doris is closely integrated with Impala, which has graduated from
> Apache Incubator.
>  * Doris uses Apache Thrift as its RPC and serialization framework of
> choice.
>
> == Known Risks ==
>
> === Orphaned Products ===
>
> The core developers of Doris team plan to work full time on this project.
> There is very little risk of Doris getting orphaned since at least one
> large company (Baidu) is extensively using it in their production. For
> example, currently there are more than 200 use cases using Doris in
> production. Furthermore, since Doris was open sourced at the beginning of
> October 2017, it has received more than 660 stars and been forked nearly
> 170 times. We plan to extend and diversify this community further through
> Apache.
>
> === Inexperience with Open Source ===
>
> The core developers are all active users and followers of open source.
> They are already committers and contributors to the Doris Github project.
> All have been involved with the source code that has been released under an
> open source license, and several of them also have experience developing
> code in an open source environment. Though the core set of Developers do
> not have Apache Open Source experience, there are plans to onboard
> individuals with Apache open source experience on to the project.
>
> === Homogenous Developers ===
>
> The most of core developers are from Baidu, but after Doris was open
> sourced, Doris received a lot of bug fixes and enhancements from other
> developers not working at Baidu.
>
> === Reliance on Salaried Developers ===
>
> Baidu invested in Doris as the OLAP solution and some of its key engineers
> are working full time on the project. In addition, since there is a growing
> Big Data need for scalable OLAP solutions, we look forward to other Apache
> developers and researchers to contribute to the project. Also key to
> addressing the risk associated with relying on Salaried developers from a
> single entity is to increase the diversity of the contributors and actively
> lobby for Domain experts in the BI space to contribute. Apache Doris
> intends to do this.
>
> === An Excessive Fascination with the Apache Brand ===
>
> Doris is proposing to enter incubation at Apache in order to help efforts
> to diversify the committer-base, not so much to capitalize on the Apache
> brand. The Doris project is in production use already inside Baidu, but is
> not expected to be an Baidu product for external customers. As such, the
> Doris project is not seeking to use the Apache brand as a marketing tool.
>
> == Documentation ==
>
> Information about Doris can be found at https://github.com/baidu/palo.
> The following links provide more information about Doris in open source:
>
>  * Doris wiki site: https://github.com/baidu/palo/wiki
>  * Codebase at Github: https://github.com/baidu/palo
>  * Issue Tracking: https://github.com/baidu/palo/issues
>  * Overview: https://github.com/baidu/Doris/wiki/palo-Overview
>  * FAQ: https://github.com/baidu/palo/wiki/palo-FAQ
>
> == Initial Source ==
>
> Doris has been under development since 2017 by a team of engineers at
> Baidu Inc. It is currently hosted on Github.com under an Apache license
> at https://github.com/baidu/palo.
>
> == External Dependencies ==
>
> Doris has the following external dependencies.
>
>  * Google gflags (BSD)
>  * Google glog (BSD)
>  * Apache Thrift (Apache Software License v2.0)
>  * Apache Commons (Apache Software License v2.0)
>  * Boost (Boost Software License)
>  * rapidjson (Tencent)
>  * Google RE2 (BSD-style)
>  * lz4 (BSD)
>  * snappy (BSD)
>  * Twitter Bootstrap (Apache Software License v2.0)
>  * d3 (BSD)
>  * LLVM (BSD-like)
>
> Build and test dependencies:
>
>  * Apache Ant (Apache Software License v2.0)
>  * Apache Maven (Apache Software License v2.0)
>  * cmake (BSD)
>  * clang (BSD)
>  * Google gtest (Apache Software License v2.0)
>
> == Required Resources ==
>
> === Mailing List ===
>
> There are currently no mailing lists. The usual mailing lists are expected
> to be set up when entering incubation:
>
>  * private@doris.incubator.apache.org
>  * dev@doris.incubator.apache.org
>  * commits@doris.incubator.apache.org
>
> === Subversion Directory ===
>
> Upon entering incubation, we want to move (or copy) the existing repo from
> https://github.com/baidu/palo to Apache infrastructure at
> https://github.com/apache/incubator-doris.
>
> === Issue Tracking ===
>
> Doris currently uses GitHub to track issues. Would like to continue to do
> so while we discuss migration possibilities with the ASF Infra committee.
>
> === Other Resources ===
>
> The existing code already has unit tests so we will make use of existing
> Apache continuous testing infrastructure. The resulting load should not be
> very large.
>
> == Initial Committers ==
>
>  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
>  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
>  * Sijie Guo (guosijie@gmail dot com)
>  * Zheng Shao (zshao@apache.org)
>
> == Affiliations ==
>
> The initial committers are employees of Baidu Inc..
>
> == Sponsors ==
>
> === Champion ===
>
>  * Dave Fisher, wave@apache.org
>
> === Nominated Mentors ===
>
>  * Luke Han, lukehan@apache.org
>  * Dave Fisher, wave@apache.org
>  * Willem Jiang, ningjiang@apache.org
>
> === Sponsoring Entity ===
>
> We are requesting the Incubator to sponsor this project.
>
>

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by "Kevin A. McGrail" <km...@apache.org>.

+1 Binding

A minor note:  Things like MPP should be spelled out at least once.  Your
proposal was technical and it helps if you think about trying to explain it
to an engineer who doesn't now Impala or Mesos or SQL, etc.

--
Kevin A. McGrail
VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

On Thu, Jul 5, 2018 at 3:22 PM, Dave Fisher <da...@comcast.net> wrote:

> Hi All,
>
> I would like to start a VOTE to bring the Doris project as an Apache
> incubator podling.
>
> The ASF voting rules are described:
>
> https://www.apache.org/foundation/voting.html
>
> A vote for accepting a new Apache Incubator podling is a majority vote for which
> only Incubator PMC member votes are binding.
>
> This vote will run for at least 72 hours. Please VOTE as follows
> [] +1 Accept Doris into the Apache Incubator
> [] +0 Abstain.
> [] -1 Do not accept Doris into the Apache Incubator because ...
>
> The proposal is listed below, but you can also access it on the wiki:
>
> https://wiki.apache.org/incubator/DorisProposal
>
> Best regards,
> Dave
>
> = Apache Doris =
>
> == Abstract ==
>
> Doris is a MPP-based interactive SQL data warehousing for reporting and
> analysis.
>
> == Proposal ==
>
> We propose to contribute the Doris codebase and associated artifacts (e.g.
> documentation, web-site content etc.) to the Apache Software Foundation,
> and aim to build an open community around Doris’s continued development in
> the ‘Apache Way’.
>
> === Overview of Doris ===
>
> Doris’s implementation consists of two daemons: Frontend (FE) and Backend
> (BE).
>
> **Frontend daemon** consists of query coordinator and catalog manager.
> Query coordinator is responsible for receiving users’ sql queries,
> compiling queries and managing queries execution. Catalog manager is
> responsible for managing metadata such as databases, tables, partitions,
> replicas and etc. Several frontend daemons could be deployed to guarantee
> fault-tolerance, and load balancing.
>
> **Backend daemon** stores the data and executes the query fragments. Many
> backend daemons could also be deployed to provide scalability and
> fault-tolerance.
>
> A typical Doris cluster generally composes of several frontend daemons and
> dozens to hundreds of backend daemons.
>
> Users can use MySQL client tools to connect any frontend daemon to submit
> SQL query. Frontend receives the query and compiles it into query plans
> executable by the Backend. Then Frontend sends the query plan fragments to
> Backend. Backend will build a query execution DAG. Data is fetched and
> pipelined into the DAG. The final result response is sent to client via
> Frontend. The distribution of query fragment execution takes minimizing
> data movement and maximizing scan locality as the main goal.
>
> == Background ==
>
> At Baidu, Prior to Doris, different tools were deployed to solve diverse
> requirements in many ways. And when a use case requires the simultaneous
> availability of capabilities that cannot all be provided by a single tool,
> users were forced to build hybrid architectures that stitch multiple tools
> together, but we believe that they shouldn’t need to accept such inherent
> complexity. A storage system built to provide great performance across a
> broad range of workloads provides a more elegant solution to the problems
> that hybrid architectures aim to solve. Doris is the solution.
>
> Doris is designed to be a simple and single tightly coupled system, not
> depending on other systems. Doris provides high concurrent low latency
> point query performance, but also provides high throughput queries of
> ad-hoc analysis. Doris provides bulk-batch data loading, but also provides
> near real-time mini-batch data loading. Doris also provides high
> availability, reliability, fault tolerance, and scalability.
>
> == Rationale ==
>
> Doris mainly integrates the technology of Google Mesa and Apache Impala.
>
> Mesa is a highly scalable analytic data storage system that stores
> critical measurement data related to Google's Internet advertising
> business. Mesa is designed to satisfy complex and challenging set of users’
> and systems’ requirements, including near real-time data ingestion and
> query ability, as well as high availability, reliability, fault tolerance,
> and scalability for large data and query volumes.
>
> Impala is a modern, open-source MPP SQL engine architected from the ground
> up for the Hadoop data processing environment. At present, by virtue of its
> superior performance and rich functionality， Impala has been comparable to
> many commercial MPP database query engine. Mesa can satisfy the needs of
> many of our storage requirements, however Mesa itself does not provide a
> SQL query engine; Impala is a very good MPP SQL query engine, but the lack
> of a perfect distributed storage engine. So in the end we chose the
> combination of these two technologies.
>
> Learning from Mesa’s data model, we developed a distributed storage
> engine. Unlike Mesa, this storage engine does not rely on any distributed
> file system. Then we deeply integrate this storage engine with Impala query
> engine. Query compiling, query execution coordination and catalog
> management of storage engine are integrated to be frontend daemon; query
> execution and data storage are integrated to be backend daemon. With this
> integration, we implemented a single, full-featured, high performance state
> the art of MPP database, as well as maintaining the simplicity.
>
> == Current Status ==
>
> Doris has been an open source project on GitHub (
> https://github.com/baidu/palo).
>
> === Meritocracy ===
>
> Doris has been deployed in production at Baidu and is applying more than
> 200 lines of business. It has demonstrated great performance benefits and
> has proved to be a better way for reporting and analysis based big data.
> Still We look forward to growing a rich user and developer community.
>
> === Community ===
>
> Doris seeks to develop developer and user communities during incubation.
>
> Doris makes use of Apache Impala. It was identified during early review of
> the proposal that the Doris community will need to work with Impala to
> define a suitable API.
>
> === Core Developers ===
>
>  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
>  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
>
> === Alignment ===
>
> Doris is related to several other Apache projects:
>
>  * Doris can also read data stored in Apache Hadoop clusters powered by
> the HDFS filesystem.
>  * Doris is closely integrated with Impala, which has graduated from
> Apache Incubator.
>  * Doris uses Apache Thrift as its RPC and serialization framework of
> choice.
>
> == Known Risks ==
>
> === Orphaned Products ===
>
> The core developers of Doris team plan to work full time on this project.
> There is very little risk of Doris getting orphaned since at least one
> large company (Baidu) is extensively using it in their production. For
> example, currently there are more than 200 use cases using Doris in
> production. Furthermore, since Doris was open sourced at the beginning of
> October 2017, it has received more than 660 stars and been forked nearly
> 170 times. We plan to extend and diversify this community further through
> Apache.
>
> === Inexperience with Open Source ===
>
> The core developers are all active users and followers of open source.
> They are already committers and contributors to the Doris Github project.
> All have been involved with the source code that has been released under an
> open source license, and several of them also have experience developing
> code in an open source environment. Though the core set of Developers do
> not have Apache Open Source experience, there are plans to onboard
> individuals with Apache open source experience on to the project.
>
> === Homogenous Developers ===
>
> The most of core developers are from Baidu, but after Doris was open
> sourced, Doris received a lot of bug fixes and enhancements from other
> developers not working at Baidu.
>
> === Reliance on Salaried Developers ===
>
> Baidu invested in Doris as the OLAP solution and some of its key engineers
> are working full time on the project. In addition, since there is a growing
> Big Data need for scalable OLAP solutions, we look forward to other Apache
> developers and researchers to contribute to the project. Also key to
> addressing the risk associated with relying on Salaried developers from a
> single entity is to increase the diversity of the contributors and actively
> lobby for Domain experts in the BI space to contribute. Apache Doris
> intends to do this.
>
> === An Excessive Fascination with the Apache Brand ===
>
> Doris is proposing to enter incubation at Apache in order to help efforts
> to diversify the committer-base, not so much to capitalize on the Apache
> brand. The Doris project is in production use already inside Baidu, but is
> not expected to be an Baidu product for external customers. As such, the
> Doris project is not seeking to use the Apache brand as a marketing tool.
>
> == Documentation ==
>
> Information about Doris can be found at https://github.com/baidu/palo.
> The following links provide more information about Doris in open source:
>
>  * Doris wiki site: https://github.com/baidu/palo/wiki
>  * Codebase at Github: https://github.com/baidu/palo
>  * Issue Tracking: https://github.com/baidu/palo/issues
>  * Overview: https://github.com/baidu/Doris/wiki/palo-Overview
>  * FAQ: https://github.com/baidu/palo/wiki/palo-FAQ
>
> == Initial Source ==
>
> Doris has been under development since 2017 by a team of engineers at
> Baidu Inc. It is currently hosted on Github.com under an Apache license
> at https://github.com/baidu/palo.
>
> == External Dependencies ==
>
> Doris has the following external dependencies.
>
>  * Google gflags (BSD)
>  * Google glog (BSD)
>  * Apache Thrift (Apache Software License v2.0)
>  * Apache Commons (Apache Software License v2.0)
>  * Boost (Boost Software License)
>  * rapidjson (Tencent)
>  * Google RE2 (BSD-style)
>  * lz4 (BSD)
>  * snappy (BSD)
>  * Twitter Bootstrap (Apache Software License v2.0)
>  * d3 (BSD)
>  * LLVM (BSD-like)
>
> Build and test dependencies:
>
>  * Apache Ant (Apache Software License v2.0)
>  * Apache Maven (Apache Software License v2.0)
>  * cmake (BSD)
>  * clang (BSD)
>  * Google gtest (Apache Software License v2.0)
>
> == Required Resources ==
>
> === Mailing List ===
>
> There are currently no mailing lists. The usual mailing lists are expected
> to be set up when entering incubation:
>
>  * private@doris.incubator.apache.org
>  * dev@doris.incubator.apache.org
>  * commits@doris.incubator.apache.org
>
> === Subversion Directory ===
>
> Upon entering incubation, we want to move (or copy) the existing repo from
> https://github.com/baidu/palo to Apache infrastructure at
> https://github.com/apache/incubator-doris.
>
> === Issue Tracking ===
>
> Doris currently uses GitHub to track issues. Would like to continue to do
> so while we discuss migration possibilities with the ASF Infra committee.
>
> === Other Resources ===
>
> The existing code already has unit tests so we will make use of existing
> Apache continuous testing infrastructure. The resulting load should not be
> very large.
>
> == Initial Committers ==
>
>  * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
>  * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
>  * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
>  * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
>  * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
>  * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
>  * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
>  * Sijie Guo (guosijie@gmail dot com)
>  * Zheng Shao (zshao@apache.org)
>
> == Affiliations ==
>
> The initial committers are employees of Baidu Inc..
>
> == Sponsors ==
>
> === Champion ===
>
>  * Dave Fisher, wave@apache.org
>
> === Nominated Mentors ===
>
>  * Luke Han, lukehan@apache.org
>  * Dave Fisher, wave@apache.org
>  * Willem Jiang, ningjiang@apache.org
>
> === Sponsoring Entity ===
>
> We are requesting the Incubator to sponsor this project.
>
>

Re: [VOTE] Accept Doris into the Apache Incubator

Posted by "Tan,Zhongyi" <ta...@baidu.com>.

Hi, dave,
Could this vote be closed ?
It seems that it has passed half a week since July 6th,
Thanks.


发件人: Dave Fisher <da...@comcast.net>>
答复: <ge...@incubator.apache.org>>
日期: 2018年7月6日 星期五 上午3:22
至: <ge...@incubator.apache.org>>
主题: [VOTE] Accept Doris into the Apache Incubator

Hi All,

I would like to start a VOTE to bring the Doris project as an Apache incubator podling.

The ASF voting rules are described:

https://www.apache.org/foundation/voting.html

A vote for accepting a new Apache Incubator podling is a majority vote for which only Incubator PMC member votes are binding.

This vote will run for at least 72 hours. Please VOTE as follows
[] +1 Accept Doris into the Apache Incubator
[] +0 Abstain.
[] -1 Do not accept Doris into the Apache Incubator because ...

The proposal is listed below, but you can also access it on the wiki:

https://wiki.apache.org/incubator/DorisProposal

Best regards,
Dave

= Apache Doris =

== Abstract ==

Doris is a MPP-based interactive SQL data warehousing for reporting and analysis.

== Proposal ==

We propose to contribute the Doris codebase and associated artifacts (e.g. documentation, web-site content etc.) to the Apache Software Foundation, and aim to build an open community around Doris’s continued development in the ‘Apache Way’.

=== Overview of Doris ===

Doris’s implementation consists of two daemons: Frontend (FE) and Backend (BE).

**Frontend daemon** consists of query coordinator and catalog manager. Query coordinator is responsible for receiving users’ sql queries, compiling queries and managing queries execution. Catalog manager is responsible for managing metadata such as databases, tables, partitions, replicas and etc. Several frontend daemons could be deployed to guarantee fault-tolerance, and load balancing.

**Backend daemon** stores the data and executes the query fragments. Many backend daemons could also be deployed to provide scalability and fault-tolerance.

A typical Doris cluster generally composes of several frontend daemons and dozens to hundreds of backend daemons.

Users can use MySQL client tools to connect any frontend daemon to submit SQL query. Frontend receives the query and compiles it into query plans executable by the Backend. Then Frontend sends the query plan fragments to Backend. Backend will build a query execution DAG. Data is fetched and pipelined into the DAG. The final result response is sent to client via Frontend. The distribution of query fragment execution takes minimizing data movement and maximizing scan locality as the main goal.

== Background ==

At Baidu, Prior to Doris, different tools were deployed to solve diverse requirements in many ways. And when a use case requires the simultaneous availability of capabilities that cannot all be provided by a single tool, users were forced to build hybrid architectures that stitch multiple tools together, but we believe that they shouldn’t need to accept such inherent complexity. A storage system built to provide great performance across a broad range of workloads provides a more elegant solution to the problems that hybrid architectures aim to solve. Doris is the solution.

Doris is designed to be a simple and single tightly coupled system, not depending on other systems. Doris provides high concurrent low latency point query performance, but also provides high throughput queries of ad-hoc analysis. Doris provides bulk-batch data loading, but also provides near real-time mini-batch data loading. Doris also provides high availability, reliability, fault tolerance, and scalability.

== Rationale ==

Doris mainly integrates the technology of Google Mesa and Apache Impala.

Mesa is a highly scalable analytic data storage system that stores critical measurement data related to Google's Internet advertising business. Mesa is designed to satisfy complex and challenging set of users’ and systems’ requirements, including near real-time data ingestion and query ability, as well as high availability, reliability, fault tolerance, and scalability for large data and query volumes.

Impala is a modern, open-source MPP SQL engine architected from the ground up for the Hadoop data processing environment. At present, by virtue of its superior performance and rich functionality， Impala has been comparable to many commercial MPP database query engine. Mesa can satisfy the needs of many of our storage requirements, however Mesa itself does not provide a SQL query engine; Impala is a very good MPP SQL query engine, but the lack of a perfect distributed storage engine. So in the end we chose the combination of these two technologies.

Learning from Mesa’s data model, we developed a distributed storage engine. Unlike Mesa, this storage engine does not rely on any distributed file system. Then we deeply integrate this storage engine with Impala query engine. Query compiling, query execution coordination and catalog management of storage engine are integrated to be frontend daemon; query execution and data storage are integrated to be backend daemon. With this integration, we implemented a single, full-featured, high performance state the art of MPP database, as well as maintaining the simplicity.

== Current Status ==

Doris has been an open source project on GitHub (https://github.com/baidu/palo).

=== Meritocracy ===

Doris has been deployed in production at Baidu and is applying more than 200 lines of business. It has demonstrated great performance benefits and has proved to be a better way for reporting and analysis based big data. Still We look forward to growing a rich user and developer community.

=== Community ===

Doris seeks to develop developer and user communities during incubation.

Doris makes use of Apache Impala. It was identified during early review of the proposal that the Doris community will need to work with Impala to define a suitable API.

=== Core Developers ===

 * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
 * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
 * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
 * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
 * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
 * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
 * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)

=== Alignment ===

Doris is related to several other Apache projects:

 * Doris can also read data stored in Apache Hadoop clusters powered by the HDFS filesystem.
 * Doris is closely integrated with Impala, which has graduated from Apache Incubator.
 * Doris uses Apache Thrift as its RPC and serialization framework of choice.

== Known Risks ==

=== Orphaned Products ===

The core developers of Doris team plan to work full time on this project. There is very little risk of Doris getting orphaned since at least one large company (Baidu) is extensively using it in their production. For example, currently there are more than 200 use cases using Doris in production. Furthermore, since Doris was open sourced at the beginning of October 2017, it has received more than 660 stars and been forked nearly 170 times. We plan to extend and diversify this community further through Apache.

=== Inexperience with Open Source ===

The core developers are all active users and followers of open source. They are already committers and contributors to the Doris Github project. All have been involved with the source code that has been released under an open source license, and several of them also have experience developing code in an open source environment. Though the core set of Developers do not have Apache Open Source experience, there are plans to onboard individuals with Apache open source experience on to the project.

=== Homogenous Developers ===

The most of core developers are from Baidu, but after Doris was open sourced, Doris received a lot of bug fixes and enhancements from other developers not working at Baidu.

=== Reliance on Salaried Developers ===

Baidu invested in Doris as the OLAP solution and some of its key engineers are working full time on the project. In addition, since there is a growing Big Data need for scalable OLAP solutions, we look forward to other Apache developers and researchers to contribute to the project. Also key to addressing the risk associated with relying on Salaried developers from a single entity is to increase the diversity of the contributors and actively lobby for Domain experts in the BI space to contribute. Apache Doris intends to do this.

=== An Excessive Fascination with the Apache Brand ===

Doris is proposing to enter incubation at Apache in order to help efforts to diversify the committer-base, not so much to capitalize on the Apache brand. The Doris project is in production use already inside Baidu, but is not expected to be an Baidu product for external customers. As such, the Doris project is not seeking to use the Apache brand as a marketing tool.

== Documentation ==

Information about Doris can be found at https://github.com/baidu/palo. The following links provide more information about Doris in open source:

 * Doris wiki site: https://github.com/baidu/palo/wiki
 * Codebase at Github: https://github.com/baidu/palo
 * Issue Tracking: https://github.com/baidu/palo/issues
 * Overview: https://github.com/baidu/Doris/wiki/palo-Overview
 * FAQ: https://github.com/baidu/palo/wiki/palo-FAQ

== Initial Source ==

Doris has been under development since 2017 by a team of engineers at Baidu Inc. It is currently hosted on Github.com<http://Github.com> under an Apache license at https://github.com/baidu/palo.

== External Dependencies ==

Doris has the following external dependencies.

 * Google gflags (BSD)
 * Google glog (BSD)
 * Apache Thrift (Apache Software License v2.0)
 * Apache Commons (Apache Software License v2.0)
 * Boost (Boost Software License)
 * rapidjson (Tencent)
 * Google RE2 (BSD-style)
 * lz4 (BSD)
 * snappy (BSD)
 * Twitter Bootstrap (Apache Software License v2.0)
 * d3 (BSD)
 * LLVM (BSD-like)

Build and test dependencies:

 * Apache Ant (Apache Software License v2.0)
 * Apache Maven (Apache Software License v2.0)
 * cmake (BSD)
 * clang (BSD)
 * Google gtest (Apache Software License v2.0)

== Required Resources ==

=== Mailing List ===

There are currently no mailing lists. The usual mailing lists are expected to be set up when entering incubation:

 * private@doris.incubator.apache.org<ma...@doris.incubator.apache.org>
 * dev@doris.incubator.apache.org<ma...@doris.incubator.apache.org>
 * commits@doris.incubator.apache.org<ma...@doris.incubator.apache.org>

=== Subversion Directory ===

Upon entering incubation, we want to move (or copy) the existing repo from https://github.com/baidu/palo to Apache infrastructure at https://github.com/apache/incubator-doris.

=== Issue Tracking ===

Doris currently uses GitHub to track issues. Would like to continue to do so while we discuss migration possibilities with the ASF Infra committee.

=== Other Resources ===

The existing code already has unit tests so we will make use of existing Apache continuous testing infrastructure. The resulting load should not be very large.

== Initial Committers ==

 * Ruyue Ma (https://github.com/maruyue, maruyue@baidu dot com)
 * Chun Zhao (https://github.com/imay, buaa.zhaoc@gmail dot com)
 * Mingyu Chen (https://github.com/morningman,chenmingyu@baidu dot com)
 * De Li（https://github.com/lide-reed, mailtolide@sina dot com）
 * Hao Chen (https://github.com/chenhao7253886, chenhao16@baidu dot com)
 * Chaoyong Li (https://github.com/cyongli, lichaoyong@baidu dot com)
 * Bin Lin (https://github.com/lingbin, lingbinlb@gmail dot com)
 * Sijie Guo (guosijie@gmail dot com)
 * Zheng Shao (zshao@apache.org<ma...@apache.org>)

== Affiliations ==

The initial committers are employees of Baidu Inc..

== Sponsors ==

=== Champion ===

 * Dave Fisher, wave@apache.org<ma...@apache.org>

=== Nominated Mentors ===

 * Luke Han, lukehan@apache.org<ma...@apache.org>
 * Dave Fisher, wave@apache.org<ma...@apache.org>
 * Willem Jiang, ningjiang@apache.org<ma...@apache.org>

=== Sponsoring Entity ===

We are requesting the Incubator to sponsor this project.