You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@incubator.apache.org by Dave Fisher <da...@comcast.net> on 2013/03/15 00:48:27 UTC
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
I think that this vote should be invalidated.
Are the five binding votes from IPMC members?
It is not 5.
Also, should proposals with only 1 valid Mentor be acceptable?
If two members of the IPMC step forward to Mentor MRQL I might change my mind.
Regards,
Dave
On Mar 12, 2013, at 2:12 AM, Edward J. Yoon wrote:
> The VOTE has passed with 5 binding +1's and no -1s.
>
> I'll start the work to get the podling started.
>
> Thank you.
>
> On Sun, Mar 10, 2013 at 3:48 AM, Mattmann, Chris A (388J)
> <ch...@jpl.nasa.gov> wrote:
>> +1 from me (binding).
>>
>> Good luck!
>>
>> Cheers,
>> Chris
>>
>>
>> On 3/6/13 9:04 AM, "Leonidas Fegaras" <fe...@cse.uta.edu> wrote:
>>
>>> Dear ASF members,
>>> I would like to call for a VOTE for acceptance of MRQL into the
>>> Incubator.
>>> The vote will close on Monday March 11, 2013.
>>>
>>> [ ] +1 Accept MRQL into the Apache incubator
>>> [ ] +0 Don't care.
>>> [ ] -1 Don't accept MRQL into the incubator because...
>>>
>>> Full proposal is pasted below and the corresponding wiki is
>>>
>>> http://wiki.apache.org/incubator/MRQLProposal
>>>
>>> Only VOTEs from Incubator PMC members are binding,
>>> but all are welcome to express their thoughts.
>>> Sincerely,
>>> Leonidas Fegaras
>>>
>>>
>>> = Abstract =
>>>
>>> MRQL is a query processing and optimization system for large-scale,
>>> distributed data analysis, built on top of Apache Hadoop and Hama.
>>>
>>> = Proposal =
>>>
>>> MRQL (pronounced ''miracle'') is a query processing and optimization
>>> system for large-scale, distributed data analysis. MRQL (the MapReduce
>>> Query Language) is an SQL-like query language for large-scale data
>>> analysis on a cluster of computers. The MRQL query processing system
>>> can evaluate MRQL queries in two modes: in MapReduce mode on top of
>>> Apache Hadoop or in Bulk Synchronous Parallel (BSP) mode on top of
>>> Apache Hama. The MRQL query language is powerful enough to express
>>> most common data analysis tasks over many forms of raw ''in-situ''
>>> data, such as XML and JSON documents, binary files, and CSV
>>> documents. MRQL is more powerful than other current high-level
>>> MapReduce languages, such as Hive and PigLatin, since it can operate
>>> on more complex data and supports more powerful query constructs, thus
>>> eliminating the need for using explicit MapReduce code. With MRQL,
>>> users will be able to express complex data analysis tasks, such as
>>> PageRank, k-means clustering, matrix factorization, etc, using
>>> SQL-like queries exclusively, while the MRQL query processing system
>>> will be able to compile these queries to efficient Java code.
>>>
>>> = Background =
>>>
>>> The initial code was developed at the University of Texas of Arlington
>>> (UTA) by a research team, led by Leonidas Fegaras. The software was
>>> first released in May 2011. The original goal of this project was to
>>> build a query processing system that translates SQL-like data analysis
>>> queries to efficient workflows of MapReduce jobs. A design goal was to
>>> use HDFS as the physical storage layer, without any indexing, data
>>> partitioning, or data normalization, and to use Hadoop (without
>>> extensions) as the run-time engine. The motivation behind this work
>>> was to build a platform to test new ideas on query processing and
>>> optimization techniques applicable to the MapReduce framework.
>>>
>>> A year ago, MRQL was extended to run on Hama. The motivation for this
>>> extension was that Hadoop MapReduce jobs were required to read their
>>> input and write their output on HDFS. This simplifies reliability and
>>> fault tolerance but it imposes a high overhead to complex MapReduce
>>> workflows and graph algorithms, such as PageRank, which require
>>> repetitive jobs. In addition, Hadoop does not preserve data in memory
>>> across consecutive MapReduce jobs. This restriction requires to read
>>> data at every step, even when the data is constant. BSP, on the other
>>> hand, does not suffer from this restriction, and, under certain
>>> circumstances, allows complex repetitive algorithms to run entirely in
>>> the collective memory of a cluster. Thus, the goal was to be able to
>>> run the same MRQL queries in both modes, MapReduce and BSP, without
>>> modifying the queries: If there are enough resources available, and
>>> low latency and speed are more important than resilience, queries may
>>> run in BSP mode; otherwise, the same queries may run in MapReduce
>>> mode. BSP evaluation was found to be a good choice when fault
>>> tolerance is not critical, data (both input and intermediate) can fit
>>> in the cluster memory, and data processing requires complex/repetitive
>>> steps.
>>>
>>> The research results of this ongoing work have already been published
>>> in conferences (WebDB'11, EDBT'12, and DataCloud'12) and the authors
>>> have already received positive feedback from researchers in academia
>>> and industry who were attending these conferences.
>>>
>>> = Rationale =
>>>
>>> * MRQL will be the first general-purpose, SQL-like query language for
>>> data analysis based on BSP.
>>> Currently, many programmers prefer to code their MapReduce
>>> applications in a higher-level query language, rather than an
>>> algorithmic language. For instance, Pig is used for 60% of Yahoo
>>> MapReduce jobs, while Hive is used for 90% of Facebook MapReduce
>>> jobs. This, we believe, will also be the trend for BSP applications,
>>> because, even though, in principle, the BSP model is very simple to
>>> understand, it is hard to develop, optimize, and maintain non-trivial
>>> BSP applications coded in a general-purpose programming
>>> language. Currently, there is no widely acceptable declarative BSP
>>> query language, although there are a few special-purpose BSP systems
>>> for graph analysis, such as Google Pregel and Apache Giraph, for
>>> machine learning, such as BSML, and for scientific data analysis.
>>>
>>> * MRQL can capture many complex data analysis algorithms in
>>> declarative form.
>>> Existing MapReduce query languages, such as HiveQL and PigLatin,
>>> provide a limited syntax for operating on data collections, in the
>>> form of relational joins and group-bys. Because of these limitations,
>>> these languages enable users to plug-in custom MapReduce scripts into
>>> their queries for those jobs that cannot be declaratively coded in
>>> their query language. This nullifies the benefits of using a
>>> declarative query language and may result to suboptimal, error-prone,
>>> and hard-to-maintain code. More importantly, these languages are
>>> inappropriate for complex scientific applications and graph analysis,
>>> because they do not directly support iteration or recursion in
>>> declarative form and are not able to handle complex, nested scientific
>>> data, which are often semi-structured. Furthermore, current MapReduce
>>> query processors apply traditional query optimization techniques that
>>> may be suboptimal in a MapReduce or BSP environment.
>>>
>>> * The MRQL design is modular, with pluggable distributed processing
>>> back-ends, query languages, and data formats.
>>> MRQL aims to be both powerful and adaptable. Although Hadoop is
>>> currently the most popular framework for large-scale data analysis,
>>> there are a few alternatives that are currently shaping form,
>>> including frameworks based on BSP (eg, Giraph, Pregel, Hama), MPI
>>> (eg, OpenMPI), etc. MRQL was designed in such a way so that it will
>>> be easy to support other distributed processing frameworks in the
>>> future. As an evidence of this claim, the MRQL processor required
>>> only 2K extra lines of Java code to support BSP evaluation.
>>>
>>> = Initial Goals =
>>>
>>> Some current goals include:
>>>
>>> * apply MRQL to graph analysis problems, such as k-means clustering
>>> and PageRank
>>>
>>> * apply MRQL to large-scale scientific analysis (develop general
>>> optimization techniques that can apply to matrix multiplication,
>>> matrix factorization, etc)
>>>
>>> * process additional data formats, such as Avro, and column-based
>>> stores, such as HBase
>>>
>>> * map MRQL to additional distributed processing frameworks, such as
>>> Spark and OpenMPI
>>>
>>> * extend the front-end to process more query languages, such as
>>> standard SQL, SPARQL, XQuery, and PigLatin
>>>
>>> = Current Status =
>>>
>>> The current MRQL release (version 0.8.10) is a beta release. It is
>>> built on top of Hadoop and Hama (no extensions are needed). It
>>> currently works on Hadoop up to 1.0.4 (but not on Yarn yet) and Hama
>>> 0.5.0. It has only been tested on a small cluster of 20 nodes (80
>>> cores).
>>>
>>> == Meritocracy ==
>>>
>>> The initial MRQL code base was developed by Leonidas Fegaras in May
>>> 2011, and was continuously improved throughout the years. We will
>>> reach out other potential contributors through open forums. We plan
>>> to do everything possible to encourage an environment that supports a
>>> meritocracy, where contributors will extend their privileges based on
>>> their contribution. MRQL's modular design will facilitate the
>>> strategic extensions to various modules, such as adding a standard-SQL
>>> interface, introducing new optimization techniques, etc.
>>>
>>> == Community ==
>>>
>>> The interest in open-source query processing systems for analyzing
>>> large datasets has been steadily increased in the last few years.
>>> Related Apache projects have already attracted a very large community
>>> from both academia and industry. We expect that MRQL will also
>>> establish an active community. Several researchers from both academia
>>> and industry who are interested in using our code have already
>>> contacted us.
>>>
>>> == Core Developers ==
>>>
>>> The initial core developer was Leonidas Fegaras, who wrote the
>>> majority of the code. He is an associate professor at UTA, with
>>> interests in cloud computing, databases, web technologies, and
>>> functional programming. He has an extensive knowledge and working
>>> experience in building complex query processing systems for databases,
>>> and compilers for functional and algorithmic programming languages.
>>>
>>> == Alignment ==
>>>
>>> MRQL is built on top of two Apache projects: Hadoop and Hama. We have
>>> plans to incorporate other products from the Hadoop ecosystem, such as
>>> Avro and HBase. MRQL can serve as a testbed for fine-tuning and
>>> evaluating the performance of the Apache Hama system. Finally, the
>>> MRQL query language and processor can be used by Apache Drill as a
>>> pluggable query language.
>>>
>>> = Known Risks =
>>>
>>> == Orphaned Products ==
>>>
>>> The initial committer is from academia, which may be a risk, since
>>> research in academia is publication-driven, rather than
>>> product-driven. It happens very often in academic research, when a
>>> project becomes outdated and doesn't produce publishable results, to
>>> be abandoned in favor of new cutting-edge projects. We do not believe
>>> that this will be the case for MRQL for the years to come, because it
>>> can be adapted to support new query languages, new optimization
>>> techniques, and new distributed back-ends, thus sustaining enough
>>> research interest. Another risk is that, when graduate students who
>>> write code graduate, they may leave their work undocumented and
>>> unfinished. We will strive to gain enough momentum to recruit
>>> additional committers from industry in order to eliminate these risks.
>>>
>>> == Inexperience with Open Source ==
>>>
>>> The initial developer has been involved with various projects whose
>>> source code has been released under open source license, but he has no
>>> prior experience on contributing to open-source projects. With the
>>> guidance from other more experienced committers and participants, we
>>> expect that the meritocracy rules will have a positive influence on
>>> this project.
>>>
>>> == Homogeneous Developers ==
>>>
>>> The initial committer comes from academia. However, given the interest
>>> we have seen in the project, we expect the diversity to improve in the
>>> near future.
>>>
>>> == Reliance on Salaried Developers ==
>>>
>>> Currently, the MRQL code was developed on the committer's volunteer
>>> time. In the future, UTA graduate students who will do some of the
>>> coding may be supported by UTA and funding agencies, such as NSF.
>>>
>>> == Relationships with Other Apache Products ==
>>>
>>> MRQL has some overlapping functionality with Hive and Tajo, which are
>>> Data Warehouse systems for Hadoop, and with Drill, which is an
>>> interactive data analysis system that can process nested data. MRQL
>>> has a more powerful data model, in which any form of nested data, such
>>> as XML and JSON, can be defined as a user-defined datatype. More
>>> importantly, complex data analysis tasks, such as PageRank, k-means
>>> clustering, and matrix multiplication and factorization, can be
>>> expressed as short SQL-like queries, while the MRQL system is able to
>>> evaluate these queries efficiently. Furthermore, the MRQL system can
>>> run these queries in BSP mode, in addition to MapReduce mode, thus
>>> achieving low latency and speed, which are also Drill's goals.
>>> Nevertheless, we will welcome and encourage any help from these
>>> projects and we will be eager to make contributions to these projects
>>> too.
>>>
>>> == An Excessive Fascination with the Apache Brand ==
>>>
>>> The Apache brand is likely to help us find contributors and reach out
>>> to the open-source community. Nevertheless, since MRQL depends on
>>> Apache projects (Hadoop and Hama), it makes sense to have our software
>>> available as part of this ecosystem.
>>>
>>> = Documentation =
>>>
>>> Information about MRQL can be found at http://lambda.uta.edu/mrql/
>>>
>>> = Initial Source =
>>>
>>> The initial MRQL code has been released as part of a research project
>>> developed at the University of Texas at Arlington under the Apache 2.0
>>> license for the past two years. The source code is currently hosted
>>> on GitHub at: https://github.com/fegaras/mrql MRQL’s release artifact
>>> would consist of a single tarball of packaging and test code.
>>>
>>> = External Dependencies =
>>>
>>> The MRQL source code is already licensed under the Apache License,
>>> Version 2.0. MRQL uses JLine which is distributed under the BSD
>>> license.
>>>
>>> = Cryptography =
>>>
>>> Not applicable.
>>>
>>> = Required Resources =
>>>
>>> == Mailing Lists ==
>>>
>>> * mrql-private
>>> * mrql-dev
>>> * mrql-user
>>>
>>> == Subversion Directory ==
>>>
>>> * Git is the preferred source control system:
>>> git://git.apache.org/mrql
>>>
>>> == Issue Tracking ==
>>>
>>> * A JIRA issue tracker, MRQL
>>>
>>> == Wiki ==
>>>
>>> * Moinmoin wiki, http://wiki.apache.org/mrql
>>>
>>> = Initial Committers =
>>>
>>> * Leonidas Fegaras <fegaras AT cse DOT uta DOT edu>
>>> * Upa Gupta <upa.gupta AT mavs DOT uta DOT edu>
>>> * Edward J. Yoon <edwardyoon AT apache DOT org>
>>> * Maqsood Alam <maqsoodalam AT hotmail DOT com>
>>> * John Hope <john.hope AT oracle DOT com>
>>> * Mark Wall <mark.wall AT oracle DOT com>
>>> * Kuassi Mensah <kuassi.mensah AT oracle DOT com>
>>> * Ambreesh Khanna <ambreesh.khanna AT oracle DOT com>
>>> * Karthik Kambatla <kasha AT cloudera DOT com>
>>>
>>> = Affiliations =
>>>
>>> * Leonidas Fegaras (University of Texas at Arlington)
>>> * Upa Gupta (University of Texas at Arlington)
>>> * Edward J. Yoon (Oracle corp)
>>> * Maqsood Alam (Oracle corp)
>>> * John Hope (Oracle corp)
>>> * Mark Wall (Oracle corp)
>>> * Kuassi Mensah (Oracle corp)
>>> * Ambreesh Khanna (Oracle corp)
>>> * Karthik Kambatla (Cloudera)
>>>
>>> = Sponsors =
>>>
>>> == Champion ==
>>>
>>> * Edward J. Yoon <edwardyoon AT apache DOT org>
>>>
>>> == Nominated Mentors ==
>>>
>>> * Alex Karasulu <akarasulu AT apache DOT org>
>>> * Edward J. Yoon <edwardyoon AT apache DOT org>
>>>
>>> == Sponsoring Entity ==
>>>
>>> Incubator PMC
>>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Mohammad Nour El-Din <no...@gmail.com>.
Hi
On Fri, Mar 15, 2013 at 2:06 AM, Marvin Humphrey <ma...@rectangular.com>wrote:
> On Thu, Mar 14, 2013 at 4:48 PM, Dave Fisher <da...@comcast.net>
> wrote:
> > Are the five binding votes from IPMC members?
> >
> > It is not 5.
>
> For the record, there were 4 binding IPMC votes in favor.
>
> Mohammed Nour El-Din
> Alex Karasulu
> Tommaso Teofili
> Chris Mattmann
>
> > Also, should proposals with only 1 valid Mentor be acceptable?
>
> In addition to Alex Karasulu, Mohammed Nour El-Din had volunteered,
> but seems to have been left off the version of the proposal which was
> VOTEd on. So unless Mohammed has withdrawn, MRQL has 2 Mentors.
>
No I didn't
>
> Marvin Humphrey
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>
--
Thanks
- Mohammad Nour
----
"Life is like riding a bicycle. To keep your balance you must keep moving"
- Albert Einstein
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Dave Fisher <da...@comcast.net>.
On Mar 15, 2013, at 7:05 AM, Matt Franklin wrote:
> On Fri, Mar 15, 2013 at 9:28 AM, Alex Karasulu <ak...@apache.org> wrote:
>> On Fri, Mar 15, 2013 at 3:01 PM, ant elder <an...@gmail.com> wrote:
>>
>>> On Fri, Mar 15, 2013 at 12:16 PM, Alex Karasulu <akarasulu@apache.org
>>>> wrote:
>>>
>>>> On Fri, Mar 15, 2013 at 10:21 AM, Edward J. Yoon <edwardyoon@apache.org
>>>>> wrote:
>>>>
>>>>>> could you please close the "Create MRQL" tasks in the Infra Jira
>>> until
>>>>>> the situation has been cleared up. Whenever this all has been sorted
>>>>>> out you can reopen the issues.
>>>>>
>>>>> Sure.
>>>>>
>>>>> Since this might not be fixed soon, proposal can be changed
>>>>> (especially corporation volunteers). And, thank you for your
>>>>> suggestion, but I'm already in initial committers list.
>>>>>
>>>>> To IPMC, I would request you to focus more on reviewing Proposal in
>>>>> the future. The opportunity of MRQL should not be faded by my mistake.
>>>>>
>>>>>
>>>> Bravo! There was no malice in the vote, just a simple mistake. If need
>>> be I
>>>> or Mo can take on the Champion role.
>>>>
>>>> Are there others from the IPMC that would be interested in mentoring
>>> MRQL?
>>>>
>>>> --
>>>> Best Regards,
>>>> -- Alex
>>>>
>>>
>>> The champion role is really just to help the proposal through any
>>> discussion prior to being accepted as a poddling so thats been done and it
>>> makes little difference now.
>>
>>
>> True. I'm totally down with your reasoning.
>>
>> But don't you think it might be a formality we need to comply with, even if
>> it does not make sense at this stage?
>>
>> Mohammed had volunteered to be a mentor so
>>> lets continue as that and I'll add myself as mentor too just to help if
>>> anymore is needed. So that gives plenty of mentors, we don't need more
>>> voting for that as the Incubator PMC can update mentors as it sees fit. So
>>> lets just continue on with that and the poddling accepted.
>>>
>>>
>> Based on your presence as a mentor along with Mo and myself I see this
>> poddling as accepted as well. If there's pressure to fill the Champion role
>> it can be handled on demand.
>
> Keep in mind that the role of champion was expanded slightly [1]. I
> think it is important to ensure that MRQL has a champion listed. I
> agree that the matter should be resolved once the champion and
> additional mentors are listed.
>
> [1]: http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Champion
>
>>
>> Ted, Dave is this situation now more acceptable and does it alleviate your
>> concerns?
In a word, yes.
Adding an appropriate champion and enough mentors solves my concerns.
Regards,
Dave
>> If not we want to know why and take steps to rectify the
>> situation accordingly. I personally really appreciate your diligence on
>> this matter. As a mentor I should have caught the problems you sited but
>> because of the Champion's ex-VP role (Hama?) I thought he was a member of
>> both the ASF and the IPMC. I should have checked.
>>
>> --
>> Thanks,
>> -- Alex
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Matt Franklin <m....@gmail.com>.
On Fri, Mar 15, 2013 at 9:28 AM, Alex Karasulu <ak...@apache.org> wrote:
> On Fri, Mar 15, 2013 at 3:01 PM, ant elder <an...@gmail.com> wrote:
>
>> On Fri, Mar 15, 2013 at 12:16 PM, Alex Karasulu <akarasulu@apache.org
>> >wrote:
>>
>> > On Fri, Mar 15, 2013 at 10:21 AM, Edward J. Yoon <edwardyoon@apache.org
>> > >wrote:
>> >
>> > > > could you please close the "Create MRQL" tasks in the Infra Jira
>> until
>> > > > the situation has been cleared up. Whenever this all has been sorted
>> > > > out you can reopen the issues.
>> > >
>> > > Sure.
>> > >
>> > > Since this might not be fixed soon, proposal can be changed
>> > > (especially corporation volunteers). And, thank you for your
>> > > suggestion, but I'm already in initial committers list.
>> > >
>> > > To IPMC, I would request you to focus more on reviewing Proposal in
>> > > the future. The opportunity of MRQL should not be faded by my mistake.
>> > >
>> > >
>> > Bravo! There was no malice in the vote, just a simple mistake. If need
>> be I
>> > or Mo can take on the Champion role.
>> >
>> > Are there others from the IPMC that would be interested in mentoring
>> MRQL?
>> >
>> > --
>> > Best Regards,
>> > -- Alex
>> >
>>
>> The champion role is really just to help the proposal through any
>> discussion prior to being accepted as a poddling so thats been done and it
>> makes little difference now.
>
>
> True. I'm totally down with your reasoning.
>
> But don't you think it might be a formality we need to comply with, even if
> it does not make sense at this stage?
>
> Mohammed had volunteered to be a mentor so
>> lets continue as that and I'll add myself as mentor too just to help if
>> anymore is needed. So that gives plenty of mentors, we don't need more
>> voting for that as the Incubator PMC can update mentors as it sees fit. So
>> lets just continue on with that and the poddling accepted.
>>
>>
> Based on your presence as a mentor along with Mo and myself I see this
> poddling as accepted as well. If there's pressure to fill the Champion role
> it can be handled on demand.
Keep in mind that the role of champion was expanded slightly [1]. I
think it is important to ensure that MRQL has a champion listed. I
agree that the matter should be resolved once the champion and
additional mentors are listed.
[1]: http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Champion
>
> Ted, Dave is this situation now more acceptable and does it alleviate your
> concerns? If not we want to know why and take steps to rectify the
> situation accordingly. I personally really appreciate your diligence on
> this matter. As a mentor I should have caught the problems you sited but
> because of the Champion's ex-VP role (Hama?) I thought he was a member of
> both the ASF and the IPMC. I should have checked.
>
> --
> Thanks,
> -- Alex
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Niall Pemberton <ni...@gmail.com>.
On Fri, Mar 15, 2013 at 1:28 PM, Alex Karasulu <ak...@apache.org> wrote:
> On Fri, Mar 15, 2013 at 3:01 PM, ant elder <an...@gmail.com> wrote:
>
>> On Fri, Mar 15, 2013 at 12:16 PM, Alex Karasulu <akarasulu@apache.org
>> >wrote:
>>
>> > On Fri, Mar 15, 2013 at 10:21 AM, Edward J. Yoon <edwardyoon@apache.org
>> > >wrote:
>> >
>> > > > could you please close the "Create MRQL" tasks in the Infra Jira
>> until
>> > > > the situation has been cleared up. Whenever this all has been sorted
>> > > > out you can reopen the issues.
>> > >
>> > > Sure.
>> > >
>> > > Since this might not be fixed soon, proposal can be changed
>> > > (especially corporation volunteers). And, thank you for your
>> > > suggestion, but I'm already in initial committers list.
>> > >
>> > > To IPMC, I would request you to focus more on reviewing Proposal in
>> > > the future. The opportunity of MRQL should not be faded by my mistake.
>> > >
>> > >
>> > Bravo! There was no malice in the vote, just a simple mistake. If need
>> be I
>> > or Mo can take on the Champion role.
>> >
>> > Are there others from the IPMC that would be interested in mentoring
>> MRQL?
>> >
>> > --
>> > Best Regards,
>> > -- Alex
>> >
>>
>> The champion role is really just to help the proposal through any
>> discussion prior to being accepted as a poddling so thats been done and it
>> makes little difference now.
>
>
> True. I'm totally down with your reasoning.
>
> But don't you think it might be a formality we need to comply with, even if
> it does not make sense at this stage?
>
> Mohammed had volunteered to be a mentor so
>> lets continue as that and I'll add myself as mentor too just to help if
>> anymore is needed. So that gives plenty of mentors, we don't need more
>> voting for that as the Incubator PMC can update mentors as it sees fit. So
>> lets just continue on with that and the poddling accepted.
>>
>>
> Based on your presence as a mentor along with Mo and myself I see this
> poddling as accepted as well. If there's pressure to fill the Champion role
> it can be handled on demand.
>
> Ted, Dave is this situation now more acceptable and does it alleviate your
> concerns? If not we want to know why and take steps to rectify the
> situation accordingly. I personally really appreciate your diligence on
> this matter. As a mentor I should have caught the problems you sited but
> because of the Champion's ex-VP role (Hama?) I thought he was a member of
> both the ASF and the IPMC. I should have checked.
I know its late - but I'm +1 to accepting MRQL
Niall
> --
> Thanks,
> -- Alex
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Roman Shaposhnik <rv...@apache.org>.
On Fri, Mar 15, 2013 at 4:01 PM, Suresh Marru <sm...@apache.org> wrote:
> I fully agree. I had a chance also to look at the proposal closely, so will +1 if the
> vote is re-opened, but I see no need for it. Its in the hands of competent mentors
> and the podling is ready to get on board.
+1 to that! 'Keep calm and carry on' as they say.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Suresh Marru <sm...@apache.org>.
On Mar 15, 2013, at 6:25 PM, ant elder <an...@gmail.com> wrote:
> On Fri, Mar 15, 2013 at 1:28 PM, Alex Karasulu <ak...@apache.org> wrote:
>
>> On Fri, Mar 15, 2013 at 3:01 PM, ant elder <an...@gmail.com> wrote:
>>
>>> On Fri, Mar 15, 2013 at 12:16 PM, Alex Karasulu <akarasulu@apache.org
>>>> wrote:
>>>
>>>> On Fri, Mar 15, 2013 at 10:21 AM, Edward J. Yoon <
>> edwardyoon@apache.org
>>>>> wrote:
>>>>
>>>>>> could you please close the "Create MRQL" tasks in the Infra Jira
>>> until
>>>>>> the situation has been cleared up. Whenever this all has been
>> sorted
>>>>>> out you can reopen the issues.
>>>>>
>>>>> Sure.
>>>>>
>>>>> Since this might not be fixed soon, proposal can be changed
>>>>> (especially corporation volunteers). And, thank you for your
>>>>> suggestion, but I'm already in initial committers list.
>>>>>
>>>>> To IPMC, I would request you to focus more on reviewing Proposal in
>>>>> the future. The opportunity of MRQL should not be faded by my
>> mistake.
>>>>>
>>>>>
>>>> Bravo! There was no malice in the vote, just a simple mistake. If need
>>> be I
>>>> or Mo can take on the Champion role.
>>>>
>>>> Are there others from the IPMC that would be interested in mentoring
>>> MRQL?
>>>>
>>>> --
>>>> Best Regards,
>>>> -- Alex
>>>>
>>>
>>> The champion role is really just to help the proposal through any
>>> discussion prior to being accepted as a poddling so thats been done and
>> it
>>> makes little difference now.
>>
>>
>> True. I'm totally down with your reasoning.
>>
>> But don't you think it might be a formality we need to comply with, even if
>> it does not make sense at this stage?
>>
>>
> There has been a vote, which passed with enough binding votes, no one has
> -1'd, and no one has retracted their vote. Lets wait till Monday and if the
> majority vote still passes just carry on with the additional mentors. I
> hope no one does -1, theres several experienced mentors now so nothing
> really to be gained from forcing some new vote.
>
> ...ant
I fully agree. I had a chance also to look at the proposal closely, so will +1 if the vote is re-opened, but I see no need for it. Its in the hands of competent mentors and the podling is ready to get on board.
Suresh
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Alan Cabrera <li...@toolazydogs.com>.
Let's move the podling chatter to the dev list. :)
Regards,
Alan
On Mar 26, 2013, at 9:55 PM, "Edward J. Yoon" <ed...@apache.org> wrote:
> Thanks Ant.
>
> Could you please also create commits@ list for wiki/source commit
> mails? The submit button of the web form doesn't change from Disabled.
>
> On Thu, Mar 21, 2013 at 5:59 PM, ant elder <an...@gmail.com> wrote:
>> Please find that the MRQL mailing lists have been created and are
>> ready to be used for further discussion:
>>
>> dev@mrql.incubator.apache.org
>> user@mrql.incubator.apache.org
>> private@mrql.incubator.apache.org
>>
>> Would everyone named on the proposal please go subscribe to them, and
>> happy MRQL'ing.
>>
>> ...ant
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by "Edward J. Yoon" <ed...@apache.org>.
Thanks Ant.
Could you please also create commits@ list for wiki/source commit
mails? The submit button of the web form doesn't change from Disabled.
On Thu, Mar 21, 2013 at 5:59 PM, ant elder <an...@gmail.com> wrote:
> Please find that the MRQL mailing lists have been created and are
> ready to be used for further discussion:
>
> dev@mrql.incubator.apache.org
> user@mrql.incubator.apache.org
> private@mrql.incubator.apache.org
>
> Would everyone named on the proposal please go subscribe to them, and
> happy MRQL'ing.
>
> ...ant
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
--
Best Regards, Edward J. Yoon
@eddieyoon
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by ant elder <an...@gmail.com>.
Please find that the MRQL mailing lists have been created and are
ready to be used for further discussion:
dev@mrql.incubator.apache.org
user@mrql.incubator.apache.org
private@mrql.incubator.apache.org
Would everyone named on the proposal please go subscribe to them, and
happy MRQL'ing.
...ant
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [RESULT][VOTE] Accept MRQL into the Incubator
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Mon, Mar 18, 2013 at 12:56 PM, ant elder <an...@gmail.com> wrote:
> ...I believe the following is is the current version following
> the vote and discussion. If there are any further personnel changes
> can we please have the discussion on private@ to make sure there is
> consensus before posting here....
Ok - what I did was copy the proposal from the wiki which is supposed
to be up to date, but I see your point.
Consider this vote suspended for now, let's discuss the
champion/mentors roster on the private list and come back here once we
have consensus among Incubator PMC members.
-Bertrand
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [RESULT][VOTE] Accept MRQL into the Incubator
Posted by ant elder <an...@gmail.com>.
On Mon, Mar 18, 2013 at 10:31 AM, Bertrand Delacretaz
<bd...@apache.org> wrote:
> On Sun, Mar 17, 2013 at 4:01 PM, Mohammad Nour El-Din
> <no...@gmail.com> wrote:
>> On Sun, Mar 17, 2013 at 11:49 AM, Alex Karasulu <ak...@apache.org>wrote:
>>> We're also missing Ant Elder from the Nominated Mentors list no?
>> Fixed...
>
> I have pasted the current version of the proposal below, so that this
> list has it correctly archived.
>
Actually, I believe the following is is the current version following
the vote and discussion. If there are any further personnel changes
can we please have the discussion on private@ to make sure there is
consensus before posting here.
= Abstract =
MRQL is a query processing and optimization system for large-scale,
distributed data analysis, built on top of Apache Hadoop and Hama.
= Proposal =
MRQL (pronounced ''miracle'') is a query processing and optimization
system for large-scale, distributed data analysis. MRQL (the
!MapReduce Query Language) is an SQL-like query language for
large-scale data analysis on a cluster of computers. The MRQL query
processing system can evaluate MRQL queries in two modes: in
!MapReduce mode on top of [[http://hama.apache.org/|Apache Hadoop]] or
in Bulk Synchronous Parallel (BSP) mode on top of
[[http://hama.apache.org/|Apache Hama]]. The MRQL query language is
powerful enough to express most common data analysis tasks over many
forms of raw ''in-situ'' data, such as XML and JSON documents, binary
files, and CSV documents. MRQL is more powerful than other current
high-level !MapReduce languages, such as Hive and !PigLatin, since it
can operate on more complex data and supports more powerful query
constructs, thus eliminating the need for using explicit !MapReduce
code. With MRQL, users will be able to express complex data analysis
tasks, such as !PageRank, k-means clustering, matrix factorization,
etc, using SQL-like queries exclusively, while the MRQL query
processing system will be able to compile these queries to efficient
Java code.
= Background =
The initial code was developed at the University of Texas of Arlington
(UTA) by a research team, led by Leonidas Fegaras. The software was
first released in May 2011. The original goal of this project was to
build a query processing system that translates SQL-like data analysis
queries to efficient workflows of !MapReduce jobs. A design goal was
to use HDFS as the physical storage layer, without any indexing, data
partitioning, or data normalization, and to use Hadoop (without
extensions) as the run-time engine. The motivation behind this work
was to build a platform to test new ideas on query processing and
optimization techniques applicable to the !MapReduce framework.
A year ago, MRQL was extended to run on Hama. The motivation for this
extension was that Hadoop !MapReduce jobs were required to read their
input and write their output on HDFS. This simplifies reliability and
fault tolerance but it imposes a high overhead to complex !MapReduce
workflows and graph algorithms, such as !PageRank, which require
repetitive jobs. In addition, Hadoop does not preserve data in memory
across consecutive !MapReduce jobs. This restriction requires to read
data at every step, even when the data is constant. BSP, on the other
hand, does not suffer from this restriction, and, under certain
circumstances, allows complex repetitive algorithms to run entirely in
the collective memory of a cluster. Thus, the goal was to be able to
run the same MRQL queries in both modes, !MapReduce and BSP, without
modifying the queries: If there are enough resources available, and
low latency and speed are more important than resilience, queries may
run in BSP mode; otherwise, the same queries may run in !MapReduce
mode. BSP evaluation was found to be a good choice when fault
tolerance is not critical, data (both input and intermediate) can fit
in the cluster memory, and data processing requires complex/repetitive
steps.
The research results of this ongoing work have already been published
in conferences (WebDB'11, EDBT'12, and !DataCloud'12) and the authors
have already received positive feedback from researchers in academia
and industry who were attending these conferences.
= Rationale =
* MRQL will be the first general-purpose, SQL-like query language for
data analysis based on BSP.
. Currently, many programmers prefer to code their !MapReduce
applications in a higher-level query language, rather than an
algorithmic language. For instance, Pig is used for 60% of Yahoo!
!MapReduce jobs, while Hive is used for 90% of Facebook !MapReduce
jobs. This, we believe, will also be the trend for BSP applications,
because, even though, in principle, the BSP model is very simple to
understand, it is hard to develop, optimize, and maintain non-trivial
BSP applications coded in a general-purpose programming language.
Currently, there is no widely acceptable declarative BSP query
language, although there are a few special-purpose BSP systems for
graph analysis, such as Google Pregel and
[[http://incubator.apache.org/giraph/|Apache Giraph]], for machine
learning, such as
[[http://frederic.loulergue.eu/research/bsml/main.html|BSML]], and for
scientific data analysis.
* MRQL can capture many complex data analysis algorithms in declarative form.
. Existing !MapReduce query languages, such as HiveQL and !PigLatin,
provide a limited syntax for operating on data collections, in the
form of relational joins and group-bys. Because of these limitations,
these languages enable users to plug-in custom !MapReduce scripts into
their queries for those jobs that cannot be declaratively coded in
their query language. This nullifies the benefits of using a declarative
query language and may result to suboptimal, error-prone, and
hard-to-maintain code. More importantly, these languages are
inappropriate for complex scientific applications and graph analysis,
because they do not directly support iteration or recursion in
declarative form and are not able to handle complex, nested scientific
data, which are often semi-structured. Furthermore, current !MapReduce
query processors apply traditional query optimization techniques that
may be suboptimal in a !MapReduce or BSP environment.
* The MRQL design is modular, with pluggable distributed processing
back-ends, query languages, and data formats.
. MRQL aims to be both powerful and adaptable. Although Hadoop is
currently the most popular framework for large-scale data analysis,
there are a few alternatives that are currently shaping form,
including frameworks based on BSP (eg, Giraph, Pregel, Hama), MPI (eg,
OpenMPI), etc. MRQL was designed in such a way so that it will be easy
to support other distributed processing frameworks in the future. As
an evidence of this claim, the MRQL processor required only 2K extra
lines of Java code to support BSP evaluation.
= Initial Goals =
Some current goals include:
* apply MRQL to graph analysis problems, such as k-means clustering
and !PageRank
* apply MRQL to large-scale scientific analysis (develop general
optimization techniques that can apply to matrix multiplication,
matrix factorization, etc)
* process additional data formats, such as
[[http://avro.apache.org/|Avro]], and column-based stores, such as
[[http://hbase.apache.org/|HBase]]
* map MRQL to additional distributed processing frameworks, such as
[[http://spark-project.org/|Spark]] and
[[http://www.open-mpi.org/|OpenMPI]]
* extend the front-end to process more query languages, such as
standard SQL, SPARQL, XQuery, and !PigLatin
= Current Status =
The current MRQL release (version 0.8.10) is a beta release. It is
built on top of Hadoop and Hama (no extensions are needed).
It currently works on Hadoop up to 1.0.4 (but not on Yarn yet) and Hama 0.5.0.
It has only been tested on a small cluster of 20 nodes (80 cores).
== Meritocracy ==
The initial MRQL code base was developed by Leonidas Fegaras in May
2011, and was continuously improved throughout the years. We will
reach out other potential contributors through open forums.
We plan to do everything possible to encourage an environment that
supports a meritocracy, where contributors will extend their
privileges based on their contribution.
MRQL's modular design will facilitate the strategic extensions to
various modules, such as adding a standard-SQL interface, introducing
new optimization techniques, etc.
== Community ==
The interest in open-source query processing systems for analyzing
large datasets has been steadily increased in the last few years.
Related Apache projects have already attracted a very large community
from both academia and industry. We expect that MRQL will also
establish an active community.
Several researchers from both academia and industry who are interested
in using our code have already contacted us.
== Core Developers ==
The initial core developer was Leonidas Fegaras, who wrote the
majority of the code. He is an associate professor at UTA, with
interests in cloud computing, databases, web technologies, and
functional programming. He has an extensive knowledge and working
experience in building complex query processing systems for databases,
and compilers for functional and algorithmic programming languages.
== Alignment ==
MRQL is built on top of two Apache projects: Hadoop and Hama.
We have plans to incorporate other products from the Hadoop ecosystem,
such as Avro and HBase.
MRQL can serve as a testbed for fine-tuning and evaluating the
performance of the [[http://hama.apache.org/|Apache Hama]] system.
Finally, the MRQL query language and processor can be used by
[[DrillProposal|Apache Drill]] as a pluggable query language.
= Known Risks =
== Orphaned Products ==
The initial committer is from academia, which may be a risk, since
research in academia is publication-driven, rather than
product-driven.
It happens very often in academic research, when a project becomes
outdated and doesn't produce publishable results, to be abandoned in
favor of new cutting-edge projects.
We do not believe that this will be the case for MRQL for the years to
come, because it can be adapted to support new query languages, new
optimization techniques, and new distributed back-ends,
thus sustaining enough research interest.
Another risk is that, when graduate students who write code graduate,
they may leave their work undocumented and unfinished.
We will strive to gain enough momentum to recruit additional
committers from industry in order to eliminate these risks.
== Inexperience with Open Source ==
The initial developer has been involved with various projects whose
source code has been released under open source license, but he has no
prior experience on contributing to open-source projects.
With the guidance from other more experienced committers and
participants, we expect that the meritocracy rules will have a
positive influence on this project.
== Homogeneous Developers ==
The initial committer comes from academia. However, given the interest
we have seen in the project, we expect the diversity to improve in the
near future.
== Reliance on Salaried Developers ==
Currently, the MRQL code was developed on the committer's volunteer
time. In the future, UTA graduate students who will do some of the
coding may be supported by UTA and funding agencies, such as NSF.
== Relationships with Other Apache Products ==
MRQL has some overlapping functionality with
[[http://hive.apache.org/|Hive]] and [[TajoProposal|Tajo]], which are
Data Warehouse systems for Hadoop,
and with [[DrillProposal|Drill]], which is an interactive data
analysis system that can process nested data. MRQL has a more powerful
data model, in which any form of nested data, such as XML and JSON,
can be defined as a user-defined datatype. More importantly, complex
data analysis tasks, such as !PageRank, k-means clustering, and matrix
multiplication and factorization, can be expressed as short SQL-like
queries, while the MRQL system is able to evaluate these queries
efficiently. Furthermore, the MRQL system can run these queries in BSP
mode, in addition to !MapReduce mode, thus achieving low latency and
speed, which are also Drill's goals. Nevertheless, we will welcome and
encourage any help from these projects and we will be eager to make
contributions to these projects too.
== An Excessive Fascination with the Apache Brand ==
The Apache brand is likely to help us find contributors and reach out
to the open-source community.
Nevertheless, since MRQL depends on Apache projects (Hadoop and Hama),
it makes sense to have our software available as part of this
ecosystem.
= Documentation =
Information about MRQL can be found at
[[http://lambda.uta.edu/mrql/|MRQL: an Optimization Framework for
Map-Reduce Queries]]
= Initial Source =
The initial MRQL code has been released as part of a research project
developed at the University of Texas at Arlington under the Apache 2.0
license for the past two years.
The source code is currently hosted on !GitHub at:
[[https://github.com/fegaras/mrql|https://github.com/fegaras/mrql]].
MRQL’s release artifact would consist of a single tarball of packaging
and test code.
= External Dependencies =
The MRQL source code is already licensed under the Apache License,
Version 2.0. MRQL uses JLine which is distributed under the BSD
license.
= Cryptography =
Not applicable.
= Required Resources =
== Mailing Lists ==
* mrql-private
* mrql-dev
* mrql-user
== Subversion Directory ==
* Git is the preferred source control system: git://git.apache.org/mrql
== Issue Tracking ==
* A JIRA issue tracker, MRQL
== Wiki ==
* Moinmoin wiki, http://wiki.apache.org/mrql
= Initial Committers =
* Leonidas Fegaras <fegaras AT cse DOT uta DOT edu>
* Upa Gupta <upa.gupta AT mavs DOT uta DOT edu>
* Edward J. Yoon <edwardyoon AT apache DOT org>
* Maqsood Alam <maqsoodalam AT hotmail DOT com>
* John Hope <john.hope AT oracle DOT com>
* Mark Wall <mark.wall AT oracle DOT com>
* Kuassi Mensah <kuassi.mensah AT oracle DOT com>
* Ambreesh Khanna <ambreesh.khanna AT oracle DOT com>
* Karthik Kambatla <kasha AT cloudera DOT com>
= Affiliations =
* Leonidas Fegaras (University of Texas at Arlington)
* Upa Gupta (University of Texas at Arlington)
* Edward J. Yoon (Oracle corp)
* Maqsood Alam (Oracle corp)
* John Hope (Oracle corp)
* Mark Wall (Oracle corp)
* Kuassi Mensah (Oracle corp)
* Ambreesh Khanna (Oracle corp)
* Karthik Kambatla (Cloudera)
= Sponsors =
== Champion ==
* Edward J. Yoon <edwardyoon AT apache DOT org>
== Nominated Mentors ==
* Alan Cabrera <adc AT apache DOT org>
* Anthony Elder <antelder AT apache DOT org>
* Alex Karasulu <akarasulu AT apache DOT org>
* Mohammad Nour <mnour AT apache DOT org>
* Edward J. Yoon <edwardyoon AT apache DOT org>
== Sponsoring Entity ==
Incubator PMC
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [RESULT][VOTE] Accept MRQL into the Incubator
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Sun, Mar 17, 2013 at 4:01 PM, Mohammad Nour El-Din
<no...@gmail.com> wrote:
> On Sun, Mar 17, 2013 at 11:49 AM, Alex Karasulu <ak...@apache.org>wrote:
>> We're also missing Ant Elder from the Nominated Mentors list no?
> Fixed...
I have pasted the current version of the proposal below, so that this
list has it correctly archived.
+1 to incubation based on that version with four mentors.
-Bertrand
** PROPOSAL **
Abstract
MRQL is a query processing and optimization system for large-scale,
distributed data analysis, built on top of Apache Hadoop and Hama.
Proposal
MRQL (pronounced miracle) is a query processing and optimization
system for large-scale, distributed data analysis. MRQL (the MapReduce
Query Language) is an SQL-like query language for large-scale data
analysis on a cluster of computers. The MRQL query processing system
can evaluate MRQL queries in two modes: in MapReduce mode on top of
Apache Hadoop or in Bulk Synchronous Parallel (BSP) mode on top of
Apache Hama. The MRQL query language is powerful enough to express
most common data analysis tasks over many forms of raw in-situ data,
such as XML and JSON documents, binary files, and CSV documents. MRQL
is more powerful than other current high-level MapReduce languages,
such as Hive and PigLatin, since it can operate on more complex data
and supports more powerful query constructs, thus eliminating the need
for using explicit MapReduce code. With MRQL, users will be able to
express complex data analysis tasks, such as PageRank, k-means
clustering, matrix factorization, etc, using SQL-like queries
exclusively, while the MRQL query processing system will be able to
compile these queries to efficient Java code.
Background
The initial code was developed at the University of Texas of Arlington
(UTA) by a research team, led by Leonidas Fegaras. The software was
first released in May 2011. The original goal of this project was to
build a query processing system that translates SQL-like data analysis
queries to efficient workflows of MapReduce jobs. A design goal was to
use HDFS as the physical storage layer, without any indexing, data
partitioning, or data normalization, and to use Hadoop (without
extensions) as the run-time engine. The motivation behind this work
was to build a platform to test new ideas on query processing and
optimization techniques applicable to the MapReduce framework.
A year ago, MRQL was extended to run on Hama. The motivation for this
extension was that Hadoop MapReduce jobs were required to read their
input and write their output on HDFS. This simplifies reliability and
fault tolerance but it imposes a high overhead to complex MapReduce
workflows and graph algorithms, such as PageRank, which require
repetitive jobs. In addition, Hadoop does not preserve data in memory
across consecutive MapReduce jobs. This restriction requires to read
data at every step, even when the data is constant. BSP, on the other
hand, does not suffer from this restriction, and, under certain
circumstances, allows complex repetitive algorithms to run entirely in
the collective memory of a cluster. Thus, the goal was to be able to
run the same MRQL queries in both modes, MapReduce and BSP, without
modifying the queries: If there are enough resources available, and
low latency and speed are more important than resilience, queries may
run in BSP mode; otherwise, the same queries may run in MapReduce
mode. BSP evaluation was found to be a good choice when fault
tolerance is not critical, data (both input and intermediate) can fit
in the cluster memory, and data processing requires complex/repetitive
steps.
The research results of this ongoing work have already been published
in conferences (WebDB'11, EDBT'12, and DataCloud'12) and the authors
have already received positive feedback from researchers in academia
and industry who were attending these conferences.
Rationale
MRQL will be the first general-purpose, SQL-like query language for
data analysis based on BSP.
Currently, many programmers prefer to code their MapReduce
applications in a higher-level query language, rather than an
algorithmic language. For instance, Pig is used for 60% of Yahoo!
MapReduce jobs, while Hive is used for 90% of Facebook MapReduce jobs.
This, we believe, will also be the trend for BSP applications,
because, even though, in principle, the BSP model is very simple to
understand, it is hard to develop, optimize, and maintain non-trivial
BSP applications coded in a general-purpose programming language.
Currently, there is no widely acceptable declarative BSP query
language, although there are a few special-purpose BSP systems for
graph analysis, such as Google Pregel and Apache Giraph, for machine
learning, such as BSML, and for scientific data analysis.
MRQL can capture many complex data analysis algorithms in declarative form.
Existing MapReduce query languages, such as HiveQL and PigLatin,
provide a limited syntax for operating on data collections, in the
form of relational joins and group-bys. Because of these limitations,
these languages enable users to plug-in custom MapReduce scripts into
their queries for those jobs that cannot be declaratively coded in
their query language. This nullifies the benefits of using a declarative
query language and may result to suboptimal, error-prone, and
hard-to-maintain code. More importantly, these languages are
inappropriate for complex scientific applications and graph analysis,
because they do not directly support iteration or recursion in
declarative form and are not able to handle complex, nested scientific
data, which are often semi-structured. Furthermore, current MapReduce
query processors apply traditional query optimization techniques that
may be suboptimal in a MapReduce or BSP environment.
The MRQL design is modular, with pluggable distributed processing
back-ends, query languages, and data formats.
MRQL aims to be both powerful and adaptable. Although Hadoop is
currently the most popular framework for large-scale data analysis,
there are a few alternatives that are currently shaping form,
including frameworks based on BSP (eg, Giraph, Pregel, Hama), MPI (eg,
OpenMPI), etc. MRQL was designed in such a way so that it will be easy
to support other distributed processing frameworks in the future. As
an evidence of this claim, the MRQL processor required only 2K extra
lines of Java code to support BSP evaluation.
Initial Goals
Some current goals include:
apply MRQL to graph analysis problems, such as k-means clustering and PageRank
apply MRQL to large-scale scientific analysis (develop general
optimization techniques that can apply to matrix multiplication,
matrix factorization, etc)
process additional data formats, such as Avro, and column-based
stores, such as HBase
map MRQL to additional distributed processing frameworks, such as
Spark and OpenMPI
extend the front-end to process more query languages, such as standard
SQL, SPARQL, XQuery, and PigLatin
Current Status
The current MRQL release (version 0.8.10) is a beta release. It is
built on top of Hadoop and Hama (no extensions are needed). It
currently works on Hadoop up to 1.0.4 (but not on Yarn yet) and Hama
0.5.0. It has only been tested on a small cluster of 20 nodes (80
cores).
Meritocracy
The initial MRQL code base was developed by Leonidas Fegaras in May
2011, and was continuously improved throughout the years. We will
reach out other potential contributors through open forums. We plan to
do everything possible to encourage an environment that supports a
meritocracy, where contributors will extend their privileges based on
their contribution. MRQL's modular design will facilitate the
strategic extensions to various modules, such as adding a standard-SQL
interface, introducing new optimization techniques, etc.
Community
The interest in open-source query processing systems for analyzing
large datasets has been steadily increased in the last few years.
Related Apache projects have already attracted a very large community
from both academia and industry. We expect that MRQL will also
establish an active community. Several researchers from both academia
and industry who are interested in using our code have already
contacted us.
Core Developers
The initial core developer was Leonidas Fegaras, who wrote the
majority of the code. He is an associate professor at UTA, with
interests in cloud computing, databases, web technologies, and
functional programming. He has an extensive knowledge and working
experience in building complex query processing systems for databases,
and compilers for functional and algorithmic programming languages.
Alignment
MRQL is built on top of two Apache projects: Hadoop and Hama. We have
plans to incorporate other products from the Hadoop ecosystem, such as
Avro and HBase. MRQL can serve as a testbed for fine-tuning and
evaluating the performance of the Apache Hama system. Finally, the
MRQL query language and processor can be used by Apache Drill as a
pluggable query language.
Known Risks
Orphaned Products
The initial committer is from academia, which may be a risk, since
research in academia is publication-driven, rather than
product-driven. It happens very often in academic research, when a
project becomes outdated and doesn't produce publishable results, to
be abandoned in favor of new cutting-edge projects. We do not believe
that this will be the case for MRQL for the years to come, because it
can be adapted to support new query languages, new optimization
techniques, and new distributed back-ends, thus sustaining enough
research interest. Another risk is that, when graduate students who
write code graduate, they may leave their work undocumented and
unfinished. We will strive to gain enough momentum to recruit
additional committers from industry in order to eliminate these risks.
Inexperience with Open Source
The initial developer has been involved with various projects whose
source code has been released under open source license, but he has no
prior experience on contributing to open-source projects. With the
guidance from other more experienced committers and participants, we
expect that the meritocracy rules will have a positive influence on
this project.
Homogeneous Developers
The initial committer comes from academia. However, given the interest
we have seen in the project, we expect the diversity to improve in the
near future.
Reliance on Salaried Developers
Currently, the MRQL code was developed on the committer's volunteer
time. In the future, UTA graduate students who will do some of the
coding may be supported by UTA and funding agencies, such as NSF.
Relationships with Other Apache Products
MRQL has some overlapping functionality with Hive and Tajo, which are
Data Warehouse systems for Hadoop, and with Drill, which is an
interactive data analysis system that can process nested data. MRQL
has a more powerful data model, in which any form of nested data, such
as XML and JSON, can be defined as a user-defined datatype. More
importantly, complex data analysis tasks, such as PageRank, k-means
clustering, and matrix multiplication and factorization, can be
expressed as short SQL-like queries, while the MRQL system is able to
evaluate these queries efficiently. Furthermore, the MRQL system can
run these queries in BSP mode, in addition to MapReduce mode, thus
achieving low latency and speed, which are also Drill's goals.
Nevertheless, we will welcome and encourage any help from these
projects and we will be eager to make contributions to these projects
too.
An Excessive Fascination with the Apache Brand
The Apache brand is likely to help us find contributors and reach out
to the open-source community. Nevertheless, since MRQL depends on
Apache projects (Hadoop and Hama), it makes sense to have our software
available as part of this ecosystem.
Documentation
Information about MRQL can be found at MRQL: an Optimization Framework
for Map-Reduce Queries
Initial Source
The initial MRQL code has been released as part of a research project
developed at the University of Texas at Arlington under the Apache 2.0
license for the past two years. The source code is currently hosted on
GitHub at: https://github.com/fegaras/mrql. MRQL’s release artifact
would consist of a single tarball of packaging and test code.
External Dependencies
The MRQL source code is already licensed under the Apache License,
Version 2.0. MRQL uses JLine which is distributed under the BSD
license.
Cryptography
Not applicable.
Required Resources
Mailing Lists
mrql-private
mrql-dev
mrql-user
Subversion Directory
Git is the preferred source control system: git://git.apache.org/mrql
Issue Tracking
A JIRA issue tracker, MRQL
Wiki
Moinmoin wiki, http://wiki.apache.org/mrql
Initial Committers
Leonidas Fegaras <fegaras AT cse DOT uta DOT edu>
Upa Gupta <upa.gupta AT mavs DOT uta DOT edu>
Edward J. Yoon <edwardyoon AT apache DOT org>
Maqsood Alam <maqsoodalam AT hotmail DOT com>
John Hope <john.hope AT oracle DOT com>
Mark Wall <mark.wall AT oracle DOT com>
Kuassi Mensah <kuassi.mensah AT oracle DOT com>
Ambreesh Khanna <ambreesh.khanna AT oracle DOT com>
Karthik Kambatla <kasha AT cloudera DOT com>
Affiliations
Leonidas Fegaras (University of Texas at Arlington)
Upa Gupta (University of Texas at Arlington)
Edward J. Yoon (Oracle corp)
Maqsood Alam (Oracle corp)
John Hope (Oracle corp)
Mark Wall (Oracle corp)
Kuassi Mensah (Oracle corp)
Ambreesh Khanna (Oracle corp)
Karthik Kambatla (Cloudera)
Sponsors
Champion
Alex Karasulu <akarasulu AT apache DOT org>
Nominated Mentors
Anthony Elder <antelder AT apache DOT org>
Alex Karasulu <akarasulu AT apache DOT org>
Mohammad Nour <mnour AT apache DOT org>
Alan Cabrera <adc AT apache DOT org>
Sponsoring Entity
Incubator PMC
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [RESULT][VOTE] Accept MRQL into the Incubator
Posted by Mohammad Nour El-Din <no...@gmail.com>.
Hi
On Sun, Mar 17, 2013 at 11:49 AM, Alex Karasulu <ak...@apache.org>wrote:
> We're also missing Ant Elder from the Nominated Mentors list no?
>
Fixed
>
>
> On Sun, Mar 17, 2013 at 9:45 AM, ant elder <an...@gmail.com> wrote:
>
> > On Sat, Mar 16, 2013 at 11:07 PM, Edward J. Yoon <edwardyoon@apache.org
> > >wrote:
> >
> > > The required action has been taken, so let me close this thread again.
> > > I apologize again for my mistake.
> > >
> > > The Sponsors are changed as following:
> > >
> > > == Champion ==
> > >
> > > * Alex Karasulu <akarasulu AT apache DOT org>
> > >
> > > == Nominated Mentors ==
> > >
> > > * Alex Karasulu <akarasulu AT apache DOT org>
> > > * Mohammad Nour <mnour AT apache DOT org>
> > > * Alan Cabrera <adc AT apache DOT org>
> > >
> > > The following IPMCers voted in favor:
> > >
> > > * Mohammed Nour El-Din
> > > * Alex Karasulu
> > > * Tommaso Teofili
> > > * Chris Mattmann
> > > * Christian Grobmeier
> > > * Niall Pemberton
> > >
> > > Thanks.
> > >
> > >
> > Actually Edward we've discussed this within the Incubator PMC and the
> plan
> > is to respect the original vote and leave you on as champion and mentor,
> > the only change being the additional mentors. You're an Officer of the
> ASF
> > which is close enough and there are now a lot of mentors who can provide
> > any additional help should it be needed. Sorry from the Incubator that
> this
> > got into such a mess.
> >
> > ...ant
> >
>
>
>
> --
> Best Regards,
> -- Alex
>
--
Thanks
- Mohammad Nour
----
"Life is like riding a bicycle. To keep your balance you must keep moving"
- Albert Einstein
Re: [RESULT][VOTE] Accept MRQL into the Incubator
Posted by Alex Karasulu <ak...@apache.org>.
We're also missing Ant Elder from the Nominated Mentors list no?
On Sun, Mar 17, 2013 at 9:45 AM, ant elder <an...@gmail.com> wrote:
> On Sat, Mar 16, 2013 at 11:07 PM, Edward J. Yoon <edwardyoon@apache.org
> >wrote:
>
> > The required action has been taken, so let me close this thread again.
> > I apologize again for my mistake.
> >
> > The Sponsors are changed as following:
> >
> > == Champion ==
> >
> > * Alex Karasulu <akarasulu AT apache DOT org>
> >
> > == Nominated Mentors ==
> >
> > * Alex Karasulu <akarasulu AT apache DOT org>
> > * Mohammad Nour <mnour AT apache DOT org>
> > * Alan Cabrera <adc AT apache DOT org>
> >
> > The following IPMCers voted in favor:
> >
> > * Mohammed Nour El-Din
> > * Alex Karasulu
> > * Tommaso Teofili
> > * Chris Mattmann
> > * Christian Grobmeier
> > * Niall Pemberton
> >
> > Thanks.
> >
> >
> Actually Edward we've discussed this within the Incubator PMC and the plan
> is to respect the original vote and leave you on as champion and mentor,
> the only change being the additional mentors. You're an Officer of the ASF
> which is close enough and there are now a lot of mentors who can provide
> any additional help should it be needed. Sorry from the Incubator that this
> got into such a mess.
>
> ...ant
>
--
Best Regards,
-- Alex
Re: [RESULT][VOTE] Accept MRQL into the Incubator
Posted by ant elder <an...@gmail.com>.
On Sat, Mar 16, 2013 at 11:07 PM, Edward J. Yoon <ed...@apache.org>wrote:
> The required action has been taken, so let me close this thread again.
> I apologize again for my mistake.
>
> The Sponsors are changed as following:
>
> == Champion ==
>
> * Alex Karasulu <akarasulu AT apache DOT org>
>
> == Nominated Mentors ==
>
> * Alex Karasulu <akarasulu AT apache DOT org>
> * Mohammad Nour <mnour AT apache DOT org>
> * Alan Cabrera <adc AT apache DOT org>
>
> The following IPMCers voted in favor:
>
> * Mohammed Nour El-Din
> * Alex Karasulu
> * Tommaso Teofili
> * Chris Mattmann
> * Christian Grobmeier
> * Niall Pemberton
>
> Thanks.
>
>
Actually Edward we've discussed this within the Incubator PMC and the plan
is to respect the original vote and leave you on as champion and mentor,
the only change being the additional mentors. You're an Officer of the ASF
which is close enough and there are now a lot of mentors who can provide
any additional help should it be needed. Sorry from the Incubator that this
got into such a mess.
...ant
[RESULT][VOTE] Accept MRQL into the Incubator
Posted by "Edward J. Yoon" <ed...@apache.org>.
The required action has been taken, so let me close this thread again.
I apologize again for my mistake.
The Sponsors are changed as following:
== Champion ==
* Alex Karasulu <akarasulu AT apache DOT org>
== Nominated Mentors ==
* Alex Karasulu <akarasulu AT apache DOT org>
* Mohammad Nour <mnour AT apache DOT org>
* Alan Cabrera <adc AT apache DOT org>
The following IPMCers voted in favor:
* Mohammed Nour El-Din
* Alex Karasulu
* Tommaso Teofili
* Chris Mattmann
* Christian Grobmeier
* Niall Pemberton
Thanks.
On Sat, Mar 16, 2013 at 10:37 PM, Alex Karasulu <ak...@apache.org> wrote:
> Looks like we can call this finally closed and approved. Edward I think we
> can carry forward now.
>
>
> On Sat, Mar 16, 2013 at 10:23 AM, Christian Grobmeier
> <gr...@gmail.com>wrote:
>
>> >> True. I'm totally down with your reasoning.
>> >>
>> >> But don't you think it might be a formality we need to comply with,
>> even if
>> >> it does not make sense at this stage?
>> >>
>> >>
>> > There has been a vote, which passed with enough binding votes, no one has
>> > -1'd, and no one has retracted their vote. Lets wait till Monday and if
>> the
>> > majority vote still passes just carry on with the additional mentors. I
>> > hope no one does -1, theres several experienced mentors now so nothing
>> > really to be gained from forcing some new vote.
>> >
>> > ...ant
>>
>> As already mentioned I am also +1 to this proposal now,
>>
>>
>> --
>> http://www.grobmeier.de
>> https://www.timeandbill.de
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>>
>
>
> --
> Best Regards,
> -- Alex
--
Best Regards, Edward J. Yoon
@eddieyoon
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Alex Karasulu <ak...@apache.org>.
Looks like we can call this finally closed and approved. Edward I think we
can carry forward now.
On Sat, Mar 16, 2013 at 10:23 AM, Christian Grobmeier
<gr...@gmail.com>wrote:
> >> True. I'm totally down with your reasoning.
> >>
> >> But don't you think it might be a formality we need to comply with,
> even if
> >> it does not make sense at this stage?
> >>
> >>
> > There has been a vote, which passed with enough binding votes, no one has
> > -1'd, and no one has retracted their vote. Lets wait till Monday and if
> the
> > majority vote still passes just carry on with the additional mentors. I
> > hope no one does -1, theres several experienced mentors now so nothing
> > really to be gained from forcing some new vote.
> >
> > ...ant
>
> As already mentioned I am also +1 to this proposal now,
>
>
> --
> http://www.grobmeier.de
> https://www.timeandbill.de
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>
--
Best Regards,
-- Alex
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Christian Grobmeier <gr...@gmail.com>.
>> True. I'm totally down with your reasoning.
>>
>> But don't you think it might be a formality we need to comply with, even if
>> it does not make sense at this stage?
>>
>>
> There has been a vote, which passed with enough binding votes, no one has
> -1'd, and no one has retracted their vote. Lets wait till Monday and if the
> majority vote still passes just carry on with the additional mentors. I
> hope no one does -1, theres several experienced mentors now so nothing
> really to be gained from forcing some new vote.
>
> ...ant
As already mentioned I am also +1 to this proposal now,
--
http://www.grobmeier.de
https://www.timeandbill.de
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by ant elder <an...@gmail.com>.
On Fri, Mar 15, 2013 at 1:28 PM, Alex Karasulu <ak...@apache.org> wrote:
> On Fri, Mar 15, 2013 at 3:01 PM, ant elder <an...@gmail.com> wrote:
>
> > On Fri, Mar 15, 2013 at 12:16 PM, Alex Karasulu <akarasulu@apache.org
> > >wrote:
> >
> > > On Fri, Mar 15, 2013 at 10:21 AM, Edward J. Yoon <
> edwardyoon@apache.org
> > > >wrote:
> > >
> > > > > could you please close the "Create MRQL" tasks in the Infra Jira
> > until
> > > > > the situation has been cleared up. Whenever this all has been
> sorted
> > > > > out you can reopen the issues.
> > > >
> > > > Sure.
> > > >
> > > > Since this might not be fixed soon, proposal can be changed
> > > > (especially corporation volunteers). And, thank you for your
> > > > suggestion, but I'm already in initial committers list.
> > > >
> > > > To IPMC, I would request you to focus more on reviewing Proposal in
> > > > the future. The opportunity of MRQL should not be faded by my
> mistake.
> > > >
> > > >
> > > Bravo! There was no malice in the vote, just a simple mistake. If need
> > be I
> > > or Mo can take on the Champion role.
> > >
> > > Are there others from the IPMC that would be interested in mentoring
> > MRQL?
> > >
> > > --
> > > Best Regards,
> > > -- Alex
> > >
> >
> > The champion role is really just to help the proposal through any
> > discussion prior to being accepted as a poddling so thats been done and
> it
> > makes little difference now.
>
>
> True. I'm totally down with your reasoning.
>
> But don't you think it might be a formality we need to comply with, even if
> it does not make sense at this stage?
>
>
There has been a vote, which passed with enough binding votes, no one has
-1'd, and no one has retracted their vote. Lets wait till Monday and if the
majority vote still passes just carry on with the additional mentors. I
hope no one does -1, theres several experienced mentors now so nothing
really to be gained from forcing some new vote.
...ant
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Alex Karasulu <ak...@apache.org>.
On Fri, Mar 15, 2013 at 3:01 PM, ant elder <an...@gmail.com> wrote:
> On Fri, Mar 15, 2013 at 12:16 PM, Alex Karasulu <akarasulu@apache.org
> >wrote:
>
> > On Fri, Mar 15, 2013 at 10:21 AM, Edward J. Yoon <edwardyoon@apache.org
> > >wrote:
> >
> > > > could you please close the "Create MRQL" tasks in the Infra Jira
> until
> > > > the situation has been cleared up. Whenever this all has been sorted
> > > > out you can reopen the issues.
> > >
> > > Sure.
> > >
> > > Since this might not be fixed soon, proposal can be changed
> > > (especially corporation volunteers). And, thank you for your
> > > suggestion, but I'm already in initial committers list.
> > >
> > > To IPMC, I would request you to focus more on reviewing Proposal in
> > > the future. The opportunity of MRQL should not be faded by my mistake.
> > >
> > >
> > Bravo! There was no malice in the vote, just a simple mistake. If need
> be I
> > or Mo can take on the Champion role.
> >
> > Are there others from the IPMC that would be interested in mentoring
> MRQL?
> >
> > --
> > Best Regards,
> > -- Alex
> >
>
> The champion role is really just to help the proposal through any
> discussion prior to being accepted as a poddling so thats been done and it
> makes little difference now.
True. I'm totally down with your reasoning.
But don't you think it might be a formality we need to comply with, even if
it does not make sense at this stage?
Mohammed had volunteered to be a mentor so
> lets continue as that and I'll add myself as mentor too just to help if
> anymore is needed. So that gives plenty of mentors, we don't need more
> voting for that as the Incubator PMC can update mentors as it sees fit. So
> lets just continue on with that and the poddling accepted.
>
>
Based on your presence as a mentor along with Mo and myself I see this
poddling as accepted as well. If there's pressure to fill the Champion role
it can be handled on demand.
Ted, Dave is this situation now more acceptable and does it alleviate your
concerns? If not we want to know why and take steps to rectify the
situation accordingly. I personally really appreciate your diligence on
this matter. As a mentor I should have caught the problems you sited but
because of the Champion's ex-VP role (Hama?) I thought he was a member of
both the ASF and the IPMC. I should have checked.
--
Thanks,
-- Alex
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by ant elder <an...@gmail.com>.
On Fri, Mar 15, 2013 at 12:16 PM, Alex Karasulu <ak...@apache.org>wrote:
> On Fri, Mar 15, 2013 at 10:21 AM, Edward J. Yoon <edwardyoon@apache.org
> >wrote:
>
> > > could you please close the "Create MRQL" tasks in the Infra Jira until
> > > the situation has been cleared up. Whenever this all has been sorted
> > > out you can reopen the issues.
> >
> > Sure.
> >
> > Since this might not be fixed soon, proposal can be changed
> > (especially corporation volunteers). And, thank you for your
> > suggestion, but I'm already in initial committers list.
> >
> > To IPMC, I would request you to focus more on reviewing Proposal in
> > the future. The opportunity of MRQL should not be faded by my mistake.
> >
> >
> Bravo! There was no malice in the vote, just a simple mistake. If need be I
> or Mo can take on the Champion role.
>
> Are there others from the IPMC that would be interested in mentoring MRQL?
>
> --
> Best Regards,
> -- Alex
>
The champion role is really just to help the proposal through any
discussion prior to being accepted as a poddling so thats been done and it
makes little difference now. Mohammed had volunteered to be a mentor so
lets continue as that and I'll add myself as mentor too just to help if
anymore is needed. So that gives plenty of mentors, we don't need more
voting for that as the Incubator PMC can update mentors as it sees fit. So
lets just continue on with that and the poddling accepted.
...ant
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Alex Karasulu <ak...@apache.org>.
On Fri, Mar 15, 2013 at 10:21 AM, Edward J. Yoon <ed...@apache.org>wrote:
> > could you please close the "Create MRQL" tasks in the Infra Jira until
> > the situation has been cleared up. Whenever this all has been sorted
> > out you can reopen the issues.
>
> Sure.
>
> Since this might not be fixed soon, proposal can be changed
> (especially corporation volunteers). And, thank you for your
> suggestion, but I'm already in initial committers list.
>
> To IPMC, I would request you to focus more on reviewing Proposal in
> the future. The opportunity of MRQL should not be faded by my mistake.
>
>
Bravo! There was no malice in the vote, just a simple mistake. If need be I
or Mo can take on the Champion role.
Are there others from the IPMC that would be interested in mentoring MRQL?
--
Best Regards,
-- Alex
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Christian Grobmeier <gr...@gmail.com>.
On Fri, Mar 15, 2013 at 9:21 AM, Edward J. Yoon <ed...@apache.org> wrote:
>> could you please close the "Create MRQL" tasks in the Infra Jira until
>> the situation has been cleared up. Whenever this all has been sorted
>> out you can reopen the issues.
>
> Sure.
>
> Since this might not be fixed soon, proposal can be changed
> (especially corporation volunteers). And, thank you for your
> suggestion, but I'm already in initial committers list.
>
> To IPMC, I would request you to focus more on reviewing Proposal in
> the future. The opportunity of MRQL should not be faded by my mistake.
+1
I can tell you this issue is already discussed. I am pretty sure there
will be soon an outcome. Thanks for your patience!
Cheers
Christian
> On Fri, Mar 15, 2013 at 4:13 PM, Christian Grobmeier
> <gr...@gmail.com> wrote:
>> Hi Eddie,
>>
>> On Fri, Mar 15, 2013 at 8:02 AM, Ted Dunning <te...@gmail.com> wrote:
>>> No. This doesn't work. That sentence is followed by another:
>>>
>>> Where the Champion is not a Member of the Foundation (i.e. is an Officer
>>>> only), the Champion shall be a member of the PMC of the Sponsor.
>>>
>>>
>>> The Sponsor here is the Incubator. The PMC of the Sponsor is the IPMC.
>>> You aren't an IPMC member. Therefore, since you are not a Member and you
>>> aren't a member of the PMC of the Sponsor, you can't be the champion.
>>
>> could you please close the "Create MRQL" tasks in the Infra Jira until
>> the situation has been cleared up. Whenever this all has been sorted
>> out you can reopen the issues.
>>
>> On another note:
>>
>> There were now multiple issues with this podling. Can I make a
>> suggestion? What, if you would not join as a Mentor/Champion but as an
>> initial committer. As you said yourself you do not know all the rules
>> but you are willing to learn. Joining this podling would be a great
>> chance to learn all the rules, which are mandatory to know when you
>> want to mentor a podling.
>>
>> If that is ok, I furthermore suggest you write to general@ that you
>> are going to add your name to the initial committers list. I guess
>> nobody will object.
>>
>> Then there is a need for a champion; please ask on general@ in another
>> mail for somebody who has the interest.
>>
>> When it comes to mentors, most people say a podling should start with
>> 3. There are a few podlings with 2 mentors. I guess it could work out.
>>
>> Whenever you have the champion, please ask him to create a new
>> acceptance vote for MRQL. I guess this one would be ok since Alex and
>> Mo already +1, but I also feel it would be a good idea to make it
>> proper.
>>
>> My 2 cents - other IPMC members might see things different.
>>
>> Cheers
>> Christian
>>
>>
>>
>>> On Thu, Mar 14, 2013 at 9:43 PM, Edward J. Yoon <ed...@apache.org>wrote:
>>>
>>>> > As you have previously pointed out Champion and Mentors are required to
>>>> be IPMC members [1], [2]. So sorry you cannot be counted as a
>>>> mentor/champion for MRQL.
>>>>
>>>> Regarding Champion, Am I not a Officer of ASF? - "A candidate project
>>>> shall be sponsored by an Officer or Member of the Foundation."
>>
>>
>>
>> --
>> http://www.grobmeier.de
>> https://www.timeandbill.de
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
--
http://www.grobmeier.de
https://www.timeandbill.de
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by "Edward J. Yoon" <ed...@apache.org>.
> could you please close the "Create MRQL" tasks in the Infra Jira until
> the situation has been cleared up. Whenever this all has been sorted
> out you can reopen the issues.
Sure.
Since this might not be fixed soon, proposal can be changed
(especially corporation volunteers). And, thank you for your
suggestion, but I'm already in initial committers list.
To IPMC, I would request you to focus more on reviewing Proposal in
the future. The opportunity of MRQL should not be faded by my mistake.
On Fri, Mar 15, 2013 at 4:13 PM, Christian Grobmeier
<gr...@gmail.com> wrote:
> Hi Eddie,
>
> On Fri, Mar 15, 2013 at 8:02 AM, Ted Dunning <te...@gmail.com> wrote:
>> No. This doesn't work. That sentence is followed by another:
>>
>> Where the Champion is not a Member of the Foundation (i.e. is an Officer
>>> only), the Champion shall be a member of the PMC of the Sponsor.
>>
>>
>> The Sponsor here is the Incubator. The PMC of the Sponsor is the IPMC.
>> You aren't an IPMC member. Therefore, since you are not a Member and you
>> aren't a member of the PMC of the Sponsor, you can't be the champion.
>
> could you please close the "Create MRQL" tasks in the Infra Jira until
> the situation has been cleared up. Whenever this all has been sorted
> out you can reopen the issues.
>
> On another note:
>
> There were now multiple issues with this podling. Can I make a
> suggestion? What, if you would not join as a Mentor/Champion but as an
> initial committer. As you said yourself you do not know all the rules
> but you are willing to learn. Joining this podling would be a great
> chance to learn all the rules, which are mandatory to know when you
> want to mentor a podling.
>
> If that is ok, I furthermore suggest you write to general@ that you
> are going to add your name to the initial committers list. I guess
> nobody will object.
>
> Then there is a need for a champion; please ask on general@ in another
> mail for somebody who has the interest.
>
> When it comes to mentors, most people say a podling should start with
> 3. There are a few podlings with 2 mentors. I guess it could work out.
>
> Whenever you have the champion, please ask him to create a new
> acceptance vote for MRQL. I guess this one would be ok since Alex and
> Mo already +1, but I also feel it would be a good idea to make it
> proper.
>
> My 2 cents - other IPMC members might see things different.
>
> Cheers
> Christian
>
>
>
>> On Thu, Mar 14, 2013 at 9:43 PM, Edward J. Yoon <ed...@apache.org>wrote:
>>
>>> > As you have previously pointed out Champion and Mentors are required to
>>> be IPMC members [1], [2]. So sorry you cannot be counted as a
>>> mentor/champion for MRQL.
>>>
>>> Regarding Champion, Am I not a Officer of ASF? - "A candidate project
>>> shall be sponsored by an Officer or Member of the Foundation."
>
>
>
> --
> http://www.grobmeier.de
> https://www.timeandbill.de
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
--
Best Regards, Edward J. Yoon
@eddieyoon
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Christian Grobmeier <gr...@gmail.com>.
Hi Eddie,
On Fri, Mar 15, 2013 at 8:02 AM, Ted Dunning <te...@gmail.com> wrote:
> No. This doesn't work. That sentence is followed by another:
>
> Where the Champion is not a Member of the Foundation (i.e. is an Officer
>> only), the Champion shall be a member of the PMC of the Sponsor.
>
>
> The Sponsor here is the Incubator. The PMC of the Sponsor is the IPMC.
> You aren't an IPMC member. Therefore, since you are not a Member and you
> aren't a member of the PMC of the Sponsor, you can't be the champion.
could you please close the "Create MRQL" tasks in the Infra Jira until
the situation has been cleared up. Whenever this all has been sorted
out you can reopen the issues.
On another note:
There were now multiple issues with this podling. Can I make a
suggestion? What, if you would not join as a Mentor/Champion but as an
initial committer. As you said yourself you do not know all the rules
but you are willing to learn. Joining this podling would be a great
chance to learn all the rules, which are mandatory to know when you
want to mentor a podling.
If that is ok, I furthermore suggest you write to general@ that you
are going to add your name to the initial committers list. I guess
nobody will object.
Then there is a need for a champion; please ask on general@ in another
mail for somebody who has the interest.
When it comes to mentors, most people say a podling should start with
3. There are a few podlings with 2 mentors. I guess it could work out.
Whenever you have the champion, please ask him to create a new
acceptance vote for MRQL. I guess this one would be ok since Alex and
Mo already +1, but I also feel it would be a good idea to make it
proper.
My 2 cents - other IPMC members might see things different.
Cheers
Christian
> On Thu, Mar 14, 2013 at 9:43 PM, Edward J. Yoon <ed...@apache.org>wrote:
>
>> > As you have previously pointed out Champion and Mentors are required to
>> be IPMC members [1], [2]. So sorry you cannot be counted as a
>> mentor/champion for MRQL.
>>
>> Regarding Champion, Am I not a Officer of ASF? - "A candidate project
>> shall be sponsored by an Officer or Member of the Foundation."
--
http://www.grobmeier.de
https://www.timeandbill.de
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Ted Dunning <te...@gmail.com>.
No. This doesn't work. That sentence is followed by another:
Where the Champion is not a Member of the Foundation (i.e. is an Officer
> only), the Champion shall be a member of the PMC of the Sponsor.
The Sponsor here is the Incubator. The PMC of the Sponsor is the IPMC.
You aren't an IPMC member. Therefore, since you are not a Member and you
aren't a member of the PMC of the Sponsor, you can't be the champion.
On Thu, Mar 14, 2013 at 9:43 PM, Edward J. Yoon <ed...@apache.org>wrote:
> > As you have previously pointed out Champion and Mentors are required to
> be IPMC members [1], [2]. So sorry you cannot be counted as a
> mentor/champion for MRQL.
>
> Regarding Champion, Am I not a Officer of ASF? - "A candidate project
> shall be sponsored by an Officer or Member of the Foundation."
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by "Edward J. Yoon" <ed...@apache.org>.
OKay I see.
> As you have previously pointed out Champion and Mentors are required to be IPMC members [1], [2]. So sorry you cannot be counted as a mentor/champion for MRQL.
Regarding Champion, Am I not a Officer of ASF? - "A candidate project
shall be sponsored by an Officer or Member of the Foundation."
On Fri, Mar 15, 2013 at 1:29 PM, Suresh Marru <sm...@apache.org> wrote:
> On Mar 14, 2013, at 9:25 PM, Edward J. Yoon <ed...@apache.org> wrote:
>
>> Maybe 3 including myself?
>
> Hi Edward,
>
> As you have previously pointed out Champion and Mentors are required to be IPMC members [1], [2]. So sorry you cannot be counted as a mentor/champion for MRQL.
>
> Since you have asked to be added to IPMC [3], I assume you realize you are not part of IPMC. Casting a binding vote on EasyAnt graduation [3] is not right. Please avoid creating such confusions.
>
> Suresh
>
> [1] - http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Mentor
> [2] - http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Champion
> [3] - http://markmail.org/message/ldoyqm3o5eifyzrl
> [4] - http://markmail.org/message/dxqqa2cf6cpvoeyp
>
>> or If we need to recruit one more Mentor and
>> re-open the vote with properly updated proposal, Please let us know.
>>
>> I apologize that I made a mistake here. I was confused with this comment[1].
>>
>> 1. http://markmail.org/message/k6s56f5wlxgqhq3k
>>
>> On Fri, Mar 15, 2013 at 10:06 AM, Marvin Humphrey
>> <ma...@rectangular.com> wrote:
>>> On Thu, Mar 14, 2013 at 4:48 PM, Dave Fisher <da...@comcast.net> wrote:
>>>> Are the five binding votes from IPMC members?
>>>>
>>>> It is not 5.
>>>
>>> For the record, there were 4 binding IPMC votes in favor.
>>>
>>> Mohammed Nour El-Din
>>> Alex Karasulu
>>> Tommaso Teofili
>>> Chris Mattmann
>>>
>>>> Also, should proposals with only 1 valid Mentor be acceptable?
>>>
>>> In addition to Alex Karasulu, Mohammed Nour El-Din had volunteered,
>>> but seems to have been left off the version of the proposal which was
>>> VOTEd on. So unless Mohammed has withdrawn, MRQL has 2 Mentors.
>>>
>>> Marvin Humphrey
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
--
Best Regards, Edward J. Yoon
@eddieyoon
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Suresh Marru <sm...@apache.org>.
On Mar 14, 2013, at 9:25 PM, Edward J. Yoon <ed...@apache.org> wrote:
> Maybe 3 including myself?
Hi Edward,
As you have previously pointed out Champion and Mentors are required to be IPMC members [1], [2]. So sorry you cannot be counted as a mentor/champion for MRQL.
Since you have asked to be added to IPMC [3], I assume you realize you are not part of IPMC. Casting a binding vote on EasyAnt graduation [3] is not right. Please avoid creating such confusions.
Suresh
[1] - http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Mentor
[2] - http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Champion
[3] - http://markmail.org/message/ldoyqm3o5eifyzrl
[4] - http://markmail.org/message/dxqqa2cf6cpvoeyp
> or If we need to recruit one more Mentor and
> re-open the vote with properly updated proposal, Please let us know.
>
> I apologize that I made a mistake here. I was confused with this comment[1].
>
> 1. http://markmail.org/message/k6s56f5wlxgqhq3k
>
> On Fri, Mar 15, 2013 at 10:06 AM, Marvin Humphrey
> <ma...@rectangular.com> wrote:
>> On Thu, Mar 14, 2013 at 4:48 PM, Dave Fisher <da...@comcast.net> wrote:
>>> Are the five binding votes from IPMC members?
>>>
>>> It is not 5.
>>
>> For the record, there were 4 binding IPMC votes in favor.
>>
>> Mohammed Nour El-Din
>> Alex Karasulu
>> Tommaso Teofili
>> Chris Mattmann
>>
>>> Also, should proposals with only 1 valid Mentor be acceptable?
>>
>> In addition to Alex Karasulu, Mohammed Nour El-Din had volunteered,
>> but seems to have been left off the version of the proposal which was
>> VOTEd on. So unless Mohammed has withdrawn, MRQL has 2 Mentors.
>>
>> Marvin Humphrey
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by "Edward J. Yoon" <ed...@apache.org>.
Maybe 3 including myself? or If we need to recruit one more Mentor and
re-open the vote with properly updated proposal, Please let us know.
I apologize that I made a mistake here. I was confused with this comment[1].
1. http://markmail.org/message/k6s56f5wlxgqhq3k
On Fri, Mar 15, 2013 at 10:06 AM, Marvin Humphrey
<ma...@rectangular.com> wrote:
> On Thu, Mar 14, 2013 at 4:48 PM, Dave Fisher <da...@comcast.net> wrote:
>> Are the five binding votes from IPMC members?
>>
>> It is not 5.
>
> For the record, there were 4 binding IPMC votes in favor.
>
> Mohammed Nour El-Din
> Alex Karasulu
> Tommaso Teofili
> Chris Mattmann
>
>> Also, should proposals with only 1 valid Mentor be acceptable?
>
> In addition to Alex Karasulu, Mohammed Nour El-Din had volunteered,
> but seems to have been left off the version of the proposal which was
> VOTEd on. So unless Mohammed has withdrawn, MRQL has 2 Mentors.
>
> Marvin Humphrey
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
--
Best Regards, Edward J. Yoon
@eddieyoon
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [INVALID][RESULT][VOTE] Accept MRQL into the Incubator
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Thu, Mar 14, 2013 at 4:48 PM, Dave Fisher <da...@comcast.net> wrote:
> Are the five binding votes from IPMC members?
>
> It is not 5.
For the record, there were 4 binding IPMC votes in favor.
Mohammed Nour El-Din
Alex Karasulu
Tommaso Teofili
Chris Mattmann
> Also, should proposals with only 1 valid Mentor be acceptable?
In addition to Alex Karasulu, Mohammed Nour El-Din had volunteered,
but seems to have been left off the version of the proposal which was
VOTEd on. So unless Mohammed has withdrawn, MRQL has 2 Mentors.
Marvin Humphrey
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org