Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Konstantin Boudnik <co...@apache.org>]

Guys,

While catching up on my emails from the last a couple of weeks, this thread
caught my attention. I am not normally paying much attention to the code
reviews traffic from GH, but this one is pretty different as it spans three
months and counting.

First, here are my five cents:
 - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be contributed to
   an ASF project this file should simply contain ASL2 text, like in [1]
 - r/pom.xml perhaps shouldn't contain a separate <developers> section, but
   Zeppelin might have different guidelines on it. Say, Bigtop doesn't
   maintain this in the source code, but we have the list of all the
   committers on the project's site [2] Every new committer's first commit is
   to update the team page ;)
 - comments like in r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java

 +/**
 + * Created by aelberg on 7/28/15.
 + */ 
   
   is better to be removed. It has been already discussed here [3]. And the
   initial discussion on the topic [4] was linked as well
 - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is R-specific
   stuff - I have no idea about R, honestly, but 

    +License: GPL (>= 2) | BSD_3_clause + file LICENSE     
    ...
    +Author: David B. Dahl
   
   shouldn't be here, IMO. Normally, if some additional licenses are used,
   they have to be listed in the top-level NOTICE file (already there).

 - I am not going to offer any comments on the technical merits of the patch,
   as I haven't tried it personally. However, I don't see any serious
   technical objections to the functionality in question. 

So, the question is - why the PR is open for three months? I hasn't been able
to get a clear answer. What I found out though is pretty unsettling, really.
The communication on the topic is almost non-existent, except for this sparse
and bitter thread in the GH.

I would love to hear from the committers what's stopping the acceptance of the
code, besides of the minor issues I've mentioned earlier? What are the reasons for it?
Is there anything wrong with it?
One of the responsibilities of the committers is to make sure the
contributions are reviewed; new contributors are guided and do understand how
the project ticks. The easy feedback flow attracts new people, allowing the
community to grow and thrive.

I am asking _explicitely_ not to re-start the bickering I have already
seen. At this point I am interested in the purely technical side of this.

[1] https://github.com/apache/bigtop/blob/master/LICENSE
[2] http://bigtop.apache.org/team-list.html
[3] http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
[4] http://s.apache.org/iZl

With regards,
  Cos

On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> Github user elbamos commented on the pull request:
> 
>     https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
>   
>     The current push should resolve some issues with changes in the
>     Spark-Zeppelin interface that had created issues for users, as well as
>     support for 1.5.1.
>     
>     Knitr support is improved, and the reason for a separate knitr interpreter may be clearer now. 
>     
>     The amount of code borrowed from rscala is reduced. 
>     
>     I did not address issues with @author tags, or files under the R/
>     folder.  The reason is, to be blunt, I don't understand what the precise
>     concerns actually are.  
>     
>     Please note that I changed .travis.yml to only use spark 1.4 and later.
>     I'm sure there is a better way to do it, but I'm just not in a position
>     to try to figure out the entire Zeppelin build process.  
>     
>     Integrating this with the rest of zeppelin is going to take some work
>     regarding pom's, travis, and so forth.  I can do a lot of that, but I'm
>     going to need to discuss it with the people who have been "owning" those
>     files.  There are just too many moving pieces here.  
>     
>     Because of the size of this (which is, unfortunately, necessary),
>     posting here is probably not the most efficient way. That is also true
>     because certain people chose to use this PR as a forum to air other
>     issues.  Therefore, it would be better if reviewers e-mail me directly.  

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Konstantin Boudnik <co...@apache.org>]

Thanks for insightful reply, Moon. 

I want to address one point specifically though. If a code change is
relatively large and touches the existing source base in multiple places, it
makes sense to break it down in a smaller chunks and get them in one by one.

Firstly, reviewing smaller patches is easier for everyone.
Secondly, maintaining and rebasing a large patch over a long period of time is
no easy job easily. Especially, if the project is active and the patch is
touching existing code. Contributors time is also valuable - they are 
volunteering as well as everyone else.

Hopefully, with the two small things in mind, the community should be able to
find what works best when it comes to the accommodation of bigger changes.

You also mentioned elsewhere on this thread, that this isn't a single PR that
sits there for a long time. That's exactly the sign that something could be
improved along the way.

Cheers,
  Cos

On Tue, Dec 01, 2015 at 06:33AM, moon soo Lee wrote:
> Hi Cos,
> 
> Thanks for opening a discussion.
> My answer about 'Why this PR is open for three months' is
> 
> 1. This PR does not passes CI
> 2. Not 100% sure this PR has no license issue. (about KniteR)
> 3. Need more time to review.
> 
> It's my personal answer, other committers may have different opinion.
> 
> 
> I think the question should be generalized. Because this PR is not the only
> PR that is in impasse. There're more. For example
> 
> Here's some examples that PR is not been merged.
> https://github.com/apache/incubator-zeppelin/pull/53,
> https://github.com/apache/incubator-zeppelin/pull/60
> and so on.
> 
> I can categorize the cases, based on experience of involving Zeppelin
> community from the beginning.
> 
> 1. Large code base change
> 
> When contribution has large code base changes, it tend to take more time to
> review and merged. Normally, most contributions merged in 1day~1 week.
> But some contribution has large code base changes take 2~4 weeks. Few
> contribution that has very large code base change take months.
> 
> 2. Communication lost
> 
> Sometimes, committer is not responding, or contributor is not responding.
> 
> 3. Opinion diverges
> 
> For some changes, included in contribution. When people have different
> opinion and it continue to diverges, those PRs are not been merged.
> 
> 
> I think having a guide such as ping other committer when a committer is not
> responding, and divide contribution into small peaces if possible, would
> help most of the cases.
> Of course committer have to pay attention more to the contribution and
> helping people. That's the first one.
> 
> Thanks,
> moon
> 
> On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org> wrote:
> 
> > To make sure we're on the same page, here are two threads that I found
> > related
> > to this topic.
> >
> > Thread 1:
> >  Subject: R?
> >  Started on: July 1, 2015
> >
> > Thread 2:
> >  Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > Zeppelin
> >  Started on: August 13, 2015
> >
> > If you want to fetch these from our archive send emails to
> >         dev-thread.1735@zeppelin.incubator.apache.org
> >         dev-thread.3573@zeppelin.incubator.apache.org
> >
> > Cos
> >
> > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > Guys,
> > >
> > > While catching up on my emails from the last a couple of weeks, this
> > thread
> > > caught my attention. I am not normally paying much attention to the code
> > > reviews traffic from GH, but this one is pretty different as it spans
> > three
> > > months and counting.
> > >
> > > First, here are my five cents:
> > >  - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > contributed to
> > >    an ASF project this file should simply contain ASL2 text, like in [1]
> > >  - r/pom.xml perhaps shouldn't contain a separate <developers> section,
> > but
> > >    Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > >    maintain this in the source code, but we have the list of all the
> > >    committers on the project's site [2] Every new committer's first
> > commit is
> > >    to update the team page ;)
> > >  - comments like in
> > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > >
> > >  +/**
> > >  + * Created by aelberg on 7/28/15.
> > >  + */
> > >
> > >    is better to be removed. It has been already discussed here [3]. And
> > the
> > >    initial discussion on the topic [4] was linked as well
> > >  - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> > R-specific
> > >    stuff - I have no idea about R, honestly, but
> > >
> > >     +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > >     ...
> > >     +Author: David B. Dahl
> > >
> > >    shouldn't be here, IMO. Normally, if some additional licenses are
> > used,
> > >    they have to be listed in the top-level NOTICE file (already there).
> > >
> > >  - I am not going to offer any comments on the technical merits of the
> > patch,
> > >    as I haven't tried it personally. However, I don't see any serious
> > >    technical objections to the functionality in question.
> > >
> > > So, the question is - why the PR is open for three months? I hasn't been
> > able
> > > to get a clear answer. What I found out though is pretty unsettling,
> > really.
> > > The communication on the topic is almost non-existent, except for this
> > sparse
> > > and bitter thread in the GH.
> > >
> > > I would love to hear from the committers what's stopping the acceptance
> > of the
> > > code, besides of the minor issues I've mentioned earlier? What are the
> > reasons for it?
> > > Is there anything wrong with it?
> > > One of the responsibilities of the committers is to make sure the
> > > contributions are reviewed; new contributors are guided and do
> > understand how
> > > the project ticks. The easy feedback flow attracts new people, allowing
> > the
> > > community to grow and thrive.
> > >
> > > I am asking _explicitely_ not to re-start the bickering I have already
> > > seen. At this point I am interested in the purely technical side of this.
> > >
> > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > [2] http://bigtop.apache.org/team-list.html
> > > [3]
> > http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > [4] http://s.apache.org/iZl
> > >
> > > With regards,
> > >   Cos
> > >
> > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > Github user elbamos commented on the pull request:
> > > >
> > > >
> > https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > >
> > > >     The current push should resolve some issues with changes in the
> > > >     Spark-Zeppelin interface that had created issues for users, as
> > well as
> > > >     support for 1.5.1.
> > > >
> > > >     Knitr support is improved, and the reason for a separate knitr
> > interpreter may be clearer now.
> > > >
> > > >     The amount of code borrowed from rscala is reduced.
> > > >
> > > >     I did not address issues with @author tags, or files under the R/
> > > >     folder.  The reason is, to be blunt, I don't understand what the
> > precise
> > > >     concerns actually are.
> > > >
> > > >     Please note that I changed .travis.yml to only use spark 1.4 and
> > later.
> > > >     I'm sure there is a better way to do it, but I'm just not in a
> > position
> > > >     to try to figure out the entire Zeppelin build process.
> > > >
> > > >     Integrating this with the rest of zeppelin is going to take some
> > work
> > > >     regarding pom's, travis, and so forth.  I can do a lot of that,
> > but I'm
> > > >     going to need to discuss it with the people who have been "owning"
> > those
> > > >     files.  There are just too many moving pieces here.
> > > >
> > > >     Because of the size of this (which is, unfortunately, necessary),
> > > >     posting here is probably not the most efficient way. That is also
> > true
> > > >     because certain people chose to use this PR as a forum to air other
> > > >     issues.  Therefore, it would be better if reviewers e-mail me
> > directly.
> >
> >
> >

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Roman Shaposhnik <rv...@apache.org>]

Hi!

On Mon, Nov 30, 2015 at 10:33 PM, moon soo Lee <mo...@apache.org> wrote:
> Hi Cos,
>
> Thanks for opening a discussion.
> My answer about 'Why this PR is open for three months' is
>
> 1. This PR does not passes CI
> 2. Not 100% sure this PR has no license issue. (about KniteR)
> 3. Need more time to review.

After inspecting both implementation AND the bundle I'd like
to render my mentor AND an ASF member opinion here: the
PR (as implemented on 5th of December 2015) clearly does
NOT have any licensing issues. Thus, I'd like this discussion
between Zeppelin community and Amos to ONLY focus on
resolving #1 and #3. To me this is definitely a loop that needs
to be closed before we start any kind of serious conversation
around graduation.

For the curious ones, here's the basis for my opinion. First of all,
the dependency on R itself is mediated by the
    org.apache.zeppelin.rinterpreter.rscala.RClient
implementation that is clearly treating R as a self-contained
external process thus avoiding GPL implications. All the
communications are done via a network protocol over
sockets.

Second of all, a further dependency on knitr is mediated by
explicitly NOT automating the step of installing it, but rather
reporting to the user that it needs to be installed via the means
that R package management provides:
    https://github.com/apache/incubator-zeppelin/pull/208/files#diff-1b1779bbf782da9507bacc3396ab6676R297

Thanks,
Roman.

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

On Thu, Dec 3, 2015 at 4:54 AM Konstantin Boudnik <co...@apache.org> wrote:

> On Wed, Dec 02, 2015 at 01:17PM, Amos B. Elberg wrote:
> > Moon — I think there is a misunderstanding about the topic of the
> discussion.
> >
> > The PR has its own userbase.  It has been and is being presented at user
> > groups.  Its been blogged and tweeted about (none of that came from me!)
> >  The features are the subject of two jiras and on the Zeppelin roadmap.
> So,
> > the discussion isn't about whether the PR is “good."
>
> We aren't talking about the user-base or who sent what twits. Moreover,
> statements about the project's road map should only be made by the
> community
> working on the project. 3rd party opinions are irrelevant, unless said
> party
> is contributing something. You might be familiar with "Words are cheap,
> show
> me the code" saying. Apache is a do-ocracy: actions (ie the contributions)
> speak louder than anything else.
>
> > But no-one responded to the PR until users began to tweet publicly
> @nflabs
> > asking why the PR had not been adopted, and e-mailing you directly.  This
> > looks really bad, especially when the project is considering applying to
> > leave incubation.
>
> This is one of the things that worries me dearly. So, let's figure out how
> to
> make sure that PRs aren't sitting there for three months.
>
I am not really interested in who said what where. I intend to give people
> a benefit of the doubt and not interpret their actions as hostile or
> intentionally bad, unless I have evidence of such.
>
> Cos
>


Thanks Cos,

Following is one of the reasons for some PRs that they're sitting there,

1. Contributor opens a pullrequest,
2. Have discussions and decided add some commit,
3. Contributor adds more commits into pullrequest
4. Contributor wait for a review.
5. But committer does respond anymore.

I received direct email message about 5) from different contributors couple
of times. I think it happens when committer does not regularly check
pullrequest while 3) does not generate any automatic notification.

But checking every pullrequest manually is not practically possible,
especially when the pullrequest is not in active discussion. So guide
contributor, always leave a comment such as "review this", "ready to
review" after adding commit would improve one more case that make
contribution impasse.

I'd like to learn how the other project who uses github pullrequest deal
this problem.

Thanks,
moon



> > The question here is what, if anything, prevents us from letting bygones
> be
> > bygones and moving forward with this now?
> >
> > Claims about CI issues, or licenses, or the PR shouldn’t have been
> rebased
> > (!?!) — well, they don’t really make sense.
> >
> > I keep offering to begin coordinating to integrate the PR with
> Zeppelin’s CI
> > and build system.
> >
> > But the answer (except from Roman) is still “nah, let us know if you
> figure
> > it out.”
> >
> > Regarding the history:
> >
> > Konstantin wisely started this thread by saying let’s keep the history
> out
> > of the discussion.  I am respecting that.
> >
> > If the PR becomes part of Zeppelin, its going to need to be maintained,
> > which means that we are going to need to be able to work together.
> >
> > I have been persuaded to give Moon the benefit of the doubt regarding
> > certain issues.  He certainly knows what my view of the history is.
> >
> > If anyone else would like to know, I am happy to share it with them
> off-list.
> >
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> > Date: December 2, 2015 at 7:45:11 AM
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull request: R Interpreter for Zeppelin
> >
> > Thanks Roman and Eran for the feedback.
> >
> > *A. About contribution impasse in general*
> >
> > I think i summarized why it happens and how it can be improved. ie.
> >
> > 1. Large code base change
> > 2. Communication lost
> > 3. Opinion diverges
> >
> > And my solution was
> >
> > Guide to ping other committer when a committer is not responding, divide
> > contribution into small peaces if possible. And committer pay more
> > attention to the contribution.
> >
> > I'd like to hear and learn any more idea to improve.
> >
> >
> > *B. About contribution impasses in R interpreter for Zeppelin*
> >
> > Although I'was the first one who reviewed and commented this contribution
> > among the committer, I feel contributor (Amos) is unhappy about the
> review.
> >
> > I want to analyze the reasons and improve this, too.
> >
> > Here's reason i guess
> >
> > 1. Late responding (first review has been made after 3 months)
> > 2. Lack of help on CI fail (Amos keep complained about CI fail)
> >
> > I think both 1 and 2 can be improved by the solution i suggested in
> section
> > A.
> >
> > Amos, if you think there're more reasons, please feel free to say and let
> > me improve. What is the history you're mentioning?
> >
> > Best,
> > moon
> >
> >
> > On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org>
> wrote:
> >
> > > Just pushing discussion back on the list
> > >
> > > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com>
> wrote:
> > >
> > > > Alex — if you genuinely do not know the history of this, then I will
> fill
> > > > you in.
> > > >
> > > > lmk…
> > > >
> > > >
> > > > --
> > > > Amos Elberg
> > > > Sent with Airmail
> > > >
> > > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > Date: December 1, 2015 at 6:14:20 AM
> > > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org
> > > >
> > > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
> > > > <am...@gmail.com> <am...@gmail.com>
> > > >
> > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > > > pull request: R Interpreter for Zeppelin
> > > >
> > > > @Amos, we had plenty of cases of CI failing and always the
> pre-condition
> > > > for a merge was a green CI. Sometimes that requires time, polite
> > > > collaboration, extra mile in direct asking for help from more
> experienced
> > > > members and fixes in different places, which indeed might take time,
> as
> > > > everyone is busy.
> > > >
> > > >
> > >
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Konstantin Boudnik <co...@apache.org>]

On Wed, Dec 02, 2015 at 01:17PM, Amos B. Elberg wrote:
> Moon — I think there is a misunderstanding about the topic of the discussion. 
> 
> The PR has its own userbase.  It has been and is being presented at user
> groups.  Its been blogged and tweeted about (none of that came from me!)
>  The features are the subject of two jiras and on the Zeppelin roadmap.  So,
> the discussion isn't about whether the PR is “good."

We aren't talking about the user-base or who sent what twits. Moreover,
statements about the project's road map should only be made by the community
working on the project. 3rd party opinions are irrelevant, unless said party
is contributing something. You might be familiar with "Words are cheap, show
me the code" saying. Apache is a do-ocracy: actions (ie the contributions)
speak louder than anything else.

> But no-one responded to the PR until users began to tweet publicly @nflabs
> asking why the PR had not been adopted, and e-mailing you directly.  This
> looks really bad, especially when the project is considering applying to
> leave incubation.  

This is one of the things that worries me dearly. So, let's figure out how to
make sure that PRs aren't sitting there for three months.

I am not really interested in who said what where. I intend to give people
a benefit of the doubt and not interpret their actions as hostile or
intentionally bad, unless I have evidence of such.

Cos

> The question here is what, if anything, prevents us from letting bygones be
> bygones and moving forward with this now?
> 
> Claims about CI issues, or licenses, or the PR shouldn’t have been rebased
> (!?!) — well, they don’t really make sense.  
> 
> I keep offering to begin coordinating to integrate the PR with Zeppelin’s CI
> and build system.  
> 
> But the answer (except from Roman) is still “nah, let us know if you figure
> it out.”
> 
> Regarding the history:
> 
> Konstantin wisely started this thread by saying let’s keep the history out
> of the discussion.  I am respecting that. 
> 
> If the PR becomes part of Zeppelin, its going to need to be maintained,
> which means that we are going to need to be able to work together. 
> 
> I have been persuaded to give Moon the benefit of the doubt regarding
> certain issues.  He certainly knows what my view of the history is. 
> 
> If anyone else would like to know, I am happy to share it with them off-list. 
> 
> 
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Date: December 2, 2015 at 7:45:11 AM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  
> 
> Thanks Roman and Eran for the feedback.  
> 
> *A. About contribution impasse in general*  
> 
> I think i summarized why it happens and how it can be improved. ie.  
> 
> 1. Large code base change  
> 2. Communication lost  
> 3. Opinion diverges  
> 
> And my solution was  
> 
> Guide to ping other committer when a committer is not responding, divide  
> contribution into small peaces if possible. And committer pay more  
> attention to the contribution.  
> 
> I'd like to hear and learn any more idea to improve.  
> 
> 
> *B. About contribution impasses in R interpreter for Zeppelin*  
> 
> Although I'was the first one who reviewed and commented this contribution  
> among the committer, I feel contributor (Amos) is unhappy about the review.  
> 
> I want to analyze the reasons and improve this, too.  
> 
> Here's reason i guess  
> 
> 1. Late responding (first review has been made after 3 months)  
> 2. Lack of help on CI fail (Amos keep complained about CI fail)  
> 
> I think both 1 and 2 can be improved by the solution i suggested in section  
> A.  
> 
> Amos, if you think there're more reasons, please feel free to say and let  
> me improve. What is the history you're mentioning?  
> 
> Best,  
> moon  
> 
> 
> On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org> wrote:  
> 
> > Just pushing discussion back on the list  
> >  
> > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com> wrote:  
> >  
> > > Alex — if you genuinely do not know the history of this, then I will fill  
> > > you in.  
> > >  
> > > lmk…  
> > >  
> > >  
> > > --  
> > > Amos Elberg  
> > > Sent with Airmail  
> > >  
> > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>  
> > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>  
> > > Date: December 1, 2015 at 6:14:20 AM  
> > > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org  
> > >  
> > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg  
> > > <am...@gmail.com> <am...@gmail.com>  
> > >  
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin  
> > > pull request: R Interpreter for Zeppelin  
> > >  
> > > @Amos, we had plenty of cases of CI failing and always the pre-condition  
> > > for a merge was a green CI. Sometimes that requires time, polite  
> > > collaboration, extra mile in direct asking for help from more experienced  
> > > members and fixes in different places, which indeed might take time, as  
> > > everyone is busy.  
> > >  
> > >  
> >  

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Moon - 

You say publicly "I have some history about something/someone. And i can send you privately." But you can not bringing anything into the public even after i asked multiple times. Then asking me 'guess' what you meant. 
I didn’t bring this issue public.   Alex did. 

And you don’t need to guess.  You know perfectly well.  

But because of i think your way of communication is very wrong.
That isn’t bickering?

***

As far as I can tell, the only progress that’s been made in this entire thread, is that Roman agreed to look at the PR. 

I am disinclined to continue responding to this. 







From: moon soo Lee <mo...@apache.org>
Reply: moon soo Lee <mo...@apache.org>
Date: December 3, 2015 at 12:15:41 AM
To: Amos B. Elberg <am...@gmail.com>, dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  



On Thu, Dec 3, 2015 at 12:32 PM Amos B. Elberg <am...@gmail.com> wrote:
Is there any reason that you can't tell here? 
Please re-read the e-mail from Konstantine that started the thread. 

If you can't tell anything, Is it okay to me assume that you just wanted make negative noise in the community with unidentified words?
Please just share your thinking and let me hear and improve from it. 

I don’t see how having that discussion in public would serve any purpose. 
 
If you genuinely don’t understand — the end of our e-mail exchange was an invitation for you to call me on the phone and attempt to resolve those issues without the misunderstandings of tone that can happen in our e-mail exchange.  

The invitation stands. 

I will continue to look for ways to move forward in the spirit of cooperation and teamwork. 



Dear amos,

The reason i'm keep replying with this unpleasant response here is, not to bickering. But because of i think your way of communication is very wrong.

You say publicly "I have some history about something/someone. And i can send you privately." But you can not bringing anything into the public even after i asked multiple times. Then asking me 'guess' what you meant. 

I'm sorry, this is one of the worst way you can communicate in opensource project. I do not want to see this way of communication never again.

I don't think this is what you call sprit of cooperation and teamwork.
So please communicate the way you call sprit of cooperation and teamwork.

Thanks,
moon

 

From: moon soo Lee <mo...@apache.org>
Reply: moon soo Lee <mo...@apache.org>
Date: December 2, 2015 at 10:24:52 PM
To: Amos B. Elberg <am...@gmail.com>, dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>

Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin



On Thu, Dec 3, 2015 at 12:05 PM Amos B. Elberg <am...@gmail.com> wrote:
Moon,

I still suggest, one of the best way you can find that person is, file an 
issue about the CI problem that you found on Zeppelin Core. 
All of the relevant people are already informed.  

I continue to be ready to work cooperatively with anyone who would like to help me resolve the issues. 

I did clearly pointed what rule could be violated and provided a link and 
tried the best at explanation. 
Maybe this is a language issue, but I have no idea what your concern about a license is.  

You did cite to two web pages.  Neither of them said anything that would explain your position.  In fact, they confirmed what I had been saying all along.  

So i'm going to continue the discussion in the other thread about the 
license and will move the discussion to legal-discuss@. 
You’re welcome to do that, of course.  I don’t know that I will participate. 

What i'm not really fine is, having not enough discussion and concern about 
license. That's sign of unhealthy community. 
As Konstantine has pointed out:  The ASF has existed for 16 years.  This cannot be the first time that this issue arose.  The ASF has a “legal faq” and other public documents that discuss various licensing issues. 

In addition, a primary responsibility of the PMCC — perhaps the most important responsibility — is making sure the project conforms to ASF licensing policy. 

In terms of community health, my concern is that there is a great deal of confusion about these licensing issues, and people are not able to find a definitive answer simply by checking the ASF documents (or even attempting to do so).

(Hint:  I reviewed the ASF materials in great detail in preparing the PR.) 

I still don't know which part of the email are you referring. … What are you referring "the history"? 

You know *exactly* what I’m talking about. 


Amos,

This mailing list is subscribed by hundreds of people. Let's not trying to make meaningless posts.

I told you i *exactly* don't know what you're talking about "the history".
So why don't you pin point what *exactly* you talking about, in public?

Is there any reason that you can't tell here? 
If you can't tell anything, Is it okay to me assume that you just wanted make negative noise in the community with unidentified words?

Please just share your thinking and let me hear and improve from it. 

Thanks,
moon

 
From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 2, 2015 at 9:19:50 PM

To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

On Thu, Dec 3, 2015 at 7:34 AM Amos B. Elberg <am...@gmail.com> wrote:

> Moon, thank you for your reply.
>
> Of course, I can claim CI, license, etc and any other issue. Especially
> when CI is not passing, I can not sure about the license. To me, these
> claims are sign of 'healthy community' not sign of 'does not make sense'.
>
> You're not meaning, your contribution always need to be accepted without
> any claim. right?
> I believe the constructive way to move forward, would be for someone who
> well-understands the CI & build structure, to begin working with me to
> resolve the CI/build integration issues.
>
> I continue to be ready to work with that person.
>


I still suggest, one of the best way you can find that person is, file an
issue about the CI problem that you found on Zeppelin Core.
That'll reduce the scope of the problem and that'll help people quickly
getting into without understanding your contribution entirely.



>
> Regarding licensing, I took extensive research steps before submitting the
> PR to make sure there was not a licensing problem.
>
> In fact, many things in the structure of the code were chosen specifically
> to avoid any licensing problem. I even negotiated with the authors of some
> libraries to switch to Apache-compatible licenses, and did some work on
> their projects in exchange.
>
> Considering all of that — If anyone thinks there is a licensing issue, if
> they want to be constructive, the *least* they can do is say clearly
> exactly what rule is being violated, how it is being violated, and provide
> a link or citation or *something* that shows the opinion is more than
> hand-waving.
>
> I am ready to engage with anyone who does that.
>

There is separate thread for the license. So i'll leave minimal comment
here.

I did clearly pointed what rule could be violated and provided a link and
tried the best at explanation. You don't agree on my concern does not mean
it's okay to pass. Just like i don't agree on your opinion does not mean
it's confirmation of license problem.

Of course my concern could be wrong, that's totally fine to me. I don't
have any problem on that. I'm not a legal expert.

What i'm not really fine is, having not enough discussion and concern about
license. That's sign of unhealthy community.

So i'm going to continue the discussion in the other thread about the
license and will move the discussion to legal-discuss@.



>
> I don't know your view of history and what you think the history is.
> Yeah you do.
>
> We had an exchange about it a week before this thread began.
>
> Some of it even spilled-over into the PR comments after I saw the video.
>
>

I still don't know which part of the email are you referring. The
suggestion i made about your code? about the review? about the conference?
about the meetup? What are you referring "the history"?

Please say in public. What is the history and show how they're related to
this thread topic. Otherwise I'll assume you just want to make a negative
noise in the community.

Thanks,
moon



> So please share them in PUBLIC on this thread NOT off-list, if you think
> that's reason you think your contribution is in impasse.
>
> I am going to follow Konstantin’s lead reading this issue, and try to give
> you the benefit of the doubt.
>
>
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> Date: December 2, 2015 at 5:06:52 PM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> Thanks Amos for replying.
>
> On Thu, Dec 3, 2015 at 3:17 AM Amos B. Elberg <am...@gmail.com>
> wrote:
>
> > Moon — I think there is a misunderstanding about the topic of the
> > discussion.
> >
> > The PR has its own userbase. It has been and is being presented at user
> > groups. Its been blogged and tweeted about (none of that came from me!)
> > The features are the subject of two jiras and on the Zeppelin roadmap.
> > So, the discussion isn't about whether the PR is “good."
> >
> > But no-one responded to the PR until users began to tweet publicly
> @nflabs
> > asking why the PR had not been adopted, and e-mailing you directly. This
> > looks really bad, especially when the project is considering applying to
> > leave incubation.
> >
> >
>
> Thanks for pinging me. Otherwise i couldn't able to know that you're still
> working on it and ready to review. I think it's good practice that pinging
> committer for review when there is no sign of response. Except for ping
> message has been made on twitter and private email instead of public
> mailing list / jira / github issue comment.
>
>
>
>
> > The question here is what, if anything, prevents us from letting bygones
> > be bygones and moving forward with this now?
> >
> > Claims about CI issues, or licenses, or the PR shouldn’t have been
> rebased
> > (!?!) — well, they don’t really make sense.
> >
> >
> Although i didn't review your contribution from day 1,
> I'm reviewing your contribution, discussing about license, discussing about
> improvement of impasse, all they're part of moving forward. I am moving
> forward.
>
> Of course, I can claim CI, license, etc and any other issue. Especially
> when CI is not passing, I can not sure about the license. To me, these
> claims are sign of 'healthy community' not sign of 'does not make sense'.
>
> You're not meaning, your contribution always need to be accepted without
> any claim. right?
>
>
>
> > I keep offering to begin coordinating to integrate the PR with Zeppelin’s
> > CI and build system.
> >
> > But the answer (except from Roman) is still “nah, let us know if you
> > figure it out.”
> >
> >
> >
> Actually, my answer was, "If you think CI is failing not because of your
> change but because of Zeppelin core problem, then file an jira issue about
> it. Everyone will look into".
>
>
>
> >
> >
> > Regarding the history:
> >
> > Konstantin wisely started this thread by saying let’s keep the history
> out
> > of the discussion. I am respecting that.
> >
> > If the PR becomes part of Zeppelin, its going to need to be maintained,
> > which means that we are going to need to be able to work together.
> >
> > I have been persuaded to give Moon the benefit of the doubt regarding
> > certain issues. He certainly knows what my view of the history is.
> >
> > If anyone else would like to know, I am happy to share it with them
> > off-list.
> >
>
>
> I don't know your view of history and what you think the history is. So
> please share them in PUBLIC on this thread NOT off-list, if you think
> that's reason you think your contribution is in impasse.
>
> Otherwise I'll never know what you're thinking and I'll not improve. More
> importantly, it's easy to make people misunderstand you that you're just
> trying to make a negative noises.
>
> So, do you mind share your view of history in this thread?
>
> Thanks,
> moon
>
>
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > Date: December 2, 2015 at 7:45:11 AM
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> > request: R Interpreter for Zeppelin
> >
> > Thanks Roman and Eran for the feedback.
> >
> > *A. About contribution impasse in general*
> >
> > I think i summarized why it happens and how it can be improved. ie.
> >
> > 1. Large code base change
> > 2. Communication lost
> > 3. Opinion diverges
> >
> > And my solution was
> >
> > Guide to ping other committer when a committer is not responding, divide
> > contribution into small peaces if possible. And committer pay more
> > attention to the contribution.
> >
> > I'd like to hear and learn any more idea to improve.
> >
> >
> > *B. About contribution impasses in R interpreter for Zeppelin*
> >
> > Although I'was the first one who reviewed and commented this contribution
> > among the committer, I feel contributor (Amos) is unhappy about the
> review.
> >
> > I want to analyze the reasons and improve this, too.
> >
> > Here's reason i guess
> >
> > 1. Late responding (first review has been made after 3 months)
> > 2. Lack of help on CI fail (Amos keep complained about CI fail)
> >
> > I think both 1 and 2 can be improved by the solution i suggested in
> section
> > A.
> >
> > Amos, if you think there're more reasons, please feel free to say and let
> > me improve. What is the history you're mentioning?
> >
> > Best,
> > moon
> >
> >
> > On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org>
> wrote:
> >
> > > Just pushing discussion back on the list
> > >
> > > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com>
> wrote:
> > >
> > > > Alex — if you genuinely do not know the history of this, then I will
> > fill
> > > > you in.
> > > >
> > > > lmk…
> > > >
> > > >
> > > > --
> > > > Amos Elberg
> > > > Sent with Airmail
> > > >
> > > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > Date: December 1, 2015 at 6:14:20 AM
> > > > To: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org
> > > >
> > > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
> > > > <am...@gmail.com> <am...@gmail.com>
> > > >
> > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > > > pull request: R Interpreter for Zeppelin
> > > >
> > > > @Amos, we had plenty of cases of CI failing and always the
> > pre-condition
> > > > for a merge was a green CI. Sometimes that requires time, polite
> > > > collaboration, extra mile in direct asking for help from more
> > experienced
> > > > members and fixes in different places, which indeed might take time,
> as
> > > > everyone is busy.
> > > >
> > > >
> > >
> >
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

On Thu, Dec 3, 2015 at 12:32 PM Amos B. Elberg <am...@gmail.com>
wrote:

> Is there any reason that you can't tell here?
>
> Please re-read the e-mail from Konstantine that started the thread.
>
> If you can't tell anything, Is it okay to me assume that you just wanted
> make negative noise in the community with unidentified words?
>
> Please just share your thinking and let me hear and improve from it.
>
>
> I don’t see how having that discussion in public would serve any purpose.
>
>
If you genuinely don’t understand — the end of our e-mail exchange was an
> invitation for you to call me on the phone and attempt to resolve those
> issues without the misunderstandings of tone that can happen in our e-mail
> exchange.
>
> The invitation stands.
>
> I will continue to look for ways to move forward in the spirit of
> cooperation and teamwork.
>
>

Dear amos,

The reason i'm keep replying with this unpleasant response here is, not to
bickering. But because of i think your way of communication is very wrong.

You say publicly "I have some history about something/someone. And i can
send you privately." But you can not bringing anything into the public even
after i asked multiple times. Then asking me 'guess' what you meant.

I'm sorry, this is one of the worst way you can communicate in opensource
project. I do not want to see this way of communication never again.

I don't think this is what you call sprit of cooperation and teamwork.
So please communicate the way you call sprit of cooperation and teamwork.

Thanks,
moon



>
> From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> Reply: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> Date: December 2, 2015 at 10:24:52 PM
> To: Amos B. Elberg <am...@gmail.com> <am...@gmail.com>,
> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> <de...@zeppelin.incubator.apache.org>
>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull request: R Interpreter for Zeppelin
>
>
>
> On Thu, Dec 3, 2015 at 12:05 PM Amos B. Elberg <am...@gmail.com>
> wrote:
>
>> Moon,
>>
>> I still suggest, one of the best way you can find that person is, file an
>> issue about the CI problem that you found on Zeppelin Core.
>>
>> All of the relevant people are already informed.
>>
>> I continue to be ready to work cooperatively with anyone who would like
>> to help me resolve the issues.
>>
>> I did clearly pointed what rule could be violated and provided a link and
>> tried the best at explanation.
>>
>> Maybe this is a language issue, but I have no idea what your concern
>> about a license is.
>>
>> You did cite to two web pages.  Neither of them said anything that would
>> explain your position.  In fact, they confirmed what I had been saying all
>> along.
>>
>> So i'm going to continue the discussion in the other thread about the
>> license and will move the discussion to legal-discuss@.
>>
>> You’re welcome to do that, of course.  I don’t know that I will
>> participate.
>>
>> What i'm not really fine is, having not enough discussion and concern
>> about
>> license. That's sign of unhealthy community.
>>
>> As Konstantine has pointed out:  The ASF has existed for 16 years.  This
>> cannot be the first time that this issue arose.  The ASF has a “legal faq”
>> and other public documents that discuss various licensing issues.
>>
>> In addition, a primary responsibility of the PMCC — perhaps the most
>> important responsibility — is making sure the project conforms to ASF
>> licensing policy.
>>
>> In terms of community health, my concern is that there is a great deal of
>> confusion about these licensing issues, and people are not able to find a
>> definitive answer simply by checking the ASF documents (or even attempting
>> to do so).
>>
>> (Hint:  I reviewed the ASF materials in great detail in preparing the
>> PR.)
>>
>> I still don't know which part of the email are you referring. … What are
>> you referring "the history"?
>>
>>
>> You know *exactly* what I’m talking about.
>>
>>
> Amos,
>
> This mailing list is subscribed by hundreds of people. Let's not trying to
> make meaningless posts.
>
> I told you i *exactly* don't know what you're talking about "the history".
> So why don't you pin point what *exactly* you talking about, in public?
>
> Is there any reason that you can't tell here?
> If you can't tell anything, Is it okay to me assume that you just wanted
> make negative noise in the community with unidentified words?
>
> Please just share your thinking and let me hear and improve from it.
>
> Thanks,
> moon
>
>
>
>> From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
>> Reply: dev@zeppelin.incubator.apache.org
>> <de...@zeppelin.incubator.apache.org> <de...@zeppelin.incubator.apache.org>
>> Date: December 2, 2015 at 9:19:50 PM
>>
>> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
>> <de...@zeppelin.incubator.apache.org>
>> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>> pull request: R Interpreter for Zeppelin
>>
>> On Thu, Dec 3, 2015 at 7:34 AM Amos B. Elberg <am...@gmail.com>
>> wrote:
>>
>> > Moon, thank you for your reply.
>> >
>> > Of course, I can claim CI, license, etc and any other issue. Especially
>> > when CI is not passing, I can not sure about the license. To me, these
>> > claims are sign of 'healthy community' not sign of 'does not make
>> sense'.
>> >
>> > You're not meaning, your contribution always need to be accepted without
>> > any claim. right?
>> > I believe the constructive way to move forward, would be for someone who
>> > well-understands the CI & build structure, to begin working with me to
>> > resolve the CI/build integration issues.
>> >
>> > I continue to be ready to work with that person.
>> >
>>
>>
>> I still suggest, one of the best way you can find that person is, file an
>> issue about the CI problem that you found on Zeppelin Core.
>> That'll reduce the scope of the problem and that'll help people quickly
>> getting into without understanding your contribution entirely.
>>
>>
>>
>> >
>> > Regarding licensing, I took extensive research steps before submitting
>> the
>> > PR to make sure there was not a licensing problem.
>> >
>> > In fact, many things in the structure of the code were chosen
>> specifically
>> > to avoid any licensing problem. I even negotiated with the authors of
>> some
>> > libraries to switch to Apache-compatible licenses, and did some work on
>> > their projects in exchange.
>> >
>> > Considering all of that — If anyone thinks there is a licensing issue,
>> if
>> > they want to be constructive, the *least* they can do is say clearly
>> > exactly what rule is being violated, how it is being violated, and
>> provide
>> > a link or citation or *something* that shows the opinion is more than
>> > hand-waving.
>> >
>> > I am ready to engage with anyone who does that.
>> >
>>
>> There is separate thread for the license. So i'll leave minimal comment
>> here.
>>
>> I did clearly pointed what rule could be violated and provided a link and
>> tried the best at explanation. You don't agree on my concern does not mean
>> it's okay to pass. Just like i don't agree on your opinion does not mean
>> it's confirmation of license problem.
>>
>> Of course my concern could be wrong, that's totally fine to me. I don't
>> have any problem on that. I'm not a legal expert.
>>
>> What i'm not really fine is, having not enough discussion and concern
>> about
>> license. That's sign of unhealthy community.
>>
>> So i'm going to continue the discussion in the other thread about the
>> license and will move the discussion to legal-discuss@.
>>
>>
>>
>> >
>> > I don't know your view of history and what you think the history is.
>> > Yeah you do.
>> >
>> > We had an exchange about it a week before this thread began.
>> >
>> > Some of it even spilled-over into the PR comments after I saw the video.
>> >
>> >
>>
>> I still don't know which part of the email are you referring. The
>> suggestion i made about your code? about the review? about the conference?
>> about the meetup? What are you referring "the history"?
>>
>> Please say in public. What is the history and show how they're related to
>> this thread topic. Otherwise I'll assume you just want to make a negative
>> noise in the community.
>>
>> Thanks,
>> moon
>>
>>
>>
>> > So please share them in PUBLIC on this thread NOT off-list, if you think
>> > that's reason you think your contribution is in impasse.
>> >
>> > I am going to follow Konstantin’s lead reading this issue, and try to
>> give
>> > you the benefit of the doubt.
>> >
>> >
>> > From: moon soo Lee <mo...@apache.org>
>> > Reply: dev@zeppelin.incubator.apache.org <
>> > dev@zeppelin.incubator.apache.org>
>> > Date: December 2, 2015 at 5:06:52 PM
>> > To: dev@zeppelin.incubator.apache.org <
>> dev@zeppelin.incubator.apache.org>
>> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>> pull
>> > request: R Interpreter for Zeppelin
>> >
>> > Thanks Amos for replying.
>> >
>> > On Thu, Dec 3, 2015 at 3:17 AM Amos B. Elberg <am...@gmail.com>
>> > wrote:
>> >
>> > > Moon — I think there is a misunderstanding about the topic of the
>> > > discussion.
>> > >
>> > > The PR has its own userbase. It has been and is being presented at
>> user
>> > > groups. Its been blogged and tweeted about (none of that came from
>> me!)
>> > > The features are the subject of two jiras and on the Zeppelin roadmap.
>> > > So, the discussion isn't about whether the PR is “good."
>> > >
>> > > But no-one responded to the PR until users began to tweet publicly
>> > @nflabs
>> > > asking why the PR had not been adopted, and e-mailing you directly.
>> This
>> > > looks really bad, especially when the project is considering applying
>> to
>> > > leave incubation.
>> > >
>> > >
>> >
>> > Thanks for pinging me. Otherwise i couldn't able to know that you're
>> still
>> > working on it and ready to review. I think it's good practice that
>> pinging
>> > committer for review when there is no sign of response. Except for ping
>> > message has been made on twitter and private email instead of public
>> > mailing list / jira / github issue comment.
>> >
>> >
>> >
>> >
>> > > The question here is what, if anything, prevents us from letting
>> bygones
>> > > be bygones and moving forward with this now?
>> > >
>> > > Claims about CI issues, or licenses, or the PR shouldn’t have been
>> > rebased
>> > > (!?!) — well, they don’t really make sense.
>> > >
>> > >
>> > Although i didn't review your contribution from day 1,
>> > I'm reviewing your contribution, discussing about license, discussing
>> about
>> > improvement of impasse, all they're part of moving forward. I am moving
>> > forward.
>> >
>> > Of course, I can claim CI, license, etc and any other issue. Especially
>> > when CI is not passing, I can not sure about the license. To me, these
>> > claims are sign of 'healthy community' not sign of 'does not make
>> sense'.
>> >
>> > You're not meaning, your contribution always need to be accepted without
>> > any claim. right?
>> >
>> >
>> >
>> > > I keep offering to begin coordinating to integrate the PR with
>> Zeppelin’s
>> > > CI and build system.
>> > >
>> > > But the answer (except from Roman) is still “nah, let us know if you
>> > > figure it out.”
>> > >
>> > >
>> > >
>> > Actually, my answer was, "If you think CI is failing not because of your
>> > change but because of Zeppelin core problem, then file an jira issue
>> about
>> > it. Everyone will look into".
>> >
>> >
>> >
>> > >
>> > >
>> > > Regarding the history:
>> > >
>> > > Konstantin wisely started this thread by saying let’s keep the history
>> > out
>> > > of the discussion. I am respecting that.
>> > >
>> > > If the PR becomes part of Zeppelin, its going to need to be
>> maintained,
>> > > which means that we are going to need to be able to work together.
>> > >
>> > > I have been persuaded to give Moon the benefit of the doubt regarding
>> > > certain issues. He certainly knows what my view of the history is.
>> > >
>> > > If anyone else would like to know, I am happy to share it with them
>> > > off-list.
>> > >
>> >
>> >
>> > I don't know your view of history and what you think the history is. So
>> > please share them in PUBLIC on this thread NOT off-list, if you think
>> > that's reason you think your contribution is in impasse.
>> >
>> > Otherwise I'll never know what you're thinking and I'll not improve.
>> More
>> > importantly, it's easy to make people misunderstand you that you're just
>> > trying to make a negative noises.
>> >
>> > So, do you mind share your view of history in this thread?
>> >
>> > Thanks,
>> > moon
>> >
>> >
>> > >
>> > > From: moon soo Lee <mo...@apache.org>
>> > > Reply: dev@zeppelin.incubator.apache.org <
>> > > dev@zeppelin.incubator.apache.org>
>> > > Date: December 2, 2015 at 7:45:11 AM
>> > > To: dev@zeppelin.incubator.apache.org <
>> dev@zeppelin.incubator.apache.org
>> > >
>> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>> pull
>> > > request: R Interpreter for Zeppelin
>> > >
>> > > Thanks Roman and Eran for the feedback.
>> > >
>> > > *A. About contribution impasse in general*
>> > >
>> > > I think i summarized why it happens and how it can be improved. ie.
>> > >
>> > > 1. Large code base change
>> > > 2. Communication lost
>> > > 3. Opinion diverges
>> > >
>> > > And my solution was
>> > >
>> > > Guide to ping other committer when a committer is not responding,
>> divide
>> > > contribution into small peaces if possible. And committer pay more
>> > > attention to the contribution.
>> > >
>> > > I'd like to hear and learn any more idea to improve.
>> > >
>> > >
>> > > *B. About contribution impasses in R interpreter for Zeppelin*
>> > >
>> > > Although I'was the first one who reviewed and commented this
>> contribution
>> > > among the committer, I feel contributor (Amos) is unhappy about the
>> > review.
>> > >
>> > > I want to analyze the reasons and improve this, too.
>> > >
>> > > Here's reason i guess
>> > >
>> > > 1. Late responding (first review has been made after 3 months)
>> > > 2. Lack of help on CI fail (Amos keep complained about CI fail)
>> > >
>> > > I think both 1 and 2 can be improved by the solution i suggested in
>> > section
>> > > A.
>> > >
>> > > Amos, if you think there're more reasons, please feel free to say and
>> let
>> > > me improve. What is the history you're mentioning?
>> > >
>> > > Best,
>> > > moon
>> > >
>> > >
>> > > On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org>
>> > wrote:
>> > >
>> > > > Just pushing discussion back on the list
>> > > >
>> > > > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com>
>> > wrote:
>> > > >
>> > > > > Alex — if you genuinely do not know the history of this, then I
>> will
>> > > fill
>> > > > > you in.
>> > > > >
>> > > > > lmk…
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Amos Elberg
>> > > > > Sent with Airmail
>> > > > >
>> > > > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
>> > > > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
>> > > > > Date: December 1, 2015 at 6:14:20 AM
>> > > > > To: dev@zeppelin.incubator.apache.org <
>> > > dev@zeppelin.incubator.apache.org
>> > > > >
>> > > > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
>> > > > > <am...@gmail.com> <am...@gmail.com>
>> > > > >
>> > > > > Subject: Re: contributions impasse. Was: [GitHub]
>> incubator-zeppelin
>> > > > > pull request: R Interpreter for Zeppelin
>> > > > >
>> > > > > @Amos, we had plenty of cases of CI failing and always the
>> > > pre-condition
>> > > > > for a merge was a green CI. Sometimes that requires time, polite
>> > > > > collaboration, extra mile in direct asking for help from more
>> > > experienced
>> > > > > members and fixes in different places, which indeed might take
>> time,
>> > as
>> > > > > everyone is busy.
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Following contribution is a recent example, that i think ping committer
after push commit will can help.

https://github.com/apache/incubator-zeppelin/pull/389

In this contribution, last comment has been made 2 weeks ago and then code
change has pushed a week later (a week ago from now), but committer
couldn't be notified about new code change push. and contribution didn't
get any chance to have attention from that time.

I think just pinging committer after push commit will save a lot of similar
cases.

Thanks,
moon

On Thu, Dec 3, 2015 at 3:57 PM moon soo Lee <mo...@apache.org> wrote:

> Apologies about being upset in this thread with non-technical issue.
>
>
> How about we start from adding such things like
>
> - Ping the other committer when no response from committer
> - Keep codebase change small if possible (or break contribution into small
> peaces when possible)
>
> in somewhere
> https://github.com/apache/incubator-zeppelin/blob/master/CONTRIBUTING.md
>
> Does it make sense?
>
> Thanks,
> moon
>
>
>
> On Thu, Dec 3, 2015 at 1:24 PM Corneau Damien <co...@gmail.com>
> wrote:
>
>> As Cos stated before:
>>
>> I am asking _explicitely_ not to re-start the bickering I have already
>>> seen. At this point I am interested in the purely technical side of this.
>>
>>
>> I guess the way to work towards resolving this PR's blocking elements
>> have already been listed before,
>> and there is another discussion to prepare a topic for legal@
>>
>> So there is no reason to continue that thread if it isn't to find a way
>> to resolve it.
>>
>> We already discussed a bit about the PR review process here and in
>> another mailing list thread.
>> We are also having a PR to document our current review process [1]
>>
>> We do have a lot of PR on stand by and I feel sorry for that (currently
>> 85), we usually tend to be more on alert on those that have some activity
>> (notifications).
>> We always try to keep track on old PR that needs review or help, but we
>> also have new PR and issues coming everyday.
>> We try to satisfy everybody but are also keen on maintaining code quality.
>> Reviewing PR is not an easy job, it's time consuming, time committers
>> would probably prefer spending coding.
>> We do it mainly on our spare time, and it's easy to fall into
>> productivity issues when it comes to prioritizing.
>>
>> So I hope you understand better how this works, and that your PR will be
>> good to go soon.
>>
>> [1] https://github.com/apache/incubator-zeppelin/pull/502
>>
>> On Thu, Dec 3, 2015 at 12:32 PM, Amos B. Elberg <am...@gmail.com>
>> wrote:
>>
>>> Is there any reason that you can't tell here?
>>> Please re-read the e-mail from Konstantine that started the thread.
>>>
>>> If you can't tell anything, Is it okay to me assume that you just wanted
>>> make negative noise in the community with unidentified words?
>>> Please just share your thinking and let me hear and improve from it.
>>>
>>> I don’t see how having that discussion in public would serve any
>>> purpose.
>>>
>>> If you genuinely don’t understand — the end of our e-mail exchange was
>>> an invitation for you to call me on the phone and attempt to resolve those
>>> issues without the misunderstandings of tone that can happen in our e-mail
>>> exchange.
>>>
>>> The invitation stands.
>>>
>>> I will continue to look for ways to move forward in the spirit of
>>> cooperation and teamwork.
>>>
>>>
>>> From: moon soo Lee <mo...@apache.org>
>>> Reply: moon soo Lee <mo...@apache.org>
>>> Date: December 2, 2015 at 10:24:52 PM
>>> To: Amos B. Elberg <am...@gmail.com>,
>>> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
>>> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>>> pull request: R Interpreter for Zeppelin
>>>
>>>
>>>
>>> On Thu, Dec 3, 2015 at 12:05 PM Amos B. Elberg <am...@gmail.com>
>>> wrote:
>>> Moon,
>>>
>>> I still suggest, one of the best way you can find that person is, file
>>> an
>>> issue about the CI problem that you found on Zeppelin Core.
>>> All of the relevant people are already informed.
>>>
>>> I continue to be ready to work cooperatively with anyone who would like
>>> to help me resolve the issues.
>>>
>>> I did clearly pointed what rule could be violated and provided a link
>>> and
>>> tried the best at explanation.
>>> Maybe this is a language issue, but I have no idea what your concern
>>> about a license is.
>>>
>>> You did cite to two web pages.  Neither of them said anything that would
>>> explain your position.  In fact, they confirmed what I had been saying all
>>> along.
>>>
>>> So i'm going to continue the discussion in the other thread about the
>>> license and will move the discussion to legal-discuss@.
>>> You’re welcome to do that, of course.  I don’t know that I will
>>> participate.
>>>
>>> What i'm not really fine is, having not enough discussion and concern
>>> about
>>> license. That's sign of unhealthy community.
>>> As Konstantine has pointed out:  The ASF has existed for 16 years.  This
>>> cannot be the first time that this issue arose.  The ASF has a “legal faq”
>>> and other public documents that discuss various licensing issues.
>>>
>>> In addition, a primary responsibility of the PMCC — perhaps the most
>>> important responsibility — is making sure the project conforms to ASF
>>> licensing policy.
>>>
>>> In terms of community health, my concern is that there is a great deal
>>> of confusion about these licensing issues, and people are not able to find
>>> a definitive answer simply by checking the ASF documents (or even
>>> attempting to do so).
>>>
>>> (Hint:  I reviewed the ASF materials in great detail in preparing the
>>> PR.)
>>>
>>> I still don't know which part of the email are you referring. … What are
>>> you referring "the history"?
>>>
>>> You know *exactly* what I’m talking about.
>>>
>>>
>>> Amos,
>>>
>>> This mailing list is subscribed by hundreds of people. Let's not trying
>>> to make meaningless posts.
>>>
>>> I told you i *exactly* don't know what you're talking about "the
>>> history".
>>> So why don't you pin point what *exactly* you talking about, in public?
>>>
>>> Is there any reason that you can't tell here?
>>> If you can't tell anything, Is it okay to me assume that you just wanted
>>> make negative noise in the community with unidentified words?
>>>
>>> Please just share your thinking and let me hear and improve from it.
>>>
>>> Thanks,
>>> moon
>>>
>>>
>>> From: moon soo Lee <mo...@apache.org>
>>> Reply: dev@zeppelin.incubator.apache.org <
>>> dev@zeppelin.incubator.apache.org>
>>> Date: December 2, 2015 at 9:19:50 PM
>>>
>>> To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
>>> >
>>> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>>> pull request: R Interpreter for Zeppelin
>>>
>>> On Thu, Dec 3, 2015 at 7:34 AM Amos B. Elberg <am...@gmail.com>
>>> wrote:
>>>
>>> > Moon, thank you for your reply.
>>> >
>>> > Of course, I can claim CI, license, etc and any other issue. Especially
>>> > when CI is not passing, I can not sure about the license. To me, these
>>> > claims are sign of 'healthy community' not sign of 'does not make
>>> sense'.
>>> >
>>> > You're not meaning, your contribution always need to be accepted
>>> without
>>> > any claim. right?
>>> > I believe the constructive way to move forward, would be for someone
>>> who
>>> > well-understands the CI & build structure, to begin working with me to
>>> > resolve the CI/build integration issues.
>>> >
>>> > I continue to be ready to work with that person.
>>> >
>>>
>>>
>>> I still suggest, one of the best way you can find that person is, file an
>>> issue about the CI problem that you found on Zeppelin Core.
>>> That'll reduce the scope of the problem and that'll help people quickly
>>> getting into without understanding your contribution entirely.
>>>
>>>
>>>
>>> >
>>> > Regarding licensing, I took extensive research steps before submitting
>>> the
>>> > PR to make sure there was not a licensing problem.
>>> >
>>> > In fact, many things in the structure of the code were chosen
>>> specifically
>>> > to avoid any licensing problem. I even negotiated with the authors of
>>> some
>>> > libraries to switch to Apache-compatible licenses, and did some work on
>>> > their projects in exchange.
>>> >
>>> > Considering all of that — If anyone thinks there is a licensing issue,
>>> if
>>> > they want to be constructive, the *least* they can do is say clearly
>>> > exactly what rule is being violated, how it is being violated, and
>>> provide
>>> > a link or citation or *something* that shows the opinion is more than
>>> > hand-waving.
>>> >
>>> > I am ready to engage with anyone who does that.
>>> >
>>>
>>> There is separate thread for the license. So i'll leave minimal comment
>>> here.
>>>
>>> I did clearly pointed what rule could be violated and provided a link and
>>> tried the best at explanation. You don't agree on my concern does not
>>> mean
>>> it's okay to pass. Just like i don't agree on your opinion does not mean
>>> it's confirmation of license problem.
>>>
>>> Of course my concern could be wrong, that's totally fine to me. I don't
>>> have any problem on that. I'm not a legal expert.
>>>
>>> What i'm not really fine is, having not enough discussion and concern
>>> about
>>> license. That's sign of unhealthy community.
>>>
>>> So i'm going to continue the discussion in the other thread about the
>>> license and will move the discussion to legal-discuss@.
>>>
>>>
>>>
>>> >
>>> > I don't know your view of history and what you think the history is.
>>> > Yeah you do.
>>> >
>>> > We had an exchange about it a week before this thread began.
>>> >
>>> > Some of it even spilled-over into the PR comments after I saw the
>>> video.
>>> >
>>> >
>>>
>>> I still don't know which part of the email are you referring. The
>>> suggestion i made about your code? about the review? about the
>>> conference?
>>> about the meetup? What are you referring "the history"?
>>>
>>> Please say in public. What is the history and show how they're related to
>>> this thread topic. Otherwise I'll assume you just want to make a negative
>>> noise in the community.
>>>
>>> Thanks,
>>> moon
>>>
>>>
>>>
>>> > So please share them in PUBLIC on this thread NOT off-list, if you
>>> think
>>> > that's reason you think your contribution is in impasse.
>>> >
>>> > I am going to follow Konstantin’s lead reading this issue, and try to
>>> give
>>> > you the benefit of the doubt.
>>> >
>>> >
>>> > From: moon soo Lee <mo...@apache.org>
>>> > Reply: dev@zeppelin.incubator.apache.org <
>>> > dev@zeppelin.incubator.apache.org>
>>> > Date: December 2, 2015 at 5:06:52 PM
>>> > To: dev@zeppelin.incubator.apache.org <
>>> dev@zeppelin.incubator.apache.org>
>>> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>>> pull
>>> > request: R Interpreter for Zeppelin
>>> >
>>> > Thanks Amos for replying.
>>> >
>>> > On Thu, Dec 3, 2015 at 3:17 AM Amos B. Elberg <am...@gmail.com>
>>> > wrote:
>>> >
>>> > > Moon — I think there is a misunderstanding about the topic of the
>>> > > discussion.
>>> > >
>>> > > The PR has its own userbase. It has been and is being presented at
>>> user
>>> > > groups. Its been blogged and tweeted about (none of that came from
>>> me!)
>>> > > The features are the subject of two jiras and on the Zeppelin
>>> roadmap.
>>> > > So, the discussion isn't about whether the PR is “good."
>>> > >
>>> > > But no-one responded to the PR until users began to tweet publicly
>>> > @nflabs
>>> > > asking why the PR had not been adopted, and e-mailing you directly.
>>> This
>>> > > looks really bad, especially when the project is considering
>>> applying to
>>> > > leave incubation.
>>> > >
>>> > >
>>> >
>>> > Thanks for pinging me. Otherwise i couldn't able to know that you're
>>> still
>>> > working on it and ready to review. I think it's good practice that
>>> pinging
>>> > committer for review when there is no sign of response. Except for ping
>>> > message has been made on twitter and private email instead of public
>>> > mailing list / jira / github issue comment.
>>> >
>>> >
>>> >
>>> >
>>> > > The question here is what, if anything, prevents us from letting
>>> bygones
>>> > > be bygones and moving forward with this now?
>>> > >
>>> > > Claims about CI issues, or licenses, or the PR shouldn’t have been
>>> > rebased
>>> > > (!?!) — well, they don’t really make sense.
>>> > >
>>> > >
>>> > Although i didn't review your contribution from day 1,
>>> > I'm reviewing your contribution, discussing about license, discussing
>>> about
>>> > improvement of impasse, all they're part of moving forward. I am moving
>>> > forward.
>>> >
>>> > Of course, I can claim CI, license, etc and any other issue. Especially
>>> > when CI is not passing, I can not sure about the license. To me, these
>>> > claims are sign of 'healthy community' not sign of 'does not make
>>> sense'.
>>> >
>>> > You're not meaning, your contribution always need to be accepted
>>> without
>>> > any claim. right?
>>> >
>>> >
>>> >
>>> > > I keep offering to begin coordinating to integrate the PR with
>>> Zeppelin’s
>>> > > CI and build system.
>>> > >
>>> > > But the answer (except from Roman) is still “nah, let us know if you
>>> > > figure it out.”
>>> > >
>>> > >
>>> > >
>>> > Actually, my answer was, "If you think CI is failing not because of
>>> your
>>> > change but because of Zeppelin core problem, then file an jira issue
>>> about
>>> > it. Everyone will look into".
>>> >
>>> >
>>> >
>>> > >
>>> > >
>>> > > Regarding the history:
>>> > >
>>> > > Konstantin wisely started this thread by saying let’s keep the
>>> history
>>> > out
>>> > > of the discussion. I am respecting that.
>>> > >
>>> > > If the PR becomes part of Zeppelin, its going to need to be
>>> maintained,
>>> > > which means that we are going to need to be able to work together.
>>> > >
>>> > > I have been persuaded to give Moon the benefit of the doubt regarding
>>> > > certain issues. He certainly knows what my view of the history is.
>>> > >
>>> > > If anyone else would like to know, I am happy to share it with them
>>> > > off-list.
>>> > >
>>> >
>>> >
>>> > I don't know your view of history and what you think the history is. So
>>> > please share them in PUBLIC on this thread NOT off-list, if you think
>>> > that's reason you think your contribution is in impasse.
>>> >
>>> > Otherwise I'll never know what you're thinking and I'll not improve.
>>> More
>>> > importantly, it's easy to make people misunderstand you that you're
>>> just
>>> > trying to make a negative noises.
>>> >
>>> > So, do you mind share your view of history in this thread?
>>> >
>>> > Thanks,
>>> > moon
>>> >
>>> >
>>> > >
>>> > > From: moon soo Lee <mo...@apache.org>
>>> > > Reply: dev@zeppelin.incubator.apache.org <
>>> > > dev@zeppelin.incubator.apache.org>
>>> > > Date: December 2, 2015 at 7:45:11 AM
>>> > > To: dev@zeppelin.incubator.apache.org <
>>> dev@zeppelin.incubator.apache.org
>>> > >
>>> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>>> pull
>>> > > request: R Interpreter for Zeppelin
>>> > >
>>> > > Thanks Roman and Eran for the feedback.
>>> > >
>>> > > *A. About contribution impasse in general*
>>> > >
>>> > > I think i summarized why it happens and how it can be improved. ie.
>>> > >
>>> > > 1. Large code base change
>>> > > 2. Communication lost
>>> > > 3. Opinion diverges
>>> > >
>>> > > And my solution was
>>> > >
>>> > > Guide to ping other committer when a committer is not responding,
>>> divide
>>> > > contribution into small peaces if possible. And committer pay more
>>> > > attention to the contribution.
>>> > >
>>> > > I'd like to hear and learn any more idea to improve.
>>> > >
>>> > >
>>> > > *B. About contribution impasses in R interpreter for Zeppelin*
>>> > >
>>> > > Although I'was the first one who reviewed and commented this
>>> contribution
>>> > > among the committer, I feel contributor (Amos) is unhappy about the
>>> > review.
>>> > >
>>> > > I want to analyze the reasons and improve this, too.
>>> > >
>>> > > Here's reason i guess
>>> > >
>>> > > 1. Late responding (first review has been made after 3 months)
>>> > > 2. Lack of help on CI fail (Amos keep complained about CI fail)
>>> > >
>>> > > I think both 1 and 2 can be improved by the solution i suggested in
>>> > section
>>> > > A.
>>> > >
>>> > > Amos, if you think there're more reasons, please feel free to say
>>> and let
>>> > > me improve. What is the history you're mentioning?
>>> > >
>>> > > Best,
>>> > > moon
>>> > >
>>> > >
>>> > > On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org>
>>> > wrote:
>>> > >
>>> > > > Just pushing discussion back on the list
>>> > > >
>>> > > > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com>
>>> > wrote:
>>> > > >
>>> > > > > Alex — if you genuinely do not know the history of this, then I
>>> will
>>> > > fill
>>> > > > > you in.
>>> > > > >
>>> > > > > lmk…
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > > Amos Elberg
>>> > > > > Sent with Airmail
>>> > > > >
>>> > > > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
>>> > > > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
>>> > > > > Date: December 1, 2015 at 6:14:20 AM
>>> > > > > To: dev@zeppelin.incubator.apache.org <
>>> > > dev@zeppelin.incubator.apache.org
>>> > > > >
>>> > > > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
>>> > > > > <am...@gmail.com> <am...@gmail.com>
>>> > > > >
>>> > > > > Subject: Re: contributions impasse. Was: [GitHub]
>>> incubator-zeppelin
>>> > > > > pull request: R Interpreter for Zeppelin
>>> > > > >
>>> > > > > @Amos, we had plenty of cases of CI failing and always the
>>> > > pre-condition
>>> > > > > for a merge was a green CI. Sometimes that requires time, polite
>>> > > > > collaboration, extra mile in direct asking for help from more
>>> > > experienced
>>> > > > > members and fixes in different places, which indeed might take
>>> time,
>>> > as
>>> > > > > everyone is busy.
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Sure, that's perfect place we can test drive.

Thanks,
moon

On Sun, Dec 6, 2015 at 12:46 PM Roman Shaposhnik <ro...@shaposhnik.org>
wrote:

> On Wed, Dec 2, 2015 at 10:56 PM, moon soo Lee <mo...@apache.org> wrote:
> > Apologies about being upset in this thread with non-technical issue.
> >
> >
> > How about we start from adding such things like
> >
> > - Ping the other committer when no response from committer
> > - Keep codebase change small if possible (or break contribution into
> small
> > peaces when possible)
> >
> > in somewhere
> > https://github.com/apache/incubator-zeppelin/blob/master/CONTRIBUTING.md
> >
> > Does it make sense?
>
> Yes it does, but I'd like you guys to test drive this process with Amos
>
> Thanks,
> Roman.
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Roman Shaposhnik <ro...@shaposhnik.org>]

On Wed, Dec 2, 2015 at 10:56 PM, moon soo Lee <mo...@apache.org> wrote:
> Apologies about being upset in this thread with non-technical issue.
>
>
> How about we start from adding such things like
>
> - Ping the other committer when no response from committer
> - Keep codebase change small if possible (or break contribution into small
> peaces when possible)
>
> in somewhere
> https://github.com/apache/incubator-zeppelin/blob/master/CONTRIBUTING.md
>
> Does it make sense?

Yes it does, but I'd like you guys to test drive this process with Amos

Thanks,
Roman.

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Apologies about being upset in this thread with non-technical issue.


How about we start from adding such things like

- Ping the other committer when no response from committer
- Keep codebase change small if possible (or break contribution into small
peaces when possible)

in somewhere
https://github.com/apache/incubator-zeppelin/blob/master/CONTRIBUTING.md

Does it make sense?

Thanks,
moon



On Thu, Dec 3, 2015 at 1:24 PM Corneau Damien <co...@gmail.com> wrote:

> As Cos stated before:
>
> I am asking _explicitely_ not to re-start the bickering I have already
>> seen. At this point I am interested in the purely technical side of this.
>
>
> I guess the way to work towards resolving this PR's blocking elements have
> already been listed before,
> and there is another discussion to prepare a topic for legal@
>
> So there is no reason to continue that thread if it isn't to find a way to
> resolve it.
>
> We already discussed a bit about the PR review process here and in another
> mailing list thread.
> We are also having a PR to document our current review process [1]
>
> We do have a lot of PR on stand by and I feel sorry for that (currently
> 85), we usually tend to be more on alert on those that have some activity
> (notifications).
> We always try to keep track on old PR that needs review or help, but we
> also have new PR and issues coming everyday.
> We try to satisfy everybody but are also keen on maintaining code quality.
> Reviewing PR is not an easy job, it's time consuming, time committers
> would probably prefer spending coding.
> We do it mainly on our spare time, and it's easy to fall into productivity
> issues when it comes to prioritizing.
>
> So I hope you understand better how this works, and that your PR will be
> good to go soon.
>
> [1] https://github.com/apache/incubator-zeppelin/pull/502
>
> On Thu, Dec 3, 2015 at 12:32 PM, Amos B. Elberg <am...@gmail.com>
> wrote:
>
>> Is there any reason that you can't tell here?
>> Please re-read the e-mail from Konstantine that started the thread.
>>
>> If you can't tell anything, Is it okay to me assume that you just wanted
>> make negative noise in the community with unidentified words?
>> Please just share your thinking and let me hear and improve from it.
>>
>> I don’t see how having that discussion in public would serve any purpose.
>>
>> If you genuinely don’t understand — the end of our e-mail exchange was an
>> invitation for you to call me on the phone and attempt to resolve those
>> issues without the misunderstandings of tone that can happen in our e-mail
>> exchange.
>>
>> The invitation stands.
>>
>> I will continue to look for ways to move forward in the spirit of
>> cooperation and teamwork.
>>
>>
>> From: moon soo Lee <mo...@apache.org>
>> Reply: moon soo Lee <mo...@apache.org>
>> Date: December 2, 2015 at 10:24:52 PM
>> To: Amos B. Elberg <am...@gmail.com>,
>> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
>> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>> pull request: R Interpreter for Zeppelin
>>
>>
>>
>> On Thu, Dec 3, 2015 at 12:05 PM Amos B. Elberg <am...@gmail.com>
>> wrote:
>> Moon,
>>
>> I still suggest, one of the best way you can find that person is, file an
>> issue about the CI problem that you found on Zeppelin Core.
>> All of the relevant people are already informed.
>>
>> I continue to be ready to work cooperatively with anyone who would like
>> to help me resolve the issues.
>>
>> I did clearly pointed what rule could be violated and provided a link and
>> tried the best at explanation.
>> Maybe this is a language issue, but I have no idea what your concern
>> about a license is.
>>
>> You did cite to two web pages.  Neither of them said anything that would
>> explain your position.  In fact, they confirmed what I had been saying all
>> along.
>>
>> So i'm going to continue the discussion in the other thread about the
>> license and will move the discussion to legal-discuss@.
>> You’re welcome to do that, of course.  I don’t know that I will
>> participate.
>>
>> What i'm not really fine is, having not enough discussion and concern
>> about
>> license. That's sign of unhealthy community.
>> As Konstantine has pointed out:  The ASF has existed for 16 years.  This
>> cannot be the first time that this issue arose.  The ASF has a “legal faq”
>> and other public documents that discuss various licensing issues.
>>
>> In addition, a primary responsibility of the PMCC — perhaps the most
>> important responsibility — is making sure the project conforms to ASF
>> licensing policy.
>>
>> In terms of community health, my concern is that there is a great deal of
>> confusion about these licensing issues, and people are not able to find a
>> definitive answer simply by checking the ASF documents (or even attempting
>> to do so).
>>
>> (Hint:  I reviewed the ASF materials in great detail in preparing the
>> PR.)
>>
>> I still don't know which part of the email are you referring. … What are
>> you referring "the history"?
>>
>> You know *exactly* what I’m talking about.
>>
>>
>> Amos,
>>
>> This mailing list is subscribed by hundreds of people. Let's not trying
>> to make meaningless posts.
>>
>> I told you i *exactly* don't know what you're talking about "the history".
>> So why don't you pin point what *exactly* you talking about, in public?
>>
>> Is there any reason that you can't tell here?
>> If you can't tell anything, Is it okay to me assume that you just wanted
>> make negative noise in the community with unidentified words?
>>
>> Please just share your thinking and let me hear and improve from it.
>>
>> Thanks,
>> moon
>>
>>
>> From: moon soo Lee <mo...@apache.org>
>> Reply: dev@zeppelin.incubator.apache.org <
>> dev@zeppelin.incubator.apache.org>
>> Date: December 2, 2015 at 9:19:50 PM
>>
>> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
>> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>> pull request: R Interpreter for Zeppelin
>>
>> On Thu, Dec 3, 2015 at 7:34 AM Amos B. Elberg <am...@gmail.com>
>> wrote:
>>
>> > Moon, thank you for your reply.
>> >
>> > Of course, I can claim CI, license, etc and any other issue. Especially
>> > when CI is not passing, I can not sure about the license. To me, these
>> > claims are sign of 'healthy community' not sign of 'does not make
>> sense'.
>> >
>> > You're not meaning, your contribution always need to be accepted without
>> > any claim. right?
>> > I believe the constructive way to move forward, would be for someone who
>> > well-understands the CI & build structure, to begin working with me to
>> > resolve the CI/build integration issues.
>> >
>> > I continue to be ready to work with that person.
>> >
>>
>>
>> I still suggest, one of the best way you can find that person is, file an
>> issue about the CI problem that you found on Zeppelin Core.
>> That'll reduce the scope of the problem and that'll help people quickly
>> getting into without understanding your contribution entirely.
>>
>>
>>
>> >
>> > Regarding licensing, I took extensive research steps before submitting
>> the
>> > PR to make sure there was not a licensing problem.
>> >
>> > In fact, many things in the structure of the code were chosen
>> specifically
>> > to avoid any licensing problem. I even negotiated with the authors of
>> some
>> > libraries to switch to Apache-compatible licenses, and did some work on
>> > their projects in exchange.
>> >
>> > Considering all of that — If anyone thinks there is a licensing issue,
>> if
>> > they want to be constructive, the *least* they can do is say clearly
>> > exactly what rule is being violated, how it is being violated, and
>> provide
>> > a link or citation or *something* that shows the opinion is more than
>> > hand-waving.
>> >
>> > I am ready to engage with anyone who does that.
>> >
>>
>> There is separate thread for the license. So i'll leave minimal comment
>> here.
>>
>> I did clearly pointed what rule could be violated and provided a link and
>> tried the best at explanation. You don't agree on my concern does not mean
>> it's okay to pass. Just like i don't agree on your opinion does not mean
>> it's confirmation of license problem.
>>
>> Of course my concern could be wrong, that's totally fine to me. I don't
>> have any problem on that. I'm not a legal expert.
>>
>> What i'm not really fine is, having not enough discussion and concern
>> about
>> license. That's sign of unhealthy community.
>>
>> So i'm going to continue the discussion in the other thread about the
>> license and will move the discussion to legal-discuss@.
>>
>>
>>
>> >
>> > I don't know your view of history and what you think the history is.
>> > Yeah you do.
>> >
>> > We had an exchange about it a week before this thread began.
>> >
>> > Some of it even spilled-over into the PR comments after I saw the video.
>> >
>> >
>>
>> I still don't know which part of the email are you referring. The
>> suggestion i made about your code? about the review? about the conference?
>> about the meetup? What are you referring "the history"?
>>
>> Please say in public. What is the history and show how they're related to
>> this thread topic. Otherwise I'll assume you just want to make a negative
>> noise in the community.
>>
>> Thanks,
>> moon
>>
>>
>>
>> > So please share them in PUBLIC on this thread NOT off-list, if you think
>> > that's reason you think your contribution is in impasse.
>> >
>> > I am going to follow Konstantin’s lead reading this issue, and try to
>> give
>> > you the benefit of the doubt.
>> >
>> >
>> > From: moon soo Lee <mo...@apache.org>
>> > Reply: dev@zeppelin.incubator.apache.org <
>> > dev@zeppelin.incubator.apache.org>
>> > Date: December 2, 2015 at 5:06:52 PM
>> > To: dev@zeppelin.incubator.apache.org <
>> dev@zeppelin.incubator.apache.org>
>> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>> pull
>> > request: R Interpreter for Zeppelin
>> >
>> > Thanks Amos for replying.
>> >
>> > On Thu, Dec 3, 2015 at 3:17 AM Amos B. Elberg <am...@gmail.com>
>> > wrote:
>> >
>> > > Moon — I think there is a misunderstanding about the topic of the
>> > > discussion.
>> > >
>> > > The PR has its own userbase. It has been and is being presented at
>> user
>> > > groups. Its been blogged and tweeted about (none of that came from
>> me!)
>> > > The features are the subject of two jiras and on the Zeppelin roadmap.
>> > > So, the discussion isn't about whether the PR is “good."
>> > >
>> > > But no-one responded to the PR until users began to tweet publicly
>> > @nflabs
>> > > asking why the PR had not been adopted, and e-mailing you directly.
>> This
>> > > looks really bad, especially when the project is considering applying
>> to
>> > > leave incubation.
>> > >
>> > >
>> >
>> > Thanks for pinging me. Otherwise i couldn't able to know that you're
>> still
>> > working on it and ready to review. I think it's good practice that
>> pinging
>> > committer for review when there is no sign of response. Except for ping
>> > message has been made on twitter and private email instead of public
>> > mailing list / jira / github issue comment.
>> >
>> >
>> >
>> >
>> > > The question here is what, if anything, prevents us from letting
>> bygones
>> > > be bygones and moving forward with this now?
>> > >
>> > > Claims about CI issues, or licenses, or the PR shouldn’t have been
>> > rebased
>> > > (!?!) — well, they don’t really make sense.
>> > >
>> > >
>> > Although i didn't review your contribution from day 1,
>> > I'm reviewing your contribution, discussing about license, discussing
>> about
>> > improvement of impasse, all they're part of moving forward. I am moving
>> > forward.
>> >
>> > Of course, I can claim CI, license, etc and any other issue. Especially
>> > when CI is not passing, I can not sure about the license. To me, these
>> > claims are sign of 'healthy community' not sign of 'does not make
>> sense'.
>> >
>> > You're not meaning, your contribution always need to be accepted without
>> > any claim. right?
>> >
>> >
>> >
>> > > I keep offering to begin coordinating to integrate the PR with
>> Zeppelin’s
>> > > CI and build system.
>> > >
>> > > But the answer (except from Roman) is still “nah, let us know if you
>> > > figure it out.”
>> > >
>> > >
>> > >
>> > Actually, my answer was, "If you think CI is failing not because of your
>> > change but because of Zeppelin core problem, then file an jira issue
>> about
>> > it. Everyone will look into".
>> >
>> >
>> >
>> > >
>> > >
>> > > Regarding the history:
>> > >
>> > > Konstantin wisely started this thread by saying let’s keep the history
>> > out
>> > > of the discussion. I am respecting that.
>> > >
>> > > If the PR becomes part of Zeppelin, its going to need to be
>> maintained,
>> > > which means that we are going to need to be able to work together.
>> > >
>> > > I have been persuaded to give Moon the benefit of the doubt regarding
>> > > certain issues. He certainly knows what my view of the history is.
>> > >
>> > > If anyone else would like to know, I am happy to share it with them
>> > > off-list.
>> > >
>> >
>> >
>> > I don't know your view of history and what you think the history is. So
>> > please share them in PUBLIC on this thread NOT off-list, if you think
>> > that's reason you think your contribution is in impasse.
>> >
>> > Otherwise I'll never know what you're thinking and I'll not improve.
>> More
>> > importantly, it's easy to make people misunderstand you that you're just
>> > trying to make a negative noises.
>> >
>> > So, do you mind share your view of history in this thread?
>> >
>> > Thanks,
>> > moon
>> >
>> >
>> > >
>> > > From: moon soo Lee <mo...@apache.org>
>> > > Reply: dev@zeppelin.incubator.apache.org <
>> > > dev@zeppelin.incubator.apache.org>
>> > > Date: December 2, 2015 at 7:45:11 AM
>> > > To: dev@zeppelin.incubator.apache.org <
>> dev@zeppelin.incubator.apache.org
>> > >
>> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>> pull
>> > > request: R Interpreter for Zeppelin
>> > >
>> > > Thanks Roman and Eran for the feedback.
>> > >
>> > > *A. About contribution impasse in general*
>> > >
>> > > I think i summarized why it happens and how it can be improved. ie.
>> > >
>> > > 1. Large code base change
>> > > 2. Communication lost
>> > > 3. Opinion diverges
>> > >
>> > > And my solution was
>> > >
>> > > Guide to ping other committer when a committer is not responding,
>> divide
>> > > contribution into small peaces if possible. And committer pay more
>> > > attention to the contribution.
>> > >
>> > > I'd like to hear and learn any more idea to improve.
>> > >
>> > >
>> > > *B. About contribution impasses in R interpreter for Zeppelin*
>> > >
>> > > Although I'was the first one who reviewed and commented this
>> contribution
>> > > among the committer, I feel contributor (Amos) is unhappy about the
>> > review.
>> > >
>> > > I want to analyze the reasons and improve this, too.
>> > >
>> > > Here's reason i guess
>> > >
>> > > 1. Late responding (first review has been made after 3 months)
>> > > 2. Lack of help on CI fail (Amos keep complained about CI fail)
>> > >
>> > > I think both 1 and 2 can be improved by the solution i suggested in
>> > section
>> > > A.
>> > >
>> > > Amos, if you think there're more reasons, please feel free to say and
>> let
>> > > me improve. What is the history you're mentioning?
>> > >
>> > > Best,
>> > > moon
>> > >
>> > >
>> > > On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org>
>> > wrote:
>> > >
>> > > > Just pushing discussion back on the list
>> > > >
>> > > > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com>
>> > wrote:
>> > > >
>> > > > > Alex — if you genuinely do not know the history of this, then I
>> will
>> > > fill
>> > > > > you in.
>> > > > >
>> > > > > lmk…
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Amos Elberg
>> > > > > Sent with Airmail
>> > > > >
>> > > > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
>> > > > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
>> > > > > Date: December 1, 2015 at 6:14:20 AM
>> > > > > To: dev@zeppelin.incubator.apache.org <
>> > > dev@zeppelin.incubator.apache.org
>> > > > >
>> > > > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
>> > > > > <am...@gmail.com> <am...@gmail.com>
>> > > > >
>> > > > > Subject: Re: contributions impasse. Was: [GitHub]
>> incubator-zeppelin
>> > > > > pull request: R Interpreter for Zeppelin
>> > > > >
>> > > > > @Amos, we had plenty of cases of CI failing and always the
>> > > pre-condition
>> > > > > for a merge was a green CI. Sometimes that requires time, polite
>> > > > > collaboration, extra mile in direct asking for help from more
>> > > experienced
>> > > > > members and fixes in different places, which indeed might take
>> time,
>> > as
>> > > > > everyone is busy.
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Corneau Damien <co...@gmail.com>]

As Cos stated before:

I am asking _explicitely_ not to re-start the bickering I have already
> seen. At this point I am interested in the purely technical side of this.


I guess the way to work towards resolving this PR's blocking elements have
already been listed before,
and there is another discussion to prepare a topic for legal@

So there is no reason to continue that thread if it isn't to find a way to
resolve it.

We already discussed a bit about the PR review process here and in another
mailing list thread.
We are also having a PR to document our current review process [1]

We do have a lot of PR on stand by and I feel sorry for that (currently
85), we usually tend to be more on alert on those that have some activity
(notifications).
We always try to keep track on old PR that needs review or help, but we
also have new PR and issues coming everyday.
We try to satisfy everybody but are also keen on maintaining code quality.
Reviewing PR is not an easy job, it's time consuming, time committers would
probably prefer spending coding.
We do it mainly on our spare time, and it's easy to fall into productivity
issues when it comes to prioritizing.

So I hope you understand better how this works, and that your PR will be
good to go soon.

[1] https://github.com/apache/incubator-zeppelin/pull/502

On Thu, Dec 3, 2015 at 12:32 PM, Amos B. Elberg <am...@gmail.com>
wrote:

> Is there any reason that you can't tell here?
> Please re-read the e-mail from Konstantine that started the thread.
>
> If you can't tell anything, Is it okay to me assume that you just wanted
> make negative noise in the community with unidentified words?
> Please just share your thinking and let me hear and improve from it.
>
> I don’t see how having that discussion in public would serve any purpose.
>
> If you genuinely don’t understand — the end of our e-mail exchange was an
> invitation for you to call me on the phone and attempt to resolve those
> issues without the misunderstandings of tone that can happen in our e-mail
> exchange.
>
> The invitation stands.
>
> I will continue to look for ways to move forward in the spirit of
> cooperation and teamwork.
>
>
> From: moon soo Lee <mo...@apache.org>
> Reply: moon soo Lee <mo...@apache.org>
> Date: December 2, 2015 at 10:24:52 PM
> To: Amos B. Elberg <am...@gmail.com>,
> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
>
>
> On Thu, Dec 3, 2015 at 12:05 PM Amos B. Elberg <am...@gmail.com>
> wrote:
> Moon,
>
> I still suggest, one of the best way you can find that person is, file an
> issue about the CI problem that you found on Zeppelin Core.
> All of the relevant people are already informed.
>
> I continue to be ready to work cooperatively with anyone who would like to
> help me resolve the issues.
>
> I did clearly pointed what rule could be violated and provided a link and
> tried the best at explanation.
> Maybe this is a language issue, but I have no idea what your concern about
> a license is.
>
> You did cite to two web pages.  Neither of them said anything that would
> explain your position.  In fact, they confirmed what I had been saying all
> along.
>
> So i'm going to continue the discussion in the other thread about the
> license and will move the discussion to legal-discuss@.
> You’re welcome to do that, of course.  I don’t know that I will
> participate.
>
> What i'm not really fine is, having not enough discussion and concern
> about
> license. That's sign of unhealthy community.
> As Konstantine has pointed out:  The ASF has existed for 16 years.  This
> cannot be the first time that this issue arose.  The ASF has a “legal faq”
> and other public documents that discuss various licensing issues.
>
> In addition, a primary responsibility of the PMCC — perhaps the most
> important responsibility — is making sure the project conforms to ASF
> licensing policy.
>
> In terms of community health, my concern is that there is a great deal of
> confusion about these licensing issues, and people are not able to find a
> definitive answer simply by checking the ASF documents (or even attempting
> to do so).
>
> (Hint:  I reviewed the ASF materials in great detail in preparing the PR.)
>
> I still don't know which part of the email are you referring. … What are
> you referring "the history"?
>
> You know *exactly* what I’m talking about.
>
>
> Amos,
>
> This mailing list is subscribed by hundreds of people. Let's not trying to
> make meaningless posts.
>
> I told you i *exactly* don't know what you're talking about "the history".
> So why don't you pin point what *exactly* you talking about, in public?
>
> Is there any reason that you can't tell here?
> If you can't tell anything, Is it okay to me assume that you just wanted
> make negative noise in the community with unidentified words?
>
> Please just share your thinking and let me hear and improve from it.
>
> Thanks,
> moon
>
>
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> Date: December 2, 2015 at 9:19:50 PM
>
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> On Thu, Dec 3, 2015 at 7:34 AM Amos B. Elberg <am...@gmail.com>
> wrote:
>
> > Moon, thank you for your reply.
> >
> > Of course, I can claim CI, license, etc and any other issue. Especially
> > when CI is not passing, I can not sure about the license. To me, these
> > claims are sign of 'healthy community' not sign of 'does not make sense'.
> >
> > You're not meaning, your contribution always need to be accepted without
> > any claim. right?
> > I believe the constructive way to move forward, would be for someone who
> > well-understands the CI & build structure, to begin working with me to
> > resolve the CI/build integration issues.
> >
> > I continue to be ready to work with that person.
> >
>
>
> I still suggest, one of the best way you can find that person is, file an
> issue about the CI problem that you found on Zeppelin Core.
> That'll reduce the scope of the problem and that'll help people quickly
> getting into without understanding your contribution entirely.
>
>
>
> >
> > Regarding licensing, I took extensive research steps before submitting
> the
> > PR to make sure there was not a licensing problem.
> >
> > In fact, many things in the structure of the code were chosen
> specifically
> > to avoid any licensing problem. I even negotiated with the authors of
> some
> > libraries to switch to Apache-compatible licenses, and did some work on
> > their projects in exchange.
> >
> > Considering all of that — If anyone thinks there is a licensing issue, if
> > they want to be constructive, the *least* they can do is say clearly
> > exactly what rule is being violated, how it is being violated, and
> provide
> > a link or citation or *something* that shows the opinion is more than
> > hand-waving.
> >
> > I am ready to engage with anyone who does that.
> >
>
> There is separate thread for the license. So i'll leave minimal comment
> here.
>
> I did clearly pointed what rule could be violated and provided a link and
> tried the best at explanation. You don't agree on my concern does not mean
> it's okay to pass. Just like i don't agree on your opinion does not mean
> it's confirmation of license problem.
>
> Of course my concern could be wrong, that's totally fine to me. I don't
> have any problem on that. I'm not a legal expert.
>
> What i'm not really fine is, having not enough discussion and concern about
> license. That's sign of unhealthy community.
>
> So i'm going to continue the discussion in the other thread about the
> license and will move the discussion to legal-discuss@.
>
>
>
> >
> > I don't know your view of history and what you think the history is.
> > Yeah you do.
> >
> > We had an exchange about it a week before this thread began.
> >
> > Some of it even spilled-over into the PR comments after I saw the video.
> >
> >
>
> I still don't know which part of the email are you referring. The
> suggestion i made about your code? about the review? about the conference?
> about the meetup? What are you referring "the history"?
>
> Please say in public. What is the history and show how they're related to
> this thread topic. Otherwise I'll assume you just want to make a negative
> noise in the community.
>
> Thanks,
> moon
>
>
>
> > So please share them in PUBLIC on this thread NOT off-list, if you think
> > that's reason you think your contribution is in impasse.
> >
> > I am going to follow Konstantin’s lead reading this issue, and try to
> give
> > you the benefit of the doubt.
> >
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > Date: December 2, 2015 at 5:06:52 PM
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> > request: R Interpreter for Zeppelin
> >
> > Thanks Amos for replying.
> >
> > On Thu, Dec 3, 2015 at 3:17 AM Amos B. Elberg <am...@gmail.com>
> > wrote:
> >
> > > Moon — I think there is a misunderstanding about the topic of the
> > > discussion.
> > >
> > > The PR has its own userbase. It has been and is being presented at user
> > > groups. Its been blogged and tweeted about (none of that came from me!)
> > > The features are the subject of two jiras and on the Zeppelin roadmap.
> > > So, the discussion isn't about whether the PR is “good."
> > >
> > > But no-one responded to the PR until users began to tweet publicly
> > @nflabs
> > > asking why the PR had not been adopted, and e-mailing you directly.
> This
> > > looks really bad, especially when the project is considering applying
> to
> > > leave incubation.
> > >
> > >
> >
> > Thanks for pinging me. Otherwise i couldn't able to know that you're
> still
> > working on it and ready to review. I think it's good practice that
> pinging
> > committer for review when there is no sign of response. Except for ping
> > message has been made on twitter and private email instead of public
> > mailing list / jira / github issue comment.
> >
> >
> >
> >
> > > The question here is what, if anything, prevents us from letting
> bygones
> > > be bygones and moving forward with this now?
> > >
> > > Claims about CI issues, or licenses, or the PR shouldn’t have been
> > rebased
> > > (!?!) — well, they don’t really make sense.
> > >
> > >
> > Although i didn't review your contribution from day 1,
> > I'm reviewing your contribution, discussing about license, discussing
> about
> > improvement of impasse, all they're part of moving forward. I am moving
> > forward.
> >
> > Of course, I can claim CI, license, etc and any other issue. Especially
> > when CI is not passing, I can not sure about the license. To me, these
> > claims are sign of 'healthy community' not sign of 'does not make sense'.
> >
> > You're not meaning, your contribution always need to be accepted without
> > any claim. right?
> >
> >
> >
> > > I keep offering to begin coordinating to integrate the PR with
> Zeppelin’s
> > > CI and build system.
> > >
> > > But the answer (except from Roman) is still “nah, let us know if you
> > > figure it out.”
> > >
> > >
> > >
> > Actually, my answer was, "If you think CI is failing not because of your
> > change but because of Zeppelin core problem, then file an jira issue
> about
> > it. Everyone will look into".
> >
> >
> >
> > >
> > >
> > > Regarding the history:
> > >
> > > Konstantin wisely started this thread by saying let’s keep the history
> > out
> > > of the discussion. I am respecting that.
> > >
> > > If the PR becomes part of Zeppelin, its going to need to be maintained,
> > > which means that we are going to need to be able to work together.
> > >
> > > I have been persuaded to give Moon the benefit of the doubt regarding
> > > certain issues. He certainly knows what my view of the history is.
> > >
> > > If anyone else would like to know, I am happy to share it with them
> > > off-list.
> > >
> >
> >
> > I don't know your view of history and what you think the history is. So
> > please share them in PUBLIC on this thread NOT off-list, if you think
> > that's reason you think your contribution is in impasse.
> >
> > Otherwise I'll never know what you're thinking and I'll not improve. More
> > importantly, it's easy to make people misunderstand you that you're just
> > trying to make a negative noises.
> >
> > So, do you mind share your view of history in this thread?
> >
> > Thanks,
> > moon
> >
> >
> > >
> > > From: moon soo Lee <mo...@apache.org>
> > > Reply: dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org>
> > > Date: December 2, 2015 at 7:45:11 AM
> > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org
> > >
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull
> > > request: R Interpreter for Zeppelin
> > >
> > > Thanks Roman and Eran for the feedback.
> > >
> > > *A. About contribution impasse in general*
> > >
> > > I think i summarized why it happens and how it can be improved. ie.
> > >
> > > 1. Large code base change
> > > 2. Communication lost
> > > 3. Opinion diverges
> > >
> > > And my solution was
> > >
> > > Guide to ping other committer when a committer is not responding,
> divide
> > > contribution into small peaces if possible. And committer pay more
> > > attention to the contribution.
> > >
> > > I'd like to hear and learn any more idea to improve.
> > >
> > >
> > > *B. About contribution impasses in R interpreter for Zeppelin*
> > >
> > > Although I'was the first one who reviewed and commented this
> contribution
> > > among the committer, I feel contributor (Amos) is unhappy about the
> > review.
> > >
> > > I want to analyze the reasons and improve this, too.
> > >
> > > Here's reason i guess
> > >
> > > 1. Late responding (first review has been made after 3 months)
> > > 2. Lack of help on CI fail (Amos keep complained about CI fail)
> > >
> > > I think both 1 and 2 can be improved by the solution i suggested in
> > section
> > > A.
> > >
> > > Amos, if you think there're more reasons, please feel free to say and
> let
> > > me improve. What is the history you're mentioning?
> > >
> > > Best,
> > > moon
> > >
> > >
> > > On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org>
> > wrote:
> > >
> > > > Just pushing discussion back on the list
> > > >
> > > > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com>
> > wrote:
> > > >
> > > > > Alex — if you genuinely do not know the history of this, then I
> will
> > > fill
> > > > > you in.
> > > > >
> > > > > lmk…
> > > > >
> > > > >
> > > > > --
> > > > > Amos Elberg
> > > > > Sent with Airmail
> > > > >
> > > > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > > Date: December 1, 2015 at 6:14:20 AM
> > > > > To: dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org
> > > > >
> > > > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
> > > > > <am...@gmail.com> <am...@gmail.com>
> > > > >
> > > > > Subject: Re: contributions impasse. Was: [GitHub]
> incubator-zeppelin
> > > > > pull request: R Interpreter for Zeppelin
> > > > >
> > > > > @Amos, we had plenty of cases of CI failing and always the
> > > pre-condition
> > > > > for a merge was a green CI. Sometimes that requires time, polite
> > > > > collaboration, extra mile in direct asking for help from more
> > > experienced
> > > > > members and fixes in different places, which indeed might take
> time,
> > as
> > > > > everyone is busy.
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Is there any reason that you can't tell here? 
Please re-read the e-mail from Konstantine that started the thread. 

If you can't tell anything, Is it okay to me assume that you just wanted make negative noise in the community with unidentified words?
Please just share your thinking and let me hear and improve from it. 

I don’t see how having that discussion in public would serve any purpose. 

If you genuinely don’t understand — the end of our e-mail exchange was an invitation for you to call me on the phone and attempt to resolve those issues without the misunderstandings of tone that can happen in our e-mail exchange.  

The invitation stands. 

I will continue to look for ways to move forward in the spirit of cooperation and teamwork. 


From: moon soo Lee <mo...@apache.org>
Reply: moon soo Lee <mo...@apache.org>
Date: December 2, 2015 at 10:24:52 PM
To: Amos B. Elberg <am...@gmail.com>, dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  



On Thu, Dec 3, 2015 at 12:05 PM Amos B. Elberg <am...@gmail.com> wrote:
Moon,

I still suggest, one of the best way you can find that person is, file an 
issue about the CI problem that you found on Zeppelin Core. 
All of the relevant people are already informed.  

I continue to be ready to work cooperatively with anyone who would like to help me resolve the issues. 

I did clearly pointed what rule could be violated and provided a link and 
tried the best at explanation. 
Maybe this is a language issue, but I have no idea what your concern about a license is.  

You did cite to two web pages.  Neither of them said anything that would explain your position.  In fact, they confirmed what I had been saying all along.  

So i'm going to continue the discussion in the other thread about the 
license and will move the discussion to legal-discuss@. 
You’re welcome to do that, of course.  I don’t know that I will participate. 

What i'm not really fine is, having not enough discussion and concern about 
license. That's sign of unhealthy community. 
As Konstantine has pointed out:  The ASF has existed for 16 years.  This cannot be the first time that this issue arose.  The ASF has a “legal faq” and other public documents that discuss various licensing issues. 

In addition, a primary responsibility of the PMCC — perhaps the most important responsibility — is making sure the project conforms to ASF licensing policy. 

In terms of community health, my concern is that there is a great deal of confusion about these licensing issues, and people are not able to find a definitive answer simply by checking the ASF documents (or even attempting to do so).

(Hint:  I reviewed the ASF materials in great detail in preparing the PR.) 

I still don't know which part of the email are you referring. … What are you referring "the history"? 

You know *exactly* what I’m talking about. 


Amos,

This mailing list is subscribed by hundreds of people. Let's not trying to make meaningless posts.

I told you i *exactly* don't know what you're talking about "the history".
So why don't you pin point what *exactly* you talking about, in public?

Is there any reason that you can't tell here? 
If you can't tell anything, Is it okay to me assume that you just wanted make negative noise in the community with unidentified words?

Please just share your thinking and let me hear and improve from it. 

Thanks,
moon

 
From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 2, 2015 at 9:19:50 PM

To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

On Thu, Dec 3, 2015 at 7:34 AM Amos B. Elberg <am...@gmail.com> wrote:

> Moon, thank you for your reply.
>
> Of course, I can claim CI, license, etc and any other issue. Especially
> when CI is not passing, I can not sure about the license. To me, these
> claims are sign of 'healthy community' not sign of 'does not make sense'.
>
> You're not meaning, your contribution always need to be accepted without
> any claim. right?
> I believe the constructive way to move forward, would be for someone who
> well-understands the CI & build structure, to begin working with me to
> resolve the CI/build integration issues.
>
> I continue to be ready to work with that person.
>


I still suggest, one of the best way you can find that person is, file an
issue about the CI problem that you found on Zeppelin Core.
That'll reduce the scope of the problem and that'll help people quickly
getting into without understanding your contribution entirely.



>
> Regarding licensing, I took extensive research steps before submitting the
> PR to make sure there was not a licensing problem.
>
> In fact, many things in the structure of the code were chosen specifically
> to avoid any licensing problem. I even negotiated with the authors of some
> libraries to switch to Apache-compatible licenses, and did some work on
> their projects in exchange.
>
> Considering all of that — If anyone thinks there is a licensing issue, if
> they want to be constructive, the *least* they can do is say clearly
> exactly what rule is being violated, how it is being violated, and provide
> a link or citation or *something* that shows the opinion is more than
> hand-waving.
>
> I am ready to engage with anyone who does that.
>

There is separate thread for the license. So i'll leave minimal comment
here.

I did clearly pointed what rule could be violated and provided a link and
tried the best at explanation. You don't agree on my concern does not mean
it's okay to pass. Just like i don't agree on your opinion does not mean
it's confirmation of license problem.

Of course my concern could be wrong, that's totally fine to me. I don't
have any problem on that. I'm not a legal expert.

What i'm not really fine is, having not enough discussion and concern about
license. That's sign of unhealthy community.

So i'm going to continue the discussion in the other thread about the
license and will move the discussion to legal-discuss@.



>
> I don't know your view of history and what you think the history is.
> Yeah you do.
>
> We had an exchange about it a week before this thread began.
>
> Some of it even spilled-over into the PR comments after I saw the video.
>
>

I still don't know which part of the email are you referring. The
suggestion i made about your code? about the review? about the conference?
about the meetup? What are you referring "the history"?

Please say in public. What is the history and show how they're related to
this thread topic. Otherwise I'll assume you just want to make a negative
noise in the community.

Thanks,
moon



> So please share them in PUBLIC on this thread NOT off-list, if you think
> that's reason you think your contribution is in impasse.
>
> I am going to follow Konstantin’s lead reading this issue, and try to give
> you the benefit of the doubt.
>
>
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> Date: December 2, 2015 at 5:06:52 PM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> Thanks Amos for replying.
>
> On Thu, Dec 3, 2015 at 3:17 AM Amos B. Elberg <am...@gmail.com>
> wrote:
>
> > Moon — I think there is a misunderstanding about the topic of the
> > discussion.
> >
> > The PR has its own userbase. It has been and is being presented at user
> > groups. Its been blogged and tweeted about (none of that came from me!)
> > The features are the subject of two jiras and on the Zeppelin roadmap.
> > So, the discussion isn't about whether the PR is “good."
> >
> > But no-one responded to the PR until users began to tweet publicly
> @nflabs
> > asking why the PR had not been adopted, and e-mailing you directly. This
> > looks really bad, especially when the project is considering applying to
> > leave incubation.
> >
> >
>
> Thanks for pinging me. Otherwise i couldn't able to know that you're still
> working on it and ready to review. I think it's good practice that pinging
> committer for review when there is no sign of response. Except for ping
> message has been made on twitter and private email instead of public
> mailing list / jira / github issue comment.
>
>
>
>
> > The question here is what, if anything, prevents us from letting bygones
> > be bygones and moving forward with this now?
> >
> > Claims about CI issues, or licenses, or the PR shouldn’t have been
> rebased
> > (!?!) — well, they don’t really make sense.
> >
> >
> Although i didn't review your contribution from day 1,
> I'm reviewing your contribution, discussing about license, discussing about
> improvement of impasse, all they're part of moving forward. I am moving
> forward.
>
> Of course, I can claim CI, license, etc and any other issue. Especially
> when CI is not passing, I can not sure about the license. To me, these
> claims are sign of 'healthy community' not sign of 'does not make sense'.
>
> You're not meaning, your contribution always need to be accepted without
> any claim. right?
>
>
>
> > I keep offering to begin coordinating to integrate the PR with Zeppelin’s
> > CI and build system.
> >
> > But the answer (except from Roman) is still “nah, let us know if you
> > figure it out.”
> >
> >
> >
> Actually, my answer was, "If you think CI is failing not because of your
> change but because of Zeppelin core problem, then file an jira issue about
> it. Everyone will look into".
>
>
>
> >
> >
> > Regarding the history:
> >
> > Konstantin wisely started this thread by saying let’s keep the history
> out
> > of the discussion. I am respecting that.
> >
> > If the PR becomes part of Zeppelin, its going to need to be maintained,
> > which means that we are going to need to be able to work together.
> >
> > I have been persuaded to give Moon the benefit of the doubt regarding
> > certain issues. He certainly knows what my view of the history is.
> >
> > If anyone else would like to know, I am happy to share it with them
> > off-list.
> >
>
>
> I don't know your view of history and what you think the history is. So
> please share them in PUBLIC on this thread NOT off-list, if you think
> that's reason you think your contribution is in impasse.
>
> Otherwise I'll never know what you're thinking and I'll not improve. More
> importantly, it's easy to make people misunderstand you that you're just
> trying to make a negative noises.
>
> So, do you mind share your view of history in this thread?
>
> Thanks,
> moon
>
>
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > Date: December 2, 2015 at 7:45:11 AM
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> > request: R Interpreter for Zeppelin
> >
> > Thanks Roman and Eran for the feedback.
> >
> > *A. About contribution impasse in general*
> >
> > I think i summarized why it happens and how it can be improved. ie.
> >
> > 1. Large code base change
> > 2. Communication lost
> > 3. Opinion diverges
> >
> > And my solution was
> >
> > Guide to ping other committer when a committer is not responding, divide
> > contribution into small peaces if possible. And committer pay more
> > attention to the contribution.
> >
> > I'd like to hear and learn any more idea to improve.
> >
> >
> > *B. About contribution impasses in R interpreter for Zeppelin*
> >
> > Although I'was the first one who reviewed and commented this contribution
> > among the committer, I feel contributor (Amos) is unhappy about the
> review.
> >
> > I want to analyze the reasons and improve this, too.
> >
> > Here's reason i guess
> >
> > 1. Late responding (first review has been made after 3 months)
> > 2. Lack of help on CI fail (Amos keep complained about CI fail)
> >
> > I think both 1 and 2 can be improved by the solution i suggested in
> section
> > A.
> >
> > Amos, if you think there're more reasons, please feel free to say and let
> > me improve. What is the history you're mentioning?
> >
> > Best,
> > moon
> >
> >
> > On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org>
> wrote:
> >
> > > Just pushing discussion back on the list
> > >
> > > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com>
> wrote:
> > >
> > > > Alex — if you genuinely do not know the history of this, then I will
> > fill
> > > > you in.
> > > >
> > > > lmk…
> > > >
> > > >
> > > > --
> > > > Amos Elberg
> > > > Sent with Airmail
> > > >
> > > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > Date: December 1, 2015 at 6:14:20 AM
> > > > To: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org
> > > >
> > > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
> > > > <am...@gmail.com> <am...@gmail.com>
> > > >
> > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > > > pull request: R Interpreter for Zeppelin
> > > >
> > > > @Amos, we had plenty of cases of CI failing and always the
> > pre-condition
> > > > for a merge was a green CI. Sometimes that requires time, polite
> > > > collaboration, extra mile in direct asking for help from more
> > experienced
> > > > members and fixes in different places, which indeed might take time,
> as
> > > > everyone is busy.
> > > >
> > > >
> > >
> >
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

On Thu, Dec 3, 2015 at 12:05 PM Amos B. Elberg <am...@gmail.com>
wrote:

> Moon,
>
> I still suggest, one of the best way you can find that person is, file an
> issue about the CI problem that you found on Zeppelin Core.
>
> All of the relevant people are already informed.
>
> I continue to be ready to work cooperatively with anyone who would like to
> help me resolve the issues.
>
> I did clearly pointed what rule could be violated and provided a link and
> tried the best at explanation.
>
> Maybe this is a language issue, but I have no idea what your concern about
> a license is.
>
> You did cite to two web pages.  Neither of them said anything that would
> explain your position.  In fact, they confirmed what I had been saying all
> along.
>
> So i'm going to continue the discussion in the other thread about the
> license and will move the discussion to legal-discuss@.
>
> You’re welcome to do that, of course.  I don’t know that I will
> participate.
>
> What i'm not really fine is, having not enough discussion and concern
> about
> license. That's sign of unhealthy community.
>
> As Konstantine has pointed out:  The ASF has existed for 16 years.  This
> cannot be the first time that this issue arose.  The ASF has a “legal faq”
> and other public documents that discuss various licensing issues.
>
> In addition, a primary responsibility of the PMCC — perhaps the most
> important responsibility — is making sure the project conforms to ASF
> licensing policy.
>
> In terms of community health, my concern is that there is a great deal of
> confusion about these licensing issues, and people are not able to find a
> definitive answer simply by checking the ASF documents (or even attempting
> to do so).
>
> (Hint:  I reviewed the ASF materials in great detail in preparing the PR.)
>
> I still don't know which part of the email are you referring. … What are
> you referring "the history"?
>
>
> You know *exactly* what I’m talking about.
>
>
Amos,

This mailing list is subscribed by hundreds of people. Let's not trying to
make meaningless posts.

I told you i *exactly* don't know what you're talking about "the history".
So why don't you pin point what *exactly* you talking about, in public?

Is there any reason that you can't tell here?
If you can't tell anything, Is it okay to me assume that you just wanted
make negative noise in the community with unidentified words?

Please just share your thinking and let me hear and improve from it.

Thanks,
moon



> From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org
> <de...@zeppelin.incubator.apache.org> <de...@zeppelin.incubator.apache.org>
> Date: December 2, 2015 at 9:19:50 PM
>
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull request: R Interpreter for Zeppelin
>
> On Thu, Dec 3, 2015 at 7:34 AM Amos B. Elberg <am...@gmail.com>
> wrote:
>
> > Moon, thank you for your reply.
> >
> > Of course, I can claim CI, license, etc and any other issue. Especially
> > when CI is not passing, I can not sure about the license. To me, these
> > claims are sign of 'healthy community' not sign of 'does not make
> sense'.
> >
> > You're not meaning, your contribution always need to be accepted without
> > any claim. right?
> > I believe the constructive way to move forward, would be for someone who
> > well-understands the CI & build structure, to begin working with me to
> > resolve the CI/build integration issues.
> >
> > I continue to be ready to work with that person.
> >
>
>
> I still suggest, one of the best way you can find that person is, file an
> issue about the CI problem that you found on Zeppelin Core.
> That'll reduce the scope of the problem and that'll help people quickly
> getting into without understanding your contribution entirely.
>
>
>
> >
> > Regarding licensing, I took extensive research steps before submitting
> the
> > PR to make sure there was not a licensing problem.
> >
> > In fact, many things in the structure of the code were chosen
> specifically
> > to avoid any licensing problem. I even negotiated with the authors of
> some
> > libraries to switch to Apache-compatible licenses, and did some work on
> > their projects in exchange.
> >
> > Considering all of that — If anyone thinks there is a licensing issue,
> if
> > they want to be constructive, the *least* they can do is say clearly
> > exactly what rule is being violated, how it is being violated, and
> provide
> > a link or citation or *something* that shows the opinion is more than
> > hand-waving.
> >
> > I am ready to engage with anyone who does that.
> >
>
> There is separate thread for the license. So i'll leave minimal comment
> here.
>
> I did clearly pointed what rule could be violated and provided a link and
> tried the best at explanation. You don't agree on my concern does not mean
> it's okay to pass. Just like i don't agree on your opinion does not mean
> it's confirmation of license problem.
>
> Of course my concern could be wrong, that's totally fine to me. I don't
> have any problem on that. I'm not a legal expert.
>
> What i'm not really fine is, having not enough discussion and concern
> about
> license. That's sign of unhealthy community.
>
> So i'm going to continue the discussion in the other thread about the
> license and will move the discussion to legal-discuss@.
>
>
>
> >
> > I don't know your view of history and what you think the history is.
> > Yeah you do.
> >
> > We had an exchange about it a week before this thread began.
> >
> > Some of it even spilled-over into the PR comments after I saw the video.
> >
> >
>
> I still don't know which part of the email are you referring. The
> suggestion i made about your code? about the review? about the conference?
> about the meetup? What are you referring "the history"?
>
> Please say in public. What is the history and show how they're related to
> this thread topic. Otherwise I'll assume you just want to make a negative
> noise in the community.
>
> Thanks,
> moon
>
>
>
> > So please share them in PUBLIC on this thread NOT off-list, if you think
> > that's reason you think your contribution is in impasse.
> >
> > I am going to follow Konstantin’s lead reading this issue, and try to
> give
> > you the benefit of the doubt.
> >
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > Date: December 2, 2015 at 5:06:52 PM
> > To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
>
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull
> > request: R Interpreter for Zeppelin
> >
> > Thanks Amos for replying.
> >
> > On Thu, Dec 3, 2015 at 3:17 AM Amos B. Elberg <am...@gmail.com>
> > wrote:
> >
> > > Moon — I think there is a misunderstanding about the topic of the
> > > discussion.
> > >
> > > The PR has its own userbase. It has been and is being presented at
> user
> > > groups. Its been blogged and tweeted about (none of that came from
> me!)
> > > The features are the subject of two jiras and on the Zeppelin roadmap.
> > > So, the discussion isn't about whether the PR is “good."
> > >
> > > But no-one responded to the PR until users began to tweet publicly
> > @nflabs
> > > asking why the PR had not been adopted, and e-mailing you directly.
> This
> > > looks really bad, especially when the project is considering applying
> to
> > > leave incubation.
> > >
> > >
> >
> > Thanks for pinging me. Otherwise i couldn't able to know that you're
> still
> > working on it and ready to review. I think it's good practice that
> pinging
> > committer for review when there is no sign of response. Except for ping
> > message has been made on twitter and private email instead of public
> > mailing list / jira / github issue comment.
> >
> >
> >
> >
> > > The question here is what, if anything, prevents us from letting
> bygones
> > > be bygones and moving forward with this now?
> > >
> > > Claims about CI issues, or licenses, or the PR shouldn’t have been
> > rebased
> > > (!?!) — well, they don’t really make sense.
> > >
> > >
> > Although i didn't review your contribution from day 1,
> > I'm reviewing your contribution, discussing about license, discussing
> about
> > improvement of impasse, all they're part of moving forward. I am moving
> > forward.
> >
> > Of course, I can claim CI, license, etc and any other issue. Especially
> > when CI is not passing, I can not sure about the license. To me, these
> > claims are sign of 'healthy community' not sign of 'does not make
> sense'.
> >
> > You're not meaning, your contribution always need to be accepted without
> > any claim. right?
> >
> >
> >
> > > I keep offering to begin coordinating to integrate the PR with
> Zeppelin’s
> > > CI and build system.
> > >
> > > But the answer (except from Roman) is still “nah, let us know if you
> > > figure it out.”
> > >
> > >
> > >
> > Actually, my answer was, "If you think CI is failing not because of your
> > change but because of Zeppelin core problem, then file an jira issue
> about
> > it. Everyone will look into".
> >
> >
> >
> > >
> > >
> > > Regarding the history:
> > >
> > > Konstantin wisely started this thread by saying let’s keep the history
> > out
> > > of the discussion. I am respecting that.
> > >
> > > If the PR becomes part of Zeppelin, its going to need to be
> maintained,
> > > which means that we are going to need to be able to work together.
> > >
> > > I have been persuaded to give Moon the benefit of the doubt regarding
> > > certain issues. He certainly knows what my view of the history is.
> > >
> > > If anyone else would like to know, I am happy to share it with them
> > > off-list.
> > >
> >
> >
> > I don't know your view of history and what you think the history is. So
> > please share them in PUBLIC on this thread NOT off-list, if you think
> > that's reason you think your contribution is in impasse.
> >
> > Otherwise I'll never know what you're thinking and I'll not improve.
> More
> > importantly, it's easy to make people misunderstand you that you're just
> > trying to make a negative noises.
> >
> > So, do you mind share your view of history in this thread?
> >
> > Thanks,
> > moon
> >
> >
> > >
> > > From: moon soo Lee <mo...@apache.org>
> > > Reply: dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org>
> > > Date: December 2, 2015 at 7:45:11 AM
> > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org
> > >
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull
> > > request: R Interpreter for Zeppelin
> > >
> > > Thanks Roman and Eran for the feedback.
> > >
> > > *A. About contribution impasse in general*
> > >
> > > I think i summarized why it happens and how it can be improved. ie.
> > >
> > > 1. Large code base change
> > > 2. Communication lost
> > > 3. Opinion diverges
> > >
> > > And my solution was
> > >
> > > Guide to ping other committer when a committer is not responding,
> divide
> > > contribution into small peaces if possible. And committer pay more
> > > attention to the contribution.
> > >
> > > I'd like to hear and learn any more idea to improve.
> > >
> > >
> > > *B. About contribution impasses in R interpreter for Zeppelin*
> > >
> > > Although I'was the first one who reviewed and commented this
> contribution
> > > among the committer, I feel contributor (Amos) is unhappy about the
> > review.
> > >
> > > I want to analyze the reasons and improve this, too.
> > >
> > > Here's reason i guess
> > >
> > > 1. Late responding (first review has been made after 3 months)
> > > 2. Lack of help on CI fail (Amos keep complained about CI fail)
> > >
> > > I think both 1 and 2 can be improved by the solution i suggested in
> > section
> > > A.
> > >
> > > Amos, if you think there're more reasons, please feel free to say and
> let
> > > me improve. What is the history you're mentioning?
> > >
> > > Best,
> > > moon
> > >
> > >
> > > On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org>
> > wrote:
> > >
> > > > Just pushing discussion back on the list
> > > >
> > > > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com>
> > wrote:
> > > >
> > > > > Alex — if you genuinely do not know the history of this, then I
> will
> > > fill
> > > > > you in.
> > > > >
> > > > > lmk…
> > > > >
> > > > >
> > > > > --
> > > > > Amos Elberg
> > > > > Sent with Airmail
> > > > >
> > > > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > > Date: December 1, 2015 at 6:14:20 AM
> > > > > To: dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org
> > > > >
> > > > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
> > > > > <am...@gmail.com> <am...@gmail.com>
> > > > >
> > > > > Subject: Re: contributions impasse. Was: [GitHub]
> incubator-zeppelin
> > > > > pull request: R Interpreter for Zeppelin
> > > > >
> > > > > @Amos, we had plenty of cases of CI failing and always the
> > > pre-condition
> > > > > for a merge was a green CI. Sometimes that requires time, polite
> > > > > collaboration, extra mile in direct asking for help from more
> > > experienced
> > > > > members and fixes in different places, which indeed might take
> time,
> > as
> > > > > everyone is busy.
> > > > >
> > > > >
> > > >
> > >
> >
>
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Moon,

I still suggest, one of the best way you can find that person is, file an 
issue about the CI problem that you found on Zeppelin Core. 
All of the relevant people are already informed.  

I continue to be ready to work cooperatively with anyone who would like to help me resolve the issues. 

I did clearly pointed what rule could be violated and provided a link and 
tried the best at explanation. 
Maybe this is a language issue, but I have no idea what your concern about a license is.  

You did cite to two web pages.  Neither of them said anything that would explain your position.  In fact, they confirmed what I had been saying all along.  

So i'm going to continue the discussion in the other thread about the 
license and will move the discussion to legal-discuss@. 
You’re welcome to do that, of course.  I don’t know that I will participate. 

What i'm not really fine is, having not enough discussion and concern about 
license. That's sign of unhealthy community. 
As Konstantine has pointed out:  The ASF has existed for 16 years.  This cannot be the first time that this issue arose.  The ASF has a “legal faq” and other public documents that discuss various licensing issues. 

In addition, a primary responsibility of the PMCC — perhaps the most important responsibility — is making sure the project conforms to ASF licensing policy. 

In terms of community health, my concern is that there is a great deal of confusion about these licensing issues, and people are not able to find a definitive answer simply by checking the ASF documents (or even attempting to do so).

(Hint:  I reviewed the ASF materials in great detail in preparing the PR.) 

I still don't know which part of the email are you referring. … What are you referring "the history"? 

You know *exactly* what I’m talking about. 

From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 2, 2015 at 9:19:50 PM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

On Thu, Dec 3, 2015 at 7:34 AM Amos B. Elberg <am...@gmail.com> wrote:  

> Moon, thank you for your reply.  
>  
> Of course, I can claim CI, license, etc and any other issue. Especially  
> when CI is not passing, I can not sure about the license. To me, these  
> claims are sign of 'healthy community' not sign of 'does not make sense'.  
>  
> You're not meaning, your contribution always need to be accepted without  
> any claim. right?  
> I believe the constructive way to move forward, would be for someone who  
> well-understands the CI & build structure, to begin working with me to  
> resolve the CI/build integration issues.  
>  
> I continue to be ready to work with that person.  
>  


I still suggest, one of the best way you can find that person is, file an  
issue about the CI problem that you found on Zeppelin Core.  
That'll reduce the scope of the problem and that'll help people quickly  
getting into without understanding your contribution entirely.  



>  
> Regarding licensing, I took extensive research steps before submitting the  
> PR to make sure there was not a licensing problem.  
>  
> In fact, many things in the structure of the code were chosen specifically  
> to avoid any licensing problem. I even negotiated with the authors of some  
> libraries to switch to Apache-compatible licenses, and did some work on  
> their projects in exchange.  
>  
> Considering all of that — If anyone thinks there is a licensing issue, if  
> they want to be constructive, the *least* they can do is say clearly  
> exactly what rule is being violated, how it is being violated, and provide  
> a link or citation or *something* that shows the opinion is more than  
> hand-waving.  
>  
> I am ready to engage with anyone who does that.  
>  

There is separate thread for the license. So i'll leave minimal comment  
here.  

I did clearly pointed what rule could be violated and provided a link and  
tried the best at explanation. You don't agree on my concern does not mean  
it's okay to pass. Just like i don't agree on your opinion does not mean  
it's confirmation of license problem.  

Of course my concern could be wrong, that's totally fine to me. I don't  
have any problem on that. I'm not a legal expert.  

What i'm not really fine is, having not enough discussion and concern about  
license. That's sign of unhealthy community.  

So i'm going to continue the discussion in the other thread about the  
license and will move the discussion to legal-discuss@.  



>  
> I don't know your view of history and what you think the history is.  
> Yeah you do.  
>  
> We had an exchange about it a week before this thread began.  
>  
> Some of it even spilled-over into the PR comments after I saw the video.  
>  
>  

I still don't know which part of the email are you referring. The  
suggestion i made about your code? about the review? about the conference?  
about the meetup? What are you referring "the history"?  

Please say in public. What is the history and show how they're related to  
this thread topic. Otherwise I'll assume you just want to make a negative  
noise in the community.  

Thanks,  
moon  



> So please share them in PUBLIC on this thread NOT off-list, if you think  
> that's reason you think your contribution is in impasse.  
>  
> I am going to follow Konstantin’s lead reading this issue, and try to give  
> you the benefit of the doubt.  
>  
>  
> From: moon soo Lee <mo...@apache.org>  
> Reply: dev@zeppelin.incubator.apache.org <  
> dev@zeppelin.incubator.apache.org>  
> Date: December 2, 2015 at 5:06:52 PM  
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> request: R Interpreter for Zeppelin  
>  
> Thanks Amos for replying.  
>  
> On Thu, Dec 3, 2015 at 3:17 AM Amos B. Elberg <am...@gmail.com>  
> wrote:  
>  
> > Moon — I think there is a misunderstanding about the topic of the  
> > discussion.  
> >  
> > The PR has its own userbase. It has been and is being presented at user  
> > groups. Its been blogged and tweeted about (none of that came from me!)  
> > The features are the subject of two jiras and on the Zeppelin roadmap.  
> > So, the discussion isn't about whether the PR is “good."  
> >  
> > But no-one responded to the PR until users began to tweet publicly  
> @nflabs  
> > asking why the PR had not been adopted, and e-mailing you directly. This  
> > looks really bad, especially when the project is considering applying to  
> > leave incubation.  
> >  
> >  
>  
> Thanks for pinging me. Otherwise i couldn't able to know that you're still  
> working on it and ready to review. I think it's good practice that pinging  
> committer for review when there is no sign of response. Except for ping  
> message has been made on twitter and private email instead of public  
> mailing list / jira / github issue comment.  
>  
>  
>  
>  
> > The question here is what, if anything, prevents us from letting bygones  
> > be bygones and moving forward with this now?  
> >  
> > Claims about CI issues, or licenses, or the PR shouldn’t have been  
> rebased  
> > (!?!) — well, they don’t really make sense.  
> >  
> >  
> Although i didn't review your contribution from day 1,  
> I'm reviewing your contribution, discussing about license, discussing about  
> improvement of impasse, all they're part of moving forward. I am moving  
> forward.  
>  
> Of course, I can claim CI, license, etc and any other issue. Especially  
> when CI is not passing, I can not sure about the license. To me, these  
> claims are sign of 'healthy community' not sign of 'does not make sense'.  
>  
> You're not meaning, your contribution always need to be accepted without  
> any claim. right?  
>  
>  
>  
> > I keep offering to begin coordinating to integrate the PR with Zeppelin’s  
> > CI and build system.  
> >  
> > But the answer (except from Roman) is still “nah, let us know if you  
> > figure it out.”  
> >  
> >  
> >  
> Actually, my answer was, "If you think CI is failing not because of your  
> change but because of Zeppelin core problem, then file an jira issue about  
> it. Everyone will look into".  
>  
>  
>  
> >  
> >  
> > Regarding the history:  
> >  
> > Konstantin wisely started this thread by saying let’s keep the history  
> out  
> > of the discussion. I am respecting that.  
> >  
> > If the PR becomes part of Zeppelin, its going to need to be maintained,  
> > which means that we are going to need to be able to work together.  
> >  
> > I have been persuaded to give Moon the benefit of the doubt regarding  
> > certain issues. He certainly knows what my view of the history is.  
> >  
> > If anyone else would like to know, I am happy to share it with them  
> > off-list.  
> >  
>  
>  
> I don't know your view of history and what you think the history is. So  
> please share them in PUBLIC on this thread NOT off-list, if you think  
> that's reason you think your contribution is in impasse.  
>  
> Otherwise I'll never know what you're thinking and I'll not improve. More  
> importantly, it's easy to make people misunderstand you that you're just  
> trying to make a negative noises.  
>  
> So, do you mind share your view of history in this thread?  
>  
> Thanks,  
> moon  
>  
>  
> >  
> > From: moon soo Lee <mo...@apache.org>  
> > Reply: dev@zeppelin.incubator.apache.org <  
> > dev@zeppelin.incubator.apache.org>  
> > Date: December 2, 2015 at 7:45:11 AM  
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org  
> >  
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> > request: R Interpreter for Zeppelin  
> >  
> > Thanks Roman and Eran for the feedback.  
> >  
> > *A. About contribution impasse in general*  
> >  
> > I think i summarized why it happens and how it can be improved. ie.  
> >  
> > 1. Large code base change  
> > 2. Communication lost  
> > 3. Opinion diverges  
> >  
> > And my solution was  
> >  
> > Guide to ping other committer when a committer is not responding, divide  
> > contribution into small peaces if possible. And committer pay more  
> > attention to the contribution.  
> >  
> > I'd like to hear and learn any more idea to improve.  
> >  
> >  
> > *B. About contribution impasses in R interpreter for Zeppelin*  
> >  
> > Although I'was the first one who reviewed and commented this contribution  
> > among the committer, I feel contributor (Amos) is unhappy about the  
> review.  
> >  
> > I want to analyze the reasons and improve this, too.  
> >  
> > Here's reason i guess  
> >  
> > 1. Late responding (first review has been made after 3 months)  
> > 2. Lack of help on CI fail (Amos keep complained about CI fail)  
> >  
> > I think both 1 and 2 can be improved by the solution i suggested in  
> section  
> > A.  
> >  
> > Amos, if you think there're more reasons, please feel free to say and let  
> > me improve. What is the history you're mentioning?  
> >  
> > Best,  
> > moon  
> >  
> >  
> > On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org>  
> wrote:  
> >  
> > > Just pushing discussion back on the list  
> > >  
> > > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com>  
> wrote:  
> > >  
> > > > Alex — if you genuinely do not know the history of this, then I will  
> > fill  
> > > > you in.  
> > > >  
> > > > lmk…  
> > > >  
> > > >  
> > > > --  
> > > > Amos Elberg  
> > > > Sent with Airmail  
> > > >  
> > > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>  
> > > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>  
> > > > Date: December 1, 2015 at 6:14:20 AM  
> > > > To: dev@zeppelin.incubator.apache.org <  
> > dev@zeppelin.incubator.apache.org  
> > > >  
> > > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg  
> > > > <am...@gmail.com> <am...@gmail.com>  
> > > >  
> > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin  
> > > > pull request: R Interpreter for Zeppelin  
> > > >  
> > > > @Amos, we had plenty of cases of CI failing and always the  
> > pre-condition  
> > > > for a merge was a green CI. Sometimes that requires time, polite  
> > > > collaboration, extra mile in direct asking for help from more  
> > experienced  
> > > > members and fixes in different places, which indeed might take time,  
> as  
> > > > everyone is busy.  
> > > >  
> > > >  
> > >  
> >  
>  

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

On Thu, Dec 3, 2015 at 7:34 AM Amos B. Elberg <am...@gmail.com> wrote:

> Moon, thank you for your reply.
>
> Of course, I can claim CI, license, etc and any other issue. Especially
> when CI is not passing, I can not sure about the license. To me, these
> claims are sign of 'healthy community' not sign of 'does not make sense'.
>
> You're not meaning, your contribution always need to be accepted without
> any claim. right?
> I believe the constructive way to move forward, would be for someone who
> well-understands the CI & build structure, to begin working with me to
> resolve the CI/build integration issues.
>
> I continue to be ready to work with that person.
>


I still suggest, one of the best way you can find that person is, file an
issue about the CI problem that you found on Zeppelin Core.
That'll reduce the scope of the problem and that'll help people quickly
getting into without understanding your contribution entirely.



>
> Regarding licensing, I took extensive research steps before submitting the
> PR to make sure there was not a licensing problem.
>
> In fact, many things in the structure of the code were chosen specifically
> to avoid any licensing problem. I even negotiated with the authors of some
> libraries to switch to Apache-compatible licenses, and did some work on
> their projects in exchange.
>
> Considering all of that — If anyone thinks there is a licensing issue, if
> they want to be constructive, the *least* they can do is say clearly
> exactly what rule is being violated, how it is being violated, and provide
> a link or citation or *something* that shows the opinion is more than
> hand-waving.
>
> I am ready to engage with anyone who does that.
>

There is separate thread for the license. So i'll leave minimal comment
here.

I did clearly pointed what rule could be violated and provided a link and
tried the best at explanation. You don't agree on my concern does not mean
it's okay to pass. Just like i don't agree on your opinion does not mean
it's confirmation of license problem.

Of course my concern could be wrong, that's totally fine to me. I don't
have any problem on that. I'm not a legal expert.

What i'm not really fine is, having not enough discussion and concern about
license. That's sign of unhealthy community.

So i'm going to continue the discussion in the other thread about the
license and will move the discussion to legal-discuss@.



>
> I don't know your view of history and what you think the history is.
> Yeah you do.
>
> We had an exchange about it a week before this thread began.
>
> Some of it even spilled-over into the PR comments after I saw the video.
>
>

I still don't know which part of the email are you referring. The
suggestion i made about your code? about the review? about the conference?
about the meetup?  What are you referring "the history"?

Please say in public. What is the history and show how they're related to
this thread topic. Otherwise I'll assume you just want to make a negative
noise in the community.

Thanks,
moon



> So please share them in PUBLIC on this thread NOT off-list, if you think
> that's reason you think your contribution is in impasse.
>
> I am going to follow Konstantin’s lead reading this issue, and try to give
> you the benefit of the doubt.
>
>
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> Date: December 2, 2015 at 5:06:52 PM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> Thanks Amos for replying.
>
> On Thu, Dec 3, 2015 at 3:17 AM Amos B. Elberg <am...@gmail.com>
> wrote:
>
> > Moon — I think there is a misunderstanding about the topic of the
> > discussion.
> >
> > The PR has its own userbase. It has been and is being presented at user
> > groups. Its been blogged and tweeted about (none of that came from me!)
> > The features are the subject of two jiras and on the Zeppelin roadmap.
> > So, the discussion isn't about whether the PR is “good."
> >
> > But no-one responded to the PR until users began to tweet publicly
> @nflabs
> > asking why the PR had not been adopted, and e-mailing you directly. This
> > looks really bad, especially when the project is considering applying to
> > leave incubation.
> >
> >
>
> Thanks for pinging me. Otherwise i couldn't able to know that you're still
> working on it and ready to review. I think it's good practice that pinging
> committer for review when there is no sign of response. Except for ping
> message has been made on twitter and private email instead of public
> mailing list / jira / github issue comment.
>
>
>
>
> > The question here is what, if anything, prevents us from letting bygones
> > be bygones and moving forward with this now?
> >
> > Claims about CI issues, or licenses, or the PR shouldn’t have been
> rebased
> > (!?!) — well, they don’t really make sense.
> >
> >
> Although i didn't review your contribution from day 1,
> I'm reviewing your contribution, discussing about license, discussing about
> improvement of impasse, all they're part of moving forward. I am moving
> forward.
>
> Of course, I can claim CI, license, etc and any other issue. Especially
> when CI is not passing, I can not sure about the license. To me, these
> claims are sign of 'healthy community' not sign of 'does not make sense'.
>
> You're not meaning, your contribution always need to be accepted without
> any claim. right?
>
>
>
> > I keep offering to begin coordinating to integrate the PR with Zeppelin’s
> > CI and build system.
> >
> > But the answer (except from Roman) is still “nah, let us know if you
> > figure it out.”
> >
> >
> >
> Actually, my answer was, "If you think CI is failing not because of your
> change but because of Zeppelin core problem, then file an jira issue about
> it. Everyone will look into".
>
>
>
> >
> >
> > Regarding the history:
> >
> > Konstantin wisely started this thread by saying let’s keep the history
> out
> > of the discussion. I am respecting that.
> >
> > If the PR becomes part of Zeppelin, its going to need to be maintained,
> > which means that we are going to need to be able to work together.
> >
> > I have been persuaded to give Moon the benefit of the doubt regarding
> > certain issues. He certainly knows what my view of the history is.
> >
> > If anyone else would like to know, I am happy to share it with them
> > off-list.
> >
>
>
> I don't know your view of history and what you think the history is. So
> please share them in PUBLIC on this thread NOT off-list, if you think
> that's reason you think your contribution is in impasse.
>
> Otherwise I'll never know what you're thinking and I'll not improve. More
> importantly, it's easy to make people misunderstand you that you're just
> trying to make a negative noises.
>
> So, do you mind share your view of history in this thread?
>
> Thanks,
> moon
>
>
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > Date: December 2, 2015 at 7:45:11 AM
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> > request: R Interpreter for Zeppelin
> >
> > Thanks Roman and Eran for the feedback.
> >
> > *A. About contribution impasse in general*
> >
> > I think i summarized why it happens and how it can be improved. ie.
> >
> > 1. Large code base change
> > 2. Communication lost
> > 3. Opinion diverges
> >
> > And my solution was
> >
> > Guide to ping other committer when a committer is not responding, divide
> > contribution into small peaces if possible. And committer pay more
> > attention to the contribution.
> >
> > I'd like to hear and learn any more idea to improve.
> >
> >
> > *B. About contribution impasses in R interpreter for Zeppelin*
> >
> > Although I'was the first one who reviewed and commented this contribution
> > among the committer, I feel contributor (Amos) is unhappy about the
> review.
> >
> > I want to analyze the reasons and improve this, too.
> >
> > Here's reason i guess
> >
> > 1. Late responding (first review has been made after 3 months)
> > 2. Lack of help on CI fail (Amos keep complained about CI fail)
> >
> > I think both 1 and 2 can be improved by the solution i suggested in
> section
> > A.
> >
> > Amos, if you think there're more reasons, please feel free to say and let
> > me improve. What is the history you're mentioning?
> >
> > Best,
> > moon
> >
> >
> > On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org>
> wrote:
> >
> > > Just pushing discussion back on the list
> > >
> > > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com>
> wrote:
> > >
> > > > Alex — if you genuinely do not know the history of this, then I will
> > fill
> > > > you in.
> > > >
> > > > lmk…
> > > >
> > > >
> > > > --
> > > > Amos Elberg
> > > > Sent with Airmail
> > > >
> > > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > > Date: December 1, 2015 at 6:14:20 AM
> > > > To: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org
> > > >
> > > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
> > > > <am...@gmail.com> <am...@gmail.com>
> > > >
> > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > > > pull request: R Interpreter for Zeppelin
> > > >
> > > > @Amos, we had plenty of cases of CI failing and always the
> > pre-condition
> > > > for a merge was a green CI. Sometimes that requires time, polite
> > > > collaboration, extra mile in direct asking for help from more
> > experienced
> > > > members and fixes in different places, which indeed might take time,
> as
> > > > everyone is busy.
> > > >
> > > >
> > >
> >
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Moon, thank you for your reply.

Of course, I can claim CI, license, etc and any other issue. Especially 
when CI is not passing, I can not sure about the license. To me, these 
claims are sign of 'healthy community' not sign of 'does not make sense'. 

You're not meaning, your contribution always need to be accepted without 
any claim. right? 
I believe the constructive way to move forward, would be for someone who well-understands the CI & build structure, to begin working with me to resolve the CI/build integration issues.  

I continue to be ready to work with that person. 

Regarding licensing, I took extensive research steps before submitting the PR to make sure there was not a licensing problem.

In fact, many things in the structure of the code were chosen specifically to avoid any licensing problem. I even negotiated with the authors of some libraries to switch to Apache-compatible licenses, and did some work on their projects in exchange. 

Considering all of that — If anyone thinks there is a licensing issue, if they want to be constructive, the *least* they can do is say clearly exactly what rule is being violated, how it is being violated, and provide a link or citation or *something* that shows the opinion is more than hand-waving. 

I am ready to engage with anyone who does that. 

I don't know your view of history and what you think the history is. 
Yeah you do.  

We had an exchange about it a week before this thread began.  

Some of it even spilled-over into the PR comments after I saw the video. 

So please share them in PUBLIC on this thread NOT off-list, if you think 
that's reason you think your contribution is in impasse. 

I am going to follow Konstantin’s lead reading this issue, and try to give you the benefit of the doubt.  


From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 2, 2015 at 5:06:52 PM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

Thanks Amos for replying.  

On Thu, Dec 3, 2015 at 3:17 AM Amos B. Elberg <am...@gmail.com> wrote:  

> Moon — I think there is a misunderstanding about the topic of the  
> discussion.  
>  
> The PR has its own userbase. It has been and is being presented at user  
> groups. Its been blogged and tweeted about (none of that came from me!)  
> The features are the subject of two jiras and on the Zeppelin roadmap.  
> So, the discussion isn't about whether the PR is “good."  
>  
> But no-one responded to the PR until users began to tweet publicly @nflabs  
> asking why the PR had not been adopted, and e-mailing you directly. This  
> looks really bad, especially when the project is considering applying to  
> leave incubation.  
>  
>  

Thanks for pinging me. Otherwise i couldn't able to know that you're still  
working on it and ready to review. I think it's good practice that pinging  
committer for review when there is no sign of response. Except for ping  
message has been made on twitter and private email instead of public  
mailing list / jira / github issue comment.  




> The question here is what, if anything, prevents us from letting bygones  
> be bygones and moving forward with this now?  
>  
> Claims about CI issues, or licenses, or the PR shouldn’t have been rebased  
> (!?!) — well, they don’t really make sense.  
>  
>  
Although i didn't review your contribution from day 1,  
I'm reviewing your contribution, discussing about license, discussing about  
improvement of impasse, all they're part of moving forward. I am moving  
forward.  

Of course, I can claim CI, license, etc and any other issue. Especially  
when CI is not passing, I can not sure about the license. To me, these  
claims are sign of 'healthy community' not sign of 'does not make sense'.  

You're not meaning, your contribution always need to be accepted without  
any claim. right?  



> I keep offering to begin coordinating to integrate the PR with Zeppelin’s  
> CI and build system.  
>  
> But the answer (except from Roman) is still “nah, let us know if you  
> figure it out.”  
>  
>  
>  
Actually, my answer was, "If you think CI is failing not because of your  
change but because of Zeppelin core problem, then file an jira issue about  
it. Everyone will look into".  



>  
>  
> Regarding the history:  
>  
> Konstantin wisely started this thread by saying let’s keep the history out  
> of the discussion. I am respecting that.  
>  
> If the PR becomes part of Zeppelin, its going to need to be maintained,  
> which means that we are going to need to be able to work together.  
>  
> I have been persuaded to give Moon the benefit of the doubt regarding  
> certain issues. He certainly knows what my view of the history is.  
>  
> If anyone else would like to know, I am happy to share it with them  
> off-list.  
>  


I don't know your view of history and what you think the history is. So  
please share them in PUBLIC on this thread NOT off-list, if you think  
that's reason you think your contribution is in impasse.  

Otherwise I'll never know what you're thinking and I'll not improve. More  
importantly, it's easy to make people misunderstand you that you're just  
trying to make a negative noises.  

So, do you mind share your view of history in this thread?  

Thanks,  
moon  


>  
> From: moon soo Lee <mo...@apache.org>  
> Reply: dev@zeppelin.incubator.apache.org <  
> dev@zeppelin.incubator.apache.org>  
> Date: December 2, 2015 at 7:45:11 AM  
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> request: R Interpreter for Zeppelin  
>  
> Thanks Roman and Eran for the feedback.  
>  
> *A. About contribution impasse in general*  
>  
> I think i summarized why it happens and how it can be improved. ie.  
>  
> 1. Large code base change  
> 2. Communication lost  
> 3. Opinion diverges  
>  
> And my solution was  
>  
> Guide to ping other committer when a committer is not responding, divide  
> contribution into small peaces if possible. And committer pay more  
> attention to the contribution.  
>  
> I'd like to hear and learn any more idea to improve.  
>  
>  
> *B. About contribution impasses in R interpreter for Zeppelin*  
>  
> Although I'was the first one who reviewed and commented this contribution  
> among the committer, I feel contributor (Amos) is unhappy about the review.  
>  
> I want to analyze the reasons and improve this, too.  
>  
> Here's reason i guess  
>  
> 1. Late responding (first review has been made after 3 months)  
> 2. Lack of help on CI fail (Amos keep complained about CI fail)  
>  
> I think both 1 and 2 can be improved by the solution i suggested in section  
> A.  
>  
> Amos, if you think there're more reasons, please feel free to say and let  
> me improve. What is the history you're mentioning?  
>  
> Best,  
> moon  
>  
>  
> On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org> wrote:  
>  
> > Just pushing discussion back on the list  
> >  
> > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com> wrote:  
> >  
> > > Alex — if you genuinely do not know the history of this, then I will  
> fill  
> > > you in.  
> > >  
> > > lmk…  
> > >  
> > >  
> > > --  
> > > Amos Elberg  
> > > Sent with Airmail  
> > >  
> > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>  
> > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>  
> > > Date: December 1, 2015 at 6:14:20 AM  
> > > To: dev@zeppelin.incubator.apache.org <  
> dev@zeppelin.incubator.apache.org  
> > >  
> > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg  
> > > <am...@gmail.com> <am...@gmail.com>  
> > >  
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin  
> > > pull request: R Interpreter for Zeppelin  
> > >  
> > > @Amos, we had plenty of cases of CI failing and always the  
> pre-condition  
> > > for a merge was a green CI. Sometimes that requires time, polite  
> > > collaboration, extra mile in direct asking for help from more  
> experienced  
> > > members and fixes in different places, which indeed might take time, as  
> > > everyone is busy.  
> > >  
> > >  
> >  
>  

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Thanks Amos for replying.

On Thu, Dec 3, 2015 at 3:17 AM Amos B. Elberg <am...@gmail.com> wrote:

> Moon — I think there is a misunderstanding about the topic of the
> discussion.
>
> The PR has its own userbase.  It has been and is being presented at user
> groups.  Its been blogged and tweeted about (none of that came from me!)
>  The features are the subject of two jiras and on the Zeppelin roadmap.
> So, the discussion isn't about whether the PR is “good."
>
> But no-one responded to the PR until users began to tweet publicly @nflabs
> asking why the PR had not been adopted, and e-mailing you directly.  This
> looks really bad, especially when the project is considering applying to
> leave incubation.
>
>

Thanks for pinging me. Otherwise i couldn't able to know that you're still
working on it and ready to review. I think it's good practice that pinging
committer for review when there is no sign of response. Except for ping
message has been made on twitter and private email instead of public
mailing list / jira / github issue comment.




> The question here is what, if anything, prevents us from letting bygones
> be bygones and moving forward with this now?
>
> Claims about CI issues, or licenses, or the PR shouldn’t have been rebased
> (!?!) — well, they don’t really make sense.
>
>
Although i didn't review your contribution from day 1,
I'm reviewing your contribution, discussing about license, discussing about
improvement of impasse, all they're part of moving forward. I am moving
forward.

Of course, I can claim CI, license, etc and any other issue. Especially
when CI is not passing, I can not sure about the license. To me, these
claims are sign of 'healthy community' not sign of 'does not make sense'.

You're not meaning, your contribution always need to be accepted without
any claim. right?



> I keep offering to begin coordinating to integrate the PR with Zeppelin’s
> CI and build system.
>
> But the answer (except from Roman) is still “nah, let us know if you
> figure it out.”
>
>
>
Actually, my answer was, "If you think CI is failing not because of your
change but because of Zeppelin core problem, then file an jira issue about
it. Everyone will look into".



>
>
> Regarding the history:
>
> Konstantin wisely started this thread by saying let’s keep the history out
> of the discussion.  I am respecting that.
>
> If the PR becomes part of Zeppelin, its going to need to be maintained,
> which means that we are going to need to be able to work together.
>
> I have been persuaded to give Moon the benefit of the doubt regarding
> certain issues.  He certainly knows what my view of the history is.
>
> If anyone else would like to know, I am happy to share it with them
> off-list.
>


I don't know your view of history and what you think the history is. So
please share them in PUBLIC on this thread NOT off-list, if you think
that's reason you think your contribution is in impasse.

Otherwise I'll never know what you're thinking and I'll not improve. More
importantly, it's easy to make people misunderstand you that you're just
trying to make a negative noises.

So, do you mind share your view of history in this thread?

Thanks,
moon


>
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> Date: December 2, 2015 at 7:45:11 AM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> Thanks Roman and Eran for the feedback.
>
> *A. About contribution impasse in general*
>
> I think i summarized why it happens and how it can be improved. ie.
>
> 1. Large code base change
> 2. Communication lost
> 3. Opinion diverges
>
> And my solution was
>
> Guide to ping other committer when a committer is not responding, divide
> contribution into small peaces if possible. And committer pay more
> attention to the contribution.
>
> I'd like to hear and learn any more idea to improve.
>
>
> *B. About contribution impasses in R interpreter for Zeppelin*
>
> Although I'was the first one who reviewed and commented this contribution
> among the committer, I feel contributor (Amos) is unhappy about the review.
>
> I want to analyze the reasons and improve this, too.
>
> Here's reason i guess
>
> 1. Late responding (first review has been made after 3 months)
> 2. Lack of help on CI fail (Amos keep complained about CI fail)
>
> I think both 1 and 2 can be improved by the solution i suggested in section
> A.
>
> Amos, if you think there're more reasons, please feel free to say and let
> me improve. What is the history you're mentioning?
>
> Best,
> moon
>
>
> On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org> wrote:
>
> > Just pushing discussion back on the list
> >
> > On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com> wrote:
> >
> > > Alex — if you genuinely do not know the history of this, then I will
> fill
> > > you in.
> > >
> > > lmk…
> > >
> > >
> > > --
> > > Amos Elberg
> > > Sent with Airmail
> > >
> > > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > > Date: December 1, 2015 at 6:14:20 AM
> > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org
> > >
> > > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
> > > <am...@gmail.com> <am...@gmail.com>
> > >
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > > pull request: R Interpreter for Zeppelin
> > >
> > > @Amos, we had plenty of cases of CI failing and always the
> pre-condition
> > > for a merge was a green CI. Sometimes that requires time, polite
> > > collaboration, extra mile in direct asking for help from more
> experienced
> > > members and fixes in different places, which indeed might take time, as
> > > everyone is busy.
> > >
> > >
> >
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Moon — I think there is a misunderstanding about the topic of the discussion. 

The PR has its own userbase.  It has been and is being presented at user groups.  Its been blogged and tweeted about (none of that came from me!)  The features are the subject of two jiras and on the Zeppelin roadmap.  So, the discussion isn't about whether the PR is “good."

But no-one responded to the PR until users began to tweet publicly @nflabs asking why the PR had not been adopted, and e-mailing you directly.  This looks really bad, especially when the project is considering applying to leave incubation.  

The question here is what, if anything, prevents us from letting bygones be bygones and moving forward with this now?

Claims about CI issues, or licenses, or the PR shouldn’t have been rebased (!?!) — well, they don’t really make sense.  

I keep offering to begin coordinating to integrate the PR with Zeppelin’s CI and build system.  

But the answer (except from Roman) is still “nah, let us know if you figure it out.”




Regarding the history:

Konstantin wisely started this thread by saying let’s keep the history out of the discussion.  I am respecting that. 

If the PR becomes part of Zeppelin, its going to need to be maintained, which means that we are going to need to be able to work together. 

I have been persuaded to give Moon the benefit of the doubt regarding certain issues.  He certainly knows what my view of the history is. 

If anyone else would like to know, I am happy to share it with them off-list. 


From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 2, 2015 at 7:45:11 AM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

Thanks Roman and Eran for the feedback.  

*A. About contribution impasse in general*  

I think i summarized why it happens and how it can be improved. ie.  

1. Large code base change  
2. Communication lost  
3. Opinion diverges  

And my solution was  

Guide to ping other committer when a committer is not responding, divide  
contribution into small peaces if possible. And committer pay more  
attention to the contribution.  

I'd like to hear and learn any more idea to improve.  


*B. About contribution impasses in R interpreter for Zeppelin*  

Although I'was the first one who reviewed and commented this contribution  
among the committer, I feel contributor (Amos) is unhappy about the review.  

I want to analyze the reasons and improve this, too.  

Here's reason i guess  

1. Late responding (first review has been made after 3 months)  
2. Lack of help on CI fail (Amos keep complained about CI fail)  

I think both 1 and 2 can be improved by the solution i suggested in section  
A.  

Amos, if you think there're more reasons, please feel free to say and let  
me improve. What is the history you're mentioning?  

Best,  
moon  


On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org> wrote:  

> Just pushing discussion back on the list  
>  
> On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com> wrote:  
>  
> > Alex — if you genuinely do not know the history of this, then I will fill  
> > you in.  
> >  
> > lmk…  
> >  
> >  
> > --  
> > Amos Elberg  
> > Sent with Airmail  
> >  
> > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>  
> > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>  
> > Date: December 1, 2015 at 6:14:20 AM  
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org  
> >  
> > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg  
> > <am...@gmail.com> <am...@gmail.com>  
> >  
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin  
> > pull request: R Interpreter for Zeppelin  
> >  
> > @Amos, we had plenty of cases of CI failing and always the pre-condition  
> > for a merge was a green CI. Sometimes that requires time, polite  
> > collaboration, extra mile in direct asking for help from more experienced  
> > members and fixes in different places, which indeed might take time, as  
> > everyone is busy.  
> >  
> >  
>  

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Thanks Roman and Eran for the feedback.

*A. About contribution impasse in general*

I think i summarized why it happens and how it can be improved. ie.

1. Large code base change
2. Communication lost
3. Opinion diverges

And my solution was

Guide to ping other committer when a committer is not responding, divide
contribution into small peaces if possible. And committer pay more
attention to the contribution.

I'd like to hear and learn any more idea to improve.


*B. About contribution impasses in R interpreter for Zeppelin*

Although I'was the first one who reviewed and commented this contribution
among the committer, I feel contributor (Amos) is unhappy about the review.

I want to analyze the reasons and improve this, too.

Here's reason i guess

1. Late responding (first review has been made after 3 months)
2. Lack of help on CI fail (Amos keep complained about CI fail)

I think both 1 and 2 can be improved by the solution i suggested in section
A.

Amos, if you think there're more reasons, please feel free to say and let
me improve. What is the history you're mentioning?

Best,
moon


On Wed, Dec 2, 2015 at 5:44 PM Alexander Bezzubov <bz...@apache.org> wrote:

> Just pushing discussion back on the list
>
> On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com> wrote:
>
> > Alex — if you genuinely do not know the history of this, then I will fill
> > you in.
> >
> > lmk…
> >
> >
> > --
> > Amos Elberg
> > Sent with Airmail
> >
> > From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> > Date: December 1, 2015 at 6:14:20 AM
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
> > <am...@gmail.com> <am...@gmail.com>
> >
> > Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > pull request: R Interpreter for Zeppelin
> >
> > @Amos, we had plenty of cases of CI failing and always the pre-condition
> > for a merge was a green CI. Sometimes that requires time, polite
> > collaboration, extra mile in direct asking for help from more experienced
> > members and fixes in different places, which indeed might take time, as
> > everyone is busy.
> >
> >
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Alexander Bezzubov <bz...@apache.org>]

Just pushing discussion back on the list

On Wed, Dec 2, 2015, 01:14 Amos B. Elberg <am...@gmail.com> wrote:

> Alex — if you genuinely do not know the history of this, then I will fill
> you in.
>
> lmk…
>
>
> --
> Amos Elberg
> Sent with Airmail
>
> From: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> Reply: Alexander Bezzubov <bz...@apache.org> <bz...@apache.org>
> Date: December 1, 2015 at 6:14:20 AM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> <de...@zeppelin.incubator.apache.org>, Amos B. Elberg
> <am...@gmail.com> <am...@gmail.com>
>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull request: R Interpreter for Zeppelin
>
> @Amos, we had plenty of cases of CI failing and always the pre-condition
> for a merge was a green CI. Sometimes that requires time, polite
> collaboration, extra mile in direct asking for help from more experienced
> members and fixes in different places, which indeed might take time, as
> everyone is busy.
>
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Alexander Bezzubov <bz...@apache.org>]

>From what I see - there is nothing special from _technical_ standpoint with
this contribution - there was an effort put in helping and reviewing it by
multiple project members.

There are other contributions, that Moon has carefully linked, that also
take time to get merged and there is ongoing effort in improving that.

Failing CI and lack of confidence in licensing  sounds like very valid
reasons for current state.

@Amos, we had plenty of cases of CI failing and always the pre-condition
for a merge was a green CI. Sometimes that requires time, polite
collaboration, extra mile in direct asking for help from more experienced
members and fixes in different places, which indeed might take time, as
everyone is busy.

@Moon thank you very much for clarifying the issue and the feedback!

On Tue, Dec 1, 2015, 18:42 moon soo Lee <mo...@apache.org> wrote:

> On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
> wrote:
>
> > Thank you, Cos.
> >
> > I’d like to briefly address the issues raised by Moon:
> >
> > 1. This PR does not passes CI
> >
> > The CI fails on core Zeppelin, *not* code in this PR.
> >
> > I’ve been seeking assistance on this since August.
> >
> > The most common reason is that SparkInterpreter is unable to launch Spark
> > and open a Spark Backend.  This is necessary to test the PR.
> >
> > 60+ hours, has been spent adapting and re-basing when the
> SparkInterpreter
> > architecture changed and broke the PR’s *tests.*
> >
>
> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> core, why do you think other PRs are passing CI?
>
> And let's say Zeppelin core has problem and that makes your PR fails on CI
> test. That's possible. But it still does not mean we can merge it with CI
> failing.
>
> If you think it's problem on Zeppelin core, then file an issue that
> reproduce the problem on Zeppelin core, that might be more efficient than
> keep trying yourself.
>
>
> 2. Not 100% sure this PR has no license issue. (about KniteR)
> >
> > What license problem *specifically* do you believe may exist?
> >
> > In preparing the PR, I:
> >
> > * Reviewed the Apache policy in detail.
> >
> > * Contacted the FSF to ask their interpretation of the “linking”
> > provisions of the GPL license.
> >
> > * Reviewed how other Apache software deals with this issue (e.g., Spark
> > talks to R, which is GPL'd).
> >
> > * No necessary *dependencies* of the PR have license conflicts.  In
> > several cases, I contacted package authors who agreed to re-issue their
> > packages under Apache-compatible licenses.  (Usually I had to do a bit of
> > coding in exchange…)
> >
> > * Where the license had to stay GPL, the packages are *not necessary* and
> > *not dependencies.*  If the optional packages are present, they will be
> > used to enable additional functionality.  Knitr is an example.  The PR
> will
> > compile and run fine without knitr.  If knitr is available (it is part of
> > the most common R distribution), the PR will enable the knitr
> interpreter.
> > * This is exactly how this issue is addressed through the Apache
> > ecosystem.
> >
> > The tl;dr is this:   When Apache code is written to talk to libraries
> that
> > may or may not be present on the user’s system, or where it talks to an
> API
> > but is agnostic about implementation, that is not “linking” in a way that
> > implicate the anti-linking provision of the GPL.
> >
> > Otherwise, no Apache code could ever call a shell script!  Let alone run
> > on Linux, or talk to R.
> >
>
> I'm not a legal expert. So following could be wrong.
>
> In my interpretation, KnitRInterpreter is not an optional feature while it
> is always enabled when compiling Zeppelin and always enabled when running
> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
> it will fail when no KnitR is installed on the system)
>
> And of course, no Apache code can ever call a shell script, on the purpose
> of dynamic linking with GPL library.
>
> I was guessing SparkR can be still in Apache License even if it is depends
> on R. Because of GPL licensed compiler generated output is not GPL license.
> and R is sort of compiler.
>
> If you can get answer from Spark community how SparkR get managed to stay
> in Apache License while R is GPL, the answer might help.
>
>
> 3. Need more time to review.
> >
> > Has any reviewer has downloaded the PR or run the demo notebook?  (Which
> > is there for the benefit of reviewers, and isn’t intended to go in final
> > distribution.)
> >
> > How many +1 comments from users would you like to see?
> >
> > How much time do you believe is required?
> >
>
> It all depends on when CI is going to pass, when license problem is going
> to be cleared, and when a committer willing to review and responsible to
> commit your contribution.
>
>
> 1. Large code base change
> >
> > Large code base changes require coordination and cooperation.  This PR
> > necessarily implicates the build scripts, testing code, the
> > SparkInterpreter, etc.
> >
> > I have been seeking to coordinate since August.
> >
> > I continue to stand ready to do so.
> >
> > -Amos
> >
>
> If i give you one suggestion, Zeppelin committers sometimes ask rebase the
> contribution branch for some reason. It is not the really the best
> practice, but still okay while most contributions are not including large
> code base changes.
>
> However, your one, has more than 4000 lines of code change. If you rebase
>  then review should be started from the beginning, again. So you might want
> to minimize the rebase your branch.
>
> Thanks,
> moon
>
>
>
> > From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org
> > <de...@zeppelin.incubator.apache.org> <de...@zeppelin.incubator.apache.org>
> > Date: December 1, 2015 at 1:34:19 AM
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > <de...@zeppelin.incubator.apache.org>
> > Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > pull request: R Interpreter for Zeppelin
> >
> > Hi Cos,
> >
> > Thanks for opening a discussion.
> > My answer about 'Why this PR is open for three months' is
> >
> > 1. This PR does not passes CI
> > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > 3. Need more time to review.
> >
> > It's my personal answer, other committers may have different opinion.
> >
> >
> > I think the question should be generalized. Because this PR is not the
> > only
> > PR that is in impasse. There're more. For example
> >
> > Here's some examples that PR is not been merged.
> > https://github.com/apache/incubator-zeppelin/pull/53,
> > https://github.com/apache/incubator-zeppelin/pull/60
> > and so on.
> >
> > I can categorize the cases, based on experience of involving Zeppelin
> > community from the beginning.
> >
> > 1. Large code base change
> >
> > When contribution has large code base changes, it tend to take more time
> > to
> > review and merged. Normally, most contributions merged in 1day~1 week.
> > But some contribution has large code base changes take 2~4 weeks. Few
> > contribution that has very large code base change take months.
> >
> > 2. Communication lost
> >
> > Sometimes, committer is not responding, or contributor is not responding.
> >
> > 3. Opinion diverges
> >
> > For some changes, included in contribution. When people have different
> > opinion and it continue to diverges, those PRs are not been merged.
> >
> >
> > I think having a guide such as ping other committer when a committer is
> > not
> > responding, and divide contribution into small peaces if possible, would
> > help most of the cases.
> > Of course committer have to pay attention more to the contribution and
> > helping people. That's the first one.
> >
> > Thanks,
> > moon
> >
> > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org>
> wrote:
> >
> > > To make sure we're on the same page, here are two threads that I found
> > > related
> > > to this topic.
> > >
> > > Thread 1:
> > > Subject: R?
> > > Started on: July 1, 2015
> > >
> > > Thread 2:
> > > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > > Zeppelin
> > > Started on: August 13, 2015
> > >
> > > If you want to fetch these from our archive send emails to
> > > dev-thread.1735@zeppelin.incubator.apache.org
> > > dev-thread.3573@zeppelin.incubator.apache.org
> > >
> > > Cos
> > >
> > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > > Guys,
> > > >
> > > > While catching up on my emails from the last a couple of weeks, this
> > > thread
> > > > caught my attention. I am not normally paying much attention to the
> > code
> > > > reviews traffic from GH, but this one is pretty different as it spans
> > > three
> > > > months and counting.
> > > >
> > > > First, here are my five cents:
> > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > > contributed to
> > > > an ASF project this file should simply contain ASL2 text, like in [1]
> > > > - r/pom.xml perhaps shouldn't contain a separate <developers>
> section,
> > > but
> > > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > > > maintain this in the source code, but we have the list of all the
> > > > committers on the project's site [2] Every new committer's first
> > > commit is
> > > > to update the team page ;)
> > > > - comments like in
> > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > >
> > > > +/**
> > > > + * Created by aelberg on 7/28/15.
> > > > + */
> > > >
> > > > is better to be removed. It has been already discussed here [3]. And
> > > the
> > > > initial discussion on the topic [4] was linked as well
> > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> > > R-specific
> > > > stuff - I have no idea about R, honestly, but
> > > >
> > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > ...
> > > > +Author: David B. Dahl
> > > >
> > > > shouldn't be here, IMO. Normally, if some additional licenses are
> > > used,
> > > > they have to be listed in the top-level NOTICE file (already there).
> > > >
> > > > - I am not going to offer any comments on the technical merits of the
> > > patch,
> > > > as I haven't tried it personally. However, I don't see any serious
> > > > technical objections to the functionality in question.
> > > >
> > > > So, the question is - why the PR is open for three months? I hasn't
> > been
> > > able
> > > > to get a clear answer. What I found out though is pretty unsettling,
> > > really.
> > > > The communication on the topic is almost non-existent, except for
> this
> > > sparse
> > > > and bitter thread in the GH.
> > > >
> > > > I would love to hear from the committers what's stopping the
> > acceptance
> > > of the
> > > > code, besides of the minor issues I've mentioned earlier? What are
> the
> > > reasons for it?
> > > > Is there anything wrong with it?
> > > > One of the responsibilities of the committers is to make sure the
> > > > contributions are reviewed; new contributors are guided and do
> > > understand how
> > > > the project ticks. The easy feedback flow attracts new people,
> > allowing
> > > the
> > > > community to grow and thrive.
> > > >
> > > > I am asking _explicitely_ not to re-start the bickering I have
> already
> > > > seen. At this point I am interested in the purely technical side of
> > this.
> > > >
> > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > > [2] http://bigtop.apache.org/team-list.html
> > > > [3]
> > >
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > [4] http://s.apache.org/iZl
> > > >
> > > > With regards,
> > > > Cos
> > > >
> > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > Github user elbamos commented on the pull request:
> > > > >
> > > > >
> > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > >
> > > > > The current push should resolve some issues with changes in the
> > > > > Spark-Zeppelin interface that had created issues for users, as
> > > well as
> > > > > support for 1.5.1.
> > > > >
> > > > > Knitr support is improved, and the reason for a separate knitr
> > > interpreter may be clearer now.
> > > > >
> > > > > The amount of code borrowed from rscala is reduced.
> > > > >
> > > > > I did not address issues with @author tags, or files under the R/
> > > > > folder. The reason is, to be blunt, I don't understand what the
> > > precise
> > > > > concerns actually are.
> > > > >
> > > > > Please note that I changed .travis.yml to only use spark 1.4 and
> > > later.
> > > > > I'm sure there is a better way to do it, but I'm just not in a
> > > position
> > > > > to try to figure out the entire Zeppelin build process.
> > > > >
> > > > > Integrating this with the rest of zeppelin is going to take some
> > > work
> > > > > regarding pom's, travis, and so forth. I can do a lot of that,
> > > but I'm
> > > > > going to need to discuss it with the people who have been "owning"
> > > those
> > > > > files. There are just too many moving pieces here.
> > > > >
> > > > > Because of the size of this (which is, unfortunately, necessary),
> > > > > posting here is probably not the most efficient way. That is also
> > > true
> > > > > because certain people chose to use this PR as a forum to air other
> > > > > issues. Therefore, it would be better if reviewers e-mail me
> > > directly.
> > >
> > >
> > >
> >
> >
>

Re: R-Interpreter CI Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Yeah that kind of error is basically what I was seeing. 

I copied that line of travis.yml from something I found online as an example of how to add R to a java-based travis build.

How can I help?

From: moon soo Lee <mo...@apache.org>
Reply: moon soo Lee <mo...@apache.org>
Date: December 26, 2015 at 10:28:34 PM
To: Amos B. Elberg <am...@gmail.com>, dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: R-Interpreter CI Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

Hi,

I did couple of test against rinterpreter pr [1]. and end up with exception 


15/12/25 02:25:54 INFO AppClient$ClientEndpoint: Connecting to master spark://testing-gce-28dccb9f-5570-4e9c-b911-a5974c99c02f.c.eco-emissary-99515.internal:7071...
15/12/25 02:26:14 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[appclient-registration-retry-thread,5,main]
java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5081a4bb rejected from java.util.concurrent.ThreadPoolExecutor@6f661a47[Running, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 1]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
        at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
        at org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anonfun$tryRegisterAllMasters$1.apply(AppClient.scala:96)
        at org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anonfun$tryRegisterAllMasters$1.apply(AppClient.scala:95)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
        at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
        at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
        at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
        at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:108)
        at org.apache.spark.deploy.client.AppClient$ClientEndpoint.tryRegisterAllMasters(AppClient.scala:95)
        at org.apache.spark.deploy.client.AppClient$ClientEndpoint.org$apache$spark$deploy$client$AppClient$ClientEndpoint$$registerWithMaster(AppClient.scala:121)
        at org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2$$anonfun$run$1.apply$mcV$sp(AppClient.scala:132)
        at org.apache.spark.util.Utils$.tryOrExit(Utils.scala:1119)
        at org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2.run(AppClient.scala:124)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

Strange thing is, above test is not related with R interpreter but it happens 
when .travis.yml has following entry in before_intall: section

 echo "deb http://cran.rstudio.com/bin/linux/ubuntu precise/" | sudo tee -a /etc/apt/sources.list


Following is build log (success) without above line in .travis.yml
https://travis-ci.org/Leemoonsoo/incubator-zeppelin/builds/98927745
and code
https://github.com/Leemoonsoo/incubator-zeppelin/tree/71791ee0362d580f670ea037f5af2c780adc2828

Following is build log (fail) with above line in .travis.yml
https://travis-ci.org/Leemoonsoo/incubator-zeppelin/builds/98927745
and code
https://github.com/Leemoonsoo/incubator-zeppelin/tree/650dfa8c3477df722fc4381af62c92b5eeb84403


This is really weird. Let me update more when i get better understanding of it.

Thanks,
moon

[1] https://github.com/apache/incubator-zeppelin/pull/208



On Tue, Dec 8, 2015 at 10:57 AM Amos B. Elberg <am...@gmail.com> wrote:
Thanks, Moon.  

If you look at the code now, the reason travis fails is because the travis.yaml is messed-up from me trying to get travis not to fail.  But with that fixed it only fails for other reasons…

I have seen both the failures you describe, but I’ve also seen a bunch of others.  The most common issue is that the PR, to test, needs to instantiate a SparkInterpreter instance without the infrastructure of the Zeppelin server, and Spark fails to start.

As a practical way to move forward I propose the following:

1.  The CI issues are broader than this PR, and we should start a separate thread to discuss and fix CI in general. 

2.  I’m going to close the current PR, and make a new PR that is based on the 0.5.5 release code.  This PR will *not* be sync’d with Zeppelin-Master.  It will be sync’d w/ 0.5.5 release because that is a stable base.  I would appreciate if people could help me with CI, using that as a base. 

-- 
Amos Elberg
Sent with Airmail

From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 8, 2015 at 9:26:38 AM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: R-Interpreter CI Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

fyi, some cases of CI failing, when PR does not impact the code,

1) Flickering test. Some tests are passing sometimes, failing the other
times. (e.g. ZEPPELIN-476)
2) Download fails. Downloading maven artifacts or other requirements from
internet are sometimes failing

both cases pass if CI triggered again.

Thanks,
moon

On Tue, Dec 8, 2015 at 6:29 PM DuyHai Doan <do...@gmail.com> wrote:

> I have also experienced CI failing in the pass for some PRs that do not
> impact the code (Cassandra documentation). I guess that during peak hours,
> the CI servers may be too busy or enter a dead lock so the build fails.
>
> At least that's my guess. What do you think ?
>
> On Tue, Dec 8, 2015 at 9:42 AM, moon soo Lee <mo...@apache.org> wrote:
>
> > Let me run some CI test with your branch and share result here.
> > Hope i can narrow down the cause and that helps involvement of more
> people.
> >
> > Thanks,
> > moon
> >
> > On Tue, Dec 8, 2015 at 3:08 PM Amos B. Elberg <am...@gmail.com>
> > wrote:
> >
> > > Is there anyone who can work with me on the CI issues? It looks like
> > > there are a number of PRs experiencing similar things.
> > >
> > > I think we should make getting CI stable to be a priority. Because it
> > > will save everyone a lot of frustration and aggravation if CI works
> > > reliably.
> > >
> > > Is there anyone other than Jongyoul and Moon who knows the CI/Build
> > > process well?
> > >
> > > (Moon - Thank you for taking another look at the licensing issue. Per
> > the
> > > e-mail I wrote about this a few days ago, I don’t feel I have more to
> > > contribute to the licensing discussion, so I’m going to try not to
> > comment
> > > further about it.)
> > >
> > > From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> > > Reply: dev@zeppelin.incubator.apache.org
> > > <de...@zeppelin.incubator.apache.org> <dev@zeppelin.incubator.apache.org
> >
> > > Date: December 7, 2015 at 5:00:08 PM
> > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org
> > >
> > > <de...@zeppelin.incubator.apache.org>
> > > Subject: Re: License of KnitRInterpreter Was: Re: contributions
> impasse.
> > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for
> Zeppelin
> > >
> > > Thanks Amos, Roman, Cos for clarifying license issue.
> > >
> > > I'm convinced that this license issue will not be a blocker.
> > >
> > > In my understanding, these are good sign,
> > >
> > > 1. any gpl licensed source codes are not included in the source package
> > > 2. any gpl licensed libraries are not included in the binary package
> > >
> > > However, i can not still 100% sure about
> > >
> > > 3. any gpl licensed libraries are not linked on runtime
> > >
> > > Even after Amos's explanation. I still think using 'knitr' is one of
> the
> > > clear case that 'knir' is linked to 'R' according to
> > > http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL.
> > >
> > > Giving input and getting output from GPL licensed interpreter (includes
> > R)
> > > from Apache licensed software is not a problem. That's not the point.
> > > Let me say in this way. There's an java code,
> > >
> > > import com.mysql.jdbc.Driver
> > > Driver driver = new Driver()
> > >
> > > Say without this java code, one of important feature of Zeppelin does
> not
> > > work. And Zeppelin does not includes GPL licensed file in the source
> > > package, GPL licensed library in the binary package, but it requires
> GPL
> > > licensed library on the runtime.
> > > In this case, will this java code be a license problem or not?
> > >
> > > In other words, my question is
> > >
> > > a) Is runtime GPL library dependency allowed in ASF release?
> > > b) is 'knitr' considered as runtime dependency?
> > >
> > > If someone can clarify a), b), then it would be extremely helpful
> > > understanding this case, and possible similar cases, too.
> > >
> > > Thanks,
> > > moon
> > >
> > > On Mon, Dec 7, 2015 at 3:20 PM Konstantin Boudnik <co...@apache.org>
> > wrote:
> > >
> > > > On Thu, Dec 03, 2015 at 11:03AM, Corneau Damien wrote:
> > > > > Thanks Cos for those answers,
> > > > >
> > > > > If I'm right you are advising to have a default build that doesn't
> > > > include
> > > > > libraries with conflicting licenses, but have an option to include
> > > them
> > > > for
> > > > > users who wants to build the project themselves.
> > > >
> > > > Yes, that's what I said. Besides, looks like Roman provided the
> second
> > > > pair of
> > > > eyes to this licensing discussion and as well didn't find any issues
> > > with
> > > > the
> > > > current approach.
> > > >
> > > > Cos
> > > >
> > > > > To refer to another thread about decentralizing interpreters, it
> > could
> > > > even
> > > > > be better in that case to have some interpreters separated from the
> > > tree,
> > > > > and easily pluggable with a release instead of forcing users to
> build
> > > the
> > > > > project to use them.
> > > > >
> > > > >
> > > > > On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <cos@apache.org
> >
> > > > wrote:
> > > > >
> > > > > > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:
> > > > > > > Konstantin thank you for getting into this.
> > > > > > >
> > > > > > >> The best way to go around it is by
> > > > > > >> providing a build-time option that will pull such binaries in.
> > > But
> > > > by
> > > > > > default
> > > > > > >> such libs shouldn't be pulled.
> > > > > > >
> > > > > > > That is basically how the PR handles this. If the GPL’d
> > > interpreter
> > > > > > scripts
> > > > > > > are missing, there’s no indication at all at build time. It
> > > doesn’t
> > > > try
> > > > > > to
> > > > > > > download them.
> > > > > > >
> > > > > > > At runtime, if the user tries to use functionality that would
> > need
> > > > such a
> > > > > > > script (i.e., if they type “knitr” to use knitr), we display an
> > > error
> > > > > > that
> > > > > > > says that the functionality is not there because the library is
> > > > missing,
> > > > > > and
> > > > > > > that the library cannot be provided because it has an
> > incompatible
> > > > > > license,
> > > > > > > but the user can download it themselves if they want.
> > > > > > >
> > > > > > > And, in the log, if the logging level is high, they will see a
> > > note
> > > > that
> > > > > > > some functionality was disabled because the libraries weren’t
> > > there.
> > > > > > >
> > > > > > > To be clear, though, none of these libraries are binaries.
> > They’re
> > > > all
> > > > > > interpreter scripts.
> > > > > >
> > > > > > If you ever in a doubt of which licenses could be used for
> > > dependncies
> > > > > > (not to
> > > > > > say about source code) are listed in Category A list of [1]
> > > > > >
> > > > > > A lot of quesitons discussed here are already covered in the
> legal
> > > > FAQ, so
> > > > > > just check against it if you have any questions.
> > > > > >
> > > > > > [1] http://www.apache.org/legal/resolved.html#category-a
> > > > > >
> > > > > > Cos
> > > > > >
> > > > > > > From: Konstantin Boudnik <co...@apache.org>
> > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>,
> > > dev@zeppelin.incubator.apache.org
> > > > <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > Date: December 2, 2015 at 3:24:50 PM
> > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > dev@zeppelin.incubator.apache.org
> > > > > > >
> > > > > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions
> > > > > > impasse. Was: [GitHub] incubator-zeppelin pull request: R
> > > Interpreter
> > > > for
> > > > > > Zeppelin
> > > > > > >
> > > > > > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:
> > > > > > > > I think that what moon means is that:
> > > > > > > > If we merge the way it is now, the KnitRInterpreter will be
> > part
> > > > of the
> > > > > > > > code base, so it isn't optional at code base level.
> > > > > > > >
> > > > > > > > To make it even more simple:
> > > > > > > > * If the code has the right licensing -> that code can be
> part
> > > of
> > > > the
> > > > > > > > source code, and can be including in a binary release
> > > > > > >
> > > > > > > We aren't concerned with binary releases - as an Apache
> community
> > > we
> > > > are
> > > > > > > voting and releasing source code. If the project wants to
> provide
> > > a
> > > > > > binary
> > > > > > > release to its users, they are better be warned about inclusion
> > of
> > > > non
> > > > > > > ASL2-friendly licensed bits.
> > > > > > >
> > > > > > > > * If the code doesn't have the right licensing -> it can't be
> > > part
> > > > of
> > > > > > the
> > > > > > > > source code, and can't be included in a binary release
> > > > > > >
> > > > > > > See above.
> > > > > > >
> > > > > > > > * If the code doesn't have the right licensing but is
> imported
> > > at
> > > > build
> > > > > > > > time (libraries for example) -> it is not in the source code,
> > it
> > > > can't
> > > > > > be
> > > > > > > > included in binary release
> > > > > > >
> > > > > > > That is unless a user doing it on his own. The best way to go
> > > around
> > > > it
> > > > > > is by
> > > > > > > providing a build-time option that will pull such binaries in.
> > But
> > > by
> > > > > > default
> > > > > > > such libs shouldn't be pulled.
> > > > > > >
> > > > > > > Cos
> > > > > > >
> > > > > > > > So in the case of licensing issues, it would need to be fully
> > > > optional
> > > > > > > > (user bring the interpreter in his directory and build
> Zeppelin
> > > > with
> > > > > > it)
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <
> > > > amos.elberg@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Moon let me clarify:
> > > > > > > > >
> > > > > > > > > Interpreted code doesn’t “link.” The wiki article actually
> > > > explains
> > > > > > it
> > > > > > > > > pretty well —
> > > > https://en.wikipedia.org/wiki/GPL_linking_exception
> > > > > > > > > “Linking” against a library means compiling its headers
> into
> > a
> > > > > > binary, the
> > > > > > > > > way a C compiler works. The 2008 e-mail Moon distributed,
> > > called
> > > > > > this the
> > > > > > > > > “interpreter exception.”
> > > > > > > > >
> > > > > > > > > As for whether GPL’d code is a “mandatory dependency,” if
> > > knitr
> > > > is
> > > > > > missing
> > > > > > > > > the PR will compile, run and test just fine. The user is
> > never
> > > > > > prompted to
> > > > > > > > > download it. The only effect is, if the user types “knitr”
> > and
> > > > knitr
> > > > > > isn’t
> > > > > > > > > there, we display an InterpreterError that knitr isn’t
> there.
> > > > > > > > >
> > > > > > > > > KnitRInterpreter is not optionally required. so it does not
> > > > matter
> > > > > > KnitR
> > > > > > > > > is
> > > > > > > > > optionally required or not.
> > > > > > > > > Aren’t all interpreters optional? You can turn them on and
> > off
> > > > in the
> > > > > > > > > config files.
> > > > > > > > >
> > > > > > > > > Do you mean that the KnitRInterpreter class gets compiled
> to
> > > > > > bytecode even
> > > > > > > > > if knitr is missing? So what? That isn't a mandatory
> > > dependency
> > > > or a
> > > > > > link.
> > > > > > > > >
> > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > Date: December 2, 2015 at 3:18:00 AM
> > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > Subject: Re: License of KnitRInterpreter Was: Re:
> > > contributions
> > > > > > impasse.
> > > > > > > > > Was: [GitHub] incubator-zeppelin pull request: R
> Interpreter
> > > for
> > > > > > Zeppelin
> > > > > > > > >
> > > > > > > > > Let me summarize license concern about KnitRInterpreter.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Amos's interpretation.
> > > > > > > > >
> > > > > > > > > KnitR is optionally required by KnitRInterpreter.
> > > > > > > > > R dependency in SparkR has no problem. So KnitR should be
> the
> > > > same.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Moon's interpretation.
> > > > > > > > >
> > > > > > > > > KnitRInterpreter is not optionally required. so it does not
> > > > matter
> > > > > > KnitR is
> > > > > > > > > optionally required or not.
> > > > > > > > > R dependency in SparkR is exception of GPL. KnitR is not
> > > applied
> > > > that
> > > > > > > > > exception.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Amos, could you confirm my understanding to your
> > > interpretation
> > > > is
> > > > > > correct?
> > > > > > > > > If it is not could you clarify it?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > moon
> > > > > > > > >
> > > > > > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Just to put the final nail in this, I looked it up.
> > > > > > > > > >
> > > > > > > > > > I see no evidence of any “compiler exception.”
> > > > > > > > > >
> > > > > > > > > > There is an exception in the license for the runtime
> > > libraries
> > > > > > that are
> > > > > > > > > > bundled with GCC. See:
> > > > > > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> > > > > > > > > >
> > > > > > > > > > But no “compiler exception.”
> > > > > > > > > >
> > > > > > > > > > In fact, I believe this is part of the reason why LLVM
> was
> > > > created.
> > > > > > > > > >
> > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Date: December 1, 2015 at 8:16:36 PM
> > > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin pull
> > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > >
> > > > > > > > > > Is knitR is commonly considered as a
> interpreter/compiler?
> > > or
> > > > is it
> > > > > > > > > > considered as a library routine?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > moon
> > > > > > > > > >
> > > > > > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > Moon - you give this as an explanation of the licensing
> > > issue:
> > > > > > > > > >
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > > > >
> > > > > > > > > > According to that, there is an exception in the GPL for
> > > > interpreter
> > > > > > > > > > languages. As long as you don’t distribute the code, its
> > > fine
> > > > to
> > > > > > talk to
> > > > > > > > > > an interpreted language.
> > > > > > > > > >
> > > > > > > > > > Well, if that’s the case, then the PR plainly does not
> have
> > > a
> > > > > > license
> > > > > > > > > > issue. It doesn’t distribute any GPL’d R code.
> > > > > > > > > >
> > > > > > > > > > I’m not sure what’s confusing about this. It seems
> > > completely
> > > > > > > > > > straightforward.
> > > > > > > > > >
> > > > > > > > > > Regarding this:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Amos Elberg
> > > > > > > > > > Sent with Airmail
> > > > > > > > > >
> > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > Date: December 1, 2015 at 6:48:47 PM
> > > > > > > > > >
> > > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org
> > > > > > > > > >
> > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin pull
> > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > >
> > > > > > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I am going to try to minimize my reaction to Moon’s
> > > e-mail.
> > > > > > > > > > >
> > > > > > > > > > > The tl;dr is this:
> > > > > > > > > > >
> > > > > > > > > > > The reason we are having this discussion now is that
> > > active
> > > > > > users of
> > > > > > > > > the
> > > > > > > > > > > PR — which now has its own user base — went public to
> > > > complain
> > > > > > about
> > > > > > > > > > this.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > The PR has been tested by an active user base for more
> > > than
> > > > three
> > > > > > > > > months.
> > > > > > > > > > > No-one has been able to identify any specific actual
> > > > licensing
> > > > > > problem,
> > > > > > > > > > and
> > > > > > > > > > > the PR was prepared based on an extensive, careful
> review
> > > of
> > > > the
> > > > > > > > > relevant
> > > > > > > > > > > licensing issues and after contacting the relevant
> > people.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I admire every software that used by user and helping
> > > people.
> > > > That
> > > > > > > > > includes
> > > > > > > > > > your work. But that's not the topic we're in discussion.
> > > Active
> > > > > > user does
> > > > > > > > > > not mean your contribution can ignore the review.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > It is not an explanation for someone who has been
> > ignoring
> > > my
> > > > > > “how can
> > > > > > > > > I
> > > > > > > > > > > move this forward…” emails for three months to point
> the
> > > > finger
> > > > > > and
> > > > > > > > > say I
> > > > > > > > > > > didn’t contact the right person or file the right
> report.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > This is also not the topic in this discussion.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > The burden for providing an explanation for the
> inaction
> > > is
> > > > on
> > > > > > the PMCC
> > > > > > > > > > at
> > > > > > > > > > > this point.
> > > > > > > > > >
> > > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's
> > problem
> > > on
> > > > > > Zeppelin
> > > > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > > > > They’re not! I often see comments on PRs to just ignore
> > > that
> > > > CI
> > > > > > is
> > > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > One of the most common reasons this PR fails CI, is CI
> > > > times-out
> > > > > > > > > > > downloading Spark to install. How could that possibly
> be
> > > > caused
> > > > > > by the
> > > > > > > > > > PR?
> > > > > > > > > > >
> > > > > > > > > > > It looks to me like the only PRs with changes to the
> > > relevant
> > > > > > parts of
> > > > > > > > > > the
> > > > > > > > > > > code — the SparkInterpreter — are being made by the
> > person
> > > > who
> > > > > > wrote
> > > > > > > > > the
> > > > > > > > > > > testing suite.
> > > > > > > > > > >
> > > > > > > > > > > So, that would explain why some other PRs pass CI:
> > Neither
> > > > the
> > > > > > > > > > > SparkInterpreter nor the testing suite are stable or
> > > robust,
> > > > but
> > > > > > since
> > > > > > > > > > the
> > > > > > > > > > > PRs are coming from the person who wrote both…
> > > > > > > > > > >
> > > > > > > > > > > And let's say Zeppelin core has problem and that makes
> > > your
> > > > PR
> > > > > > fails on
> > > > > > > > > > CI
> > > > > > > > > > > test. That's possible. But it still does not mean we
> can
> > > > merge
> > > > > > it with
> > > > > > > > > CI
> > > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > It means you should be working with me to figure out
> why
> > > the
> > > > CI
> > > > > > is
> > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > This PR has been tested by an active user base for the
> > > past
> > > > three
> > > > > > > > > months.
> > > > > > > > > > > If CI is continuing to fail, and dozens of hours of
> > effort
> > > > have
> > > > > > not
> > > > > > > > > > > resolved the CI issues, then it is time to start
> > > considering
> > > > > > whether
> > > > > > > > > the
> > > > > > > > > > > testing suite is part of the problem.
> > > > > > > > > > >
> > > > > > > > > > > The level of defensiveness about the CI and
> > > SparkInterpreter
> > > > is
> > > > > > not
> > > > > > > > > > > helping to resolve these issues.
> > > > > > > > > > >
> > > > > > > > > > > If you think it's problem on Zeppelin core, then file
> an
> > > > issue
> > > > > > that
> > > > > > > > > > > reproduce the problem on Zeppelin core, that might be
> > more
> > > > > > efficient
> > > > > > > > > than
> > > > > > > > > > > keep trying yourself.
> > > > > > > > > > > I contacted you numerous times about such issues...
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I remember i commented your issue about CI. but you just
> > > keep
> > > > > > repeated
> > > > > > > > > it's
> > > > > > > > > > not your problem but Zeppelin core problem.
> > > > > > > > > >
> > > > > > > > > > Then please file an issue about the problem you found in
> > > > Zeppelin
> > > > > > Core.
> > > > > > > > > > Then everyone will get into the problem.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > In my interpretation, KnitRInterpreter is not an
> optional
> > > > > > feature while
> > > > > > > > > > it
> > > > > > > > > > > is always enabled when compiling Zeppelin and always
> > > enabled
> > > > when
> > > > > > > > > running
> > > > > > > > > > > Zeppelin. And it requires dynamically linked GPL
> library
> > > on
> > > > > > runtime.
> > > > > > > > > (yes
> > > > > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > > > > >
> > > > > > > > > > > Its not always enabled.
> > > > > > > > > > > It is not dynamically linked at runtime.
> > > > > > > > > > > It will not fail when knitr is missing. If knitr is not
> > > > present,
> > > > > > the
> > > > > > > > > repl
> > > > > > > > > > > interpreter starts and a note is written to to the log
> > > that
> > > > the
> > > > > > knitr
> > > > > > > > > > > interpreter isn’t available because knitr is not
> present.
> > > > > > > > > > >
> > > > > > > > > > > no Apache code can ever call a shell script, on the
> > > purpose
> > > > of
> > > > > > dynamic
> > > > > > > > > > > linking with GPL library.
> > > > > > > > > > > You misunderstand.
> > > > > > > > > > >
> > > > > > > > > > > The *shell* is GPL'd.
> > > > > > > > > > >
> > > > > > > > > > > Is Zeppelin “linked" against the GPL’d shell because
> > > Zeppelin
> > > > > > depends
> > > > > > > > > on
> > > > > > > > > > a
> > > > > > > > > > > shell script to launch?
> > > > > > > > > > >
> > > > > > > > > > Obviously not.
> > > > > > > > > > >
> > > > > > > > > > > The interaction with R in the PR is the same.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Again, bash is one of exceptions of GPL, like other GPL
> > > > licensed
> > > > > > > > > > compiler/interpreter.
> > > > > > > > > >
> > > > > > > > > > Check here why Bash and R is okay with Apache License.
> > > > > > > > > >
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > > > >
> > > > > > > > > > I'm not sure we can apply the same exception for 'using'
> > > KnitR.
> > > > > > > > > >
> > > > > > > > > > My point is not 'KnitR' is optional or not. Point is
> > > > > > 'KnitRInterpreter'
> > > > > > > > > you
> > > > > > > > > > wrote is not an optional feature. Which is clearly not
> > > > optionally
> > > > > > enabled
> > > > > > > > > > code and feature. And that depends on KnitR library which
> > is
> > > > GPL.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > I was guessing SparkR can be still in Apache License
> even
> > > if
> > > > it
> > > > > > is
> > > > > > > > > > depends
> > > > > > > > > > > on R. Because of GPL licensed compiler generated output
> > is
> > > > not
> > > > > > GPL
> > > > > > > > > > license.
> > > > > > > > > > > and R is sort of compiler. If you can get answer from
> > > Spark
> > > > > > community
> > > > > > > > > how
> > > > > > > > > > > SparkR get managed to stay in Apache License while R is
> > > GPL,
> > > > the
> > > > > > answer
> > > > > > > > > > > might help.
> > > > > > > > > > > The description of SparkR is not accurate in any
> respect.
> > > (Do
> > > > > > you think
> > > > > > > > > > > SparkR is not talking to GPL-licensed libraries?)
> > > > > > > > > > >
> > > > > > > > > > > I don’t see that any genuine issue is being raised
> here.
> > > > > > > > > > >
> > > > > > > > > > > If there is an issue, the burden is on you to identify
> > it.
> > > > > > > > > > >
> > > > > > > > > > > If i give you one suggestion, Zeppelin committers
> > > sometimes
> > > > ask
> > > > > > rebase
> > > > > > > > > > the
> > > > > > > > > > > contribution branch for some reason. It is not the
> really
> > > the
> > > > > > best
> > > > > > > > > > > practice, but still okay while most contributions are
> not
> > > > > > including
> > > > > > > > > large
> > > > > > > > > > > code base changes
> > > > > > > > > > > However, your one, has more than 4000 lines of code
> > > change.
> > > > If
> > > > > > you
> > > > > > > > > rebase
> > > > > > > > > > > then review should be started from the beginning,
> again.
> > > So
> > > > you
> > > > > > might
> > > > > > > > > > want
> > > > > > > > > > > to minimize the rebase your branch.
> > > > > > > > > > >
> > > > > > > > > > > Are you actually complaining that the problem is that I
> > > > rebased
> > > > > > the
> > > > > > > > > code
> > > > > > > > > > > during the three-month period when no-one looked at it
> > and
> > > > > > Zeppelin
> > > > > > > > > went
> > > > > > > > > > > through a release?
> > > > > > > > > > >
> > > > > > > > > > > I cannot take it seriously when you say things like
> this.
> > > > > > > > > > >
> > > > > > > > > > > Having to “start from the beginning” cannot be a
> problem
> > > if
> > > > you
> > > > > > never
> > > > > > > > > > > actually started the first time...
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > You wanted coordination and cooperation. So i gave you
> > > > suggestion
> > > > > > that
> > > > > > > > > > helping review process. For example, your code has been
> > > rebased
> > > > > > since my
> > > > > > > > > > comment and jongyoul's comment. that means committers
> will
> > > > need to
> > > > > > look
> > > > > > > > > > from the beginning. That'll require more time. And
> maybe, i
> > > > guess
> > > > > > that's
> > > > > > > > > > not what you want. But If you don't care, feel free to
> > > rebase.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > moon
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Date: December 1, 2015 at 4:42:06 AM
> > > > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin
> > > > > > > > > pull
> > > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > Thank you, Cos.
> > > > > > > > > > >
> > > > > > > > > > > I’d like to briefly address the issues raised by Moon:
> > > > > > > > > > >
> > > > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > > > The CI fails on core Zeppelin, *not* code in this PR.
> > > > > > > > > > >
> > > > > > > > > > > I’ve been seeking assistance on this since August.
> > > > > > > > > > >
> > > > > > > > > > > The most common reason is that SparkInterpreter is
> unable
> > > to
> > > > > > launch
> > > > > > > > > Spark
> > > > > > > > > > > and open a Spark Backend. This is necessary to test the
> > > PR.
> > > > > > > > > > >
> > > > > > > > > > > 60+ hours, has been spent adapting and re-basing when
> the
> > > > > > > > > > SparkInterpreter
> > > > > > > > > > > architecture changed and broke the PR’s *tests.*
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's
> > > problem
> > > > on
> > > > > > > > > Zeppelin
> > > > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > > > >
> > > > > > > > > > > And let's say Zeppelin core has problem and that makes
> > > your
> > > > PR
> > > > > > fails on
> > > > > > > > > > CI
> > > > > > > > > > > test. That's possible. But it still does not mean we
> can
> > > > merge
> > > > > > it with
> > > > > > > > > CI
> > > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > If you think it's problem on Zeppelin core, then file
> an
> > > > issue
> > > > > > that
> > > > > > > > > > > reproduce the problem on Zeppelin core, that might be
> > more
> > > > > > efficient
> > > > > > > > > than
> > > > > > > > > > > keep trying yourself.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 2. Not 100% sure this PR has no license issue. (about
> > > KniteR)
> > > > > > > > > > > What license problem *specifically* do you believe may
> > > exist?
> > > > > > > > > > >
> > > > > > > > > > > In preparing the PR, I:
> > > > > > > > > > >
> > > > > > > > > > > * Reviewed the Apache policy in detail.
> > > > > > > > > > >
> > > > > > > > > > > * Contacted the FSF to ask their interpretation of the
> > > > “linking”
> > > > > > > > > > > provisions of the GPL license.
> > > > > > > > > > >
> > > > > > > > > > > * Reviewed how other Apache software deals with this
> > issue
> > > > > > (e.g., Spark
> > > > > > > > > > > talks to R, which is GPL'd).
> > > > > > > > > > >
> > > > > > > > > > > * No necessary *dependencies* of the PR have license
> > > > conflicts.
> > > > > > In
> > > > > > > > > > > several cases, I contacted package authors who agreed
> to
> > > > > > re-issue their
> > > > > > > > > > > packages under Apache-compatible licenses. (Usually I
> had
> > > to
> > > > do
> > > > > > a bit
> > > > > > > > > of
> > > > > > > > > > > coding in exchange…)
> > > > > > > > > > >
> > > > > > > > > > > * Where the license had to stay GPL, the packages are
> > *not
> > > > > > necessary*
> > > > > > > > > and
> > > > > > > > > > > *not dependencies.* If the optional packages are
> present,
> > > > they
> > > > > > will be
> > > > > > > > > > > used to enable additional functionality. Knitr is an
> > > example.
> > > > > > The PR
> > > > > > > > > will
> > > > > > > > > > > compile and run fine without knitr. If knitr is
> available
> > > > (it is
> > > > > > part
> > > > > > > > > of
> > > > > > > > > > > the most common R distribution), the PR will enable the
> > > knitr
> > > > > > > > > > interpreter.
> > > > > > > > > > >
> > > > > > > > > > > * This is exactly how this issue is addressed through
> the
> > > > Apache
> > > > > > > > > > > ecosystem.
> > > > > > > > > > > The tl;dr is this: When Apache code is written to talk
> to
> > > > > > libraries
> > > > > > > > > that
> > > > > > > > > > > may or may not be present on the user’s system, or
> where
> > > it
> > > > > > talks to an
> > > > > > > > > > API
> > > > > > > > > > > but is agnostic about implementation, that is not
> > > “linking”
> > > > in a
> > > > > > way
> > > > > > > > > that
> > > > > > > > > > > implicate the anti-linking provision of the GPL.
> > > > > > > > > > >
> > > > > > > > > > > Otherwise, no Apache code could ever call a shell
> script!
> > > Let
> > > > > > alone run
> > > > > > > > > > > on Linux, or talk to R.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I'm not a legal expert. So following could be wrong.
> > > > > > > > > > >
> > > > > > > > > > > In my interpretation, KnitRInterpreter is not an
> optional
> > > > > > feature while
> > > > > > > > > > it
> > > > > > > > > > > is always enabled when compiling Zeppelin and always
> > > enabled
> > > > when
> > > > > > > > > running
> > > > > > > > > > > Zeppelin. And it requires dynamically linked GPL
> library
> > > on
> > > > > > runtime.
> > > > > > > > > (yes
> > > > > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > > > > >
> > > > > > > > > > > And of course, no Apache code can ever call a shell
> > > script,
> > > > on
> > > > > > the
> > > > > > > > > > purpose
> > > > > > > > > > > of dynamic linking with GPL library.
> > > > > > > > > > >
> > > > > > > > > > > I was guessing SparkR can be still in Apache License
> even
> > > if
> > > > it
> > > > > > is
> > > > > > > > > > depends
> > > > > > > > > > > on R. Because of GPL licensed compiler generated output
> > is
> > > > not
> > > > > > GPL
> > > > > > > > > > license.
> > > > > > > > > > > and R is sort of compiler.
> > > > > > > > > > >
> > > > > > > > > > > If you can get answer from Spark community how SparkR
> get
> > > > > > managed to
> > > > > > > > > stay
> > > > > > > > > > > in Apache License while R is GPL, the answer might
> help.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 3. Need more time to review.
> > > > > > > > > > > Has any reviewer has downloaded the PR or run the demo
> > > > notebook?
> > > > > > (Which
> > > > > > > > > > > is there for the benefit of reviewers, and isn’t
> intended
> > > to
> > > > go
> > > > > > in
> > > > > > > > > final
> > > > > > > > > > > distribution.)
> > > > > > > > > > >
> > > > > > > > > > > How many +1 comments from users would you like to see?
> > > > > > > > > > >
> > > > > > > > > > > How much time do you believe is required?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > It all depends on when CI is going to pass, when
> license
> > > > problem
> > > > > > is
> > > > > > > > > going
> > > > > > > > > > > to be cleared, and when a committer willing to review
> and
> > > > > > responsible
> > > > > > > > > to
> > > > > > > > > > > commit your contribution.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 1. Large code base change
> > > > > > > > > > > Large code base changes require coordination and
> > > cooperation.
> > > > > > This PR
> > > > > > > > > > > necessarily implicates the build scripts, testing code,
> > > the
> > > > > > > > > > > SparkInterpreter, etc.
> > > > > > > > > > >
> > > > > > > > > > > I have been seeking to coordinate since August.
> > > > > > > > > > >
> > > > > > > > > > > I continue to stand ready to do so.
> > > > > > > > > > >
> > > > > > > > > > > -Amos
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > If i give you one suggestion, Zeppelin committers
> > > sometimes
> > > > ask
> > > > > > rebase
> > > > > > > > > > the
> > > > > > > > > > > contribution branch for some reason. It is not the
> really
> > > the
> > > > > > best
> > > > > > > > > > > practice, but still okay while most contributions are
> not
> > > > > > including
> > > > > > > > > large
> > > > > > > > > > > code base changes.
> > > > > > > > > > >
> > > > > > > > > > > However, your one, has more than 4000 lines of code
> > > change.
> > > > If
> > > > > > you
> > > > > > > > > rebase
> > > > > > > > > > > then review should be started from the beginning,
> again.
> > > So
> > > > you
> > > > > > might
> > > > > > > > > > want
> > > > > > > > > > > to minimize the rebase your branch.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > moon
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > > Date: December 1, 2015 at 1:34:19 AM
> > > > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > > > > dev@zeppelin.incubator.apache.org
> > > > > > > > > > >
> > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin
> > > > > > > > > pull
> > > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > > >
> > > > > > > > > > > Hi Cos,
> > > > > > > > > > >
> > > > > > > > > > > Thanks for opening a discussion.
> > > > > > > > > > > My answer about 'Why this PR is open for three months'
> is
> > > > > > > > > > >
> > > > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > > > 2. Not 100% sure this PR has no license issue. (about
> > > KniteR)
> > > > > > > > > > > 3. Need more time to review.
> > > > > > > > > > >
> > > > > > > > > > > It's my personal answer, other committers may have
> > > different
> > > > > > opinion.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I think the question should be generalized. Because
> this
> > > PR
> > > > is
> > > > > > not the
> > > > > > > > > > only
> > > > > > > > > > > PR that is in impasse. There're more. For example
> > > > > > > > > > >
> > > > > > > > > > > Here's some examples that PR is not been merged.
> > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,
> > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/60
> > > > > > > > > > > and so on.
> > > > > > > > > > >
> > > > > > > > > > > I can categorize the cases, based on experience of
> > > involving
> > > > > > Zeppelin
> > > > > > > > > > > community from the beginning.
> > > > > > > > > > >
> > > > > > > > > > > 1. Large code base change
> > > > > > > > > > >
> > > > > > > > > > > When contribution has large code base changes, it tend
> to
> > > > take
> > > > > > more
> > > > > > > > > time
> > > > > > > > > > to
> > > > > > > > > > > review and merged. Normally, most contributions merged
> in
> > > > 1day~1
> > > > > > week.
> > > > > > > > > > > But some contribution has large code base changes take
> > 2~4
> > > > > > weeks. Few
> > > > > > > > > > > contribution that has very large code base change take
> > > > months.
> > > > > > > > > > >
> > > > > > > > > > > 2. Communication lost
> > > > > > > > > > >
> > > > > > > > > > > Sometimes, committer is not responding, or contributor
> is
> > > not
> > > > > > > > > responding.
> > > > > > > > > > >
> > > > > > > > > > > 3. Opinion diverges
> > > > > > > > > > >
> > > > > > > > > > > For some changes, included in contribution. When people
> > > have
> > > > > > different
> > > > > > > > > > > opinion and it continue to diverges, those PRs are not
> > > been
> > > > > > merged.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I think having a guide such as ping other committer
> when
> > a
> > > > > > committer is
> > > > > > > > > > not
> > > > > > > > > > > responding, and divide contribution into small peaces
> if
> > > > > > possible,
> > > > > > > > > would
> > > > > > > > > > > help most of the cases.
> > > > > > > > > > > Of course committer have to pay attention more to the
> > > > > > contribution and
> > > > > > > > > > > helping people. That's the first one.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > moon
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <
> > > > > > cos@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > To make sure we're on the same page, here are two
> > > threads
> > > > that
> > > > > > I
> > > > > > > > > found
> > > > > > > > > > > > related
> > > > > > > > > > > > to this topic.
> > > > > > > > > > > >
> > > > > > > > > > > > Thread 1:
> > > > > > > > > > > > Subject: R?
> > > > > > > > > > > > Started on: July 1, 2015
> > > > > > > > > > > >
> > > > > > > > > > > > Thread 2:
> > > > > > > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R
> > > > > > Interpreter for
> > > > > > > > > > > > Zeppelin
> > > > > > > > > > > > Started on: August 13, 2015
> > > > > > > > > > > >
> > > > > > > > > > > > If you want to fetch these from our archive send
> emails
> > > to
> > > > > > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > > > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > > > > > > > > > >
> > > > > > > > > > > > Cos
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik
> > > wrote:
> > > > > > > > > > > > > Guys,
> > > > > > > > > > > > >
> > > > > > > > > > > > > While catching up on my emails from the last a
> couple
> > > of
> > > > > > weeks,
> > > > > > > > > this
> > > > > > > > > > > > thread
> > > > > > > > > > > > > caught my attention. I am not normally paying much
> > > > attention
> > > > > > to the
> > > > > > > > > > > code
> > > > > > > > > > > > > reviews traffic from GH, but this one is pretty
> > > > different as
> > > > > > it
> > > > > > > > > spans
> > > > > > > > > > > > three
> > > > > > > > > > > > > months and counting.
> > > > > > > > > > > > >
> > > > > > > > > > > > > First, here are my five cents:
> > > > > > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is
> > aimed
> > > > to be
> > > > > > > > > > > > contributed to
> > > > > > > > > > > > > an ASF project this file should simply contain ASL2
> > > text,
> > > > > > like in
> > > > > > > > > [1]
> > > > > > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate
> > > > <developers>
> > > > > > > > > > section,
> > > > > > > > > > > > but
> > > > > > > > > > > > > Zeppelin might have different guidelines on it.
> Say,
> > > > Bigtop
> > > > > > doesn't
> > > > > > > > > > > > > maintain this in the source code, but we have the
> > list
> > > of
> > > > > > all the
> > > > > > > > > > > > > committers on the project's site [2] Every new
> > > > committer's
> > > > > > first
> > > > > > > > > > > > commit is
> > > > > > > > > > > > > to update the team page ;)
> > > > > > > > > > > > > - comments like in
> > > > > > > > > > > >
> > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > > > > > > > > > >
> > > > > > > > > > > > > +/**
> > > > > > > > > > > > > + * Created by aelberg on 7/28/15.
> > > > > > > > > > > > > + */
> > > > > > > > > > > > >
> > > > > > > > > > > > > is better to be removed. It has been already
> > discussed
> > > > here
> > > > > > [3].
> > > > > > > > > And
> > > > > > > > > > > > the
> > > > > > > > > > > > > initial discussion on the topic [4] was linked as
> > well
> > > > > > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not
> > > sure
> > > > if
> > > > > > this is
> > > > > > > > > > > > R-specific
> > > > > > > > > > > > > stuff - I have no idea about R, honestly, but
> > > > > > > > > > > > >
> > > > > > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > > > > > > > > > > ...
> > > > > > > > > > > > > +Author: David B. Dahl
> > > > > > > > > > > > >
> > > > > > > > > > > > > shouldn't be here, IMO. Normally, if some
> additional
> > > > > > licenses are
> > > > > > > > > > > > used,
> > > > > > > > > > > > > they have to be listed in the top-level NOTICE file
> > > > (already
> > > > > > > > > there).
> > > > > > > > > > > > >
> > > > > > > > > > > > > - I am not going to offer any comments on the
> > > technical
> > > > > > merits of
> > > > > > > > > the
> > > > > > > > > > > > patch,
> > > > > > > > > > > > > as I haven't tried it personally. However, I don't
> > see
> > > > any
> > > > > > serious
> > > > > > > > > > > > > technical objections to the functionality in
> > question.
> > > > > > > > > > > > >
> > > > > > > > > > > > > So, the question is - why the PR is open for three
> > > > months? I
> > > > > > hasn't
> > > > > > > > > > > been
> > > > > > > > > > > > able
> > > > > > > > > > > > > to get a clear answer. What I found out though is
> > > pretty
> > > > > > > > > unsettling,
> > > > > > > > > > > > really.
> > > > > > > > > > > > > The communication on the topic is almost
> > non-existent,
> > > > > > except for
> > > > > > > > > > this
> > > > > > > > > > > > sparse
> > > > > > > > > > > > > and bitter thread in the GH.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I would love to hear from the committers what's
> > > stopping
> > > > the
> > > > > > > > > > acceptance
> > > > > > > > > > > > of the
> > > > > > > > > > > > > code, besides of the minor issues I've mentioned
> > > earlier?
> > > > > > What are
> > > > > > > > > > the
> > > > > > > > > > > > reasons for it?
> > > > > > > > > > > > > Is there anything wrong with it?
> > > > > > > > > > > > > One of the responsibilities of the committers is to
> > > make
> > > > > > sure the
> > > > > > > > > > > > > contributions are reviewed; new contributors are
> > > guided
> > > > and
> > > > > > do
> > > > > > > > > > > > understand how
> > > > > > > > > > > > > the project ticks. The easy feedback flow attracts
> > new
> > > > > > people,
> > > > > > > > > > allowing
> > > > > > > > > > > > the
> > > > > > > > > > > > > community to grow and thrive.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I am asking _explicitely_ not to re-start the
> > > bickering I
> > > > > > have
> > > > > > > > > > already
> > > > > > > > > > > > > seen. At this point I am interested in the purely
> > > > technical
> > > > > > side of
> > > > > > > > > > > this.
> > > > > > > > > > > > >
> > > > > > > > > > > > > [1]
> > > https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > > > > > > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > > > > > > > > > [3]
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > >
> > >
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > > > > > > > > > [4] http://s.apache.org/iZl
> > > > > > > > > > > > >
> > > > > > > > > > > > > With regards,
> > > > > > > > > > > > > Cos
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > > > > > > > > > Github user elbamos commented on the pull
> request:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The current push should resolve some issues with
> > > > changes
> > > > > > in the
> > > > > > > > > > > > > > Spark-Zeppelin interface that had created issues
> > for
> > > > > > users, as
> > > > > > > > > > > > well as
> > > > > > > > > > > > > > support for 1.5.1.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Knitr support is improved, and the reason for a
> > > > separate
> > > > > > knitr
> > > > > > > > > > > > interpreter may be clearer now.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The amount of code borrowed from rscala is
> reduced.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I did not address issues with @author tags, or
> > files
> > > > under
> > > > > > the R/
> > > > > > > > > > > > > > folder. The reason is, to be blunt, I don't
> > > understand
> > > > > > what the
> > > > > > > > > > > > precise
> > > > > > > > > > > > > > concerns actually are.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Please note that I changed .travis.yml to only
> use
> > > > spark
> > > > > > 1.4 and
> > > > > > > > > > > > later.
> > > > > > > > > > > > > > I'm sure there is a better way to do it, but I'm
> > > just
> > > > not
> > > > > > in a
> > > > > > > > > > > > position
> > > > > > > > > > > > > > to try to figure out the entire Zeppelin build
> > > process.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Integrating this with the rest of zeppelin is
> going
> > > to
> > > > > > take some
> > > > > > > > > > > > work
> > > > > > > > > > > > > > regarding pom's, travis, and so forth. I can do a
> > > lot
> > > > of
> > > > > > that,
> > > > > > > > > > > > but I'm
> > > > > > > > > > > > > > going to need to discuss it with the people who
> > have
> > > > been
> > > > > > > > > "owning"
> > > > > > > > > > > > those
> > > > > > > > > > > > > > files. There are just too many moving pieces
> here.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Because of the size of this (which is,
> > > unfortunately,
> > > > > > necessary),
> > > > > > > > > > > > > > posting here is probably not the most efficient
> > way.
> > > > That
> > > > > > is also
> > > > > > > > > > > > true
> > > > > > > > > > > > > > because certain people chose to use this PR as a
> > > forum
> > > > to
> > > > > > air
> > > > > > > > > other
> > > > > > > > > > > > > > issues. Therefore, it would be better if
> reviewers
> > > > e-mail
> > > > > > me
> > > > > > > > > > > > directly.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > >
> > >
> > >
> >
>

Re: R-Interpreter CI Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Hi,

I did couple of test against rinterpreter pr [1]. and end up with exception

15/12/25 02:25:54 INFO AppClient$ClientEndpoint: Connecting to master
spark://testing-gce-28dccb9f-5570-4e9c-b911-a5974c99c02f.c.eco-emissary-99515.internal:7071...
15/12/25 02:26:14 ERROR SparkUncaughtExceptionHandler: Uncaught
exception in thread Thread[appclient-registration-retry-thread,5,main]
java.util.concurrent.RejectedExecutionException: Task
java.util.concurrent.FutureTask@5081a4bb rejected from
java.util.concurrent.ThreadPoolExecutor@6f661a47[Running, pool size =
1, active threads = 0, queued tasks = 0, completed tasks = 1]
	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
	at org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anonfun$tryRegisterAllMasters$1.apply(AppClient.scala:96)
	at org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anonfun$tryRegisterAllMasters$1.apply(AppClient.scala:95)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
	at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
	at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
	at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:108)
	at org.apache.spark.deploy.client.AppClient$ClientEndpoint.tryRegisterAllMasters(AppClient.scala:95)
	at org.apache.spark.deploy.client.AppClient$ClientEndpoint.org$apache$spark$deploy$client$AppClient$ClientEndpoint$$registerWithMaster(AppClient.scala:121)
	at org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2$$anonfun$run$1.apply$mcV$sp(AppClient.scala:132)
	at org.apache.spark.util.Utils$.tryOrExit(Utils.scala:1119)
	at org.apache.spark.deploy.client.AppClient$ClientEndpoint$$anon$2.run(AppClient.scala:124)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


Strange thing is, above test is not related with R interpreter but it
happens
when .travis.yml has following entry in before_intall: section

 echo "deb http://cran.rstudio.com/bin/linux/ubuntu precise/" | sudo tee -a
/etc/apt/sources.list


Following is build log (success) without above line in .travis.yml
https://travis-ci.org/Leemoonsoo/incubator-zeppelin/builds/98927745
and code
https://github.com/Leemoonsoo/incubator-zeppelin/tree/71791ee0362d580f670ea037f5af2c780adc2828

Following is build log (fail) with above line in .travis.yml
https://travis-ci.org/Leemoonsoo/incubator-zeppelin/builds/98927745
and code
https://github.com/Leemoonsoo/incubator-zeppelin/tree/650dfa8c3477df722fc4381af62c92b5eeb84403


This is really weird. Let me update more when i get better understanding of
it.

Thanks,
moon

[1] https://github.com/apache/incubator-zeppelin/pull/208



On Tue, Dec 8, 2015 at 10:57 AM Amos B. Elberg <am...@gmail.com>
wrote:

> Thanks, Moon.
>
> If you look at the code now, the reason travis fails is because the
> travis.yaml is messed-up from me trying to get travis not to fail.  But
> with that fixed it only fails for other reasons…
>
> I have seen both the failures you describe, but I’ve also seen a bunch of
> others.  The most common issue is that the PR, to test, needs to
> instantiate a SparkInterpreter instance without the infrastructure of the
> Zeppelin server, and Spark fails to start.
>
> As a practical way to move forward I propose the following:
>
> 1.  The CI issues are broader than this PR, and we should start a separate
> thread to discuss and fix CI in general.
>
> 2.  I’m going to close the current PR, and make a new PR that is based on
> the 0.5.5 release code.  This PR will *not* be sync’d with
> Zeppelin-Master.  It will be sync’d w/ 0.5.5 release because that is a
> stable base.  I would appreciate if people could help me with CI, using
> that as a base.
>
> --
> Amos Elberg
> Sent with Airmail
>
> From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org
> <de...@zeppelin.incubator.apache.org> <de...@zeppelin.incubator.apache.org>
> Date: December 8, 2015 at 9:26:38 AM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> <de...@zeppelin.incubator.apache.org>
> Subject:  Re: R-Interpreter CI Was: Re: contributions impasse. Was:
> [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin
>
> fyi, some cases of CI failing, when PR does not impact the code,
>
> 1) Flickering test. Some tests are passing sometimes, failing the other
> times. (e.g. ZEPPELIN-476)
> 2) Download fails. Downloading maven artifacts or other requirements from
> internet are sometimes failing
>
> both cases pass if CI triggered again.
>
> Thanks,
> moon
>
> On Tue, Dec 8, 2015 at 6:29 PM DuyHai Doan <do...@gmail.com> wrote:
>
> > I have also experienced CI failing in the pass for some PRs that do not
> > impact the code (Cassandra documentation). I guess that during peak
> hours,
> > the CI servers may be too busy or enter a dead lock so the build fails.
> >
> > At least that's my guess. What do you think ?
> >
> > On Tue, Dec 8, 2015 at 9:42 AM, moon soo Lee <mo...@apache.org> wrote:
> >
> > > Let me run some CI test with your branch and share result here.
> > > Hope i can narrow down the cause and that helps involvement of more
> > people.
> > >
> > > Thanks,
> > > moon
> > >
> > > On Tue, Dec 8, 2015 at 3:08 PM Amos B. Elberg <am...@gmail.com>
> > > wrote:
> > >
> > > > Is there anyone who can work with me on the CI issues? It looks like
> > > > there are a number of PRs experiencing similar things.
> > > >
> > > > I think we should make getting CI stable to be a priority. Because
> it
> > > > will save everyone a lot of frustration and aggravation if CI works
> > > > reliably.
> > > >
> > > > Is there anyone other than Jongyoul and Moon who knows the CI/Build
> > > > process well?
> > > >
> > > > (Moon - Thank you for taking another look at the licensing issue.
> Per
> > > the
> > > > e-mail I wrote about this a few days ago, I don’t feel I have more
> to
> > > > contribute to the licensing discussion, so I’m going to try not to
> > > comment
> > > > further about it.)
> > > >
> > > > From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> > > > Reply: dev@zeppelin.incubator.apache.org
> > > > <de...@zeppelin.incubator.apache.org> <
> dev@zeppelin.incubator.apache.org
> > >
> > > > Date: December 7, 2015 at 5:00:08 PM
> > > > To: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org
> > > >
> > > > <de...@zeppelin.incubator.apache.org>
> > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions
> > impasse.
> > > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > Zeppelin
> > > >
> > > > Thanks Amos, Roman, Cos for clarifying license issue.
> > > >
> > > > I'm convinced that this license issue will not be a blocker.
> > > >
> > > > In my understanding, these are good sign,
> > > >
> > > > 1. any gpl licensed source codes are not included in the source
> package
> > > > 2. any gpl licensed libraries are not included in the binary package
> > > >
> > > > However, i can not still 100% sure about
> > > >
> > > > 3. any gpl licensed libraries are not linked on runtime
> > > >
> > > > Even after Amos's explanation. I still think using 'knitr' is one of
> > the
> > > > clear case that 'knir' is linked to 'R' according to
> > > > http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL.
> > > >
> > > > Giving input and getting output from GPL licensed interpreter
> (includes
> > > R)
> > > > from Apache licensed software is not a problem. That's not the
> point.
> > > > Let me say in this way. There's an java code,
> > > >
> > > > import com.mysql.jdbc.Driver
> > > > Driver driver = new Driver()
> > > >
> > > > Say without this java code, one of important feature of Zeppelin
> does
> > not
> > > > work. And Zeppelin does not includes GPL licensed file in the source
> > > > package, GPL licensed library in the binary package, but it requires
> > GPL
> > > > licensed library on the runtime.
> > > > In this case, will this java code be a license problem or not?
> > > >
> > > > In other words, my question is
> > > >
> > > > a) Is runtime GPL library dependency allowed in ASF release?
> > > > b) is 'knitr' considered as runtime dependency?
> > > >
> > > > If someone can clarify a), b), then it would be extremely helpful
> > > > understanding this case, and possible similar cases, too.
> > > >
> > > > Thanks,
> > > > moon
> > > >
> > > > On Mon, Dec 7, 2015 at 3:20 PM Konstantin Boudnik <co...@apache.org>
> > > wrote:
> > > >
> > > > > On Thu, Dec 03, 2015 at 11:03AM, Corneau Damien wrote:
> > > > > > Thanks Cos for those answers,
> > > > > >
> > > > > > If I'm right you are advising to have a default build that
> doesn't
> > > > > include
> > > > > > libraries with conflicting licenses, but have an option to
> include
> > > > them
> > > > > for
> > > > > > users who wants to build the project themselves.
> > > > >
> > > > > Yes, that's what I said. Besides, looks like Roman provided the
> > second
> > > > > pair of
> > > > > eyes to this licensing discussion and as well didn't find any
> issues
> > > > with
> > > > > the
> > > > > current approach.
> > > > >
> > > > > Cos
> > > > >
> > > > > > To refer to another thread about decentralizing interpreters, it
> > > could
> > > > > even
> > > > > > be better in that case to have some interpreters separated from
> the
> > > > tree,
> > > > > > and easily pluggable with a release instead of forcing users to
> > build
> > > > the
> > > > > > project to use them.
> > > > > >
> > > > > >
> > > > > > On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <
> cos@apache.org
> > >
> > > > > wrote:
> > > > > >
> > > > > > > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:
> > > > > > > > Konstantin thank you for getting into this.
> > > > > > > >
> > > > > > > >> The best way to go around it is by
> > > > > > > >> providing a build-time option that will pull such binaries
> in.
> > > > But
> > > > > by
> > > > > > > default
> > > > > > > >> such libs shouldn't be pulled.
> > > > > > > >
> > > > > > > > That is basically how the PR handles this. If the GPL’d
> > > > interpreter
> > > > > > > scripts
> > > > > > > > are missing, there’s no indication at all at build time. It
> > > > doesn’t
> > > > > try
> > > > > > > to
> > > > > > > > download them.
> > > > > > > >
> > > > > > > > At runtime, if the user tries to use functionality that
> would
> > > need
> > > > > such a
> > > > > > > > script (i.e., if they type “knitr” to use knitr), we display
> an
> > > > error
> > > > > > > that
> > > > > > > > says that the functionality is not there because the library
> is
> > > > > missing,
> > > > > > > and
> > > > > > > > that the library cannot be provided because it has an
> > > incompatible
> > > > > > > license,
> > > > > > > > but the user can download it themselves if they want.
> > > > > > > >
> > > > > > > > And, in the log, if the logging level is high, they will see
> a
> > > > note
> > > > > that
> > > > > > > > some functionality was disabled because the libraries
> weren’t
> > > > there.
> > > > > > > >
> > > > > > > > To be clear, though, none of these libraries are binaries.
> > > They’re
> > > > > all
> > > > > > > interpreter scripts.
> > > > > > >
> > > > > > > If you ever in a doubt of which licenses could be used for
> > > > dependncies
> > > > > > > (not to
> > > > > > > say about source code) are listed in Category A list of [1]
> > > > > > >
> > > > > > > A lot of quesitons discussed here are already covered in the
> > legal
> > > > > FAQ, so
> > > > > > > just check against it if you have any questions.
> > > > > > >
> > > > > > > [1] http://www.apache.org/legal/resolved.html#category-a
> > > > > > >
> > > > > > > Cos
> > > > > > >
> > > > > > > > From: Konstantin Boudnik <co...@apache.org>
> > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > dev@zeppelin.incubator.apache.org>,
> > > > dev@zeppelin.incubator.apache.org
> > > > > <
> > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > Date: December 2, 2015 at 3:24:50 PM
> > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > dev@zeppelin.incubator.apache.org
> > > > > > > >
> > > > > > > > Subject: Re: License of KnitRInterpreter Was: Re:
> contributions
> > > > > > > impasse. Was: [GitHub] incubator-zeppelin pull request: R
> > > > Interpreter
> > > > > for
> > > > > > > Zeppelin
> > > > > > > >
> > > > > > > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:
> > > > > > > > > I think that what moon means is that:
> > > > > > > > > If we merge the way it is now, the KnitRInterpreter will
> be
> > > part
> > > > > of the
> > > > > > > > > code base, so it isn't optional at code base level.
> > > > > > > > >
> > > > > > > > > To make it even more simple:
> > > > > > > > > * If the code has the right licensing -> that code can be
> > part
> > > > of
> > > > > the
> > > > > > > > > source code, and can be including in a binary release
> > > > > > > >
> > > > > > > > We aren't concerned with binary releases - as an Apache
> > community
> > > > we
> > > > > are
> > > > > > > > voting and releasing source code. If the project wants to
> > provide
> > > > a
> > > > > > > binary
> > > > > > > > release to its users, they are better be warned about
> inclusion
> > > of
> > > > > non
> > > > > > > > ASL2-friendly licensed bits.
> > > > > > > >
> > > > > > > > > * If the code doesn't have the right licensing -> it can't
> be
> > > > part
> > > > > of
> > > > > > > the
> > > > > > > > > source code, and can't be included in a binary release
> > > > > > > >
> > > > > > > > See above.
> > > > > > > >
> > > > > > > > > * If the code doesn't have the right licensing but is
> > imported
> > > > at
> > > > > build
> > > > > > > > > time (libraries for example) -> it is not in the source
> code,
> > > it
> > > > > can't
> > > > > > > be
> > > > > > > > > included in binary release
> > > > > > > >
> > > > > > > > That is unless a user doing it on his own. The best way to
> go
> > > > around
> > > > > it
> > > > > > > is by
> > > > > > > > providing a build-time option that will pull such binaries
> in.
> > > But
> > > > by
> > > > > > > default
> > > > > > > > such libs shouldn't be pulled.
> > > > > > > >
> > > > > > > > Cos
> > > > > > > >
> > > > > > > > > So in the case of licensing issues, it would need to be
> fully
> > > > > optional
> > > > > > > > > (user bring the interpreter in his directory and build
> > Zeppelin
> > > > > with
> > > > > > > it)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <
> > > > > amos.elberg@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Moon let me clarify:
> > > > > > > > > >
> > > > > > > > > > Interpreted code doesn’t “link.” The wiki article
> actually
> > > > > explains
> > > > > > > it
> > > > > > > > > > pretty well —
> > > > > https://en.wikipedia.org/wiki/GPL_linking_exception
> > > > > > > > > > “Linking” against a library means compiling its headers
> > into
> > > a
> > > > > > > binary, the
> > > > > > > > > > way a C compiler works. The 2008 e-mail Moon
> distributed,
> > > > called
> > > > > > > this the
> > > > > > > > > > “interpreter exception.”
> > > > > > > > > >
> > > > > > > > > > As for whether GPL’d code is a “mandatory dependency,”
> if
> > > > knitr
> > > > > is
> > > > > > > missing
> > > > > > > > > > the PR will compile, run and test just fine. The user is
> > > never
> > > > > > > prompted to
> > > > > > > > > > download it. The only effect is, if the user types
> “knitr”
> > > and
> > > > > knitr
> > > > > > > isn’t
> > > > > > > > > > there, we display an InterpreterError that knitr isn’t
> > there.
> > > > > > > > > >
> > > > > > > > > > KnitRInterpreter is not optionally required. so it does
> not
> > > > > matter
> > > > > > > KnitR
> > > > > > > > > > is
> > > > > > > > > > optionally required or not.
> > > > > > > > > > Aren’t all interpreters optional? You can turn them on
> and
> > > off
> > > > > in the
> > > > > > > > > > config files.
> > > > > > > > > >
> > > > > > > > > > Do you mean that the KnitRInterpreter class gets
> compiled
> > to
> > > > > > > bytecode even
> > > > > > > > > > if knitr is missing? So what? That isn't a mandatory
> > > > dependency
> > > > > or a
> > > > > > > link.
> > > > > > > > > >
> > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > Date: December 2, 2015 at 3:18:00 AM
> > > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > Subject: Re: License of KnitRInterpreter Was: Re:
> > > > contributions
> > > > > > > impasse.
> > > > > > > > > > Was: [GitHub] incubator-zeppelin pull request: R
> > Interpreter
> > > > for
> > > > > > > Zeppelin
> > > > > > > > > >
> > > > > > > > > > Let me summarize license concern about KnitRInterpreter.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Amos's interpretation.
> > > > > > > > > >
> > > > > > > > > > KnitR is optionally required by KnitRInterpreter.
> > > > > > > > > > R dependency in SparkR has no problem. So KnitR should
> be
> > the
> > > > > same.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Moon's interpretation.
> > > > > > > > > >
> > > > > > > > > > KnitRInterpreter is not optionally required. so it does
> not
> > > > > matter
> > > > > > > KnitR is
> > > > > > > > > > optionally required or not.
> > > > > > > > > > R dependency in SparkR is exception of GPL. KnitR is not
> > > > applied
> > > > > that
> > > > > > > > > > exception.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Amos, could you confirm my understanding to your
> > > > interpretation
> > > > > is
> > > > > > > correct?
> > > > > > > > > > If it is not could you clarify it?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > moon
> > > > > > > > > >
> > > > > > > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <
> > > > > > > amos.elberg@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Just to put the final nail in this, I looked it up.
> > > > > > > > > > >
> > > > > > > > > > > I see no evidence of any “compiler exception.”
> > > > > > > > > > >
> > > > > > > > > > > There is an exception in the license for the runtime
> > > > libraries
> > > > > > > that are
> > > > > > > > > > > bundled with GCC. See:
> > > > > > > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> > > > > > > > > > >
> > > > > > > > > > > But no “compiler exception.”
> > > > > > > > > > >
> > > > > > > > > > > In fact, I believe this is part of the reason why LLVM
> > was
> > > > > created.
> > > > > > > > > > >
> > > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Date: December 1, 2015 at 8:16:36 PM
> > > > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > > incubator-zeppelin pull
> > > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > > >
> > > > > > > > > > > Is knitR is commonly considered as a
> > interpreter/compiler?
> > > > or
> > > > > is it
> > > > > > > > > > > considered as a library routine?
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > moon
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <
> > > > > > > amos.elberg@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > Moon - you give this as an explanation of the
> licensing
> > > > issue:
> > > > > > > > > > >
> > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > > > > >
> > > > > > > > > > > According to that, there is an exception in the GPL
> for
> > > > > interpreter
> > > > > > > > > > > languages. As long as you don’t distribute the code,
> its
> > > > fine
> > > > > to
> > > > > > > talk to
> > > > > > > > > > > an interpreted language.
> > > > > > > > > > >
> > > > > > > > > > > Well, if that’s the case, then the PR plainly does not
> > have
> > > > a
> > > > > > > license
> > > > > > > > > > > issue. It doesn’t distribute any GPL’d R code.
> > > > > > > > > > >
> > > > > > > > > > > I’m not sure what’s confusing about this. It seems
> > > > completely
> > > > > > > > > > > straightforward.
> > > > > > > > > > >
> > > > > > > > > > > Regarding this:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Amos Elberg
> > > > > > > > > > > Sent with Airmail
> > > > > > > > > > >
> > > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > > Date: December 1, 2015 at 6:48:47 PM
> > > > > > > > > > >
> > > > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > > dev@zeppelin.incubator.apache.org
> > > > > > > > > > >
> > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > > incubator-zeppelin pull
> > > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <
> > > > > > > amos.elberg@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > I am going to try to minimize my reaction to Moon’s
> > > > e-mail.
> > > > > > > > > > > >
> > > > > > > > > > > > The tl;dr is this:
> > > > > > > > > > > >
> > > > > > > > > > > > The reason we are having this discussion now is that
> > > > active
> > > > > > > users of
> > > > > > > > > > the
> > > > > > > > > > > > PR — which now has its own user base — went public
> to
> > > > > complain
> > > > > > > about
> > > > > > > > > > > this.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > The PR has been tested by an active user base for
> more
> > > > than
> > > > > three
> > > > > > > > > > months.
> > > > > > > > > > > > No-one has been able to identify any specific actual
> > > > > licensing
> > > > > > > problem,
> > > > > > > > > > > and
> > > > > > > > > > > > the PR was prepared based on an extensive, careful
> > review
> > > > of
> > > > > the
> > > > > > > > > > relevant
> > > > > > > > > > > > licensing issues and after contacting the relevant
> > > people.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I admire every software that used by user and helping
> > > > people.
> > > > > That
> > > > > > > > > > includes
> > > > > > > > > > > your work. But that's not the topic we're in
> discussion.
> > > > Active
> > > > > > > user does
> > > > > > > > > > > not mean your contribution can ignore the review.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > It is not an explanation for someone who has been
> > > ignoring
> > > > my
> > > > > > > “how can
> > > > > > > > > > I
> > > > > > > > > > > > move this forward…” emails for three months to point
> > the
> > > > > finger
> > > > > > > and
> > > > > > > > > > say I
> > > > > > > > > > > > didn’t contact the right person or file the right
> > report.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > This is also not the topic in this discussion.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > The burden for providing an explanation for the
> > inaction
> > > > is
> > > > > on
> > > > > > > the PMCC
> > > > > > > > > > > at
> > > > > > > > > > > > this point.
> > > > > > > > > > >
> > > > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's
> > > problem
> > > > on
> > > > > > > Zeppelin
> > > > > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > > > > > They’re not! I often see comments on PRs to just
> ignore
> > > > that
> > > > > CI
> > > > > > > is
> > > > > > > > > > > > failing.
> > > > > > > > > > > >
> > > > > > > > > > > > One of the most common reasons this PR fails CI, is
> CI
> > > > > times-out
> > > > > > > > > > > > downloading Spark to install. How could that
> possibly
> > be
> > > > > caused
> > > > > > > by the
> > > > > > > > > > > PR?
> > > > > > > > > > > >
> > > > > > > > > > > > It looks to me like the only PRs with changes to the
> > > > relevant
> > > > > > > parts of
> > > > > > > > > > > the
> > > > > > > > > > > > code — the SparkInterpreter — are being made by the
> > > person
> > > > > who
> > > > > > > wrote
> > > > > > > > > > the
> > > > > > > > > > > > testing suite.
> > > > > > > > > > > >
> > > > > > > > > > > > So, that would explain why some other PRs pass CI:
> > > Neither
> > > > > the
> > > > > > > > > > > > SparkInterpreter nor the testing suite are stable or
> > > > robust,
> > > > > but
> > > > > > > since
> > > > > > > > > > > the
> > > > > > > > > > > > PRs are coming from the person who wrote both…
> > > > > > > > > > > >
> > > > > > > > > > > > And let's say Zeppelin core has problem and that
> makes
> > > > your
> > > > > PR
> > > > > > > fails on
> > > > > > > > > > > CI
> > > > > > > > > > > > test. That's possible. But it still does not mean we
> > can
> > > > > merge
> > > > > > > it with
> > > > > > > > > > CI
> > > > > > > > > > > > failing.
> > > > > > > > > > > >
> > > > > > > > > > > > It means you should be working with me to figure out
> > why
> > > > the
> > > > > CI
> > > > > > > is
> > > > > > > > > > > failing.
> > > > > > > > > > > >
> > > > > > > > > > > > This PR has been tested by an active user base for
> the
> > > > past
> > > > > three
> > > > > > > > > > months.
> > > > > > > > > > > > If CI is continuing to fail, and dozens of hours of
> > > effort
> > > > > have
> > > > > > > not
> > > > > > > > > > > > resolved the CI issues, then it is time to start
> > > > considering
> > > > > > > whether
> > > > > > > > > > the
> > > > > > > > > > > > testing suite is part of the problem.
> > > > > > > > > > > >
> > > > > > > > > > > > The level of defensiveness about the CI and
> > > > SparkInterpreter
> > > > > is
> > > > > > > not
> > > > > > > > > > > > helping to resolve these issues.
> > > > > > > > > > > >
> > > > > > > > > > > > If you think it's problem on Zeppelin core, then
> file
> > an
> > > > > issue
> > > > > > > that
> > > > > > > > > > > > reproduce the problem on Zeppelin core, that might
> be
> > > more
> > > > > > > efficient
> > > > > > > > > > than
> > > > > > > > > > > > keep trying yourself.
> > > > > > > > > > > > I contacted you numerous times about such issues...
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I remember i commented your issue about CI. but you
> just
> > > > keep
> > > > > > > repeated
> > > > > > > > > > it's
> > > > > > > > > > > not your problem but Zeppelin core problem.
> > > > > > > > > > >
> > > > > > > > > > > Then please file an issue about the problem you found
> in
> > > > > Zeppelin
> > > > > > > Core.
> > > > > > > > > > > Then everyone will get into the problem.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > In my interpretation, KnitRInterpreter is not an
> > optional
> > > > > > > feature while
> > > > > > > > > > > it
> > > > > > > > > > > > is always enabled when compiling Zeppelin and always
> > > > enabled
> > > > > when
> > > > > > > > > > running
> > > > > > > > > > > > Zeppelin. And it requires dynamically linked GPL
> > library
> > > > on
> > > > > > > runtime.
> > > > > > > > > > (yes
> > > > > > > > > > > > it will fail when no KnitR is installed on the
> system)
> > > > > > > > > > > >
> > > > > > > > > > > > Its not always enabled.
> > > > > > > > > > > > It is not dynamically linked at runtime.
> > > > > > > > > > > > It will not fail when knitr is missing. If knitr is
> not
> > > > > present,
> > > > > > > the
> > > > > > > > > > repl
> > > > > > > > > > > > interpreter starts and a note is written to to the
> log
> > > > that
> > > > > the
> > > > > > > knitr
> > > > > > > > > > > > interpreter isn’t available because knitr is not
> > present.
> > > > > > > > > > > >
> > > > > > > > > > > > no Apache code can ever call a shell script, on the
> > > > purpose
> > > > > of
> > > > > > > dynamic
> > > > > > > > > > > > linking with GPL library.
> > > > > > > > > > > > You misunderstand.
> > > > > > > > > > > >
> > > > > > > > > > > > The *shell* is GPL'd.
> > > > > > > > > > > >
> > > > > > > > > > > > Is Zeppelin “linked" against the GPL’d shell because
> > > > Zeppelin
> > > > > > > depends
> > > > > > > > > > on
> > > > > > > > > > > a
> > > > > > > > > > > > shell script to launch?
> > > > > > > > > > > >
> > > > > > > > > > > Obviously not.
> > > > > > > > > > > >
> > > > > > > > > > > > The interaction with R in the PR is the same.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Again, bash is one of exceptions of GPL, like other
> GPL
> > > > > licensed
> > > > > > > > > > > compiler/interpreter.
> > > > > > > > > > >
> > > > > > > > > > > Check here why Bash and R is okay with Apache License.
> > > > > > > > > > >
> > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > > > > >
> > > > > > > > > > > I'm not sure we can apply the same exception for
> 'using'
> > > > KnitR.
> > > > > > > > > > >
> > > > > > > > > > > My point is not 'KnitR' is optional or not. Point is
> > > > > > > 'KnitRInterpreter'
> > > > > > > > > > you
> > > > > > > > > > > wrote is not an optional feature. Which is clearly not
> > > > > optionally
> > > > > > > enabled
> > > > > > > > > > > code and feature. And that depends on KnitR library
> which
> > > is
> > > > > GPL.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > I was guessing SparkR can be still in Apache License
> > even
> > > > if
> > > > > it
> > > > > > > is
> > > > > > > > > > > depends
> > > > > > > > > > > > on R. Because of GPL licensed compiler generated
> output
> > > is
> > > > > not
> > > > > > > GPL
> > > > > > > > > > > license.
> > > > > > > > > > > > and R is sort of compiler. If you can get answer
> from
> > > > Spark
> > > > > > > community
> > > > > > > > > > how
> > > > > > > > > > > > SparkR get managed to stay in Apache License while R
> is
> > > > GPL,
> > > > > the
> > > > > > > answer
> > > > > > > > > > > > might help.
> > > > > > > > > > > > The description of SparkR is not accurate in any
> > respect.
> > > > (Do
> > > > > > > you think
> > > > > > > > > > > > SparkR is not talking to GPL-licensed libraries?)
> > > > > > > > > > > >
> > > > > > > > > > > > I don’t see that any genuine issue is being raised
> > here.
> > > > > > > > > > > >
> > > > > > > > > > > > If there is an issue, the burden is on you to
> identify
> > > it.
> > > > > > > > > > > >
> > > > > > > > > > > > If i give you one suggestion, Zeppelin committers
> > > > sometimes
> > > > > ask
> > > > > > > rebase
> > > > > > > > > > > the
> > > > > > > > > > > > contribution branch for some reason. It is not the
> > really
> > > > the
> > > > > > > best
> > > > > > > > > > > > practice, but still okay while most contributions
> are
> > not
> > > > > > > including
> > > > > > > > > > large
> > > > > > > > > > > > code base changes
> > > > > > > > > > > > However, your one, has more than 4000 lines of code
> > > > change.
> > > > > If
> > > > > > > you
> > > > > > > > > > rebase
> > > > > > > > > > > > then review should be started from the beginning,
> > again.
> > > > So
> > > > > you
> > > > > > > might
> > > > > > > > > > > want
> > > > > > > > > > > > to minimize the rebase your branch.
> > > > > > > > > > > >
> > > > > > > > > > > > Are you actually complaining that the problem is
> that I
> > > > > rebased
> > > > > > > the
> > > > > > > > > > code
> > > > > > > > > > > > during the three-month period when no-one looked at
> it
> > > and
> > > > > > > Zeppelin
> > > > > > > > > > went
> > > > > > > > > > > > through a release?
> > > > > > > > > > > >
> > > > > > > > > > > > I cannot take it seriously when you say things like
> > this.
> > > > > > > > > > > >
> > > > > > > > > > > > Having to “start from the beginning” cannot be a
> > problem
> > > > if
> > > > > you
> > > > > > > never
> > > > > > > > > > > > actually started the first time...
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > You wanted coordination and cooperation. So i gave you
> > > > > suggestion
> > > > > > > that
> > > > > > > > > > > helping review process. For example, your code has
> been
> > > > rebased
> > > > > > > since my
> > > > > > > > > > > comment and jongyoul's comment. that means committers
> > will
> > > > > need to
> > > > > > > look
> > > > > > > > > > > from the beginning. That'll require more time. And
> > maybe, i
> > > > > guess
> > > > > > > that's
> > > > > > > > > > > not what you want. But If you don't care, feel free to
> > > > rebase.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > moon
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > > Date: December 1, 2015 at 4:42:06 AM
> > > > > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > > incubator-zeppelin
> > > > > > > > > > pull
> > > > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <
> > > > > > > amos.elberg@gmail.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > Thank you, Cos.
> > > > > > > > > > > >
> > > > > > > > > > > > I’d like to briefly address the issues raised by
> Moon:
> > > > > > > > > > > >
> > > > > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > > > > The CI fails on core Zeppelin, *not* code in this
> PR.
> > > > > > > > > > > >
> > > > > > > > > > > > I’ve been seeking assistance on this since August.
> > > > > > > > > > > >
> > > > > > > > > > > > The most common reason is that SparkInterpreter is
> > unable
> > > > to
> > > > > > > launch
> > > > > > > > > > Spark
> > > > > > > > > > > > and open a Spark Backend. This is necessary to test
> the
> > > > PR.
> > > > > > > > > > > >
> > > > > > > > > > > > 60+ hours, has been spent adapting and re-basing
> when
> > the
> > > > > > > > > > > SparkInterpreter
> > > > > > > > > > > > architecture changed and broke the PR’s *tests.*
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's
> > > > problem
> > > > > on
> > > > > > > > > > Zeppelin
> > > > > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > > > > >
> > > > > > > > > > > > And let's say Zeppelin core has problem and that
> makes
> > > > your
> > > > > PR
> > > > > > > fails on
> > > > > > > > > > > CI
> > > > > > > > > > > > test. That's possible. But it still does not mean we
> > can
> > > > > merge
> > > > > > > it with
> > > > > > > > > > CI
> > > > > > > > > > > > failing.
> > > > > > > > > > > >
> > > > > > > > > > > > If you think it's problem on Zeppelin core, then
> file
> > an
> > > > > issue
> > > > > > > that
> > > > > > > > > > > > reproduce the problem on Zeppelin core, that might
> be
> > > more
> > > > > > > efficient
> > > > > > > > > > than
> > > > > > > > > > > > keep trying yourself.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > 2. Not 100% sure this PR has no license issue.
> (about
> > > > KniteR)
> > > > > > > > > > > > What license problem *specifically* do you believe
> may
> > > > exist?
> > > > > > > > > > > >
> > > > > > > > > > > > In preparing the PR, I:
> > > > > > > > > > > >
> > > > > > > > > > > > * Reviewed the Apache policy in detail.
> > > > > > > > > > > >
> > > > > > > > > > > > * Contacted the FSF to ask their interpretation of
> the
> > > > > “linking”
> > > > > > > > > > > > provisions of the GPL license.
> > > > > > > > > > > >
> > > > > > > > > > > > * Reviewed how other Apache software deals with this
> > > issue
> > > > > > > (e.g., Spark
> > > > > > > > > > > > talks to R, which is GPL'd).
> > > > > > > > > > > >
> > > > > > > > > > > > * No necessary *dependencies* of the PR have license
> > > > > conflicts.
> > > > > > > In
> > > > > > > > > > > > several cases, I contacted package authors who
> agreed
> > to
> > > > > > > re-issue their
> > > > > > > > > > > > packages under Apache-compatible licenses. (Usually
> I
> > had
> > > > to
> > > > > do
> > > > > > > a bit
> > > > > > > > > > of
> > > > > > > > > > > > coding in exchange…)
> > > > > > > > > > > >
> > > > > > > > > > > > * Where the license had to stay GPL, the packages
> are
> > > *not
> > > > > > > necessary*
> > > > > > > > > > and
> > > > > > > > > > > > *not dependencies.* If the optional packages are
> > present,
> > > > > they
> > > > > > > will be
> > > > > > > > > > > > used to enable additional functionality. Knitr is an
> > > > example.
> > > > > > > The PR
> > > > > > > > > > will
> > > > > > > > > > > > compile and run fine without knitr. If knitr is
> > available
> > > > > (it is
> > > > > > > part
> > > > > > > > > > of
> > > > > > > > > > > > the most common R distribution), the PR will enable
> the
> > > > knitr
> > > > > > > > > > > interpreter.
> > > > > > > > > > > >
> > > > > > > > > > > > * This is exactly how this issue is addressed
> through
> > the
> > > > > Apache
> > > > > > > > > > > > ecosystem.
> > > > > > > > > > > > The tl;dr is this: When Apache code is written to
> talk
> > to
> > > > > > > libraries
> > > > > > > > > > that
> > > > > > > > > > > > may or may not be present on the user’s system, or
> > where
> > > > it
> > > > > > > talks to an
> > > > > > > > > > > API
> > > > > > > > > > > > but is agnostic about implementation, that is not
> > > > “linking”
> > > > > in a
> > > > > > > way
> > > > > > > > > > that
> > > > > > > > > > > > implicate the anti-linking provision of the GPL.
> > > > > > > > > > > >
> > > > > > > > > > > > Otherwise, no Apache code could ever call a shell
> > script!
> > > > Let
> > > > > > > alone run
> > > > > > > > > > > > on Linux, or talk to R.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > I'm not a legal expert. So following could be wrong.
> > > > > > > > > > > >
> > > > > > > > > > > > In my interpretation, KnitRInterpreter is not an
> > optional
> > > > > > > feature while
> > > > > > > > > > > it
> > > > > > > > > > > > is always enabled when compiling Zeppelin and always
> > > > enabled
> > > > > when
> > > > > > > > > > running
> > > > > > > > > > > > Zeppelin. And it requires dynamically linked GPL
> > library
> > > > on
> > > > > > > runtime.
> > > > > > > > > > (yes
> > > > > > > > > > > > it will fail when no KnitR is installed on the
> system)
> > > > > > > > > > > >
> > > > > > > > > > > > And of course, no Apache code can ever call a shell
> > > > script,
> > > > > on
> > > > > > > the
> > > > > > > > > > > purpose
> > > > > > > > > > > > of dynamic linking with GPL library.
> > > > > > > > > > > >
> > > > > > > > > > > > I was guessing SparkR can be still in Apache License
> > even
> > > > if
> > > > > it
> > > > > > > is
> > > > > > > > > > > depends
> > > > > > > > > > > > on R. Because of GPL licensed compiler generated
> output
> > > is
> > > > > not
> > > > > > > GPL
> > > > > > > > > > > license.
> > > > > > > > > > > > and R is sort of compiler.
> > > > > > > > > > > >
> > > > > > > > > > > > If you can get answer from Spark community how
> SparkR
> > get
> > > > > > > managed to
> > > > > > > > > > stay
> > > > > > > > > > > > in Apache License while R is GPL, the answer might
> > help.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > 3. Need more time to review.
> > > > > > > > > > > > Has any reviewer has downloaded the PR or run the
> demo
> > > > > notebook?
> > > > > > > (Which
> > > > > > > > > > > > is there for the benefit of reviewers, and isn’t
> > intended
> > > > to
> > > > > go
> > > > > > > in
> > > > > > > > > > final
> > > > > > > > > > > > distribution.)
> > > > > > > > > > > >
> > > > > > > > > > > > How many +1 comments from users would you like to
> see?
> > > > > > > > > > > >
> > > > > > > > > > > > How much time do you believe is required?
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > It all depends on when CI is going to pass, when
> > license
> > > > > problem
> > > > > > > is
> > > > > > > > > > going
> > > > > > > > > > > > to be cleared, and when a committer willing to
> review
> > and
> > > > > > > responsible
> > > > > > > > > > to
> > > > > > > > > > > > commit your contribution.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > 1. Large code base change
> > > > > > > > > > > > Large code base changes require coordination and
> > > > cooperation.
> > > > > > > This PR
> > > > > > > > > > > > necessarily implicates the build scripts, testing
> code,
> > > > the
> > > > > > > > > > > > SparkInterpreter, etc.
> > > > > > > > > > > >
> > > > > > > > > > > > I have been seeking to coordinate since August.
> > > > > > > > > > > >
> > > > > > > > > > > > I continue to stand ready to do so.
> > > > > > > > > > > >
> > > > > > > > > > > > -Amos
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > If i give you one suggestion, Zeppelin committers
> > > > sometimes
> > > > > ask
> > > > > > > rebase
> > > > > > > > > > > the
> > > > > > > > > > > > contribution branch for some reason. It is not the
> > really
> > > > the
> > > > > > > best
> > > > > > > > > > > > practice, but still okay while most contributions
> are
> > not
> > > > > > > including
> > > > > > > > > > large
> > > > > > > > > > > > code base changes.
> > > > > > > > > > > >
> > > > > > > > > > > > However, your one, has more than 4000 lines of code
> > > > change.
> > > > > If
> > > > > > > you
> > > > > > > > > > rebase
> > > > > > > > > > > > then review should be started from the beginning,
> > again.
> > > > So
> > > > > you
> > > > > > > might
> > > > > > > > > > > want
> > > > > > > > > > > > to minimize the rebase your branch.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > > moon
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > > > Date: December 1, 2015 at 1:34:19 AM
> > > > > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > > > > > dev@zeppelin.incubator.apache.org
> > > > > > > > > > > >
> > > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > > incubator-zeppelin
> > > > > > > > > > pull
> > > > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > > > >
> > > > > > > > > > > > Hi Cos,
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks for opening a discussion.
> > > > > > > > > > > > My answer about 'Why this PR is open for three
> months'
> > is
> > > > > > > > > > > >
> > > > > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > > > > 2. Not 100% sure this PR has no license issue.
> (about
> > > > KniteR)
> > > > > > > > > > > > 3. Need more time to review.
> > > > > > > > > > > >
> > > > > > > > > > > > It's my personal answer, other committers may have
> > > > different
> > > > > > > opinion.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > I think the question should be generalized. Because
> > this
> > > > PR
> > > > > is
> > > > > > > not the
> > > > > > > > > > > only
> > > > > > > > > > > > PR that is in impasse. There're more. For example
> > > > > > > > > > > >
> > > > > > > > > > > > Here's some examples that PR is not been merged.
> > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,
>
> > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/60
> > > > > > > > > > > > and so on.
> > > > > > > > > > > >
> > > > > > > > > > > > I can categorize the cases, based on experience of
> > > > involving
> > > > > > > Zeppelin
> > > > > > > > > > > > community from the beginning.
> > > > > > > > > > > >
> > > > > > > > > > > > 1. Large code base change
> > > > > > > > > > > >
> > > > > > > > > > > > When contribution has large code base changes, it
> tend
> > to
> > > > > take
> > > > > > > more
> > > > > > > > > > time
> > > > > > > > > > > to
> > > > > > > > > > > > review and merged. Normally, most contributions
> merged
> > in
> > > > > 1day~1
> > > > > > > week.
> > > > > > > > > > > > But some contribution has large code base changes
> take
> > > 2~4
> > > > > > > weeks. Few
> > > > > > > > > > > > contribution that has very large code base change
> take
> > > > > months.
> > > > > > > > > > > >
> > > > > > > > > > > > 2. Communication lost
> > > > > > > > > > > >
> > > > > > > > > > > > Sometimes, committer is not responding, or
> contributor
> > is
> > > > not
> > > > > > > > > > responding.
> > > > > > > > > > > >
> > > > > > > > > > > > 3. Opinion diverges
> > > > > > > > > > > >
> > > > > > > > > > > > For some changes, included in contribution. When
> people
> > > > have
> > > > > > > different
> > > > > > > > > > > > opinion and it continue to diverges, those PRs are
> not
> > > > been
> > > > > > > merged.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > I think having a guide such as ping other committer
> > when
> > > a
> > > > > > > committer is
> > > > > > > > > > > not
> > > > > > > > > > > > responding, and divide contribution into small
> peaces
> > if
> > > > > > > possible,
> > > > > > > > > > would
> > > > > > > > > > > > help most of the cases.
> > > > > > > > > > > > Of course committer have to pay attention more to
> the
> > > > > > > contribution and
> > > > > > > > > > > > helping people. That's the first one.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > > moon
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <
> > > > > > > cos@apache.org>
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > To make sure we're on the same page, here are two
> > > > threads
> > > > > that
> > > > > > > I
> > > > > > > > > > found
> > > > > > > > > > > > > related
> > > > > > > > > > > > > to this topic.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thread 1:
> > > > > > > > > > > > > Subject: R?
> > > > > > > > > > > > > Started on: July 1, 2015
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thread 2:
> > > > > > > > > > > > > Subject: [GitHub] incubator-zeppelin pull request:
> R
> > > > > > > Interpreter for
> > > > > > > > > > > > > Zeppelin
> > > > > > > > > > > > > Started on: August 13, 2015
> > > > > > > > > > > > >
> > > > > > > > > > > > > If you want to fetch these from our archive send
> > emails
> > > > to
> > > > > > > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > > > > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > > > Cos
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin
> Boudnik
> > > > wrote:
> > > > > > > > > > > > > > Guys,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > While catching up on my emails from the last a
> > couple
> > > > of
> > > > > > > weeks,
> > > > > > > > > > this
> > > > > > > > > > > > > thread
> > > > > > > > > > > > > > caught my attention. I am not normally paying
> much
> > > > > attention
> > > > > > > to the
> > > > > > > > > > > > code
> > > > > > > > > > > > > > reviews traffic from GH, but this one is pretty
> > > > > different as
> > > > > > > it
> > > > > > > > > > spans
> > > > > > > > > > > > > three
> > > > > > > > > > > > > > months and counting.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > First, here are my five cents:
> > > > > > > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is
> > > aimed
> > > > > to be
> > > > > > > > > > > > > contributed to
> > > > > > > > > > > > > > an ASF project this file should simply contain
> ASL2
> > > > text,
> > > > > > > like in
> > > > > > > > > > [1]
> > > > > > > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate
> > > > > <developers>
> > > > > > > > > > > section,
> > > > > > > > > > > > > but
> > > > > > > > > > > > > > Zeppelin might have different guidelines on it.
> > Say,
> > > > > Bigtop
> > > > > > > doesn't
> > > > > > > > > > > > > > maintain this in the source code, but we have
> the
> > > list
> > > > of
> > > > > > > all the
> > > > > > > > > > > > > > committers on the project's site [2] Every new
> > > > > committer's
> > > > > > > first
> > > > > > > > > > > > > commit is
> > > > > > > > > > > > > > to update the team page ;)
> > > > > > > > > > > > > > - comments like in
> > > > > > > > > > > > >
> > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > +/**
> > > > > > > > > > > > > > + * Created by aelberg on 7/28/15.
> > > > > > > > > > > > > > + */
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > is better to be removed. It has been already
> > > discussed
> > > > > here
> > > > > > > [3].
> > > > > > > > > > And
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > initial discussion on the topic [4] was linked
> as
> > > well
> > > > > > > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am
> not
> > > > sure
> > > > > if
> > > > > > > this is
> > > > > > > > > > > > > R-specific
> > > > > > > > > > > > > > stuff - I have no idea about R, honestly, but
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file
> LICENSE
> > > > > > > > > > > > > > ...
> > > > > > > > > > > > > > +Author: David B. Dahl
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > shouldn't be here, IMO. Normally, if some
> > additional
> > > > > > > licenses are
> > > > > > > > > > > > > used,
> > > > > > > > > > > > > > they have to be listed in the top-level NOTICE
> file
> > > > > (already
> > > > > > > > > > there).
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > - I am not going to offer any comments on the
> > > > technical
> > > > > > > merits of
> > > > > > > > > > the
> > > > > > > > > > > > > patch,
> > > > > > > > > > > > > > as I haven't tried it personally. However, I
> don't
> > > see
> > > > > any
> > > > > > > serious
> > > > > > > > > > > > > > technical objections to the functionality in
> > > question.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > So, the question is - why the PR is open for
> three
> > > > > months? I
> > > > > > > hasn't
> > > > > > > > > > > > been
> > > > > > > > > > > > > able
> > > > > > > > > > > > > > to get a clear answer. What I found out though
> is
> > > > pretty
> > > > > > > > > > unsettling,
> > > > > > > > > > > > > really.
> > > > > > > > > > > > > > The communication on the topic is almost
> > > non-existent,
> > > > > > > except for
> > > > > > > > > > > this
> > > > > > > > > > > > > sparse
> > > > > > > > > > > > > > and bitter thread in the GH.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I would love to hear from the committers what's
> > > > stopping
> > > > > the
> > > > > > > > > > > acceptance
> > > > > > > > > > > > > of the
> > > > > > > > > > > > > > code, besides of the minor issues I've mentioned
> > > > earlier?
> > > > > > > What are
> > > > > > > > > > > the
> > > > > > > > > > > > > reasons for it?
> > > > > > > > > > > > > > Is there anything wrong with it?
> > > > > > > > > > > > > > One of the responsibilities of the committers is
> to
> > > > make
> > > > > > > sure the
> > > > > > > > > > > > > > contributions are reviewed; new contributors are
> > > > guided
> > > > > and
> > > > > > > do
> > > > > > > > > > > > > understand how
> > > > > > > > > > > > > > the project ticks. The easy feedback flow
> attracts
> > > new
> > > > > > > people,
> > > > > > > > > > > allowing
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > community to grow and thrive.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I am asking _explicitely_ not to re-start the
> > > > bickering I
> > > > > > > have
> > > > > > > > > > > already
> > > > > > > > > > > > > > seen. At this point I am interested in the
> purely
> > > > > technical
> > > > > > > side of
> > > > > > > > > > > > this.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > [1]
> > > > https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > > > > > > > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > > > > > > > > > > [3]
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > > > > > > > > > > [4] http://s.apache.org/iZl
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > With regards,
> > > > > > > > > > > > > > Cos
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > > > > > > > > > > Github user elbamos commented on the pull
> > request:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The current push should resolve some issues
> with
> > > > > changes
> > > > > > > in the
> > > > > > > > > > > > > > > Spark-Zeppelin interface that had created
> issues
> > > for
> > > > > > > users, as
> > > > > > > > > > > > > well as
> > > > > > > > > > > > > > > support for 1.5.1.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Knitr support is improved, and the reason for
> a
> > > > > separate
> > > > > > > knitr
> > > > > > > > > > > > > interpreter may be clearer now.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The amount of code borrowed from rscala is
> > reduced.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I did not address issues with @author tags, or
> > > files
> > > > > under
> > > > > > > the R/
> > > > > > > > > > > > > > > folder. The reason is, to be blunt, I don't
> > > > understand
> > > > > > > what the
> > > > > > > > > > > > > precise
> > > > > > > > > > > > > > > concerns actually are.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Please note that I changed .travis.yml to only
> > use
> > > > > spark
> > > > > > > 1.4 and
> > > > > > > > > > > > > later.
> > > > > > > > > > > > > > > I'm sure there is a better way to do it, but
> I'm
> > > > just
> > > > > not
> > > > > > > in a
> > > > > > > > > > > > > position
> > > > > > > > > > > > > > > to try to figure out the entire Zeppelin build
> > > > process.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Integrating this with the rest of zeppelin is
> > going
> > > > to
> > > > > > > take some
> > > > > > > > > > > > > work
> > > > > > > > > > > > > > > regarding pom's, travis, and so forth. I can
> do a
> > > > lot
> > > > > of
> > > > > > > that,
> > > > > > > > > > > > > but I'm
> > > > > > > > > > > > > > > going to need to discuss it with the people
> who
> > > have
> > > > > been
> > > > > > > > > > "owning"
> > > > > > > > > > > > > those
> > > > > > > > > > > > > > > files. There are just too many moving pieces
> > here.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Because of the size of this (which is,
> > > > unfortunately,
> > > > > > > necessary),
> > > > > > > > > > > > > > > posting here is probably not the most
> efficient
> > > way.
> > > > > That
> > > > > > > is also
> > > > > > > > > > > > > true
> > > > > > > > > > > > > > > because certain people chose to use this PR as
> a
> > > > forum
> > > > > to
> > > > > > > air
> > > > > > > > > > other
> > > > > > > > > > > > > > > issues. Therefore, it would be better if
> > reviewers
> > > > > e-mail
> > > > > > > me
> > > > > > > > > > > > > directly.
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > > >
> > >
> >
>
>

Re: R-Interpreter CI Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Thanks, Moon.  

If you look at the code now, the reason travis fails is because the travis.yaml is messed-up from me trying to get travis not to fail.  But with that fixed it only fails for other reasons…

I have seen both the failures you describe, but I’ve also seen a bunch of others.  The most common issue is that the PR, to test, needs to instantiate a SparkInterpreter instance without the infrastructure of the Zeppelin server, and Spark fails to start.

As a practical way to move forward I propose the following:

1.  The CI issues are broader than this PR, and we should start a separate thread to discuss and fix CI in general. 

2.  I’m going to close the current PR, and make a new PR that is based on the 0.5.5 release code.  This PR will *not* be sync’d with Zeppelin-Master.  It will be sync’d w/ 0.5.5 release because that is a stable base.  I would appreciate if people could help me with CI, using that as a base. 

-- 
Amos Elberg
Sent with Airmail

From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 8, 2015 at 9:26:38 AM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: R-Interpreter CI Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

fyi, some cases of CI failing, when PR does not impact the code,  

1) Flickering test. Some tests are passing sometimes, failing the other  
times. (e.g. ZEPPELIN-476)  
2) Download fails. Downloading maven artifacts or other requirements from  
internet are sometimes failing  

both cases pass if CI triggered again.  

Thanks,  
moon  

On Tue, Dec 8, 2015 at 6:29 PM DuyHai Doan <do...@gmail.com> wrote:  

> I have also experienced CI failing in the pass for some PRs that do not  
> impact the code (Cassandra documentation). I guess that during peak hours,  
> the CI servers may be too busy or enter a dead lock so the build fails.  
>  
> At least that's my guess. What do you think ?  
>  
> On Tue, Dec 8, 2015 at 9:42 AM, moon soo Lee <mo...@apache.org> wrote:  
>  
> > Let me run some CI test with your branch and share result here.  
> > Hope i can narrow down the cause and that helps involvement of more  
> people.  
> >  
> > Thanks,  
> > moon  
> >  
> > On Tue, Dec 8, 2015 at 3:08 PM Amos B. Elberg <am...@gmail.com>  
> > wrote:  
> >  
> > > Is there anyone who can work with me on the CI issues? It looks like  
> > > there are a number of PRs experiencing similar things.  
> > >  
> > > I think we should make getting CI stable to be a priority. Because it  
> > > will save everyone a lot of frustration and aggravation if CI works  
> > > reliably.  
> > >  
> > > Is there anyone other than Jongyoul and Moon who knows the CI/Build  
> > > process well?  
> > >  
> > > (Moon - Thank you for taking another look at the licensing issue. Per  
> > the  
> > > e-mail I wrote about this a few days ago, I don’t feel I have more to  
> > > contribute to the licensing discussion, so I’m going to try not to  
> > comment  
> > > further about it.)  
> > >  
> > > From: moon soo Lee <mo...@apache.org> <mo...@apache.org>  
> > > Reply: dev@zeppelin.incubator.apache.org  
> > > <de...@zeppelin.incubator.apache.org> <dev@zeppelin.incubator.apache.org  
> >  
> > > Date: December 7, 2015 at 5:00:08 PM  
> > > To: dev@zeppelin.incubator.apache.org <  
> dev@zeppelin.incubator.apache.org  
> > >  
> > > <de...@zeppelin.incubator.apache.org>  
> > > Subject: Re: License of KnitRInterpreter Was: Re: contributions  
> impasse.  
> > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for  
> Zeppelin  
> > >  
> > > Thanks Amos, Roman, Cos for clarifying license issue.  
> > >  
> > > I'm convinced that this license issue will not be a blocker.  
> > >  
> > > In my understanding, these are good sign,  
> > >  
> > > 1. any gpl licensed source codes are not included in the source package  
> > > 2. any gpl licensed libraries are not included in the binary package  
> > >  
> > > However, i can not still 100% sure about  
> > >  
> > > 3. any gpl licensed libraries are not linked on runtime  
> > >  
> > > Even after Amos's explanation. I still think using 'knitr' is one of  
> the  
> > > clear case that 'knir' is linked to 'R' according to  
> > > http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL.  
> > >  
> > > Giving input and getting output from GPL licensed interpreter (includes  
> > R)  
> > > from Apache licensed software is not a problem. That's not the point.  
> > > Let me say in this way. There's an java code,  
> > >  
> > > import com.mysql.jdbc.Driver  
> > > Driver driver = new Driver()  
> > >  
> > > Say without this java code, one of important feature of Zeppelin does  
> not  
> > > work. And Zeppelin does not includes GPL licensed file in the source  
> > > package, GPL licensed library in the binary package, but it requires  
> GPL  
> > > licensed library on the runtime.  
> > > In this case, will this java code be a license problem or not?  
> > >  
> > > In other words, my question is  
> > >  
> > > a) Is runtime GPL library dependency allowed in ASF release?  
> > > b) is 'knitr' considered as runtime dependency?  
> > >  
> > > If someone can clarify a), b), then it would be extremely helpful  
> > > understanding this case, and possible similar cases, too.  
> > >  
> > > Thanks,  
> > > moon  
> > >  
> > > On Mon, Dec 7, 2015 at 3:20 PM Konstantin Boudnik <co...@apache.org>  
> > wrote:  
> > >  
> > > > On Thu, Dec 03, 2015 at 11:03AM, Corneau Damien wrote:  
> > > > > Thanks Cos for those answers,  
> > > > >  
> > > > > If I'm right you are advising to have a default build that doesn't  
> > > > include  
> > > > > libraries with conflicting licenses, but have an option to include  
> > > them  
> > > > for  
> > > > > users who wants to build the project themselves.  
> > > >  
> > > > Yes, that's what I said. Besides, looks like Roman provided the  
> second  
> > > > pair of  
> > > > eyes to this licensing discussion and as well didn't find any issues  
> > > with  
> > > > the  
> > > > current approach.  
> > > >  
> > > > Cos  
> > > >  
> > > > > To refer to another thread about decentralizing interpreters, it  
> > could  
> > > > even  
> > > > > be better in that case to have some interpreters separated from the  
> > > tree,  
> > > > > and easily pluggable with a release instead of forcing users to  
> build  
> > > the  
> > > > > project to use them.  
> > > > >  
> > > > >  
> > > > > On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <cos@apache.org  
> >  
> > > > wrote:  
> > > > >  
> > > > > > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:  
> > > > > > > Konstantin thank you for getting into this.  
> > > > > > >  
> > > > > > >> The best way to go around it is by  
> > > > > > >> providing a build-time option that will pull such binaries in.  
> > > But  
> > > > by  
> > > > > > default  
> > > > > > >> such libs shouldn't be pulled.  
> > > > > > >  
> > > > > > > That is basically how the PR handles this. If the GPL’d  
> > > interpreter  
> > > > > > scripts  
> > > > > > > are missing, there’s no indication at all at build time. It  
> > > doesn’t  
> > > > try  
> > > > > > to  
> > > > > > > download them.  
> > > > > > >  
> > > > > > > At runtime, if the user tries to use functionality that would  
> > need  
> > > > such a  
> > > > > > > script (i.e., if they type “knitr” to use knitr), we display an  
> > > error  
> > > > > > that  
> > > > > > > says that the functionality is not there because the library is  
> > > > missing,  
> > > > > > and  
> > > > > > > that the library cannot be provided because it has an  
> > incompatible  
> > > > > > license,  
> > > > > > > but the user can download it themselves if they want.  
> > > > > > >  
> > > > > > > And, in the log, if the logging level is high, they will see a  
> > > note  
> > > > that  
> > > > > > > some functionality was disabled because the libraries weren’t  
> > > there.  
> > > > > > >  
> > > > > > > To be clear, though, none of these libraries are binaries.  
> > They’re  
> > > > all  
> > > > > > interpreter scripts.  
> > > > > >  
> > > > > > If you ever in a doubt of which licenses could be used for  
> > > dependncies  
> > > > > > (not to  
> > > > > > say about source code) are listed in Category A list of [1]  
> > > > > >  
> > > > > > A lot of quesitons discussed here are already covered in the  
> legal  
> > > > FAQ, so  
> > > > > > just check against it if you have any questions.  
> > > > > >  
> > > > > > [1] http://www.apache.org/legal/resolved.html#category-a  
> > > > > >  
> > > > > > Cos  
> > > > > >  
> > > > > > > From: Konstantin Boudnik <co...@apache.org>  
> > > > > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > > > dev@zeppelin.incubator.apache.org>,  
> > > dev@zeppelin.incubator.apache.org  
> > > > <  
> > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > > Date: December 2, 2015 at 3:24:50 PM  
> > > > > > > To: dev@zeppelin.incubator.apache.org <  
> > > > dev@zeppelin.incubator.apache.org  
> > > > > > >  
> > > > > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions  
> > > > > > impasse. Was: [GitHub] incubator-zeppelin pull request: R  
> > > Interpreter  
> > > > for  
> > > > > > Zeppelin  
> > > > > > >  
> > > > > > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:  
> > > > > > > > I think that what moon means is that:  
> > > > > > > > If we merge the way it is now, the KnitRInterpreter will be  
> > part  
> > > > of the  
> > > > > > > > code base, so it isn't optional at code base level.  
> > > > > > > >  
> > > > > > > > To make it even more simple:  
> > > > > > > > * If the code has the right licensing -> that code can be  
> part  
> > > of  
> > > > the  
> > > > > > > > source code, and can be including in a binary release  
> > > > > > >  
> > > > > > > We aren't concerned with binary releases - as an Apache  
> community  
> > > we  
> > > > are  
> > > > > > > voting and releasing source code. If the project wants to  
> provide  
> > > a  
> > > > > > binary  
> > > > > > > release to its users, they are better be warned about inclusion  
> > of  
> > > > non  
> > > > > > > ASL2-friendly licensed bits.  
> > > > > > >  
> > > > > > > > * If the code doesn't have the right licensing -> it can't be  
> > > part  
> > > > of  
> > > > > > the  
> > > > > > > > source code, and can't be included in a binary release  
> > > > > > >  
> > > > > > > See above.  
> > > > > > >  
> > > > > > > > * If the code doesn't have the right licensing but is  
> imported  
> > > at  
> > > > build  
> > > > > > > > time (libraries for example) -> it is not in the source code,  
> > it  
> > > > can't  
> > > > > > be  
> > > > > > > > included in binary release  
> > > > > > >  
> > > > > > > That is unless a user doing it on his own. The best way to go  
> > > around  
> > > > it  
> > > > > > is by  
> > > > > > > providing a build-time option that will pull such binaries in.  
> > But  
> > > by  
> > > > > > default  
> > > > > > > such libs shouldn't be pulled.  
> > > > > > >  
> > > > > > > Cos  
> > > > > > >  
> > > > > > > > So in the case of licensing issues, it would need to be fully  
> > > > optional  
> > > > > > > > (user bring the interpreter in his directory and build  
> Zeppelin  
> > > > with  
> > > > > > it)  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <  
> > > > amos.elberg@gmail.com>  
> > > > > > > > wrote:  
> > > > > > > >  
> > > > > > > > > Moon let me clarify:  
> > > > > > > > >  
> > > > > > > > > Interpreted code doesn’t “link.” The wiki article actually  
> > > > explains  
> > > > > > it  
> > > > > > > > > pretty well —  
> > > > https://en.wikipedia.org/wiki/GPL_linking_exception  
> > > > > > > > > “Linking” against a library means compiling its headers  
> into  
> > a  
> > > > > > binary, the  
> > > > > > > > > way a C compiler works. The 2008 e-mail Moon distributed,  
> > > called  
> > > > > > this the  
> > > > > > > > > “interpreter exception.”  
> > > > > > > > >  
> > > > > > > > > As for whether GPL’d code is a “mandatory dependency,” if  
> > > knitr  
> > > > is  
> > > > > > missing  
> > > > > > > > > the PR will compile, run and test just fine. The user is  
> > never  
> > > > > > prompted to  
> > > > > > > > > download it. The only effect is, if the user types “knitr”  
> > and  
> > > > knitr  
> > > > > > isn’t  
> > > > > > > > > there, we display an InterpreterError that knitr isn’t  
> there.  
> > > > > > > > >  
> > > > > > > > > KnitRInterpreter is not optionally required. so it does not  
> > > > matter  
> > > > > > KnitR  
> > > > > > > > > is  
> > > > > > > > > optionally required or not.  
> > > > > > > > > Aren’t all interpreters optional? You can turn them on and  
> > off  
> > > > in the  
> > > > > > > > > config files.  
> > > > > > > > >  
> > > > > > > > > Do you mean that the KnitRInterpreter class gets compiled  
> to  
> > > > > > bytecode even  
> > > > > > > > > if knitr is missing? So what? That isn't a mandatory  
> > > dependency  
> > > > or a  
> > > > > > link.  
> > > > > > > > >  
> > > > > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > > > > Date: December 2, 2015 at 3:18:00 AM  
> > > > > > > > > To: dev@zeppelin.incubator.apache.org <  
> > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > > > > Subject: Re: License of KnitRInterpreter Was: Re:  
> > > contributions  
> > > > > > impasse.  
> > > > > > > > > Was: [GitHub] incubator-zeppelin pull request: R  
> Interpreter  
> > > for  
> > > > > > Zeppelin  
> > > > > > > > >  
> > > > > > > > > Let me summarize license concern about KnitRInterpreter.  
> > > > > > > > >  
> > > > > > > > >  
> > > > > > > > > Amos's interpretation.  
> > > > > > > > >  
> > > > > > > > > KnitR is optionally required by KnitRInterpreter.  
> > > > > > > > > R dependency in SparkR has no problem. So KnitR should be  
> the  
> > > > same.  
> > > > > > > > >  
> > > > > > > > >  
> > > > > > > > > Moon's interpretation.  
> > > > > > > > >  
> > > > > > > > > KnitRInterpreter is not optionally required. so it does not  
> > > > matter  
> > > > > > KnitR is  
> > > > > > > > > optionally required or not.  
> > > > > > > > > R dependency in SparkR is exception of GPL. KnitR is not  
> > > applied  
> > > > that  
> > > > > > > > > exception.  
> > > > > > > > >  
> > > > > > > > >  
> > > > > > > > > Amos, could you confirm my understanding to your  
> > > interpretation  
> > > > is  
> > > > > > correct?  
> > > > > > > > > If it is not could you clarify it?  
> > > > > > > > >  
> > > > > > > > > Thanks,  
> > > > > > > > > moon  
> > > > > > > > >  
> > > > > > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <  
> > > > > > amos.elberg@gmail.com>  
> > > > > > > > > wrote:  
> > > > > > > > >  
> > > > > > > > > > Just to put the final nail in this, I looked it up.  
> > > > > > > > > >  
> > > > > > > > > > I see no evidence of any “compiler exception.”  
> > > > > > > > > >  
> > > > > > > > > > There is an exception in the license for the runtime  
> > > libraries  
> > > > > > that are  
> > > > > > > > > > bundled with GCC. See:  
> > > > > > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html  
> > > > > > > > > >  
> > > > > > > > > > But no “compiler exception.”  
> > > > > > > > > >  
> > > > > > > > > > In fact, I believe this is part of the reason why LLVM  
> was  
> > > > created.  
> > > > > > > > > >  
> > > > > > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > > > > > Reply: moon soo Lee <mo...@apache.org>  
> > > > > > > > > > Date: December 1, 2015 at 8:16:36 PM  
> > > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,  
> > > > > > > > > > dev@zeppelin.incubator.apache.org <  
> > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > > > > > incubator-zeppelin pull  
> > > > > > > > > > request: R Interpreter for Zeppelin  
> > > > > > > > > >  
> > > > > > > > > > Is knitR is commonly considered as a  
> interpreter/compiler?  
> > > or  
> > > > is it  
> > > > > > > > > > considered as a library routine?  
> > > > > > > > > >  
> > > > > > > > > > Thanks,  
> > > > > > > > > > moon  
> > > > > > > > > >  
> > > > > > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <  
> > > > > > amos.elberg@gmail.com>  
> > > > > > > > > > wrote:  
> > > > > > > > > > Moon - you give this as an explanation of the licensing  
> > > issue:  
> > > > > > > > > >  
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
> > > > > > > > > >  
> > > > > > > > > > According to that, there is an exception in the GPL for  
> > > > interpreter  
> > > > > > > > > > languages. As long as you don’t distribute the code, its  
> > > fine  
> > > > to  
> > > > > > talk to  
> > > > > > > > > > an interpreted language.  
> > > > > > > > > >  
> > > > > > > > > > Well, if that’s the case, then the PR plainly does not  
> have  
> > > a  
> > > > > > license  
> > > > > > > > > > issue. It doesn’t distribute any GPL’d R code.  
> > > > > > > > > >  
> > > > > > > > > > I’m not sure what’s confusing about this. It seems  
> > > completely  
> > > > > > > > > > straightforward.  
> > > > > > > > > >  
> > > > > > > > > > Regarding this:  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > --  
> > > > > > > > > > Amos Elberg  
> > > > > > > > > > Sent with Airmail  
> > > > > > > > > >  
> > > > > > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > > > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > > > > > Date: December 1, 2015 at 6:48:47 PM  
> > > > > > > > > >  
> > > > > > > > > > To: dev@zeppelin.incubator.apache.org <  
> > > > > > dev@zeppelin.incubator.apache.org  
> > > > > > > > > >  
> > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > > > > > incubator-zeppelin pull  
> > > > > > > > > > request: R Interpreter for Zeppelin  
> > > > > > > > > >  
> > > > > > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <  
> > > > > > amos.elberg@gmail.com>  
> > > > > > > > > > wrote:  
> > > > > > > > > >  
> > > > > > > > > > > I am going to try to minimize my reaction to Moon’s  
> > > e-mail.  
> > > > > > > > > > >  
> > > > > > > > > > > The tl;dr is this:  
> > > > > > > > > > >  
> > > > > > > > > > > The reason we are having this discussion now is that  
> > > active  
> > > > > > users of  
> > > > > > > > > the  
> > > > > > > > > > > PR — which now has its own user base — went public to  
> > > > complain  
> > > > > > about  
> > > > > > > > > > this.  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > > The PR has been tested by an active user base for more  
> > > than  
> > > > three  
> > > > > > > > > months.  
> > > > > > > > > > > No-one has been able to identify any specific actual  
> > > > licensing  
> > > > > > problem,  
> > > > > > > > > > and  
> > > > > > > > > > > the PR was prepared based on an extensive, careful  
> review  
> > > of  
> > > > the  
> > > > > > > > > relevant  
> > > > > > > > > > > licensing issues and after contacting the relevant  
> > people.  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > I admire every software that used by user and helping  
> > > people.  
> > > > That  
> > > > > > > > > includes  
> > > > > > > > > > your work. But that's not the topic we're in discussion.  
> > > Active  
> > > > > > user does  
> > > > > > > > > > not mean your contribution can ignore the review.  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > > It is not an explanation for someone who has been  
> > ignoring  
> > > my  
> > > > > > “how can  
> > > > > > > > > I  
> > > > > > > > > > > move this forward…” emails for three months to point  
> the  
> > > > finger  
> > > > > > and  
> > > > > > > > > say I  
> > > > > > > > > > > didn’t contact the right person or file the right  
> report.  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > This is also not the topic in this discussion.  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > > The burden for providing an explanation for the  
> inaction  
> > > is  
> > > > on  
> > > > > > the PMCC  
> > > > > > > > > > at  
> > > > > > > > > > > this point.  
> > > > > > > > > >  
> > > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's  
> > problem  
> > > on  
> > > > > > Zeppelin  
> > > > > > > > > > > core, why do you think other PRs are passing CI?  
> > > > > > > > > > > They’re not! I often see comments on PRs to just ignore  
> > > that  
> > > > CI  
> > > > > > is  
> > > > > > > > > > > failing.  
> > > > > > > > > > >  
> > > > > > > > > > > One of the most common reasons this PR fails CI, is CI  
> > > > times-out  
> > > > > > > > > > > downloading Spark to install. How could that possibly  
> be  
> > > > caused  
> > > > > > by the  
> > > > > > > > > > PR?  
> > > > > > > > > > >  
> > > > > > > > > > > It looks to me like the only PRs with changes to the  
> > > relevant  
> > > > > > parts of  
> > > > > > > > > > the  
> > > > > > > > > > > code — the SparkInterpreter — are being made by the  
> > person  
> > > > who  
> > > > > > wrote  
> > > > > > > > > the  
> > > > > > > > > > > testing suite.  
> > > > > > > > > > >  
> > > > > > > > > > > So, that would explain why some other PRs pass CI:  
> > Neither  
> > > > the  
> > > > > > > > > > > SparkInterpreter nor the testing suite are stable or  
> > > robust,  
> > > > but  
> > > > > > since  
> > > > > > > > > > the  
> > > > > > > > > > > PRs are coming from the person who wrote both…  
> > > > > > > > > > >  
> > > > > > > > > > > And let's say Zeppelin core has problem and that makes  
> > > your  
> > > > PR  
> > > > > > fails on  
> > > > > > > > > > CI  
> > > > > > > > > > > test. That's possible. But it still does not mean we  
> can  
> > > > merge  
> > > > > > it with  
> > > > > > > > > CI  
> > > > > > > > > > > failing.  
> > > > > > > > > > >  
> > > > > > > > > > > It means you should be working with me to figure out  
> why  
> > > the  
> > > > CI  
> > > > > > is  
> > > > > > > > > > failing.  
> > > > > > > > > > >  
> > > > > > > > > > > This PR has been tested by an active user base for the  
> > > past  
> > > > three  
> > > > > > > > > months.  
> > > > > > > > > > > If CI is continuing to fail, and dozens of hours of  
> > effort  
> > > > have  
> > > > > > not  
> > > > > > > > > > > resolved the CI issues, then it is time to start  
> > > considering  
> > > > > > whether  
> > > > > > > > > the  
> > > > > > > > > > > testing suite is part of the problem.  
> > > > > > > > > > >  
> > > > > > > > > > > The level of defensiveness about the CI and  
> > > SparkInterpreter  
> > > > is  
> > > > > > not  
> > > > > > > > > > > helping to resolve these issues.  
> > > > > > > > > > >  
> > > > > > > > > > > If you think it's problem on Zeppelin core, then file  
> an  
> > > > issue  
> > > > > > that  
> > > > > > > > > > > reproduce the problem on Zeppelin core, that might be  
> > more  
> > > > > > efficient  
> > > > > > > > > than  
> > > > > > > > > > > keep trying yourself.  
> > > > > > > > > > > I contacted you numerous times about such issues...  
> > > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > I remember i commented your issue about CI. but you just  
> > > keep  
> > > > > > repeated  
> > > > > > > > > it's  
> > > > > > > > > > not your problem but Zeppelin core problem.  
> > > > > > > > > >  
> > > > > > > > > > Then please file an issue about the problem you found in  
> > > > Zeppelin  
> > > > > > Core.  
> > > > > > > > > > Then everyone will get into the problem.  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > In my interpretation, KnitRInterpreter is not an  
> optional  
> > > > > > feature while  
> > > > > > > > > > it  
> > > > > > > > > > > is always enabled when compiling Zeppelin and always  
> > > enabled  
> > > > when  
> > > > > > > > > running  
> > > > > > > > > > > Zeppelin. And it requires dynamically linked GPL  
> library  
> > > on  
> > > > > > runtime.  
> > > > > > > > > (yes  
> > > > > > > > > > > it will fail when no KnitR is installed on the system)  
> > > > > > > > > > >  
> > > > > > > > > > > Its not always enabled.  
> > > > > > > > > > > It is not dynamically linked at runtime.  
> > > > > > > > > > > It will not fail when knitr is missing. If knitr is not  
> > > > present,  
> > > > > > the  
> > > > > > > > > repl  
> > > > > > > > > > > interpreter starts and a note is written to to the log  
> > > that  
> > > > the  
> > > > > > knitr  
> > > > > > > > > > > interpreter isn’t available because knitr is not  
> present.  
> > > > > > > > > > >  
> > > > > > > > > > > no Apache code can ever call a shell script, on the  
> > > purpose  
> > > > of  
> > > > > > dynamic  
> > > > > > > > > > > linking with GPL library.  
> > > > > > > > > > > You misunderstand.  
> > > > > > > > > > >  
> > > > > > > > > > > The *shell* is GPL'd.  
> > > > > > > > > > >  
> > > > > > > > > > > Is Zeppelin “linked" against the GPL’d shell because  
> > > Zeppelin  
> > > > > > depends  
> > > > > > > > > on  
> > > > > > > > > > a  
> > > > > > > > > > > shell script to launch?  
> > > > > > > > > > >  
> > > > > > > > > > Obviously not.  
> > > > > > > > > > >  
> > > > > > > > > > > The interaction with R in the PR is the same.  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > Again, bash is one of exceptions of GPL, like other GPL  
> > > > licensed  
> > > > > > > > > > compiler/interpreter.  
> > > > > > > > > >  
> > > > > > > > > > Check here why Bash and R is okay with Apache License.  
> > > > > > > > > >  
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
> > > > > > > > > >  
> > > > > > > > > > I'm not sure we can apply the same exception for 'using'  
> > > KnitR.  
> > > > > > > > > >  
> > > > > > > > > > My point is not 'KnitR' is optional or not. Point is  
> > > > > > 'KnitRInterpreter'  
> > > > > > > > > you  
> > > > > > > > > > wrote is not an optional feature. Which is clearly not  
> > > > optionally  
> > > > > > enabled  
> > > > > > > > > > code and feature. And that depends on KnitR library which  
> > is  
> > > > GPL.  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > > I was guessing SparkR can be still in Apache License  
> even  
> > > if  
> > > > it  
> > > > > > is  
> > > > > > > > > > depends  
> > > > > > > > > > > on R. Because of GPL licensed compiler generated output  
> > is  
> > > > not  
> > > > > > GPL  
> > > > > > > > > > license.  
> > > > > > > > > > > and R is sort of compiler. If you can get answer from  
> > > Spark  
> > > > > > community  
> > > > > > > > > how  
> > > > > > > > > > > SparkR get managed to stay in Apache License while R is  
> > > GPL,  
> > > > the  
> > > > > > answer  
> > > > > > > > > > > might help.  
> > > > > > > > > > > The description of SparkR is not accurate in any  
> respect.  
> > > (Do  
> > > > > > you think  
> > > > > > > > > > > SparkR is not talking to GPL-licensed libraries?)  
> > > > > > > > > > >  
> > > > > > > > > > > I don’t see that any genuine issue is being raised  
> here.  
> > > > > > > > > > >  
> > > > > > > > > > > If there is an issue, the burden is on you to identify  
> > it.  
> > > > > > > > > > >  
> > > > > > > > > > > If i give you one suggestion, Zeppelin committers  
> > > sometimes  
> > > > ask  
> > > > > > rebase  
> > > > > > > > > > the  
> > > > > > > > > > > contribution branch for some reason. It is not the  
> really  
> > > the  
> > > > > > best  
> > > > > > > > > > > practice, but still okay while most contributions are  
> not  
> > > > > > including  
> > > > > > > > > large  
> > > > > > > > > > > code base changes  
> > > > > > > > > > > However, your one, has more than 4000 lines of code  
> > > change.  
> > > > If  
> > > > > > you  
> > > > > > > > > rebase  
> > > > > > > > > > > then review should be started from the beginning,  
> again.  
> > > So  
> > > > you  
> > > > > > might  
> > > > > > > > > > want  
> > > > > > > > > > > to minimize the rebase your branch.  
> > > > > > > > > > >  
> > > > > > > > > > > Are you actually complaining that the problem is that I  
> > > > rebased  
> > > > > > the  
> > > > > > > > > code  
> > > > > > > > > > > during the three-month period when no-one looked at it  
> > and  
> > > > > > Zeppelin  
> > > > > > > > > went  
> > > > > > > > > > > through a release?  
> > > > > > > > > > >  
> > > > > > > > > > > I cannot take it seriously when you say things like  
> this.  
> > > > > > > > > > >  
> > > > > > > > > > > Having to “start from the beginning” cannot be a  
> problem  
> > > if  
> > > > you  
> > > > > > never  
> > > > > > > > > > > actually started the first time...  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > You wanted coordination and cooperation. So i gave you  
> > > > suggestion  
> > > > > > that  
> > > > > > > > > > helping review process. For example, your code has been  
> > > rebased  
> > > > > > since my  
> > > > > > > > > > comment and jongyoul's comment. that means committers  
> will  
> > > > need to  
> > > > > > look  
> > > > > > > > > > from the beginning. That'll require more time. And  
> maybe, i  
> > > > guess  
> > > > > > that's  
> > > > > > > > > > not what you want. But If you don't care, feel free to  
> > > rebase.  
> > > > > > > > > >  
> > > > > > > > > > Thanks,  
> > > > > > > > > > moon  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > > > > > > Reply: moon soo Lee <mo...@apache.org>  
> > > > > > > > > > > Date: December 1, 2015 at 4:42:06 AM  
> > > > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,  
> > > > > > > > > > > dev@zeppelin.incubator.apache.org <  
> > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > > > > > incubator-zeppelin  
> > > > > > > > > pull  
> > > > > > > > > > > request: R Interpreter for Zeppelin  
> > > > > > > > > > >  
> > > > > > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <  
> > > > > > amos.elberg@gmail.com>  
> > > > > > > > > > > wrote:  
> > > > > > > > > > > Thank you, Cos.  
> > > > > > > > > > >  
> > > > > > > > > > > I’d like to briefly address the issues raised by Moon:  
> > > > > > > > > > >  
> > > > > > > > > > > 1. This PR does not passes CI  
> > > > > > > > > > > The CI fails on core Zeppelin, *not* code in this PR.  
> > > > > > > > > > >  
> > > > > > > > > > > I’ve been seeking assistance on this since August.  
> > > > > > > > > > >  
> > > > > > > > > > > The most common reason is that SparkInterpreter is  
> unable  
> > > to  
> > > > > > launch  
> > > > > > > > > Spark  
> > > > > > > > > > > and open a Spark Backend. This is necessary to test the  
> > > PR.  
> > > > > > > > > > >  
> > > > > > > > > > > 60+ hours, has been spent adapting and re-basing when  
> the  
> > > > > > > > > > SparkInterpreter  
> > > > > > > > > > > architecture changed and broke the PR’s *tests.*  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's  
> > > problem  
> > > > on  
> > > > > > > > > Zeppelin  
> > > > > > > > > > > core, why do you think other PRs are passing CI?  
> > > > > > > > > > >  
> > > > > > > > > > > And let's say Zeppelin core has problem and that makes  
> > > your  
> > > > PR  
> > > > > > fails on  
> > > > > > > > > > CI  
> > > > > > > > > > > test. That's possible. But it still does not mean we  
> can  
> > > > merge  
> > > > > > it with  
> > > > > > > > > CI  
> > > > > > > > > > > failing.  
> > > > > > > > > > >  
> > > > > > > > > > > If you think it's problem on Zeppelin core, then file  
> an  
> > > > issue  
> > > > > > that  
> > > > > > > > > > > reproduce the problem on Zeppelin core, that might be  
> > more  
> > > > > > efficient  
> > > > > > > > > than  
> > > > > > > > > > > keep trying yourself.  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > 2. Not 100% sure this PR has no license issue. (about  
> > > KniteR)  
> > > > > > > > > > > What license problem *specifically* do you believe may  
> > > exist?  
> > > > > > > > > > >  
> > > > > > > > > > > In preparing the PR, I:  
> > > > > > > > > > >  
> > > > > > > > > > > * Reviewed the Apache policy in detail.  
> > > > > > > > > > >  
> > > > > > > > > > > * Contacted the FSF to ask their interpretation of the  
> > > > “linking”  
> > > > > > > > > > > provisions of the GPL license.  
> > > > > > > > > > >  
> > > > > > > > > > > * Reviewed how other Apache software deals with this  
> > issue  
> > > > > > (e.g., Spark  
> > > > > > > > > > > talks to R, which is GPL'd).  
> > > > > > > > > > >  
> > > > > > > > > > > * No necessary *dependencies* of the PR have license  
> > > > conflicts.  
> > > > > > In  
> > > > > > > > > > > several cases, I contacted package authors who agreed  
> to  
> > > > > > re-issue their  
> > > > > > > > > > > packages under Apache-compatible licenses. (Usually I  
> had  
> > > to  
> > > > do  
> > > > > > a bit  
> > > > > > > > > of  
> > > > > > > > > > > coding in exchange…)  
> > > > > > > > > > >  
> > > > > > > > > > > * Where the license had to stay GPL, the packages are  
> > *not  
> > > > > > necessary*  
> > > > > > > > > and  
> > > > > > > > > > > *not dependencies.* If the optional packages are  
> present,  
> > > > they  
> > > > > > will be  
> > > > > > > > > > > used to enable additional functionality. Knitr is an  
> > > example.  
> > > > > > The PR  
> > > > > > > > > will  
> > > > > > > > > > > compile and run fine without knitr. If knitr is  
> available  
> > > > (it is  
> > > > > > part  
> > > > > > > > > of  
> > > > > > > > > > > the most common R distribution), the PR will enable the  
> > > knitr  
> > > > > > > > > > interpreter.  
> > > > > > > > > > >  
> > > > > > > > > > > * This is exactly how this issue is addressed through  
> the  
> > > > Apache  
> > > > > > > > > > > ecosystem.  
> > > > > > > > > > > The tl;dr is this: When Apache code is written to talk  
> to  
> > > > > > libraries  
> > > > > > > > > that  
> > > > > > > > > > > may or may not be present on the user’s system, or  
> where  
> > > it  
> > > > > > talks to an  
> > > > > > > > > > API  
> > > > > > > > > > > but is agnostic about implementation, that is not  
> > > “linking”  
> > > > in a  
> > > > > > way  
> > > > > > > > > that  
> > > > > > > > > > > implicate the anti-linking provision of the GPL.  
> > > > > > > > > > >  
> > > > > > > > > > > Otherwise, no Apache code could ever call a shell  
> script!  
> > > Let  
> > > > > > alone run  
> > > > > > > > > > > on Linux, or talk to R.  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > I'm not a legal expert. So following could be wrong.  
> > > > > > > > > > >  
> > > > > > > > > > > In my interpretation, KnitRInterpreter is not an  
> optional  
> > > > > > feature while  
> > > > > > > > > > it  
> > > > > > > > > > > is always enabled when compiling Zeppelin and always  
> > > enabled  
> > > > when  
> > > > > > > > > running  
> > > > > > > > > > > Zeppelin. And it requires dynamically linked GPL  
> library  
> > > on  
> > > > > > runtime.  
> > > > > > > > > (yes  
> > > > > > > > > > > it will fail when no KnitR is installed on the system)  
> > > > > > > > > > >  
> > > > > > > > > > > And of course, no Apache code can ever call a shell  
> > > script,  
> > > > on  
> > > > > > the  
> > > > > > > > > > purpose  
> > > > > > > > > > > of dynamic linking with GPL library.  
> > > > > > > > > > >  
> > > > > > > > > > > I was guessing SparkR can be still in Apache License  
> even  
> > > if  
> > > > it  
> > > > > > is  
> > > > > > > > > > depends  
> > > > > > > > > > > on R. Because of GPL licensed compiler generated output  
> > is  
> > > > not  
> > > > > > GPL  
> > > > > > > > > > license.  
> > > > > > > > > > > and R is sort of compiler.  
> > > > > > > > > > >  
> > > > > > > > > > > If you can get answer from Spark community how SparkR  
> get  
> > > > > > managed to  
> > > > > > > > > stay  
> > > > > > > > > > > in Apache License while R is GPL, the answer might  
> help.  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > 3. Need more time to review.  
> > > > > > > > > > > Has any reviewer has downloaded the PR or run the demo  
> > > > notebook?  
> > > > > > (Which  
> > > > > > > > > > > is there for the benefit of reviewers, and isn’t  
> intended  
> > > to  
> > > > go  
> > > > > > in  
> > > > > > > > > final  
> > > > > > > > > > > distribution.)  
> > > > > > > > > > >  
> > > > > > > > > > > How many +1 comments from users would you like to see?  
> > > > > > > > > > >  
> > > > > > > > > > > How much time do you believe is required?  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > It all depends on when CI is going to pass, when  
> license  
> > > > problem  
> > > > > > is  
> > > > > > > > > going  
> > > > > > > > > > > to be cleared, and when a committer willing to review  
> and  
> > > > > > responsible  
> > > > > > > > > to  
> > > > > > > > > > > commit your contribution.  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > 1. Large code base change  
> > > > > > > > > > > Large code base changes require coordination and  
> > > cooperation.  
> > > > > > This PR  
> > > > > > > > > > > necessarily implicates the build scripts, testing code,  
> > > the  
> > > > > > > > > > > SparkInterpreter, etc.  
> > > > > > > > > > >  
> > > > > > > > > > > I have been seeking to coordinate since August.  
> > > > > > > > > > >  
> > > > > > > > > > > I continue to stand ready to do so.  
> > > > > > > > > > >  
> > > > > > > > > > > -Amos  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > If i give you one suggestion, Zeppelin committers  
> > > sometimes  
> > > > ask  
> > > > > > rebase  
> > > > > > > > > > the  
> > > > > > > > > > > contribution branch for some reason. It is not the  
> really  
> > > the  
> > > > > > best  
> > > > > > > > > > > practice, but still okay while most contributions are  
> not  
> > > > > > including  
> > > > > > > > > large  
> > > > > > > > > > > code base changes.  
> > > > > > > > > > >  
> > > > > > > > > > > However, your one, has more than 4000 lines of code  
> > > change.  
> > > > If  
> > > > > > you  
> > > > > > > > > rebase  
> > > > > > > > > > > then review should be started from the beginning,  
> again.  
> > > So  
> > > > you  
> > > > > > might  
> > > > > > > > > > want  
> > > > > > > > > > > to minimize the rebase your branch.  
> > > > > > > > > > >  
> > > > > > > > > > > Thanks,  
> > > > > > > > > > > moon  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > > > > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > > > > > > Date: December 1, 2015 at 1:34:19 AM  
> > > > > > > > > > > To: dev@zeppelin.incubator.apache.org <  
> > > > > > > > > dev@zeppelin.incubator.apache.org  
> > > > > > > > > > >  
> > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > > > > > incubator-zeppelin  
> > > > > > > > > pull  
> > > > > > > > > > > request: R Interpreter for Zeppelin  
> > > > > > > > > > >  
> > > > > > > > > > > Hi Cos,  
> > > > > > > > > > >  
> > > > > > > > > > > Thanks for opening a discussion.  
> > > > > > > > > > > My answer about 'Why this PR is open for three months'  
> is  
> > > > > > > > > > >  
> > > > > > > > > > > 1. This PR does not passes CI  
> > > > > > > > > > > 2. Not 100% sure this PR has no license issue. (about  
> > > KniteR)  
> > > > > > > > > > > 3. Need more time to review.  
> > > > > > > > > > >  
> > > > > > > > > > > It's my personal answer, other committers may have  
> > > different  
> > > > > > opinion.  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > I think the question should be generalized. Because  
> this  
> > > PR  
> > > > is  
> > > > > > not the  
> > > > > > > > > > only  
> > > > > > > > > > > PR that is in impasse. There're more. For example  
> > > > > > > > > > >  
> > > > > > > > > > > Here's some examples that PR is not been merged.  
> > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,  
> > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/60  
> > > > > > > > > > > and so on.  
> > > > > > > > > > >  
> > > > > > > > > > > I can categorize the cases, based on experience of  
> > > involving  
> > > > > > Zeppelin  
> > > > > > > > > > > community from the beginning.  
> > > > > > > > > > >  
> > > > > > > > > > > 1. Large code base change  
> > > > > > > > > > >  
> > > > > > > > > > > When contribution has large code base changes, it tend  
> to  
> > > > take  
> > > > > > more  
> > > > > > > > > time  
> > > > > > > > > > to  
> > > > > > > > > > > review and merged. Normally, most contributions merged  
> in  
> > > > 1day~1  
> > > > > > week.  
> > > > > > > > > > > But some contribution has large code base changes take  
> > 2~4  
> > > > > > weeks. Few  
> > > > > > > > > > > contribution that has very large code base change take  
> > > > months.  
> > > > > > > > > > >  
> > > > > > > > > > > 2. Communication lost  
> > > > > > > > > > >  
> > > > > > > > > > > Sometimes, committer is not responding, or contributor  
> is  
> > > not  
> > > > > > > > > responding.  
> > > > > > > > > > >  
> > > > > > > > > > > 3. Opinion diverges  
> > > > > > > > > > >  
> > > > > > > > > > > For some changes, included in contribution. When people  
> > > have  
> > > > > > different  
> > > > > > > > > > > opinion and it continue to diverges, those PRs are not  
> > > been  
> > > > > > merged.  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > > > I think having a guide such as ping other committer  
> when  
> > a  
> > > > > > committer is  
> > > > > > > > > > not  
> > > > > > > > > > > responding, and divide contribution into small peaces  
> if  
> > > > > > possible,  
> > > > > > > > > would  
> > > > > > > > > > > help most of the cases.  
> > > > > > > > > > > Of course committer have to pay attention more to the  
> > > > > > contribution and  
> > > > > > > > > > > helping people. That's the first one.  
> > > > > > > > > > >  
> > > > > > > > > > > Thanks,  
> > > > > > > > > > > moon  
> > > > > > > > > > >  
> > > > > > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <  
> > > > > > cos@apache.org>  
> > > > > > > > > > wrote:  
> > > > > > > > > > >  
> > > > > > > > > > > > To make sure we're on the same page, here are two  
> > > threads  
> > > > that  
> > > > > > I  
> > > > > > > > > found  
> > > > > > > > > > > > related  
> > > > > > > > > > > > to this topic.  
> > > > > > > > > > > >  
> > > > > > > > > > > > Thread 1:  
> > > > > > > > > > > > Subject: R?  
> > > > > > > > > > > > Started on: July 1, 2015  
> > > > > > > > > > > >  
> > > > > > > > > > > > Thread 2:  
> > > > > > > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R  
> > > > > > Interpreter for  
> > > > > > > > > > > > Zeppelin  
> > > > > > > > > > > > Started on: August 13, 2015  
> > > > > > > > > > > >  
> > > > > > > > > > > > If you want to fetch these from our archive send  
> emails  
> > > to  
> > > > > > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org  
> > > > > > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org  
> > > > > > > > > > > >  
> > > > > > > > > > > > Cos  
> > > > > > > > > > > >  
> > > > > > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik  
> > > wrote:  
> > > > > > > > > > > > > Guys,  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > While catching up on my emails from the last a  
> couple  
> > > of  
> > > > > > weeks,  
> > > > > > > > > this  
> > > > > > > > > > > > thread  
> > > > > > > > > > > > > caught my attention. I am not normally paying much  
> > > > attention  
> > > > > > to the  
> > > > > > > > > > > code  
> > > > > > > > > > > > > reviews traffic from GH, but this one is pretty  
> > > > different as  
> > > > > > it  
> > > > > > > > > spans  
> > > > > > > > > > > > three  
> > > > > > > > > > > > > months and counting.  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > First, here are my five cents:  
> > > > > > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is  
> > aimed  
> > > > to be  
> > > > > > > > > > > > contributed to  
> > > > > > > > > > > > > an ASF project this file should simply contain ASL2  
> > > text,  
> > > > > > like in  
> > > > > > > > > [1]  
> > > > > > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate  
> > > > <developers>  
> > > > > > > > > > section,  
> > > > > > > > > > > > but  
> > > > > > > > > > > > > Zeppelin might have different guidelines on it.  
> Say,  
> > > > Bigtop  
> > > > > > doesn't  
> > > > > > > > > > > > > maintain this in the source code, but we have the  
> > list  
> > > of  
> > > > > > all the  
> > > > > > > > > > > > > committers on the project's site [2] Every new  
> > > > committer's  
> > > > > > first  
> > > > > > > > > > > > commit is  
> > > > > > > > > > > > > to update the team page ;)  
> > > > > > > > > > > > > - comments like in  
> > > > > > > > > > > >  
> > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > +/**  
> > > > > > > > > > > > > + * Created by aelberg on 7/28/15.  
> > > > > > > > > > > > > + */  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > is better to be removed. It has been already  
> > discussed  
> > > > here  
> > > > > > [3].  
> > > > > > > > > And  
> > > > > > > > > > > > the  
> > > > > > > > > > > > > initial discussion on the topic [4] was linked as  
> > well  
> > > > > > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not  
> > > sure  
> > > > if  
> > > > > > this is  
> > > > > > > > > > > > R-specific  
> > > > > > > > > > > > > stuff - I have no idea about R, honestly, but  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE  
> > > > > > > > > > > > > ...  
> > > > > > > > > > > > > +Author: David B. Dahl  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > shouldn't be here, IMO. Normally, if some  
> additional  
> > > > > > licenses are  
> > > > > > > > > > > > used,  
> > > > > > > > > > > > > they have to be listed in the top-level NOTICE file  
> > > > (already  
> > > > > > > > > there).  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > - I am not going to offer any comments on the  
> > > technical  
> > > > > > merits of  
> > > > > > > > > the  
> > > > > > > > > > > > patch,  
> > > > > > > > > > > > > as I haven't tried it personally. However, I don't  
> > see  
> > > > any  
> > > > > > serious  
> > > > > > > > > > > > > technical objections to the functionality in  
> > question.  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > So, the question is - why the PR is open for three  
> > > > months? I  
> > > > > > hasn't  
> > > > > > > > > > > been  
> > > > > > > > > > > > able  
> > > > > > > > > > > > > to get a clear answer. What I found out though is  
> > > pretty  
> > > > > > > > > unsettling,  
> > > > > > > > > > > > really.  
> > > > > > > > > > > > > The communication on the topic is almost  
> > non-existent,  
> > > > > > except for  
> > > > > > > > > > this  
> > > > > > > > > > > > sparse  
> > > > > > > > > > > > > and bitter thread in the GH.  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > I would love to hear from the committers what's  
> > > stopping  
> > > > the  
> > > > > > > > > > acceptance  
> > > > > > > > > > > > of the  
> > > > > > > > > > > > > code, besides of the minor issues I've mentioned  
> > > earlier?  
> > > > > > What are  
> > > > > > > > > > the  
> > > > > > > > > > > > reasons for it?  
> > > > > > > > > > > > > Is there anything wrong with it?  
> > > > > > > > > > > > > One of the responsibilities of the committers is to  
> > > make  
> > > > > > sure the  
> > > > > > > > > > > > > contributions are reviewed; new contributors are  
> > > guided  
> > > > and  
> > > > > > do  
> > > > > > > > > > > > understand how  
> > > > > > > > > > > > > the project ticks. The easy feedback flow attracts  
> > new  
> > > > > > people,  
> > > > > > > > > > allowing  
> > > > > > > > > > > > the  
> > > > > > > > > > > > > community to grow and thrive.  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > I am asking _explicitely_ not to re-start the  
> > > bickering I  
> > > > > > have  
> > > > > > > > > > already  
> > > > > > > > > > > > > seen. At this point I am interested in the purely  
> > > > technical  
> > > > > > side of  
> > > > > > > > > > > this.  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > [1]  
> > > https://github.com/apache/bigtop/blob/master/LICENSE  
> > > > > > > > > > > > > [2] http://bigtop.apache.org/team-list.html  
> > > > > > > > > > > > > [3]  
> > > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > >  
> > > > > >  
> > > >  
> > >  
> >  
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html  
> > > > > > > > > > > > > [4] http://s.apache.org/iZl  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > With regards,  
> > > > > > > > > > > > > Cos  
> > > > > > > > > > > > >  
> > > > > > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:  
> > > > > > > > > > > > > > Github user elbamos commented on the pull  
> request:  
> > > > > > > > > > > > > >  
> > > > > > > > > > > > > >  
> > > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > >  
> > > > > >  
> > > >  
> > >  
> >  
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411  
> > > > > > > > > > > > > >  
> > > > > > > > > > > > > > The current push should resolve some issues with  
> > > > changes  
> > > > > > in the  
> > > > > > > > > > > > > > Spark-Zeppelin interface that had created issues  
> > for  
> > > > > > users, as  
> > > > > > > > > > > > well as  
> > > > > > > > > > > > > > support for 1.5.1.  
> > > > > > > > > > > > > >  
> > > > > > > > > > > > > > Knitr support is improved, and the reason for a  
> > > > separate  
> > > > > > knitr  
> > > > > > > > > > > > interpreter may be clearer now.  
> > > > > > > > > > > > > >  
> > > > > > > > > > > > > > The amount of code borrowed from rscala is  
> reduced.  
> > > > > > > > > > > > > >  
> > > > > > > > > > > > > > I did not address issues with @author tags, or  
> > files  
> > > > under  
> > > > > > the R/  
> > > > > > > > > > > > > > folder. The reason is, to be blunt, I don't  
> > > understand  
> > > > > > what the  
> > > > > > > > > > > > precise  
> > > > > > > > > > > > > > concerns actually are.  
> > > > > > > > > > > > > >  
> > > > > > > > > > > > > > Please note that I changed .travis.yml to only  
> use  
> > > > spark  
> > > > > > 1.4 and  
> > > > > > > > > > > > later.  
> > > > > > > > > > > > > > I'm sure there is a better way to do it, but I'm  
> > > just  
> > > > not  
> > > > > > in a  
> > > > > > > > > > > > position  
> > > > > > > > > > > > > > to try to figure out the entire Zeppelin build  
> > > process.  
> > > > > > > > > > > > > >  
> > > > > > > > > > > > > > Integrating this with the rest of zeppelin is  
> going  
> > > to  
> > > > > > take some  
> > > > > > > > > > > > work  
> > > > > > > > > > > > > > regarding pom's, travis, and so forth. I can do a  
> > > lot  
> > > > of  
> > > > > > that,  
> > > > > > > > > > > > but I'm  
> > > > > > > > > > > > > > going to need to discuss it with the people who  
> > have  
> > > > been  
> > > > > > > > > "owning"  
> > > > > > > > > > > > those  
> > > > > > > > > > > > > > files. There are just too many moving pieces  
> here.  
> > > > > > > > > > > > > >  
> > > > > > > > > > > > > > Because of the size of this (which is,  
> > > unfortunately,  
> > > > > > necessary),  
> > > > > > > > > > > > > > posting here is probably not the most efficient  
> > way.  
> > > > That  
> > > > > > is also  
> > > > > > > > > > > > true  
> > > > > > > > > > > > > > because certain people chose to use this PR as a  
> > > forum  
> > > > to  
> > > > > > air  
> > > > > > > > > other  
> > > > > > > > > > > > > > issues. Therefore, it would be better if  
> reviewers  
> > > > e-mail  
> > > > > > me  
> > > > > > > > > > > > directly.  
> > > > > > > > > > > >  
> > > > > > > > > > > >  
> > > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > > >  
> > > > > >  
> > > >  
> > >  
> > >  
> >  
>  

Re: R-Interpreter CI Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

fyi, some cases of CI failing, when PR does not impact the code,

1) Flickering test. Some tests are passing sometimes, failing the other
times. (e.g. ZEPPELIN-476)
2) Download fails. Downloading maven artifacts or other requirements from
internet are sometimes failing

both cases pass if CI triggered again.

Thanks,
moon

On Tue, Dec 8, 2015 at 6:29 PM DuyHai Doan <do...@gmail.com> wrote:

> I have also experienced CI failing in the pass for some PRs that do not
> impact the code (Cassandra documentation). I guess that during peak hours,
> the CI servers may be too busy or enter a dead lock so the build fails.
>
>  At least that's my guess. What do you think ?
>
> On Tue, Dec 8, 2015 at 9:42 AM, moon soo Lee <mo...@apache.org> wrote:
>
> > Let me run some CI test with your branch and share result here.
> > Hope i can narrow down the cause and that helps involvement of more
> people.
> >
> > Thanks,
> > moon
> >
> > On Tue, Dec 8, 2015 at 3:08 PM Amos B. Elberg <am...@gmail.com>
> > wrote:
> >
> > > Is there anyone who can work with me on the CI issues?  It looks like
> > > there are a number of PRs experiencing similar things.
> > >
> > > I think we should make getting CI stable to be a priority.  Because it
> > > will save everyone a lot of frustration and aggravation if CI works
> > > reliably.
> > >
> > > Is there anyone other than Jongyoul and Moon who knows the CI/Build
> > > process well?
> > >
> > > (Moon - Thank you for taking another look at the licensing issue.  Per
> > the
> > > e-mail I wrote about this a few days ago, I don’t feel I have more to
> > > contribute to the licensing discussion, so I’m going to try not to
> > comment
> > > further about it.)
> > >
> > > From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> > > Reply: dev@zeppelin.incubator.apache.org
> > > <de...@zeppelin.incubator.apache.org> <dev@zeppelin.incubator.apache.org
> >
> > > Date: December 7, 2015 at 5:00:08 PM
> > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org
> > >
> > > <de...@zeppelin.incubator.apache.org>
> > > Subject:  Re: License of KnitRInterpreter Was: Re: contributions
> impasse.
> > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for
> Zeppelin
> > >
> > > Thanks Amos, Roman, Cos for clarifying license issue.
> > >
> > > I'm convinced that this license issue will not be a blocker.
> > >
> > > In my understanding, these are good sign,
> > >
> > > 1. any gpl licensed source codes are not included in the source package
> > > 2. any gpl licensed libraries are not included in the binary package
> > >
> > > However, i can not still 100% sure about
> > >
> > > 3. any gpl licensed libraries are not linked on runtime
> > >
> > > Even after Amos's explanation. I still think using 'knitr' is one of
> the
> > > clear case that 'knir' is linked to 'R' according to
> > > http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL.
> > >
> > > Giving input and getting output from GPL licensed interpreter (includes
> > R)
> > > from Apache licensed software is not a problem. That's not the point.
> > > Let me say in this way. There's an java code,
> > >
> > > import com.mysql.jdbc.Driver
> > > Driver driver = new Driver()
> > >
> > > Say without this java code, one of important feature of Zeppelin does
> not
> > > work. And Zeppelin does not includes GPL licensed file in the source
> > > package, GPL licensed library in the binary package, but it requires
> GPL
> > > licensed library on the runtime.
> > > In this case, will this java code be a license problem or not?
> > >
> > > In other words, my question is
> > >
> > > a) Is runtime GPL library dependency allowed in ASF release?
> > > b) is 'knitr' considered as runtime dependency?
> > >
> > > If someone can clarify a), b), then it would be extremely helpful
> > > understanding this case, and possible similar cases, too.
> > >
> > > Thanks,
> > > moon
> > >
> > > On Mon, Dec 7, 2015 at 3:20 PM Konstantin Boudnik <co...@apache.org>
> > wrote:
> > >
> > > > On Thu, Dec 03, 2015 at 11:03AM, Corneau Damien wrote:
> > > > > Thanks Cos for those answers,
> > > > >
> > > > > If I'm right you are advising to have a default build that doesn't
> > > > include
> > > > > libraries with conflicting licenses, but have an option to include
> > > them
> > > > for
> > > > > users who wants to build the project themselves.
> > > >
> > > > Yes, that's what I said. Besides, looks like Roman provided the
> second
> > > > pair of
> > > > eyes to this licensing discussion and as well didn't find any issues
> > > with
> > > > the
> > > > current approach.
> > > >
> > > > Cos
> > > >
> > > > > To refer to another thread about decentralizing interpreters, it
> > could
> > > > even
> > > > > be better in that case to have some interpreters separated from the
> > > tree,
> > > > > and easily pluggable with a release instead of forcing users to
> build
> > > the
> > > > > project to use them.
> > > > >
> > > > >
> > > > > On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <cos@apache.org
> >
> > > > wrote:
> > > > >
> > > > > > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:
> > > > > > > Konstantin thank you for getting into this.
> > > > > > >
> > > > > > >> The best way to go around it is by
> > > > > > >> providing a build-time option that will pull such binaries in.
> > > But
> > > > by
> > > > > > default
> > > > > > >> such libs shouldn't be pulled.
> > > > > > >
> > > > > > > That is basically how the PR handles this. If the GPL’d
> > > interpreter
> > > > > > scripts
> > > > > > > are missing, there’s no indication at all at build time. It
> > > doesn’t
> > > > try
> > > > > > to
> > > > > > > download them.
> > > > > > >
> > > > > > > At runtime, if the user tries to use functionality that would
> > need
> > > > such a
> > > > > > > script (i.e., if they type “knitr” to use knitr), we display an
> > > error
> > > > > > that
> > > > > > > says that the functionality is not there because the library is
> > > > missing,
> > > > > > and
> > > > > > > that the library cannot be provided because it has an
> > incompatible
> > > > > > license,
> > > > > > > but the user can download it themselves if they want.
> > > > > > >
> > > > > > > And, in the log, if the logging level is high, they will see a
> > > note
> > > > that
> > > > > > > some functionality was disabled because the libraries weren’t
> > > there.
> > > > > > >
> > > > > > > To be clear, though, none of these libraries are binaries.
> > They’re
> > > > all
> > > > > > interpreter scripts.
> > > > > >
> > > > > > If you ever in a doubt of which licenses could be used for
> > > dependncies
> > > > > > (not to
> > > > > > say about source code) are listed in Category A list of [1]
> > > > > >
> > > > > > A lot of quesitons discussed here are already covered in the
> legal
> > > > FAQ, so
> > > > > > just check against it if you have any questions.
> > > > > >
> > > > > > [1] http://www.apache.org/legal/resolved.html#category-a
> > > > > >
> > > > > > Cos
> > > > > >
> > > > > > > From: Konstantin Boudnik <co...@apache.org>
> > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>,
> > > dev@zeppelin.incubator.apache.org
> > > > <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > Date: December 2, 2015 at 3:24:50 PM
> > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > dev@zeppelin.incubator.apache.org
> > > > > > >
> > > > > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions
> > > > > > impasse. Was: [GitHub] incubator-zeppelin pull request: R
> > > Interpreter
> > > > for
> > > > > > Zeppelin
> > > > > > >
> > > > > > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:
> > > > > > > > I think that what moon means is that:
> > > > > > > > If we merge the way it is now, the KnitRInterpreter will be
> > part
> > > > of the
> > > > > > > > code base, so it isn't optional at code base level.
> > > > > > > >
> > > > > > > > To make it even more simple:
> > > > > > > > * If the code has the right licensing -> that code can be
> part
> > > of
> > > > the
> > > > > > > > source code, and can be including in a binary release
> > > > > > >
> > > > > > > We aren't concerned with binary releases - as an Apache
> community
> > > we
> > > > are
> > > > > > > voting and releasing source code. If the project wants to
> provide
> > > a
> > > > > > binary
> > > > > > > release to its users, they are better be warned about inclusion
> > of
> > > > non
> > > > > > > ASL2-friendly licensed bits.
> > > > > > >
> > > > > > > > * If the code doesn't have the right licensing -> it can't be
> > > part
> > > > of
> > > > > > the
> > > > > > > > source code, and can't be included in a binary release
> > > > > > >
> > > > > > > See above.
> > > > > > >
> > > > > > > > * If the code doesn't have the right licensing but is
> imported
> > > at
> > > > build
> > > > > > > > time (libraries for example) -> it is not in the source code,
> > it
> > > > can't
> > > > > > be
> > > > > > > > included in binary release
> > > > > > >
> > > > > > > That is unless a user doing it on his own. The best way to go
> > > around
> > > > it
> > > > > > is by
> > > > > > > providing a build-time option that will pull such binaries in.
> > But
> > > by
> > > > > > default
> > > > > > > such libs shouldn't be pulled.
> > > > > > >
> > > > > > > Cos
> > > > > > >
> > > > > > > > So in the case of licensing issues, it would need to be fully
> > > > optional
> > > > > > > > (user bring the interpreter in his directory and build
> Zeppelin
> > > > with
> > > > > > it)
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <
> > > > amos.elberg@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Moon let me clarify:
> > > > > > > > >
> > > > > > > > > Interpreted code doesn’t “link.” The wiki article actually
> > > > explains
> > > > > > it
> > > > > > > > > pretty well —
> > > > https://en.wikipedia.org/wiki/GPL_linking_exception
> > > > > > > > > “Linking” against a library means compiling its headers
> into
> > a
> > > > > > binary, the
> > > > > > > > > way a C compiler works. The 2008 e-mail Moon distributed,
> > > called
> > > > > > this the
> > > > > > > > > “interpreter exception.”
> > > > > > > > >
> > > > > > > > > As for whether GPL’d code is a “mandatory dependency,” if
> > > knitr
> > > > is
> > > > > > missing
> > > > > > > > > the PR will compile, run and test just fine. The user is
> > never
> > > > > > prompted to
> > > > > > > > > download it. The only effect is, if the user types “knitr”
> > and
> > > > knitr
> > > > > > isn’t
> > > > > > > > > there, we display an InterpreterError that knitr isn’t
> there.
> > > > > > > > >
> > > > > > > > > KnitRInterpreter is not optionally required. so it does not
> > > > matter
> > > > > > KnitR
> > > > > > > > > is
> > > > > > > > > optionally required or not.
> > > > > > > > > Aren’t all interpreters optional? You can turn them on and
> > off
> > > > in the
> > > > > > > > > config files.
> > > > > > > > >
> > > > > > > > > Do you mean that the KnitRInterpreter class gets compiled
> to
> > > > > > bytecode even
> > > > > > > > > if knitr is missing? So what? That isn't a mandatory
> > > dependency
> > > > or a
> > > > > > link.
> > > > > > > > >
> > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > Date: December 2, 2015 at 3:18:00 AM
> > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > Subject: Re: License of KnitRInterpreter Was: Re:
> > > contributions
> > > > > > impasse.
> > > > > > > > > Was: [GitHub] incubator-zeppelin pull request: R
> Interpreter
> > > for
> > > > > > Zeppelin
> > > > > > > > >
> > > > > > > > > Let me summarize license concern about KnitRInterpreter.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Amos's interpretation.
> > > > > > > > >
> > > > > > > > > KnitR is optionally required by KnitRInterpreter.
> > > > > > > > > R dependency in SparkR has no problem. So KnitR should be
> the
> > > > same.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Moon's interpretation.
> > > > > > > > >
> > > > > > > > > KnitRInterpreter is not optionally required. so it does not
> > > > matter
> > > > > > KnitR is
> > > > > > > > > optionally required or not.
> > > > > > > > > R dependency in SparkR is exception of GPL. KnitR is not
> > > applied
> > > > that
> > > > > > > > > exception.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Amos, could you confirm my understanding to your
> > > interpretation
> > > > is
> > > > > > correct?
> > > > > > > > > If it is not could you clarify it?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > moon
> > > > > > > > >
> > > > > > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Just to put the final nail in this, I looked it up.
> > > > > > > > > >
> > > > > > > > > > I see no evidence of any “compiler exception.”
> > > > > > > > > >
> > > > > > > > > > There is an exception in the license for the runtime
> > > libraries
> > > > > > that are
> > > > > > > > > > bundled with GCC. See:
> > > > > > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> > > > > > > > > >
> > > > > > > > > > But no “compiler exception.”
> > > > > > > > > >
> > > > > > > > > > In fact, I believe this is part of the reason why LLVM
> was
> > > > created.
> > > > > > > > > >
> > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Date: December 1, 2015 at 8:16:36 PM
> > > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin pull
> > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > >
> > > > > > > > > > Is knitR is commonly considered as a
> interpreter/compiler?
> > > or
> > > > is it
> > > > > > > > > > considered as a library routine?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > moon
> > > > > > > > > >
> > > > > > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > Moon - you give this as an explanation of the licensing
> > > issue:
> > > > > > > > > >
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > > > >
> > > > > > > > > > According to that, there is an exception in the GPL for
> > > > interpreter
> > > > > > > > > > languages. As long as you don’t distribute the code, its
> > > fine
> > > > to
> > > > > > talk to
> > > > > > > > > > an interpreted language.
> > > > > > > > > >
> > > > > > > > > > Well, if that’s the case, then the PR plainly does not
> have
> > > a
> > > > > > license
> > > > > > > > > > issue. It doesn’t distribute any GPL’d R code.
> > > > > > > > > >
> > > > > > > > > > I’m not sure what’s confusing about this. It seems
> > > completely
> > > > > > > > > > straightforward.
> > > > > > > > > >
> > > > > > > > > > Regarding this:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Amos Elberg
> > > > > > > > > > Sent with Airmail
> > > > > > > > > >
> > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > Date: December 1, 2015 at 6:48:47 PM
> > > > > > > > > >
> > > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org
> > > > > > > > > >
> > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin pull
> > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > >
> > > > > > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I am going to try to minimize my reaction to Moon’s
> > > e-mail.
> > > > > > > > > > >
> > > > > > > > > > > The tl;dr is this:
> > > > > > > > > > >
> > > > > > > > > > > The reason we are having this discussion now is that
> > > active
> > > > > > users of
> > > > > > > > > the
> > > > > > > > > > > PR — which now has its own user base — went public to
> > > > complain
> > > > > > about
> > > > > > > > > > this.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > The PR has been tested by an active user base for more
> > > than
> > > > three
> > > > > > > > > months.
> > > > > > > > > > > No-one has been able to identify any specific actual
> > > > licensing
> > > > > > problem,
> > > > > > > > > > and
> > > > > > > > > > > the PR was prepared based on an extensive, careful
> review
> > > of
> > > > the
> > > > > > > > > relevant
> > > > > > > > > > > licensing issues and after contacting the relevant
> > people.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I admire every software that used by user and helping
> > > people.
> > > > That
> > > > > > > > > includes
> > > > > > > > > > your work. But that's not the topic we're in discussion.
> > > Active
> > > > > > user does
> > > > > > > > > > not mean your contribution can ignore the review.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > It is not an explanation for someone who has been
> > ignoring
> > > my
> > > > > > “how can
> > > > > > > > > I
> > > > > > > > > > > move this forward…” emails for three months to point
> the
> > > > finger
> > > > > > and
> > > > > > > > > say I
> > > > > > > > > > > didn’t contact the right person or file the right
> report.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > This is also not the topic in this discussion.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > The burden for providing an explanation for the
> inaction
> > > is
> > > > on
> > > > > > the PMCC
> > > > > > > > > > at
> > > > > > > > > > > this point.
> > > > > > > > > >
> > > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's
> > problem
> > > on
> > > > > > Zeppelin
> > > > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > > > > They’re not! I often see comments on PRs to just ignore
> > > that
> > > > CI
> > > > > > is
> > > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > One of the most common reasons this PR fails CI, is CI
> > > > times-out
> > > > > > > > > > > downloading Spark to install. How could that possibly
> be
> > > > caused
> > > > > > by the
> > > > > > > > > > PR?
> > > > > > > > > > >
> > > > > > > > > > > It looks to me like the only PRs with changes to the
> > > relevant
> > > > > > parts of
> > > > > > > > > > the
> > > > > > > > > > > code — the SparkInterpreter — are being made by the
> > person
> > > > who
> > > > > > wrote
> > > > > > > > > the
> > > > > > > > > > > testing suite.
> > > > > > > > > > >
> > > > > > > > > > > So, that would explain why some other PRs pass CI:
> > Neither
> > > > the
> > > > > > > > > > > SparkInterpreter nor the testing suite are stable or
> > > robust,
> > > > but
> > > > > > since
> > > > > > > > > > the
> > > > > > > > > > > PRs are coming from the person who wrote both…
> > > > > > > > > > >
> > > > > > > > > > > And let's say Zeppelin core has problem and that makes
> > > your
> > > > PR
> > > > > > fails on
> > > > > > > > > > CI
> > > > > > > > > > > test. That's possible. But it still does not mean we
> can
> > > > merge
> > > > > > it with
> > > > > > > > > CI
> > > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > It means you should be working with me to figure out
> why
> > > the
> > > > CI
> > > > > > is
> > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > This PR has been tested by an active user base for the
> > > past
> > > > three
> > > > > > > > > months.
> > > > > > > > > > > If CI is continuing to fail, and dozens of hours of
> > effort
> > > > have
> > > > > > not
> > > > > > > > > > > resolved the CI issues, then it is time to start
> > > considering
> > > > > > whether
> > > > > > > > > the
> > > > > > > > > > > testing suite is part of the problem.
> > > > > > > > > > >
> > > > > > > > > > > The level of defensiveness about the CI and
> > > SparkInterpreter
> > > > is
> > > > > > not
> > > > > > > > > > > helping to resolve these issues.
> > > > > > > > > > >
> > > > > > > > > > > If you think it's problem on Zeppelin core, then file
> an
> > > > issue
> > > > > > that
> > > > > > > > > > > reproduce the problem on Zeppelin core, that might be
> > more
> > > > > > efficient
> > > > > > > > > than
> > > > > > > > > > > keep trying yourself.
> > > > > > > > > > > I contacted you numerous times about such issues...
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I remember i commented your issue about CI. but you just
> > > keep
> > > > > > repeated
> > > > > > > > > it's
> > > > > > > > > > not your problem but Zeppelin core problem.
> > > > > > > > > >
> > > > > > > > > > Then please file an issue about the problem you found in
> > > > Zeppelin
> > > > > > Core.
> > > > > > > > > > Then everyone will get into the problem.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > In my interpretation, KnitRInterpreter is not an
> optional
> > > > > > feature while
> > > > > > > > > > it
> > > > > > > > > > > is always enabled when compiling Zeppelin and always
> > > enabled
> > > > when
> > > > > > > > > running
> > > > > > > > > > > Zeppelin. And it requires dynamically linked GPL
> library
> > > on
> > > > > > runtime.
> > > > > > > > > (yes
> > > > > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > > > > >
> > > > > > > > > > > Its not always enabled.
> > > > > > > > > > > It is not dynamically linked at runtime.
> > > > > > > > > > > It will not fail when knitr is missing. If knitr is not
> > > > present,
> > > > > > the
> > > > > > > > > repl
> > > > > > > > > > > interpreter starts and a note is written to to the log
> > > that
> > > > the
> > > > > > knitr
> > > > > > > > > > > interpreter isn’t available because knitr is not
> present.
> > > > > > > > > > >
> > > > > > > > > > > no Apache code can ever call a shell script, on the
> > > purpose
> > > > of
> > > > > > dynamic
> > > > > > > > > > > linking with GPL library.
> > > > > > > > > > > You misunderstand.
> > > > > > > > > > >
> > > > > > > > > > > The *shell* is GPL'd.
> > > > > > > > > > >
> > > > > > > > > > > Is Zeppelin “linked" against the GPL’d shell because
> > > Zeppelin
> > > > > > depends
> > > > > > > > > on
> > > > > > > > > > a
> > > > > > > > > > > shell script to launch?
> > > > > > > > > > >
> > > > > > > > > > Obviously not.
> > > > > > > > > > >
> > > > > > > > > > > The interaction with R in the PR is the same.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Again, bash is one of exceptions of GPL, like other GPL
> > > > licensed
> > > > > > > > > > compiler/interpreter.
> > > > > > > > > >
> > > > > > > > > > Check here why Bash and R is okay with Apache License.
> > > > > > > > > >
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > > > >
> > > > > > > > > > I'm not sure we can apply the same exception for 'using'
> > > KnitR.
> > > > > > > > > >
> > > > > > > > > > My point is not 'KnitR' is optional or not. Point is
> > > > > > 'KnitRInterpreter'
> > > > > > > > > you
> > > > > > > > > > wrote is not an optional feature. Which is clearly not
> > > > optionally
> > > > > > enabled
> > > > > > > > > > code and feature. And that depends on KnitR library which
> > is
> > > > GPL.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > I was guessing SparkR can be still in Apache License
> even
> > > if
> > > > it
> > > > > > is
> > > > > > > > > > depends
> > > > > > > > > > > on R. Because of GPL licensed compiler generated output
> > is
> > > > not
> > > > > > GPL
> > > > > > > > > > license.
> > > > > > > > > > > and R is sort of compiler. If you can get answer from
> > > Spark
> > > > > > community
> > > > > > > > > how
> > > > > > > > > > > SparkR get managed to stay in Apache License while R is
> > > GPL,
> > > > the
> > > > > > answer
> > > > > > > > > > > might help.
> > > > > > > > > > > The description of SparkR is not accurate in any
> respect.
> > > (Do
> > > > > > you think
> > > > > > > > > > > SparkR is not talking to GPL-licensed libraries?)
> > > > > > > > > > >
> > > > > > > > > > > I don’t see that any genuine issue is being raised
> here.
> > > > > > > > > > >
> > > > > > > > > > > If there is an issue, the burden is on you to identify
> > it.
> > > > > > > > > > >
> > > > > > > > > > > If i give you one suggestion, Zeppelin committers
> > > sometimes
> > > > ask
> > > > > > rebase
> > > > > > > > > > the
> > > > > > > > > > > contribution branch for some reason. It is not the
> really
> > > the
> > > > > > best
> > > > > > > > > > > practice, but still okay while most contributions are
> not
> > > > > > including
> > > > > > > > > large
> > > > > > > > > > > code base changes
> > > > > > > > > > > However, your one, has more than 4000 lines of code
> > > change.
> > > > If
> > > > > > you
> > > > > > > > > rebase
> > > > > > > > > > > then review should be started from the beginning,
> again.
> > > So
> > > > you
> > > > > > might
> > > > > > > > > > want
> > > > > > > > > > > to minimize the rebase your branch.
> > > > > > > > > > >
> > > > > > > > > > > Are you actually complaining that the problem is that I
> > > > rebased
> > > > > > the
> > > > > > > > > code
> > > > > > > > > > > during the three-month period when no-one looked at it
> > and
> > > > > > Zeppelin
> > > > > > > > > went
> > > > > > > > > > > through a release?
> > > > > > > > > > >
> > > > > > > > > > > I cannot take it seriously when you say things like
> this.
> > > > > > > > > > >
> > > > > > > > > > > Having to “start from the beginning” cannot be a
> problem
> > > if
> > > > you
> > > > > > never
> > > > > > > > > > > actually started the first time...
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > You wanted coordination and cooperation. So i gave you
> > > > suggestion
> > > > > > that
> > > > > > > > > > helping review process. For example, your code has been
> > > rebased
> > > > > > since my
> > > > > > > > > > comment and jongyoul's comment. that means committers
> will
> > > > need to
> > > > > > look
> > > > > > > > > > from the beginning. That'll require more time. And
> maybe, i
> > > > guess
> > > > > > that's
> > > > > > > > > > not what you want. But If you don't care, feel free to
> > > rebase.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > moon
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Date: December 1, 2015 at 4:42:06 AM
> > > > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin
> > > > > > > > > pull
> > > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > Thank you, Cos.
> > > > > > > > > > >
> > > > > > > > > > > I’d like to briefly address the issues raised by Moon:
> > > > > > > > > > >
> > > > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > > > The CI fails on core Zeppelin, *not* code in this PR.
> > > > > > > > > > >
> > > > > > > > > > > I’ve been seeking assistance on this since August.
> > > > > > > > > > >
> > > > > > > > > > > The most common reason is that SparkInterpreter is
> unable
> > > to
> > > > > > launch
> > > > > > > > > Spark
> > > > > > > > > > > and open a Spark Backend. This is necessary to test the
> > > PR.
> > > > > > > > > > >
> > > > > > > > > > > 60+ hours, has been spent adapting and re-basing when
> the
> > > > > > > > > > SparkInterpreter
> > > > > > > > > > > architecture changed and broke the PR’s *tests.*
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's
> > > problem
> > > > on
> > > > > > > > > Zeppelin
> > > > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > > > >
> > > > > > > > > > > And let's say Zeppelin core has problem and that makes
> > > your
> > > > PR
> > > > > > fails on
> > > > > > > > > > CI
> > > > > > > > > > > test. That's possible. But it still does not mean we
> can
> > > > merge
> > > > > > it with
> > > > > > > > > CI
> > > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > If you think it's problem on Zeppelin core, then file
> an
> > > > issue
> > > > > > that
> > > > > > > > > > > reproduce the problem on Zeppelin core, that might be
> > more
> > > > > > efficient
> > > > > > > > > than
> > > > > > > > > > > keep trying yourself.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 2. Not 100% sure this PR has no license issue. (about
> > > KniteR)
> > > > > > > > > > > What license problem *specifically* do you believe may
> > > exist?
> > > > > > > > > > >
> > > > > > > > > > > In preparing the PR, I:
> > > > > > > > > > >
> > > > > > > > > > > * Reviewed the Apache policy in detail.
> > > > > > > > > > >
> > > > > > > > > > > * Contacted the FSF to ask their interpretation of the
> > > > “linking”
> > > > > > > > > > > provisions of the GPL license.
> > > > > > > > > > >
> > > > > > > > > > > * Reviewed how other Apache software deals with this
> > issue
> > > > > > (e.g., Spark
> > > > > > > > > > > talks to R, which is GPL'd).
> > > > > > > > > > >
> > > > > > > > > > > * No necessary *dependencies* of the PR have license
> > > > conflicts.
> > > > > > In
> > > > > > > > > > > several cases, I contacted package authors who agreed
> to
> > > > > > re-issue their
> > > > > > > > > > > packages under Apache-compatible licenses. (Usually I
> had
> > > to
> > > > do
> > > > > > a bit
> > > > > > > > > of
> > > > > > > > > > > coding in exchange…)
> > > > > > > > > > >
> > > > > > > > > > > * Where the license had to stay GPL, the packages are
> > *not
> > > > > > necessary*
> > > > > > > > > and
> > > > > > > > > > > *not dependencies.* If the optional packages are
> present,
> > > > they
> > > > > > will be
> > > > > > > > > > > used to enable additional functionality. Knitr is an
> > > example.
> > > > > > The PR
> > > > > > > > > will
> > > > > > > > > > > compile and run fine without knitr. If knitr is
> available
> > > > (it is
> > > > > > part
> > > > > > > > > of
> > > > > > > > > > > the most common R distribution), the PR will enable the
> > > knitr
> > > > > > > > > > interpreter.
> > > > > > > > > > >
> > > > > > > > > > > * This is exactly how this issue is addressed through
> the
> > > > Apache
> > > > > > > > > > > ecosystem.
> > > > > > > > > > > The tl;dr is this: When Apache code is written to talk
> to
> > > > > > libraries
> > > > > > > > > that
> > > > > > > > > > > may or may not be present on the user’s system, or
> where
> > > it
> > > > > > talks to an
> > > > > > > > > > API
> > > > > > > > > > > but is agnostic about implementation, that is not
> > > “linking”
> > > > in a
> > > > > > way
> > > > > > > > > that
> > > > > > > > > > > implicate the anti-linking provision of the GPL.
> > > > > > > > > > >
> > > > > > > > > > > Otherwise, no Apache code could ever call a shell
> script!
> > > Let
> > > > > > alone run
> > > > > > > > > > > on Linux, or talk to R.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I'm not a legal expert. So following could be wrong.
> > > > > > > > > > >
> > > > > > > > > > > In my interpretation, KnitRInterpreter is not an
> optional
> > > > > > feature while
> > > > > > > > > > it
> > > > > > > > > > > is always enabled when compiling Zeppelin and always
> > > enabled
> > > > when
> > > > > > > > > running
> > > > > > > > > > > Zeppelin. And it requires dynamically linked GPL
> library
> > > on
> > > > > > runtime.
> > > > > > > > > (yes
> > > > > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > > > > >
> > > > > > > > > > > And of course, no Apache code can ever call a shell
> > > script,
> > > > on
> > > > > > the
> > > > > > > > > > purpose
> > > > > > > > > > > of dynamic linking with GPL library.
> > > > > > > > > > >
> > > > > > > > > > > I was guessing SparkR can be still in Apache License
> even
> > > if
> > > > it
> > > > > > is
> > > > > > > > > > depends
> > > > > > > > > > > on R. Because of GPL licensed compiler generated output
> > is
> > > > not
> > > > > > GPL
> > > > > > > > > > license.
> > > > > > > > > > > and R is sort of compiler.
> > > > > > > > > > >
> > > > > > > > > > > If you can get answer from Spark community how SparkR
> get
> > > > > > managed to
> > > > > > > > > stay
> > > > > > > > > > > in Apache License while R is GPL, the answer might
> help.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 3. Need more time to review.
> > > > > > > > > > > Has any reviewer has downloaded the PR or run the demo
> > > > notebook?
> > > > > > (Which
> > > > > > > > > > > is there for the benefit of reviewers, and isn’t
> intended
> > > to
> > > > go
> > > > > > in
> > > > > > > > > final
> > > > > > > > > > > distribution.)
> > > > > > > > > > >
> > > > > > > > > > > How many +1 comments from users would you like to see?
> > > > > > > > > > >
> > > > > > > > > > > How much time do you believe is required?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > It all depends on when CI is going to pass, when
> license
> > > > problem
> > > > > > is
> > > > > > > > > going
> > > > > > > > > > > to be cleared, and when a committer willing to review
> and
> > > > > > responsible
> > > > > > > > > to
> > > > > > > > > > > commit your contribution.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 1. Large code base change
> > > > > > > > > > > Large code base changes require coordination and
> > > cooperation.
> > > > > > This PR
> > > > > > > > > > > necessarily implicates the build scripts, testing code,
> > > the
> > > > > > > > > > > SparkInterpreter, etc.
> > > > > > > > > > >
> > > > > > > > > > > I have been seeking to coordinate since August.
> > > > > > > > > > >
> > > > > > > > > > > I continue to stand ready to do so.
> > > > > > > > > > >
> > > > > > > > > > > -Amos
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > If i give you one suggestion, Zeppelin committers
> > > sometimes
> > > > ask
> > > > > > rebase
> > > > > > > > > > the
> > > > > > > > > > > contribution branch for some reason. It is not the
> really
> > > the
> > > > > > best
> > > > > > > > > > > practice, but still okay while most contributions are
> not
> > > > > > including
> > > > > > > > > large
> > > > > > > > > > > code base changes.
> > > > > > > > > > >
> > > > > > > > > > > However, your one, has more than 4000 lines of code
> > > change.
> > > > If
> > > > > > you
> > > > > > > > > rebase
> > > > > > > > > > > then review should be started from the beginning,
> again.
> > > So
> > > > you
> > > > > > might
> > > > > > > > > > want
> > > > > > > > > > > to minimize the rebase your branch.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > moon
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > > Date: December 1, 2015 at 1:34:19 AM
> > > > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > > > > dev@zeppelin.incubator.apache.org
> > > > > > > > > > >
> > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin
> > > > > > > > > pull
> > > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > > >
> > > > > > > > > > > Hi Cos,
> > > > > > > > > > >
> > > > > > > > > > > Thanks for opening a discussion.
> > > > > > > > > > > My answer about 'Why this PR is open for three months'
> is
> > > > > > > > > > >
> > > > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > > > 2. Not 100% sure this PR has no license issue. (about
> > > KniteR)
> > > > > > > > > > > 3. Need more time to review.
> > > > > > > > > > >
> > > > > > > > > > > It's my personal answer, other committers may have
> > > different
> > > > > > opinion.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I think the question should be generalized. Because
> this
> > > PR
> > > > is
> > > > > > not the
> > > > > > > > > > only
> > > > > > > > > > > PR that is in impasse. There're more. For example
> > > > > > > > > > >
> > > > > > > > > > > Here's some examples that PR is not been merged.
> > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,
> > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/60
> > > > > > > > > > > and so on.
> > > > > > > > > > >
> > > > > > > > > > > I can categorize the cases, based on experience of
> > > involving
> > > > > > Zeppelin
> > > > > > > > > > > community from the beginning.
> > > > > > > > > > >
> > > > > > > > > > > 1. Large code base change
> > > > > > > > > > >
> > > > > > > > > > > When contribution has large code base changes, it tend
> to
> > > > take
> > > > > > more
> > > > > > > > > time
> > > > > > > > > > to
> > > > > > > > > > > review and merged. Normally, most contributions merged
> in
> > > > 1day~1
> > > > > > week.
> > > > > > > > > > > But some contribution has large code base changes take
> > 2~4
> > > > > > weeks. Few
> > > > > > > > > > > contribution that has very large code base change take
> > > > months.
> > > > > > > > > > >
> > > > > > > > > > > 2. Communication lost
> > > > > > > > > > >
> > > > > > > > > > > Sometimes, committer is not responding, or contributor
> is
> > > not
> > > > > > > > > responding.
> > > > > > > > > > >
> > > > > > > > > > > 3. Opinion diverges
> > > > > > > > > > >
> > > > > > > > > > > For some changes, included in contribution. When people
> > > have
> > > > > > different
> > > > > > > > > > > opinion and it continue to diverges, those PRs are not
> > > been
> > > > > > merged.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I think having a guide such as ping other committer
> when
> > a
> > > > > > committer is
> > > > > > > > > > not
> > > > > > > > > > > responding, and divide contribution into small peaces
> if
> > > > > > possible,
> > > > > > > > > would
> > > > > > > > > > > help most of the cases.
> > > > > > > > > > > Of course committer have to pay attention more to the
> > > > > > contribution and
> > > > > > > > > > > helping people. That's the first one.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > moon
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <
> > > > > > cos@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > To make sure we're on the same page, here are two
> > > threads
> > > > that
> > > > > > I
> > > > > > > > > found
> > > > > > > > > > > > related
> > > > > > > > > > > > to this topic.
> > > > > > > > > > > >
> > > > > > > > > > > > Thread 1:
> > > > > > > > > > > > Subject: R?
> > > > > > > > > > > > Started on: July 1, 2015
> > > > > > > > > > > >
> > > > > > > > > > > > Thread 2:
> > > > > > > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R
> > > > > > Interpreter for
> > > > > > > > > > > > Zeppelin
> > > > > > > > > > > > Started on: August 13, 2015
> > > > > > > > > > > >
> > > > > > > > > > > > If you want to fetch these from our archive send
> emails
> > > to
> > > > > > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > > > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > > > > > > > > > >
> > > > > > > > > > > > Cos
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik
> > > wrote:
> > > > > > > > > > > > > Guys,
> > > > > > > > > > > > >
> > > > > > > > > > > > > While catching up on my emails from the last a
> couple
> > > of
> > > > > > weeks,
> > > > > > > > > this
> > > > > > > > > > > > thread
> > > > > > > > > > > > > caught my attention. I am not normally paying much
> > > > attention
> > > > > > to the
> > > > > > > > > > > code
> > > > > > > > > > > > > reviews traffic from GH, but this one is pretty
> > > > different as
> > > > > > it
> > > > > > > > > spans
> > > > > > > > > > > > three
> > > > > > > > > > > > > months and counting.
> > > > > > > > > > > > >
> > > > > > > > > > > > > First, here are my five cents:
> > > > > > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is
> > aimed
> > > > to be
> > > > > > > > > > > > contributed to
> > > > > > > > > > > > > an ASF project this file should simply contain ASL2
> > > text,
> > > > > > like in
> > > > > > > > > [1]
> > > > > > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate
> > > > <developers>
> > > > > > > > > > section,
> > > > > > > > > > > > but
> > > > > > > > > > > > > Zeppelin might have different guidelines on it.
> Say,
> > > > Bigtop
> > > > > > doesn't
> > > > > > > > > > > > > maintain this in the source code, but we have the
> > list
> > > of
> > > > > > all the
> > > > > > > > > > > > > committers on the project's site [2] Every new
> > > > committer's
> > > > > > first
> > > > > > > > > > > > commit is
> > > > > > > > > > > > > to update the team page ;)
> > > > > > > > > > > > > - comments like in
> > > > > > > > > > > >
> > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > > > > > > > > > >
> > > > > > > > > > > > > +/**
> > > > > > > > > > > > > + * Created by aelberg on 7/28/15.
> > > > > > > > > > > > > + */
> > > > > > > > > > > > >
> > > > > > > > > > > > > is better to be removed. It has been already
> > discussed
> > > > here
> > > > > > [3].
> > > > > > > > > And
> > > > > > > > > > > > the
> > > > > > > > > > > > > initial discussion on the topic [4] was linked as
> > well
> > > > > > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not
> > > sure
> > > > if
> > > > > > this is
> > > > > > > > > > > > R-specific
> > > > > > > > > > > > > stuff - I have no idea about R, honestly, but
> > > > > > > > > > > > >
> > > > > > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > > > > > > > > > > ...
> > > > > > > > > > > > > +Author: David B. Dahl
> > > > > > > > > > > > >
> > > > > > > > > > > > > shouldn't be here, IMO. Normally, if some
> additional
> > > > > > licenses are
> > > > > > > > > > > > used,
> > > > > > > > > > > > > they have to be listed in the top-level NOTICE file
> > > > (already
> > > > > > > > > there).
> > > > > > > > > > > > >
> > > > > > > > > > > > > - I am not going to offer any comments on the
> > > technical
> > > > > > merits of
> > > > > > > > > the
> > > > > > > > > > > > patch,
> > > > > > > > > > > > > as I haven't tried it personally. However, I don't
> > see
> > > > any
> > > > > > serious
> > > > > > > > > > > > > technical objections to the functionality in
> > question.
> > > > > > > > > > > > >
> > > > > > > > > > > > > So, the question is - why the PR is open for three
> > > > months? I
> > > > > > hasn't
> > > > > > > > > > > been
> > > > > > > > > > > > able
> > > > > > > > > > > > > to get a clear answer. What I found out though is
> > > pretty
> > > > > > > > > unsettling,
> > > > > > > > > > > > really.
> > > > > > > > > > > > > The communication on the topic is almost
> > non-existent,
> > > > > > except for
> > > > > > > > > > this
> > > > > > > > > > > > sparse
> > > > > > > > > > > > > and bitter thread in the GH.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I would love to hear from the committers what's
> > > stopping
> > > > the
> > > > > > > > > > acceptance
> > > > > > > > > > > > of the
> > > > > > > > > > > > > code, besides of the minor issues I've mentioned
> > > earlier?
> > > > > > What are
> > > > > > > > > > the
> > > > > > > > > > > > reasons for it?
> > > > > > > > > > > > > Is there anything wrong with it?
> > > > > > > > > > > > > One of the responsibilities of the committers is to
> > > make
> > > > > > sure the
> > > > > > > > > > > > > contributions are reviewed; new contributors are
> > > guided
> > > > and
> > > > > > do
> > > > > > > > > > > > understand how
> > > > > > > > > > > > > the project ticks. The easy feedback flow attracts
> > new
> > > > > > people,
> > > > > > > > > > allowing
> > > > > > > > > > > > the
> > > > > > > > > > > > > community to grow and thrive.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I am asking _explicitely_ not to re-start the
> > > bickering I
> > > > > > have
> > > > > > > > > > already
> > > > > > > > > > > > > seen. At this point I am interested in the purely
> > > > technical
> > > > > > side of
> > > > > > > > > > > this.
> > > > > > > > > > > > >
> > > > > > > > > > > > > [1]
> > > https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > > > > > > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > > > > > > > > > [3]
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > >
> > >
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > > > > > > > > > [4] http://s.apache.org/iZl
> > > > > > > > > > > > >
> > > > > > > > > > > > > With regards,
> > > > > > > > > > > > > Cos
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > > > > > > > > > Github user elbamos commented on the pull
> request:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The current push should resolve some issues with
> > > > changes
> > > > > > in the
> > > > > > > > > > > > > > Spark-Zeppelin interface that had created issues
> > for
> > > > > > users, as
> > > > > > > > > > > > well as
> > > > > > > > > > > > > > support for 1.5.1.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Knitr support is improved, and the reason for a
> > > > separate
> > > > > > knitr
> > > > > > > > > > > > interpreter may be clearer now.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The amount of code borrowed from rscala is
> reduced.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I did not address issues with @author tags, or
> > files
> > > > under
> > > > > > the R/
> > > > > > > > > > > > > > folder. The reason is, to be blunt, I don't
> > > understand
> > > > > > what the
> > > > > > > > > > > > precise
> > > > > > > > > > > > > > concerns actually are.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Please note that I changed .travis.yml to only
> use
> > > > spark
> > > > > > 1.4 and
> > > > > > > > > > > > later.
> > > > > > > > > > > > > > I'm sure there is a better way to do it, but I'm
> > > just
> > > > not
> > > > > > in a
> > > > > > > > > > > > position
> > > > > > > > > > > > > > to try to figure out the entire Zeppelin build
> > > process.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Integrating this with the rest of zeppelin is
> going
> > > to
> > > > > > take some
> > > > > > > > > > > > work
> > > > > > > > > > > > > > regarding pom's, travis, and so forth. I can do a
> > > lot
> > > > of
> > > > > > that,
> > > > > > > > > > > > but I'm
> > > > > > > > > > > > > > going to need to discuss it with the people who
> > have
> > > > been
> > > > > > > > > "owning"
> > > > > > > > > > > > those
> > > > > > > > > > > > > > files. There are just too many moving pieces
> here.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Because of the size of this (which is,
> > > unfortunately,
> > > > > > necessary),
> > > > > > > > > > > > > > posting here is probably not the most efficient
> > way.
> > > > That
> > > > > > is also
> > > > > > > > > > > > true
> > > > > > > > > > > > > > because certain people chose to use this PR as a
> > > forum
> > > > to
> > > > > > air
> > > > > > > > > other
> > > > > > > > > > > > > > issues. Therefore, it would be better if
> reviewers
> > > > e-mail
> > > > > > me
> > > > > > > > > > > > directly.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > >
> > >
> > >
> >
>

Re: R-Interpreter CI Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Alexander Bezzubov <bz...@apache.org>]

Hi,
I have experience.thouse and also think its definitly a good time to take
some actions.

May be, as a first step in dealing with such test failures, we could
document the process i.e "file a jira issue labeld 'flacky-test' with the
link to CI logs"?

https://issues.apache.org/jira/browse/ZEPPELIN-476?jql=labels%20%3D%20flaky-test

This will narow down the list of 'suspects' that need to be taken care of.

Its a bit easear to collaborate on particular issues then just helping to
"make it pass in tis PR"

What do you guys think?

On Tue, Dec 8, 2015, 18:29 DuyHai Doan <do...@gmail.com> wrote:

> I have also experienced CI failing in the pass for some PRs that do not
> impact the code (Cassandra documentation). I guess that during peak hours,
> the CI servers may be too busy or enter a dead lock so the build fails.
>
>  At least that's my guess. What do you think ?
>
> On Tue, Dec 8, 2015 at 9:42 AM, moon soo Lee <mo...@apache.org> wrote:
>
> > Let me run some CI test with your branch and share result here.
> > Hope i can narrow down the cause and that helps involvement of more
> people.
> >
> > Thanks,
> > moon
> >
> > On Tue, Dec 8, 2015 at 3:08 PM Amos B. Elberg <am...@gmail.com>
> > wrote:
> >
> > > Is there anyone who can work with me on the CI issues?  It looks like
> > > there are a number of PRs experiencing similar things.
> > >
> > > I think we should make getting CI stable to be a priority.  Because it
> > > will save everyone a lot of frustration and aggravation if CI works
> > > reliably.
> > >
> > > Is there anyone other than Jongyoul and Moon who knows the CI/Build
> > > process well?
> > >
> > > (Moon - Thank you for taking another look at the licensing issue.  Per
> > the
> > > e-mail I wrote about this a few days ago, I don’t feel I have more to
> > > contribute to the licensing discussion, so I’m going to try not to
> > comment
> > > further about it.)
> > >
> > > From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> > > Reply: dev@zeppelin.incubator.apache.org
> > > <de...@zeppelin.incubator.apache.org> <dev@zeppelin.incubator.apache.org
> >
> > > Date: December 7, 2015 at 5:00:08 PM
> > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org
> > >
> > > <de...@zeppelin.incubator.apache.org>
> > > Subject:  Re: License of KnitRInterpreter Was: Re: contributions
> impasse.
> > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for
> Zeppelin
> > >
> > > Thanks Amos, Roman, Cos for clarifying license issue.
> > >
> > > I'm convinced that this license issue will not be a blocker.
> > >
> > > In my understanding, these are good sign,
> > >
> > > 1. any gpl licensed source codes are not included in the source package
> > > 2. any gpl licensed libraries are not included in the binary package
> > >
> > > However, i can not still 100% sure about
> > >
> > > 3. any gpl licensed libraries are not linked on runtime
> > >
> > > Even after Amos's explanation. I still think using 'knitr' is one of
> the
> > > clear case that 'knir' is linked to 'R' according to
> > > http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL.
> > >
> > > Giving input and getting output from GPL licensed interpreter (includes
> > R)
> > > from Apache licensed software is not a problem. That's not the point.
> > > Let me say in this way. There's an java code,
> > >
> > > import com.mysql.jdbc.Driver
> > > Driver driver = new Driver()
> > >
> > > Say without this java code, one of important feature of Zeppelin does
> not
> > > work. And Zeppelin does not includes GPL licensed file in the source
> > > package, GPL licensed library in the binary package, but it requires
> GPL
> > > licensed library on the runtime.
> > > In this case, will this java code be a license problem or not?
> > >
> > > In other words, my question is
> > >
> > > a) Is runtime GPL library dependency allowed in ASF release?
> > > b) is 'knitr' considered as runtime dependency?
> > >
> > > If someone can clarify a), b), then it would be extremely helpful
> > > understanding this case, and possible similar cases, too.
> > >
> > > Thanks,
> > > moon
> > >
> > > On Mon, Dec 7, 2015 at 3:20 PM Konstantin Boudnik <co...@apache.org>
> > wrote:
> > >
> > > > On Thu, Dec 03, 2015 at 11:03AM, Corneau Damien wrote:
> > > > > Thanks Cos for those answers,
> > > > >
> > > > > If I'm right you are advising to have a default build that doesn't
> > > > include
> > > > > libraries with conflicting licenses, but have an option to include
> > > them
> > > > for
> > > > > users who wants to build the project themselves.
> > > >
> > > > Yes, that's what I said. Besides, looks like Roman provided the
> second
> > > > pair of
> > > > eyes to this licensing discussion and as well didn't find any issues
> > > with
> > > > the
> > > > current approach.
> > > >
> > > > Cos
> > > >
> > > > > To refer to another thread about decentralizing interpreters, it
> > could
> > > > even
> > > > > be better in that case to have some interpreters separated from the
> > > tree,
> > > > > and easily pluggable with a release instead of forcing users to
> build
> > > the
> > > > > project to use them.
> > > > >
> > > > >
> > > > > On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <cos@apache.org
> >
> > > > wrote:
> > > > >
> > > > > > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:
> > > > > > > Konstantin thank you for getting into this.
> > > > > > >
> > > > > > >> The best way to go around it is by
> > > > > > >> providing a build-time option that will pull such binaries in.
> > > But
> > > > by
> > > > > > default
> > > > > > >> such libs shouldn't be pulled.
> > > > > > >
> > > > > > > That is basically how the PR handles this. If the GPL’d
> > > interpreter
> > > > > > scripts
> > > > > > > are missing, there’s no indication at all at build time. It
> > > doesn’t
> > > > try
> > > > > > to
> > > > > > > download them.
> > > > > > >
> > > > > > > At runtime, if the user tries to use functionality that would
> > need
> > > > such a
> > > > > > > script (i.e., if they type “knitr” to use knitr), we display an
> > > error
> > > > > > that
> > > > > > > says that the functionality is not there because the library is
> > > > missing,
> > > > > > and
> > > > > > > that the library cannot be provided because it has an
> > incompatible
> > > > > > license,
> > > > > > > but the user can download it themselves if they want.
> > > > > > >
> > > > > > > And, in the log, if the logging level is high, they will see a
> > > note
> > > > that
> > > > > > > some functionality was disabled because the libraries weren’t
> > > there.
> > > > > > >
> > > > > > > To be clear, though, none of these libraries are binaries.
> > They’re
> > > > all
> > > > > > interpreter scripts.
> > > > > >
> > > > > > If you ever in a doubt of which licenses could be used for
> > > dependncies
> > > > > > (not to
> > > > > > say about source code) are listed in Category A list of [1]
> > > > > >
> > > > > > A lot of quesitons discussed here are already covered in the
> legal
> > > > FAQ, so
> > > > > > just check against it if you have any questions.
> > > > > >
> > > > > > [1] http://www.apache.org/legal/resolved.html#category-a
> > > > > >
> > > > > > Cos
> > > > > >
> > > > > > > From: Konstantin Boudnik <co...@apache.org>
> > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>,
> > > dev@zeppelin.incubator.apache.org
> > > > <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > Date: December 2, 2015 at 3:24:50 PM
> > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > dev@zeppelin.incubator.apache.org
> > > > > > >
> > > > > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions
> > > > > > impasse. Was: [GitHub] incubator-zeppelin pull request: R
> > > Interpreter
> > > > for
> > > > > > Zeppelin
> > > > > > >
> > > > > > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:
> > > > > > > > I think that what moon means is that:
> > > > > > > > If we merge the way it is now, the KnitRInterpreter will be
> > part
> > > > of the
> > > > > > > > code base, so it isn't optional at code base level.
> > > > > > > >
> > > > > > > > To make it even more simple:
> > > > > > > > * If the code has the right licensing -> that code can be
> part
> > > of
> > > > the
> > > > > > > > source code, and can be including in a binary release
> > > > > > >
> > > > > > > We aren't concerned with binary releases - as an Apache
> community
> > > we
> > > > are
> > > > > > > voting and releasing source code. If the project wants to
> provide
> > > a
> > > > > > binary
> > > > > > > release to its users, they are better be warned about inclusion
> > of
> > > > non
> > > > > > > ASL2-friendly licensed bits.
> > > > > > >
> > > > > > > > * If the code doesn't have the right licensing -> it can't be
> > > part
> > > > of
> > > > > > the
> > > > > > > > source code, and can't be included in a binary release
> > > > > > >
> > > > > > > See above.
> > > > > > >
> > > > > > > > * If the code doesn't have the right licensing but is
> imported
> > > at
> > > > build
> > > > > > > > time (libraries for example) -> it is not in the source code,
> > it
> > > > can't
> > > > > > be
> > > > > > > > included in binary release
> > > > > > >
> > > > > > > That is unless a user doing it on his own. The best way to go
> > > around
> > > > it
> > > > > > is by
> > > > > > > providing a build-time option that will pull such binaries in.
> > But
> > > by
> > > > > > default
> > > > > > > such libs shouldn't be pulled.
> > > > > > >
> > > > > > > Cos
> > > > > > >
> > > > > > > > So in the case of licensing issues, it would need to be fully
> > > > optional
> > > > > > > > (user bring the interpreter in his directory and build
> Zeppelin
> > > > with
> > > > > > it)
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <
> > > > amos.elberg@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Moon let me clarify:
> > > > > > > > >
> > > > > > > > > Interpreted code doesn’t “link.” The wiki article actually
> > > > explains
> > > > > > it
> > > > > > > > > pretty well —
> > > > https://en.wikipedia.org/wiki/GPL_linking_exception
> > > > > > > > > “Linking” against a library means compiling its headers
> into
> > a
> > > > > > binary, the
> > > > > > > > > way a C compiler works. The 2008 e-mail Moon distributed,
> > > called
> > > > > > this the
> > > > > > > > > “interpreter exception.”
> > > > > > > > >
> > > > > > > > > As for whether GPL’d code is a “mandatory dependency,” if
> > > knitr
> > > > is
> > > > > > missing
> > > > > > > > > the PR will compile, run and test just fine. The user is
> > never
> > > > > > prompted to
> > > > > > > > > download it. The only effect is, if the user types “knitr”
> > and
> > > > knitr
> > > > > > isn’t
> > > > > > > > > there, we display an InterpreterError that knitr isn’t
> there.
> > > > > > > > >
> > > > > > > > > KnitRInterpreter is not optionally required. so it does not
> > > > matter
> > > > > > KnitR
> > > > > > > > > is
> > > > > > > > > optionally required or not.
> > > > > > > > > Aren’t all interpreters optional? You can turn them on and
> > off
> > > > in the
> > > > > > > > > config files.
> > > > > > > > >
> > > > > > > > > Do you mean that the KnitRInterpreter class gets compiled
> to
> > > > > > bytecode even
> > > > > > > > > if knitr is missing? So what? That isn't a mandatory
> > > dependency
> > > > or a
> > > > > > link.
> > > > > > > > >
> > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > Date: December 2, 2015 at 3:18:00 AM
> > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > Subject: Re: License of KnitRInterpreter Was: Re:
> > > contributions
> > > > > > impasse.
> > > > > > > > > Was: [GitHub] incubator-zeppelin pull request: R
> Interpreter
> > > for
> > > > > > Zeppelin
> > > > > > > > >
> > > > > > > > > Let me summarize license concern about KnitRInterpreter.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Amos's interpretation.
> > > > > > > > >
> > > > > > > > > KnitR is optionally required by KnitRInterpreter.
> > > > > > > > > R dependency in SparkR has no problem. So KnitR should be
> the
> > > > same.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Moon's interpretation.
> > > > > > > > >
> > > > > > > > > KnitRInterpreter is not optionally required. so it does not
> > > > matter
> > > > > > KnitR is
> > > > > > > > > optionally required or not.
> > > > > > > > > R dependency in SparkR is exception of GPL. KnitR is not
> > > applied
> > > > that
> > > > > > > > > exception.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Amos, could you confirm my understanding to your
> > > interpretation
> > > > is
> > > > > > correct?
> > > > > > > > > If it is not could you clarify it?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > moon
> > > > > > > > >
> > > > > > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Just to put the final nail in this, I looked it up.
> > > > > > > > > >
> > > > > > > > > > I see no evidence of any “compiler exception.”
> > > > > > > > > >
> > > > > > > > > > There is an exception in the license for the runtime
> > > libraries
> > > > > > that are
> > > > > > > > > > bundled with GCC. See:
> > > > > > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> > > > > > > > > >
> > > > > > > > > > But no “compiler exception.”
> > > > > > > > > >
> > > > > > > > > > In fact, I believe this is part of the reason why LLVM
> was
> > > > created.
> > > > > > > > > >
> > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Date: December 1, 2015 at 8:16:36 PM
> > > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin pull
> > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > >
> > > > > > > > > > Is knitR is commonly considered as a
> interpreter/compiler?
> > > or
> > > > is it
> > > > > > > > > > considered as a library routine?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > moon
> > > > > > > > > >
> > > > > > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > Moon - you give this as an explanation of the licensing
> > > issue:
> > > > > > > > > >
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > > > >
> > > > > > > > > > According to that, there is an exception in the GPL for
> > > > interpreter
> > > > > > > > > > languages. As long as you don’t distribute the code, its
> > > fine
> > > > to
> > > > > > talk to
> > > > > > > > > > an interpreted language.
> > > > > > > > > >
> > > > > > > > > > Well, if that’s the case, then the PR plainly does not
> have
> > > a
> > > > > > license
> > > > > > > > > > issue. It doesn’t distribute any GPL’d R code.
> > > > > > > > > >
> > > > > > > > > > I’m not sure what’s confusing about this. It seems
> > > completely
> > > > > > > > > > straightforward.
> > > > > > > > > >
> > > > > > > > > > Regarding this:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Amos Elberg
> > > > > > > > > > Sent with Airmail
> > > > > > > > > >
> > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > Date: December 1, 2015 at 6:48:47 PM
> > > > > > > > > >
> > > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org
> > > > > > > > > >
> > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin pull
> > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > >
> > > > > > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I am going to try to minimize my reaction to Moon’s
> > > e-mail.
> > > > > > > > > > >
> > > > > > > > > > > The tl;dr is this:
> > > > > > > > > > >
> > > > > > > > > > > The reason we are having this discussion now is that
> > > active
> > > > > > users of
> > > > > > > > > the
> > > > > > > > > > > PR — which now has its own user base — went public to
> > > > complain
> > > > > > about
> > > > > > > > > > this.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > The PR has been tested by an active user base for more
> > > than
> > > > three
> > > > > > > > > months.
> > > > > > > > > > > No-one has been able to identify any specific actual
> > > > licensing
> > > > > > problem,
> > > > > > > > > > and
> > > > > > > > > > > the PR was prepared based on an extensive, careful
> review
> > > of
> > > > the
> > > > > > > > > relevant
> > > > > > > > > > > licensing issues and after contacting the relevant
> > people.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I admire every software that used by user and helping
> > > people.
> > > > That
> > > > > > > > > includes
> > > > > > > > > > your work. But that's not the topic we're in discussion.
> > > Active
> > > > > > user does
> > > > > > > > > > not mean your contribution can ignore the review.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > It is not an explanation for someone who has been
> > ignoring
> > > my
> > > > > > “how can
> > > > > > > > > I
> > > > > > > > > > > move this forward…” emails for three months to point
> the
> > > > finger
> > > > > > and
> > > > > > > > > say I
> > > > > > > > > > > didn’t contact the right person or file the right
> report.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > This is also not the topic in this discussion.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > The burden for providing an explanation for the
> inaction
> > > is
> > > > on
> > > > > > the PMCC
> > > > > > > > > > at
> > > > > > > > > > > this point.
> > > > > > > > > >
> > > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's
> > problem
> > > on
> > > > > > Zeppelin
> > > > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > > > > They’re not! I often see comments on PRs to just ignore
> > > that
> > > > CI
> > > > > > is
> > > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > One of the most common reasons this PR fails CI, is CI
> > > > times-out
> > > > > > > > > > > downloading Spark to install. How could that possibly
> be
> > > > caused
> > > > > > by the
> > > > > > > > > > PR?
> > > > > > > > > > >
> > > > > > > > > > > It looks to me like the only PRs with changes to the
> > > relevant
> > > > > > parts of
> > > > > > > > > > the
> > > > > > > > > > > code — the SparkInterpreter — are being made by the
> > person
> > > > who
> > > > > > wrote
> > > > > > > > > the
> > > > > > > > > > > testing suite.
> > > > > > > > > > >
> > > > > > > > > > > So, that would explain why some other PRs pass CI:
> > Neither
> > > > the
> > > > > > > > > > > SparkInterpreter nor the testing suite are stable or
> > > robust,
> > > > but
> > > > > > since
> > > > > > > > > > the
> > > > > > > > > > > PRs are coming from the person who wrote both…
> > > > > > > > > > >
> > > > > > > > > > > And let's say Zeppelin core has problem and that makes
> > > your
> > > > PR
> > > > > > fails on
> > > > > > > > > > CI
> > > > > > > > > > > test. That's possible. But it still does not mean we
> can
> > > > merge
> > > > > > it with
> > > > > > > > > CI
> > > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > It means you should be working with me to figure out
> why
> > > the
> > > > CI
> > > > > > is
> > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > This PR has been tested by an active user base for the
> > > past
> > > > three
> > > > > > > > > months.
> > > > > > > > > > > If CI is continuing to fail, and dozens of hours of
> > effort
> > > > have
> > > > > > not
> > > > > > > > > > > resolved the CI issues, then it is time to start
> > > considering
> > > > > > whether
> > > > > > > > > the
> > > > > > > > > > > testing suite is part of the problem.
> > > > > > > > > > >
> > > > > > > > > > > The level of defensiveness about the CI and
> > > SparkInterpreter
> > > > is
> > > > > > not
> > > > > > > > > > > helping to resolve these issues.
> > > > > > > > > > >
> > > > > > > > > > > If you think it's problem on Zeppelin core, then file
> an
> > > > issue
> > > > > > that
> > > > > > > > > > > reproduce the problem on Zeppelin core, that might be
> > more
> > > > > > efficient
> > > > > > > > > than
> > > > > > > > > > > keep trying yourself.
> > > > > > > > > > > I contacted you numerous times about such issues...
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I remember i commented your issue about CI. but you just
> > > keep
> > > > > > repeated
> > > > > > > > > it's
> > > > > > > > > > not your problem but Zeppelin core problem.
> > > > > > > > > >
> > > > > > > > > > Then please file an issue about the problem you found in
> > > > Zeppelin
> > > > > > Core.
> > > > > > > > > > Then everyone will get into the problem.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > In my interpretation, KnitRInterpreter is not an
> optional
> > > > > > feature while
> > > > > > > > > > it
> > > > > > > > > > > is always enabled when compiling Zeppelin and always
> > > enabled
> > > > when
> > > > > > > > > running
> > > > > > > > > > > Zeppelin. And it requires dynamically linked GPL
> library
> > > on
> > > > > > runtime.
> > > > > > > > > (yes
> > > > > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > > > > >
> > > > > > > > > > > Its not always enabled.
> > > > > > > > > > > It is not dynamically linked at runtime.
> > > > > > > > > > > It will not fail when knitr is missing. If knitr is not
> > > > present,
> > > > > > the
> > > > > > > > > repl
> > > > > > > > > > > interpreter starts and a note is written to to the log
> > > that
> > > > the
> > > > > > knitr
> > > > > > > > > > > interpreter isn’t available because knitr is not
> present.
> > > > > > > > > > >
> > > > > > > > > > > no Apache code can ever call a shell script, on the
> > > purpose
> > > > of
> > > > > > dynamic
> > > > > > > > > > > linking with GPL library.
> > > > > > > > > > > You misunderstand.
> > > > > > > > > > >
> > > > > > > > > > > The *shell* is GPL'd.
> > > > > > > > > > >
> > > > > > > > > > > Is Zeppelin “linked" against the GPL’d shell because
> > > Zeppelin
> > > > > > depends
> > > > > > > > > on
> > > > > > > > > > a
> > > > > > > > > > > shell script to launch?
> > > > > > > > > > >
> > > > > > > > > > Obviously not.
> > > > > > > > > > >
> > > > > > > > > > > The interaction with R in the PR is the same.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Again, bash is one of exceptions of GPL, like other GPL
> > > > licensed
> > > > > > > > > > compiler/interpreter.
> > > > > > > > > >
> > > > > > > > > > Check here why Bash and R is okay with Apache License.
> > > > > > > > > >
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > > > >
> > > > > > > > > > I'm not sure we can apply the same exception for 'using'
> > > KnitR.
> > > > > > > > > >
> > > > > > > > > > My point is not 'KnitR' is optional or not. Point is
> > > > > > 'KnitRInterpreter'
> > > > > > > > > you
> > > > > > > > > > wrote is not an optional feature. Which is clearly not
> > > > optionally
> > > > > > enabled
> > > > > > > > > > code and feature. And that depends on KnitR library which
> > is
> > > > GPL.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > I was guessing SparkR can be still in Apache License
> even
> > > if
> > > > it
> > > > > > is
> > > > > > > > > > depends
> > > > > > > > > > > on R. Because of GPL licensed compiler generated output
> > is
> > > > not
> > > > > > GPL
> > > > > > > > > > license.
> > > > > > > > > > > and R is sort of compiler. If you can get answer from
> > > Spark
> > > > > > community
> > > > > > > > > how
> > > > > > > > > > > SparkR get managed to stay in Apache License while R is
> > > GPL,
> > > > the
> > > > > > answer
> > > > > > > > > > > might help.
> > > > > > > > > > > The description of SparkR is not accurate in any
> respect.
> > > (Do
> > > > > > you think
> > > > > > > > > > > SparkR is not talking to GPL-licensed libraries?)
> > > > > > > > > > >
> > > > > > > > > > > I don’t see that any genuine issue is being raised
> here.
> > > > > > > > > > >
> > > > > > > > > > > If there is an issue, the burden is on you to identify
> > it.
> > > > > > > > > > >
> > > > > > > > > > > If i give you one suggestion, Zeppelin committers
> > > sometimes
> > > > ask
> > > > > > rebase
> > > > > > > > > > the
> > > > > > > > > > > contribution branch for some reason. It is not the
> really
> > > the
> > > > > > best
> > > > > > > > > > > practice, but still okay while most contributions are
> not
> > > > > > including
> > > > > > > > > large
> > > > > > > > > > > code base changes
> > > > > > > > > > > However, your one, has more than 4000 lines of code
> > > change.
> > > > If
> > > > > > you
> > > > > > > > > rebase
> > > > > > > > > > > then review should be started from the beginning,
> again.
> > > So
> > > > you
> > > > > > might
> > > > > > > > > > want
> > > > > > > > > > > to minimize the rebase your branch.
> > > > > > > > > > >
> > > > > > > > > > > Are you actually complaining that the problem is that I
> > > > rebased
> > > > > > the
> > > > > > > > > code
> > > > > > > > > > > during the three-month period when no-one looked at it
> > and
> > > > > > Zeppelin
> > > > > > > > > went
> > > > > > > > > > > through a release?
> > > > > > > > > > >
> > > > > > > > > > > I cannot take it seriously when you say things like
> this.
> > > > > > > > > > >
> > > > > > > > > > > Having to “start from the beginning” cannot be a
> problem
> > > if
> > > > you
> > > > > > never
> > > > > > > > > > > actually started the first time...
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > You wanted coordination and cooperation. So i gave you
> > > > suggestion
> > > > > > that
> > > > > > > > > > helping review process. For example, your code has been
> > > rebased
> > > > > > since my
> > > > > > > > > > comment and jongyoul's comment. that means committers
> will
> > > > need to
> > > > > > look
> > > > > > > > > > from the beginning. That'll require more time. And
> maybe, i
> > > > guess
> > > > > > that's
> > > > > > > > > > not what you want. But If you don't care, feel free to
> > > rebase.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > moon
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Date: December 1, 2015 at 4:42:06 AM
> > > > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin
> > > > > > > > > pull
> > > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <
> > > > > > amos.elberg@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > Thank you, Cos.
> > > > > > > > > > >
> > > > > > > > > > > I’d like to briefly address the issues raised by Moon:
> > > > > > > > > > >
> > > > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > > > The CI fails on core Zeppelin, *not* code in this PR.
> > > > > > > > > > >
> > > > > > > > > > > I’ve been seeking assistance on this since August.
> > > > > > > > > > >
> > > > > > > > > > > The most common reason is that SparkInterpreter is
> unable
> > > to
> > > > > > launch
> > > > > > > > > Spark
> > > > > > > > > > > and open a Spark Backend. This is necessary to test the
> > > PR.
> > > > > > > > > > >
> > > > > > > > > > > 60+ hours, has been spent adapting and re-basing when
> the
> > > > > > > > > > SparkInterpreter
> > > > > > > > > > > architecture changed and broke the PR’s *tests.*
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's
> > > problem
> > > > on
> > > > > > > > > Zeppelin
> > > > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > > > >
> > > > > > > > > > > And let's say Zeppelin core has problem and that makes
> > > your
> > > > PR
> > > > > > fails on
> > > > > > > > > > CI
> > > > > > > > > > > test. That's possible. But it still does not mean we
> can
> > > > merge
> > > > > > it with
> > > > > > > > > CI
> > > > > > > > > > > failing.
> > > > > > > > > > >
> > > > > > > > > > > If you think it's problem on Zeppelin core, then file
> an
> > > > issue
> > > > > > that
> > > > > > > > > > > reproduce the problem on Zeppelin core, that might be
> > more
> > > > > > efficient
> > > > > > > > > than
> > > > > > > > > > > keep trying yourself.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 2. Not 100% sure this PR has no license issue. (about
> > > KniteR)
> > > > > > > > > > > What license problem *specifically* do you believe may
> > > exist?
> > > > > > > > > > >
> > > > > > > > > > > In preparing the PR, I:
> > > > > > > > > > >
> > > > > > > > > > > * Reviewed the Apache policy in detail.
> > > > > > > > > > >
> > > > > > > > > > > * Contacted the FSF to ask their interpretation of the
> > > > “linking”
> > > > > > > > > > > provisions of the GPL license.
> > > > > > > > > > >
> > > > > > > > > > > * Reviewed how other Apache software deals with this
> > issue
> > > > > > (e.g., Spark
> > > > > > > > > > > talks to R, which is GPL'd).
> > > > > > > > > > >
> > > > > > > > > > > * No necessary *dependencies* of the PR have license
> > > > conflicts.
> > > > > > In
> > > > > > > > > > > several cases, I contacted package authors who agreed
> to
> > > > > > re-issue their
> > > > > > > > > > > packages under Apache-compatible licenses. (Usually I
> had
> > > to
> > > > do
> > > > > > a bit
> > > > > > > > > of
> > > > > > > > > > > coding in exchange…)
> > > > > > > > > > >
> > > > > > > > > > > * Where the license had to stay GPL, the packages are
> > *not
> > > > > > necessary*
> > > > > > > > > and
> > > > > > > > > > > *not dependencies.* If the optional packages are
> present,
> > > > they
> > > > > > will be
> > > > > > > > > > > used to enable additional functionality. Knitr is an
> > > example.
> > > > > > The PR
> > > > > > > > > will
> > > > > > > > > > > compile and run fine without knitr. If knitr is
> available
> > > > (it is
> > > > > > part
> > > > > > > > > of
> > > > > > > > > > > the most common R distribution), the PR will enable the
> > > knitr
> > > > > > > > > > interpreter.
> > > > > > > > > > >
> > > > > > > > > > > * This is exactly how this issue is addressed through
> the
> > > > Apache
> > > > > > > > > > > ecosystem.
> > > > > > > > > > > The tl;dr is this: When Apache code is written to talk
> to
> > > > > > libraries
> > > > > > > > > that
> > > > > > > > > > > may or may not be present on the user’s system, or
> where
> > > it
> > > > > > talks to an
> > > > > > > > > > API
> > > > > > > > > > > but is agnostic about implementation, that is not
> > > “linking”
> > > > in a
> > > > > > way
> > > > > > > > > that
> > > > > > > > > > > implicate the anti-linking provision of the GPL.
> > > > > > > > > > >
> > > > > > > > > > > Otherwise, no Apache code could ever call a shell
> script!
> > > Let
> > > > > > alone run
> > > > > > > > > > > on Linux, or talk to R.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I'm not a legal expert. So following could be wrong.
> > > > > > > > > > >
> > > > > > > > > > > In my interpretation, KnitRInterpreter is not an
> optional
> > > > > > feature while
> > > > > > > > > > it
> > > > > > > > > > > is always enabled when compiling Zeppelin and always
> > > enabled
> > > > when
> > > > > > > > > running
> > > > > > > > > > > Zeppelin. And it requires dynamically linked GPL
> library
> > > on
> > > > > > runtime.
> > > > > > > > > (yes
> > > > > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > > > > >
> > > > > > > > > > > And of course, no Apache code can ever call a shell
> > > script,
> > > > on
> > > > > > the
> > > > > > > > > > purpose
> > > > > > > > > > > of dynamic linking with GPL library.
> > > > > > > > > > >
> > > > > > > > > > > I was guessing SparkR can be still in Apache License
> even
> > > if
> > > > it
> > > > > > is
> > > > > > > > > > depends
> > > > > > > > > > > on R. Because of GPL licensed compiler generated output
> > is
> > > > not
> > > > > > GPL
> > > > > > > > > > license.
> > > > > > > > > > > and R is sort of compiler.
> > > > > > > > > > >
> > > > > > > > > > > If you can get answer from Spark community how SparkR
> get
> > > > > > managed to
> > > > > > > > > stay
> > > > > > > > > > > in Apache License while R is GPL, the answer might
> help.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 3. Need more time to review.
> > > > > > > > > > > Has any reviewer has downloaded the PR or run the demo
> > > > notebook?
> > > > > > (Which
> > > > > > > > > > > is there for the benefit of reviewers, and isn’t
> intended
> > > to
> > > > go
> > > > > > in
> > > > > > > > > final
> > > > > > > > > > > distribution.)
> > > > > > > > > > >
> > > > > > > > > > > How many +1 comments from users would you like to see?
> > > > > > > > > > >
> > > > > > > > > > > How much time do you believe is required?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > It all depends on when CI is going to pass, when
> license
> > > > problem
> > > > > > is
> > > > > > > > > going
> > > > > > > > > > > to be cleared, and when a committer willing to review
> and
> > > > > > responsible
> > > > > > > > > to
> > > > > > > > > > > commit your contribution.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 1. Large code base change
> > > > > > > > > > > Large code base changes require coordination and
> > > cooperation.
> > > > > > This PR
> > > > > > > > > > > necessarily implicates the build scripts, testing code,
> > > the
> > > > > > > > > > > SparkInterpreter, etc.
> > > > > > > > > > >
> > > > > > > > > > > I have been seeking to coordinate since August.
> > > > > > > > > > >
> > > > > > > > > > > I continue to stand ready to do so.
> > > > > > > > > > >
> > > > > > > > > > > -Amos
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > If i give you one suggestion, Zeppelin committers
> > > sometimes
> > > > ask
> > > > > > rebase
> > > > > > > > > > the
> > > > > > > > > > > contribution branch for some reason. It is not the
> really
> > > the
> > > > > > best
> > > > > > > > > > > practice, but still okay while most contributions are
> not
> > > > > > including
> > > > > > > > > large
> > > > > > > > > > > code base changes.
> > > > > > > > > > >
> > > > > > > > > > > However, your one, has more than 4000 lines of code
> > > change.
> > > > If
> > > > > > you
> > > > > > > > > rebase
> > > > > > > > > > > then review should be started from the beginning,
> again.
> > > So
> > > > you
> > > > > > might
> > > > > > > > > > want
> > > > > > > > > > > to minimize the rebase your branch.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > moon
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > > Date: December 1, 2015 at 1:34:19 AM
> > > > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > > > > dev@zeppelin.incubator.apache.org
> > > > > > > > > > >
> > > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > > incubator-zeppelin
> > > > > > > > > pull
> > > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > > >
> > > > > > > > > > > Hi Cos,
> > > > > > > > > > >
> > > > > > > > > > > Thanks for opening a discussion.
> > > > > > > > > > > My answer about 'Why this PR is open for three months'
> is
> > > > > > > > > > >
> > > > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > > > 2. Not 100% sure this PR has no license issue. (about
> > > KniteR)
> > > > > > > > > > > 3. Need more time to review.
> > > > > > > > > > >
> > > > > > > > > > > It's my personal answer, other committers may have
> > > different
> > > > > > opinion.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I think the question should be generalized. Because
> this
> > > PR
> > > > is
> > > > > > not the
> > > > > > > > > > only
> > > > > > > > > > > PR that is in impasse. There're more. For example
> > > > > > > > > > >
> > > > > > > > > > > Here's some examples that PR is not been merged.
> > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,
> > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/60
> > > > > > > > > > > and so on.
> > > > > > > > > > >
> > > > > > > > > > > I can categorize the cases, based on experience of
> > > involving
> > > > > > Zeppelin
> > > > > > > > > > > community from the beginning.
> > > > > > > > > > >
> > > > > > > > > > > 1. Large code base change
> > > > > > > > > > >
> > > > > > > > > > > When contribution has large code base changes, it tend
> to
> > > > take
> > > > > > more
> > > > > > > > > time
> > > > > > > > > > to
> > > > > > > > > > > review and merged. Normally, most contributions merged
> in
> > > > 1day~1
> > > > > > week.
> > > > > > > > > > > But some contribution has large code base changes take
> > 2~4
> > > > > > weeks. Few
> > > > > > > > > > > contribution that has very large code base change take
> > > > months.
> > > > > > > > > > >
> > > > > > > > > > > 2. Communication lost
> > > > > > > > > > >
> > > > > > > > > > > Sometimes, committer is not responding, or contributor
> is
> > > not
> > > > > > > > > responding.
> > > > > > > > > > >
> > > > > > > > > > > 3. Opinion diverges
> > > > > > > > > > >
> > > > > > > > > > > For some changes, included in contribution. When people
> > > have
> > > > > > different
> > > > > > > > > > > opinion and it continue to diverges, those PRs are not
> > > been
> > > > > > merged.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I think having a guide such as ping other committer
> when
> > a
> > > > > > committer is
> > > > > > > > > > not
> > > > > > > > > > > responding, and divide contribution into small peaces
> if
> > > > > > possible,
> > > > > > > > > would
> > > > > > > > > > > help most of the cases.
> > > > > > > > > > > Of course committer have to pay attention more to the
> > > > > > contribution and
> > > > > > > > > > > helping people. That's the first one.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > moon
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <
> > > > > > cos@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > To make sure we're on the same page, here are two
> > > threads
> > > > that
> > > > > > I
> > > > > > > > > found
> > > > > > > > > > > > related
> > > > > > > > > > > > to this topic.
> > > > > > > > > > > >
> > > > > > > > > > > > Thread 1:
> > > > > > > > > > > > Subject: R?
> > > > > > > > > > > > Started on: July 1, 2015
> > > > > > > > > > > >
> > > > > > > > > > > > Thread 2:
> > > > > > > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R
> > > > > > Interpreter for
> > > > > > > > > > > > Zeppelin
> > > > > > > > > > > > Started on: August 13, 2015
> > > > > > > > > > > >
> > > > > > > > > > > > If you want to fetch these from our archive send
> emails
> > > to
> > > > > > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > > > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > > > > > > > > > >
> > > > > > > > > > > > Cos
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik
> > > wrote:
> > > > > > > > > > > > > Guys,
> > > > > > > > > > > > >
> > > > > > > > > > > > > While catching up on my emails from the last a
> couple
> > > of
> > > > > > weeks,
> > > > > > > > > this
> > > > > > > > > > > > thread
> > > > > > > > > > > > > caught my attention. I am not normally paying much
> > > > attention
> > > > > > to the
> > > > > > > > > > > code
> > > > > > > > > > > > > reviews traffic from GH, but this one is pretty
> > > > different as
> > > > > > it
> > > > > > > > > spans
> > > > > > > > > > > > three
> > > > > > > > > > > > > months and counting.
> > > > > > > > > > > > >
> > > > > > > > > > > > > First, here are my five cents:
> > > > > > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is
> > aimed
> > > > to be
> > > > > > > > > > > > contributed to
> > > > > > > > > > > > > an ASF project this file should simply contain ASL2
> > > text,
> > > > > > like in
> > > > > > > > > [1]
> > > > > > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate
> > > > <developers>
> > > > > > > > > > section,
> > > > > > > > > > > > but
> > > > > > > > > > > > > Zeppelin might have different guidelines on it.
> Say,
> > > > Bigtop
> > > > > > doesn't
> > > > > > > > > > > > > maintain this in the source code, but we have the
> > list
> > > of
> > > > > > all the
> > > > > > > > > > > > > committers on the project's site [2] Every new
> > > > committer's
> > > > > > first
> > > > > > > > > > > > commit is
> > > > > > > > > > > > > to update the team page ;)
> > > > > > > > > > > > > - comments like in
> > > > > > > > > > > >
> > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > > > > > > > > > >
> > > > > > > > > > > > > +/**
> > > > > > > > > > > > > + * Created by aelberg on 7/28/15.
> > > > > > > > > > > > > + */
> > > > > > > > > > > > >
> > > > > > > > > > > > > is better to be removed. It has been already
> > discussed
> > > > here
> > > > > > [3].
> > > > > > > > > And
> > > > > > > > > > > > the
> > > > > > > > > > > > > initial discussion on the topic [4] was linked as
> > well
> > > > > > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not
> > > sure
> > > > if
> > > > > > this is
> > > > > > > > > > > > R-specific
> > > > > > > > > > > > > stuff - I have no idea about R, honestly, but
> > > > > > > > > > > > >
> > > > > > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > > > > > > > > > > ...
> > > > > > > > > > > > > +Author: David B. Dahl
> > > > > > > > > > > > >
> > > > > > > > > > > > > shouldn't be here, IMO. Normally, if some
> additional
> > > > > > licenses are
> > > > > > > > > > > > used,
> > > > > > > > > > > > > they have to be listed in the top-level NOTICE file
> > > > (already
> > > > > > > > > there).
> > > > > > > > > > > > >
> > > > > > > > > > > > > - I am not going to offer any comments on the
> > > technical
> > > > > > merits of
> > > > > > > > > the
> > > > > > > > > > > > patch,
> > > > > > > > > > > > > as I haven't tried it personally. However, I don't
> > see
> > > > any
> > > > > > serious
> > > > > > > > > > > > > technical objections to the functionality in
> > question.
> > > > > > > > > > > > >
> > > > > > > > > > > > > So, the question is - why the PR is open for three
> > > > months? I
> > > > > > hasn't
> > > > > > > > > > > been
> > > > > > > > > > > > able
> > > > > > > > > > > > > to get a clear answer. What I found out though is
> > > pretty
> > > > > > > > > unsettling,
> > > > > > > > > > > > really.
> > > > > > > > > > > > > The communication on the topic is almost
> > non-existent,
> > > > > > except for
> > > > > > > > > > this
> > > > > > > > > > > > sparse
> > > > > > > > > > > > > and bitter thread in the GH.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I would love to hear from the committers what's
> > > stopping
> > > > the
> > > > > > > > > > acceptance
> > > > > > > > > > > > of the
> > > > > > > > > > > > > code, besides of the minor issues I've mentioned
> > > earlier?
> > > > > > What are
> > > > > > > > > > the
> > > > > > > > > > > > reasons for it?
> > > > > > > > > > > > > Is there anything wrong with it?
> > > > > > > > > > > > > One of the responsibilities of the committers is to
> > > make
> > > > > > sure the
> > > > > > > > > > > > > contributions are reviewed; new contributors are
> > > guided
> > > > and
> > > > > > do
> > > > > > > > > > > > understand how
> > > > > > > > > > > > > the project ticks. The easy feedback flow attracts
> > new
> > > > > > people,
> > > > > > > > > > allowing
> > > > > > > > > > > > the
> > > > > > > > > > > > > community to grow and thrive.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I am asking _explicitely_ not to re-start the
> > > bickering I
> > > > > > have
> > > > > > > > > > already
> > > > > > > > > > > > > seen. At this point I am interested in the purely
> > > > technical
> > > > > > side of
> > > > > > > > > > > this.
> > > > > > > > > > > > >
> > > > > > > > > > > > > [1]
> > > https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > > > > > > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > > > > > > > > > [3]
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > >
> > >
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > > > > > > > > > [4] http://s.apache.org/iZl
> > > > > > > > > > > > >
> > > > > > > > > > > > > With regards,
> > > > > > > > > > > > > Cos
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > > > > > > > > > Github user elbamos commented on the pull
> request:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The current push should resolve some issues with
> > > > changes
> > > > > > in the
> > > > > > > > > > > > > > Spark-Zeppelin interface that had created issues
> > for
> > > > > > users, as
> > > > > > > > > > > > well as
> > > > > > > > > > > > > > support for 1.5.1.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Knitr support is improved, and the reason for a
> > > > separate
> > > > > > knitr
> > > > > > > > > > > > interpreter may be clearer now.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The amount of code borrowed from rscala is
> reduced.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I did not address issues with @author tags, or
> > files
> > > > under
> > > > > > the R/
> > > > > > > > > > > > > > folder. The reason is, to be blunt, I don't
> > > understand
> > > > > > what the
> > > > > > > > > > > > precise
> > > > > > > > > > > > > > concerns actually are.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Please note that I changed .travis.yml to only
> use
> > > > spark
> > > > > > 1.4 and
> > > > > > > > > > > > later.
> > > > > > > > > > > > > > I'm sure there is a better way to do it, but I'm
> > > just
> > > > not
> > > > > > in a
> > > > > > > > > > > > position
> > > > > > > > > > > > > > to try to figure out the entire Zeppelin build
> > > process.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Integrating this with the rest of zeppelin is
> going
> > > to
> > > > > > take some
> > > > > > > > > > > > work
> > > > > > > > > > > > > > regarding pom's, travis, and so forth. I can do a
> > > lot
> > > > of
> > > > > > that,
> > > > > > > > > > > > but I'm
> > > > > > > > > > > > > > going to need to discuss it with the people who
> > have
> > > > been
> > > > > > > > > "owning"
> > > > > > > > > > > > those
> > > > > > > > > > > > > > files. There are just too many moving pieces
> here.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Because of the size of this (which is,
> > > unfortunately,
> > > > > > necessary),
> > > > > > > > > > > > > > posting here is probably not the most efficient
> > way.
> > > > That
> > > > > > is also
> > > > > > > > > > > > true
> > > > > > > > > > > > > > because certain people chose to use this PR as a
> > > forum
> > > > to
> > > > > > air
> > > > > > > > > other
> > > > > > > > > > > > > > issues. Therefore, it would be better if
> reviewers
> > > > e-mail
> > > > > > me
> > > > > > > > > > > > directly.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > >
> > > >
> > >
> > >
> >
>

Re: R-Interpreter CI Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[DuyHai Doan <do...@gmail.com>]

I have also experienced CI failing in the pass for some PRs that do not
impact the code (Cassandra documentation). I guess that during peak hours,
the CI servers may be too busy or enter a dead lock so the build fails.

 At least that's my guess. What do you think ?

On Tue, Dec 8, 2015 at 9:42 AM, moon soo Lee <mo...@apache.org> wrote:

> Let me run some CI test with your branch and share result here.
> Hope i can narrow down the cause and that helps involvement of more people.
>
> Thanks,
> moon
>
> On Tue, Dec 8, 2015 at 3:08 PM Amos B. Elberg <am...@gmail.com>
> wrote:
>
> > Is there anyone who can work with me on the CI issues?  It looks like
> > there are a number of PRs experiencing similar things.
> >
> > I think we should make getting CI stable to be a priority.  Because it
> > will save everyone a lot of frustration and aggravation if CI works
> > reliably.
> >
> > Is there anyone other than Jongyoul and Moon who knows the CI/Build
> > process well?
> >
> > (Moon - Thank you for taking another look at the licensing issue.  Per
> the
> > e-mail I wrote about this a few days ago, I don’t feel I have more to
> > contribute to the licensing discussion, so I’m going to try not to
> comment
> > further about it.)
> >
> > From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org
> > <de...@zeppelin.incubator.apache.org> <de...@zeppelin.incubator.apache.org>
> > Date: December 7, 2015 at 5:00:08 PM
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > <de...@zeppelin.incubator.apache.org>
> > Subject:  Re: License of KnitRInterpreter Was: Re: contributions impasse.
> > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin
> >
> > Thanks Amos, Roman, Cos for clarifying license issue.
> >
> > I'm convinced that this license issue will not be a blocker.
> >
> > In my understanding, these are good sign,
> >
> > 1. any gpl licensed source codes are not included in the source package
> > 2. any gpl licensed libraries are not included in the binary package
> >
> > However, i can not still 100% sure about
> >
> > 3. any gpl licensed libraries are not linked on runtime
> >
> > Even after Amos's explanation. I still think using 'knitr' is one of the
> > clear case that 'knir' is linked to 'R' according to
> > http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL.
> >
> > Giving input and getting output from GPL licensed interpreter (includes
> R)
> > from Apache licensed software is not a problem. That's not the point.
> > Let me say in this way. There's an java code,
> >
> > import com.mysql.jdbc.Driver
> > Driver driver = new Driver()
> >
> > Say without this java code, one of important feature of Zeppelin does not
> > work. And Zeppelin does not includes GPL licensed file in the source
> > package, GPL licensed library in the binary package, but it requires GPL
> > licensed library on the runtime.
> > In this case, will this java code be a license problem or not?
> >
> > In other words, my question is
> >
> > a) Is runtime GPL library dependency allowed in ASF release?
> > b) is 'knitr' considered as runtime dependency?
> >
> > If someone can clarify a), b), then it would be extremely helpful
> > understanding this case, and possible similar cases, too.
> >
> > Thanks,
> > moon
> >
> > On Mon, Dec 7, 2015 at 3:20 PM Konstantin Boudnik <co...@apache.org>
> wrote:
> >
> > > On Thu, Dec 03, 2015 at 11:03AM, Corneau Damien wrote:
> > > > Thanks Cos for those answers,
> > > >
> > > > If I'm right you are advising to have a default build that doesn't
> > > include
> > > > libraries with conflicting licenses, but have an option to include
> > them
> > > for
> > > > users who wants to build the project themselves.
> > >
> > > Yes, that's what I said. Besides, looks like Roman provided the second
> > > pair of
> > > eyes to this licensing discussion and as well didn't find any issues
> > with
> > > the
> > > current approach.
> > >
> > > Cos
> > >
> > > > To refer to another thread about decentralizing interpreters, it
> could
> > > even
> > > > be better in that case to have some interpreters separated from the
> > tree,
> > > > and easily pluggable with a release instead of forcing users to build
> > the
> > > > project to use them.
> > > >
> > > >
> > > > On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <co...@apache.org>
> > > wrote:
> > > >
> > > > > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:
> > > > > > Konstantin thank you for getting into this.
> > > > > >
> > > > > >> The best way to go around it is by
> > > > > >> providing a build-time option that will pull such binaries in.
> > But
> > > by
> > > > > default
> > > > > >> such libs shouldn't be pulled.
> > > > > >
> > > > > > That is basically how the PR handles this. If the GPL’d
> > interpreter
> > > > > scripts
> > > > > > are missing, there’s no indication at all at build time. It
> > doesn’t
> > > try
> > > > > to
> > > > > > download them.
> > > > > >
> > > > > > At runtime, if the user tries to use functionality that would
> need
> > > such a
> > > > > > script (i.e., if they type “knitr” to use knitr), we display an
> > error
> > > > > that
> > > > > > says that the functionality is not there because the library is
> > > missing,
> > > > > and
> > > > > > that the library cannot be provided because it has an
> incompatible
> > > > > license,
> > > > > > but the user can download it themselves if they want.
> > > > > >
> > > > > > And, in the log, if the logging level is high, they will see a
> > note
> > > that
> > > > > > some functionality was disabled because the libraries weren’t
> > there.
> > > > > >
> > > > > > To be clear, though, none of these libraries are binaries.
> They’re
> > > all
> > > > > interpreter scripts.
> > > > >
> > > > > If you ever in a doubt of which licenses could be used for
> > dependncies
> > > > > (not to
> > > > > say about source code) are listed in Category A list of [1]
> > > > >
> > > > > A lot of quesitons discussed here are already covered in the legal
> > > FAQ, so
> > > > > just check against it if you have any questions.
> > > > >
> > > > > [1] http://www.apache.org/legal/resolved.html#category-a
> > > > >
> > > > > Cos
> > > > >
> > > > > > From: Konstantin Boudnik <co...@apache.org>
> > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > dev@zeppelin.incubator.apache.org>,
> > dev@zeppelin.incubator.apache.org
> > > <
> > > > > dev@zeppelin.incubator.apache.org>
> > > > > > Date: December 2, 2015 at 3:24:50 PM
> > > > > > To: dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org
> > > > > >
> > > > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions
> > > > > impasse. Was: [GitHub] incubator-zeppelin pull request: R
> > Interpreter
> > > for
> > > > > Zeppelin
> > > > > >
> > > > > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:
> > > > > > > I think that what moon means is that:
> > > > > > > If we merge the way it is now, the KnitRInterpreter will be
> part
> > > of the
> > > > > > > code base, so it isn't optional at code base level.
> > > > > > >
> > > > > > > To make it even more simple:
> > > > > > > * If the code has the right licensing -> that code can be part
> > of
> > > the
> > > > > > > source code, and can be including in a binary release
> > > > > >
> > > > > > We aren't concerned with binary releases - as an Apache community
> > we
> > > are
> > > > > > voting and releasing source code. If the project wants to provide
> > a
> > > > > binary
> > > > > > release to its users, they are better be warned about inclusion
> of
> > > non
> > > > > > ASL2-friendly licensed bits.
> > > > > >
> > > > > > > * If the code doesn't have the right licensing -> it can't be
> > part
> > > of
> > > > > the
> > > > > > > source code, and can't be included in a binary release
> > > > > >
> > > > > > See above.
> > > > > >
> > > > > > > * If the code doesn't have the right licensing but is imported
> > at
> > > build
> > > > > > > time (libraries for example) -> it is not in the source code,
> it
> > > can't
> > > > > be
> > > > > > > included in binary release
> > > > > >
> > > > > > That is unless a user doing it on his own. The best way to go
> > around
> > > it
> > > > > is by
> > > > > > providing a build-time option that will pull such binaries in.
> But
> > by
> > > > > default
> > > > > > such libs shouldn't be pulled.
> > > > > >
> > > > > > Cos
> > > > > >
> > > > > > > So in the case of licensing issues, it would need to be fully
> > > optional
> > > > > > > (user bring the interpreter in his directory and build Zeppelin
> > > with
> > > > > it)
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <
> > > amos.elberg@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Moon let me clarify:
> > > > > > > >
> > > > > > > > Interpreted code doesn’t “link.” The wiki article actually
> > > explains
> > > > > it
> > > > > > > > pretty well —
> > > https://en.wikipedia.org/wiki/GPL_linking_exception
> > > > > > > > “Linking” against a library means compiling its headers into
> a
> > > > > binary, the
> > > > > > > > way a C compiler works. The 2008 e-mail Moon distributed,
> > called
> > > > > this the
> > > > > > > > “interpreter exception.”
> > > > > > > >
> > > > > > > > As for whether GPL’d code is a “mandatory dependency,” if
> > knitr
> > > is
> > > > > missing
> > > > > > > > the PR will compile, run and test just fine. The user is
> never
> > > > > prompted to
> > > > > > > > download it. The only effect is, if the user types “knitr”
> and
> > > knitr
> > > > > isn’t
> > > > > > > > there, we display an InterpreterError that knitr isn’t there.
> > > > > > > >
> > > > > > > > KnitRInterpreter is not optionally required. so it does not
> > > matter
> > > > > KnitR
> > > > > > > > is
> > > > > > > > optionally required or not.
> > > > > > > > Aren’t all interpreters optional? You can turn them on and
> off
> > > in the
> > > > > > > > config files.
> > > > > > > >
> > > > > > > > Do you mean that the KnitRInterpreter class gets compiled to
> > > > > bytecode even
> > > > > > > > if knitr is missing? So what? That isn't a mandatory
> > dependency
> > > or a
> > > > > link.
> > > > > > > >
> > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > Date: December 2, 2015 at 3:18:00 AM
> > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > Subject: Re: License of KnitRInterpreter Was: Re:
> > contributions
> > > > > impasse.
> > > > > > > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter
> > for
> > > > > Zeppelin
> > > > > > > >
> > > > > > > > Let me summarize license concern about KnitRInterpreter.
> > > > > > > >
> > > > > > > >
> > > > > > > > Amos's interpretation.
> > > > > > > >
> > > > > > > > KnitR is optionally required by KnitRInterpreter.
> > > > > > > > R dependency in SparkR has no problem. So KnitR should be the
> > > same.
> > > > > > > >
> > > > > > > >
> > > > > > > > Moon's interpretation.
> > > > > > > >
> > > > > > > > KnitRInterpreter is not optionally required. so it does not
> > > matter
> > > > > KnitR is
> > > > > > > > optionally required or not.
> > > > > > > > R dependency in SparkR is exception of GPL. KnitR is not
> > applied
> > > that
> > > > > > > > exception.
> > > > > > > >
> > > > > > > >
> > > > > > > > Amos, could you confirm my understanding to your
> > interpretation
> > > is
> > > > > correct?
> > > > > > > > If it is not could you clarify it?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > moon
> > > > > > > >
> > > > > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <
> > > > > amos.elberg@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Just to put the final nail in this, I looked it up.
> > > > > > > > >
> > > > > > > > > I see no evidence of any “compiler exception.”
> > > > > > > > >
> > > > > > > > > There is an exception in the license for the runtime
> > libraries
> > > > > that are
> > > > > > > > > bundled with GCC. See:
> > > > > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> > > > > > > > >
> > > > > > > > > But no “compiler exception.”
> > > > > > > > >
> > > > > > > > > In fact, I believe this is part of the reason why LLVM was
> > > created.
> > > > > > > > >
> > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > > Date: December 1, 2015 at 8:16:36 PM
> > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > incubator-zeppelin pull
> > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > >
> > > > > > > > > Is knitR is commonly considered as a interpreter/compiler?
> > or
> > > is it
> > > > > > > > > considered as a library routine?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > moon
> > > > > > > > >
> > > > > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <
> > > > > amos.elberg@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > Moon - you give this as an explanation of the licensing
> > issue:
> > > > > > > > >
> https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > > >
> > > > > > > > > According to that, there is an exception in the GPL for
> > > interpreter
> > > > > > > > > languages. As long as you don’t distribute the code, its
> > fine
> > > to
> > > > > talk to
> > > > > > > > > an interpreted language.
> > > > > > > > >
> > > > > > > > > Well, if that’s the case, then the PR plainly does not have
> > a
> > > > > license
> > > > > > > > > issue. It doesn’t distribute any GPL’d R code.
> > > > > > > > >
> > > > > > > > > I’m not sure what’s confusing about this. It seems
> > completely
> > > > > > > > > straightforward.
> > > > > > > > >
> > > > > > > > > Regarding this:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Amos Elberg
> > > > > > > > > Sent with Airmail
> > > > > > > > >
> > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > Date: December 1, 2015 at 6:48:47 PM
> > > > > > > > >
> > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > dev@zeppelin.incubator.apache.org
> > > > > > > > >
> > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > incubator-zeppelin pull
> > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > >
> > > > > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <
> > > > > amos.elberg@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > I am going to try to minimize my reaction to Moon’s
> > e-mail.
> > > > > > > > > >
> > > > > > > > > > The tl;dr is this:
> > > > > > > > > >
> > > > > > > > > > The reason we are having this discussion now is that
> > active
> > > > > users of
> > > > > > > > the
> > > > > > > > > > PR — which now has its own user base — went public to
> > > complain
> > > > > about
> > > > > > > > > this.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > The PR has been tested by an active user base for more
> > than
> > > three
> > > > > > > > months.
> > > > > > > > > > No-one has been able to identify any specific actual
> > > licensing
> > > > > problem,
> > > > > > > > > and
> > > > > > > > > > the PR was prepared based on an extensive, careful review
> > of
> > > the
> > > > > > > > relevant
> > > > > > > > > > licensing issues and after contacting the relevant
> people.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I admire every software that used by user and helping
> > people.
> > > That
> > > > > > > > includes
> > > > > > > > > your work. But that's not the topic we're in discussion.
> > Active
> > > > > user does
> > > > > > > > > not mean your contribution can ignore the review.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > It is not an explanation for someone who has been
> ignoring
> > my
> > > > > “how can
> > > > > > > > I
> > > > > > > > > > move this forward…” emails for three months to point the
> > > finger
> > > > > and
> > > > > > > > say I
> > > > > > > > > > didn’t contact the right person or file the right report.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > This is also not the topic in this discussion.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > The burden for providing an explanation for the inaction
> > is
> > > on
> > > > > the PMCC
> > > > > > > > > at
> > > > > > > > > > this point.
> > > > > > > > >
> > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's
> problem
> > on
> > > > > Zeppelin
> > > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > > > They’re not! I often see comments on PRs to just ignore
> > that
> > > CI
> > > > > is
> > > > > > > > > > failing.
> > > > > > > > > >
> > > > > > > > > > One of the most common reasons this PR fails CI, is CI
> > > times-out
> > > > > > > > > > downloading Spark to install. How could that possibly be
> > > caused
> > > > > by the
> > > > > > > > > PR?
> > > > > > > > > >
> > > > > > > > > > It looks to me like the only PRs with changes to the
> > relevant
> > > > > parts of
> > > > > > > > > the
> > > > > > > > > > code — the SparkInterpreter — are being made by the
> person
> > > who
> > > > > wrote
> > > > > > > > the
> > > > > > > > > > testing suite.
> > > > > > > > > >
> > > > > > > > > > So, that would explain why some other PRs pass CI:
> Neither
> > > the
> > > > > > > > > > SparkInterpreter nor the testing suite are stable or
> > robust,
> > > but
> > > > > since
> > > > > > > > > the
> > > > > > > > > > PRs are coming from the person who wrote both…
> > > > > > > > > >
> > > > > > > > > > And let's say Zeppelin core has problem and that makes
> > your
> > > PR
> > > > > fails on
> > > > > > > > > CI
> > > > > > > > > > test. That's possible. But it still does not mean we can
> > > merge
> > > > > it with
> > > > > > > > CI
> > > > > > > > > > failing.
> > > > > > > > > >
> > > > > > > > > > It means you should be working with me to figure out why
> > the
> > > CI
> > > > > is
> > > > > > > > > failing.
> > > > > > > > > >
> > > > > > > > > > This PR has been tested by an active user base for the
> > past
> > > three
> > > > > > > > months.
> > > > > > > > > > If CI is continuing to fail, and dozens of hours of
> effort
> > > have
> > > > > not
> > > > > > > > > > resolved the CI issues, then it is time to start
> > considering
> > > > > whether
> > > > > > > > the
> > > > > > > > > > testing suite is part of the problem.
> > > > > > > > > >
> > > > > > > > > > The level of defensiveness about the CI and
> > SparkInterpreter
> > > is
> > > > > not
> > > > > > > > > > helping to resolve these issues.
> > > > > > > > > >
> > > > > > > > > > If you think it's problem on Zeppelin core, then file an
> > > issue
> > > > > that
> > > > > > > > > > reproduce the problem on Zeppelin core, that might be
> more
> > > > > efficient
> > > > > > > > than
> > > > > > > > > > keep trying yourself.
> > > > > > > > > > I contacted you numerous times about such issues...
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I remember i commented your issue about CI. but you just
> > keep
> > > > > repeated
> > > > > > > > it's
> > > > > > > > > not your problem but Zeppelin core problem.
> > > > > > > > >
> > > > > > > > > Then please file an issue about the problem you found in
> > > Zeppelin
> > > > > Core.
> > > > > > > > > Then everyone will get into the problem.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > In my interpretation, KnitRInterpreter is not an optional
> > > > > feature while
> > > > > > > > > it
> > > > > > > > > > is always enabled when compiling Zeppelin and always
> > enabled
> > > when
> > > > > > > > running
> > > > > > > > > > Zeppelin. And it requires dynamically linked GPL library
> > on
> > > > > runtime.
> > > > > > > > (yes
> > > > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > > > >
> > > > > > > > > > Its not always enabled.
> > > > > > > > > > It is not dynamically linked at runtime.
> > > > > > > > > > It will not fail when knitr is missing. If knitr is not
> > > present,
> > > > > the
> > > > > > > > repl
> > > > > > > > > > interpreter starts and a note is written to to the log
> > that
> > > the
> > > > > knitr
> > > > > > > > > > interpreter isn’t available because knitr is not present.
> > > > > > > > > >
> > > > > > > > > > no Apache code can ever call a shell script, on the
> > purpose
> > > of
> > > > > dynamic
> > > > > > > > > > linking with GPL library.
> > > > > > > > > > You misunderstand.
> > > > > > > > > >
> > > > > > > > > > The *shell* is GPL'd.
> > > > > > > > > >
> > > > > > > > > > Is Zeppelin “linked" against the GPL’d shell because
> > Zeppelin
> > > > > depends
> > > > > > > > on
> > > > > > > > > a
> > > > > > > > > > shell script to launch?
> > > > > > > > > >
> > > > > > > > > Obviously not.
> > > > > > > > > >
> > > > > > > > > > The interaction with R in the PR is the same.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Again, bash is one of exceptions of GPL, like other GPL
> > > licensed
> > > > > > > > > compiler/interpreter.
> > > > > > > > >
> > > > > > > > > Check here why Bash and R is okay with Apache License.
> > > > > > > > >
> https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > > >
> > > > > > > > > I'm not sure we can apply the same exception for 'using'
> > KnitR.
> > > > > > > > >
> > > > > > > > > My point is not 'KnitR' is optional or not. Point is
> > > > > 'KnitRInterpreter'
> > > > > > > > you
> > > > > > > > > wrote is not an optional feature. Which is clearly not
> > > optionally
> > > > > enabled
> > > > > > > > > code and feature. And that depends on KnitR library which
> is
> > > GPL.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > I was guessing SparkR can be still in Apache License even
> > if
> > > it
> > > > > is
> > > > > > > > > depends
> > > > > > > > > > on R. Because of GPL licensed compiler generated output
> is
> > > not
> > > > > GPL
> > > > > > > > > license.
> > > > > > > > > > and R is sort of compiler. If you can get answer from
> > Spark
> > > > > community
> > > > > > > > how
> > > > > > > > > > SparkR get managed to stay in Apache License while R is
> > GPL,
> > > the
> > > > > answer
> > > > > > > > > > might help.
> > > > > > > > > > The description of SparkR is not accurate in any respect.
> > (Do
> > > > > you think
> > > > > > > > > > SparkR is not talking to GPL-licensed libraries?)
> > > > > > > > > >
> > > > > > > > > > I don’t see that any genuine issue is being raised here.
> > > > > > > > > >
> > > > > > > > > > If there is an issue, the burden is on you to identify
> it.
> > > > > > > > > >
> > > > > > > > > > If i give you one suggestion, Zeppelin committers
> > sometimes
> > > ask
> > > > > rebase
> > > > > > > > > the
> > > > > > > > > > contribution branch for some reason. It is not the really
> > the
> > > > > best
> > > > > > > > > > practice, but still okay while most contributions are not
> > > > > including
> > > > > > > > large
> > > > > > > > > > code base changes
> > > > > > > > > > However, your one, has more than 4000 lines of code
> > change.
> > > If
> > > > > you
> > > > > > > > rebase
> > > > > > > > > > then review should be started from the beginning, again.
> > So
> > > you
> > > > > might
> > > > > > > > > want
> > > > > > > > > > to minimize the rebase your branch.
> > > > > > > > > >
> > > > > > > > > > Are you actually complaining that the problem is that I
> > > rebased
> > > > > the
> > > > > > > > code
> > > > > > > > > > during the three-month period when no-one looked at it
> and
> > > > > Zeppelin
> > > > > > > > went
> > > > > > > > > > through a release?
> > > > > > > > > >
> > > > > > > > > > I cannot take it seriously when you say things like this.
> > > > > > > > > >
> > > > > > > > > > Having to “start from the beginning” cannot be a problem
> > if
> > > you
> > > > > never
> > > > > > > > > > actually started the first time...
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > You wanted coordination and cooperation. So i gave you
> > > suggestion
> > > > > that
> > > > > > > > > helping review process. For example, your code has been
> > rebased
> > > > > since my
> > > > > > > > > comment and jongyoul's comment. that means committers will
> > > need to
> > > > > look
> > > > > > > > > from the beginning. That'll require more time. And maybe, i
> > > guess
> > > > > that's
> > > > > > > > > not what you want. But If you don't care, feel free to
> > rebase.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > moon
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Date: December 1, 2015 at 4:42:06 AM
> > > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > incubator-zeppelin
> > > > > > > > pull
> > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > >
> > > > > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <
> > > > > amos.elberg@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > Thank you, Cos.
> > > > > > > > > >
> > > > > > > > > > I’d like to briefly address the issues raised by Moon:
> > > > > > > > > >
> > > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > > The CI fails on core Zeppelin, *not* code in this PR.
> > > > > > > > > >
> > > > > > > > > > I’ve been seeking assistance on this since August.
> > > > > > > > > >
> > > > > > > > > > The most common reason is that SparkInterpreter is unable
> > to
> > > > > launch
> > > > > > > > Spark
> > > > > > > > > > and open a Spark Backend. This is necessary to test the
> > PR.
> > > > > > > > > >
> > > > > > > > > > 60+ hours, has been spent adapting and re-basing when the
> > > > > > > > > SparkInterpreter
> > > > > > > > > > architecture changed and broke the PR’s *tests.*
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's
> > problem
> > > on
> > > > > > > > Zeppelin
> > > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > > >
> > > > > > > > > > And let's say Zeppelin core has problem and that makes
> > your
> > > PR
> > > > > fails on
> > > > > > > > > CI
> > > > > > > > > > test. That's possible. But it still does not mean we can
> > > merge
> > > > > it with
> > > > > > > > CI
> > > > > > > > > > failing.
> > > > > > > > > >
> > > > > > > > > > If you think it's problem on Zeppelin core, then file an
> > > issue
> > > > > that
> > > > > > > > > > reproduce the problem on Zeppelin core, that might be
> more
> > > > > efficient
> > > > > > > > than
> > > > > > > > > > keep trying yourself.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 2. Not 100% sure this PR has no license issue. (about
> > KniteR)
> > > > > > > > > > What license problem *specifically* do you believe may
> > exist?
> > > > > > > > > >
> > > > > > > > > > In preparing the PR, I:
> > > > > > > > > >
> > > > > > > > > > * Reviewed the Apache policy in detail.
> > > > > > > > > >
> > > > > > > > > > * Contacted the FSF to ask their interpretation of the
> > > “linking”
> > > > > > > > > > provisions of the GPL license.
> > > > > > > > > >
> > > > > > > > > > * Reviewed how other Apache software deals with this
> issue
> > > > > (e.g., Spark
> > > > > > > > > > talks to R, which is GPL'd).
> > > > > > > > > >
> > > > > > > > > > * No necessary *dependencies* of the PR have license
> > > conflicts.
> > > > > In
> > > > > > > > > > several cases, I contacted package authors who agreed to
> > > > > re-issue their
> > > > > > > > > > packages under Apache-compatible licenses. (Usually I had
> > to
> > > do
> > > > > a bit
> > > > > > > > of
> > > > > > > > > > coding in exchange…)
> > > > > > > > > >
> > > > > > > > > > * Where the license had to stay GPL, the packages are
> *not
> > > > > necessary*
> > > > > > > > and
> > > > > > > > > > *not dependencies.* If the optional packages are present,
> > > they
> > > > > will be
> > > > > > > > > > used to enable additional functionality. Knitr is an
> > example.
> > > > > The PR
> > > > > > > > will
> > > > > > > > > > compile and run fine without knitr. If knitr is available
> > > (it is
> > > > > part
> > > > > > > > of
> > > > > > > > > > the most common R distribution), the PR will enable the
> > knitr
> > > > > > > > > interpreter.
> > > > > > > > > >
> > > > > > > > > > * This is exactly how this issue is addressed through the
> > > Apache
> > > > > > > > > > ecosystem.
> > > > > > > > > > The tl;dr is this: When Apache code is written to talk to
> > > > > libraries
> > > > > > > > that
> > > > > > > > > > may or may not be present on the user’s system, or where
> > it
> > > > > talks to an
> > > > > > > > > API
> > > > > > > > > > but is agnostic about implementation, that is not
> > “linking”
> > > in a
> > > > > way
> > > > > > > > that
> > > > > > > > > > implicate the anti-linking provision of the GPL.
> > > > > > > > > >
> > > > > > > > > > Otherwise, no Apache code could ever call a shell script!
> > Let
> > > > > alone run
> > > > > > > > > > on Linux, or talk to R.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I'm not a legal expert. So following could be wrong.
> > > > > > > > > >
> > > > > > > > > > In my interpretation, KnitRInterpreter is not an optional
> > > > > feature while
> > > > > > > > > it
> > > > > > > > > > is always enabled when compiling Zeppelin and always
> > enabled
> > > when
> > > > > > > > running
> > > > > > > > > > Zeppelin. And it requires dynamically linked GPL library
> > on
> > > > > runtime.
> > > > > > > > (yes
> > > > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > > > >
> > > > > > > > > > And of course, no Apache code can ever call a shell
> > script,
> > > on
> > > > > the
> > > > > > > > > purpose
> > > > > > > > > > of dynamic linking with GPL library.
> > > > > > > > > >
> > > > > > > > > > I was guessing SparkR can be still in Apache License even
> > if
> > > it
> > > > > is
> > > > > > > > > depends
> > > > > > > > > > on R. Because of GPL licensed compiler generated output
> is
> > > not
> > > > > GPL
> > > > > > > > > license.
> > > > > > > > > > and R is sort of compiler.
> > > > > > > > > >
> > > > > > > > > > If you can get answer from Spark community how SparkR get
> > > > > managed to
> > > > > > > > stay
> > > > > > > > > > in Apache License while R is GPL, the answer might help.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 3. Need more time to review.
> > > > > > > > > > Has any reviewer has downloaded the PR or run the demo
> > > notebook?
> > > > > (Which
> > > > > > > > > > is there for the benefit of reviewers, and isn’t intended
> > to
> > > go
> > > > > in
> > > > > > > > final
> > > > > > > > > > distribution.)
> > > > > > > > > >
> > > > > > > > > > How many +1 comments from users would you like to see?
> > > > > > > > > >
> > > > > > > > > > How much time do you believe is required?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > It all depends on when CI is going to pass, when license
> > > problem
> > > > > is
> > > > > > > > going
> > > > > > > > > > to be cleared, and when a committer willing to review and
> > > > > responsible
> > > > > > > > to
> > > > > > > > > > commit your contribution.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 1. Large code base change
> > > > > > > > > > Large code base changes require coordination and
> > cooperation.
> > > > > This PR
> > > > > > > > > > necessarily implicates the build scripts, testing code,
> > the
> > > > > > > > > > SparkInterpreter, etc.
> > > > > > > > > >
> > > > > > > > > > I have been seeking to coordinate since August.
> > > > > > > > > >
> > > > > > > > > > I continue to stand ready to do so.
> > > > > > > > > >
> > > > > > > > > > -Amos
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > If i give you one suggestion, Zeppelin committers
> > sometimes
> > > ask
> > > > > rebase
> > > > > > > > > the
> > > > > > > > > > contribution branch for some reason. It is not the really
> > the
> > > > > best
> > > > > > > > > > practice, but still okay while most contributions are not
> > > > > including
> > > > > > > > large
> > > > > > > > > > code base changes.
> > > > > > > > > >
> > > > > > > > > > However, your one, has more than 4000 lines of code
> > change.
> > > If
> > > > > you
> > > > > > > > rebase
> > > > > > > > > > then review should be started from the beginning, again.
> > So
> > > you
> > > > > might
> > > > > > > > > want
> > > > > > > > > > to minimize the rebase your branch.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > moon
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > > Date: December 1, 2015 at 1:34:19 AM
> > > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > > > dev@zeppelin.incubator.apache.org
> > > > > > > > > >
> > > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > > incubator-zeppelin
> > > > > > > > pull
> > > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > > >
> > > > > > > > > > Hi Cos,
> > > > > > > > > >
> > > > > > > > > > Thanks for opening a discussion.
> > > > > > > > > > My answer about 'Why this PR is open for three months' is
> > > > > > > > > >
> > > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > > 2. Not 100% sure this PR has no license issue. (about
> > KniteR)
> > > > > > > > > > 3. Need more time to review.
> > > > > > > > > >
> > > > > > > > > > It's my personal answer, other committers may have
> > different
> > > > > opinion.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I think the question should be generalized. Because this
> > PR
> > > is
> > > > > not the
> > > > > > > > > only
> > > > > > > > > > PR that is in impasse. There're more. For example
> > > > > > > > > >
> > > > > > > > > > Here's some examples that PR is not been merged.
> > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,
> > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/60
> > > > > > > > > > and so on.
> > > > > > > > > >
> > > > > > > > > > I can categorize the cases, based on experience of
> > involving
> > > > > Zeppelin
> > > > > > > > > > community from the beginning.
> > > > > > > > > >
> > > > > > > > > > 1. Large code base change
> > > > > > > > > >
> > > > > > > > > > When contribution has large code base changes, it tend to
> > > take
> > > > > more
> > > > > > > > time
> > > > > > > > > to
> > > > > > > > > > review and merged. Normally, most contributions merged in
> > > 1day~1
> > > > > week.
> > > > > > > > > > But some contribution has large code base changes take
> 2~4
> > > > > weeks. Few
> > > > > > > > > > contribution that has very large code base change take
> > > months.
> > > > > > > > > >
> > > > > > > > > > 2. Communication lost
> > > > > > > > > >
> > > > > > > > > > Sometimes, committer is not responding, or contributor is
> > not
> > > > > > > > responding.
> > > > > > > > > >
> > > > > > > > > > 3. Opinion diverges
> > > > > > > > > >
> > > > > > > > > > For some changes, included in contribution. When people
> > have
> > > > > different
> > > > > > > > > > opinion and it continue to diverges, those PRs are not
> > been
> > > > > merged.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I think having a guide such as ping other committer when
> a
> > > > > committer is
> > > > > > > > > not
> > > > > > > > > > responding, and divide contribution into small peaces if
> > > > > possible,
> > > > > > > > would
> > > > > > > > > > help most of the cases.
> > > > > > > > > > Of course committer have to pay attention more to the
> > > > > contribution and
> > > > > > > > > > helping people. That's the first one.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > moon
> > > > > > > > > >
> > > > > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <
> > > > > cos@apache.org>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > To make sure we're on the same page, here are two
> > threads
> > > that
> > > > > I
> > > > > > > > found
> > > > > > > > > > > related
> > > > > > > > > > > to this topic.
> > > > > > > > > > >
> > > > > > > > > > > Thread 1:
> > > > > > > > > > > Subject: R?
> > > > > > > > > > > Started on: July 1, 2015
> > > > > > > > > > >
> > > > > > > > > > > Thread 2:
> > > > > > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R
> > > > > Interpreter for
> > > > > > > > > > > Zeppelin
> > > > > > > > > > > Started on: August 13, 2015
> > > > > > > > > > >
> > > > > > > > > > > If you want to fetch these from our archive send emails
> > to
> > > > > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > > > > > > > > >
> > > > > > > > > > > Cos
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik
> > wrote:
> > > > > > > > > > > > Guys,
> > > > > > > > > > > >
> > > > > > > > > > > > While catching up on my emails from the last a couple
> > of
> > > > > weeks,
> > > > > > > > this
> > > > > > > > > > > thread
> > > > > > > > > > > > caught my attention. I am not normally paying much
> > > attention
> > > > > to the
> > > > > > > > > > code
> > > > > > > > > > > > reviews traffic from GH, but this one is pretty
> > > different as
> > > > > it
> > > > > > > > spans
> > > > > > > > > > > three
> > > > > > > > > > > > months and counting.
> > > > > > > > > > > >
> > > > > > > > > > > > First, here are my five cents:
> > > > > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is
> aimed
> > > to be
> > > > > > > > > > > contributed to
> > > > > > > > > > > > an ASF project this file should simply contain ASL2
> > text,
> > > > > like in
> > > > > > > > [1]
> > > > > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate
> > > <developers>
> > > > > > > > > section,
> > > > > > > > > > > but
> > > > > > > > > > > > Zeppelin might have different guidelines on it. Say,
> > > Bigtop
> > > > > doesn't
> > > > > > > > > > > > maintain this in the source code, but we have the
> list
> > of
> > > > > all the
> > > > > > > > > > > > committers on the project's site [2] Every new
> > > committer's
> > > > > first
> > > > > > > > > > > commit is
> > > > > > > > > > > > to update the team page ;)
> > > > > > > > > > > > - comments like in
> > > > > > > > > > >
> > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > > > > > > > > >
> > > > > > > > > > > > +/**
> > > > > > > > > > > > + * Created by aelberg on 7/28/15.
> > > > > > > > > > > > + */
> > > > > > > > > > > >
> > > > > > > > > > > > is better to be removed. It has been already
> discussed
> > > here
> > > > > [3].
> > > > > > > > And
> > > > > > > > > > > the
> > > > > > > > > > > > initial discussion on the topic [4] was linked as
> well
> > > > > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not
> > sure
> > > if
> > > > > this is
> > > > > > > > > > > R-specific
> > > > > > > > > > > > stuff - I have no idea about R, honestly, but
> > > > > > > > > > > >
> > > > > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > > > > > > > > > ...
> > > > > > > > > > > > +Author: David B. Dahl
> > > > > > > > > > > >
> > > > > > > > > > > > shouldn't be here, IMO. Normally, if some additional
> > > > > licenses are
> > > > > > > > > > > used,
> > > > > > > > > > > > they have to be listed in the top-level NOTICE file
> > > (already
> > > > > > > > there).
> > > > > > > > > > > >
> > > > > > > > > > > > - I am not going to offer any comments on the
> > technical
> > > > > merits of
> > > > > > > > the
> > > > > > > > > > > patch,
> > > > > > > > > > > > as I haven't tried it personally. However, I don't
> see
> > > any
> > > > > serious
> > > > > > > > > > > > technical objections to the functionality in
> question.
> > > > > > > > > > > >
> > > > > > > > > > > > So, the question is - why the PR is open for three
> > > months? I
> > > > > hasn't
> > > > > > > > > > been
> > > > > > > > > > > able
> > > > > > > > > > > > to get a clear answer. What I found out though is
> > pretty
> > > > > > > > unsettling,
> > > > > > > > > > > really.
> > > > > > > > > > > > The communication on the topic is almost
> non-existent,
> > > > > except for
> > > > > > > > > this
> > > > > > > > > > > sparse
> > > > > > > > > > > > and bitter thread in the GH.
> > > > > > > > > > > >
> > > > > > > > > > > > I would love to hear from the committers what's
> > stopping
> > > the
> > > > > > > > > acceptance
> > > > > > > > > > > of the
> > > > > > > > > > > > code, besides of the minor issues I've mentioned
> > earlier?
> > > > > What are
> > > > > > > > > the
> > > > > > > > > > > reasons for it?
> > > > > > > > > > > > Is there anything wrong with it?
> > > > > > > > > > > > One of the responsibilities of the committers is to
> > make
> > > > > sure the
> > > > > > > > > > > > contributions are reviewed; new contributors are
> > guided
> > > and
> > > > > do
> > > > > > > > > > > understand how
> > > > > > > > > > > > the project ticks. The easy feedback flow attracts
> new
> > > > > people,
> > > > > > > > > allowing
> > > > > > > > > > > the
> > > > > > > > > > > > community to grow and thrive.
> > > > > > > > > > > >
> > > > > > > > > > > > I am asking _explicitely_ not to re-start the
> > bickering I
> > > > > have
> > > > > > > > > already
> > > > > > > > > > > > seen. At this point I am interested in the purely
> > > technical
> > > > > side of
> > > > > > > > > > this.
> > > > > > > > > > > >
> > > > > > > > > > > > [1]
> > https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > > > > > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > > > > > > > > [3]
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > >
> > >
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > > > > > > > > [4] http://s.apache.org/iZl
> > > > > > > > > > > >
> > > > > > > > > > > > With regards,
> > > > > > > > > > > > Cos
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > > > > > > > > Github user elbamos commented on the pull request:
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > >
> > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > > > > > > > > >
> > > > > > > > > > > > > The current push should resolve some issues with
> > > changes
> > > > > in the
> > > > > > > > > > > > > Spark-Zeppelin interface that had created issues
> for
> > > > > users, as
> > > > > > > > > > > well as
> > > > > > > > > > > > > support for 1.5.1.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Knitr support is improved, and the reason for a
> > > separate
> > > > > knitr
> > > > > > > > > > > interpreter may be clearer now.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The amount of code borrowed from rscala is reduced.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I did not address issues with @author tags, or
> files
> > > under
> > > > > the R/
> > > > > > > > > > > > > folder. The reason is, to be blunt, I don't
> > understand
> > > > > what the
> > > > > > > > > > > precise
> > > > > > > > > > > > > concerns actually are.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Please note that I changed .travis.yml to only use
> > > spark
> > > > > 1.4 and
> > > > > > > > > > > later.
> > > > > > > > > > > > > I'm sure there is a better way to do it, but I'm
> > just
> > > not
> > > > > in a
> > > > > > > > > > > position
> > > > > > > > > > > > > to try to figure out the entire Zeppelin build
> > process.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Integrating this with the rest of zeppelin is going
> > to
> > > > > take some
> > > > > > > > > > > work
> > > > > > > > > > > > > regarding pom's, travis, and so forth. I can do a
> > lot
> > > of
> > > > > that,
> > > > > > > > > > > but I'm
> > > > > > > > > > > > > going to need to discuss it with the people who
> have
> > > been
> > > > > > > > "owning"
> > > > > > > > > > > those
> > > > > > > > > > > > > files. There are just too many moving pieces here.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Because of the size of this (which is,
> > unfortunately,
> > > > > necessary),
> > > > > > > > > > > > > posting here is probably not the most efficient
> way.
> > > That
> > > > > is also
> > > > > > > > > > > true
> > > > > > > > > > > > > because certain people chose to use this PR as a
> > forum
> > > to
> > > > > air
> > > > > > > > other
> > > > > > > > > > > > > issues. Therefore, it would be better if reviewers
> > > e-mail
> > > > > me
> > > > > > > > > > > directly.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > >
> > >
> >
> >
>

Re: R-Interpreter CI Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Let me run some CI test with your branch and share result here.
Hope i can narrow down the cause and that helps involvement of more people.

Thanks,
moon

On Tue, Dec 8, 2015 at 3:08 PM Amos B. Elberg <am...@gmail.com> wrote:

> Is there anyone who can work with me on the CI issues?  It looks like
> there are a number of PRs experiencing similar things.
>
> I think we should make getting CI stable to be a priority.  Because it
> will save everyone a lot of frustration and aggravation if CI works
> reliably.
>
> Is there anyone other than Jongyoul and Moon who knows the CI/Build
> process well?
>
> (Moon - Thank you for taking another look at the licensing issue.  Per the
> e-mail I wrote about this a few days ago, I don’t feel I have more to
> contribute to the licensing discussion, so I’m going to try not to comment
> further about it.)
>
> From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org
> <de...@zeppelin.incubator.apache.org> <de...@zeppelin.incubator.apache.org>
> Date: December 7, 2015 at 5:00:08 PM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> <de...@zeppelin.incubator.apache.org>
> Subject:  Re: License of KnitRInterpreter Was: Re: contributions impasse.
> Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin
>
> Thanks Amos, Roman, Cos for clarifying license issue.
>
> I'm convinced that this license issue will not be a blocker.
>
> In my understanding, these are good sign,
>
> 1. any gpl licensed source codes are not included in the source package
> 2. any gpl licensed libraries are not included in the binary package
>
> However, i can not still 100% sure about
>
> 3. any gpl licensed libraries are not linked on runtime
>
> Even after Amos's explanation. I still think using 'knitr' is one of the
> clear case that 'knir' is linked to 'R' according to
> http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL.
>
> Giving input and getting output from GPL licensed interpreter (includes R)
> from Apache licensed software is not a problem. That's not the point.
> Let me say in this way. There's an java code,
>
> import com.mysql.jdbc.Driver
> Driver driver = new Driver()
>
> Say without this java code, one of important feature of Zeppelin does not
> work. And Zeppelin does not includes GPL licensed file in the source
> package, GPL licensed library in the binary package, but it requires GPL
> licensed library on the runtime.
> In this case, will this java code be a license problem or not?
>
> In other words, my question is
>
> a) Is runtime GPL library dependency allowed in ASF release?
> b) is 'knitr' considered as runtime dependency?
>
> If someone can clarify a), b), then it would be extremely helpful
> understanding this case, and possible similar cases, too.
>
> Thanks,
> moon
>
> On Mon, Dec 7, 2015 at 3:20 PM Konstantin Boudnik <co...@apache.org> wrote:
>
> > On Thu, Dec 03, 2015 at 11:03AM, Corneau Damien wrote:
> > > Thanks Cos for those answers,
> > >
> > > If I'm right you are advising to have a default build that doesn't
> > include
> > > libraries with conflicting licenses, but have an option to include
> them
> > for
> > > users who wants to build the project themselves.
> >
> > Yes, that's what I said. Besides, looks like Roman provided the second
> > pair of
> > eyes to this licensing discussion and as well didn't find any issues
> with
> > the
> > current approach.
> >
> > Cos
> >
> > > To refer to another thread about decentralizing interpreters, it could
> > even
> > > be better in that case to have some interpreters separated from the
> tree,
> > > and easily pluggable with a release instead of forcing users to build
> the
> > > project to use them.
> > >
> > >
> > > On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <co...@apache.org>
> > wrote:
> > >
> > > > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:
> > > > > Konstantin thank you for getting into this.
> > > > >
> > > > >> The best way to go around it is by
> > > > >> providing a build-time option that will pull such binaries in.
> But
> > by
> > > > default
> > > > >> such libs shouldn't be pulled.
> > > > >
> > > > > That is basically how the PR handles this. If the GPL’d
> interpreter
> > > > scripts
> > > > > are missing, there’s no indication at all at build time. It
> doesn’t
> > try
> > > > to
> > > > > download them.
> > > > >
> > > > > At runtime, if the user tries to use functionality that would need
> > such a
> > > > > script (i.e., if they type “knitr” to use knitr), we display an
> error
> > > > that
> > > > > says that the functionality is not there because the library is
> > missing,
> > > > and
> > > > > that the library cannot be provided because it has an incompatible
> > > > license,
> > > > > but the user can download it themselves if they want.
> > > > >
> > > > > And, in the log, if the logging level is high, they will see a
> note
> > that
> > > > > some functionality was disabled because the libraries weren’t
> there.
> > > > >
> > > > > To be clear, though, none of these libraries are binaries. They’re
> > all
> > > > interpreter scripts.
> > > >
> > > > If you ever in a doubt of which licenses could be used for
> dependncies
> > > > (not to
> > > > say about source code) are listed in Category A list of [1]
> > > >
> > > > A lot of quesitons discussed here are already covered in the legal
> > FAQ, so
> > > > just check against it if you have any questions.
> > > >
> > > > [1] http://www.apache.org/legal/resolved.html#category-a
> > > >
> > > > Cos
> > > >
> > > > > From: Konstantin Boudnik <co...@apache.org>
> > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > dev@zeppelin.incubator.apache.org>,
> dev@zeppelin.incubator.apache.org
> > <
> > > > dev@zeppelin.incubator.apache.org>
> > > > > Date: December 2, 2015 at 3:24:50 PM
> > > > > To: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org
> > > > >
> > > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions
> > > > impasse. Was: [GitHub] incubator-zeppelin pull request: R
> Interpreter
> > for
> > > > Zeppelin
> > > > >
> > > > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:
> > > > > > I think that what moon means is that:
> > > > > > If we merge the way it is now, the KnitRInterpreter will be part
> > of the
> > > > > > code base, so it isn't optional at code base level.
> > > > > >
> > > > > > To make it even more simple:
> > > > > > * If the code has the right licensing -> that code can be part
> of
> > the
> > > > > > source code, and can be including in a binary release
> > > > >
> > > > > We aren't concerned with binary releases - as an Apache community
> we
> > are
> > > > > voting and releasing source code. If the project wants to provide
> a
> > > > binary
> > > > > release to its users, they are better be warned about inclusion of
> > non
> > > > > ASL2-friendly licensed bits.
> > > > >
> > > > > > * If the code doesn't have the right licensing -> it can't be
> part
> > of
> > > > the
> > > > > > source code, and can't be included in a binary release
> > > > >
> > > > > See above.
> > > > >
> > > > > > * If the code doesn't have the right licensing but is imported
> at
> > build
> > > > > > time (libraries for example) -> it is not in the source code, it
> > can't
> > > > be
> > > > > > included in binary release
> > > > >
> > > > > That is unless a user doing it on his own. The best way to go
> around
> > it
> > > > is by
> > > > > providing a build-time option that will pull such binaries in. But
> by
> > > > default
> > > > > such libs shouldn't be pulled.
> > > > >
> > > > > Cos
> > > > >
> > > > > > So in the case of licensing issues, it would need to be fully
> > optional
> > > > > > (user bring the interpreter in his directory and build Zeppelin
> > with
> > > > it)
> > > > > >
> > > > > >
> > > > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <
> > amos.elberg@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Moon let me clarify:
> > > > > > >
> > > > > > > Interpreted code doesn’t “link.” The wiki article actually
> > explains
> > > > it
> > > > > > > pretty well —
> > https://en.wikipedia.org/wiki/GPL_linking_exception
> > > > > > > “Linking” against a library means compiling its headers into a
> > > > binary, the
> > > > > > > way a C compiler works. The 2008 e-mail Moon distributed,
> called
> > > > this the
> > > > > > > “interpreter exception.”
> > > > > > >
> > > > > > > As for whether GPL’d code is a “mandatory dependency,” if
> knitr
> > is
> > > > missing
> > > > > > > the PR will compile, run and test just fine. The user is never
> > > > prompted to
> > > > > > > download it. The only effect is, if the user types “knitr” and
> > knitr
> > > > isn’t
> > > > > > > there, we display an InterpreterError that knitr isn’t there.
> > > > > > >
> > > > > > > KnitRInterpreter is not optionally required. so it does not
> > matter
> > > > KnitR
> > > > > > > is
> > > > > > > optionally required or not.
> > > > > > > Aren’t all interpreters optional? You can turn them on and off
> > in the
> > > > > > > config files.
> > > > > > >
> > > > > > > Do you mean that the KnitRInterpreter class gets compiled to
> > > > bytecode even
> > > > > > > if knitr is missing? So what? That isn't a mandatory
> dependency
> > or a
> > > > link.
> > > > > > >
> > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > Date: December 2, 2015 at 3:18:00 AM
> > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > dev@zeppelin.incubator.apache.org>
> > > > > > > Subject: Re: License of KnitRInterpreter Was: Re:
> contributions
> > > > impasse.
> > > > > > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter
> for
> > > > Zeppelin
> > > > > > >
> > > > > > > Let me summarize license concern about KnitRInterpreter.
> > > > > > >
> > > > > > >
> > > > > > > Amos's interpretation.
> > > > > > >
> > > > > > > KnitR is optionally required by KnitRInterpreter.
> > > > > > > R dependency in SparkR has no problem. So KnitR should be the
> > same.
> > > > > > >
> > > > > > >
> > > > > > > Moon's interpretation.
> > > > > > >
> > > > > > > KnitRInterpreter is not optionally required. so it does not
> > matter
> > > > KnitR is
> > > > > > > optionally required or not.
> > > > > > > R dependency in SparkR is exception of GPL. KnitR is not
> applied
> > that
> > > > > > > exception.
> > > > > > >
> > > > > > >
> > > > > > > Amos, could you confirm my understanding to your
> interpretation
> > is
> > > > correct?
> > > > > > > If it is not could you clarify it?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > moon
> > > > > > >
> > > > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <
> > > > amos.elberg@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Just to put the final nail in this, I looked it up.
> > > > > > > >
> > > > > > > > I see no evidence of any “compiler exception.”
> > > > > > > >
> > > > > > > > There is an exception in the license for the runtime
> libraries
> > > > that are
> > > > > > > > bundled with GCC. See:
> > > > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> > > > > > > >
> > > > > > > > But no “compiler exception.”
> > > > > > > >
> > > > > > > > In fact, I believe this is part of the reason why LLVM was
> > created.
> > > > > > > >
> > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > Date: December 1, 2015 at 8:16:36 PM
> > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > incubator-zeppelin pull
> > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > >
> > > > > > > > Is knitR is commonly considered as a interpreter/compiler?
> or
> > is it
> > > > > > > > considered as a library routine?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > moon
> > > > > > > >
> > > > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <
> > > > amos.elberg@gmail.com>
> > > > > > > > wrote:
> > > > > > > > Moon - you give this as an explanation of the licensing
> issue:
> > > > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > >
> > > > > > > > According to that, there is an exception in the GPL for
> > interpreter
> > > > > > > > languages. As long as you don’t distribute the code, its
> fine
> > to
> > > > talk to
> > > > > > > > an interpreted language.
> > > > > > > >
> > > > > > > > Well, if that’s the case, then the PR plainly does not have
> a
> > > > license
> > > > > > > > issue. It doesn’t distribute any GPL’d R code.
> > > > > > > >
> > > > > > > > I’m not sure what’s confusing about this. It seems
> completely
> > > > > > > > straightforward.
> > > > > > > >
> > > > > > > > Regarding this:
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Amos Elberg
> > > > > > > > Sent with Airmail
> > > > > > > >
> > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > Date: December 1, 2015 at 6:48:47 PM
> > > > > > > >
> > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > dev@zeppelin.incubator.apache.org
> > > > > > > >
> > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > incubator-zeppelin pull
> > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > >
> > > > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <
> > > > amos.elberg@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > I am going to try to minimize my reaction to Moon’s
> e-mail.
> > > > > > > > >
> > > > > > > > > The tl;dr is this:
> > > > > > > > >
> > > > > > > > > The reason we are having this discussion now is that
> active
> > > > users of
> > > > > > > the
> > > > > > > > > PR — which now has its own user base — went public to
> > complain
> > > > about
> > > > > > > > this.
> > > > > > > >
> > > > > > > >
> > > > > > > > > The PR has been tested by an active user base for more
> than
> > three
> > > > > > > months.
> > > > > > > > > No-one has been able to identify any specific actual
> > licensing
> > > > problem,
> > > > > > > > and
> > > > > > > > > the PR was prepared based on an extensive, careful review
> of
> > the
> > > > > > > relevant
> > > > > > > > > licensing issues and after contacting the relevant people.
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > I admire every software that used by user and helping
> people.
> > That
> > > > > > > includes
> > > > > > > > your work. But that's not the topic we're in discussion.
> Active
> > > > user does
> > > > > > > > not mean your contribution can ignore the review.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > It is not an explanation for someone who has been ignoring
> my
> > > > “how can
> > > > > > > I
> > > > > > > > > move this forward…” emails for three months to point the
> > finger
> > > > and
> > > > > > > say I
> > > > > > > > > didn’t contact the right person or file the right report.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > This is also not the topic in this discussion.
> > > > > > > >
> > > > > > > >
> > > > > > > > > The burden for providing an explanation for the inaction
> is
> > on
> > > > the PMCC
> > > > > > > > at
> > > > > > > > > this point.
> > > > > > > >
> > > > > > > > I'm sorry, but the other PRs are passing CI. If it's problem
> on
> > > > Zeppelin
> > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > > They’re not! I often see comments on PRs to just ignore
> that
> > CI
> > > > is
> > > > > > > > > failing.
> > > > > > > > >
> > > > > > > > > One of the most common reasons this PR fails CI, is CI
> > times-out
> > > > > > > > > downloading Spark to install. How could that possibly be
> > caused
> > > > by the
> > > > > > > > PR?
> > > > > > > > >
> > > > > > > > > It looks to me like the only PRs with changes to the
> relevant
> > > > parts of
> > > > > > > > the
> > > > > > > > > code — the SparkInterpreter — are being made by the person
> > who
> > > > wrote
> > > > > > > the
> > > > > > > > > testing suite.
> > > > > > > > >
> > > > > > > > > So, that would explain why some other PRs pass CI: Neither
> > the
> > > > > > > > > SparkInterpreter nor the testing suite are stable or
> robust,
> > but
> > > > since
> > > > > > > > the
> > > > > > > > > PRs are coming from the person who wrote both…
> > > > > > > > >
> > > > > > > > > And let's say Zeppelin core has problem and that makes
> your
> > PR
> > > > fails on
> > > > > > > > CI
> > > > > > > > > test. That's possible. But it still does not mean we can
> > merge
> > > > it with
> > > > > > > CI
> > > > > > > > > failing.
> > > > > > > > >
> > > > > > > > > It means you should be working with me to figure out why
> the
> > CI
> > > > is
> > > > > > > > failing.
> > > > > > > > >
> > > > > > > > > This PR has been tested by an active user base for the
> past
> > three
> > > > > > > months.
> > > > > > > > > If CI is continuing to fail, and dozens of hours of effort
> > have
> > > > not
> > > > > > > > > resolved the CI issues, then it is time to start
> considering
> > > > whether
> > > > > > > the
> > > > > > > > > testing suite is part of the problem.
> > > > > > > > >
> > > > > > > > > The level of defensiveness about the CI and
> SparkInterpreter
> > is
> > > > not
> > > > > > > > > helping to resolve these issues.
> > > > > > > > >
> > > > > > > > > If you think it's problem on Zeppelin core, then file an
> > issue
> > > > that
> > > > > > > > > reproduce the problem on Zeppelin core, that might be more
> > > > efficient
> > > > > > > than
> > > > > > > > > keep trying yourself.
> > > > > > > > > I contacted you numerous times about such issues...
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > I remember i commented your issue about CI. but you just
> keep
> > > > repeated
> > > > > > > it's
> > > > > > > > not your problem but Zeppelin core problem.
> > > > > > > >
> > > > > > > > Then please file an issue about the problem you found in
> > Zeppelin
> > > > Core.
> > > > > > > > Then everyone will get into the problem.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > In my interpretation, KnitRInterpreter is not an optional
> > > > feature while
> > > > > > > > it
> > > > > > > > > is always enabled when compiling Zeppelin and always
> enabled
> > when
> > > > > > > running
> > > > > > > > > Zeppelin. And it requires dynamically linked GPL library
> on
> > > > runtime.
> > > > > > > (yes
> > > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > > >
> > > > > > > > > Its not always enabled.
> > > > > > > > > It is not dynamically linked at runtime.
> > > > > > > > > It will not fail when knitr is missing. If knitr is not
> > present,
> > > > the
> > > > > > > repl
> > > > > > > > > interpreter starts and a note is written to to the log
> that
> > the
> > > > knitr
> > > > > > > > > interpreter isn’t available because knitr is not present.
> > > > > > > > >
> > > > > > > > > no Apache code can ever call a shell script, on the
> purpose
> > of
> > > > dynamic
> > > > > > > > > linking with GPL library.
> > > > > > > > > You misunderstand.
> > > > > > > > >
> > > > > > > > > The *shell* is GPL'd.
> > > > > > > > >
> > > > > > > > > Is Zeppelin “linked" against the GPL’d shell because
> Zeppelin
> > > > depends
> > > > > > > on
> > > > > > > > a
> > > > > > > > > shell script to launch?
> > > > > > > > >
> > > > > > > > Obviously not.
> > > > > > > > >
> > > > > > > > > The interaction with R in the PR is the same.
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > Again, bash is one of exceptions of GPL, like other GPL
> > licensed
> > > > > > > > compiler/interpreter.
> > > > > > > >
> > > > > > > > Check here why Bash and R is okay with Apache License.
> > > > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > > >
> > > > > > > > I'm not sure we can apply the same exception for 'using'
> KnitR.
> > > > > > > >
> > > > > > > > My point is not 'KnitR' is optional or not. Point is
> > > > 'KnitRInterpreter'
> > > > > > > you
> > > > > > > > wrote is not an optional feature. Which is clearly not
> > optionally
> > > > enabled
> > > > > > > > code and feature. And that depends on KnitR library which is
> > GPL.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > I was guessing SparkR can be still in Apache License even
> if
> > it
> > > > is
> > > > > > > > depends
> > > > > > > > > on R. Because of GPL licensed compiler generated output is
> > not
> > > > GPL
> > > > > > > > license.
> > > > > > > > > and R is sort of compiler. If you can get answer from
> Spark
> > > > community
> > > > > > > how
> > > > > > > > > SparkR get managed to stay in Apache License while R is
> GPL,
> > the
> > > > answer
> > > > > > > > > might help.
> > > > > > > > > The description of SparkR is not accurate in any respect.
> (Do
> > > > you think
> > > > > > > > > SparkR is not talking to GPL-licensed libraries?)
> > > > > > > > >
> > > > > > > > > I don’t see that any genuine issue is being raised here.
> > > > > > > > >
> > > > > > > > > If there is an issue, the burden is on you to identify it.
> > > > > > > > >
> > > > > > > > > If i give you one suggestion, Zeppelin committers
> sometimes
> > ask
> > > > rebase
> > > > > > > > the
> > > > > > > > > contribution branch for some reason. It is not the really
> the
> > > > best
> > > > > > > > > practice, but still okay while most contributions are not
> > > > including
> > > > > > > large
> > > > > > > > > code base changes
> > > > > > > > > However, your one, has more than 4000 lines of code
> change.
> > If
> > > > you
> > > > > > > rebase
> > > > > > > > > then review should be started from the beginning, again.
> So
> > you
> > > > might
> > > > > > > > want
> > > > > > > > > to minimize the rebase your branch.
> > > > > > > > >
> > > > > > > > > Are you actually complaining that the problem is that I
> > rebased
> > > > the
> > > > > > > code
> > > > > > > > > during the three-month period when no-one looked at it and
> > > > Zeppelin
> > > > > > > went
> > > > > > > > > through a release?
> > > > > > > > >
> > > > > > > > > I cannot take it seriously when you say things like this.
> > > > > > > > >
> > > > > > > > > Having to “start from the beginning” cannot be a problem
> if
> > you
> > > > never
> > > > > > > > > actually started the first time...
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > You wanted coordination and cooperation. So i gave you
> > suggestion
> > > > that
> > > > > > > > helping review process. For example, your code has been
> rebased
> > > > since my
> > > > > > > > comment and jongyoul's comment. that means committers will
> > need to
> > > > look
> > > > > > > > from the beginning. That'll require more time. And maybe, i
> > guess
> > > > that's
> > > > > > > > not what you want. But If you don't care, feel free to
> rebase.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > moon
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > > Date: December 1, 2015 at 4:42:06 AM
> > > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > > dev@zeppelin.incubator.apache.org <
> > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > incubator-zeppelin
> > > > > > > pull
> > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > >
> > > > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <
> > > > amos.elberg@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > Thank you, Cos.
> > > > > > > > >
> > > > > > > > > I’d like to briefly address the issues raised by Moon:
> > > > > > > > >
> > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > The CI fails on core Zeppelin, *not* code in this PR.
> > > > > > > > >
> > > > > > > > > I’ve been seeking assistance on this since August.
> > > > > > > > >
> > > > > > > > > The most common reason is that SparkInterpreter is unable
> to
> > > > launch
> > > > > > > Spark
> > > > > > > > > and open a Spark Backend. This is necessary to test the
> PR.
> > > > > > > > >
> > > > > > > > > 60+ hours, has been spent adapting and re-basing when the
> > > > > > > > SparkInterpreter
> > > > > > > > > architecture changed and broke the PR’s *tests.*
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I'm sorry, but the other PRs are passing CI. If it's
> problem
> > on
> > > > > > > Zeppelin
> > > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > >
> > > > > > > > > And let's say Zeppelin core has problem and that makes
> your
> > PR
> > > > fails on
> > > > > > > > CI
> > > > > > > > > test. That's possible. But it still does not mean we can
> > merge
> > > > it with
> > > > > > > CI
> > > > > > > > > failing.
> > > > > > > > >
> > > > > > > > > If you think it's problem on Zeppelin core, then file an
> > issue
> > > > that
> > > > > > > > > reproduce the problem on Zeppelin core, that might be more
> > > > efficient
> > > > > > > than
> > > > > > > > > keep trying yourself.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 2. Not 100% sure this PR has no license issue. (about
> KniteR)
> > > > > > > > > What license problem *specifically* do you believe may
> exist?
> > > > > > > > >
> > > > > > > > > In preparing the PR, I:
> > > > > > > > >
> > > > > > > > > * Reviewed the Apache policy in detail.
> > > > > > > > >
> > > > > > > > > * Contacted the FSF to ask their interpretation of the
> > “linking”
> > > > > > > > > provisions of the GPL license.
> > > > > > > > >
> > > > > > > > > * Reviewed how other Apache software deals with this issue
> > > > (e.g., Spark
> > > > > > > > > talks to R, which is GPL'd).
> > > > > > > > >
> > > > > > > > > * No necessary *dependencies* of the PR have license
> > conflicts.
> > > > In
> > > > > > > > > several cases, I contacted package authors who agreed to
> > > > re-issue their
> > > > > > > > > packages under Apache-compatible licenses. (Usually I had
> to
> > do
> > > > a bit
> > > > > > > of
> > > > > > > > > coding in exchange…)
> > > > > > > > >
> > > > > > > > > * Where the license had to stay GPL, the packages are *not
> > > > necessary*
> > > > > > > and
> > > > > > > > > *not dependencies.* If the optional packages are present,
> > they
> > > > will be
> > > > > > > > > used to enable additional functionality. Knitr is an
> example.
> > > > The PR
> > > > > > > will
> > > > > > > > > compile and run fine without knitr. If knitr is available
> > (it is
> > > > part
> > > > > > > of
> > > > > > > > > the most common R distribution), the PR will enable the
> knitr
> > > > > > > > interpreter.
> > > > > > > > >
> > > > > > > > > * This is exactly how this issue is addressed through the
> > Apache
> > > > > > > > > ecosystem.
> > > > > > > > > The tl;dr is this: When Apache code is written to talk to
> > > > libraries
> > > > > > > that
> > > > > > > > > may or may not be present on the user’s system, or where
> it
> > > > talks to an
> > > > > > > > API
> > > > > > > > > but is agnostic about implementation, that is not
> “linking”
> > in a
> > > > way
> > > > > > > that
> > > > > > > > > implicate the anti-linking provision of the GPL.
> > > > > > > > >
> > > > > > > > > Otherwise, no Apache code could ever call a shell script!
> Let
> > > > alone run
> > > > > > > > > on Linux, or talk to R.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I'm not a legal expert. So following could be wrong.
> > > > > > > > >
> > > > > > > > > In my interpretation, KnitRInterpreter is not an optional
> > > > feature while
> > > > > > > > it
> > > > > > > > > is always enabled when compiling Zeppelin and always
> enabled
> > when
> > > > > > > running
> > > > > > > > > Zeppelin. And it requires dynamically linked GPL library
> on
> > > > runtime.
> > > > > > > (yes
> > > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > > >
> > > > > > > > > And of course, no Apache code can ever call a shell
> script,
> > on
> > > > the
> > > > > > > > purpose
> > > > > > > > > of dynamic linking with GPL library.
> > > > > > > > >
> > > > > > > > > I was guessing SparkR can be still in Apache License even
> if
> > it
> > > > is
> > > > > > > > depends
> > > > > > > > > on R. Because of GPL licensed compiler generated output is
> > not
> > > > GPL
> > > > > > > > license.
> > > > > > > > > and R is sort of compiler.
> > > > > > > > >
> > > > > > > > > If you can get answer from Spark community how SparkR get
> > > > managed to
> > > > > > > stay
> > > > > > > > > in Apache License while R is GPL, the answer might help.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 3. Need more time to review.
> > > > > > > > > Has any reviewer has downloaded the PR or run the demo
> > notebook?
> > > > (Which
> > > > > > > > > is there for the benefit of reviewers, and isn’t intended
> to
> > go
> > > > in
> > > > > > > final
> > > > > > > > > distribution.)
> > > > > > > > >
> > > > > > > > > How many +1 comments from users would you like to see?
> > > > > > > > >
> > > > > > > > > How much time do you believe is required?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > It all depends on when CI is going to pass, when license
> > problem
> > > > is
> > > > > > > going
> > > > > > > > > to be cleared, and when a committer willing to review and
> > > > responsible
> > > > > > > to
> > > > > > > > > commit your contribution.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1. Large code base change
> > > > > > > > > Large code base changes require coordination and
> cooperation.
> > > > This PR
> > > > > > > > > necessarily implicates the build scripts, testing code,
> the
> > > > > > > > > SparkInterpreter, etc.
> > > > > > > > >
> > > > > > > > > I have been seeking to coordinate since August.
> > > > > > > > >
> > > > > > > > > I continue to stand ready to do so.
> > > > > > > > >
> > > > > > > > > -Amos
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > If i give you one suggestion, Zeppelin committers
> sometimes
> > ask
> > > > rebase
> > > > > > > > the
> > > > > > > > > contribution branch for some reason. It is not the really
> the
> > > > best
> > > > > > > > > practice, but still okay while most contributions are not
> > > > including
> > > > > > > large
> > > > > > > > > code base changes.
> > > > > > > > >
> > > > > > > > > However, your one, has more than 4000 lines of code
> change.
> > If
> > > > you
> > > > > > > rebase
> > > > > > > > > then review should be started from the beginning, again.
> So
> > you
> > > > might
> > > > > > > > want
> > > > > > > > > to minimize the rebase your branch.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > moon
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > > Date: December 1, 2015 at 1:34:19 AM
> > > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > > dev@zeppelin.incubator.apache.org
> > > > > > > > >
> > > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > > incubator-zeppelin
> > > > > > > pull
> > > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > > >
> > > > > > > > > Hi Cos,
> > > > > > > > >
> > > > > > > > > Thanks for opening a discussion.
> > > > > > > > > My answer about 'Why this PR is open for three months' is
> > > > > > > > >
> > > > > > > > > 1. This PR does not passes CI
> > > > > > > > > 2. Not 100% sure this PR has no license issue. (about
> KniteR)
> > > > > > > > > 3. Need more time to review.
> > > > > > > > >
> > > > > > > > > It's my personal answer, other committers may have
> different
> > > > opinion.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I think the question should be generalized. Because this
> PR
> > is
> > > > not the
> > > > > > > > only
> > > > > > > > > PR that is in impasse. There're more. For example
> > > > > > > > >
> > > > > > > > > Here's some examples that PR is not been merged.
> > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,
> > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/60
> > > > > > > > > and so on.
> > > > > > > > >
> > > > > > > > > I can categorize the cases, based on experience of
> involving
> > > > Zeppelin
> > > > > > > > > community from the beginning.
> > > > > > > > >
> > > > > > > > > 1. Large code base change
> > > > > > > > >
> > > > > > > > > When contribution has large code base changes, it tend to
> > take
> > > > more
> > > > > > > time
> > > > > > > > to
> > > > > > > > > review and merged. Normally, most contributions merged in
> > 1day~1
> > > > week.
> > > > > > > > > But some contribution has large code base changes take 2~4
> > > > weeks. Few
> > > > > > > > > contribution that has very large code base change take
> > months.
> > > > > > > > >
> > > > > > > > > 2. Communication lost
> > > > > > > > >
> > > > > > > > > Sometimes, committer is not responding, or contributor is
> not
> > > > > > > responding.
> > > > > > > > >
> > > > > > > > > 3. Opinion diverges
> > > > > > > > >
> > > > > > > > > For some changes, included in contribution. When people
> have
> > > > different
> > > > > > > > > opinion and it continue to diverges, those PRs are not
> been
> > > > merged.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I think having a guide such as ping other committer when a
> > > > committer is
> > > > > > > > not
> > > > > > > > > responding, and divide contribution into small peaces if
> > > > possible,
> > > > > > > would
> > > > > > > > > help most of the cases.
> > > > > > > > > Of course committer have to pay attention more to the
> > > > contribution and
> > > > > > > > > helping people. That's the first one.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > moon
> > > > > > > > >
> > > > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <
> > > > cos@apache.org>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > To make sure we're on the same page, here are two
> threads
> > that
> > > > I
> > > > > > > found
> > > > > > > > > > related
> > > > > > > > > > to this topic.
> > > > > > > > > >
> > > > > > > > > > Thread 1:
> > > > > > > > > > Subject: R?
> > > > > > > > > > Started on: July 1, 2015
> > > > > > > > > >
> > > > > > > > > > Thread 2:
> > > > > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R
> > > > Interpreter for
> > > > > > > > > > Zeppelin
> > > > > > > > > > Started on: August 13, 2015
> > > > > > > > > >
> > > > > > > > > > If you want to fetch these from our archive send emails
> to
> > > > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > > > > > > > >
> > > > > > > > > > Cos
> > > > > > > > > >
> > > > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik
> wrote:
> > > > > > > > > > > Guys,
> > > > > > > > > > >
> > > > > > > > > > > While catching up on my emails from the last a couple
> of
> > > > weeks,
> > > > > > > this
> > > > > > > > > > thread
> > > > > > > > > > > caught my attention. I am not normally paying much
> > attention
> > > > to the
> > > > > > > > > code
> > > > > > > > > > > reviews traffic from GH, but this one is pretty
> > different as
> > > > it
> > > > > > > spans
> > > > > > > > > > three
> > > > > > > > > > > months and counting.
> > > > > > > > > > >
> > > > > > > > > > > First, here are my five cents:
> > > > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed
> > to be
> > > > > > > > > > contributed to
> > > > > > > > > > > an ASF project this file should simply contain ASL2
> text,
> > > > like in
> > > > > > > [1]
> > > > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate
> > <developers>
> > > > > > > > section,
> > > > > > > > > > but
> > > > > > > > > > > Zeppelin might have different guidelines on it. Say,
> > Bigtop
> > > > doesn't
> > > > > > > > > > > maintain this in the source code, but we have the list
> of
> > > > all the
> > > > > > > > > > > committers on the project's site [2] Every new
> > committer's
> > > > first
> > > > > > > > > > commit is
> > > > > > > > > > > to update the team page ;)
> > > > > > > > > > > - comments like in
> > > > > > > > > >
> r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > > > > > > > >
> > > > > > > > > > > +/**
> > > > > > > > > > > + * Created by aelberg on 7/28/15.
> > > > > > > > > > > + */
> > > > > > > > > > >
> > > > > > > > > > > is better to be removed. It has been already discussed
> > here
> > > > [3].
> > > > > > > And
> > > > > > > > > > the
> > > > > > > > > > > initial discussion on the topic [4] was linked as well
> > > > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not
> sure
> > if
> > > > this is
> > > > > > > > > > R-specific
> > > > > > > > > > > stuff - I have no idea about R, honestly, but
> > > > > > > > > > >
> > > > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > > > > > > > > ...
> > > > > > > > > > > +Author: David B. Dahl
> > > > > > > > > > >
> > > > > > > > > > > shouldn't be here, IMO. Normally, if some additional
> > > > licenses are
> > > > > > > > > > used,
> > > > > > > > > > > they have to be listed in the top-level NOTICE file
> > (already
> > > > > > > there).
> > > > > > > > > > >
> > > > > > > > > > > - I am not going to offer any comments on the
> technical
> > > > merits of
> > > > > > > the
> > > > > > > > > > patch,
> > > > > > > > > > > as I haven't tried it personally. However, I don't see
> > any
> > > > serious
> > > > > > > > > > > technical objections to the functionality in question.
> > > > > > > > > > >
> > > > > > > > > > > So, the question is - why the PR is open for three
> > months? I
> > > > hasn't
> > > > > > > > > been
> > > > > > > > > > able
> > > > > > > > > > > to get a clear answer. What I found out though is
> pretty
> > > > > > > unsettling,
> > > > > > > > > > really.
> > > > > > > > > > > The communication on the topic is almost non-existent,
> > > > except for
> > > > > > > > this
> > > > > > > > > > sparse
> > > > > > > > > > > and bitter thread in the GH.
> > > > > > > > > > >
> > > > > > > > > > > I would love to hear from the committers what's
> stopping
> > the
> > > > > > > > acceptance
> > > > > > > > > > of the
> > > > > > > > > > > code, besides of the minor issues I've mentioned
> earlier?
> > > > What are
> > > > > > > > the
> > > > > > > > > > reasons for it?
> > > > > > > > > > > Is there anything wrong with it?
> > > > > > > > > > > One of the responsibilities of the committers is to
> make
> > > > sure the
> > > > > > > > > > > contributions are reviewed; new contributors are
> guided
> > and
> > > > do
> > > > > > > > > > understand how
> > > > > > > > > > > the project ticks. The easy feedback flow attracts new
> > > > people,
> > > > > > > > allowing
> > > > > > > > > > the
> > > > > > > > > > > community to grow and thrive.
> > > > > > > > > > >
> > > > > > > > > > > I am asking _explicitely_ not to re-start the
> bickering I
> > > > have
> > > > > > > > already
> > > > > > > > > > > seen. At this point I am interested in the purely
> > technical
> > > > side of
> > > > > > > > > this.
> > > > > > > > > > >
> > > > > > > > > > > [1]
> https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > > > > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > > > > > > > [3]
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > >
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > > > > > > > [4] http://s.apache.org/iZl
> > > > > > > > > > >
> > > > > > > > > > > With regards,
> > > > > > > > > > > Cos
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > > > > > > > Github user elbamos commented on the pull request:
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > > > > > > > >
> > > > > > > > > > > > The current push should resolve some issues with
> > changes
> > > > in the
> > > > > > > > > > > > Spark-Zeppelin interface that had created issues for
> > > > users, as
> > > > > > > > > > well as
> > > > > > > > > > > > support for 1.5.1.
> > > > > > > > > > > >
> > > > > > > > > > > > Knitr support is improved, and the reason for a
> > separate
> > > > knitr
> > > > > > > > > > interpreter may be clearer now.
> > > > > > > > > > > >
> > > > > > > > > > > > The amount of code borrowed from rscala is reduced.
> > > > > > > > > > > >
> > > > > > > > > > > > I did not address issues with @author tags, or files
> > under
> > > > the R/
> > > > > > > > > > > > folder. The reason is, to be blunt, I don't
> understand
> > > > what the
> > > > > > > > > > precise
> > > > > > > > > > > > concerns actually are.
> > > > > > > > > > > >
> > > > > > > > > > > > Please note that I changed .travis.yml to only use
> > spark
> > > > 1.4 and
> > > > > > > > > > later.
> > > > > > > > > > > > I'm sure there is a better way to do it, but I'm
> just
> > not
> > > > in a
> > > > > > > > > > position
> > > > > > > > > > > > to try to figure out the entire Zeppelin build
> process.
> > > > > > > > > > > >
> > > > > > > > > > > > Integrating this with the rest of zeppelin is going
> to
> > > > take some
> > > > > > > > > > work
> > > > > > > > > > > > regarding pom's, travis, and so forth. I can do a
> lot
> > of
> > > > that,
> > > > > > > > > > but I'm
> > > > > > > > > > > > going to need to discuss it with the people who have
> > been
> > > > > > > "owning"
> > > > > > > > > > those
> > > > > > > > > > > > files. There are just too many moving pieces here.
> > > > > > > > > > > >
> > > > > > > > > > > > Because of the size of this (which is,
> unfortunately,
> > > > necessary),
> > > > > > > > > > > > posting here is probably not the most efficient way.
> > That
> > > > is also
> > > > > > > > > > true
> > > > > > > > > > > > because certain people chose to use this PR as a
> forum
> > to
> > > > air
> > > > > > > other
> > > > > > > > > > > > issues. Therefore, it would be better if reviewers
> > e-mail
> > > > me
> > > > > > > > > > directly.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > >
> >
>
>

R-Interpreter CI Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Is there anyone who can work with me on the CI issues?  It looks like there are a number of PRs experiencing similar things.  

I think we should make getting CI stable to be a priority.  Because it will save everyone a lot of frustration and aggravation if CI works reliably. 

Is there anyone other than Jongyoul and Moon who knows the CI/Build process well?

(Moon - Thank you for taking another look at the licensing issue.  Per the e-mail I wrote about this a few days ago, I don’t feel I have more to contribute to the licensing discussion, so I’m going to try not to comment further about it.)

From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 7, 2015 at 5:00:08 PM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

Thanks Amos, Roman, Cos for clarifying license issue.  

I'm convinced that this license issue will not be a blocker.  

In my understanding, these are good sign,  

1. any gpl licensed source codes are not included in the source package  
2. any gpl licensed libraries are not included in the binary package  

However, i can not still 100% sure about  

3. any gpl licensed libraries are not linked on runtime  

Even after Amos's explanation. I still think using 'knitr' is one of the  
clear case that 'knir' is linked to 'R' according to  
http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL.  

Giving input and getting output from GPL licensed interpreter (includes R)  
from Apache licensed software is not a problem. That's not the point.  
Let me say in this way. There's an java code,  

import com.mysql.jdbc.Driver  
Driver driver = new Driver()  

Say without this java code, one of important feature of Zeppelin does not  
work. And Zeppelin does not includes GPL licensed file in the source  
package, GPL licensed library in the binary package, but it requires GPL  
licensed library on the runtime.  
In this case, will this java code be a license problem or not?  

In other words, my question is  

a) Is runtime GPL library dependency allowed in ASF release?  
b) is 'knitr' considered as runtime dependency?  

If someone can clarify a), b), then it would be extremely helpful  
understanding this case, and possible similar cases, too.  

Thanks,  
moon  

On Mon, Dec 7, 2015 at 3:20 PM Konstantin Boudnik <co...@apache.org> wrote:  

> On Thu, Dec 03, 2015 at 11:03AM, Corneau Damien wrote:  
> > Thanks Cos for those answers,  
> >  
> > If I'm right you are advising to have a default build that doesn't  
> include  
> > libraries with conflicting licenses, but have an option to include them  
> for  
> > users who wants to build the project themselves.  
>  
> Yes, that's what I said. Besides, looks like Roman provided the second  
> pair of  
> eyes to this licensing discussion and as well didn't find any issues with  
> the  
> current approach.  
>  
> Cos  
>  
> > To refer to another thread about decentralizing interpreters, it could  
> even  
> > be better in that case to have some interpreters separated from the tree,  
> > and easily pluggable with a release instead of forcing users to build the  
> > project to use them.  
> >  
> >  
> > On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <co...@apache.org>  
> wrote:  
> >  
> > > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:  
> > > > Konstantin thank you for getting into this.  
> > > >  
> > > >> The best way to go around it is by  
> > > >> providing a build-time option that will pull such binaries in. But  
> by  
> > > default  
> > > >> such libs shouldn't be pulled.  
> > > >  
> > > > That is basically how the PR handles this. If the GPL’d interpreter  
> > > scripts  
> > > > are missing, there’s no indication at all at build time. It doesn’t  
> try  
> > > to  
> > > > download them.  
> > > >  
> > > > At runtime, if the user tries to use functionality that would need  
> such a  
> > > > script (i.e., if they type “knitr” to use knitr), we display an error  
> > > that  
> > > > says that the functionality is not there because the library is  
> missing,  
> > > and  
> > > > that the library cannot be provided because it has an incompatible  
> > > license,  
> > > > but the user can download it themselves if they want.  
> > > >  
> > > > And, in the log, if the logging level is high, they will see a note  
> that  
> > > > some functionality was disabled because the libraries weren’t there.  
> > > >  
> > > > To be clear, though, none of these libraries are binaries. They’re  
> all  
> > > interpreter scripts.  
> > >  
> > > If you ever in a doubt of which licenses could be used for dependncies  
> > > (not to  
> > > say about source code) are listed in Category A list of [1]  
> > >  
> > > A lot of quesitons discussed here are already covered in the legal  
> FAQ, so  
> > > just check against it if you have any questions.  
> > >  
> > > [1] http://www.apache.org/legal/resolved.html#category-a  
> > >  
> > > Cos  
> > >  
> > > > From: Konstantin Boudnik <co...@apache.org>  
> > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > dev@zeppelin.incubator.apache.org>, dev@zeppelin.incubator.apache.org  
> <  
> > > dev@zeppelin.incubator.apache.org>  
> > > > Date: December 2, 2015 at 3:24:50 PM  
> > > > To: dev@zeppelin.incubator.apache.org <  
> dev@zeppelin.incubator.apache.org  
> > > >  
> > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions  
> > > impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter  
> for  
> > > Zeppelin  
> > > >  
> > > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:  
> > > > > I think that what moon means is that:  
> > > > > If we merge the way it is now, the KnitRInterpreter will be part  
> of the  
> > > > > code base, so it isn't optional at code base level.  
> > > > >  
> > > > > To make it even more simple:  
> > > > > * If the code has the right licensing -> that code can be part of  
> the  
> > > > > source code, and can be including in a binary release  
> > > >  
> > > > We aren't concerned with binary releases - as an Apache community we  
> are  
> > > > voting and releasing source code. If the project wants to provide a  
> > > binary  
> > > > release to its users, they are better be warned about inclusion of  
> non  
> > > > ASL2-friendly licensed bits.  
> > > >  
> > > > > * If the code doesn't have the right licensing -> it can't be part  
> of  
> > > the  
> > > > > source code, and can't be included in a binary release  
> > > >  
> > > > See above.  
> > > >  
> > > > > * If the code doesn't have the right licensing but is imported at  
> build  
> > > > > time (libraries for example) -> it is not in the source code, it  
> can't  
> > > be  
> > > > > included in binary release  
> > > >  
> > > > That is unless a user doing it on his own. The best way to go around  
> it  
> > > is by  
> > > > providing a build-time option that will pull such binaries in. But by  
> > > default  
> > > > such libs shouldn't be pulled.  
> > > >  
> > > > Cos  
> > > >  
> > > > > So in the case of licensing issues, it would need to be fully  
> optional  
> > > > > (user bring the interpreter in his directory and build Zeppelin  
> with  
> > > it)  
> > > > >  
> > > > >  
> > > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <  
> amos.elberg@gmail.com>  
> > > > > wrote:  
> > > > >  
> > > > > > Moon let me clarify:  
> > > > > >  
> > > > > > Interpreted code doesn’t “link.” The wiki article actually  
> explains  
> > > it  
> > > > > > pretty well —  
> https://en.wikipedia.org/wiki/GPL_linking_exception  
> > > > > > “Linking” against a library means compiling its headers into a  
> > > binary, the  
> > > > > > way a C compiler works. The 2008 e-mail Moon distributed, called  
> > > this the  
> > > > > > “interpreter exception.”  
> > > > > >  
> > > > > > As for whether GPL’d code is a “mandatory dependency,” if knitr  
> is  
> > > missing  
> > > > > > the PR will compile, run and test just fine. The user is never  
> > > prompted to  
> > > > > > download it. The only effect is, if the user types “knitr” and  
> knitr  
> > > isn’t  
> > > > > > there, we display an InterpreterError that knitr isn’t there.  
> > > > > >  
> > > > > > KnitRInterpreter is not optionally required. so it does not  
> matter  
> > > KnitR  
> > > > > > is  
> > > > > > optionally required or not.  
> > > > > > Aren’t all interpreters optional? You can turn them on and off  
> in the  
> > > > > > config files.  
> > > > > >  
> > > > > > Do you mean that the KnitRInterpreter class gets compiled to  
> > > bytecode even  
> > > > > > if knitr is missing? So what? That isn't a mandatory dependency  
> or a  
> > > link.  
> > > > > >  
> > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > Date: December 2, 2015 at 3:18:00 AM  
> > > > > > To: dev@zeppelin.incubator.apache.org <  
> > > dev@zeppelin.incubator.apache.org>  
> > > > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions  
> > > impasse.  
> > > > > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for  
> > > Zeppelin  
> > > > > >  
> > > > > > Let me summarize license concern about KnitRInterpreter.  
> > > > > >  
> > > > > >  
> > > > > > Amos's interpretation.  
> > > > > >  
> > > > > > KnitR is optionally required by KnitRInterpreter.  
> > > > > > R dependency in SparkR has no problem. So KnitR should be the  
> same.  
> > > > > >  
> > > > > >  
> > > > > > Moon's interpretation.  
> > > > > >  
> > > > > > KnitRInterpreter is not optionally required. so it does not  
> matter  
> > > KnitR is  
> > > > > > optionally required or not.  
> > > > > > R dependency in SparkR is exception of GPL. KnitR is not applied  
> that  
> > > > > > exception.  
> > > > > >  
> > > > > >  
> > > > > > Amos, could you confirm my understanding to your interpretation  
> is  
> > > correct?  
> > > > > > If it is not could you clarify it?  
> > > > > >  
> > > > > > Thanks,  
> > > > > > moon  
> > > > > >  
> > > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <  
> > > amos.elberg@gmail.com>  
> > > > > > wrote:  
> > > > > >  
> > > > > > > Just to put the final nail in this, I looked it up.  
> > > > > > >  
> > > > > > > I see no evidence of any “compiler exception.”  
> > > > > > >  
> > > > > > > There is an exception in the license for the runtime libraries  
> > > that are  
> > > > > > > bundled with GCC. See:  
> > > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html  
> > > > > > >  
> > > > > > > But no “compiler exception.”  
> > > > > > >  
> > > > > > > In fact, I believe this is part of the reason why LLVM was  
> created.  
> > > > > > >  
> > > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > > Reply: moon soo Lee <mo...@apache.org>  
> > > > > > > Date: December 1, 2015 at 8:16:36 PM  
> > > > > > > To: Amos B. Elberg <am...@gmail.com>,  
> > > > > > > dev@zeppelin.incubator.apache.org <  
> > > dev@zeppelin.incubator.apache.org>  
> > > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > > incubator-zeppelin pull  
> > > > > > > request: R Interpreter for Zeppelin  
> > > > > > >  
> > > > > > > Is knitR is commonly considered as a interpreter/compiler? or  
> is it  
> > > > > > > considered as a library routine?  
> > > > > > >  
> > > > > > > Thanks,  
> > > > > > > moon  
> > > > > > >  
> > > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <  
> > > amos.elberg@gmail.com>  
> > > > > > > wrote:  
> > > > > > > Moon - you give this as an explanation of the licensing issue:  
> > > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
> > > > > > >  
> > > > > > > According to that, there is an exception in the GPL for  
> interpreter  
> > > > > > > languages. As long as you don’t distribute the code, its fine  
> to  
> > > talk to  
> > > > > > > an interpreted language.  
> > > > > > >  
> > > > > > > Well, if that’s the case, then the PR plainly does not have a  
> > > license  
> > > > > > > issue. It doesn’t distribute any GPL’d R code.  
> > > > > > >  
> > > > > > > I’m not sure what’s confusing about this. It seems completely  
> > > > > > > straightforward.  
> > > > > > >  
> > > > > > > Regarding this:  
> > > > > > >  
> > > > > > >  
> > > > > > > --  
> > > > > > > Amos Elberg  
> > > > > > > Sent with Airmail  
> > > > > > >  
> > > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > > Date: December 1, 2015 at 6:48:47 PM  
> > > > > > >  
> > > > > > > To: dev@zeppelin.incubator.apache.org <  
> > > dev@zeppelin.incubator.apache.org  
> > > > > > >  
> > > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > > incubator-zeppelin pull  
> > > > > > > request: R Interpreter for Zeppelin  
> > > > > > >  
> > > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <  
> > > amos.elberg@gmail.com>  
> > > > > > > wrote:  
> > > > > > >  
> > > > > > > > I am going to try to minimize my reaction to Moon’s e-mail.  
> > > > > > > >  
> > > > > > > > The tl;dr is this:  
> > > > > > > >  
> > > > > > > > The reason we are having this discussion now is that active  
> > > users of  
> > > > > > the  
> > > > > > > > PR — which now has its own user base — went public to  
> complain  
> > > about  
> > > > > > > this.  
> > > > > > >  
> > > > > > >  
> > > > > > > > The PR has been tested by an active user base for more than  
> three  
> > > > > > months.  
> > > > > > > > No-one has been able to identify any specific actual  
> licensing  
> > > problem,  
> > > > > > > and  
> > > > > > > > the PR was prepared based on an extensive, careful review of  
> the  
> > > > > > relevant  
> > > > > > > > licensing issues and after contacting the relevant people.  
> > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > > > I admire every software that used by user and helping people.  
> That  
> > > > > > includes  
> > > > > > > your work. But that's not the topic we're in discussion. Active  
> > > user does  
> > > > > > > not mean your contribution can ignore the review.  
> > > > > > >  
> > > > > > >  
> > > > > > >  
> > > > > > > > It is not an explanation for someone who has been ignoring my  
> > > “how can  
> > > > > > I  
> > > > > > > > move this forward…” emails for three months to point the  
> finger  
> > > and  
> > > > > > say I  
> > > > > > > > didn’t contact the right person or file the right report.  
> > > > > > > >  
> > > > > > > >  
> > > > > > > This is also not the topic in this discussion.  
> > > > > > >  
> > > > > > >  
> > > > > > > > The burden for providing an explanation for the inaction is  
> on  
> > > the PMCC  
> > > > > > > at  
> > > > > > > > this point.  
> > > > > > >  
> > > > > > > I'm sorry, but the other PRs are passing CI. If it's problem on  
> > > Zeppelin  
> > > > > > > > core, why do you think other PRs are passing CI?  
> > > > > > > > They’re not! I often see comments on PRs to just ignore that  
> CI  
> > > is  
> > > > > > > > failing.  
> > > > > > > >  
> > > > > > > > One of the most common reasons this PR fails CI, is CI  
> times-out  
> > > > > > > > downloading Spark to install. How could that possibly be  
> caused  
> > > by the  
> > > > > > > PR?  
> > > > > > > >  
> > > > > > > > It looks to me like the only PRs with changes to the relevant  
> > > parts of  
> > > > > > > the  
> > > > > > > > code — the SparkInterpreter — are being made by the person  
> who  
> > > wrote  
> > > > > > the  
> > > > > > > > testing suite.  
> > > > > > > >  
> > > > > > > > So, that would explain why some other PRs pass CI: Neither  
> the  
> > > > > > > > SparkInterpreter nor the testing suite are stable or robust,  
> but  
> > > since  
> > > > > > > the  
> > > > > > > > PRs are coming from the person who wrote both…  
> > > > > > > >  
> > > > > > > > And let's say Zeppelin core has problem and that makes your  
> PR  
> > > fails on  
> > > > > > > CI  
> > > > > > > > test. That's possible. But it still does not mean we can  
> merge  
> > > it with  
> > > > > > CI  
> > > > > > > > failing.  
> > > > > > > >  
> > > > > > > > It means you should be working with me to figure out why the  
> CI  
> > > is  
> > > > > > > failing.  
> > > > > > > >  
> > > > > > > > This PR has been tested by an active user base for the past  
> three  
> > > > > > months.  
> > > > > > > > If CI is continuing to fail, and dozens of hours of effort  
> have  
> > > not  
> > > > > > > > resolved the CI issues, then it is time to start considering  
> > > whether  
> > > > > > the  
> > > > > > > > testing suite is part of the problem.  
> > > > > > > >  
> > > > > > > > The level of defensiveness about the CI and SparkInterpreter  
> is  
> > > not  
> > > > > > > > helping to resolve these issues.  
> > > > > > > >  
> > > > > > > > If you think it's problem on Zeppelin core, then file an  
> issue  
> > > that  
> > > > > > > > reproduce the problem on Zeppelin core, that might be more  
> > > efficient  
> > > > > > than  
> > > > > > > > keep trying yourself.  
> > > > > > > > I contacted you numerous times about such issues...  
> > > > > > > >  
> > > > > > >  
> > > > > > >  
> > > > > > > I remember i commented your issue about CI. but you just keep  
> > > repeated  
> > > > > > it's  
> > > > > > > not your problem but Zeppelin core problem.  
> > > > > > >  
> > > > > > > Then please file an issue about the problem you found in  
> Zeppelin  
> > > Core.  
> > > > > > > Then everyone will get into the problem.  
> > > > > > >  
> > > > > > >  
> > > > > > >  
> > > > > > > >  
> > > > > > > > In my interpretation, KnitRInterpreter is not an optional  
> > > feature while  
> > > > > > > it  
> > > > > > > > is always enabled when compiling Zeppelin and always enabled  
> when  
> > > > > > running  
> > > > > > > > Zeppelin. And it requires dynamically linked GPL library on  
> > > runtime.  
> > > > > > (yes  
> > > > > > > > it will fail when no KnitR is installed on the system)  
> > > > > > > >  
> > > > > > > > Its not always enabled.  
> > > > > > > > It is not dynamically linked at runtime.  
> > > > > > > > It will not fail when knitr is missing. If knitr is not  
> present,  
> > > the  
> > > > > > repl  
> > > > > > > > interpreter starts and a note is written to to the log that  
> the  
> > > knitr  
> > > > > > > > interpreter isn’t available because knitr is not present.  
> > > > > > > >  
> > > > > > > > no Apache code can ever call a shell script, on the purpose  
> of  
> > > dynamic  
> > > > > > > > linking with GPL library.  
> > > > > > > > You misunderstand.  
> > > > > > > >  
> > > > > > > > The *shell* is GPL'd.  
> > > > > > > >  
> > > > > > > > Is Zeppelin “linked" against the GPL’d shell because Zeppelin  
> > > depends  
> > > > > > on  
> > > > > > > a  
> > > > > > > > shell script to launch?  
> > > > > > > >  
> > > > > > > Obviously not.  
> > > > > > > >  
> > > > > > > > The interaction with R in the PR is the same.  
> > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > > > Again, bash is one of exceptions of GPL, like other GPL  
> licensed  
> > > > > > > compiler/interpreter.  
> > > > > > >  
> > > > > > > Check here why Bash and R is okay with Apache License.  
> > > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
> > > > > > >  
> > > > > > > I'm not sure we can apply the same exception for 'using' KnitR.  
> > > > > > >  
> > > > > > > My point is not 'KnitR' is optional or not. Point is  
> > > 'KnitRInterpreter'  
> > > > > > you  
> > > > > > > wrote is not an optional feature. Which is clearly not  
> optionally  
> > > enabled  
> > > > > > > code and feature. And that depends on KnitR library which is  
> GPL.  
> > > > > > >  
> > > > > > >  
> > > > > > >  
> > > > > > > > I was guessing SparkR can be still in Apache License even if  
> it  
> > > is  
> > > > > > > depends  
> > > > > > > > on R. Because of GPL licensed compiler generated output is  
> not  
> > > GPL  
> > > > > > > license.  
> > > > > > > > and R is sort of compiler. If you can get answer from Spark  
> > > community  
> > > > > > how  
> > > > > > > > SparkR get managed to stay in Apache License while R is GPL,  
> the  
> > > answer  
> > > > > > > > might help.  
> > > > > > > > The description of SparkR is not accurate in any respect. (Do  
> > > you think  
> > > > > > > > SparkR is not talking to GPL-licensed libraries?)  
> > > > > > > >  
> > > > > > > > I don’t see that any genuine issue is being raised here.  
> > > > > > > >  
> > > > > > > > If there is an issue, the burden is on you to identify it.  
> > > > > > > >  
> > > > > > > > If i give you one suggestion, Zeppelin committers sometimes  
> ask  
> > > rebase  
> > > > > > > the  
> > > > > > > > contribution branch for some reason. It is not the really the  
> > > best  
> > > > > > > > practice, but still okay while most contributions are not  
> > > including  
> > > > > > large  
> > > > > > > > code base changes  
> > > > > > > > However, your one, has more than 4000 lines of code change.  
> If  
> > > you  
> > > > > > rebase  
> > > > > > > > then review should be started from the beginning, again. So  
> you  
> > > might  
> > > > > > > want  
> > > > > > > > to minimize the rebase your branch.  
> > > > > > > >  
> > > > > > > > Are you actually complaining that the problem is that I  
> rebased  
> > > the  
> > > > > > code  
> > > > > > > > during the three-month period when no-one looked at it and  
> > > Zeppelin  
> > > > > > went  
> > > > > > > > through a release?  
> > > > > > > >  
> > > > > > > > I cannot take it seriously when you say things like this.  
> > > > > > > >  
> > > > > > > > Having to “start from the beginning” cannot be a problem if  
> you  
> > > never  
> > > > > > > > actually started the first time...  
> > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > > > You wanted coordination and cooperation. So i gave you  
> suggestion  
> > > that  
> > > > > > > helping review process. For example, your code has been rebased  
> > > since my  
> > > > > > > comment and jongyoul's comment. that means committers will  
> need to  
> > > look  
> > > > > > > from the beginning. That'll require more time. And maybe, i  
> guess  
> > > that's  
> > > > > > > not what you want. But If you don't care, feel free to rebase.  
> > > > > > >  
> > > > > > > Thanks,  
> > > > > > > moon  
> > > > > > >  
> > > > > > >  
> > > > > > >  
> > > > > > > >  
> > > > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > > > Reply: moon soo Lee <mo...@apache.org>  
> > > > > > > > Date: December 1, 2015 at 4:42:06 AM  
> > > > > > > > To: Amos B. Elberg <am...@gmail.com>,  
> > > > > > > > dev@zeppelin.incubator.apache.org <  
> > > dev@zeppelin.incubator.apache.org>  
> > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > > incubator-zeppelin  
> > > > > > pull  
> > > > > > > > request: R Interpreter for Zeppelin  
> > > > > > > >  
> > > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <  
> > > amos.elberg@gmail.com>  
> > > > > > > > wrote:  
> > > > > > > > Thank you, Cos.  
> > > > > > > >  
> > > > > > > > I’d like to briefly address the issues raised by Moon:  
> > > > > > > >  
> > > > > > > > 1. This PR does not passes CI  
> > > > > > > > The CI fails on core Zeppelin, *not* code in this PR.  
> > > > > > > >  
> > > > > > > > I’ve been seeking assistance on this since August.  
> > > > > > > >  
> > > > > > > > The most common reason is that SparkInterpreter is unable to  
> > > launch  
> > > > > > Spark  
> > > > > > > > and open a Spark Backend. This is necessary to test the PR.  
> > > > > > > >  
> > > > > > > > 60+ hours, has been spent adapting and re-basing when the  
> > > > > > > SparkInterpreter  
> > > > > > > > architecture changed and broke the PR’s *tests.*  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > I'm sorry, but the other PRs are passing CI. If it's problem  
> on  
> > > > > > Zeppelin  
> > > > > > > > core, why do you think other PRs are passing CI?  
> > > > > > > >  
> > > > > > > > And let's say Zeppelin core has problem and that makes your  
> PR  
> > > fails on  
> > > > > > > CI  
> > > > > > > > test. That's possible. But it still does not mean we can  
> merge  
> > > it with  
> > > > > > CI  
> > > > > > > > failing.  
> > > > > > > >  
> > > > > > > > If you think it's problem on Zeppelin core, then file an  
> issue  
> > > that  
> > > > > > > > reproduce the problem on Zeppelin core, that might be more  
> > > efficient  
> > > > > > than  
> > > > > > > > keep trying yourself.  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)  
> > > > > > > > What license problem *specifically* do you believe may exist?  
> > > > > > > >  
> > > > > > > > In preparing the PR, I:  
> > > > > > > >  
> > > > > > > > * Reviewed the Apache policy in detail.  
> > > > > > > >  
> > > > > > > > * Contacted the FSF to ask their interpretation of the  
> “linking”  
> > > > > > > > provisions of the GPL license.  
> > > > > > > >  
> > > > > > > > * Reviewed how other Apache software deals with this issue  
> > > (e.g., Spark  
> > > > > > > > talks to R, which is GPL'd).  
> > > > > > > >  
> > > > > > > > * No necessary *dependencies* of the PR have license  
> conflicts.  
> > > In  
> > > > > > > > several cases, I contacted package authors who agreed to  
> > > re-issue their  
> > > > > > > > packages under Apache-compatible licenses. (Usually I had to  
> do  
> > > a bit  
> > > > > > of  
> > > > > > > > coding in exchange…)  
> > > > > > > >  
> > > > > > > > * Where the license had to stay GPL, the packages are *not  
> > > necessary*  
> > > > > > and  
> > > > > > > > *not dependencies.* If the optional packages are present,  
> they  
> > > will be  
> > > > > > > > used to enable additional functionality. Knitr is an example.  
> > > The PR  
> > > > > > will  
> > > > > > > > compile and run fine without knitr. If knitr is available  
> (it is  
> > > part  
> > > > > > of  
> > > > > > > > the most common R distribution), the PR will enable the knitr  
> > > > > > > interpreter.  
> > > > > > > >  
> > > > > > > > * This is exactly how this issue is addressed through the  
> Apache  
> > > > > > > > ecosystem.  
> > > > > > > > The tl;dr is this: When Apache code is written to talk to  
> > > libraries  
> > > > > > that  
> > > > > > > > may or may not be present on the user’s system, or where it  
> > > talks to an  
> > > > > > > API  
> > > > > > > > but is agnostic about implementation, that is not “linking”  
> in a  
> > > way  
> > > > > > that  
> > > > > > > > implicate the anti-linking provision of the GPL.  
> > > > > > > >  
> > > > > > > > Otherwise, no Apache code could ever call a shell script! Let  
> > > alone run  
> > > > > > > > on Linux, or talk to R.  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > I'm not a legal expert. So following could be wrong.  
> > > > > > > >  
> > > > > > > > In my interpretation, KnitRInterpreter is not an optional  
> > > feature while  
> > > > > > > it  
> > > > > > > > is always enabled when compiling Zeppelin and always enabled  
> when  
> > > > > > running  
> > > > > > > > Zeppelin. And it requires dynamically linked GPL library on  
> > > runtime.  
> > > > > > (yes  
> > > > > > > > it will fail when no KnitR is installed on the system)  
> > > > > > > >  
> > > > > > > > And of course, no Apache code can ever call a shell script,  
> on  
> > > the  
> > > > > > > purpose  
> > > > > > > > of dynamic linking with GPL library.  
> > > > > > > >  
> > > > > > > > I was guessing SparkR can be still in Apache License even if  
> it  
> > > is  
> > > > > > > depends  
> > > > > > > > on R. Because of GPL licensed compiler generated output is  
> not  
> > > GPL  
> > > > > > > license.  
> > > > > > > > and R is sort of compiler.  
> > > > > > > >  
> > > > > > > > If you can get answer from Spark community how SparkR get  
> > > managed to  
> > > > > > stay  
> > > > > > > > in Apache License while R is GPL, the answer might help.  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > 3. Need more time to review.  
> > > > > > > > Has any reviewer has downloaded the PR or run the demo  
> notebook?  
> > > (Which  
> > > > > > > > is there for the benefit of reviewers, and isn’t intended to  
> go  
> > > in  
> > > > > > final  
> > > > > > > > distribution.)  
> > > > > > > >  
> > > > > > > > How many +1 comments from users would you like to see?  
> > > > > > > >  
> > > > > > > > How much time do you believe is required?  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > It all depends on when CI is going to pass, when license  
> problem  
> > > is  
> > > > > > going  
> > > > > > > > to be cleared, and when a committer willing to review and  
> > > responsible  
> > > > > > to  
> > > > > > > > commit your contribution.  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > 1. Large code base change  
> > > > > > > > Large code base changes require coordination and cooperation.  
> > > This PR  
> > > > > > > > necessarily implicates the build scripts, testing code, the  
> > > > > > > > SparkInterpreter, etc.  
> > > > > > > >  
> > > > > > > > I have been seeking to coordinate since August.  
> > > > > > > >  
> > > > > > > > I continue to stand ready to do so.  
> > > > > > > >  
> > > > > > > > -Amos  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > If i give you one suggestion, Zeppelin committers sometimes  
> ask  
> > > rebase  
> > > > > > > the  
> > > > > > > > contribution branch for some reason. It is not the really the  
> > > best  
> > > > > > > > practice, but still okay while most contributions are not  
> > > including  
> > > > > > large  
> > > > > > > > code base changes.  
> > > > > > > >  
> > > > > > > > However, your one, has more than 4000 lines of code change.  
> If  
> > > you  
> > > > > > rebase  
> > > > > > > > then review should be started from the beginning, again. So  
> you  
> > > might  
> > > > > > > want  
> > > > > > > > to minimize the rebase your branch.  
> > > > > > > >  
> > > > > > > > Thanks,  
> > > > > > > > moon  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > > > Date: December 1, 2015 at 1:34:19 AM  
> > > > > > > > To: dev@zeppelin.incubator.apache.org <  
> > > > > > dev@zeppelin.incubator.apache.org  
> > > > > > > >  
> > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > > incubator-zeppelin  
> > > > > > pull  
> > > > > > > > request: R Interpreter for Zeppelin  
> > > > > > > >  
> > > > > > > > Hi Cos,  
> > > > > > > >  
> > > > > > > > Thanks for opening a discussion.  
> > > > > > > > My answer about 'Why this PR is open for three months' is  
> > > > > > > >  
> > > > > > > > 1. This PR does not passes CI  
> > > > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)  
> > > > > > > > 3. Need more time to review.  
> > > > > > > >  
> > > > > > > > It's my personal answer, other committers may have different  
> > > opinion.  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > I think the question should be generalized. Because this PR  
> is  
> > > not the  
> > > > > > > only  
> > > > > > > > PR that is in impasse. There're more. For example  
> > > > > > > >  
> > > > > > > > Here's some examples that PR is not been merged.  
> > > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,  
> > > > > > > > https://github.com/apache/incubator-zeppelin/pull/60  
> > > > > > > > and so on.  
> > > > > > > >  
> > > > > > > > I can categorize the cases, based on experience of involving  
> > > Zeppelin  
> > > > > > > > community from the beginning.  
> > > > > > > >  
> > > > > > > > 1. Large code base change  
> > > > > > > >  
> > > > > > > > When contribution has large code base changes, it tend to  
> take  
> > > more  
> > > > > > time  
> > > > > > > to  
> > > > > > > > review and merged. Normally, most contributions merged in  
> 1day~1  
> > > week.  
> > > > > > > > But some contribution has large code base changes take 2~4  
> > > weeks. Few  
> > > > > > > > contribution that has very large code base change take  
> months.  
> > > > > > > >  
> > > > > > > > 2. Communication lost  
> > > > > > > >  
> > > > > > > > Sometimes, committer is not responding, or contributor is not  
> > > > > > responding.  
> > > > > > > >  
> > > > > > > > 3. Opinion diverges  
> > > > > > > >  
> > > > > > > > For some changes, included in contribution. When people have  
> > > different  
> > > > > > > > opinion and it continue to diverges, those PRs are not been  
> > > merged.  
> > > > > > > >  
> > > > > > > >  
> > > > > > > > I think having a guide such as ping other committer when a  
> > > committer is  
> > > > > > > not  
> > > > > > > > responding, and divide contribution into small peaces if  
> > > possible,  
> > > > > > would  
> > > > > > > > help most of the cases.  
> > > > > > > > Of course committer have to pay attention more to the  
> > > contribution and  
> > > > > > > > helping people. That's the first one.  
> > > > > > > >  
> > > > > > > > Thanks,  
> > > > > > > > moon  
> > > > > > > >  
> > > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <  
> > > cos@apache.org>  
> > > > > > > wrote:  
> > > > > > > >  
> > > > > > > > > To make sure we're on the same page, here are two threads  
> that  
> > > I  
> > > > > > found  
> > > > > > > > > related  
> > > > > > > > > to this topic.  
> > > > > > > > >  
> > > > > > > > > Thread 1:  
> > > > > > > > > Subject: R?  
> > > > > > > > > Started on: July 1, 2015  
> > > > > > > > >  
> > > > > > > > > Thread 2:  
> > > > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R  
> > > Interpreter for  
> > > > > > > > > Zeppelin  
> > > > > > > > > Started on: August 13, 2015  
> > > > > > > > >  
> > > > > > > > > If you want to fetch these from our archive send emails to  
> > > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org  
> > > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org  
> > > > > > > > >  
> > > > > > > > > Cos  
> > > > > > > > >  
> > > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:  
> > > > > > > > > > Guys,  
> > > > > > > > > >  
> > > > > > > > > > While catching up on my emails from the last a couple of  
> > > weeks,  
> > > > > > this  
> > > > > > > > > thread  
> > > > > > > > > > caught my attention. I am not normally paying much  
> attention  
> > > to the  
> > > > > > > > code  
> > > > > > > > > > reviews traffic from GH, but this one is pretty  
> different as  
> > > it  
> > > > > > spans  
> > > > > > > > > three  
> > > > > > > > > > months and counting.  
> > > > > > > > > >  
> > > > > > > > > > First, here are my five cents:  
> > > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed  
> to be  
> > > > > > > > > contributed to  
> > > > > > > > > > an ASF project this file should simply contain ASL2 text,  
> > > like in  
> > > > > > [1]  
> > > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate  
> <developers>  
> > > > > > > section,  
> > > > > > > > > but  
> > > > > > > > > > Zeppelin might have different guidelines on it. Say,  
> Bigtop  
> > > doesn't  
> > > > > > > > > > maintain this in the source code, but we have the list of  
> > > all the  
> > > > > > > > > > committers on the project's site [2] Every new  
> committer's  
> > > first  
> > > > > > > > > commit is  
> > > > > > > > > > to update the team page ;)  
> > > > > > > > > > - comments like in  
> > > > > > > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java  
> > > > > > > > > >  
> > > > > > > > > > +/**  
> > > > > > > > > > + * Created by aelberg on 7/28/15.  
> > > > > > > > > > + */  
> > > > > > > > > >  
> > > > > > > > > > is better to be removed. It has been already discussed  
> here  
> > > [3].  
> > > > > > And  
> > > > > > > > > the  
> > > > > > > > > > initial discussion on the topic [4] was linked as well  
> > > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure  
> if  
> > > this is  
> > > > > > > > > R-specific  
> > > > > > > > > > stuff - I have no idea about R, honestly, but  
> > > > > > > > > >  
> > > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE  
> > > > > > > > > > ...  
> > > > > > > > > > +Author: David B. Dahl  
> > > > > > > > > >  
> > > > > > > > > > shouldn't be here, IMO. Normally, if some additional  
> > > licenses are  
> > > > > > > > > used,  
> > > > > > > > > > they have to be listed in the top-level NOTICE file  
> (already  
> > > > > > there).  
> > > > > > > > > >  
> > > > > > > > > > - I am not going to offer any comments on the technical  
> > > merits of  
> > > > > > the  
> > > > > > > > > patch,  
> > > > > > > > > > as I haven't tried it personally. However, I don't see  
> any  
> > > serious  
> > > > > > > > > > technical objections to the functionality in question.  
> > > > > > > > > >  
> > > > > > > > > > So, the question is - why the PR is open for three  
> months? I  
> > > hasn't  
> > > > > > > > been  
> > > > > > > > > able  
> > > > > > > > > > to get a clear answer. What I found out though is pretty  
> > > > > > unsettling,  
> > > > > > > > > really.  
> > > > > > > > > > The communication on the topic is almost non-existent,  
> > > except for  
> > > > > > > this  
> > > > > > > > > sparse  
> > > > > > > > > > and bitter thread in the GH.  
> > > > > > > > > >  
> > > > > > > > > > I would love to hear from the committers what's stopping  
> the  
> > > > > > > acceptance  
> > > > > > > > > of the  
> > > > > > > > > > code, besides of the minor issues I've mentioned earlier?  
> > > What are  
> > > > > > > the  
> > > > > > > > > reasons for it?  
> > > > > > > > > > Is there anything wrong with it?  
> > > > > > > > > > One of the responsibilities of the committers is to make  
> > > sure the  
> > > > > > > > > > contributions are reviewed; new contributors are guided  
> and  
> > > do  
> > > > > > > > > understand how  
> > > > > > > > > > the project ticks. The easy feedback flow attracts new  
> > > people,  
> > > > > > > allowing  
> > > > > > > > > the  
> > > > > > > > > > community to grow and thrive.  
> > > > > > > > > >  
> > > > > > > > > > I am asking _explicitely_ not to re-start the bickering I  
> > > have  
> > > > > > > already  
> > > > > > > > > > seen. At this point I am interested in the purely  
> technical  
> > > side of  
> > > > > > > > this.  
> > > > > > > > > >  
> > > > > > > > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE  
> > > > > > > > > > [2] http://bigtop.apache.org/team-list.html  
> > > > > > > > > > [3]  
> > > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > >  
> > >  
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html  
> > > > > > > > > > [4] http://s.apache.org/iZl  
> > > > > > > > > >  
> > > > > > > > > > With regards,  
> > > > > > > > > > Cos  
> > > > > > > > > >  
> > > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:  
> > > > > > > > > > > Github user elbamos commented on the pull request:  
> > > > > > > > > > >  
> > > > > > > > > > >  
> > > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > >  
> > >  
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411  
> > > > > > > > > > >  
> > > > > > > > > > > The current push should resolve some issues with  
> changes  
> > > in the  
> > > > > > > > > > > Spark-Zeppelin interface that had created issues for  
> > > users, as  
> > > > > > > > > well as  
> > > > > > > > > > > support for 1.5.1.  
> > > > > > > > > > >  
> > > > > > > > > > > Knitr support is improved, and the reason for a  
> separate  
> > > knitr  
> > > > > > > > > interpreter may be clearer now.  
> > > > > > > > > > >  
> > > > > > > > > > > The amount of code borrowed from rscala is reduced.  
> > > > > > > > > > >  
> > > > > > > > > > > I did not address issues with @author tags, or files  
> under  
> > > the R/  
> > > > > > > > > > > folder. The reason is, to be blunt, I don't understand  
> > > what the  
> > > > > > > > > precise  
> > > > > > > > > > > concerns actually are.  
> > > > > > > > > > >  
> > > > > > > > > > > Please note that I changed .travis.yml to only use  
> spark  
> > > 1.4 and  
> > > > > > > > > later.  
> > > > > > > > > > > I'm sure there is a better way to do it, but I'm just  
> not  
> > > in a  
> > > > > > > > > position  
> > > > > > > > > > > to try to figure out the entire Zeppelin build process.  
> > > > > > > > > > >  
> > > > > > > > > > > Integrating this with the rest of zeppelin is going to  
> > > take some  
> > > > > > > > > work  
> > > > > > > > > > > regarding pom's, travis, and so forth. I can do a lot  
> of  
> > > that,  
> > > > > > > > > but I'm  
> > > > > > > > > > > going to need to discuss it with the people who have  
> been  
> > > > > > "owning"  
> > > > > > > > > those  
> > > > > > > > > > > files. There are just too many moving pieces here.  
> > > > > > > > > > >  
> > > > > > > > > > > Because of the size of this (which is, unfortunately,  
> > > necessary),  
> > > > > > > > > > > posting here is probably not the most efficient way.  
> That  
> > > is also  
> > > > > > > > > true  
> > > > > > > > > > > because certain people chose to use this PR as a forum  
> to  
> > > air  
> > > > > > other  
> > > > > > > > > > > issues. Therefore, it would be better if reviewers  
> e-mail  
> > > me  
> > > > > > > > > directly.  
> > > > > > > > >  
> > > > > > > > >  
> > > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > >  
> > >  
>  

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Thanks Amos, Roman, Cos for clarifying license issue.

I'm convinced that this license issue will not be a blocker.

In my understanding, these are good sign,

1. any gpl licensed source codes are not included in the source package
2. any gpl licensed libraries are not included in the binary package

However, i can not still 100% sure about

3. any gpl licensed libraries are not linked on runtime

Even after Amos's explanation. I still think using 'knitr' is one of the
clear case that 'knir' is linked to 'R' according to
http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL.

Giving input and getting output from GPL licensed interpreter (includes R)
from Apache licensed software is not a problem. That's not the point.
Let me say in this way. There's an java code,

import com.mysql.jdbc.Driver
Driver driver = new Driver()

Say without this java code, one of important feature of Zeppelin does not
work. And Zeppelin does not includes GPL licensed file in the source
package, GPL licensed library in the binary package, but it requires GPL
licensed library on the runtime.
In this case, will this java code be a license problem or not?

In other words, my question is

a) Is runtime GPL library dependency allowed in ASF release?
b) is 'knitr' considered as runtime dependency?

If someone can clarify a), b), then it would be extremely helpful
understanding this case, and possible similar cases, too.

Thanks,
moon

On Mon, Dec 7, 2015 at 3:20 PM Konstantin Boudnik <co...@apache.org> wrote:

> On Thu, Dec 03, 2015 at 11:03AM, Corneau Damien wrote:
> > Thanks Cos for those answers,
> >
> > If I'm right you are advising to have a default build that doesn't
> include
> > libraries with conflicting licenses, but have an option to include them
> for
> > users who wants to build the project themselves.
>
> Yes, that's what I said. Besides, looks like Roman provided the second
> pair of
> eyes to this licensing discussion and as well didn't find any issues with
> the
> current approach.
>
> Cos
>
> > To refer to another thread about decentralizing interpreters, it could
> even
> > be better in that case to have some interpreters separated from the tree,
> > and easily pluggable with a release instead of forcing users to build the
> > project to use them.
> >
> >
> > On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <co...@apache.org>
> wrote:
> >
> > > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:
> > > > Konstantin thank you for getting into this.
> > > >
> > > >> The best way to go around it is by
> > > >> providing a build-time option that will pull such binaries in. But
> by
> > > default
> > > >> such libs shouldn't be pulled.
> > > >
> > > > That is basically how the PR handles this. If the GPL’d interpreter
> > > scripts
> > > > are missing, there’s no indication at all at build time.  It doesn’t
> try
> > > to
> > > > download them.
> > > >
> > > > At runtime, if the user tries to use functionality that would need
> such a
> > > > script (i.e., if they type “knitr” to use knitr), we display an error
> > > that
> > > > says that the functionality is not there because the library is
> missing,
> > > and
> > > > that the library cannot be provided because it has an incompatible
> > > license,
> > > > but the user can download it themselves if they want.
> > > >
> > > > And, in the log, if the logging level is high, they will see a note
> that
> > > > some functionality was disabled because the libraries weren’t there.
> > > >
> > > > To be clear, though, none of these libraries are binaries.  They’re
> all
> > > interpreter scripts.
> > >
> > > If you ever in a doubt of which licenses could be used for dependncies
> > > (not to
> > > say about source code) are listed in Category A list of [1]
> > >
> > > A lot of quesitons discussed here are already covered in the legal
> FAQ, so
> > > just check against it if you have any questions.
> > >
> > > [1] http://www.apache.org/legal/resolved.html#category-a
> > >
> > > Cos
> > >
> > > > From: Konstantin Boudnik <co...@apache.org>
> > > > Reply: dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org>, dev@zeppelin.incubator.apache.org
> <
> > > dev@zeppelin.incubator.apache.org>
> > > > Date: December 2, 2015 at 3:24:50 PM
> > > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org
> > > >
> > > > Subject:  Re: License of KnitRInterpreter Was: Re: contributions
> > > impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter
> for
> > > Zeppelin
> > > >
> > > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:
> > > > > I think that what moon means is that:
> > > > > If we merge the way it is now, the KnitRInterpreter will be part
> of the
> > > > > code base, so it isn't optional at code base level.
> > > > >
> > > > > To make it even more simple:
> > > > > * If the code has the right licensing -> that code can be part of
> the
> > > > > source code, and can be including in a binary release
> > > >
> > > > We aren't concerned with binary releases - as an Apache community we
> are
> > > > voting and releasing source code. If the project wants to provide a
> > > binary
> > > > release to its users, they are better be warned about inclusion of
> non
> > > > ASL2-friendly licensed bits.
> > > >
> > > > > * If the code doesn't have the right licensing -> it can't be part
> of
> > > the
> > > > > source code, and can't be included in a binary release
> > > >
> > > > See above.
> > > >
> > > > > * If the code doesn't have the right licensing but is imported at
> build
> > > > > time (libraries for example) -> it is not in the source code, it
> can't
> > > be
> > > > > included in binary release
> > > >
> > > > That is unless a user doing it on his own. The best way to go around
> it
> > > is by
> > > > providing a build-time option that will pull such binaries in. But by
> > > default
> > > > such libs shouldn't be pulled.
> > > >
> > > > Cos
> > > >
> > > > > So in the case of licensing issues, it would need to be fully
> optional
> > > > > (user bring the interpreter in his directory and build Zeppelin
> with
> > > it)
> > > > >
> > > > >
> > > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <
> amos.elberg@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Moon let me clarify:
> > > > > >
> > > > > > Interpreted code doesn’t “link.” The wiki article actually
> explains
> > > it
> > > > > > pretty well —
> https://en.wikipedia.org/wiki/GPL_linking_exception
> > > > > > “Linking” against a library means compiling its headers into a
> > > binary, the
> > > > > > way a C compiler works. The 2008 e-mail Moon distributed, called
> > > this the
> > > > > > “interpreter exception.”
> > > > > >
> > > > > > As for whether GPL’d code is a “mandatory dependency,” if knitr
> is
> > > missing
> > > > > > the PR will compile, run and test just fine. The user is never
> > > prompted to
> > > > > > download it. The only effect is, if the user types “knitr” and
> knitr
> > > isn’t
> > > > > > there, we display an InterpreterError that knitr isn’t there.
> > > > > >
> > > > > > KnitRInterpreter is not optionally required. so it does not
> matter
> > > KnitR
> > > > > > is
> > > > > > optionally required or not.
> > > > > > Aren’t all interpreters optional? You can turn them on and off
> in the
> > > > > > config files.
> > > > > >
> > > > > > Do you mean that the KnitRInterpreter class gets compiled to
> > > bytecode even
> > > > > > if knitr is missing? So what? That isn't a mandatory dependency
> or a
> > > link.
> > > > > >
> > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > Date: December 2, 2015 at 3:18:00 AM
> > > > > > To: dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org>
> > > > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions
> > > impasse.
> > > > > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > > Zeppelin
> > > > > >
> > > > > > Let me summarize license concern about KnitRInterpreter.
> > > > > >
> > > > > >
> > > > > > Amos's interpretation.
> > > > > >
> > > > > > KnitR is optionally required by KnitRInterpreter.
> > > > > > R dependency in SparkR has no problem. So KnitR should be the
> same.
> > > > > >
> > > > > >
> > > > > > Moon's interpretation.
> > > > > >
> > > > > > KnitRInterpreter is not optionally required. so it does not
> matter
> > > KnitR is
> > > > > > optionally required or not.
> > > > > > R dependency in SparkR is exception of GPL. KnitR is not applied
> that
> > > > > > exception.
> > > > > >
> > > > > >
> > > > > > Amos, could you confirm my understanding to your interpretation
> is
> > > correct?
> > > > > > If it is not could you clarify it?
> > > > > >
> > > > > > Thanks,
> > > > > > moon
> > > > > >
> > > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <
> > > amos.elberg@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Just to put the final nail in this, I looked it up.
> > > > > > >
> > > > > > > I see no evidence of any “compiler exception.”
> > > > > > >
> > > > > > > There is an exception in the license for the runtime libraries
> > > that are
> > > > > > > bundled with GCC. See:
> > > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> > > > > > >
> > > > > > > But no “compiler exception.”
> > > > > > >
> > > > > > > In fact, I believe this is part of the reason why LLVM was
> created.
> > > > > > >
> > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > Date: December 1, 2015 at 8:16:36 PM
> > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org>
> > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > incubator-zeppelin pull
> > > > > > > request: R Interpreter for Zeppelin
> > > > > > >
> > > > > > > Is knitR is commonly considered as a interpreter/compiler? or
> is it
> > > > > > > considered as a library routine?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > moon
> > > > > > >
> > > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <
> > > amos.elberg@gmail.com>
> > > > > > > wrote:
> > > > > > > Moon - you give this as an explanation of the licensing issue:
> > > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > >
> > > > > > > According to that, there is an exception in the GPL for
> interpreter
> > > > > > > languages. As long as you don’t distribute the code, its fine
> to
> > > talk to
> > > > > > > an interpreted language.
> > > > > > >
> > > > > > > Well, if that’s the case, then the PR plainly does not have a
> > > license
> > > > > > > issue. It doesn’t distribute any GPL’d R code.
> > > > > > >
> > > > > > > I’m not sure what’s confusing about this. It seems completely
> > > > > > > straightforward.
> > > > > > >
> > > > > > > Regarding this:
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Amos Elberg
> > > > > > > Sent with Airmail
> > > > > > >
> > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > Date: December 1, 2015 at 6:48:47 PM
> > > > > > >
> > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org
> > > > > > >
> > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > incubator-zeppelin pull
> > > > > > > request: R Interpreter for Zeppelin
> > > > > > >
> > > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <
> > > amos.elberg@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I am going to try to minimize my reaction to Moon’s e-mail.
> > > > > > > >
> > > > > > > > The tl;dr is this:
> > > > > > > >
> > > > > > > > The reason we are having this discussion now is that active
> > > users of
> > > > > > the
> > > > > > > > PR — which now has its own user base — went public to
> complain
> > > about
> > > > > > > this.
> > > > > > >
> > > > > > >
> > > > > > > > The PR has been tested by an active user base for more than
> three
> > > > > > months.
> > > > > > > > No-one has been able to identify any specific actual
> licensing
> > > problem,
> > > > > > > and
> > > > > > > > the PR was prepared based on an extensive, careful review of
> the
> > > > > > relevant
> > > > > > > > licensing issues and after contacting the relevant people.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > I admire every software that used by user and helping people.
> That
> > > > > > includes
> > > > > > > your work. But that's not the topic we're in discussion. Active
> > > user does
> > > > > > > not mean your contribution can ignore the review.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > It is not an explanation for someone who has been ignoring my
> > > “how can
> > > > > > I
> > > > > > > > move this forward…” emails for three months to point the
> finger
> > > and
> > > > > > say I
> > > > > > > > didn’t contact the right person or file the right report.
> > > > > > > >
> > > > > > > >
> > > > > > > This is also not the topic in this discussion.
> > > > > > >
> > > > > > >
> > > > > > > > The burden for providing an explanation for the inaction is
> on
> > > the PMCC
> > > > > > > at
> > > > > > > > this point.
> > > > > > >
> > > > > > > I'm sorry, but the other PRs are passing CI. If it's problem on
> > > Zeppelin
> > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > > They’re not! I often see comments on PRs to just ignore that
> CI
> > > is
> > > > > > > > failing.
> > > > > > > >
> > > > > > > > One of the most common reasons this PR fails CI, is CI
> times-out
> > > > > > > > downloading Spark to install. How could that possibly be
> caused
> > > by the
> > > > > > > PR?
> > > > > > > >
> > > > > > > > It looks to me like the only PRs with changes to the relevant
> > > parts of
> > > > > > > the
> > > > > > > > code — the SparkInterpreter — are being made by the person
> who
> > > wrote
> > > > > > the
> > > > > > > > testing suite.
> > > > > > > >
> > > > > > > > So, that would explain why some other PRs pass CI: Neither
> the
> > > > > > > > SparkInterpreter nor the testing suite are stable or robust,
> but
> > > since
> > > > > > > the
> > > > > > > > PRs are coming from the person who wrote both…
> > > > > > > >
> > > > > > > > And let's say Zeppelin core has problem and that makes your
> PR
> > > fails on
> > > > > > > CI
> > > > > > > > test. That's possible. But it still does not mean we can
> merge
> > > it with
> > > > > > CI
> > > > > > > > failing.
> > > > > > > >
> > > > > > > > It means you should be working with me to figure out why the
> CI
> > > is
> > > > > > > failing.
> > > > > > > >
> > > > > > > > This PR has been tested by an active user base for the past
> three
> > > > > > months.
> > > > > > > > If CI is continuing to fail, and dozens of hours of effort
> have
> > > not
> > > > > > > > resolved the CI issues, then it is time to start considering
> > > whether
> > > > > > the
> > > > > > > > testing suite is part of the problem.
> > > > > > > >
> > > > > > > > The level of defensiveness about the CI and SparkInterpreter
> is
> > > not
> > > > > > > > helping to resolve these issues.
> > > > > > > >
> > > > > > > > If you think it's problem on Zeppelin core, then file an
> issue
> > > that
> > > > > > > > reproduce the problem on Zeppelin core, that might be more
> > > efficient
> > > > > > than
> > > > > > > > keep trying yourself.
> > > > > > > > I contacted you numerous times about such issues...
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I remember i commented your issue about CI. but you just keep
> > > repeated
> > > > > > it's
> > > > > > > not your problem but Zeppelin core problem.
> > > > > > >
> > > > > > > Then please file an issue about the problem you found in
> Zeppelin
> > > Core.
> > > > > > > Then everyone will get into the problem.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > In my interpretation, KnitRInterpreter is not an optional
> > > feature while
> > > > > > > it
> > > > > > > > is always enabled when compiling Zeppelin and always enabled
> when
> > > > > > running
> > > > > > > > Zeppelin. And it requires dynamically linked GPL library on
> > > runtime.
> > > > > > (yes
> > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > >
> > > > > > > > Its not always enabled.
> > > > > > > > It is not dynamically linked at runtime.
> > > > > > > > It will not fail when knitr is missing. If knitr is not
> present,
> > > the
> > > > > > repl
> > > > > > > > interpreter starts and a note is written to to the log that
> the
> > > knitr
> > > > > > > > interpreter isn’t available because knitr is not present.
> > > > > > > >
> > > > > > > > no Apache code can ever call a shell script, on the purpose
> of
> > > dynamic
> > > > > > > > linking with GPL library.
> > > > > > > > You misunderstand.
> > > > > > > >
> > > > > > > > The *shell* is GPL'd.
> > > > > > > >
> > > > > > > > Is Zeppelin “linked" against the GPL’d shell because Zeppelin
> > > depends
> > > > > > on
> > > > > > > a
> > > > > > > > shell script to launch?
> > > > > > > >
> > > > > > > Obviously not.
> > > > > > > >
> > > > > > > > The interaction with R in the PR is the same.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > Again, bash is one of exceptions of GPL, like other GPL
> licensed
> > > > > > > compiler/interpreter.
> > > > > > >
> > > > > > > Check here why Bash and R is okay with Apache License.
> > > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > > >
> > > > > > > I'm not sure we can apply the same exception for 'using' KnitR.
> > > > > > >
> > > > > > > My point is not 'KnitR' is optional or not. Point is
> > > 'KnitRInterpreter'
> > > > > > you
> > > > > > > wrote is not an optional feature. Which is clearly not
> optionally
> > > enabled
> > > > > > > code and feature. And that depends on KnitR library which is
> GPL.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > I was guessing SparkR can be still in Apache License even if
> it
> > > is
> > > > > > > depends
> > > > > > > > on R. Because of GPL licensed compiler generated output is
> not
> > > GPL
> > > > > > > license.
> > > > > > > > and R is sort of compiler. If you can get answer from Spark
> > > community
> > > > > > how
> > > > > > > > SparkR get managed to stay in Apache License while R is GPL,
> the
> > > answer
> > > > > > > > might help.
> > > > > > > > The description of SparkR is not accurate in any respect. (Do
> > > you think
> > > > > > > > SparkR is not talking to GPL-licensed libraries?)
> > > > > > > >
> > > > > > > > I don’t see that any genuine issue is being raised here.
> > > > > > > >
> > > > > > > > If there is an issue, the burden is on you to identify it.
> > > > > > > >
> > > > > > > > If i give you one suggestion, Zeppelin committers sometimes
> ask
> > > rebase
> > > > > > > the
> > > > > > > > contribution branch for some reason. It is not the really the
> > > best
> > > > > > > > practice, but still okay while most contributions are not
> > > including
> > > > > > large
> > > > > > > > code base changes
> > > > > > > > However, your one, has more than 4000 lines of code change.
> If
> > > you
> > > > > > rebase
> > > > > > > > then review should be started from the beginning, again. So
> you
> > > might
> > > > > > > want
> > > > > > > > to minimize the rebase your branch.
> > > > > > > >
> > > > > > > > Are you actually complaining that the problem is that I
> rebased
> > > the
> > > > > > code
> > > > > > > > during the three-month period when no-one looked at it and
> > > Zeppelin
> > > > > > went
> > > > > > > > through a release?
> > > > > > > >
> > > > > > > > I cannot take it seriously when you say things like this.
> > > > > > > >
> > > > > > > > Having to “start from the beginning” cannot be a problem if
> you
> > > never
> > > > > > > > actually started the first time...
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > You wanted coordination and cooperation. So i gave you
> suggestion
> > > that
> > > > > > > helping review process. For example, your code has been rebased
> > > since my
> > > > > > > comment and jongyoul's comment. that means committers will
> need to
> > > look
> > > > > > > from the beginning. That'll require more time. And maybe, i
> guess
> > > that's
> > > > > > > not what you want. But If you don't care, feel free to rebase.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > moon
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > > Date: December 1, 2015 at 4:42:06 AM
> > > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > > dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org>
> > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > incubator-zeppelin
> > > > > > pull
> > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > >
> > > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <
> > > amos.elberg@gmail.com>
> > > > > > > > wrote:
> > > > > > > > Thank you, Cos.
> > > > > > > >
> > > > > > > > I’d like to briefly address the issues raised by Moon:
> > > > > > > >
> > > > > > > > 1. This PR does not passes CI
> > > > > > > > The CI fails on core Zeppelin, *not* code in this PR.
> > > > > > > >
> > > > > > > > I’ve been seeking assistance on this since August.
> > > > > > > >
> > > > > > > > The most common reason is that SparkInterpreter is unable to
> > > launch
> > > > > > Spark
> > > > > > > > and open a Spark Backend. This is necessary to test the PR.
> > > > > > > >
> > > > > > > > 60+ hours, has been spent adapting and re-basing when the
> > > > > > > SparkInterpreter
> > > > > > > > architecture changed and broke the PR’s *tests.*
> > > > > > > >
> > > > > > > >
> > > > > > > > I'm sorry, but the other PRs are passing CI. If it's problem
> on
> > > > > > Zeppelin
> > > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > >
> > > > > > > > And let's say Zeppelin core has problem and that makes your
> PR
> > > fails on
> > > > > > > CI
> > > > > > > > test. That's possible. But it still does not mean we can
> merge
> > > it with
> > > > > > CI
> > > > > > > > failing.
> > > > > > > >
> > > > > > > > If you think it's problem on Zeppelin core, then file an
> issue
> > > that
> > > > > > > > reproduce the problem on Zeppelin core, that might be more
> > > efficient
> > > > > > than
> > > > > > > > keep trying yourself.
> > > > > > > >
> > > > > > > >
> > > > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > > > > > > What license problem *specifically* do you believe may exist?
> > > > > > > >
> > > > > > > > In preparing the PR, I:
> > > > > > > >
> > > > > > > > * Reviewed the Apache policy in detail.
> > > > > > > >
> > > > > > > > * Contacted the FSF to ask their interpretation of the
> “linking”
> > > > > > > > provisions of the GPL license.
> > > > > > > >
> > > > > > > > * Reviewed how other Apache software deals with this issue
> > > (e.g., Spark
> > > > > > > > talks to R, which is GPL'd).
> > > > > > > >
> > > > > > > > * No necessary *dependencies* of the PR have license
> conflicts.
> > > In
> > > > > > > > several cases, I contacted package authors who agreed to
> > > re-issue their
> > > > > > > > packages under Apache-compatible licenses. (Usually I had to
> do
> > > a bit
> > > > > > of
> > > > > > > > coding in exchange…)
> > > > > > > >
> > > > > > > > * Where the license had to stay GPL, the packages are *not
> > > necessary*
> > > > > > and
> > > > > > > > *not dependencies.* If the optional packages are present,
> they
> > > will be
> > > > > > > > used to enable additional functionality. Knitr is an example.
> > > The PR
> > > > > > will
> > > > > > > > compile and run fine without knitr. If knitr is available
> (it is
> > > part
> > > > > > of
> > > > > > > > the most common R distribution), the PR will enable the knitr
> > > > > > > interpreter.
> > > > > > > >
> > > > > > > > * This is exactly how this issue is addressed through the
> Apache
> > > > > > > > ecosystem.
> > > > > > > > The tl;dr is this: When Apache code is written to talk to
> > > libraries
> > > > > > that
> > > > > > > > may or may not be present on the user’s system, or where it
> > > talks to an
> > > > > > > API
> > > > > > > > but is agnostic about implementation, that is not “linking”
> in a
> > > way
> > > > > > that
> > > > > > > > implicate the anti-linking provision of the GPL.
> > > > > > > >
> > > > > > > > Otherwise, no Apache code could ever call a shell script! Let
> > > alone run
> > > > > > > > on Linux, or talk to R.
> > > > > > > >
> > > > > > > >
> > > > > > > > I'm not a legal expert. So following could be wrong.
> > > > > > > >
> > > > > > > > In my interpretation, KnitRInterpreter is not an optional
> > > feature while
> > > > > > > it
> > > > > > > > is always enabled when compiling Zeppelin and always enabled
> when
> > > > > > running
> > > > > > > > Zeppelin. And it requires dynamically linked GPL library on
> > > runtime.
> > > > > > (yes
> > > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > > >
> > > > > > > > And of course, no Apache code can ever call a shell script,
> on
> > > the
> > > > > > > purpose
> > > > > > > > of dynamic linking with GPL library.
> > > > > > > >
> > > > > > > > I was guessing SparkR can be still in Apache License even if
> it
> > > is
> > > > > > > depends
> > > > > > > > on R. Because of GPL licensed compiler generated output is
> not
> > > GPL
> > > > > > > license.
> > > > > > > > and R is sort of compiler.
> > > > > > > >
> > > > > > > > If you can get answer from Spark community how SparkR get
> > > managed to
> > > > > > stay
> > > > > > > > in Apache License while R is GPL, the answer might help.
> > > > > > > >
> > > > > > > >
> > > > > > > > 3. Need more time to review.
> > > > > > > > Has any reviewer has downloaded the PR or run the demo
> notebook?
> > > (Which
> > > > > > > > is there for the benefit of reviewers, and isn’t intended to
> go
> > > in
> > > > > > final
> > > > > > > > distribution.)
> > > > > > > >
> > > > > > > > How many +1 comments from users would you like to see?
> > > > > > > >
> > > > > > > > How much time do you believe is required?
> > > > > > > >
> > > > > > > >
> > > > > > > > It all depends on when CI is going to pass, when license
> problem
> > > is
> > > > > > going
> > > > > > > > to be cleared, and when a committer willing to review and
> > > responsible
> > > > > > to
> > > > > > > > commit your contribution.
> > > > > > > >
> > > > > > > >
> > > > > > > > 1. Large code base change
> > > > > > > > Large code base changes require coordination and cooperation.
> > > This PR
> > > > > > > > necessarily implicates the build scripts, testing code, the
> > > > > > > > SparkInterpreter, etc.
> > > > > > > >
> > > > > > > > I have been seeking to coordinate since August.
> > > > > > > >
> > > > > > > > I continue to stand ready to do so.
> > > > > > > >
> > > > > > > > -Amos
> > > > > > > >
> > > > > > > >
> > > > > > > > If i give you one suggestion, Zeppelin committers sometimes
> ask
> > > rebase
> > > > > > > the
> > > > > > > > contribution branch for some reason. It is not the really the
> > > best
> > > > > > > > practice, but still okay while most contributions are not
> > > including
> > > > > > large
> > > > > > > > code base changes.
> > > > > > > >
> > > > > > > > However, your one, has more than 4000 lines of code change.
> If
> > > you
> > > > > > rebase
> > > > > > > > then review should be started from the beginning, again. So
> you
> > > might
> > > > > > > want
> > > > > > > > to minimize the rebase your branch.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > moon
> > > > > > > >
> > > > > > > >
> > > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > > Date: December 1, 2015 at 1:34:19 AM
> > > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org
> > > > > > > >
> > > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > > incubator-zeppelin
> > > > > > pull
> > > > > > > > request: R Interpreter for Zeppelin
> > > > > > > >
> > > > > > > > Hi Cos,
> > > > > > > >
> > > > > > > > Thanks for opening a discussion.
> > > > > > > > My answer about 'Why this PR is open for three months' is
> > > > > > > >
> > > > > > > > 1. This PR does not passes CI
> > > > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > > > > > > 3. Need more time to review.
> > > > > > > >
> > > > > > > > It's my personal answer, other committers may have different
> > > opinion.
> > > > > > > >
> > > > > > > >
> > > > > > > > I think the question should be generalized. Because this PR
> is
> > > not the
> > > > > > > only
> > > > > > > > PR that is in impasse. There're more. For example
> > > > > > > >
> > > > > > > > Here's some examples that PR is not been merged.
> > > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,
> > > > > > > > https://github.com/apache/incubator-zeppelin/pull/60
> > > > > > > > and so on.
> > > > > > > >
> > > > > > > > I can categorize the cases, based on experience of involving
> > > Zeppelin
> > > > > > > > community from the beginning.
> > > > > > > >
> > > > > > > > 1. Large code base change
> > > > > > > >
> > > > > > > > When contribution has large code base changes, it tend to
> take
> > > more
> > > > > > time
> > > > > > > to
> > > > > > > > review and merged. Normally, most contributions merged in
> 1day~1
> > > week.
> > > > > > > > But some contribution has large code base changes take 2~4
> > > weeks. Few
> > > > > > > > contribution that has very large code base change take
> months.
> > > > > > > >
> > > > > > > > 2. Communication lost
> > > > > > > >
> > > > > > > > Sometimes, committer is not responding, or contributor is not
> > > > > > responding.
> > > > > > > >
> > > > > > > > 3. Opinion diverges
> > > > > > > >
> > > > > > > > For some changes, included in contribution. When people have
> > > different
> > > > > > > > opinion and it continue to diverges, those PRs are not been
> > > merged.
> > > > > > > >
> > > > > > > >
> > > > > > > > I think having a guide such as ping other committer when a
> > > committer is
> > > > > > > not
> > > > > > > > responding, and divide contribution into small peaces if
> > > possible,
> > > > > > would
> > > > > > > > help most of the cases.
> > > > > > > > Of course committer have to pay attention more to the
> > > contribution and
> > > > > > > > helping people. That's the first one.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > moon
> > > > > > > >
> > > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <
> > > cos@apache.org>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > To make sure we're on the same page, here are two threads
> that
> > > I
> > > > > > found
> > > > > > > > > related
> > > > > > > > > to this topic.
> > > > > > > > >
> > > > > > > > > Thread 1:
> > > > > > > > > Subject: R?
> > > > > > > > > Started on: July 1, 2015
> > > > > > > > >
> > > > > > > > > Thread 2:
> > > > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R
> > > Interpreter for
> > > > > > > > > Zeppelin
> > > > > > > > > Started on: August 13, 2015
> > > > > > > > >
> > > > > > > > > If you want to fetch these from our archive send emails to
> > > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > > > > > > >
> > > > > > > > > Cos
> > > > > > > > >
> > > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > > > > > > > > Guys,
> > > > > > > > > >
> > > > > > > > > > While catching up on my emails from the last a couple of
> > > weeks,
> > > > > > this
> > > > > > > > > thread
> > > > > > > > > > caught my attention. I am not normally paying much
> attention
> > > to the
> > > > > > > > code
> > > > > > > > > > reviews traffic from GH, but this one is pretty
> different as
> > > it
> > > > > > spans
> > > > > > > > > three
> > > > > > > > > > months and counting.
> > > > > > > > > >
> > > > > > > > > > First, here are my five cents:
> > > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed
> to be
> > > > > > > > > contributed to
> > > > > > > > > > an ASF project this file should simply contain ASL2 text,
> > > like in
> > > > > > [1]
> > > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate
> <developers>
> > > > > > > section,
> > > > > > > > > but
> > > > > > > > > > Zeppelin might have different guidelines on it. Say,
> Bigtop
> > > doesn't
> > > > > > > > > > maintain this in the source code, but we have the list of
> > > all the
> > > > > > > > > > committers on the project's site [2] Every new
> committer's
> > > first
> > > > > > > > > commit is
> > > > > > > > > > to update the team page ;)
> > > > > > > > > > - comments like in
> > > > > > > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > > > > > > >
> > > > > > > > > > +/**
> > > > > > > > > > + * Created by aelberg on 7/28/15.
> > > > > > > > > > + */
> > > > > > > > > >
> > > > > > > > > > is better to be removed. It has been already discussed
> here
> > > [3].
> > > > > > And
> > > > > > > > > the
> > > > > > > > > > initial discussion on the topic [4] was linked as well
> > > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure
> if
> > > this is
> > > > > > > > > R-specific
> > > > > > > > > > stuff - I have no idea about R, honestly, but
> > > > > > > > > >
> > > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > > > > > > > ...
> > > > > > > > > > +Author: David B. Dahl
> > > > > > > > > >
> > > > > > > > > > shouldn't be here, IMO. Normally, if some additional
> > > licenses are
> > > > > > > > > used,
> > > > > > > > > > they have to be listed in the top-level NOTICE file
> (already
> > > > > > there).
> > > > > > > > > >
> > > > > > > > > > - I am not going to offer any comments on the technical
> > > merits of
> > > > > > the
> > > > > > > > > patch,
> > > > > > > > > > as I haven't tried it personally. However, I don't see
> any
> > > serious
> > > > > > > > > > technical objections to the functionality in question.
> > > > > > > > > >
> > > > > > > > > > So, the question is - why the PR is open for three
> months? I
> > > hasn't
> > > > > > > > been
> > > > > > > > > able
> > > > > > > > > > to get a clear answer. What I found out though is pretty
> > > > > > unsettling,
> > > > > > > > > really.
> > > > > > > > > > The communication on the topic is almost non-existent,
> > > except for
> > > > > > > this
> > > > > > > > > sparse
> > > > > > > > > > and bitter thread in the GH.
> > > > > > > > > >
> > > > > > > > > > I would love to hear from the committers what's stopping
> the
> > > > > > > acceptance
> > > > > > > > > of the
> > > > > > > > > > code, besides of the minor issues I've mentioned earlier?
> > > What are
> > > > > > > the
> > > > > > > > > reasons for it?
> > > > > > > > > > Is there anything wrong with it?
> > > > > > > > > > One of the responsibilities of the committers is to make
> > > sure the
> > > > > > > > > > contributions are reviewed; new contributors are guided
> and
> > > do
> > > > > > > > > understand how
> > > > > > > > > > the project ticks. The easy feedback flow attracts new
> > > people,
> > > > > > > allowing
> > > > > > > > > the
> > > > > > > > > > community to grow and thrive.
> > > > > > > > > >
> > > > > > > > > > I am asking _explicitely_ not to re-start the bickering I
> > > have
> > > > > > > already
> > > > > > > > > > seen. At this point I am interested in the purely
> technical
> > > side of
> > > > > > > > this.
> > > > > > > > > >
> > > > > > > > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > > > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > > > > > > [3]
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > > > > > > [4] http://s.apache.org/iZl
> > > > > > > > > >
> > > > > > > > > > With regards,
> > > > > > > > > > Cos
> > > > > > > > > >
> > > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > > > > > > Github user elbamos commented on the pull request:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > > > > > > >
> > > > > > > > > > > The current push should resolve some issues with
> changes
> > > in the
> > > > > > > > > > > Spark-Zeppelin interface that had created issues for
> > > users, as
> > > > > > > > > well as
> > > > > > > > > > > support for 1.5.1.
> > > > > > > > > > >
> > > > > > > > > > > Knitr support is improved, and the reason for a
> separate
> > > knitr
> > > > > > > > > interpreter may be clearer now.
> > > > > > > > > > >
> > > > > > > > > > > The amount of code borrowed from rscala is reduced.
> > > > > > > > > > >
> > > > > > > > > > > I did not address issues with @author tags, or files
> under
> > > the R/
> > > > > > > > > > > folder. The reason is, to be blunt, I don't understand
> > > what the
> > > > > > > > > precise
> > > > > > > > > > > concerns actually are.
> > > > > > > > > > >
> > > > > > > > > > > Please note that I changed .travis.yml to only use
> spark
> > > 1.4 and
> > > > > > > > > later.
> > > > > > > > > > > I'm sure there is a better way to do it, but I'm just
> not
> > > in a
> > > > > > > > > position
> > > > > > > > > > > to try to figure out the entire Zeppelin build process.
> > > > > > > > > > >
> > > > > > > > > > > Integrating this with the rest of zeppelin is going to
> > > take some
> > > > > > > > > work
> > > > > > > > > > > regarding pom's, travis, and so forth. I can do a lot
> of
> > > that,
> > > > > > > > > but I'm
> > > > > > > > > > > going to need to discuss it with the people who have
> been
> > > > > > "owning"
> > > > > > > > > those
> > > > > > > > > > > files. There are just too many moving pieces here.
> > > > > > > > > > >
> > > > > > > > > > > Because of the size of this (which is, unfortunately,
> > > necessary),
> > > > > > > > > > > posting here is probably not the most efficient way.
> That
> > > is also
> > > > > > > > > true
> > > > > > > > > > > because certain people chose to use this PR as a forum
> to
> > > air
> > > > > > other
> > > > > > > > > > > issues. Therefore, it would be better if reviewers
> e-mail
> > > me
> > > > > > > > > directly.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > >
>

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Konstantin Boudnik <co...@apache.org>]

On Thu, Dec 03, 2015 at 11:03AM, Corneau Damien wrote:
> Thanks Cos for those answers,
> 
> If I'm right you are advising to have a default build that doesn't include
> libraries with conflicting licenses, but have an option to include them for
> users who wants to build the project themselves.

Yes, that's what I said. Besides, looks like Roman provided the second pair of
eyes to this licensing discussion and as well didn't find any issues with the
current approach.

Cos

> To refer to another thread about decentralizing interpreters, it could even
> be better in that case to have some interpreters separated from the tree,
> and easily pluggable with a release instead of forcing users to build the
> project to use them.
> 
> 
> On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <co...@apache.org> wrote:
> 
> > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:
> > > Konstantin thank you for getting into this.
> > >
> > >> The best way to go around it is by
> > >> providing a build-time option that will pull such binaries in. But by
> > default
> > >> such libs shouldn't be pulled.
> > >
> > > That is basically how the PR handles this. If the GPL’d interpreter
> > scripts
> > > are missing, there’s no indication at all at build time.  It doesn’t try
> > to
> > > download them.
> > >
> > > At runtime, if the user tries to use functionality that would need such a
> > > script (i.e., if they type “knitr” to use knitr), we display an error
> > that
> > > says that the functionality is not there because the library is missing,
> > and
> > > that the library cannot be provided because it has an incompatible
> > license,
> > > but the user can download it themselves if they want.
> > >
> > > And, in the log, if the logging level is high, they will see a note that
> > > some functionality was disabled because the libraries weren’t there.
> > >
> > > To be clear, though, none of these libraries are binaries.  They’re all
> > interpreter scripts.
> >
> > If you ever in a doubt of which licenses could be used for dependncies
> > (not to
> > say about source code) are listed in Category A list of [1]
> >
> > A lot of quesitons discussed here are already covered in the legal FAQ, so
> > just check against it if you have any questions.
> >
> > [1] http://www.apache.org/legal/resolved.html#category-a
> >
> > Cos
> >
> > > From: Konstantin Boudnik <co...@apache.org>
> > > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>, dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > > Date: December 2, 2015 at 3:24:50 PM
> > > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> > >
> > > Subject:  Re: License of KnitRInterpreter Was: Re: contributions
> > impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > Zeppelin
> > >
> > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:
> > > > I think that what moon means is that:
> > > > If we merge the way it is now, the KnitRInterpreter will be part of the
> > > > code base, so it isn't optional at code base level.
> > > >
> > > > To make it even more simple:
> > > > * If the code has the right licensing -> that code can be part of the
> > > > source code, and can be including in a binary release
> > >
> > > We aren't concerned with binary releases - as an Apache community we are
> > > voting and releasing source code. If the project wants to provide a
> > binary
> > > release to its users, they are better be warned about inclusion of non
> > > ASL2-friendly licensed bits.
> > >
> > > > * If the code doesn't have the right licensing -> it can't be part of
> > the
> > > > source code, and can't be included in a binary release
> > >
> > > See above.
> > >
> > > > * If the code doesn't have the right licensing but is imported at build
> > > > time (libraries for example) -> it is not in the source code, it can't
> > be
> > > > included in binary release
> > >
> > > That is unless a user doing it on his own. The best way to go around it
> > is by
> > > providing a build-time option that will pull such binaries in. But by
> > default
> > > such libs shouldn't be pulled.
> > >
> > > Cos
> > >
> > > > So in the case of licensing issues, it would need to be fully optional
> > > > (user bring the interpreter in his directory and build Zeppelin with
> > it)
> > > >
> > > >
> > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <am...@gmail.com>
> > > > wrote:
> > > >
> > > > > Moon let me clarify:
> > > > >
> > > > > Interpreted code doesn’t “link.” The wiki article actually explains
> > it
> > > > > pretty well — https://en.wikipedia.org/wiki/GPL_linking_exception
> > > > > “Linking” against a library means compiling its headers into a
> > binary, the
> > > > > way a C compiler works. The 2008 e-mail Moon distributed, called
> > this the
> > > > > “interpreter exception.”
> > > > >
> > > > > As for whether GPL’d code is a “mandatory dependency,” if knitr is
> > missing
> > > > > the PR will compile, run and test just fine. The user is never
> > prompted to
> > > > > download it. The only effect is, if the user types “knitr” and knitr
> > isn’t
> > > > > there, we display an InterpreterError that knitr isn’t there.
> > > > >
> > > > > KnitRInterpreter is not optionally required. so it does not matter
> > KnitR
> > > > > is
> > > > > optionally required or not.
> > > > > Aren’t all interpreters optional? You can turn them on and off in the
> > > > > config files.
> > > > >
> > > > > Do you mean that the KnitRInterpreter class gets compiled to
> > bytecode even
> > > > > if knitr is missing? So what? That isn't a mandatory dependency or a
> > link.
> > > > >
> > > > > From: moon soo Lee <mo...@apache.org>
> > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > dev@zeppelin.incubator.apache.org>
> > > > > Date: December 2, 2015 at 3:18:00 AM
> > > > > To: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions
> > impasse.
> > > > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > Zeppelin
> > > > >
> > > > > Let me summarize license concern about KnitRInterpreter.
> > > > >
> > > > >
> > > > > Amos's interpretation.
> > > > >
> > > > > KnitR is optionally required by KnitRInterpreter.
> > > > > R dependency in SparkR has no problem. So KnitR should be the same.
> > > > >
> > > > >
> > > > > Moon's interpretation.
> > > > >
> > > > > KnitRInterpreter is not optionally required. so it does not matter
> > KnitR is
> > > > > optionally required or not.
> > > > > R dependency in SparkR is exception of GPL. KnitR is not applied that
> > > > > exception.
> > > > >
> > > > >
> > > > > Amos, could you confirm my understanding to your interpretation is
> > correct?
> > > > > If it is not could you clarify it?
> > > > >
> > > > > Thanks,
> > > > > moon
> > > > >
> > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <
> > amos.elberg@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Just to put the final nail in this, I looked it up.
> > > > > >
> > > > > > I see no evidence of any “compiler exception.”
> > > > > >
> > > > > > There is an exception in the license for the runtime libraries
> > that are
> > > > > > bundled with GCC. See:
> > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> > > > > >
> > > > > > But no “compiler exception.”
> > > > > >
> > > > > > In fact, I believe this is part of the reason why LLVM was created.
> > > > > >
> > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > Date: December 1, 2015 at 8:16:36 PM
> > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > incubator-zeppelin pull
> > > > > > request: R Interpreter for Zeppelin
> > > > > >
> > > > > > Is knitR is commonly considered as a interpreter/compiler? or is it
> > > > > > considered as a library routine?
> > > > > >
> > > > > > Thanks,
> > > > > > moon
> > > > > >
> > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <
> > amos.elberg@gmail.com>
> > > > > > wrote:
> > > > > > Moon - you give this as an explanation of the licensing issue:
> > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > >
> > > > > > According to that, there is an exception in the GPL for interpreter
> > > > > > languages. As long as you don’t distribute the code, its fine to
> > talk to
> > > > > > an interpreted language.
> > > > > >
> > > > > > Well, if that’s the case, then the PR plainly does not have a
> > license
> > > > > > issue. It doesn’t distribute any GPL’d R code.
> > > > > >
> > > > > > I’m not sure what’s confusing about this. It seems completely
> > > > > > straightforward.
> > > > > >
> > > > > > Regarding this:
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Amos Elberg
> > > > > > Sent with Airmail
> > > > > >
> > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > Date: December 1, 2015 at 6:48:47 PM
> > > > > >
> > > > > > To: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org
> > > > > >
> > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > incubator-zeppelin pull
> > > > > > request: R Interpreter for Zeppelin
> > > > > >
> > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <
> > amos.elberg@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > I am going to try to minimize my reaction to Moon’s e-mail.
> > > > > > >
> > > > > > > The tl;dr is this:
> > > > > > >
> > > > > > > The reason we are having this discussion now is that active
> > users of
> > > > > the
> > > > > > > PR — which now has its own user base — went public to complain
> > about
> > > > > > this.
> > > > > >
> > > > > >
> > > > > > > The PR has been tested by an active user base for more than three
> > > > > months.
> > > > > > > No-one has been able to identify any specific actual licensing
> > problem,
> > > > > > and
> > > > > > > the PR was prepared based on an extensive, careful review of the
> > > > > relevant
> > > > > > > licensing issues and after contacting the relevant people.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > I admire every software that used by user and helping people. That
> > > > > includes
> > > > > > your work. But that's not the topic we're in discussion. Active
> > user does
> > > > > > not mean your contribution can ignore the review.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > It is not an explanation for someone who has been ignoring my
> > “how can
> > > > > I
> > > > > > > move this forward…” emails for three months to point the finger
> > and
> > > > > say I
> > > > > > > didn’t contact the right person or file the right report.
> > > > > > >
> > > > > > >
> > > > > > This is also not the topic in this discussion.
> > > > > >
> > > > > >
> > > > > > > The burden for providing an explanation for the inaction is on
> > the PMCC
> > > > > > at
> > > > > > > this point.
> > > > > >
> > > > > > I'm sorry, but the other PRs are passing CI. If it's problem on
> > Zeppelin
> > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > They’re not! I often see comments on PRs to just ignore that CI
> > is
> > > > > > > failing.
> > > > > > >
> > > > > > > One of the most common reasons this PR fails CI, is CI times-out
> > > > > > > downloading Spark to install. How could that possibly be caused
> > by the
> > > > > > PR?
> > > > > > >
> > > > > > > It looks to me like the only PRs with changes to the relevant
> > parts of
> > > > > > the
> > > > > > > code — the SparkInterpreter — are being made by the person who
> > wrote
> > > > > the
> > > > > > > testing suite.
> > > > > > >
> > > > > > > So, that would explain why some other PRs pass CI: Neither the
> > > > > > > SparkInterpreter nor the testing suite are stable or robust, but
> > since
> > > > > > the
> > > > > > > PRs are coming from the person who wrote both…
> > > > > > >
> > > > > > > And let's say Zeppelin core has problem and that makes your PR
> > fails on
> > > > > > CI
> > > > > > > test. That's possible. But it still does not mean we can merge
> > it with
> > > > > CI
> > > > > > > failing.
> > > > > > >
> > > > > > > It means you should be working with me to figure out why the CI
> > is
> > > > > > failing.
> > > > > > >
> > > > > > > This PR has been tested by an active user base for the past three
> > > > > months.
> > > > > > > If CI is continuing to fail, and dozens of hours of effort have
> > not
> > > > > > > resolved the CI issues, then it is time to start considering
> > whether
> > > > > the
> > > > > > > testing suite is part of the problem.
> > > > > > >
> > > > > > > The level of defensiveness about the CI and SparkInterpreter is
> > not
> > > > > > > helping to resolve these issues.
> > > > > > >
> > > > > > > If you think it's problem on Zeppelin core, then file an issue
> > that
> > > > > > > reproduce the problem on Zeppelin core, that might be more
> > efficient
> > > > > than
> > > > > > > keep trying yourself.
> > > > > > > I contacted you numerous times about such issues...
> > > > > > >
> > > > > >
> > > > > >
> > > > > > I remember i commented your issue about CI. but you just keep
> > repeated
> > > > > it's
> > > > > > not your problem but Zeppelin core problem.
> > > > > >
> > > > > > Then please file an issue about the problem you found in Zeppelin
> > Core.
> > > > > > Then everyone will get into the problem.
> > > > > >
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > In my interpretation, KnitRInterpreter is not an optional
> > feature while
> > > > > > it
> > > > > > > is always enabled when compiling Zeppelin and always enabled when
> > > > > running
> > > > > > > Zeppelin. And it requires dynamically linked GPL library on
> > runtime.
> > > > > (yes
> > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > >
> > > > > > > Its not always enabled.
> > > > > > > It is not dynamically linked at runtime.
> > > > > > > It will not fail when knitr is missing. If knitr is not present,
> > the
> > > > > repl
> > > > > > > interpreter starts and a note is written to to the log that the
> > knitr
> > > > > > > interpreter isn’t available because knitr is not present.
> > > > > > >
> > > > > > > no Apache code can ever call a shell script, on the purpose of
> > dynamic
> > > > > > > linking with GPL library.
> > > > > > > You misunderstand.
> > > > > > >
> > > > > > > The *shell* is GPL'd.
> > > > > > >
> > > > > > > Is Zeppelin “linked" against the GPL’d shell because Zeppelin
> > depends
> > > > > on
> > > > > > a
> > > > > > > shell script to launch?
> > > > > > >
> > > > > > Obviously not.
> > > > > > >
> > > > > > > The interaction with R in the PR is the same.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > Again, bash is one of exceptions of GPL, like other GPL licensed
> > > > > > compiler/interpreter.
> > > > > >
> > > > > > Check here why Bash and R is okay with Apache License.
> > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > >
> > > > > > I'm not sure we can apply the same exception for 'using' KnitR.
> > > > > >
> > > > > > My point is not 'KnitR' is optional or not. Point is
> > 'KnitRInterpreter'
> > > > > you
> > > > > > wrote is not an optional feature. Which is clearly not optionally
> > enabled
> > > > > > code and feature. And that depends on KnitR library which is GPL.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > I was guessing SparkR can be still in Apache License even if it
> > is
> > > > > > depends
> > > > > > > on R. Because of GPL licensed compiler generated output is not
> > GPL
> > > > > > license.
> > > > > > > and R is sort of compiler. If you can get answer from Spark
> > community
> > > > > how
> > > > > > > SparkR get managed to stay in Apache License while R is GPL, the
> > answer
> > > > > > > might help.
> > > > > > > The description of SparkR is not accurate in any respect. (Do
> > you think
> > > > > > > SparkR is not talking to GPL-licensed libraries?)
> > > > > > >
> > > > > > > I don’t see that any genuine issue is being raised here.
> > > > > > >
> > > > > > > If there is an issue, the burden is on you to identify it.
> > > > > > >
> > > > > > > If i give you one suggestion, Zeppelin committers sometimes ask
> > rebase
> > > > > > the
> > > > > > > contribution branch for some reason. It is not the really the
> > best
> > > > > > > practice, but still okay while most contributions are not
> > including
> > > > > large
> > > > > > > code base changes
> > > > > > > However, your one, has more than 4000 lines of code change. If
> > you
> > > > > rebase
> > > > > > > then review should be started from the beginning, again. So you
> > might
> > > > > > want
> > > > > > > to minimize the rebase your branch.
> > > > > > >
> > > > > > > Are you actually complaining that the problem is that I rebased
> > the
> > > > > code
> > > > > > > during the three-month period when no-one looked at it and
> > Zeppelin
> > > > > went
> > > > > > > through a release?
> > > > > > >
> > > > > > > I cannot take it seriously when you say things like this.
> > > > > > >
> > > > > > > Having to “start from the beginning” cannot be a problem if you
> > never
> > > > > > > actually started the first time...
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > You wanted coordination and cooperation. So i gave you suggestion
> > that
> > > > > > helping review process. For example, your code has been rebased
> > since my
> > > > > > comment and jongyoul's comment. that means committers will need to
> > look
> > > > > > from the beginning. That'll require more time. And maybe, i guess
> > that's
> > > > > > not what you want. But If you don't care, feel free to rebase.
> > > > > >
> > > > > > Thanks,
> > > > > > moon
> > > > > >
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > Date: December 1, 2015 at 4:42:06 AM
> > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > incubator-zeppelin
> > > > > pull
> > > > > > > request: R Interpreter for Zeppelin
> > > > > > >
> > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <
> > amos.elberg@gmail.com>
> > > > > > > wrote:
> > > > > > > Thank you, Cos.
> > > > > > >
> > > > > > > I’d like to briefly address the issues raised by Moon:
> > > > > > >
> > > > > > > 1. This PR does not passes CI
> > > > > > > The CI fails on core Zeppelin, *not* code in this PR.
> > > > > > >
> > > > > > > I’ve been seeking assistance on this since August.
> > > > > > >
> > > > > > > The most common reason is that SparkInterpreter is unable to
> > launch
> > > > > Spark
> > > > > > > and open a Spark Backend. This is necessary to test the PR.
> > > > > > >
> > > > > > > 60+ hours, has been spent adapting and re-basing when the
> > > > > > SparkInterpreter
> > > > > > > architecture changed and broke the PR’s *tests.*
> > > > > > >
> > > > > > >
> > > > > > > I'm sorry, but the other PRs are passing CI. If it's problem on
> > > > > Zeppelin
> > > > > > > core, why do you think other PRs are passing CI?
> > > > > > >
> > > > > > > And let's say Zeppelin core has problem and that makes your PR
> > fails on
> > > > > > CI
> > > > > > > test. That's possible. But it still does not mean we can merge
> > it with
> > > > > CI
> > > > > > > failing.
> > > > > > >
> > > > > > > If you think it's problem on Zeppelin core, then file an issue
> > that
> > > > > > > reproduce the problem on Zeppelin core, that might be more
> > efficient
> > > > > than
> > > > > > > keep trying yourself.
> > > > > > >
> > > > > > >
> > > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > > > > > What license problem *specifically* do you believe may exist?
> > > > > > >
> > > > > > > In preparing the PR, I:
> > > > > > >
> > > > > > > * Reviewed the Apache policy in detail.
> > > > > > >
> > > > > > > * Contacted the FSF to ask their interpretation of the “linking”
> > > > > > > provisions of the GPL license.
> > > > > > >
> > > > > > > * Reviewed how other Apache software deals with this issue
> > (e.g., Spark
> > > > > > > talks to R, which is GPL'd).
> > > > > > >
> > > > > > > * No necessary *dependencies* of the PR have license conflicts.
> > In
> > > > > > > several cases, I contacted package authors who agreed to
> > re-issue their
> > > > > > > packages under Apache-compatible licenses. (Usually I had to do
> > a bit
> > > > > of
> > > > > > > coding in exchange…)
> > > > > > >
> > > > > > > * Where the license had to stay GPL, the packages are *not
> > necessary*
> > > > > and
> > > > > > > *not dependencies.* If the optional packages are present, they
> > will be
> > > > > > > used to enable additional functionality. Knitr is an example.
> > The PR
> > > > > will
> > > > > > > compile and run fine without knitr. If knitr is available (it is
> > part
> > > > > of
> > > > > > > the most common R distribution), the PR will enable the knitr
> > > > > > interpreter.
> > > > > > >
> > > > > > > * This is exactly how this issue is addressed through the Apache
> > > > > > > ecosystem.
> > > > > > > The tl;dr is this: When Apache code is written to talk to
> > libraries
> > > > > that
> > > > > > > may or may not be present on the user’s system, or where it
> > talks to an
> > > > > > API
> > > > > > > but is agnostic about implementation, that is not “linking” in a
> > way
> > > > > that
> > > > > > > implicate the anti-linking provision of the GPL.
> > > > > > >
> > > > > > > Otherwise, no Apache code could ever call a shell script! Let
> > alone run
> > > > > > > on Linux, or talk to R.
> > > > > > >
> > > > > > >
> > > > > > > I'm not a legal expert. So following could be wrong.
> > > > > > >
> > > > > > > In my interpretation, KnitRInterpreter is not an optional
> > feature while
> > > > > > it
> > > > > > > is always enabled when compiling Zeppelin and always enabled when
> > > > > running
> > > > > > > Zeppelin. And it requires dynamically linked GPL library on
> > runtime.
> > > > > (yes
> > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > >
> > > > > > > And of course, no Apache code can ever call a shell script, on
> > the
> > > > > > purpose
> > > > > > > of dynamic linking with GPL library.
> > > > > > >
> > > > > > > I was guessing SparkR can be still in Apache License even if it
> > is
> > > > > > depends
> > > > > > > on R. Because of GPL licensed compiler generated output is not
> > GPL
> > > > > > license.
> > > > > > > and R is sort of compiler.
> > > > > > >
> > > > > > > If you can get answer from Spark community how SparkR get
> > managed to
> > > > > stay
> > > > > > > in Apache License while R is GPL, the answer might help.
> > > > > > >
> > > > > > >
> > > > > > > 3. Need more time to review.
> > > > > > > Has any reviewer has downloaded the PR or run the demo notebook?
> > (Which
> > > > > > > is there for the benefit of reviewers, and isn’t intended to go
> > in
> > > > > final
> > > > > > > distribution.)
> > > > > > >
> > > > > > > How many +1 comments from users would you like to see?
> > > > > > >
> > > > > > > How much time do you believe is required?
> > > > > > >
> > > > > > >
> > > > > > > It all depends on when CI is going to pass, when license problem
> > is
> > > > > going
> > > > > > > to be cleared, and when a committer willing to review and
> > responsible
> > > > > to
> > > > > > > commit your contribution.
> > > > > > >
> > > > > > >
> > > > > > > 1. Large code base change
> > > > > > > Large code base changes require coordination and cooperation.
> > This PR
> > > > > > > necessarily implicates the build scripts, testing code, the
> > > > > > > SparkInterpreter, etc.
> > > > > > >
> > > > > > > I have been seeking to coordinate since August.
> > > > > > >
> > > > > > > I continue to stand ready to do so.
> > > > > > >
> > > > > > > -Amos
> > > > > > >
> > > > > > >
> > > > > > > If i give you one suggestion, Zeppelin committers sometimes ask
> > rebase
> > > > > > the
> > > > > > > contribution branch for some reason. It is not the really the
> > best
> > > > > > > practice, but still okay while most contributions are not
> > including
> > > > > large
> > > > > > > code base changes.
> > > > > > >
> > > > > > > However, your one, has more than 4000 lines of code change. If
> > you
> > > > > rebase
> > > > > > > then review should be started from the beginning, again. So you
> > might
> > > > > > want
> > > > > > > to minimize the rebase your branch.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > moon
> > > > > > >
> > > > > > >
> > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > Date: December 1, 2015 at 1:34:19 AM
> > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > dev@zeppelin.incubator.apache.org
> > > > > > >
> > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > incubator-zeppelin
> > > > > pull
> > > > > > > request: R Interpreter for Zeppelin
> > > > > > >
> > > > > > > Hi Cos,
> > > > > > >
> > > > > > > Thanks for opening a discussion.
> > > > > > > My answer about 'Why this PR is open for three months' is
> > > > > > >
> > > > > > > 1. This PR does not passes CI
> > > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > > > > > 3. Need more time to review.
> > > > > > >
> > > > > > > It's my personal answer, other committers may have different
> > opinion.
> > > > > > >
> > > > > > >
> > > > > > > I think the question should be generalized. Because this PR is
> > not the
> > > > > > only
> > > > > > > PR that is in impasse. There're more. For example
> > > > > > >
> > > > > > > Here's some examples that PR is not been merged.
> > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,
> > > > > > > https://github.com/apache/incubator-zeppelin/pull/60
> > > > > > > and so on.
> > > > > > >
> > > > > > > I can categorize the cases, based on experience of involving
> > Zeppelin
> > > > > > > community from the beginning.
> > > > > > >
> > > > > > > 1. Large code base change
> > > > > > >
> > > > > > > When contribution has large code base changes, it tend to take
> > more
> > > > > time
> > > > > > to
> > > > > > > review and merged. Normally, most contributions merged in 1day~1
> > week.
> > > > > > > But some contribution has large code base changes take 2~4
> > weeks. Few
> > > > > > > contribution that has very large code base change take months.
> > > > > > >
> > > > > > > 2. Communication lost
> > > > > > >
> > > > > > > Sometimes, committer is not responding, or contributor is not
> > > > > responding.
> > > > > > >
> > > > > > > 3. Opinion diverges
> > > > > > >
> > > > > > > For some changes, included in contribution. When people have
> > different
> > > > > > > opinion and it continue to diverges, those PRs are not been
> > merged.
> > > > > > >
> > > > > > >
> > > > > > > I think having a guide such as ping other committer when a
> > committer is
> > > > > > not
> > > > > > > responding, and divide contribution into small peaces if
> > possible,
> > > > > would
> > > > > > > help most of the cases.
> > > > > > > Of course committer have to pay attention more to the
> > contribution and
> > > > > > > helping people. That's the first one.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > moon
> > > > > > >
> > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <
> > cos@apache.org>
> > > > > > wrote:
> > > > > > >
> > > > > > > > To make sure we're on the same page, here are two threads that
> > I
> > > > > found
> > > > > > > > related
> > > > > > > > to this topic.
> > > > > > > >
> > > > > > > > Thread 1:
> > > > > > > > Subject: R?
> > > > > > > > Started on: July 1, 2015
> > > > > > > >
> > > > > > > > Thread 2:
> > > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R
> > Interpreter for
> > > > > > > > Zeppelin
> > > > > > > > Started on: August 13, 2015
> > > > > > > >
> > > > > > > > If you want to fetch these from our archive send emails to
> > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > > > > > >
> > > > > > > > Cos
> > > > > > > >
> > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > > > > > > > Guys,
> > > > > > > > >
> > > > > > > > > While catching up on my emails from the last a couple of
> > weeks,
> > > > > this
> > > > > > > > thread
> > > > > > > > > caught my attention. I am not normally paying much attention
> > to the
> > > > > > > code
> > > > > > > > > reviews traffic from GH, but this one is pretty different as
> > it
> > > > > spans
> > > > > > > > three
> > > > > > > > > months and counting.
> > > > > > > > >
> > > > > > > > > First, here are my five cents:
> > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > > > > > > > contributed to
> > > > > > > > > an ASF project this file should simply contain ASL2 text,
> > like in
> > > > > [1]
> > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate <developers>
> > > > > > section,
> > > > > > > > but
> > > > > > > > > Zeppelin might have different guidelines on it. Say, Bigtop
> > doesn't
> > > > > > > > > maintain this in the source code, but we have the list of
> > all the
> > > > > > > > > committers on the project's site [2] Every new committer's
> > first
> > > > > > > > commit is
> > > > > > > > > to update the team page ;)
> > > > > > > > > - comments like in
> > > > > > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > > > > > >
> > > > > > > > > +/**
> > > > > > > > > + * Created by aelberg on 7/28/15.
> > > > > > > > > + */
> > > > > > > > >
> > > > > > > > > is better to be removed. It has been already discussed here
> > [3].
> > > > > And
> > > > > > > > the
> > > > > > > > > initial discussion on the topic [4] was linked as well
> > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if
> > this is
> > > > > > > > R-specific
> > > > > > > > > stuff - I have no idea about R, honestly, but
> > > > > > > > >
> > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > > > > > > ...
> > > > > > > > > +Author: David B. Dahl
> > > > > > > > >
> > > > > > > > > shouldn't be here, IMO. Normally, if some additional
> > licenses are
> > > > > > > > used,
> > > > > > > > > they have to be listed in the top-level NOTICE file (already
> > > > > there).
> > > > > > > > >
> > > > > > > > > - I am not going to offer any comments on the technical
> > merits of
> > > > > the
> > > > > > > > patch,
> > > > > > > > > as I haven't tried it personally. However, I don't see any
> > serious
> > > > > > > > > technical objections to the functionality in question.
> > > > > > > > >
> > > > > > > > > So, the question is - why the PR is open for three months? I
> > hasn't
> > > > > > > been
> > > > > > > > able
> > > > > > > > > to get a clear answer. What I found out though is pretty
> > > > > unsettling,
> > > > > > > > really.
> > > > > > > > > The communication on the topic is almost non-existent,
> > except for
> > > > > > this
> > > > > > > > sparse
> > > > > > > > > and bitter thread in the GH.
> > > > > > > > >
> > > > > > > > > I would love to hear from the committers what's stopping the
> > > > > > acceptance
> > > > > > > > of the
> > > > > > > > > code, besides of the minor issues I've mentioned earlier?
> > What are
> > > > > > the
> > > > > > > > reasons for it?
> > > > > > > > > Is there anything wrong with it?
> > > > > > > > > One of the responsibilities of the committers is to make
> > sure the
> > > > > > > > > contributions are reviewed; new contributors are guided and
> > do
> > > > > > > > understand how
> > > > > > > > > the project ticks. The easy feedback flow attracts new
> > people,
> > > > > > allowing
> > > > > > > > the
> > > > > > > > > community to grow and thrive.
> > > > > > > > >
> > > > > > > > > I am asking _explicitely_ not to re-start the bickering I
> > have
> > > > > > already
> > > > > > > > > seen. At this point I am interested in the purely technical
> > side of
> > > > > > > this.
> > > > > > > > >
> > > > > > > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > > > > > [3]
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > > > > > [4] http://s.apache.org/iZl
> > > > > > > > >
> > > > > > > > > With regards,
> > > > > > > > > Cos
> > > > > > > > >
> > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > > > > > Github user elbamos commented on the pull request:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > > > > > >
> > > > > > > > > > The current push should resolve some issues with changes
> > in the
> > > > > > > > > > Spark-Zeppelin interface that had created issues for
> > users, as
> > > > > > > > well as
> > > > > > > > > > support for 1.5.1.
> > > > > > > > > >
> > > > > > > > > > Knitr support is improved, and the reason for a separate
> > knitr
> > > > > > > > interpreter may be clearer now.
> > > > > > > > > >
> > > > > > > > > > The amount of code borrowed from rscala is reduced.
> > > > > > > > > >
> > > > > > > > > > I did not address issues with @author tags, or files under
> > the R/
> > > > > > > > > > folder. The reason is, to be blunt, I don't understand
> > what the
> > > > > > > > precise
> > > > > > > > > > concerns actually are.
> > > > > > > > > >
> > > > > > > > > > Please note that I changed .travis.yml to only use spark
> > 1.4 and
> > > > > > > > later.
> > > > > > > > > > I'm sure there is a better way to do it, but I'm just not
> > in a
> > > > > > > > position
> > > > > > > > > > to try to figure out the entire Zeppelin build process.
> > > > > > > > > >
> > > > > > > > > > Integrating this with the rest of zeppelin is going to
> > take some
> > > > > > > > work
> > > > > > > > > > regarding pom's, travis, and so forth. I can do a lot of
> > that,
> > > > > > > > but I'm
> > > > > > > > > > going to need to discuss it with the people who have been
> > > > > "owning"
> > > > > > > > those
> > > > > > > > > > files. There are just too many moving pieces here.
> > > > > > > > > >
> > > > > > > > > > Because of the size of this (which is, unfortunately,
> > necessary),
> > > > > > > > > > posting here is probably not the most efficient way. That
> > is also
> > > > > > > > true
> > > > > > > > > > because certain people chose to use this PR as a forum to
> > air
> > > > > other
> > > > > > > > > > issues. Therefore, it would be better if reviewers e-mail
> > me
> > > > > > > > directly.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> >

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Moon - 

In this e-mail you seem to be saying that you believe the problem is *linking* against knitr.

Before I address your e-mail I want to be very clear that “linking” is not the same issue as whether something is a mandatory dependency.  

These are separate things that seem to be getting conflated.   

Building Zeppelin with the instruction as documented will build 
KnitRInterpreter and enable KnitRInterpreter on runtime. 
So KnitRInterpreter is not an optional feature. 

“Mandatory” != “Default.”

class “Default” extends “Optional” … 

The KnitRIntepreter is only compiled if R is installed.  It is only enabled at runtime if knitr is installed.  It is only used at runtime if the user types “knitr.”  

But, if this is your concern, it is very easy to resolve:   Don’t include KnitRInterpreter in the default configuration xml.

It seems obvious to me that if your concern could be resolved by changing the default config, then its not a licensing issue. 

So KnitRInterpreter now have features that dynamic report generation 
oriented from knitr. That's clear design goal in my understanding. 
knitr is an interface to R that is an alternative to a REPL.  This is the same way that Zeppelin is an interface to Spark that is an alternative to the bundled spark cli client.  knitr is also written in R. 

3. knitr is licensed under GPL 

knitr is licensed under GPL.  R is licensed under GPL.  R’s standard libraries are licensed under GPL. 

We have all agreed that we want Zeppelin to integrate with R, like SparkR does, right?

 Binding knitr library through R language interpreter requires Zeppelin 
to be GPL ...

"However, when the interpreter is extended to provide “bindings” to other 
facilities (often, but not necessarily, libraries), the interpreted program 
is effectively linked to the facilities it uses through these bindings. S...

In our case, 'interpreter' is R, 'facilities' is knitr. 

Please correct me if i'm wrong. 

The key words here are “binding” and “link.”  The example given by the GPL is Java that makes calls to a system library through the JNI.  Because that is a *link* to the JVM.  

There is no “binding” or “link” between *knitr* and *R* .   Knitr itself is just interpreted code, what the GPL FAQ calls “just data.”

knitr is not a library.  (Sometimes the language gets confused, and I am surely as guilty of that as anyone.)  It’s a package.  http://www.r-bloggers.com/packages-v-libraries-in-r/

If the user has on their system a package they wrote themselves, and for whatever reason they decided to call it “knitr,” the PR will call that code.  

There is no “binding” or “link” between the PR and R either. 

If the user gets R from a source other than the R foundation (there are, in fact, non-GPL’d, commercial versions of R), the PR will use that instead of R-foundation-GLP'd-R.  

R scripts don’t bind, and they don’t link.  (Well, they will if you want them to, but none of the ones involved here do.)  This is a difference between R and, say, python or perl or Java.  This is a feature of R. 

Here is a short R script:

system(command = “Word.exe”)
system(command = “gpg”) 

If you exported this email from your mail client as text and told R to source the file (and you’re on windows), it would launch Word.  If you’re on Linux, it will launch Gnu privacy guard. 

Does this e-mail link against MS Office?   The e-mail is going to be maintained on an ASF server.  Does the entire ASF now have to convert to GPL 3?


From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 2, 2015 at 10:03:35 PM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

Let me make my concern clear.  

1. KnitRInterpreter is enabled by default  

Building Zeppelin with the instruction as documented will build  
KnitRInterpreter and enable KnitRInterpreter on runtime.  
So KnitRInterpreter is not an optional feature.  

2. KnitRInterpreter is designed to interact with knitr library.  

Purpose of KnitRInterpreter is to call a knitr library.  
You'll able to see [1] KnitRInterpreter.java:open(), [2]  
KnitRInterpreter.java:interpret() to see how KnitRInterpreter works.  

knitr library is called from JVM (Zeppelin) using R language interpreter.  

So KnitRInterpreter now have features that dynamic report generation  
oriented from knitr. That's clear design goal in my understanding.  


3. knitr is licensed under GPL  

https://github.com/yihui/knitr  


4. Binding knitr library through R language interpreter requires Zeppelin  
to be GPL  

According to http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL,  

"However, when the interpreter is extended to provide “bindings” to other  
facilities (often, but not necessarily, libraries), the interpreted program  
is effectively linked to the facilities it uses through these bindings. So  
if these facilities are released under the GPL, the interpreted program  
that uses them must be released in a GPL-compatible way"  

In our case, 'interpreter' is R, 'facilities' is knitr.  



Please correct me if i'm wrong.  

[1]  
https://github.com/apache/incubator-zeppelin/pull/208/files#diff-1bb44f3ddb1a6e92144352d8b54084c2R33  
[2]  
https://github.com/apache/incubator-zeppelin/pull/208/files#diff-1bb44f3ddb1a6e92144352d8b54084c2R54  


Thanks,  
moon  

On Thu, Dec 3, 2015 at 11:03 AM Corneau Damien <co...@gmail.com> wrote:  

> Thanks Cos for those answers,  
>  
> If I'm right you are advising to have a default build that doesn't include  
> libraries with conflicting licenses, but have an option to include them for  
> users who wants to build the project themselves.  
>  
> To refer to another thread about decentralizing interpreters, it could even  
> be better in that case to have some interpreters separated from the tree,  
> and easily pluggable with a release instead of forcing users to build the  
> project to use them.  
>  
>  
> On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <co...@apache.org> wrote:  
>  
> > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:  
> > > Konstantin thank you for getting into this.  
> > >  
> > >> The best way to go around it is by  
> > >> providing a build-time option that will pull such binaries in. But by  
> > default  
> > >> such libs shouldn't be pulled.  
> > >  
> > > That is basically how the PR handles this. If the GPL’d interpreter  
> > scripts  
> > > are missing, there’s no indication at all at build time. It doesn’t  
> try  
> > to  
> > > download them.  
> > >  
> > > At runtime, if the user tries to use functionality that would need  
> such a  
> > > script (i.e., if they type “knitr” to use knitr), we display an error  
> > that  
> > > says that the functionality is not there because the library is  
> missing,  
> > and  
> > > that the library cannot be provided because it has an incompatible  
> > license,  
> > > but the user can download it themselves if they want.  
> > >  
> > > And, in the log, if the logging level is high, they will see a note  
> that  
> > > some functionality was disabled because the libraries weren’t there.  
> > >  
> > > To be clear, though, none of these libraries are binaries. They’re all  
> > interpreter scripts.  
> >  
> > If you ever in a doubt of which licenses could be used for dependncies  
> > (not to  
> > say about source code) are listed in Category A list of [1]  
> >  
> > A lot of quesitons discussed here are already covered in the legal FAQ,  
> so  
> > just check against it if you have any questions.  
> >  
> > [1] http://www.apache.org/legal/resolved.html#category-a  
> >  
> > Cos  
> >  
> > > From: Konstantin Boudnik <co...@apache.org>  
> > > Reply: dev@zeppelin.incubator.apache.org <  
> > dev@zeppelin.incubator.apache.org>, dev@zeppelin.incubator.apache.org <  
> > dev@zeppelin.incubator.apache.org>  
> > > Date: December 2, 2015 at 3:24:50 PM  
> > > To: dev@zeppelin.incubator.apache.org <  
> dev@zeppelin.incubator.apache.org  
> > >  
> > > Subject: Re: License of KnitRInterpreter Was: Re: contributions  
> > impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for  
> > Zeppelin  
> > >  
> > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:  
> > > > I think that what moon means is that:  
> > > > If we merge the way it is now, the KnitRInterpreter will be part of  
> the  
> > > > code base, so it isn't optional at code base level.  
> > > >  
> > > > To make it even more simple:  
> > > > * If the code has the right licensing -> that code can be part of the  
> > > > source code, and can be including in a binary release  
> > >  
> > > We aren't concerned with binary releases - as an Apache community we  
> are  
> > > voting and releasing source code. If the project wants to provide a  
> > binary  
> > > release to its users, they are better be warned about inclusion of non  
> > > ASL2-friendly licensed bits.  
> > >  
> > > > * If the code doesn't have the right licensing -> it can't be part of  
> > the  
> > > > source code, and can't be included in a binary release  
> > >  
> > > See above.  
> > >  
> > > > * If the code doesn't have the right licensing but is imported at  
> build  
> > > > time (libraries for example) -> it is not in the source code, it  
> can't  
> > be  
> > > > included in binary release  
> > >  
> > > That is unless a user doing it on his own. The best way to go around it  
> > is by  
> > > providing a build-time option that will pull such binaries in. But by  
> > default  
> > > such libs shouldn't be pulled.  
> > >  
> > > Cos  
> > >  
> > > > So in the case of licensing issues, it would need to be fully  
> optional  
> > > > (user bring the interpreter in his directory and build Zeppelin with  
> > it)  
> > > >  
> > > >  
> > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <  
> amos.elberg@gmail.com>  
> > > > wrote:  
> > > >  
> > > > > Moon let me clarify:  
> > > > >  
> > > > > Interpreted code doesn’t “link.” The wiki article actually explains  
> > it  
> > > > > pretty well — https://en.wikipedia.org/wiki/GPL_linking_exception  
> > > > > “Linking” against a library means compiling its headers into a  
> > binary, the  
> > > > > way a C compiler works. The 2008 e-mail Moon distributed, called  
> > this the  
> > > > > “interpreter exception.”  
> > > > >  
> > > > > As for whether GPL’d code is a “mandatory dependency,” if knitr is  
> > missing  
> > > > > the PR will compile, run and test just fine. The user is never  
> > prompted to  
> > > > > download it. The only effect is, if the user types “knitr” and  
> knitr  
> > isn’t  
> > > > > there, we display an InterpreterError that knitr isn’t there.  
> > > > >  
> > > > > KnitRInterpreter is not optionally required. so it does not matter  
> > KnitR  
> > > > > is  
> > > > > optionally required or not.  
> > > > > Aren’t all interpreters optional? You can turn them on and off in  
> the  
> > > > > config files.  
> > > > >  
> > > > > Do you mean that the KnitRInterpreter class gets compiled to  
> > bytecode even  
> > > > > if knitr is missing? So what? That isn't a mandatory dependency or  
> a  
> > link.  
> > > > >  
> > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > > dev@zeppelin.incubator.apache.org>  
> > > > > Date: December 2, 2015 at 3:18:00 AM  
> > > > > To: dev@zeppelin.incubator.apache.org <  
> > dev@zeppelin.incubator.apache.org>  
> > > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions  
> > impasse.  
> > > > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for  
> > Zeppelin  
> > > > >  
> > > > > Let me summarize license concern about KnitRInterpreter.  
> > > > >  
> > > > >  
> > > > > Amos's interpretation.  
> > > > >  
> > > > > KnitR is optionally required by KnitRInterpreter.  
> > > > > R dependency in SparkR has no problem. So KnitR should be the same.  
> > > > >  
> > > > >  
> > > > > Moon's interpretation.  
> > > > >  
> > > > > KnitRInterpreter is not optionally required. so it does not matter  
> > KnitR is  
> > > > > optionally required or not.  
> > > > > R dependency in SparkR is exception of GPL. KnitR is not applied  
> that  
> > > > > exception.  
> > > > >  
> > > > >  
> > > > > Amos, could you confirm my understanding to your interpretation is  
> > correct?  
> > > > > If it is not could you clarify it?  
> > > > >  
> > > > > Thanks,  
> > > > > moon  
> > > > >  
> > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <  
> > amos.elberg@gmail.com>  
> > > > > wrote:  
> > > > >  
> > > > > > Just to put the final nail in this, I looked it up.  
> > > > > >  
> > > > > > I see no evidence of any “compiler exception.”  
> > > > > >  
> > > > > > There is an exception in the license for the runtime libraries  
> > that are  
> > > > > > bundled with GCC. See:  
> > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html  
> > > > > >  
> > > > > > But no “compiler exception.”  
> > > > > >  
> > > > > > In fact, I believe this is part of the reason why LLVM was  
> created.  
> > > > > >  
> > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > Reply: moon soo Lee <mo...@apache.org>  
> > > > > > Date: December 1, 2015 at 8:16:36 PM  
> > > > > > To: Amos B. Elberg <am...@gmail.com>,  
> > > > > > dev@zeppelin.incubator.apache.org <  
> > dev@zeppelin.incubator.apache.org>  
> > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > incubator-zeppelin pull  
> > > > > > request: R Interpreter for Zeppelin  
> > > > > >  
> > > > > > Is knitR is commonly considered as a interpreter/compiler? or is  
> it  
> > > > > > considered as a library routine?  
> > > > > >  
> > > > > > Thanks,  
> > > > > > moon  
> > > > > >  
> > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <  
> > amos.elberg@gmail.com>  
> > > > > > wrote:  
> > > > > > Moon - you give this as an explanation of the licensing issue:  
> > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
> > > > > >  
> > > > > > According to that, there is an exception in the GPL for  
> interpreter  
> > > > > > languages. As long as you don’t distribute the code, its fine to  
> > talk to  
> > > > > > an interpreted language.  
> > > > > >  
> > > > > > Well, if that’s the case, then the PR plainly does not have a  
> > license  
> > > > > > issue. It doesn’t distribute any GPL’d R code.  
> > > > > >  
> > > > > > I’m not sure what’s confusing about this. It seems completely  
> > > > > > straightforward.  
> > > > > >  
> > > > > > Regarding this:  
> > > > > >  
> > > > > >  
> > > > > > --  
> > > > > > Amos Elberg  
> > > > > > Sent with Airmail  
> > > > > >  
> > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > Date: December 1, 2015 at 6:48:47 PM  
> > > > > >  
> > > > > > To: dev@zeppelin.incubator.apache.org <  
> > dev@zeppelin.incubator.apache.org  
> > > > > >  
> > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > incubator-zeppelin pull  
> > > > > > request: R Interpreter for Zeppelin  
> > > > > >  
> > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <  
> > amos.elberg@gmail.com>  
> > > > > > wrote:  
> > > > > >  
> > > > > > > I am going to try to minimize my reaction to Moon’s e-mail.  
> > > > > > >  
> > > > > > > The tl;dr is this:  
> > > > > > >  
> > > > > > > The reason we are having this discussion now is that active  
> > users of  
> > > > > the  
> > > > > > > PR — which now has its own user base — went public to complain  
> > about  
> > > > > > this.  
> > > > > >  
> > > > > >  
> > > > > > > The PR has been tested by an active user base for more than  
> three  
> > > > > months.  
> > > > > > > No-one has been able to identify any specific actual licensing  
> > problem,  
> > > > > > and  
> > > > > > > the PR was prepared based on an extensive, careful review of  
> the  
> > > > > relevant  
> > > > > > > licensing issues and after contacting the relevant people.  
> > > > > > >  
> > > > > > >  
> > > > > >  
> > > > > > I admire every software that used by user and helping people.  
> That  
> > > > > includes  
> > > > > > your work. But that's not the topic we're in discussion. Active  
> > user does  
> > > > > > not mean your contribution can ignore the review.  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > > > It is not an explanation for someone who has been ignoring my  
> > “how can  
> > > > > I  
> > > > > > > move this forward…” emails for three months to point the finger  
> > and  
> > > > > say I  
> > > > > > > didn’t contact the right person or file the right report.  
> > > > > > >  
> > > > > > >  
> > > > > > This is also not the topic in this discussion.  
> > > > > >  
> > > > > >  
> > > > > > > The burden for providing an explanation for the inaction is on  
> > the PMCC  
> > > > > > at  
> > > > > > > this point.  
> > > > > >  
> > > > > > I'm sorry, but the other PRs are passing CI. If it's problem on  
> > Zeppelin  
> > > > > > > core, why do you think other PRs are passing CI?  
> > > > > > > They’re not! I often see comments on PRs to just ignore that CI  
> > is  
> > > > > > > failing.  
> > > > > > >  
> > > > > > > One of the most common reasons this PR fails CI, is CI  
> times-out  
> > > > > > > downloading Spark to install. How could that possibly be caused  
> > by the  
> > > > > > PR?  
> > > > > > >  
> > > > > > > It looks to me like the only PRs with changes to the relevant  
> > parts of  
> > > > > > the  
> > > > > > > code — the SparkInterpreter — are being made by the person who  
> > wrote  
> > > > > the  
> > > > > > > testing suite.  
> > > > > > >  
> > > > > > > So, that would explain why some other PRs pass CI: Neither the  
> > > > > > > SparkInterpreter nor the testing suite are stable or robust,  
> but  
> > since  
> > > > > > the  
> > > > > > > PRs are coming from the person who wrote both…  
> > > > > > >  
> > > > > > > And let's say Zeppelin core has problem and that makes your PR  
> > fails on  
> > > > > > CI  
> > > > > > > test. That's possible. But it still does not mean we can merge  
> > it with  
> > > > > CI  
> > > > > > > failing.  
> > > > > > >  
> > > > > > > It means you should be working with me to figure out why the CI  
> > is  
> > > > > > failing.  
> > > > > > >  
> > > > > > > This PR has been tested by an active user base for the past  
> three  
> > > > > months.  
> > > > > > > If CI is continuing to fail, and dozens of hours of effort have  
> > not  
> > > > > > > resolved the CI issues, then it is time to start considering  
> > whether  
> > > > > the  
> > > > > > > testing suite is part of the problem.  
> > > > > > >  
> > > > > > > The level of defensiveness about the CI and SparkInterpreter is  
> > not  
> > > > > > > helping to resolve these issues.  
> > > > > > >  
> > > > > > > If you think it's problem on Zeppelin core, then file an issue  
> > that  
> > > > > > > reproduce the problem on Zeppelin core, that might be more  
> > efficient  
> > > > > than  
> > > > > > > keep trying yourself.  
> > > > > > > I contacted you numerous times about such issues...  
> > > > > > >  
> > > > > >  
> > > > > >  
> > > > > > I remember i commented your issue about CI. but you just keep  
> > repeated  
> > > > > it's  
> > > > > > not your problem but Zeppelin core problem.  
> > > > > >  
> > > > > > Then please file an issue about the problem you found in Zeppelin  
> > Core.  
> > > > > > Then everyone will get into the problem.  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > > >  
> > > > > > > In my interpretation, KnitRInterpreter is not an optional  
> > feature while  
> > > > > > it  
> > > > > > > is always enabled when compiling Zeppelin and always enabled  
> when  
> > > > > running  
> > > > > > > Zeppelin. And it requires dynamically linked GPL library on  
> > runtime.  
> > > > > (yes  
> > > > > > > it will fail when no KnitR is installed on the system)  
> > > > > > >  
> > > > > > > Its not always enabled.  
> > > > > > > It is not dynamically linked at runtime.  
> > > > > > > It will not fail when knitr is missing. If knitr is not  
> present,  
> > the  
> > > > > repl  
> > > > > > > interpreter starts and a note is written to to the log that the  
> > knitr  
> > > > > > > interpreter isn’t available because knitr is not present.  
> > > > > > >  
> > > > > > > no Apache code can ever call a shell script, on the purpose of  
> > dynamic  
> > > > > > > linking with GPL library.  
> > > > > > > You misunderstand.  
> > > > > > >  
> > > > > > > The *shell* is GPL'd.  
> > > > > > >  
> > > > > > > Is Zeppelin “linked" against the GPL’d shell because Zeppelin  
> > depends  
> > > > > on  
> > > > > > a  
> > > > > > > shell script to launch?  
> > > > > > >  
> > > > > > Obviously not.  
> > > > > > >  
> > > > > > > The interaction with R in the PR is the same.  
> > > > > > >  
> > > > > > >  
> > > > > >  
> > > > > > Again, bash is one of exceptions of GPL, like other GPL licensed  
> > > > > > compiler/interpreter.  
> > > > > >  
> > > > > > Check here why Bash and R is okay with Apache License.  
> > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
> > > > > >  
> > > > > > I'm not sure we can apply the same exception for 'using' KnitR.  
> > > > > >  
> > > > > > My point is not 'KnitR' is optional or not. Point is  
> > 'KnitRInterpreter'  
> > > > > you  
> > > > > > wrote is not an optional feature. Which is clearly not optionally  
> > enabled  
> > > > > > code and feature. And that depends on KnitR library which is GPL.  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > > > I was guessing SparkR can be still in Apache License even if it  
> > is  
> > > > > > depends  
> > > > > > > on R. Because of GPL licensed compiler generated output is not  
> > GPL  
> > > > > > license.  
> > > > > > > and R is sort of compiler. If you can get answer from Spark  
> > community  
> > > > > how  
> > > > > > > SparkR get managed to stay in Apache License while R is GPL,  
> the  
> > answer  
> > > > > > > might help.  
> > > > > > > The description of SparkR is not accurate in any respect. (Do  
> > you think  
> > > > > > > SparkR is not talking to GPL-licensed libraries?)  
> > > > > > >  
> > > > > > > I don’t see that any genuine issue is being raised here.  
> > > > > > >  
> > > > > > > If there is an issue, the burden is on you to identify it.  
> > > > > > >  
> > > > > > > If i give you one suggestion, Zeppelin committers sometimes ask  
> > rebase  
> > > > > > the  
> > > > > > > contribution branch for some reason. It is not the really the  
> > best  
> > > > > > > practice, but still okay while most contributions are not  
> > including  
> > > > > large  
> > > > > > > code base changes  
> > > > > > > However, your one, has more than 4000 lines of code change. If  
> > you  
> > > > > rebase  
> > > > > > > then review should be started from the beginning, again. So you  
> > might  
> > > > > > want  
> > > > > > > to minimize the rebase your branch.  
> > > > > > >  
> > > > > > > Are you actually complaining that the problem is that I rebased  
> > the  
> > > > > code  
> > > > > > > during the three-month period when no-one looked at it and  
> > Zeppelin  
> > > > > went  
> > > > > > > through a release?  
> > > > > > >  
> > > > > > > I cannot take it seriously when you say things like this.  
> > > > > > >  
> > > > > > > Having to “start from the beginning” cannot be a problem if you  
> > never  
> > > > > > > actually started the first time...  
> > > > > > >  
> > > > > > >  
> > > > > >  
> > > > > > You wanted coordination and cooperation. So i gave you suggestion  
> > that  
> > > > > > helping review process. For example, your code has been rebased  
> > since my  
> > > > > > comment and jongyoul's comment. that means committers will need  
> to  
> > look  
> > > > > > from the beginning. That'll require more time. And maybe, i guess  
> > that's  
> > > > > > not what you want. But If you don't care, feel free to rebase.  
> > > > > >  
> > > > > > Thanks,  
> > > > > > moon  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > > >  
> > > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > > Reply: moon soo Lee <mo...@apache.org>  
> > > > > > > Date: December 1, 2015 at 4:42:06 AM  
> > > > > > > To: Amos B. Elberg <am...@gmail.com>,  
> > > > > > > dev@zeppelin.incubator.apache.org <  
> > dev@zeppelin.incubator.apache.org>  
> > > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > incubator-zeppelin  
> > > > > pull  
> > > > > > > request: R Interpreter for Zeppelin  
> > > > > > >  
> > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <  
> > amos.elberg@gmail.com>  
> > > > > > > wrote:  
> > > > > > > Thank you, Cos.  
> > > > > > >  
> > > > > > > I’d like to briefly address the issues raised by Moon:  
> > > > > > >  
> > > > > > > 1. This PR does not passes CI  
> > > > > > > The CI fails on core Zeppelin, *not* code in this PR.  
> > > > > > >  
> > > > > > > I’ve been seeking assistance on this since August.  
> > > > > > >  
> > > > > > > The most common reason is that SparkInterpreter is unable to  
> > launch  
> > > > > Spark  
> > > > > > > and open a Spark Backend. This is necessary to test the PR.  
> > > > > > >  
> > > > > > > 60+ hours, has been spent adapting and re-basing when the  
> > > > > > SparkInterpreter  
> > > > > > > architecture changed and broke the PR’s *tests.*  
> > > > > > >  
> > > > > > >  
> > > > > > > I'm sorry, but the other PRs are passing CI. If it's problem on  
> > > > > Zeppelin  
> > > > > > > core, why do you think other PRs are passing CI?  
> > > > > > >  
> > > > > > > And let's say Zeppelin core has problem and that makes your PR  
> > fails on  
> > > > > > CI  
> > > > > > > test. That's possible. But it still does not mean we can merge  
> > it with  
> > > > > CI  
> > > > > > > failing.  
> > > > > > >  
> > > > > > > If you think it's problem on Zeppelin core, then file an issue  
> > that  
> > > > > > > reproduce the problem on Zeppelin core, that might be more  
> > efficient  
> > > > > than  
> > > > > > > keep trying yourself.  
> > > > > > >  
> > > > > > >  
> > > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)  
> > > > > > > What license problem *specifically* do you believe may exist?  
> > > > > > >  
> > > > > > > In preparing the PR, I:  
> > > > > > >  
> > > > > > > * Reviewed the Apache policy in detail.  
> > > > > > >  
> > > > > > > * Contacted the FSF to ask their interpretation of the  
> “linking”  
> > > > > > > provisions of the GPL license.  
> > > > > > >  
> > > > > > > * Reviewed how other Apache software deals with this issue  
> > (e.g., Spark  
> > > > > > > talks to R, which is GPL'd).  
> > > > > > >  
> > > > > > > * No necessary *dependencies* of the PR have license conflicts.  
> > In  
> > > > > > > several cases, I contacted package authors who agreed to  
> > re-issue their  
> > > > > > > packages under Apache-compatible licenses. (Usually I had to do  
> > a bit  
> > > > > of  
> > > > > > > coding in exchange…)  
> > > > > > >  
> > > > > > > * Where the license had to stay GPL, the packages are *not  
> > necessary*  
> > > > > and  
> > > > > > > *not dependencies.* If the optional packages are present, they  
> > will be  
> > > > > > > used to enable additional functionality. Knitr is an example.  
> > The PR  
> > > > > will  
> > > > > > > compile and run fine without knitr. If knitr is available (it  
> is  
> > part  
> > > > > of  
> > > > > > > the most common R distribution), the PR will enable the knitr  
> > > > > > interpreter.  
> > > > > > >  
> > > > > > > * This is exactly how this issue is addressed through the  
> Apache  
> > > > > > > ecosystem.  
> > > > > > > The tl;dr is this: When Apache code is written to talk to  
> > libraries  
> > > > > that  
> > > > > > > may or may not be present on the user’s system, or where it  
> > talks to an  
> > > > > > API  
> > > > > > > but is agnostic about implementation, that is not “linking” in  
> a  
> > way  
> > > > > that  
> > > > > > > implicate the anti-linking provision of the GPL.  
> > > > > > >  
> > > > > > > Otherwise, no Apache code could ever call a shell script! Let  
> > alone run  
> > > > > > > on Linux, or talk to R.  
> > > > > > >  
> > > > > > >  
> > > > > > > I'm not a legal expert. So following could be wrong.  
> > > > > > >  
> > > > > > > In my interpretation, KnitRInterpreter is not an optional  
> > feature while  
> > > > > > it  
> > > > > > > is always enabled when compiling Zeppelin and always enabled  
> when  
> > > > > running  
> > > > > > > Zeppelin. And it requires dynamically linked GPL library on  
> > runtime.  
> > > > > (yes  
> > > > > > > it will fail when no KnitR is installed on the system)  
> > > > > > >  
> > > > > > > And of course, no Apache code can ever call a shell script, on  
> > the  
> > > > > > purpose  
> > > > > > > of dynamic linking with GPL library.  
> > > > > > >  
> > > > > > > I was guessing SparkR can be still in Apache License even if it  
> > is  
> > > > > > depends  
> > > > > > > on R. Because of GPL licensed compiler generated output is not  
> > GPL  
> > > > > > license.  
> > > > > > > and R is sort of compiler.  
> > > > > > >  
> > > > > > > If you can get answer from Spark community how SparkR get  
> > managed to  
> > > > > stay  
> > > > > > > in Apache License while R is GPL, the answer might help.  
> > > > > > >  
> > > > > > >  
> > > > > > > 3. Need more time to review.  
> > > > > > > Has any reviewer has downloaded the PR or run the demo  
> notebook?  
> > (Which  
> > > > > > > is there for the benefit of reviewers, and isn’t intended to go  
> > in  
> > > > > final  
> > > > > > > distribution.)  
> > > > > > >  
> > > > > > > How many +1 comments from users would you like to see?  
> > > > > > >  
> > > > > > > How much time do you believe is required?  
> > > > > > >  
> > > > > > >  
> > > > > > > It all depends on when CI is going to pass, when license  
> problem  
> > is  
> > > > > going  
> > > > > > > to be cleared, and when a committer willing to review and  
> > responsible  
> > > > > to  
> > > > > > > commit your contribution.  
> > > > > > >  
> > > > > > >  
> > > > > > > 1. Large code base change  
> > > > > > > Large code base changes require coordination and cooperation.  
> > This PR  
> > > > > > > necessarily implicates the build scripts, testing code, the  
> > > > > > > SparkInterpreter, etc.  
> > > > > > >  
> > > > > > > I have been seeking to coordinate since August.  
> > > > > > >  
> > > > > > > I continue to stand ready to do so.  
> > > > > > >  
> > > > > > > -Amos  
> > > > > > >  
> > > > > > >  
> > > > > > > If i give you one suggestion, Zeppelin committers sometimes ask  
> > rebase  
> > > > > > the  
> > > > > > > contribution branch for some reason. It is not the really the  
> > best  
> > > > > > > practice, but still okay while most contributions are not  
> > including  
> > > > > large  
> > > > > > > code base changes.  
> > > > > > >  
> > > > > > > However, your one, has more than 4000 lines of code change. If  
> > you  
> > > > > rebase  
> > > > > > > then review should be started from the beginning, again. So you  
> > might  
> > > > > > want  
> > > > > > > to minimize the rebase your branch.  
> > > > > > >  
> > > > > > > Thanks,  
> > > > > > > moon  
> > > > > > >  
> > > > > > >  
> > > > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > > > > dev@zeppelin.incubator.apache.org>  
> > > > > > > Date: December 1, 2015 at 1:34:19 AM  
> > > > > > > To: dev@zeppelin.incubator.apache.org <  
> > > > > dev@zeppelin.incubator.apache.org  
> > > > > > >  
> > > > > > > Subject: Re: contributions impasse. Was: [GitHub]  
> > incubator-zeppelin  
> > > > > pull  
> > > > > > > request: R Interpreter for Zeppelin  
> > > > > > >  
> > > > > > > Hi Cos,  
> > > > > > >  
> > > > > > > Thanks for opening a discussion.  
> > > > > > > My answer about 'Why this PR is open for three months' is  
> > > > > > >  
> > > > > > > 1. This PR does not passes CI  
> > > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)  
> > > > > > > 3. Need more time to review.  
> > > > > > >  
> > > > > > > It's my personal answer, other committers may have different  
> > opinion.  
> > > > > > >  
> > > > > > >  
> > > > > > > I think the question should be generalized. Because this PR is  
> > not the  
> > > > > > only  
> > > > > > > PR that is in impasse. There're more. For example  
> > > > > > >  
> > > > > > > Here's some examples that PR is not been merged.  
> > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,  
> > > > > > > https://github.com/apache/incubator-zeppelin/pull/60  
> > > > > > > and so on.  
> > > > > > >  
> > > > > > > I can categorize the cases, based on experience of involving  
> > Zeppelin  
> > > > > > > community from the beginning.  
> > > > > > >  
> > > > > > > 1. Large code base change  
> > > > > > >  
> > > > > > > When contribution has large code base changes, it tend to take  
> > more  
> > > > > time  
> > > > > > to  
> > > > > > > review and merged. Normally, most contributions merged in  
> 1day~1  
> > week.  
> > > > > > > But some contribution has large code base changes take 2~4  
> > weeks. Few  
> > > > > > > contribution that has very large code base change take months.  
> > > > > > >  
> > > > > > > 2. Communication lost  
> > > > > > >  
> > > > > > > Sometimes, committer is not responding, or contributor is not  
> > > > > responding.  
> > > > > > >  
> > > > > > > 3. Opinion diverges  
> > > > > > >  
> > > > > > > For some changes, included in contribution. When people have  
> > different  
> > > > > > > opinion and it continue to diverges, those PRs are not been  
> > merged.  
> > > > > > >  
> > > > > > >  
> > > > > > > I think having a guide such as ping other committer when a  
> > committer is  
> > > > > > not  
> > > > > > > responding, and divide contribution into small peaces if  
> > possible,  
> > > > > would  
> > > > > > > help most of the cases.  
> > > > > > > Of course committer have to pay attention more to the  
> > contribution and  
> > > > > > > helping people. That's the first one.  
> > > > > > >  
> > > > > > > Thanks,  
> > > > > > > moon  
> > > > > > >  
> > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <  
> > cos@apache.org>  
> > > > > > wrote:  
> > > > > > >  
> > > > > > > > To make sure we're on the same page, here are two threads  
> that  
> > I  
> > > > > found  
> > > > > > > > related  
> > > > > > > > to this topic.  
> > > > > > > >  
> > > > > > > > Thread 1:  
> > > > > > > > Subject: R?  
> > > > > > > > Started on: July 1, 2015  
> > > > > > > >  
> > > > > > > > Thread 2:  
> > > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R  
> > Interpreter for  
> > > > > > > > Zeppelin  
> > > > > > > > Started on: August 13, 2015  
> > > > > > > >  
> > > > > > > > If you want to fetch these from our archive send emails to  
> > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org  
> > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org  
> > > > > > > >  
> > > > > > > > Cos  
> > > > > > > >  
> > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:  
> > > > > > > > > Guys,  
> > > > > > > > >  
> > > > > > > > > While catching up on my emails from the last a couple of  
> > weeks,  
> > > > > this  
> > > > > > > > thread  
> > > > > > > > > caught my attention. I am not normally paying much  
> attention  
> > to the  
> > > > > > > code  
> > > > > > > > > reviews traffic from GH, but this one is pretty different  
> as  
> > it  
> > > > > spans  
> > > > > > > > three  
> > > > > > > > > months and counting.  
> > > > > > > > >  
> > > > > > > > > First, here are my five cents:  
> > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to  
> be  
> > > > > > > > contributed to  
> > > > > > > > > an ASF project this file should simply contain ASL2 text,  
> > like in  
> > > > > [1]  
> > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate  
> <developers>  
> > > > > > section,  
> > > > > > > > but  
> > > > > > > > > Zeppelin might have different guidelines on it. Say, Bigtop  
> > doesn't  
> > > > > > > > > maintain this in the source code, but we have the list of  
> > all the  
> > > > > > > > > committers on the project's site [2] Every new committer's  
> > first  
> > > > > > > > commit is  
> > > > > > > > > to update the team page ;)  
> > > > > > > > > - comments like in  
> > > > > > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java  
> > > > > > > > >  
> > > > > > > > > +/**  
> > > > > > > > > + * Created by aelberg on 7/28/15.  
> > > > > > > > > + */  
> > > > > > > > >  
> > > > > > > > > is better to be removed. It has been already discussed here  
> > [3].  
> > > > > And  
> > > > > > > > the  
> > > > > > > > > initial discussion on the topic [4] was linked as well  
> > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if  
> > this is  
> > > > > > > > R-specific  
> > > > > > > > > stuff - I have no idea about R, honestly, but  
> > > > > > > > >  
> > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE  
> > > > > > > > > ...  
> > > > > > > > > +Author: David B. Dahl  
> > > > > > > > >  
> > > > > > > > > shouldn't be here, IMO. Normally, if some additional  
> > licenses are  
> > > > > > > > used,  
> > > > > > > > > they have to be listed in the top-level NOTICE file  
> (already  
> > > > > there).  
> > > > > > > > >  
> > > > > > > > > - I am not going to offer any comments on the technical  
> > merits of  
> > > > > the  
> > > > > > > > patch,  
> > > > > > > > > as I haven't tried it personally. However, I don't see any  
> > serious  
> > > > > > > > > technical objections to the functionality in question.  
> > > > > > > > >  
> > > > > > > > > So, the question is - why the PR is open for three months?  
> I  
> > hasn't  
> > > > > > > been  
> > > > > > > > able  
> > > > > > > > > to get a clear answer. What I found out though is pretty  
> > > > > unsettling,  
> > > > > > > > really.  
> > > > > > > > > The communication on the topic is almost non-existent,  
> > except for  
> > > > > > this  
> > > > > > > > sparse  
> > > > > > > > > and bitter thread in the GH.  
> > > > > > > > >  
> > > > > > > > > I would love to hear from the committers what's stopping  
> the  
> > > > > > acceptance  
> > > > > > > > of the  
> > > > > > > > > code, besides of the minor issues I've mentioned earlier?  
> > What are  
> > > > > > the  
> > > > > > > > reasons for it?  
> > > > > > > > > Is there anything wrong with it?  
> > > > > > > > > One of the responsibilities of the committers is to make  
> > sure the  
> > > > > > > > > contributions are reviewed; new contributors are guided and  
> > do  
> > > > > > > > understand how  
> > > > > > > > > the project ticks. The easy feedback flow attracts new  
> > people,  
> > > > > > allowing  
> > > > > > > > the  
> > > > > > > > > community to grow and thrive.  
> > > > > > > > >  
> > > > > > > > > I am asking _explicitely_ not to re-start the bickering I  
> > have  
> > > > > > already  
> > > > > > > > > seen. At this point I am interested in the purely technical  
> > side of  
> > > > > > > this.  
> > > > > > > > >  
> > > > > > > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE  
> > > > > > > > > [2] http://bigtop.apache.org/team-list.html  
> > > > > > > > > [3]  
> > > > > > > >  
> > > > > > >  
> > > > > >  
> > > > >  
> >  
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html  
> > > > > > > > > [4] http://s.apache.org/iZl  
> > > > > > > > >  
> > > > > > > > > With regards,  
> > > > > > > > > Cos  
> > > > > > > > >  
> > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:  
> > > > > > > > > > Github user elbamos commented on the pull request:  
> > > > > > > > > >  
> > > > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > >  
> > > > >  
> >  
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411  
> > > > > > > > > >  
> > > > > > > > > > The current push should resolve some issues with changes  
> > in the  
> > > > > > > > > > Spark-Zeppelin interface that had created issues for  
> > users, as  
> > > > > > > > well as  
> > > > > > > > > > support for 1.5.1.  
> > > > > > > > > >  
> > > > > > > > > > Knitr support is improved, and the reason for a separate  
> > knitr  
> > > > > > > > interpreter may be clearer now.  
> > > > > > > > > >  
> > > > > > > > > > The amount of code borrowed from rscala is reduced.  
> > > > > > > > > >  
> > > > > > > > > > I did not address issues with @author tags, or files  
> under  
> > the R/  
> > > > > > > > > > folder. The reason is, to be blunt, I don't understand  
> > what the  
> > > > > > > > precise  
> > > > > > > > > > concerns actually are.  
> > > > > > > > > >  
> > > > > > > > > > Please note that I changed .travis.yml to only use spark  
> > 1.4 and  
> > > > > > > > later.  
> > > > > > > > > > I'm sure there is a better way to do it, but I'm just not  
> > in a  
> > > > > > > > position  
> > > > > > > > > > to try to figure out the entire Zeppelin build process.  
> > > > > > > > > >  
> > > > > > > > > > Integrating this with the rest of zeppelin is going to  
> > take some  
> > > > > > > > work  
> > > > > > > > > > regarding pom's, travis, and so forth. I can do a lot of  
> > that,  
> > > > > > > > but I'm  
> > > > > > > > > > going to need to discuss it with the people who have been  
> > > > > "owning"  
> > > > > > > > those  
> > > > > > > > > > files. There are just too many moving pieces here.  
> > > > > > > > > >  
> > > > > > > > > > Because of the size of this (which is, unfortunately,  
> > necessary),  
> > > > > > > > > > posting here is probably not the most efficient way. That  
> > is also  
> > > > > > > > true  
> > > > > > > > > > because certain people chose to use this PR as a forum to  
> > air  
> > > > > other  
> > > > > > > > > > issues. Therefore, it would be better if reviewers e-mail  
> > me  
> > > > > > > > directly.  
> > > > > > > >  
> > > > > > > >  
> > > > > > > >  
> > > > > > >  
> > > > > >  
> > > > >  
> >  
>  

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Let me make my concern clear.

1. KnitRInterpreter is enabled by default

Building Zeppelin with the instruction as documented will build
KnitRInterpreter and enable KnitRInterpreter on runtime.
So KnitRInterpreter is not an optional feature.

2. KnitRInterpreter is designed to interact with knitr library.

Purpose of KnitRInterpreter is to call a knitr library.
You'll able to see [1] KnitRInterpreter.java:open(), [2]
KnitRInterpreter.java:interpret() to see how KnitRInterpreter works.

knitr library is called from JVM (Zeppelin) using R language interpreter.

So KnitRInterpreter now have features that dynamic report generation
oriented from knitr. That's clear design goal in my understanding.


3. knitr is licensed under GPL

https://github.com/yihui/knitr


4. Binding knitr library through R language interpreter requires Zeppelin
to be GPL

According to http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL,

"However, when the interpreter is extended to provide “bindings” to other
facilities (often, but not necessarily, libraries), the interpreted program
is effectively linked to the facilities it uses through these bindings. So
if these facilities are released under the GPL, the interpreted program
that uses them must be released in a GPL-compatible way"

In our case, 'interpreter' is R, 'facilities' is knitr.



Please correct me if i'm wrong.

[1]
https://github.com/apache/incubator-zeppelin/pull/208/files#diff-1bb44f3ddb1a6e92144352d8b54084c2R33
[2]
https://github.com/apache/incubator-zeppelin/pull/208/files#diff-1bb44f3ddb1a6e92144352d8b54084c2R54


Thanks,
moon

On Thu, Dec 3, 2015 at 11:03 AM Corneau Damien <co...@gmail.com> wrote:

> Thanks Cos for those answers,
>
> If I'm right you are advising to have a default build that doesn't include
> libraries with conflicting licenses, but have an option to include them for
> users who wants to build the project themselves.
>
> To refer to another thread about decentralizing interpreters, it could even
> be better in that case to have some interpreters separated from the tree,
> and easily pluggable with a release instead of forcing users to build the
> project to use them.
>
>
> On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <co...@apache.org> wrote:
>
> > On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:
> > > Konstantin thank you for getting into this.
> > >
> > >> The best way to go around it is by
> > >> providing a build-time option that will pull such binaries in. But by
> > default
> > >> such libs shouldn't be pulled.
> > >
> > > That is basically how the PR handles this. If the GPL’d interpreter
> > scripts
> > > are missing, there’s no indication at all at build time.  It doesn’t
> try
> > to
> > > download them.
> > >
> > > At runtime, if the user tries to use functionality that would need
> such a
> > > script (i.e., if they type “knitr” to use knitr), we display an error
> > that
> > > says that the functionality is not there because the library is
> missing,
> > and
> > > that the library cannot be provided because it has an incompatible
> > license,
> > > but the user can download it themselves if they want.
> > >
> > > And, in the log, if the logging level is high, they will see a note
> that
> > > some functionality was disabled because the libraries weren’t there.
> > >
> > > To be clear, though, none of these libraries are binaries.  They’re all
> > interpreter scripts.
> >
> > If you ever in a doubt of which licenses could be used for dependncies
> > (not to
> > say about source code) are listed in Category A list of [1]
> >
> > A lot of quesitons discussed here are already covered in the legal FAQ,
> so
> > just check against it if you have any questions.
> >
> > [1] http://www.apache.org/legal/resolved.html#category-a
> >
> > Cos
> >
> > > From: Konstantin Boudnik <co...@apache.org>
> > > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>, dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > > Date: December 2, 2015 at 3:24:50 PM
> > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org
> > >
> > > Subject:  Re: License of KnitRInterpreter Was: Re: contributions
> > impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > Zeppelin
> > >
> > > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:
> > > > I think that what moon means is that:
> > > > If we merge the way it is now, the KnitRInterpreter will be part of
> the
> > > > code base, so it isn't optional at code base level.
> > > >
> > > > To make it even more simple:
> > > > * If the code has the right licensing -> that code can be part of the
> > > > source code, and can be including in a binary release
> > >
> > > We aren't concerned with binary releases - as an Apache community we
> are
> > > voting and releasing source code. If the project wants to provide a
> > binary
> > > release to its users, they are better be warned about inclusion of non
> > > ASL2-friendly licensed bits.
> > >
> > > > * If the code doesn't have the right licensing -> it can't be part of
> > the
> > > > source code, and can't be included in a binary release
> > >
> > > See above.
> > >
> > > > * If the code doesn't have the right licensing but is imported at
> build
> > > > time (libraries for example) -> it is not in the source code, it
> can't
> > be
> > > > included in binary release
> > >
> > > That is unless a user doing it on his own. The best way to go around it
> > is by
> > > providing a build-time option that will pull such binaries in. But by
> > default
> > > such libs shouldn't be pulled.
> > >
> > > Cos
> > >
> > > > So in the case of licensing issues, it would need to be fully
> optional
> > > > (user bring the interpreter in his directory and build Zeppelin with
> > it)
> > > >
> > > >
> > > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <
> amos.elberg@gmail.com>
> > > > wrote:
> > > >
> > > > > Moon let me clarify:
> > > > >
> > > > > Interpreted code doesn’t “link.” The wiki article actually explains
> > it
> > > > > pretty well — https://en.wikipedia.org/wiki/GPL_linking_exception
> > > > > “Linking” against a library means compiling its headers into a
> > binary, the
> > > > > way a C compiler works. The 2008 e-mail Moon distributed, called
> > this the
> > > > > “interpreter exception.”
> > > > >
> > > > > As for whether GPL’d code is a “mandatory dependency,” if knitr is
> > missing
> > > > > the PR will compile, run and test just fine. The user is never
> > prompted to
> > > > > download it. The only effect is, if the user types “knitr” and
> knitr
> > isn’t
> > > > > there, we display an InterpreterError that knitr isn’t there.
> > > > >
> > > > > KnitRInterpreter is not optionally required. so it does not matter
> > KnitR
> > > > > is
> > > > > optionally required or not.
> > > > > Aren’t all interpreters optional? You can turn them on and off in
> the
> > > > > config files.
> > > > >
> > > > > Do you mean that the KnitRInterpreter class gets compiled to
> > bytecode even
> > > > > if knitr is missing? So what? That isn't a mandatory dependency or
> a
> > link.
> > > > >
> > > > > From: moon soo Lee <mo...@apache.org>
> > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > dev@zeppelin.incubator.apache.org>
> > > > > Date: December 2, 2015 at 3:18:00 AM
> > > > > To: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions
> > impasse.
> > > > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > Zeppelin
> > > > >
> > > > > Let me summarize license concern about KnitRInterpreter.
> > > > >
> > > > >
> > > > > Amos's interpretation.
> > > > >
> > > > > KnitR is optionally required by KnitRInterpreter.
> > > > > R dependency in SparkR has no problem. So KnitR should be the same.
> > > > >
> > > > >
> > > > > Moon's interpretation.
> > > > >
> > > > > KnitRInterpreter is not optionally required. so it does not matter
> > KnitR is
> > > > > optionally required or not.
> > > > > R dependency in SparkR is exception of GPL. KnitR is not applied
> that
> > > > > exception.
> > > > >
> > > > >
> > > > > Amos, could you confirm my understanding to your interpretation is
> > correct?
> > > > > If it is not could you clarify it?
> > > > >
> > > > > Thanks,
> > > > > moon
> > > > >
> > > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <
> > amos.elberg@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Just to put the final nail in this, I looked it up.
> > > > > >
> > > > > > I see no evidence of any “compiler exception.”
> > > > > >
> > > > > > There is an exception in the license for the runtime libraries
> > that are
> > > > > > bundled with GCC. See:
> > > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> > > > > >
> > > > > > But no “compiler exception.”
> > > > > >
> > > > > > In fact, I believe this is part of the reason why LLVM was
> created.
> > > > > >
> > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > Date: December 1, 2015 at 8:16:36 PM
> > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > incubator-zeppelin pull
> > > > > > request: R Interpreter for Zeppelin
> > > > > >
> > > > > > Is knitR is commonly considered as a interpreter/compiler? or is
> it
> > > > > > considered as a library routine?
> > > > > >
> > > > > > Thanks,
> > > > > > moon
> > > > > >
> > > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <
> > amos.elberg@gmail.com>
> > > > > > wrote:
> > > > > > Moon - you give this as an explanation of the licensing issue:
> > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > >
> > > > > > According to that, there is an exception in the GPL for
> interpreter
> > > > > > languages. As long as you don’t distribute the code, its fine to
> > talk to
> > > > > > an interpreted language.
> > > > > >
> > > > > > Well, if that’s the case, then the PR plainly does not have a
> > license
> > > > > > issue. It doesn’t distribute any GPL’d R code.
> > > > > >
> > > > > > I’m not sure what’s confusing about this. It seems completely
> > > > > > straightforward.
> > > > > >
> > > > > > Regarding this:
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Amos Elberg
> > > > > > Sent with Airmail
> > > > > >
> > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > Date: December 1, 2015 at 6:48:47 PM
> > > > > >
> > > > > > To: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org
> > > > > >
> > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > incubator-zeppelin pull
> > > > > > request: R Interpreter for Zeppelin
> > > > > >
> > > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <
> > amos.elberg@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > I am going to try to minimize my reaction to Moon’s e-mail.
> > > > > > >
> > > > > > > The tl;dr is this:
> > > > > > >
> > > > > > > The reason we are having this discussion now is that active
> > users of
> > > > > the
> > > > > > > PR — which now has its own user base — went public to complain
> > about
> > > > > > this.
> > > > > >
> > > > > >
> > > > > > > The PR has been tested by an active user base for more than
> three
> > > > > months.
> > > > > > > No-one has been able to identify any specific actual licensing
> > problem,
> > > > > > and
> > > > > > > the PR was prepared based on an extensive, careful review of
> the
> > > > > relevant
> > > > > > > licensing issues and after contacting the relevant people.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > I admire every software that used by user and helping people.
> That
> > > > > includes
> > > > > > your work. But that's not the topic we're in discussion. Active
> > user does
> > > > > > not mean your contribution can ignore the review.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > It is not an explanation for someone who has been ignoring my
> > “how can
> > > > > I
> > > > > > > move this forward…” emails for three months to point the finger
> > and
> > > > > say I
> > > > > > > didn’t contact the right person or file the right report.
> > > > > > >
> > > > > > >
> > > > > > This is also not the topic in this discussion.
> > > > > >
> > > > > >
> > > > > > > The burden for providing an explanation for the inaction is on
> > the PMCC
> > > > > > at
> > > > > > > this point.
> > > > > >
> > > > > > I'm sorry, but the other PRs are passing CI. If it's problem on
> > Zeppelin
> > > > > > > core, why do you think other PRs are passing CI?
> > > > > > > They’re not! I often see comments on PRs to just ignore that CI
> > is
> > > > > > > failing.
> > > > > > >
> > > > > > > One of the most common reasons this PR fails CI, is CI
> times-out
> > > > > > > downloading Spark to install. How could that possibly be caused
> > by the
> > > > > > PR?
> > > > > > >
> > > > > > > It looks to me like the only PRs with changes to the relevant
> > parts of
> > > > > > the
> > > > > > > code — the SparkInterpreter — are being made by the person who
> > wrote
> > > > > the
> > > > > > > testing suite.
> > > > > > >
> > > > > > > So, that would explain why some other PRs pass CI: Neither the
> > > > > > > SparkInterpreter nor the testing suite are stable or robust,
> but
> > since
> > > > > > the
> > > > > > > PRs are coming from the person who wrote both…
> > > > > > >
> > > > > > > And let's say Zeppelin core has problem and that makes your PR
> > fails on
> > > > > > CI
> > > > > > > test. That's possible. But it still does not mean we can merge
> > it with
> > > > > CI
> > > > > > > failing.
> > > > > > >
> > > > > > > It means you should be working with me to figure out why the CI
> > is
> > > > > > failing.
> > > > > > >
> > > > > > > This PR has been tested by an active user base for the past
> three
> > > > > months.
> > > > > > > If CI is continuing to fail, and dozens of hours of effort have
> > not
> > > > > > > resolved the CI issues, then it is time to start considering
> > whether
> > > > > the
> > > > > > > testing suite is part of the problem.
> > > > > > >
> > > > > > > The level of defensiveness about the CI and SparkInterpreter is
> > not
> > > > > > > helping to resolve these issues.
> > > > > > >
> > > > > > > If you think it's problem on Zeppelin core, then file an issue
> > that
> > > > > > > reproduce the problem on Zeppelin core, that might be more
> > efficient
> > > > > than
> > > > > > > keep trying yourself.
> > > > > > > I contacted you numerous times about such issues...
> > > > > > >
> > > > > >
> > > > > >
> > > > > > I remember i commented your issue about CI. but you just keep
> > repeated
> > > > > it's
> > > > > > not your problem but Zeppelin core problem.
> > > > > >
> > > > > > Then please file an issue about the problem you found in Zeppelin
> > Core.
> > > > > > Then everyone will get into the problem.
> > > > > >
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > In my interpretation, KnitRInterpreter is not an optional
> > feature while
> > > > > > it
> > > > > > > is always enabled when compiling Zeppelin and always enabled
> when
> > > > > running
> > > > > > > Zeppelin. And it requires dynamically linked GPL library on
> > runtime.
> > > > > (yes
> > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > >
> > > > > > > Its not always enabled.
> > > > > > > It is not dynamically linked at runtime.
> > > > > > > It will not fail when knitr is missing. If knitr is not
> present,
> > the
> > > > > repl
> > > > > > > interpreter starts and a note is written to to the log that the
> > knitr
> > > > > > > interpreter isn’t available because knitr is not present.
> > > > > > >
> > > > > > > no Apache code can ever call a shell script, on the purpose of
> > dynamic
> > > > > > > linking with GPL library.
> > > > > > > You misunderstand.
> > > > > > >
> > > > > > > The *shell* is GPL'd.
> > > > > > >
> > > > > > > Is Zeppelin “linked" against the GPL’d shell because Zeppelin
> > depends
> > > > > on
> > > > > > a
> > > > > > > shell script to launch?
> > > > > > >
> > > > > > Obviously not.
> > > > > > >
> > > > > > > The interaction with R in the PR is the same.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > Again, bash is one of exceptions of GPL, like other GPL licensed
> > > > > > compiler/interpreter.
> > > > > >
> > > > > > Check here why Bash and R is okay with Apache License.
> > > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > > >
> > > > > > I'm not sure we can apply the same exception for 'using' KnitR.
> > > > > >
> > > > > > My point is not 'KnitR' is optional or not. Point is
> > 'KnitRInterpreter'
> > > > > you
> > > > > > wrote is not an optional feature. Which is clearly not optionally
> > enabled
> > > > > > code and feature. And that depends on KnitR library which is GPL.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > I was guessing SparkR can be still in Apache License even if it
> > is
> > > > > > depends
> > > > > > > on R. Because of GPL licensed compiler generated output is not
> > GPL
> > > > > > license.
> > > > > > > and R is sort of compiler. If you can get answer from Spark
> > community
> > > > > how
> > > > > > > SparkR get managed to stay in Apache License while R is GPL,
> the
> > answer
> > > > > > > might help.
> > > > > > > The description of SparkR is not accurate in any respect. (Do
> > you think
> > > > > > > SparkR is not talking to GPL-licensed libraries?)
> > > > > > >
> > > > > > > I don’t see that any genuine issue is being raised here.
> > > > > > >
> > > > > > > If there is an issue, the burden is on you to identify it.
> > > > > > >
> > > > > > > If i give you one suggestion, Zeppelin committers sometimes ask
> > rebase
> > > > > > the
> > > > > > > contribution branch for some reason. It is not the really the
> > best
> > > > > > > practice, but still okay while most contributions are not
> > including
> > > > > large
> > > > > > > code base changes
> > > > > > > However, your one, has more than 4000 lines of code change. If
> > you
> > > > > rebase
> > > > > > > then review should be started from the beginning, again. So you
> > might
> > > > > > want
> > > > > > > to minimize the rebase your branch.
> > > > > > >
> > > > > > > Are you actually complaining that the problem is that I rebased
> > the
> > > > > code
> > > > > > > during the three-month period when no-one looked at it and
> > Zeppelin
> > > > > went
> > > > > > > through a release?
> > > > > > >
> > > > > > > I cannot take it seriously when you say things like this.
> > > > > > >
> > > > > > > Having to “start from the beginning” cannot be a problem if you
> > never
> > > > > > > actually started the first time...
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > You wanted coordination and cooperation. So i gave you suggestion
> > that
> > > > > > helping review process. For example, your code has been rebased
> > since my
> > > > > > comment and jongyoul's comment. that means committers will need
> to
> > look
> > > > > > from the beginning. That'll require more time. And maybe, i guess
> > that's
> > > > > > not what you want. But If you don't care, feel free to rebase.
> > > > > >
> > > > > > Thanks,
> > > > > > moon
> > > > > >
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > > Date: December 1, 2015 at 4:42:06 AM
> > > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > > dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > incubator-zeppelin
> > > > > pull
> > > > > > > request: R Interpreter for Zeppelin
> > > > > > >
> > > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <
> > amos.elberg@gmail.com>
> > > > > > > wrote:
> > > > > > > Thank you, Cos.
> > > > > > >
> > > > > > > I’d like to briefly address the issues raised by Moon:
> > > > > > >
> > > > > > > 1. This PR does not passes CI
> > > > > > > The CI fails on core Zeppelin, *not* code in this PR.
> > > > > > >
> > > > > > > I’ve been seeking assistance on this since August.
> > > > > > >
> > > > > > > The most common reason is that SparkInterpreter is unable to
> > launch
> > > > > Spark
> > > > > > > and open a Spark Backend. This is necessary to test the PR.
> > > > > > >
> > > > > > > 60+ hours, has been spent adapting and re-basing when the
> > > > > > SparkInterpreter
> > > > > > > architecture changed and broke the PR’s *tests.*
> > > > > > >
> > > > > > >
> > > > > > > I'm sorry, but the other PRs are passing CI. If it's problem on
> > > > > Zeppelin
> > > > > > > core, why do you think other PRs are passing CI?
> > > > > > >
> > > > > > > And let's say Zeppelin core has problem and that makes your PR
> > fails on
> > > > > > CI
> > > > > > > test. That's possible. But it still does not mean we can merge
> > it with
> > > > > CI
> > > > > > > failing.
> > > > > > >
> > > > > > > If you think it's problem on Zeppelin core, then file an issue
> > that
> > > > > > > reproduce the problem on Zeppelin core, that might be more
> > efficient
> > > > > than
> > > > > > > keep trying yourself.
> > > > > > >
> > > > > > >
> > > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > > > > > What license problem *specifically* do you believe may exist?
> > > > > > >
> > > > > > > In preparing the PR, I:
> > > > > > >
> > > > > > > * Reviewed the Apache policy in detail.
> > > > > > >
> > > > > > > * Contacted the FSF to ask their interpretation of the
> “linking”
> > > > > > > provisions of the GPL license.
> > > > > > >
> > > > > > > * Reviewed how other Apache software deals with this issue
> > (e.g., Spark
> > > > > > > talks to R, which is GPL'd).
> > > > > > >
> > > > > > > * No necessary *dependencies* of the PR have license conflicts.
> > In
> > > > > > > several cases, I contacted package authors who agreed to
> > re-issue their
> > > > > > > packages under Apache-compatible licenses. (Usually I had to do
> > a bit
> > > > > of
> > > > > > > coding in exchange…)
> > > > > > >
> > > > > > > * Where the license had to stay GPL, the packages are *not
> > necessary*
> > > > > and
> > > > > > > *not dependencies.* If the optional packages are present, they
> > will be
> > > > > > > used to enable additional functionality. Knitr is an example.
> > The PR
> > > > > will
> > > > > > > compile and run fine without knitr. If knitr is available (it
> is
> > part
> > > > > of
> > > > > > > the most common R distribution), the PR will enable the knitr
> > > > > > interpreter.
> > > > > > >
> > > > > > > * This is exactly how this issue is addressed through the
> Apache
> > > > > > > ecosystem.
> > > > > > > The tl;dr is this: When Apache code is written to talk to
> > libraries
> > > > > that
> > > > > > > may or may not be present on the user’s system, or where it
> > talks to an
> > > > > > API
> > > > > > > but is agnostic about implementation, that is not “linking” in
> a
> > way
> > > > > that
> > > > > > > implicate the anti-linking provision of the GPL.
> > > > > > >
> > > > > > > Otherwise, no Apache code could ever call a shell script! Let
> > alone run
> > > > > > > on Linux, or talk to R.
> > > > > > >
> > > > > > >
> > > > > > > I'm not a legal expert. So following could be wrong.
> > > > > > >
> > > > > > > In my interpretation, KnitRInterpreter is not an optional
> > feature while
> > > > > > it
> > > > > > > is always enabled when compiling Zeppelin and always enabled
> when
> > > > > running
> > > > > > > Zeppelin. And it requires dynamically linked GPL library on
> > runtime.
> > > > > (yes
> > > > > > > it will fail when no KnitR is installed on the system)
> > > > > > >
> > > > > > > And of course, no Apache code can ever call a shell script, on
> > the
> > > > > > purpose
> > > > > > > of dynamic linking with GPL library.
> > > > > > >
> > > > > > > I was guessing SparkR can be still in Apache License even if it
> > is
> > > > > > depends
> > > > > > > on R. Because of GPL licensed compiler generated output is not
> > GPL
> > > > > > license.
> > > > > > > and R is sort of compiler.
> > > > > > >
> > > > > > > If you can get answer from Spark community how SparkR get
> > managed to
> > > > > stay
> > > > > > > in Apache License while R is GPL, the answer might help.
> > > > > > >
> > > > > > >
> > > > > > > 3. Need more time to review.
> > > > > > > Has any reviewer has downloaded the PR or run the demo
> notebook?
> > (Which
> > > > > > > is there for the benefit of reviewers, and isn’t intended to go
> > in
> > > > > final
> > > > > > > distribution.)
> > > > > > >
> > > > > > > How many +1 comments from users would you like to see?
> > > > > > >
> > > > > > > How much time do you believe is required?
> > > > > > >
> > > > > > >
> > > > > > > It all depends on when CI is going to pass, when license
> problem
> > is
> > > > > going
> > > > > > > to be cleared, and when a committer willing to review and
> > responsible
> > > > > to
> > > > > > > commit your contribution.
> > > > > > >
> > > > > > >
> > > > > > > 1. Large code base change
> > > > > > > Large code base changes require coordination and cooperation.
> > This PR
> > > > > > > necessarily implicates the build scripts, testing code, the
> > > > > > > SparkInterpreter, etc.
> > > > > > >
> > > > > > > I have been seeking to coordinate since August.
> > > > > > >
> > > > > > > I continue to stand ready to do so.
> > > > > > >
> > > > > > > -Amos
> > > > > > >
> > > > > > >
> > > > > > > If i give you one suggestion, Zeppelin committers sometimes ask
> > rebase
> > > > > > the
> > > > > > > contribution branch for some reason. It is not the really the
> > best
> > > > > > > practice, but still okay while most contributions are not
> > including
> > > > > large
> > > > > > > code base changes.
> > > > > > >
> > > > > > > However, your one, has more than 4000 lines of code change. If
> > you
> > > > > rebase
> > > > > > > then review should be started from the beginning, again. So you
> > might
> > > > > > want
> > > > > > > to minimize the rebase your branch.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > moon
> > > > > > >
> > > > > > >
> > > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > > Date: December 1, 2015 at 1:34:19 AM
> > > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > > dev@zeppelin.incubator.apache.org
> > > > > > >
> > > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> > incubator-zeppelin
> > > > > pull
> > > > > > > request: R Interpreter for Zeppelin
> > > > > > >
> > > > > > > Hi Cos,
> > > > > > >
> > > > > > > Thanks for opening a discussion.
> > > > > > > My answer about 'Why this PR is open for three months' is
> > > > > > >
> > > > > > > 1. This PR does not passes CI
> > > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > > > > > 3. Need more time to review.
> > > > > > >
> > > > > > > It's my personal answer, other committers may have different
> > opinion.
> > > > > > >
> > > > > > >
> > > > > > > I think the question should be generalized. Because this PR is
> > not the
> > > > > > only
> > > > > > > PR that is in impasse. There're more. For example
> > > > > > >
> > > > > > > Here's some examples that PR is not been merged.
> > > > > > > https://github.com/apache/incubator-zeppelin/pull/53,
> > > > > > > https://github.com/apache/incubator-zeppelin/pull/60
> > > > > > > and so on.
> > > > > > >
> > > > > > > I can categorize the cases, based on experience of involving
> > Zeppelin
> > > > > > > community from the beginning.
> > > > > > >
> > > > > > > 1. Large code base change
> > > > > > >
> > > > > > > When contribution has large code base changes, it tend to take
> > more
> > > > > time
> > > > > > to
> > > > > > > review and merged. Normally, most contributions merged in
> 1day~1
> > week.
> > > > > > > But some contribution has large code base changes take 2~4
> > weeks. Few
> > > > > > > contribution that has very large code base change take months.
> > > > > > >
> > > > > > > 2. Communication lost
> > > > > > >
> > > > > > > Sometimes, committer is not responding, or contributor is not
> > > > > responding.
> > > > > > >
> > > > > > > 3. Opinion diverges
> > > > > > >
> > > > > > > For some changes, included in contribution. When people have
> > different
> > > > > > > opinion and it continue to diverges, those PRs are not been
> > merged.
> > > > > > >
> > > > > > >
> > > > > > > I think having a guide such as ping other committer when a
> > committer is
> > > > > > not
> > > > > > > responding, and divide contribution into small peaces if
> > possible,
> > > > > would
> > > > > > > help most of the cases.
> > > > > > > Of course committer have to pay attention more to the
> > contribution and
> > > > > > > helping people. That's the first one.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > moon
> > > > > > >
> > > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <
> > cos@apache.org>
> > > > > > wrote:
> > > > > > >
> > > > > > > > To make sure we're on the same page, here are two threads
> that
> > I
> > > > > found
> > > > > > > > related
> > > > > > > > to this topic.
> > > > > > > >
> > > > > > > > Thread 1:
> > > > > > > > Subject: R?
> > > > > > > > Started on: July 1, 2015
> > > > > > > >
> > > > > > > > Thread 2:
> > > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R
> > Interpreter for
> > > > > > > > Zeppelin
> > > > > > > > Started on: August 13, 2015
> > > > > > > >
> > > > > > > > If you want to fetch these from our archive send emails to
> > > > > > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > > > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > > > > > >
> > > > > > > > Cos
> > > > > > > >
> > > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > > > > > > > Guys,
> > > > > > > > >
> > > > > > > > > While catching up on my emails from the last a couple of
> > weeks,
> > > > > this
> > > > > > > > thread
> > > > > > > > > caught my attention. I am not normally paying much
> attention
> > to the
> > > > > > > code
> > > > > > > > > reviews traffic from GH, but this one is pretty different
> as
> > it
> > > > > spans
> > > > > > > > three
> > > > > > > > > months and counting.
> > > > > > > > >
> > > > > > > > > First, here are my five cents:
> > > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to
> be
> > > > > > > > contributed to
> > > > > > > > > an ASF project this file should simply contain ASL2 text,
> > like in
> > > > > [1]
> > > > > > > > > - r/pom.xml perhaps shouldn't contain a separate
> <developers>
> > > > > > section,
> > > > > > > > but
> > > > > > > > > Zeppelin might have different guidelines on it. Say, Bigtop
> > doesn't
> > > > > > > > > maintain this in the source code, but we have the list of
> > all the
> > > > > > > > > committers on the project's site [2] Every new committer's
> > first
> > > > > > > > commit is
> > > > > > > > > to update the team page ;)
> > > > > > > > > - comments like in
> > > > > > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > > > > > >
> > > > > > > > > +/**
> > > > > > > > > + * Created by aelberg on 7/28/15.
> > > > > > > > > + */
> > > > > > > > >
> > > > > > > > > is better to be removed. It has been already discussed here
> > [3].
> > > > > And
> > > > > > > > the
> > > > > > > > > initial discussion on the topic [4] was linked as well
> > > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if
> > this is
> > > > > > > > R-specific
> > > > > > > > > stuff - I have no idea about R, honestly, but
> > > > > > > > >
> > > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > > > > > > ...
> > > > > > > > > +Author: David B. Dahl
> > > > > > > > >
> > > > > > > > > shouldn't be here, IMO. Normally, if some additional
> > licenses are
> > > > > > > > used,
> > > > > > > > > they have to be listed in the top-level NOTICE file
> (already
> > > > > there).
> > > > > > > > >
> > > > > > > > > - I am not going to offer any comments on the technical
> > merits of
> > > > > the
> > > > > > > > patch,
> > > > > > > > > as I haven't tried it personally. However, I don't see any
> > serious
> > > > > > > > > technical objections to the functionality in question.
> > > > > > > > >
> > > > > > > > > So, the question is - why the PR is open for three months?
> I
> > hasn't
> > > > > > > been
> > > > > > > > able
> > > > > > > > > to get a clear answer. What I found out though is pretty
> > > > > unsettling,
> > > > > > > > really.
> > > > > > > > > The communication on the topic is almost non-existent,
> > except for
> > > > > > this
> > > > > > > > sparse
> > > > > > > > > and bitter thread in the GH.
> > > > > > > > >
> > > > > > > > > I would love to hear from the committers what's stopping
> the
> > > > > > acceptance
> > > > > > > > of the
> > > > > > > > > code, besides of the minor issues I've mentioned earlier?
> > What are
> > > > > > the
> > > > > > > > reasons for it?
> > > > > > > > > Is there anything wrong with it?
> > > > > > > > > One of the responsibilities of the committers is to make
> > sure the
> > > > > > > > > contributions are reviewed; new contributors are guided and
> > do
> > > > > > > > understand how
> > > > > > > > > the project ticks. The easy feedback flow attracts new
> > people,
> > > > > > allowing
> > > > > > > > the
> > > > > > > > > community to grow and thrive.
> > > > > > > > >
> > > > > > > > > I am asking _explicitely_ not to re-start the bickering I
> > have
> > > > > > already
> > > > > > > > > seen. At this point I am interested in the purely technical
> > side of
> > > > > > > this.
> > > > > > > > >
> > > > > > > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > > > > > [3]
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > > > > > [4] http://s.apache.org/iZl
> > > > > > > > >
> > > > > > > > > With regards,
> > > > > > > > > Cos
> > > > > > > > >
> > > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > > > > > Github user elbamos commented on the pull request:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > > > > > >
> > > > > > > > > > The current push should resolve some issues with changes
> > in the
> > > > > > > > > > Spark-Zeppelin interface that had created issues for
> > users, as
> > > > > > > > well as
> > > > > > > > > > support for 1.5.1.
> > > > > > > > > >
> > > > > > > > > > Knitr support is improved, and the reason for a separate
> > knitr
> > > > > > > > interpreter may be clearer now.
> > > > > > > > > >
> > > > > > > > > > The amount of code borrowed from rscala is reduced.
> > > > > > > > > >
> > > > > > > > > > I did not address issues with @author tags, or files
> under
> > the R/
> > > > > > > > > > folder. The reason is, to be blunt, I don't understand
> > what the
> > > > > > > > precise
> > > > > > > > > > concerns actually are.
> > > > > > > > > >
> > > > > > > > > > Please note that I changed .travis.yml to only use spark
> > 1.4 and
> > > > > > > > later.
> > > > > > > > > > I'm sure there is a better way to do it, but I'm just not
> > in a
> > > > > > > > position
> > > > > > > > > > to try to figure out the entire Zeppelin build process.
> > > > > > > > > >
> > > > > > > > > > Integrating this with the rest of zeppelin is going to
> > take some
> > > > > > > > work
> > > > > > > > > > regarding pom's, travis, and so forth. I can do a lot of
> > that,
> > > > > > > > but I'm
> > > > > > > > > > going to need to discuss it with the people who have been
> > > > > "owning"
> > > > > > > > those
> > > > > > > > > > files. There are just too many moving pieces here.
> > > > > > > > > >
> > > > > > > > > > Because of the size of this (which is, unfortunately,
> > necessary),
> > > > > > > > > > posting here is probably not the most efficient way. That
> > is also
> > > > > > > > true
> > > > > > > > > > because certain people chose to use this PR as a forum to
> > air
> > > > > other
> > > > > > > > > > issues. Therefore, it would be better if reviewers e-mail
> > me
> > > > > > > > directly.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> >
>

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Corneau Damien <co...@gmail.com>]

Thanks Cos for those answers,

If I'm right you are advising to have a default build that doesn't include
libraries with conflicting licenses, but have an option to include them for
users who wants to build the project themselves.

To refer to another thread about decentralizing interpreters, it could even
be better in that case to have some interpreters separated from the tree,
and easily pluggable with a release instead of forcing users to build the
project to use them.


On Thu, Dec 3, 2015 at 9:26 AM, Konstantin Boudnik <co...@apache.org> wrote:

> On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:
> > Konstantin thank you for getting into this.
> >
> >> The best way to go around it is by
> >> providing a build-time option that will pull such binaries in. But by
> default
> >> such libs shouldn't be pulled.
> >
> > That is basically how the PR handles this. If the GPL’d interpreter
> scripts
> > are missing, there’s no indication at all at build time.  It doesn’t try
> to
> > download them.
> >
> > At runtime, if the user tries to use functionality that would need such a
> > script (i.e., if they type “knitr” to use knitr), we display an error
> that
> > says that the functionality is not there because the library is missing,
> and
> > that the library cannot be provided because it has an incompatible
> license,
> > but the user can download it themselves if they want.
> >
> > And, in the log, if the logging level is high, they will see a note that
> > some functionality was disabled because the libraries weren’t there.
> >
> > To be clear, though, none of these libraries are binaries.  They’re all
> interpreter scripts.
>
> If you ever in a doubt of which licenses could be used for dependncies
> (not to
> say about source code) are listed in Category A list of [1]
>
> A lot of quesitons discussed here are already covered in the legal FAQ, so
> just check against it if you have any questions.
>
> [1] http://www.apache.org/legal/resolved.html#category-a
>
> Cos
>
> > From: Konstantin Boudnik <co...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>, dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> > Date: December 2, 2015 at 3:24:50 PM
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > Subject:  Re: License of KnitRInterpreter Was: Re: contributions
> impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for
> Zeppelin
> >
> > On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:
> > > I think that what moon means is that:
> > > If we merge the way it is now, the KnitRInterpreter will be part of the
> > > code base, so it isn't optional at code base level.
> > >
> > > To make it even more simple:
> > > * If the code has the right licensing -> that code can be part of the
> > > source code, and can be including in a binary release
> >
> > We aren't concerned with binary releases - as an Apache community we are
> > voting and releasing source code. If the project wants to provide a
> binary
> > release to its users, they are better be warned about inclusion of non
> > ASL2-friendly licensed bits.
> >
> > > * If the code doesn't have the right licensing -> it can't be part of
> the
> > > source code, and can't be included in a binary release
> >
> > See above.
> >
> > > * If the code doesn't have the right licensing but is imported at build
> > > time (libraries for example) -> it is not in the source code, it can't
> be
> > > included in binary release
> >
> > That is unless a user doing it on his own. The best way to go around it
> is by
> > providing a build-time option that will pull such binaries in. But by
> default
> > such libs shouldn't be pulled.
> >
> > Cos
> >
> > > So in the case of licensing issues, it would need to be fully optional
> > > (user bring the interpreter in his directory and build Zeppelin with
> it)
> > >
> > >
> > > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <am...@gmail.com>
> > > wrote:
> > >
> > > > Moon let me clarify:
> > > >
> > > > Interpreted code doesn’t “link.” The wiki article actually explains
> it
> > > > pretty well — https://en.wikipedia.org/wiki/GPL_linking_exception
> > > > “Linking” against a library means compiling its headers into a
> binary, the
> > > > way a C compiler works. The 2008 e-mail Moon distributed, called
> this the
> > > > “interpreter exception.”
> > > >
> > > > As for whether GPL’d code is a “mandatory dependency,” if knitr is
> missing
> > > > the PR will compile, run and test just fine. The user is never
> prompted to
> > > > download it. The only effect is, if the user types “knitr” and knitr
> isn’t
> > > > there, we display an InterpreterError that knitr isn’t there.
> > > >
> > > > KnitRInterpreter is not optionally required. so it does not matter
> KnitR
> > > > is
> > > > optionally required or not.
> > > > Aren’t all interpreters optional? You can turn them on and off in the
> > > > config files.
> > > >
> > > > Do you mean that the KnitRInterpreter class gets compiled to
> bytecode even
> > > > if knitr is missing? So what? That isn't a mandatory dependency or a
> link.
> > > >
> > > > From: moon soo Lee <mo...@apache.org>
> > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > dev@zeppelin.incubator.apache.org>
> > > > Date: December 2, 2015 at 3:18:00 AM
> > > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> > > > Subject: Re: License of KnitRInterpreter Was: Re: contributions
> impasse.
> > > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for
> Zeppelin
> > > >
> > > > Let me summarize license concern about KnitRInterpreter.
> > > >
> > > >
> > > > Amos's interpretation.
> > > >
> > > > KnitR is optionally required by KnitRInterpreter.
> > > > R dependency in SparkR has no problem. So KnitR should be the same.
> > > >
> > > >
> > > > Moon's interpretation.
> > > >
> > > > KnitRInterpreter is not optionally required. so it does not matter
> KnitR is
> > > > optionally required or not.
> > > > R dependency in SparkR is exception of GPL. KnitR is not applied that
> > > > exception.
> > > >
> > > >
> > > > Amos, could you confirm my understanding to your interpretation is
> correct?
> > > > If it is not could you clarify it?
> > > >
> > > > Thanks,
> > > > moon
> > > >
> > > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <
> amos.elberg@gmail.com>
> > > > wrote:
> > > >
> > > > > Just to put the final nail in this, I looked it up.
> > > > >
> > > > > I see no evidence of any “compiler exception.”
> > > > >
> > > > > There is an exception in the license for the runtime libraries
> that are
> > > > > bundled with GCC. See:
> > > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> > > > >
> > > > > But no “compiler exception.”
> > > > >
> > > > > In fact, I believe this is part of the reason why LLVM was created.
> > > > >
> > > > > From: moon soo Lee <mo...@apache.org>
> > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > Date: December 1, 2015 at 8:16:36 PM
> > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> > > > > Subject: Re: contributions impasse. Was: [GitHub]
> incubator-zeppelin pull
> > > > > request: R Interpreter for Zeppelin
> > > > >
> > > > > Is knitR is commonly considered as a interpreter/compiler? or is it
> > > > > considered as a library routine?
> > > > >
> > > > > Thanks,
> > > > > moon
> > > > >
> > > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <
> amos.elberg@gmail.com>
> > > > > wrote:
> > > > > Moon - you give this as an explanation of the licensing issue:
> > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > >
> > > > > According to that, there is an exception in the GPL for interpreter
> > > > > languages. As long as you don’t distribute the code, its fine to
> talk to
> > > > > an interpreted language.
> > > > >
> > > > > Well, if that’s the case, then the PR plainly does not have a
> license
> > > > > issue. It doesn’t distribute any GPL’d R code.
> > > > >
> > > > > I’m not sure what’s confusing about this. It seems completely
> > > > > straightforward.
> > > > >
> > > > > Regarding this:
> > > > >
> > > > >
> > > > > --
> > > > > Amos Elberg
> > > > > Sent with Airmail
> > > > >
> > > > > From: moon soo Lee <mo...@apache.org>
> > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > dev@zeppelin.incubator.apache.org>
> > > > > Date: December 1, 2015 at 6:48:47 PM
> > > > >
> > > > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org
> > > > >
> > > > > Subject: Re: contributions impasse. Was: [GitHub]
> incubator-zeppelin pull
> > > > > request: R Interpreter for Zeppelin
> > > > >
> > > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <
> amos.elberg@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > I am going to try to minimize my reaction to Moon’s e-mail.
> > > > > >
> > > > > > The tl;dr is this:
> > > > > >
> > > > > > The reason we are having this discussion now is that active
> users of
> > > > the
> > > > > > PR — which now has its own user base — went public to complain
> about
> > > > > this.
> > > > >
> > > > >
> > > > > > The PR has been tested by an active user base for more than three
> > > > months.
> > > > > > No-one has been able to identify any specific actual licensing
> problem,
> > > > > and
> > > > > > the PR was prepared based on an extensive, careful review of the
> > > > relevant
> > > > > > licensing issues and after contacting the relevant people.
> > > > > >
> > > > > >
> > > > >
> > > > > I admire every software that used by user and helping people. That
> > > > includes
> > > > > your work. But that's not the topic we're in discussion. Active
> user does
> > > > > not mean your contribution can ignore the review.
> > > > >
> > > > >
> > > > >
> > > > > > It is not an explanation for someone who has been ignoring my
> “how can
> > > > I
> > > > > > move this forward…” emails for three months to point the finger
> and
> > > > say I
> > > > > > didn’t contact the right person or file the right report.
> > > > > >
> > > > > >
> > > > > This is also not the topic in this discussion.
> > > > >
> > > > >
> > > > > > The burden for providing an explanation for the inaction is on
> the PMCC
> > > > > at
> > > > > > this point.
> > > > >
> > > > > I'm sorry, but the other PRs are passing CI. If it's problem on
> Zeppelin
> > > > > > core, why do you think other PRs are passing CI?
> > > > > > They’re not! I often see comments on PRs to just ignore that CI
> is
> > > > > > failing.
> > > > > >
> > > > > > One of the most common reasons this PR fails CI, is CI times-out
> > > > > > downloading Spark to install. How could that possibly be caused
> by the
> > > > > PR?
> > > > > >
> > > > > > It looks to me like the only PRs with changes to the relevant
> parts of
> > > > > the
> > > > > > code — the SparkInterpreter — are being made by the person who
> wrote
> > > > the
> > > > > > testing suite.
> > > > > >
> > > > > > So, that would explain why some other PRs pass CI: Neither the
> > > > > > SparkInterpreter nor the testing suite are stable or robust, but
> since
> > > > > the
> > > > > > PRs are coming from the person who wrote both…
> > > > > >
> > > > > > And let's say Zeppelin core has problem and that makes your PR
> fails on
> > > > > CI
> > > > > > test. That's possible. But it still does not mean we can merge
> it with
> > > > CI
> > > > > > failing.
> > > > > >
> > > > > > It means you should be working with me to figure out why the CI
> is
> > > > > failing.
> > > > > >
> > > > > > This PR has been tested by an active user base for the past three
> > > > months.
> > > > > > If CI is continuing to fail, and dozens of hours of effort have
> not
> > > > > > resolved the CI issues, then it is time to start considering
> whether
> > > > the
> > > > > > testing suite is part of the problem.
> > > > > >
> > > > > > The level of defensiveness about the CI and SparkInterpreter is
> not
> > > > > > helping to resolve these issues.
> > > > > >
> > > > > > If you think it's problem on Zeppelin core, then file an issue
> that
> > > > > > reproduce the problem on Zeppelin core, that might be more
> efficient
> > > > than
> > > > > > keep trying yourself.
> > > > > > I contacted you numerous times about such issues...
> > > > > >
> > > > >
> > > > >
> > > > > I remember i commented your issue about CI. but you just keep
> repeated
> > > > it's
> > > > > not your problem but Zeppelin core problem.
> > > > >
> > > > > Then please file an issue about the problem you found in Zeppelin
> Core.
> > > > > Then everyone will get into the problem.
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > > In my interpretation, KnitRInterpreter is not an optional
> feature while
> > > > > it
> > > > > > is always enabled when compiling Zeppelin and always enabled when
> > > > running
> > > > > > Zeppelin. And it requires dynamically linked GPL library on
> runtime.
> > > > (yes
> > > > > > it will fail when no KnitR is installed on the system)
> > > > > >
> > > > > > Its not always enabled.
> > > > > > It is not dynamically linked at runtime.
> > > > > > It will not fail when knitr is missing. If knitr is not present,
> the
> > > > repl
> > > > > > interpreter starts and a note is written to to the log that the
> knitr
> > > > > > interpreter isn’t available because knitr is not present.
> > > > > >
> > > > > > no Apache code can ever call a shell script, on the purpose of
> dynamic
> > > > > > linking with GPL library.
> > > > > > You misunderstand.
> > > > > >
> > > > > > The *shell* is GPL'd.
> > > > > >
> > > > > > Is Zeppelin “linked" against the GPL’d shell because Zeppelin
> depends
> > > > on
> > > > > a
> > > > > > shell script to launch?
> > > > > >
> > > > > Obviously not.
> > > > > >
> > > > > > The interaction with R in the PR is the same.
> > > > > >
> > > > > >
> > > > >
> > > > > Again, bash is one of exceptions of GPL, like other GPL licensed
> > > > > compiler/interpreter.
> > > > >
> > > > > Check here why Bash and R is okay with Apache License.
> > > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > > > >
> > > > > I'm not sure we can apply the same exception for 'using' KnitR.
> > > > >
> > > > > My point is not 'KnitR' is optional or not. Point is
> 'KnitRInterpreter'
> > > > you
> > > > > wrote is not an optional feature. Which is clearly not optionally
> enabled
> > > > > code and feature. And that depends on KnitR library which is GPL.
> > > > >
> > > > >
> > > > >
> > > > > > I was guessing SparkR can be still in Apache License even if it
> is
> > > > > depends
> > > > > > on R. Because of GPL licensed compiler generated output is not
> GPL
> > > > > license.
> > > > > > and R is sort of compiler. If you can get answer from Spark
> community
> > > > how
> > > > > > SparkR get managed to stay in Apache License while R is GPL, the
> answer
> > > > > > might help.
> > > > > > The description of SparkR is not accurate in any respect. (Do
> you think
> > > > > > SparkR is not talking to GPL-licensed libraries?)
> > > > > >
> > > > > > I don’t see that any genuine issue is being raised here.
> > > > > >
> > > > > > If there is an issue, the burden is on you to identify it.
> > > > > >
> > > > > > If i give you one suggestion, Zeppelin committers sometimes ask
> rebase
> > > > > the
> > > > > > contribution branch for some reason. It is not the really the
> best
> > > > > > practice, but still okay while most contributions are not
> including
> > > > large
> > > > > > code base changes
> > > > > > However, your one, has more than 4000 lines of code change. If
> you
> > > > rebase
> > > > > > then review should be started from the beginning, again. So you
> might
> > > > > want
> > > > > > to minimize the rebase your branch.
> > > > > >
> > > > > > Are you actually complaining that the problem is that I rebased
> the
> > > > code
> > > > > > during the three-month period when no-one looked at it and
> Zeppelin
> > > > went
> > > > > > through a release?
> > > > > >
> > > > > > I cannot take it seriously when you say things like this.
> > > > > >
> > > > > > Having to “start from the beginning” cannot be a problem if you
> never
> > > > > > actually started the first time...
> > > > > >
> > > > > >
> > > > >
> > > > > You wanted coordination and cooperation. So i gave you suggestion
> that
> > > > > helping review process. For example, your code has been rebased
> since my
> > > > > comment and jongyoul's comment. that means committers will need to
> look
> > > > > from the beginning. That'll require more time. And maybe, i guess
> that's
> > > > > not what you want. But If you don't care, feel free to rebase.
> > > > >
> > > > > Thanks,
> > > > > moon
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > Reply: moon soo Lee <mo...@apache.org>
> > > > > > Date: December 1, 2015 at 4:42:06 AM
> > > > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > > > dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> incubator-zeppelin
> > > > pull
> > > > > > request: R Interpreter for Zeppelin
> > > > > >
> > > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <
> amos.elberg@gmail.com>
> > > > > > wrote:
> > > > > > Thank you, Cos.
> > > > > >
> > > > > > I’d like to briefly address the issues raised by Moon:
> > > > > >
> > > > > > 1. This PR does not passes CI
> > > > > > The CI fails on core Zeppelin, *not* code in this PR.
> > > > > >
> > > > > > I’ve been seeking assistance on this since August.
> > > > > >
> > > > > > The most common reason is that SparkInterpreter is unable to
> launch
> > > > Spark
> > > > > > and open a Spark Backend. This is necessary to test the PR.
> > > > > >
> > > > > > 60+ hours, has been spent adapting and re-basing when the
> > > > > SparkInterpreter
> > > > > > architecture changed and broke the PR’s *tests.*
> > > > > >
> > > > > >
> > > > > > I'm sorry, but the other PRs are passing CI. If it's problem on
> > > > Zeppelin
> > > > > > core, why do you think other PRs are passing CI?
> > > > > >
> > > > > > And let's say Zeppelin core has problem and that makes your PR
> fails on
> > > > > CI
> > > > > > test. That's possible. But it still does not mean we can merge
> it with
> > > > CI
> > > > > > failing.
> > > > > >
> > > > > > If you think it's problem on Zeppelin core, then file an issue
> that
> > > > > > reproduce the problem on Zeppelin core, that might be more
> efficient
> > > > than
> > > > > > keep trying yourself.
> > > > > >
> > > > > >
> > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > > > > What license problem *specifically* do you believe may exist?
> > > > > >
> > > > > > In preparing the PR, I:
> > > > > >
> > > > > > * Reviewed the Apache policy in detail.
> > > > > >
> > > > > > * Contacted the FSF to ask their interpretation of the “linking”
> > > > > > provisions of the GPL license.
> > > > > >
> > > > > > * Reviewed how other Apache software deals with this issue
> (e.g., Spark
> > > > > > talks to R, which is GPL'd).
> > > > > >
> > > > > > * No necessary *dependencies* of the PR have license conflicts.
> In
> > > > > > several cases, I contacted package authors who agreed to
> re-issue their
> > > > > > packages under Apache-compatible licenses. (Usually I had to do
> a bit
> > > > of
> > > > > > coding in exchange…)
> > > > > >
> > > > > > * Where the license had to stay GPL, the packages are *not
> necessary*
> > > > and
> > > > > > *not dependencies.* If the optional packages are present, they
> will be
> > > > > > used to enable additional functionality. Knitr is an example.
> The PR
> > > > will
> > > > > > compile and run fine without knitr. If knitr is available (it is
> part
> > > > of
> > > > > > the most common R distribution), the PR will enable the knitr
> > > > > interpreter.
> > > > > >
> > > > > > * This is exactly how this issue is addressed through the Apache
> > > > > > ecosystem.
> > > > > > The tl;dr is this: When Apache code is written to talk to
> libraries
> > > > that
> > > > > > may or may not be present on the user’s system, or where it
> talks to an
> > > > > API
> > > > > > but is agnostic about implementation, that is not “linking” in a
> way
> > > > that
> > > > > > implicate the anti-linking provision of the GPL.
> > > > > >
> > > > > > Otherwise, no Apache code could ever call a shell script! Let
> alone run
> > > > > > on Linux, or talk to R.
> > > > > >
> > > > > >
> > > > > > I'm not a legal expert. So following could be wrong.
> > > > > >
> > > > > > In my interpretation, KnitRInterpreter is not an optional
> feature while
> > > > > it
> > > > > > is always enabled when compiling Zeppelin and always enabled when
> > > > running
> > > > > > Zeppelin. And it requires dynamically linked GPL library on
> runtime.
> > > > (yes
> > > > > > it will fail when no KnitR is installed on the system)
> > > > > >
> > > > > > And of course, no Apache code can ever call a shell script, on
> the
> > > > > purpose
> > > > > > of dynamic linking with GPL library.
> > > > > >
> > > > > > I was guessing SparkR can be still in Apache License even if it
> is
> > > > > depends
> > > > > > on R. Because of GPL licensed compiler generated output is not
> GPL
> > > > > license.
> > > > > > and R is sort of compiler.
> > > > > >
> > > > > > If you can get answer from Spark community how SparkR get
> managed to
> > > > stay
> > > > > > in Apache License while R is GPL, the answer might help.
> > > > > >
> > > > > >
> > > > > > 3. Need more time to review.
> > > > > > Has any reviewer has downloaded the PR or run the demo notebook?
> (Which
> > > > > > is there for the benefit of reviewers, and isn’t intended to go
> in
> > > > final
> > > > > > distribution.)
> > > > > >
> > > > > > How many +1 comments from users would you like to see?
> > > > > >
> > > > > > How much time do you believe is required?
> > > > > >
> > > > > >
> > > > > > It all depends on when CI is going to pass, when license problem
> is
> > > > going
> > > > > > to be cleared, and when a committer willing to review and
> responsible
> > > > to
> > > > > > commit your contribution.
> > > > > >
> > > > > >
> > > > > > 1. Large code base change
> > > > > > Large code base changes require coordination and cooperation.
> This PR
> > > > > > necessarily implicates the build scripts, testing code, the
> > > > > > SparkInterpreter, etc.
> > > > > >
> > > > > > I have been seeking to coordinate since August.
> > > > > >
> > > > > > I continue to stand ready to do so.
> > > > > >
> > > > > > -Amos
> > > > > >
> > > > > >
> > > > > > If i give you one suggestion, Zeppelin committers sometimes ask
> rebase
> > > > > the
> > > > > > contribution branch for some reason. It is not the really the
> best
> > > > > > practice, but still okay while most contributions are not
> including
> > > > large
> > > > > > code base changes.
> > > > > >
> > > > > > However, your one, has more than 4000 lines of code change. If
> you
> > > > rebase
> > > > > > then review should be started from the beginning, again. So you
> might
> > > > > want
> > > > > > to minimize the rebase your branch.
> > > > > >
> > > > > > Thanks,
> > > > > > moon
> > > > > >
> > > > > >
> > > > > > From: moon soo Lee <mo...@apache.org>
> > > > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > > > dev@zeppelin.incubator.apache.org>
> > > > > > Date: December 1, 2015 at 1:34:19 AM
> > > > > > To: dev@zeppelin.incubator.apache.org <
> > > > dev@zeppelin.incubator.apache.org
> > > > > >
> > > > > > Subject: Re: contributions impasse. Was: [GitHub]
> incubator-zeppelin
> > > > pull
> > > > > > request: R Interpreter for Zeppelin
> > > > > >
> > > > > > Hi Cos,
> > > > > >
> > > > > > Thanks for opening a discussion.
> > > > > > My answer about 'Why this PR is open for three months' is
> > > > > >
> > > > > > 1. This PR does not passes CI
> > > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > > > > 3. Need more time to review.
> > > > > >
> > > > > > It's my personal answer, other committers may have different
> opinion.
> > > > > >
> > > > > >
> > > > > > I think the question should be generalized. Because this PR is
> not the
> > > > > only
> > > > > > PR that is in impasse. There're more. For example
> > > > > >
> > > > > > Here's some examples that PR is not been merged.
> > > > > > https://github.com/apache/incubator-zeppelin/pull/53,
> > > > > > https://github.com/apache/incubator-zeppelin/pull/60
> > > > > > and so on.
> > > > > >
> > > > > > I can categorize the cases, based on experience of involving
> Zeppelin
> > > > > > community from the beginning.
> > > > > >
> > > > > > 1. Large code base change
> > > > > >
> > > > > > When contribution has large code base changes, it tend to take
> more
> > > > time
> > > > > to
> > > > > > review and merged. Normally, most contributions merged in 1day~1
> week.
> > > > > > But some contribution has large code base changes take 2~4
> weeks. Few
> > > > > > contribution that has very large code base change take months.
> > > > > >
> > > > > > 2. Communication lost
> > > > > >
> > > > > > Sometimes, committer is not responding, or contributor is not
> > > > responding.
> > > > > >
> > > > > > 3. Opinion diverges
> > > > > >
> > > > > > For some changes, included in contribution. When people have
> different
> > > > > > opinion and it continue to diverges, those PRs are not been
> merged.
> > > > > >
> > > > > >
> > > > > > I think having a guide such as ping other committer when a
> committer is
> > > > > not
> > > > > > responding, and divide contribution into small peaces if
> possible,
> > > > would
> > > > > > help most of the cases.
> > > > > > Of course committer have to pay attention more to the
> contribution and
> > > > > > helping people. That's the first one.
> > > > > >
> > > > > > Thanks,
> > > > > > moon
> > > > > >
> > > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <
> cos@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > To make sure we're on the same page, here are two threads that
> I
> > > > found
> > > > > > > related
> > > > > > > to this topic.
> > > > > > >
> > > > > > > Thread 1:
> > > > > > > Subject: R?
> > > > > > > Started on: July 1, 2015
> > > > > > >
> > > > > > > Thread 2:
> > > > > > > Subject: [GitHub] incubator-zeppelin pull request: R
> Interpreter for
> > > > > > > Zeppelin
> > > > > > > Started on: August 13, 2015
> > > > > > >
> > > > > > > If you want to fetch these from our archive send emails to
> > > > > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > > > > >
> > > > > > > Cos
> > > > > > >
> > > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > > > > > > Guys,
> > > > > > > >
> > > > > > > > While catching up on my emails from the last a couple of
> weeks,
> > > > this
> > > > > > > thread
> > > > > > > > caught my attention. I am not normally paying much attention
> to the
> > > > > > code
> > > > > > > > reviews traffic from GH, but this one is pretty different as
> it
> > > > spans
> > > > > > > three
> > > > > > > > months and counting.
> > > > > > > >
> > > > > > > > First, here are my five cents:
> > > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > > > > > > contributed to
> > > > > > > > an ASF project this file should simply contain ASL2 text,
> like in
> > > > [1]
> > > > > > > > - r/pom.xml perhaps shouldn't contain a separate <developers>
> > > > > section,
> > > > > > > but
> > > > > > > > Zeppelin might have different guidelines on it. Say, Bigtop
> doesn't
> > > > > > > > maintain this in the source code, but we have the list of
> all the
> > > > > > > > committers on the project's site [2] Every new committer's
> first
> > > > > > > commit is
> > > > > > > > to update the team page ;)
> > > > > > > > - comments like in
> > > > > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > > > > >
> > > > > > > > +/**
> > > > > > > > + * Created by aelberg on 7/28/15.
> > > > > > > > + */
> > > > > > > >
> > > > > > > > is better to be removed. It has been already discussed here
> [3].
> > > > And
> > > > > > > the
> > > > > > > > initial discussion on the topic [4] was linked as well
> > > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if
> this is
> > > > > > > R-specific
> > > > > > > > stuff - I have no idea about R, honestly, but
> > > > > > > >
> > > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > > > > > ...
> > > > > > > > +Author: David B. Dahl
> > > > > > > >
> > > > > > > > shouldn't be here, IMO. Normally, if some additional
> licenses are
> > > > > > > used,
> > > > > > > > they have to be listed in the top-level NOTICE file (already
> > > > there).
> > > > > > > >
> > > > > > > > - I am not going to offer any comments on the technical
> merits of
> > > > the
> > > > > > > patch,
> > > > > > > > as I haven't tried it personally. However, I don't see any
> serious
> > > > > > > > technical objections to the functionality in question.
> > > > > > > >
> > > > > > > > So, the question is - why the PR is open for three months? I
> hasn't
> > > > > > been
> > > > > > > able
> > > > > > > > to get a clear answer. What I found out though is pretty
> > > > unsettling,
> > > > > > > really.
> > > > > > > > The communication on the topic is almost non-existent,
> except for
> > > > > this
> > > > > > > sparse
> > > > > > > > and bitter thread in the GH.
> > > > > > > >
> > > > > > > > I would love to hear from the committers what's stopping the
> > > > > acceptance
> > > > > > > of the
> > > > > > > > code, besides of the minor issues I've mentioned earlier?
> What are
> > > > > the
> > > > > > > reasons for it?
> > > > > > > > Is there anything wrong with it?
> > > > > > > > One of the responsibilities of the committers is to make
> sure the
> > > > > > > > contributions are reviewed; new contributors are guided and
> do
> > > > > > > understand how
> > > > > > > > the project ticks. The easy feedback flow attracts new
> people,
> > > > > allowing
> > > > > > > the
> > > > > > > > community to grow and thrive.
> > > > > > > >
> > > > > > > > I am asking _explicitely_ not to re-start the bickering I
> have
> > > > > already
> > > > > > > > seen. At this point I am interested in the purely technical
> side of
> > > > > > this.
> > > > > > > >
> > > > > > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > > > > [3]
> > > > > > >
> > > > > >
> > > > >
> > > >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > > > > [4] http://s.apache.org/iZl
> > > > > > > >
> > > > > > > > With regards,
> > > > > > > > Cos
> > > > > > > >
> > > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > > > > Github user elbamos commented on the pull request:
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > > > > >
> > > > > > > > > The current push should resolve some issues with changes
> in the
> > > > > > > > > Spark-Zeppelin interface that had created issues for
> users, as
> > > > > > > well as
> > > > > > > > > support for 1.5.1.
> > > > > > > > >
> > > > > > > > > Knitr support is improved, and the reason for a separate
> knitr
> > > > > > > interpreter may be clearer now.
> > > > > > > > >
> > > > > > > > > The amount of code borrowed from rscala is reduced.
> > > > > > > > >
> > > > > > > > > I did not address issues with @author tags, or files under
> the R/
> > > > > > > > > folder. The reason is, to be blunt, I don't understand
> what the
> > > > > > > precise
> > > > > > > > > concerns actually are.
> > > > > > > > >
> > > > > > > > > Please note that I changed .travis.yml to only use spark
> 1.4 and
> > > > > > > later.
> > > > > > > > > I'm sure there is a better way to do it, but I'm just not
> in a
> > > > > > > position
> > > > > > > > > to try to figure out the entire Zeppelin build process.
> > > > > > > > >
> > > > > > > > > Integrating this with the rest of zeppelin is going to
> take some
> > > > > > > work
> > > > > > > > > regarding pom's, travis, and so forth. I can do a lot of
> that,
> > > > > > > but I'm
> > > > > > > > > going to need to discuss it with the people who have been
> > > > "owning"
> > > > > > > those
> > > > > > > > > files. There are just too many moving pieces here.
> > > > > > > > >
> > > > > > > > > Because of the size of this (which is, unfortunately,
> necessary),
> > > > > > > > > posting here is probably not the most efficient way. That
> is also
> > > > > > > true
> > > > > > > > > because certain people chose to use this PR as a forum to
> air
> > > > other
> > > > > > > > > issues. Therefore, it would be better if reviewers e-mail
> me
> > > > > > > directly.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
>

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Konstantin Boudnik <co...@apache.org>]

On Wed, Dec 02, 2015 at 04:28PM, Amos B. Elberg wrote:
> Konstantin thank you for getting into this. 
> 
>> The best way to go around it is by 
>> providing a build-time option that will pull such binaries in. But by default 
>> such libs shouldn't be pulled. 
>
> That is basically how the PR handles this. If the GPL’d interpreter scripts
> are missing, there’s no indication at all at build time.  It doesn’t try to
> download them. 
> 
> At runtime, if the user tries to use functionality that would need such a
> script (i.e., if they type “knitr” to use knitr), we display an error that
> says that the functionality is not there because the library is missing, and
> that the library cannot be provided because it has an incompatible license,
> but the user can download it themselves if they want.
> 
> And, in the log, if the logging level is high, they will see a note that
> some functionality was disabled because the libraries weren’t there.
> 
> To be clear, though, none of these libraries are binaries.  They’re all interpreter scripts. 

If you ever in a doubt of which licenses could be used for dependncies (not to
say about source code) are listed in Category A list of [1]

A lot of quesitons discussed here are already covered in the legal FAQ, so
just check against it if you have any questions.

[1] http://www.apache.org/legal/resolved.html#category-a

Cos

> From: Konstantin Boudnik <co...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>, dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Date: December 2, 2015 at 3:24:50 PM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  
> 
> On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:  
> > I think that what moon means is that:  
> > If we merge the way it is now, the KnitRInterpreter will be part of the  
> > code base, so it isn't optional at code base level.  
> >  
> > To make it even more simple:  
> > * If the code has the right licensing -> that code can be part of the  
> > source code, and can be including in a binary release  
> 
> We aren't concerned with binary releases - as an Apache community we are  
> voting and releasing source code. If the project wants to provide a binary  
> release to its users, they are better be warned about inclusion of non  
> ASL2-friendly licensed bits.  
> 
> > * If the code doesn't have the right licensing -> it can't be part of the  
> > source code, and can't be included in a binary release  
> 
> See above.  
> 
> > * If the code doesn't have the right licensing but is imported at build  
> > time (libraries for example) -> it is not in the source code, it can't be  
> > included in binary release  
> 
> That is unless a user doing it on his own. The best way to go around it is by  
> providing a build-time option that will pull such binaries in. But by default  
> such libs shouldn't be pulled.  
> 
> Cos  
> 
> > So in the case of licensing issues, it would need to be fully optional  
> > (user bring the interpreter in his directory and build Zeppelin with it)  
> >  
> >  
> > On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <am...@gmail.com>  
> > wrote:  
> >  
> > > Moon let me clarify:  
> > >  
> > > Interpreted code doesn’t “link.” The wiki article actually explains it  
> > > pretty well — https://en.wikipedia.org/wiki/GPL_linking_exception  
> > > “Linking” against a library means compiling its headers into a binary, the  
> > > way a C compiler works. The 2008 e-mail Moon distributed, called this the  
> > > “interpreter exception.”  
> > >  
> > > As for whether GPL’d code is a “mandatory dependency,” if knitr is missing  
> > > the PR will compile, run and test just fine. The user is never prompted to  
> > > download it. The only effect is, if the user types “knitr” and knitr isn’t  
> > > there, we display an InterpreterError that knitr isn’t there.  
> > >  
> > > KnitRInterpreter is not optionally required. so it does not matter KnitR  
> > > is  
> > > optionally required or not.  
> > > Aren’t all interpreters optional? You can turn them on and off in the  
> > > config files.  
> > >  
> > > Do you mean that the KnitRInterpreter class gets compiled to bytecode even  
> > > if knitr is missing? So what? That isn't a mandatory dependency or a link.  
> > >  
> > > From: moon soo Lee <mo...@apache.org>  
> > > Reply: dev@zeppelin.incubator.apache.org <  
> > > dev@zeppelin.incubator.apache.org>  
> > > Date: December 2, 2015 at 3:18:00 AM  
> > > To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> > > Subject: Re: License of KnitRInterpreter Was: Re: contributions impasse.  
> > > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  
> > >  
> > > Let me summarize license concern about KnitRInterpreter.  
> > >  
> > >  
> > > Amos's interpretation.  
> > >  
> > > KnitR is optionally required by KnitRInterpreter.  
> > > R dependency in SparkR has no problem. So KnitR should be the same.  
> > >  
> > >  
> > > Moon's interpretation.  
> > >  
> > > KnitRInterpreter is not optionally required. so it does not matter KnitR is  
> > > optionally required or not.  
> > > R dependency in SparkR is exception of GPL. KnitR is not applied that  
> > > exception.  
> > >  
> > >  
> > > Amos, could you confirm my understanding to your interpretation is correct?  
> > > If it is not could you clarify it?  
> > >  
> > > Thanks,  
> > > moon  
> > >  
> > > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <am...@gmail.com>  
> > > wrote:  
> > >  
> > > > Just to put the final nail in this, I looked it up.  
> > > >  
> > > > I see no evidence of any “compiler exception.”  
> > > >  
> > > > There is an exception in the license for the runtime libraries that are  
> > > > bundled with GCC. See:  
> > > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html  
> > > >  
> > > > But no “compiler exception.”  
> > > >  
> > > > In fact, I believe this is part of the reason why LLVM was created.  
> > > >  
> > > > From: moon soo Lee <mo...@apache.org>  
> > > > Reply: moon soo Lee <mo...@apache.org>  
> > > > Date: December 1, 2015 at 8:16:36 PM  
> > > > To: Amos B. Elberg <am...@gmail.com>,  
> > > > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> > > > request: R Interpreter for Zeppelin  
> > > >  
> > > > Is knitR is commonly considered as a interpreter/compiler? or is it  
> > > > considered as a library routine?  
> > > >  
> > > > Thanks,  
> > > > moon  
> > > >  
> > > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com>  
> > > > wrote:  
> > > > Moon - you give this as an explanation of the licensing issue:  
> > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
> > > >  
> > > > According to that, there is an exception in the GPL for interpreter  
> > > > languages. As long as you don’t distribute the code, its fine to talk to  
> > > > an interpreted language.  
> > > >  
> > > > Well, if that’s the case, then the PR plainly does not have a license  
> > > > issue. It doesn’t distribute any GPL’d R code.  
> > > >  
> > > > I’m not sure what’s confusing about this. It seems completely  
> > > > straightforward.  
> > > >  
> > > > Regarding this:  
> > > >  
> > > >  
> > > > --  
> > > > Amos Elberg  
> > > > Sent with Airmail  
> > > >  
> > > > From: moon soo Lee <mo...@apache.org>  
> > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > dev@zeppelin.incubator.apache.org>  
> > > > Date: December 1, 2015 at 6:48:47 PM  
> > > >  
> > > > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org  
> > > >  
> > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> > > > request: R Interpreter for Zeppelin  
> > > >  
> > > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com>  
> > > > wrote:  
> > > >  
> > > > > I am going to try to minimize my reaction to Moon’s e-mail.  
> > > > >  
> > > > > The tl;dr is this:  
> > > > >  
> > > > > The reason we are having this discussion now is that active users of  
> > > the  
> > > > > PR — which now has its own user base — went public to complain about  
> > > > this.  
> > > >  
> > > >  
> > > > > The PR has been tested by an active user base for more than three  
> > > months.  
> > > > > No-one has been able to identify any specific actual licensing problem,  
> > > > and  
> > > > > the PR was prepared based on an extensive, careful review of the  
> > > relevant  
> > > > > licensing issues and after contacting the relevant people.  
> > > > >  
> > > > >  
> > > >  
> > > > I admire every software that used by user and helping people. That  
> > > includes  
> > > > your work. But that's not the topic we're in discussion. Active user does  
> > > > not mean your contribution can ignore the review.  
> > > >  
> > > >  
> > > >  
> > > > > It is not an explanation for someone who has been ignoring my “how can  
> > > I  
> > > > > move this forward…” emails for three months to point the finger and  
> > > say I  
> > > > > didn’t contact the right person or file the right report.  
> > > > >  
> > > > >  
> > > > This is also not the topic in this discussion.  
> > > >  
> > > >  
> > > > > The burden for providing an explanation for the inaction is on the PMCC  
> > > > at  
> > > > > this point.  
> > > >  
> > > > I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin  
> > > > > core, why do you think other PRs are passing CI?  
> > > > > They’re not! I often see comments on PRs to just ignore that CI is  
> > > > > failing.  
> > > > >  
> > > > > One of the most common reasons this PR fails CI, is CI times-out  
> > > > > downloading Spark to install. How could that possibly be caused by the  
> > > > PR?  
> > > > >  
> > > > > It looks to me like the only PRs with changes to the relevant parts of  
> > > > the  
> > > > > code — the SparkInterpreter — are being made by the person who wrote  
> > > the  
> > > > > testing suite.  
> > > > >  
> > > > > So, that would explain why some other PRs pass CI: Neither the  
> > > > > SparkInterpreter nor the testing suite are stable or robust, but since  
> > > > the  
> > > > > PRs are coming from the person who wrote both…  
> > > > >  
> > > > > And let's say Zeppelin core has problem and that makes your PR fails on  
> > > > CI  
> > > > > test. That's possible. But it still does not mean we can merge it with  
> > > CI  
> > > > > failing.  
> > > > >  
> > > > > It means you should be working with me to figure out why the CI is  
> > > > failing.  
> > > > >  
> > > > > This PR has been tested by an active user base for the past three  
> > > months.  
> > > > > If CI is continuing to fail, and dozens of hours of effort have not  
> > > > > resolved the CI issues, then it is time to start considering whether  
> > > the  
> > > > > testing suite is part of the problem.  
> > > > >  
> > > > > The level of defensiveness about the CI and SparkInterpreter is not  
> > > > > helping to resolve these issues.  
> > > > >  
> > > > > If you think it's problem on Zeppelin core, then file an issue that  
> > > > > reproduce the problem on Zeppelin core, that might be more efficient  
> > > than  
> > > > > keep trying yourself.  
> > > > > I contacted you numerous times about such issues...  
> > > > >  
> > > >  
> > > >  
> > > > I remember i commented your issue about CI. but you just keep repeated  
> > > it's  
> > > > not your problem but Zeppelin core problem.  
> > > >  
> > > > Then please file an issue about the problem you found in Zeppelin Core.  
> > > > Then everyone will get into the problem.  
> > > >  
> > > >  
> > > >  
> > > > >  
> > > > > In my interpretation, KnitRInterpreter is not an optional feature while  
> > > > it  
> > > > > is always enabled when compiling Zeppelin and always enabled when  
> > > running  
> > > > > Zeppelin. And it requires dynamically linked GPL library on runtime.  
> > > (yes  
> > > > > it will fail when no KnitR is installed on the system)  
> > > > >  
> > > > > Its not always enabled.  
> > > > > It is not dynamically linked at runtime.  
> > > > > It will not fail when knitr is missing. If knitr is not present, the  
> > > repl  
> > > > > interpreter starts and a note is written to to the log that the knitr  
> > > > > interpreter isn’t available because knitr is not present.  
> > > > >  
> > > > > no Apache code can ever call a shell script, on the purpose of dynamic  
> > > > > linking with GPL library.  
> > > > > You misunderstand.  
> > > > >  
> > > > > The *shell* is GPL'd.  
> > > > >  
> > > > > Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends  
> > > on  
> > > > a  
> > > > > shell script to launch?  
> > > > >  
> > > > Obviously not.  
> > > > >  
> > > > > The interaction with R in the PR is the same.  
> > > > >  
> > > > >  
> > > >  
> > > > Again, bash is one of exceptions of GPL, like other GPL licensed  
> > > > compiler/interpreter.  
> > > >  
> > > > Check here why Bash and R is okay with Apache License.  
> > > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
> > > >  
> > > > I'm not sure we can apply the same exception for 'using' KnitR.  
> > > >  
> > > > My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter'  
> > > you  
> > > > wrote is not an optional feature. Which is clearly not optionally enabled  
> > > > code and feature. And that depends on KnitR library which is GPL.  
> > > >  
> > > >  
> > > >  
> > > > > I was guessing SparkR can be still in Apache License even if it is  
> > > > depends  
> > > > > on R. Because of GPL licensed compiler generated output is not GPL  
> > > > license.  
> > > > > and R is sort of compiler. If you can get answer from Spark community  
> > > how  
> > > > > SparkR get managed to stay in Apache License while R is GPL, the answer  
> > > > > might help.  
> > > > > The description of SparkR is not accurate in any respect. (Do you think  
> > > > > SparkR is not talking to GPL-licensed libraries?)  
> > > > >  
> > > > > I don’t see that any genuine issue is being raised here.  
> > > > >  
> > > > > If there is an issue, the burden is on you to identify it.  
> > > > >  
> > > > > If i give you one suggestion, Zeppelin committers sometimes ask rebase  
> > > > the  
> > > > > contribution branch for some reason. It is not the really the best  
> > > > > practice, but still okay while most contributions are not including  
> > > large  
> > > > > code base changes  
> > > > > However, your one, has more than 4000 lines of code change. If you  
> > > rebase  
> > > > > then review should be started from the beginning, again. So you might  
> > > > want  
> > > > > to minimize the rebase your branch.  
> > > > >  
> > > > > Are you actually complaining that the problem is that I rebased the  
> > > code  
> > > > > during the three-month period when no-one looked at it and Zeppelin  
> > > went  
> > > > > through a release?  
> > > > >  
> > > > > I cannot take it seriously when you say things like this.  
> > > > >  
> > > > > Having to “start from the beginning” cannot be a problem if you never  
> > > > > actually started the first time...  
> > > > >  
> > > > >  
> > > >  
> > > > You wanted coordination and cooperation. So i gave you suggestion that  
> > > > helping review process. For example, your code has been rebased since my  
> > > > comment and jongyoul's comment. that means committers will need to look  
> > > > from the beginning. That'll require more time. And maybe, i guess that's  
> > > > not what you want. But If you don't care, feel free to rebase.  
> > > >  
> > > > Thanks,  
> > > > moon  
> > > >  
> > > >  
> > > >  
> > > > >  
> > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > Reply: moon soo Lee <mo...@apache.org>  
> > > > > Date: December 1, 2015 at 4:42:06 AM  
> > > > > To: Amos B. Elberg <am...@gmail.com>,  
> > > > > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> > > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin  
> > > pull  
> > > > > request: R Interpreter for Zeppelin  
> > > > >  
> > > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>  
> > > > > wrote:  
> > > > > Thank you, Cos.  
> > > > >  
> > > > > I’d like to briefly address the issues raised by Moon:  
> > > > >  
> > > > > 1. This PR does not passes CI  
> > > > > The CI fails on core Zeppelin, *not* code in this PR.  
> > > > >  
> > > > > I’ve been seeking assistance on this since August.  
> > > > >  
> > > > > The most common reason is that SparkInterpreter is unable to launch  
> > > Spark  
> > > > > and open a Spark Backend. This is necessary to test the PR.  
> > > > >  
> > > > > 60+ hours, has been spent adapting and re-basing when the  
> > > > SparkInterpreter  
> > > > > architecture changed and broke the PR’s *tests.*  
> > > > >  
> > > > >  
> > > > > I'm sorry, but the other PRs are passing CI. If it's problem on  
> > > Zeppelin  
> > > > > core, why do you think other PRs are passing CI?  
> > > > >  
> > > > > And let's say Zeppelin core has problem and that makes your PR fails on  
> > > > CI  
> > > > > test. That's possible. But it still does not mean we can merge it with  
> > > CI  
> > > > > failing.  
> > > > >  
> > > > > If you think it's problem on Zeppelin core, then file an issue that  
> > > > > reproduce the problem on Zeppelin core, that might be more efficient  
> > > than  
> > > > > keep trying yourself.  
> > > > >  
> > > > >  
> > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)  
> > > > > What license problem *specifically* do you believe may exist?  
> > > > >  
> > > > > In preparing the PR, I:  
> > > > >  
> > > > > * Reviewed the Apache policy in detail.  
> > > > >  
> > > > > * Contacted the FSF to ask their interpretation of the “linking”  
> > > > > provisions of the GPL license.  
> > > > >  
> > > > > * Reviewed how other Apache software deals with this issue (e.g., Spark  
> > > > > talks to R, which is GPL'd).  
> > > > >  
> > > > > * No necessary *dependencies* of the PR have license conflicts. In  
> > > > > several cases, I contacted package authors who agreed to re-issue their  
> > > > > packages under Apache-compatible licenses. (Usually I had to do a bit  
> > > of  
> > > > > coding in exchange…)  
> > > > >  
> > > > > * Where the license had to stay GPL, the packages are *not necessary*  
> > > and  
> > > > > *not dependencies.* If the optional packages are present, they will be  
> > > > > used to enable additional functionality. Knitr is an example. The PR  
> > > will  
> > > > > compile and run fine without knitr. If knitr is available (it is part  
> > > of  
> > > > > the most common R distribution), the PR will enable the knitr  
> > > > interpreter.  
> > > > >  
> > > > > * This is exactly how this issue is addressed through the Apache  
> > > > > ecosystem.  
> > > > > The tl;dr is this: When Apache code is written to talk to libraries  
> > > that  
> > > > > may or may not be present on the user’s system, or where it talks to an  
> > > > API  
> > > > > but is agnostic about implementation, that is not “linking” in a way  
> > > that  
> > > > > implicate the anti-linking provision of the GPL.  
> > > > >  
> > > > > Otherwise, no Apache code could ever call a shell script! Let alone run  
> > > > > on Linux, or talk to R.  
> > > > >  
> > > > >  
> > > > > I'm not a legal expert. So following could be wrong.  
> > > > >  
> > > > > In my interpretation, KnitRInterpreter is not an optional feature while  
> > > > it  
> > > > > is always enabled when compiling Zeppelin and always enabled when  
> > > running  
> > > > > Zeppelin. And it requires dynamically linked GPL library on runtime.  
> > > (yes  
> > > > > it will fail when no KnitR is installed on the system)  
> > > > >  
> > > > > And of course, no Apache code can ever call a shell script, on the  
> > > > purpose  
> > > > > of dynamic linking with GPL library.  
> > > > >  
> > > > > I was guessing SparkR can be still in Apache License even if it is  
> > > > depends  
> > > > > on R. Because of GPL licensed compiler generated output is not GPL  
> > > > license.  
> > > > > and R is sort of compiler.  
> > > > >  
> > > > > If you can get answer from Spark community how SparkR get managed to  
> > > stay  
> > > > > in Apache License while R is GPL, the answer might help.  
> > > > >  
> > > > >  
> > > > > 3. Need more time to review.  
> > > > > Has any reviewer has downloaded the PR or run the demo notebook? (Which  
> > > > > is there for the benefit of reviewers, and isn’t intended to go in  
> > > final  
> > > > > distribution.)  
> > > > >  
> > > > > How many +1 comments from users would you like to see?  
> > > > >  
> > > > > How much time do you believe is required?  
> > > > >  
> > > > >  
> > > > > It all depends on when CI is going to pass, when license problem is  
> > > going  
> > > > > to be cleared, and when a committer willing to review and responsible  
> > > to  
> > > > > commit your contribution.  
> > > > >  
> > > > >  
> > > > > 1. Large code base change  
> > > > > Large code base changes require coordination and cooperation. This PR  
> > > > > necessarily implicates the build scripts, testing code, the  
> > > > > SparkInterpreter, etc.  
> > > > >  
> > > > > I have been seeking to coordinate since August.  
> > > > >  
> > > > > I continue to stand ready to do so.  
> > > > >  
> > > > > -Amos  
> > > > >  
> > > > >  
> > > > > If i give you one suggestion, Zeppelin committers sometimes ask rebase  
> > > > the  
> > > > > contribution branch for some reason. It is not the really the best  
> > > > > practice, but still okay while most contributions are not including  
> > > large  
> > > > > code base changes.  
> > > > >  
> > > > > However, your one, has more than 4000 lines of code change. If you  
> > > rebase  
> > > > > then review should be started from the beginning, again. So you might  
> > > > want  
> > > > > to minimize the rebase your branch.  
> > > > >  
> > > > > Thanks,  
> > > > > moon  
> > > > >  
> > > > >  
> > > > > From: moon soo Lee <mo...@apache.org>  
> > > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > > dev@zeppelin.incubator.apache.org>  
> > > > > Date: December 1, 2015 at 1:34:19 AM  
> > > > > To: dev@zeppelin.incubator.apache.org <  
> > > dev@zeppelin.incubator.apache.org  
> > > > >  
> > > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin  
> > > pull  
> > > > > request: R Interpreter for Zeppelin  
> > > > >  
> > > > > Hi Cos,  
> > > > >  
> > > > > Thanks for opening a discussion.  
> > > > > My answer about 'Why this PR is open for three months' is  
> > > > >  
> > > > > 1. This PR does not passes CI  
> > > > > 2. Not 100% sure this PR has no license issue. (about KniteR)  
> > > > > 3. Need more time to review.  
> > > > >  
> > > > > It's my personal answer, other committers may have different opinion.  
> > > > >  
> > > > >  
> > > > > I think the question should be generalized. Because this PR is not the  
> > > > only  
> > > > > PR that is in impasse. There're more. For example  
> > > > >  
> > > > > Here's some examples that PR is not been merged.  
> > > > > https://github.com/apache/incubator-zeppelin/pull/53,  
> > > > > https://github.com/apache/incubator-zeppelin/pull/60  
> > > > > and so on.  
> > > > >  
> > > > > I can categorize the cases, based on experience of involving Zeppelin  
> > > > > community from the beginning.  
> > > > >  
> > > > > 1. Large code base change  
> > > > >  
> > > > > When contribution has large code base changes, it tend to take more  
> > > time  
> > > > to  
> > > > > review and merged. Normally, most contributions merged in 1day~1 week.  
> > > > > But some contribution has large code base changes take 2~4 weeks. Few  
> > > > > contribution that has very large code base change take months.  
> > > > >  
> > > > > 2. Communication lost  
> > > > >  
> > > > > Sometimes, committer is not responding, or contributor is not  
> > > responding.  
> > > > >  
> > > > > 3. Opinion diverges  
> > > > >  
> > > > > For some changes, included in contribution. When people have different  
> > > > > opinion and it continue to diverges, those PRs are not been merged.  
> > > > >  
> > > > >  
> > > > > I think having a guide such as ping other committer when a committer is  
> > > > not  
> > > > > responding, and divide contribution into small peaces if possible,  
> > > would  
> > > > > help most of the cases.  
> > > > > Of course committer have to pay attention more to the contribution and  
> > > > > helping people. That's the first one.  
> > > > >  
> > > > > Thanks,  
> > > > > moon  
> > > > >  
> > > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org>  
> > > > wrote:  
> > > > >  
> > > > > > To make sure we're on the same page, here are two threads that I  
> > > found  
> > > > > > related  
> > > > > > to this topic.  
> > > > > >  
> > > > > > Thread 1:  
> > > > > > Subject: R?  
> > > > > > Started on: July 1, 2015  
> > > > > >  
> > > > > > Thread 2:  
> > > > > > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for  
> > > > > > Zeppelin  
> > > > > > Started on: August 13, 2015  
> > > > > >  
> > > > > > If you want to fetch these from our archive send emails to  
> > > > > > dev-thread.1735@zeppelin.incubator.apache.org  
> > > > > > dev-thread.3573@zeppelin.incubator.apache.org  
> > > > > >  
> > > > > > Cos  
> > > > > >  
> > > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:  
> > > > > > > Guys,  
> > > > > > >  
> > > > > > > While catching up on my emails from the last a couple of weeks,  
> > > this  
> > > > > > thread  
> > > > > > > caught my attention. I am not normally paying much attention to the  
> > > > > code  
> > > > > > > reviews traffic from GH, but this one is pretty different as it  
> > > spans  
> > > > > > three  
> > > > > > > months and counting.  
> > > > > > >  
> > > > > > > First, here are my five cents:  
> > > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be  
> > > > > > contributed to  
> > > > > > > an ASF project this file should simply contain ASL2 text, like in  
> > > [1]  
> > > > > > > - r/pom.xml perhaps shouldn't contain a separate <developers>  
> > > > section,  
> > > > > > but  
> > > > > > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't  
> > > > > > > maintain this in the source code, but we have the list of all the  
> > > > > > > committers on the project's site [2] Every new committer's first  
> > > > > > commit is  
> > > > > > > to update the team page ;)  
> > > > > > > - comments like in  
> > > > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java  
> > > > > > >  
> > > > > > > +/**  
> > > > > > > + * Created by aelberg on 7/28/15.  
> > > > > > > + */  
> > > > > > >  
> > > > > > > is better to be removed. It has been already discussed here [3].  
> > > And  
> > > > > > the  
> > > > > > > initial discussion on the topic [4] was linked as well  
> > > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is  
> > > > > > R-specific  
> > > > > > > stuff - I have no idea about R, honestly, but  
> > > > > > >  
> > > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE  
> > > > > > > ...  
> > > > > > > +Author: David B. Dahl  
> > > > > > >  
> > > > > > > shouldn't be here, IMO. Normally, if some additional licenses are  
> > > > > > used,  
> > > > > > > they have to be listed in the top-level NOTICE file (already  
> > > there).  
> > > > > > >  
> > > > > > > - I am not going to offer any comments on the technical merits of  
> > > the  
> > > > > > patch,  
> > > > > > > as I haven't tried it personally. However, I don't see any serious  
> > > > > > > technical objections to the functionality in question.  
> > > > > > >  
> > > > > > > So, the question is - why the PR is open for three months? I hasn't  
> > > > > been  
> > > > > > able  
> > > > > > > to get a clear answer. What I found out though is pretty  
> > > unsettling,  
> > > > > > really.  
> > > > > > > The communication on the topic is almost non-existent, except for  
> > > > this  
> > > > > > sparse  
> > > > > > > and bitter thread in the GH.  
> > > > > > >  
> > > > > > > I would love to hear from the committers what's stopping the  
> > > > acceptance  
> > > > > > of the  
> > > > > > > code, besides of the minor issues I've mentioned earlier? What are  
> > > > the  
> > > > > > reasons for it?  
> > > > > > > Is there anything wrong with it?  
> > > > > > > One of the responsibilities of the committers is to make sure the  
> > > > > > > contributions are reviewed; new contributors are guided and do  
> > > > > > understand how  
> > > > > > > the project ticks. The easy feedback flow attracts new people,  
> > > > allowing  
> > > > > > the  
> > > > > > > community to grow and thrive.  
> > > > > > >  
> > > > > > > I am asking _explicitely_ not to re-start the bickering I have  
> > > > already  
> > > > > > > seen. At this point I am interested in the purely technical side of  
> > > > > this.  
> > > > > > >  
> > > > > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE  
> > > > > > > [2] http://bigtop.apache.org/team-list.html  
> > > > > > > [3]  
> > > > > >  
> > > > >  
> > > >  
> > > http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html  
> > > > > > > [4] http://s.apache.org/iZl  
> > > > > > >  
> > > > > > > With regards,  
> > > > > > > Cos  
> > > > > > >  
> > > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:  
> > > > > > > > Github user elbamos commented on the pull request:  
> > > > > > > >  
> > > > > > > >  
> > > > > >  
> > > > >  
> > > >  
> > > https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411  
> > > > > > > >  
> > > > > > > > The current push should resolve some issues with changes in the  
> > > > > > > > Spark-Zeppelin interface that had created issues for users, as  
> > > > > > well as  
> > > > > > > > support for 1.5.1.  
> > > > > > > >  
> > > > > > > > Knitr support is improved, and the reason for a separate knitr  
> > > > > > interpreter may be clearer now.  
> > > > > > > >  
> > > > > > > > The amount of code borrowed from rscala is reduced.  
> > > > > > > >  
> > > > > > > > I did not address issues with @author tags, or files under the R/  
> > > > > > > > folder. The reason is, to be blunt, I don't understand what the  
> > > > > > precise  
> > > > > > > > concerns actually are.  
> > > > > > > >  
> > > > > > > > Please note that I changed .travis.yml to only use spark 1.4 and  
> > > > > > later.  
> > > > > > > > I'm sure there is a better way to do it, but I'm just not in a  
> > > > > > position  
> > > > > > > > to try to figure out the entire Zeppelin build process.  
> > > > > > > >  
> > > > > > > > Integrating this with the rest of zeppelin is going to take some  
> > > > > > work  
> > > > > > > > regarding pom's, travis, and so forth. I can do a lot of that,  
> > > > > > but I'm  
> > > > > > > > going to need to discuss it with the people who have been  
> > > "owning"  
> > > > > > those  
> > > > > > > > files. There are just too many moving pieces here.  
> > > > > > > >  
> > > > > > > > Because of the size of this (which is, unfortunately, necessary),  
> > > > > > > > posting here is probably not the most efficient way. That is also  
> > > > > > true  
> > > > > > > > because certain people chose to use this PR as a forum to air  
> > > other  
> > > > > > > > issues. Therefore, it would be better if reviewers e-mail me  
> > > > > > directly.  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > >  
> > > >  
> > >  

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Konstantin thank you for getting into this. 

The best way to go around it is by 
providing a build-time option that will pull such binaries in. But by default 
such libs shouldn't be pulled. 
That is basically how the PR handles this. If the GPL’d interpreter scripts are missing, there’s no indication at all at build time.  It doesn’t try to download them. 

At runtime, if the user tries to use functionality that would need such a script (i.e., if they type “knitr” to use knitr), we display an error that says that the functionality is not there because the library is missing, and that the library cannot be provided because it has an incompatible license, but the user can download it themselves if they want.

And, in the log, if the logging level is high, they will see a note that some functionality was disabled because the libraries weren’t there. 

To be clear, though, none of these libraries are binaries.  They’re all interpreter scripts. 

From: Konstantin Boudnik <co...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>, dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 2, 2015 at 3:24:50 PM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:  
> I think that what moon means is that:  
> If we merge the way it is now, the KnitRInterpreter will be part of the  
> code base, so it isn't optional at code base level.  
>  
> To make it even more simple:  
> * If the code has the right licensing -> that code can be part of the  
> source code, and can be including in a binary release  

We aren't concerned with binary releases - as an Apache community we are  
voting and releasing source code. If the project wants to provide a binary  
release to its users, they are better be warned about inclusion of non  
ASL2-friendly licensed bits.  

> * If the code doesn't have the right licensing -> it can't be part of the  
> source code, and can't be included in a binary release  

See above.  

> * If the code doesn't have the right licensing but is imported at build  
> time (libraries for example) -> it is not in the source code, it can't be  
> included in binary release  

That is unless a user doing it on his own. The best way to go around it is by  
providing a build-time option that will pull such binaries in. But by default  
such libs shouldn't be pulled.  

Cos  

> So in the case of licensing issues, it would need to be fully optional  
> (user bring the interpreter in his directory and build Zeppelin with it)  
>  
>  
> On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <am...@gmail.com>  
> wrote:  
>  
> > Moon let me clarify:  
> >  
> > Interpreted code doesn’t “link.” The wiki article actually explains it  
> > pretty well — https://en.wikipedia.org/wiki/GPL_linking_exception  
> > “Linking” against a library means compiling its headers into a binary, the  
> > way a C compiler works. The 2008 e-mail Moon distributed, called this the  
> > “interpreter exception.”  
> >  
> > As for whether GPL’d code is a “mandatory dependency,” if knitr is missing  
> > the PR will compile, run and test just fine. The user is never prompted to  
> > download it. The only effect is, if the user types “knitr” and knitr isn’t  
> > there, we display an InterpreterError that knitr isn’t there.  
> >  
> > KnitRInterpreter is not optionally required. so it does not matter KnitR  
> > is  
> > optionally required or not.  
> > Aren’t all interpreters optional? You can turn them on and off in the  
> > config files.  
> >  
> > Do you mean that the KnitRInterpreter class gets compiled to bytecode even  
> > if knitr is missing? So what? That isn't a mandatory dependency or a link.  
> >  
> > From: moon soo Lee <mo...@apache.org>  
> > Reply: dev@zeppelin.incubator.apache.org <  
> > dev@zeppelin.incubator.apache.org>  
> > Date: December 2, 2015 at 3:18:00 AM  
> > To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> > Subject: Re: License of KnitRInterpreter Was: Re: contributions impasse.  
> > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  
> >  
> > Let me summarize license concern about KnitRInterpreter.  
> >  
> >  
> > Amos's interpretation.  
> >  
> > KnitR is optionally required by KnitRInterpreter.  
> > R dependency in SparkR has no problem. So KnitR should be the same.  
> >  
> >  
> > Moon's interpretation.  
> >  
> > KnitRInterpreter is not optionally required. so it does not matter KnitR is  
> > optionally required or not.  
> > R dependency in SparkR is exception of GPL. KnitR is not applied that  
> > exception.  
> >  
> >  
> > Amos, could you confirm my understanding to your interpretation is correct?  
> > If it is not could you clarify it?  
> >  
> > Thanks,  
> > moon  
> >  
> > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <am...@gmail.com>  
> > wrote:  
> >  
> > > Just to put the final nail in this, I looked it up.  
> > >  
> > > I see no evidence of any “compiler exception.”  
> > >  
> > > There is an exception in the license for the runtime libraries that are  
> > > bundled with GCC. See:  
> > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html  
> > >  
> > > But no “compiler exception.”  
> > >  
> > > In fact, I believe this is part of the reason why LLVM was created.  
> > >  
> > > From: moon soo Lee <mo...@apache.org>  
> > > Reply: moon soo Lee <mo...@apache.org>  
> > > Date: December 1, 2015 at 8:16:36 PM  
> > > To: Amos B. Elberg <am...@gmail.com>,  
> > > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> > > request: R Interpreter for Zeppelin  
> > >  
> > > Is knitR is commonly considered as a interpreter/compiler? or is it  
> > > considered as a library routine?  
> > >  
> > > Thanks,  
> > > moon  
> > >  
> > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com>  
> > > wrote:  
> > > Moon - you give this as an explanation of the licensing issue:  
> > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
> > >  
> > > According to that, there is an exception in the GPL for interpreter  
> > > languages. As long as you don’t distribute the code, its fine to talk to  
> > > an interpreted language.  
> > >  
> > > Well, if that’s the case, then the PR plainly does not have a license  
> > > issue. It doesn’t distribute any GPL’d R code.  
> > >  
> > > I’m not sure what’s confusing about this. It seems completely  
> > > straightforward.  
> > >  
> > > Regarding this:  
> > >  
> > >  
> > > --  
> > > Amos Elberg  
> > > Sent with Airmail  
> > >  
> > > From: moon soo Lee <mo...@apache.org>  
> > > Reply: dev@zeppelin.incubator.apache.org <  
> > > dev@zeppelin.incubator.apache.org>  
> > > Date: December 1, 2015 at 6:48:47 PM  
> > >  
> > > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org  
> > >  
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> > > request: R Interpreter for Zeppelin  
> > >  
> > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com>  
> > > wrote:  
> > >  
> > > > I am going to try to minimize my reaction to Moon’s e-mail.  
> > > >  
> > > > The tl;dr is this:  
> > > >  
> > > > The reason we are having this discussion now is that active users of  
> > the  
> > > > PR — which now has its own user base — went public to complain about  
> > > this.  
> > >  
> > >  
> > > > The PR has been tested by an active user base for more than three  
> > months.  
> > > > No-one has been able to identify any specific actual licensing problem,  
> > > and  
> > > > the PR was prepared based on an extensive, careful review of the  
> > relevant  
> > > > licensing issues and after contacting the relevant people.  
> > > >  
> > > >  
> > >  
> > > I admire every software that used by user and helping people. That  
> > includes  
> > > your work. But that's not the topic we're in discussion. Active user does  
> > > not mean your contribution can ignore the review.  
> > >  
> > >  
> > >  
> > > > It is not an explanation for someone who has been ignoring my “how can  
> > I  
> > > > move this forward…” emails for three months to point the finger and  
> > say I  
> > > > didn’t contact the right person or file the right report.  
> > > >  
> > > >  
> > > This is also not the topic in this discussion.  
> > >  
> > >  
> > > > The burden for providing an explanation for the inaction is on the PMCC  
> > > at  
> > > > this point.  
> > >  
> > > I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin  
> > > > core, why do you think other PRs are passing CI?  
> > > > They’re not! I often see comments on PRs to just ignore that CI is  
> > > > failing.  
> > > >  
> > > > One of the most common reasons this PR fails CI, is CI times-out  
> > > > downloading Spark to install. How could that possibly be caused by the  
> > > PR?  
> > > >  
> > > > It looks to me like the only PRs with changes to the relevant parts of  
> > > the  
> > > > code — the SparkInterpreter — are being made by the person who wrote  
> > the  
> > > > testing suite.  
> > > >  
> > > > So, that would explain why some other PRs pass CI: Neither the  
> > > > SparkInterpreter nor the testing suite are stable or robust, but since  
> > > the  
> > > > PRs are coming from the person who wrote both…  
> > > >  
> > > > And let's say Zeppelin core has problem and that makes your PR fails on  
> > > CI  
> > > > test. That's possible. But it still does not mean we can merge it with  
> > CI  
> > > > failing.  
> > > >  
> > > > It means you should be working with me to figure out why the CI is  
> > > failing.  
> > > >  
> > > > This PR has been tested by an active user base for the past three  
> > months.  
> > > > If CI is continuing to fail, and dozens of hours of effort have not  
> > > > resolved the CI issues, then it is time to start considering whether  
> > the  
> > > > testing suite is part of the problem.  
> > > >  
> > > > The level of defensiveness about the CI and SparkInterpreter is not  
> > > > helping to resolve these issues.  
> > > >  
> > > > If you think it's problem on Zeppelin core, then file an issue that  
> > > > reproduce the problem on Zeppelin core, that might be more efficient  
> > than  
> > > > keep trying yourself.  
> > > > I contacted you numerous times about such issues...  
> > > >  
> > >  
> > >  
> > > I remember i commented your issue about CI. but you just keep repeated  
> > it's  
> > > not your problem but Zeppelin core problem.  
> > >  
> > > Then please file an issue about the problem you found in Zeppelin Core.  
> > > Then everyone will get into the problem.  
> > >  
> > >  
> > >  
> > > >  
> > > > In my interpretation, KnitRInterpreter is not an optional feature while  
> > > it  
> > > > is always enabled when compiling Zeppelin and always enabled when  
> > running  
> > > > Zeppelin. And it requires dynamically linked GPL library on runtime.  
> > (yes  
> > > > it will fail when no KnitR is installed on the system)  
> > > >  
> > > > Its not always enabled.  
> > > > It is not dynamically linked at runtime.  
> > > > It will not fail when knitr is missing. If knitr is not present, the  
> > repl  
> > > > interpreter starts and a note is written to to the log that the knitr  
> > > > interpreter isn’t available because knitr is not present.  
> > > >  
> > > > no Apache code can ever call a shell script, on the purpose of dynamic  
> > > > linking with GPL library.  
> > > > You misunderstand.  
> > > >  
> > > > The *shell* is GPL'd.  
> > > >  
> > > > Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends  
> > on  
> > > a  
> > > > shell script to launch?  
> > > >  
> > > Obviously not.  
> > > >  
> > > > The interaction with R in the PR is the same.  
> > > >  
> > > >  
> > >  
> > > Again, bash is one of exceptions of GPL, like other GPL licensed  
> > > compiler/interpreter.  
> > >  
> > > Check here why Bash and R is okay with Apache License.  
> > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
> > >  
> > > I'm not sure we can apply the same exception for 'using' KnitR.  
> > >  
> > > My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter'  
> > you  
> > > wrote is not an optional feature. Which is clearly not optionally enabled  
> > > code and feature. And that depends on KnitR library which is GPL.  
> > >  
> > >  
> > >  
> > > > I was guessing SparkR can be still in Apache License even if it is  
> > > depends  
> > > > on R. Because of GPL licensed compiler generated output is not GPL  
> > > license.  
> > > > and R is sort of compiler. If you can get answer from Spark community  
> > how  
> > > > SparkR get managed to stay in Apache License while R is GPL, the answer  
> > > > might help.  
> > > > The description of SparkR is not accurate in any respect. (Do you think  
> > > > SparkR is not talking to GPL-licensed libraries?)  
> > > >  
> > > > I don’t see that any genuine issue is being raised here.  
> > > >  
> > > > If there is an issue, the burden is on you to identify it.  
> > > >  
> > > > If i give you one suggestion, Zeppelin committers sometimes ask rebase  
> > > the  
> > > > contribution branch for some reason. It is not the really the best  
> > > > practice, but still okay while most contributions are not including  
> > large  
> > > > code base changes  
> > > > However, your one, has more than 4000 lines of code change. If you  
> > rebase  
> > > > then review should be started from the beginning, again. So you might  
> > > want  
> > > > to minimize the rebase your branch.  
> > > >  
> > > > Are you actually complaining that the problem is that I rebased the  
> > code  
> > > > during the three-month period when no-one looked at it and Zeppelin  
> > went  
> > > > through a release?  
> > > >  
> > > > I cannot take it seriously when you say things like this.  
> > > >  
> > > > Having to “start from the beginning” cannot be a problem if you never  
> > > > actually started the first time...  
> > > >  
> > > >  
> > >  
> > > You wanted coordination and cooperation. So i gave you suggestion that  
> > > helping review process. For example, your code has been rebased since my  
> > > comment and jongyoul's comment. that means committers will need to look  
> > > from the beginning. That'll require more time. And maybe, i guess that's  
> > > not what you want. But If you don't care, feel free to rebase.  
> > >  
> > > Thanks,  
> > > moon  
> > >  
> > >  
> > >  
> > > >  
> > > > From: moon soo Lee <mo...@apache.org>  
> > > > Reply: moon soo Lee <mo...@apache.org>  
> > > > Date: December 1, 2015 at 4:42:06 AM  
> > > > To: Amos B. Elberg <am...@gmail.com>,  
> > > > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin  
> > pull  
> > > > request: R Interpreter for Zeppelin  
> > > >  
> > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>  
> > > > wrote:  
> > > > Thank you, Cos.  
> > > >  
> > > > I’d like to briefly address the issues raised by Moon:  
> > > >  
> > > > 1. This PR does not passes CI  
> > > > The CI fails on core Zeppelin, *not* code in this PR.  
> > > >  
> > > > I’ve been seeking assistance on this since August.  
> > > >  
> > > > The most common reason is that SparkInterpreter is unable to launch  
> > Spark  
> > > > and open a Spark Backend. This is necessary to test the PR.  
> > > >  
> > > > 60+ hours, has been spent adapting and re-basing when the  
> > > SparkInterpreter  
> > > > architecture changed and broke the PR’s *tests.*  
> > > >  
> > > >  
> > > > I'm sorry, but the other PRs are passing CI. If it's problem on  
> > Zeppelin  
> > > > core, why do you think other PRs are passing CI?  
> > > >  
> > > > And let's say Zeppelin core has problem and that makes your PR fails on  
> > > CI  
> > > > test. That's possible. But it still does not mean we can merge it with  
> > CI  
> > > > failing.  
> > > >  
> > > > If you think it's problem on Zeppelin core, then file an issue that  
> > > > reproduce the problem on Zeppelin core, that might be more efficient  
> > than  
> > > > keep trying yourself.  
> > > >  
> > > >  
> > > > 2. Not 100% sure this PR has no license issue. (about KniteR)  
> > > > What license problem *specifically* do you believe may exist?  
> > > >  
> > > > In preparing the PR, I:  
> > > >  
> > > > * Reviewed the Apache policy in detail.  
> > > >  
> > > > * Contacted the FSF to ask their interpretation of the “linking”  
> > > > provisions of the GPL license.  
> > > >  
> > > > * Reviewed how other Apache software deals with this issue (e.g., Spark  
> > > > talks to R, which is GPL'd).  
> > > >  
> > > > * No necessary *dependencies* of the PR have license conflicts. In  
> > > > several cases, I contacted package authors who agreed to re-issue their  
> > > > packages under Apache-compatible licenses. (Usually I had to do a bit  
> > of  
> > > > coding in exchange…)  
> > > >  
> > > > * Where the license had to stay GPL, the packages are *not necessary*  
> > and  
> > > > *not dependencies.* If the optional packages are present, they will be  
> > > > used to enable additional functionality. Knitr is an example. The PR  
> > will  
> > > > compile and run fine without knitr. If knitr is available (it is part  
> > of  
> > > > the most common R distribution), the PR will enable the knitr  
> > > interpreter.  
> > > >  
> > > > * This is exactly how this issue is addressed through the Apache  
> > > > ecosystem.  
> > > > The tl;dr is this: When Apache code is written to talk to libraries  
> > that  
> > > > may or may not be present on the user’s system, or where it talks to an  
> > > API  
> > > > but is agnostic about implementation, that is not “linking” in a way  
> > that  
> > > > implicate the anti-linking provision of the GPL.  
> > > >  
> > > > Otherwise, no Apache code could ever call a shell script! Let alone run  
> > > > on Linux, or talk to R.  
> > > >  
> > > >  
> > > > I'm not a legal expert. So following could be wrong.  
> > > >  
> > > > In my interpretation, KnitRInterpreter is not an optional feature while  
> > > it  
> > > > is always enabled when compiling Zeppelin and always enabled when  
> > running  
> > > > Zeppelin. And it requires dynamically linked GPL library on runtime.  
> > (yes  
> > > > it will fail when no KnitR is installed on the system)  
> > > >  
> > > > And of course, no Apache code can ever call a shell script, on the  
> > > purpose  
> > > > of dynamic linking with GPL library.  
> > > >  
> > > > I was guessing SparkR can be still in Apache License even if it is  
> > > depends  
> > > > on R. Because of GPL licensed compiler generated output is not GPL  
> > > license.  
> > > > and R is sort of compiler.  
> > > >  
> > > > If you can get answer from Spark community how SparkR get managed to  
> > stay  
> > > > in Apache License while R is GPL, the answer might help.  
> > > >  
> > > >  
> > > > 3. Need more time to review.  
> > > > Has any reviewer has downloaded the PR or run the demo notebook? (Which  
> > > > is there for the benefit of reviewers, and isn’t intended to go in  
> > final  
> > > > distribution.)  
> > > >  
> > > > How many +1 comments from users would you like to see?  
> > > >  
> > > > How much time do you believe is required?  
> > > >  
> > > >  
> > > > It all depends on when CI is going to pass, when license problem is  
> > going  
> > > > to be cleared, and when a committer willing to review and responsible  
> > to  
> > > > commit your contribution.  
> > > >  
> > > >  
> > > > 1. Large code base change  
> > > > Large code base changes require coordination and cooperation. This PR  
> > > > necessarily implicates the build scripts, testing code, the  
> > > > SparkInterpreter, etc.  
> > > >  
> > > > I have been seeking to coordinate since August.  
> > > >  
> > > > I continue to stand ready to do so.  
> > > >  
> > > > -Amos  
> > > >  
> > > >  
> > > > If i give you one suggestion, Zeppelin committers sometimes ask rebase  
> > > the  
> > > > contribution branch for some reason. It is not the really the best  
> > > > practice, but still okay while most contributions are not including  
> > large  
> > > > code base changes.  
> > > >  
> > > > However, your one, has more than 4000 lines of code change. If you  
> > rebase  
> > > > then review should be started from the beginning, again. So you might  
> > > want  
> > > > to minimize the rebase your branch.  
> > > >  
> > > > Thanks,  
> > > > moon  
> > > >  
> > > >  
> > > > From: moon soo Lee <mo...@apache.org>  
> > > > Reply: dev@zeppelin.incubator.apache.org <  
> > > > dev@zeppelin.incubator.apache.org>  
> > > > Date: December 1, 2015 at 1:34:19 AM  
> > > > To: dev@zeppelin.incubator.apache.org <  
> > dev@zeppelin.incubator.apache.org  
> > > >  
> > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin  
> > pull  
> > > > request: R Interpreter for Zeppelin  
> > > >  
> > > > Hi Cos,  
> > > >  
> > > > Thanks for opening a discussion.  
> > > > My answer about 'Why this PR is open for three months' is  
> > > >  
> > > > 1. This PR does not passes CI  
> > > > 2. Not 100% sure this PR has no license issue. (about KniteR)  
> > > > 3. Need more time to review.  
> > > >  
> > > > It's my personal answer, other committers may have different opinion.  
> > > >  
> > > >  
> > > > I think the question should be generalized. Because this PR is not the  
> > > only  
> > > > PR that is in impasse. There're more. For example  
> > > >  
> > > > Here's some examples that PR is not been merged.  
> > > > https://github.com/apache/incubator-zeppelin/pull/53,  
> > > > https://github.com/apache/incubator-zeppelin/pull/60  
> > > > and so on.  
> > > >  
> > > > I can categorize the cases, based on experience of involving Zeppelin  
> > > > community from the beginning.  
> > > >  
> > > > 1. Large code base change  
> > > >  
> > > > When contribution has large code base changes, it tend to take more  
> > time  
> > > to  
> > > > review and merged. Normally, most contributions merged in 1day~1 week.  
> > > > But some contribution has large code base changes take 2~4 weeks. Few  
> > > > contribution that has very large code base change take months.  
> > > >  
> > > > 2. Communication lost  
> > > >  
> > > > Sometimes, committer is not responding, or contributor is not  
> > responding.  
> > > >  
> > > > 3. Opinion diverges  
> > > >  
> > > > For some changes, included in contribution. When people have different  
> > > > opinion and it continue to diverges, those PRs are not been merged.  
> > > >  
> > > >  
> > > > I think having a guide such as ping other committer when a committer is  
> > > not  
> > > > responding, and divide contribution into small peaces if possible,  
> > would  
> > > > help most of the cases.  
> > > > Of course committer have to pay attention more to the contribution and  
> > > > helping people. That's the first one.  
> > > >  
> > > > Thanks,  
> > > > moon  
> > > >  
> > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org>  
> > > wrote:  
> > > >  
> > > > > To make sure we're on the same page, here are two threads that I  
> > found  
> > > > > related  
> > > > > to this topic.  
> > > > >  
> > > > > Thread 1:  
> > > > > Subject: R?  
> > > > > Started on: July 1, 2015  
> > > > >  
> > > > > Thread 2:  
> > > > > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for  
> > > > > Zeppelin  
> > > > > Started on: August 13, 2015  
> > > > >  
> > > > > If you want to fetch these from our archive send emails to  
> > > > > dev-thread.1735@zeppelin.incubator.apache.org  
> > > > > dev-thread.3573@zeppelin.incubator.apache.org  
> > > > >  
> > > > > Cos  
> > > > >  
> > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:  
> > > > > > Guys,  
> > > > > >  
> > > > > > While catching up on my emails from the last a couple of weeks,  
> > this  
> > > > > thread  
> > > > > > caught my attention. I am not normally paying much attention to the  
> > > > code  
> > > > > > reviews traffic from GH, but this one is pretty different as it  
> > spans  
> > > > > three  
> > > > > > months and counting.  
> > > > > >  
> > > > > > First, here are my five cents:  
> > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be  
> > > > > contributed to  
> > > > > > an ASF project this file should simply contain ASL2 text, like in  
> > [1]  
> > > > > > - r/pom.xml perhaps shouldn't contain a separate <developers>  
> > > section,  
> > > > > but  
> > > > > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't  
> > > > > > maintain this in the source code, but we have the list of all the  
> > > > > > committers on the project's site [2] Every new committer's first  
> > > > > commit is  
> > > > > > to update the team page ;)  
> > > > > > - comments like in  
> > > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java  
> > > > > >  
> > > > > > +/**  
> > > > > > + * Created by aelberg on 7/28/15.  
> > > > > > + */  
> > > > > >  
> > > > > > is better to be removed. It has been already discussed here [3].  
> > And  
> > > > > the  
> > > > > > initial discussion on the topic [4] was linked as well  
> > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is  
> > > > > R-specific  
> > > > > > stuff - I have no idea about R, honestly, but  
> > > > > >  
> > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE  
> > > > > > ...  
> > > > > > +Author: David B. Dahl  
> > > > > >  
> > > > > > shouldn't be here, IMO. Normally, if some additional licenses are  
> > > > > used,  
> > > > > > they have to be listed in the top-level NOTICE file (already  
> > there).  
> > > > > >  
> > > > > > - I am not going to offer any comments on the technical merits of  
> > the  
> > > > > patch,  
> > > > > > as I haven't tried it personally. However, I don't see any serious  
> > > > > > technical objections to the functionality in question.  
> > > > > >  
> > > > > > So, the question is - why the PR is open for three months? I hasn't  
> > > > been  
> > > > > able  
> > > > > > to get a clear answer. What I found out though is pretty  
> > unsettling,  
> > > > > really.  
> > > > > > The communication on the topic is almost non-existent, except for  
> > > this  
> > > > > sparse  
> > > > > > and bitter thread in the GH.  
> > > > > >  
> > > > > > I would love to hear from the committers what's stopping the  
> > > acceptance  
> > > > > of the  
> > > > > > code, besides of the minor issues I've mentioned earlier? What are  
> > > the  
> > > > > reasons for it?  
> > > > > > Is there anything wrong with it?  
> > > > > > One of the responsibilities of the committers is to make sure the  
> > > > > > contributions are reviewed; new contributors are guided and do  
> > > > > understand how  
> > > > > > the project ticks. The easy feedback flow attracts new people,  
> > > allowing  
> > > > > the  
> > > > > > community to grow and thrive.  
> > > > > >  
> > > > > > I am asking _explicitely_ not to re-start the bickering I have  
> > > already  
> > > > > > seen. At this point I am interested in the purely technical side of  
> > > > this.  
> > > > > >  
> > > > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE  
> > > > > > [2] http://bigtop.apache.org/team-list.html  
> > > > > > [3]  
> > > > >  
> > > >  
> > >  
> > http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html  
> > > > > > [4] http://s.apache.org/iZl  
> > > > > >  
> > > > > > With regards,  
> > > > > > Cos  
> > > > > >  
> > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:  
> > > > > > > Github user elbamos commented on the pull request:  
> > > > > > >  
> > > > > > >  
> > > > >  
> > > >  
> > >  
> > https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411  
> > > > > > >  
> > > > > > > The current push should resolve some issues with changes in the  
> > > > > > > Spark-Zeppelin interface that had created issues for users, as  
> > > > > well as  
> > > > > > > support for 1.5.1.  
> > > > > > >  
> > > > > > > Knitr support is improved, and the reason for a separate knitr  
> > > > > interpreter may be clearer now.  
> > > > > > >  
> > > > > > > The amount of code borrowed from rscala is reduced.  
> > > > > > >  
> > > > > > > I did not address issues with @author tags, or files under the R/  
> > > > > > > folder. The reason is, to be blunt, I don't understand what the  
> > > > > precise  
> > > > > > > concerns actually are.  
> > > > > > >  
> > > > > > > Please note that I changed .travis.yml to only use spark 1.4 and  
> > > > > later.  
> > > > > > > I'm sure there is a better way to do it, but I'm just not in a  
> > > > > position  
> > > > > > > to try to figure out the entire Zeppelin build process.  
> > > > > > >  
> > > > > > > Integrating this with the rest of zeppelin is going to take some  
> > > > > work  
> > > > > > > regarding pom's, travis, and so forth. I can do a lot of that,  
> > > > > but I'm  
> > > > > > > going to need to discuss it with the people who have been  
> > "owning"  
> > > > > those  
> > > > > > > files. There are just too many moving pieces here.  
> > > > > > >  
> > > > > > > Because of the size of this (which is, unfortunately, necessary),  
> > > > > > > posting here is probably not the most efficient way. That is also  
> > > > > true  
> > > > > > > because certain people chose to use this PR as a forum to air  
> > other  
> > > > > > > issues. Therefore, it would be better if reviewers e-mail me  
> > > > > directly.  
> > > > >  
> > > > >  
> > > > >  
> > > >  
> > >  
> >  

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Konstantin Boudnik <co...@apache.org>]

On Wed, Dec 02, 2015 at 06:56PM, Corneau Damien wrote:
> I think that what moon means is that:
> If we merge the way it is now, the KnitRInterpreter will be part of the
> code base, so it isn't optional at code base level.
> 
> To make it even more simple:
> * If the code has the right licensing -> that code can be part of the
> source code, and can be including in a binary release

We aren't concerned with binary releases - as an Apache community we are
voting and releasing source code. If the project wants to provide a binary
release to its users, they are better be warned about inclusion of non
ASL2-friendly licensed bits.
 
> * If the code doesn't have the right licensing -> it can't be part of the
> source code, and can't be included in a binary release

See above.

> * If the code doesn't have the right licensing but is imported at build
> time (libraries for example) -> it is not in the source code, it can't be
> included in binary release

That is unless a user doing it on his own. The best way to go around it is by
providing a build-time option that will pull such binaries in. But by default
such libs shouldn't be pulled.

Cos

> So in the case of licensing issues, it would need to be fully optional
> (user bring the interpreter in his directory and build Zeppelin with it)
> 
> 
> On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <am...@gmail.com>
> wrote:
> 
> > Moon let me clarify:
> >
> > Interpreted code doesn’t “link.”  The wiki article actually explains it
> > pretty well — https://en.wikipedia.org/wiki/GPL_linking_exception
> >  “Linking” against a library means compiling its headers into a binary, the
> > way a C compiler works.  The 2008 e-mail Moon distributed, called this the
> > “interpreter exception.”
> >
> > As for whether GPL’d code is a “mandatory dependency,” if knitr is missing
> > the PR will compile, run and test just fine.  The user is never prompted to
> > download it.  The only effect is, if the user types “knitr” and knitr isn’t
> > there, we display an InterpreterError that knitr isn’t there.
> >
> > KnitRInterpreter is not optionally required. so it does not matter KnitR
> > is
> > optionally required or not.
> > Aren’t all interpreters optional?  You can turn them on and off in the
> > config files.
> >
> > Do you mean that the KnitRInterpreter class gets compiled to bytecode even
> > if knitr is missing?  So what?  That isn't a mandatory dependency or a link.
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > Date: December 2, 2015 at 3:18:00 AM
> > To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> > Subject:  Re: License of KnitRInterpreter Was: Re: contributions impasse.
> > Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin
> >
> > Let me summarize license concern about KnitRInterpreter.
> >
> >
> > Amos's interpretation.
> >
> > KnitR is optionally required by KnitRInterpreter.
> > R dependency in SparkR has no problem. So KnitR should be the same.
> >
> >
> > Moon's interpretation.
> >
> > KnitRInterpreter is not optionally required. so it does not matter KnitR is
> > optionally required or not.
> > R dependency in SparkR is exception of GPL. KnitR is not applied that
> > exception.
> >
> >
> > Amos, could you confirm my understanding to your interpretation is correct?
> > If it is not could you clarify it?
> >
> > Thanks,
> > moon
> >
> > On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <am...@gmail.com>
> > wrote:
> >
> > > Just to put the final nail in this, I looked it up.
> > >
> > > I see no evidence of any “compiler exception.”
> > >
> > > There is an exception in the license for the runtime libraries that are
> > > bundled with GCC. See:
> > > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> > >
> > > But no “compiler exception.”
> > >
> > > In fact, I believe this is part of the reason why LLVM was created.
> > >
> > > From: moon soo Lee <mo...@apache.org>
> > > Reply: moon soo Lee <mo...@apache.org>
> > > Date: December 1, 2015 at 8:16:36 PM
> > > To: Amos B. Elberg <am...@gmail.com>,
> > > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> > > request: R Interpreter for Zeppelin
> > >
> > > Is knitR is commonly considered as a interpreter/compiler? or is it
> > > considered as a library routine?
> > >
> > > Thanks,
> > > moon
> > >
> > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com>
> > > wrote:
> > > Moon - you give this as an explanation of the licensing issue:
> > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > >
> > > According to that, there is an exception in the GPL for interpreter
> > > languages. As long as you don’t distribute the code, its fine to talk to
> > > an interpreted language.
> > >
> > > Well, if that’s the case, then the PR plainly does not have a license
> > > issue. It doesn’t distribute any GPL’d R code.
> > >
> > > I’m not sure what’s confusing about this. It seems completely
> > > straightforward.
> > >
> > > Regarding this:
> > >
> > >
> > > --
> > > Amos Elberg
> > > Sent with Airmail
> > >
> > > From: moon soo Lee <mo...@apache.org>
> > > Reply: dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org>
> > > Date: December 1, 2015 at 6:48:47 PM
> > >
> > > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> > >
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> > > request: R Interpreter for Zeppelin
> > >
> > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com>
> > > wrote:
> > >
> > > > I am going to try to minimize my reaction to Moon’s e-mail.
> > > >
> > > > The tl;dr is this:
> > > >
> > > > The reason we are having this discussion now is that active users of
> > the
> > > > PR — which now has its own user base — went public to complain about
> > > this.
> > >
> > >
> > > > The PR has been tested by an active user base for more than three
> > months.
> > > > No-one has been able to identify any specific actual licensing problem,
> > > and
> > > > the PR was prepared based on an extensive, careful review of the
> > relevant
> > > > licensing issues and after contacting the relevant people.
> > > >
> > > >
> > >
> > > I admire every software that used by user and helping people. That
> > includes
> > > your work. But that's not the topic we're in discussion. Active user does
> > > not mean your contribution can ignore the review.
> > >
> > >
> > >
> > > > It is not an explanation for someone who has been ignoring my “how can
> > I
> > > > move this forward…” emails for three months to point the finger and
> > say I
> > > > didn’t contact the right person or file the right report.
> > > >
> > > >
> > > This is also not the topic in this discussion.
> > >
> > >
> > > > The burden for providing an explanation for the inaction is on the PMCC
> > > at
> > > > this point.
> > >
> > > I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> > > > core, why do you think other PRs are passing CI?
> > > > They’re not! I often see comments on PRs to just ignore that CI is
> > > > failing.
> > > >
> > > > One of the most common reasons this PR fails CI, is CI times-out
> > > > downloading Spark to install. How could that possibly be caused by the
> > > PR?
> > > >
> > > > It looks to me like the only PRs with changes to the relevant parts of
> > > the
> > > > code — the SparkInterpreter — are being made by the person who wrote
> > the
> > > > testing suite.
> > > >
> > > > So, that would explain why some other PRs pass CI: Neither the
> > > > SparkInterpreter nor the testing suite are stable or robust, but since
> > > the
> > > > PRs are coming from the person who wrote both…
> > > >
> > > > And let's say Zeppelin core has problem and that makes your PR fails on
> > > CI
> > > > test. That's possible. But it still does not mean we can merge it with
> > CI
> > > > failing.
> > > >
> > > > It means you should be working with me to figure out why the CI is
> > > failing.
> > > >
> > > > This PR has been tested by an active user base for the past three
> > months.
> > > > If CI is continuing to fail, and dozens of hours of effort have not
> > > > resolved the CI issues, then it is time to start considering whether
> > the
> > > > testing suite is part of the problem.
> > > >
> > > > The level of defensiveness about the CI and SparkInterpreter is not
> > > > helping to resolve these issues.
> > > >
> > > > If you think it's problem on Zeppelin core, then file an issue that
> > > > reproduce the problem on Zeppelin core, that might be more efficient
> > than
> > > > keep trying yourself.
> > > > I contacted you numerous times about such issues...
> > > >
> > >
> > >
> > > I remember i commented your issue about CI. but you just keep repeated
> > it's
> > > not your problem but Zeppelin core problem.
> > >
> > > Then please file an issue about the problem you found in Zeppelin Core.
> > > Then everyone will get into the problem.
> > >
> > >
> > >
> > > >
> > > > In my interpretation, KnitRInterpreter is not an optional feature while
> > > it
> > > > is always enabled when compiling Zeppelin and always enabled when
> > running
> > > > Zeppelin. And it requires dynamically linked GPL library on runtime.
> > (yes
> > > > it will fail when no KnitR is installed on the system)
> > > >
> > > > Its not always enabled.
> > > > It is not dynamically linked at runtime.
> > > > It will not fail when knitr is missing. If knitr is not present, the
> > repl
> > > > interpreter starts and a note is written to to the log that the knitr
> > > > interpreter isn’t available because knitr is not present.
> > > >
> > > > no Apache code can ever call a shell script, on the purpose of dynamic
> > > > linking with GPL library.
> > > > You misunderstand.
> > > >
> > > > The *shell* is GPL'd.
> > > >
> > > > Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends
> > on
> > > a
> > > > shell script to launch?
> > > >
> > > Obviously not.
> > > >
> > > > The interaction with R in the PR is the same.
> > > >
> > > >
> > >
> > > Again, bash is one of exceptions of GPL, like other GPL licensed
> > > compiler/interpreter.
> > >
> > > Check here why Bash and R is okay with Apache License.
> > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > >
> > > I'm not sure we can apply the same exception for 'using' KnitR.
> > >
> > > My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter'
> > you
> > > wrote is not an optional feature. Which is clearly not optionally enabled
> > > code and feature. And that depends on KnitR library which is GPL.
> > >
> > >
> > >
> > > > I was guessing SparkR can be still in Apache License even if it is
> > > depends
> > > > on R. Because of GPL licensed compiler generated output is not GPL
> > > license.
> > > > and R is sort of compiler. If you can get answer from Spark community
> > how
> > > > SparkR get managed to stay in Apache License while R is GPL, the answer
> > > > might help.
> > > > The description of SparkR is not accurate in any respect. (Do you think
> > > > SparkR is not talking to GPL-licensed libraries?)
> > > >
> > > > I don’t see that any genuine issue is being raised here.
> > > >
> > > > If there is an issue, the burden is on you to identify it.
> > > >
> > > > If i give you one suggestion, Zeppelin committers sometimes ask rebase
> > > the
> > > > contribution branch for some reason. It is not the really the best
> > > > practice, but still okay while most contributions are not including
> > large
> > > > code base changes
> > > > However, your one, has more than 4000 lines of code change. If you
> > rebase
> > > > then review should be started from the beginning, again. So you might
> > > want
> > > > to minimize the rebase your branch.
> > > >
> > > > Are you actually complaining that the problem is that I rebased the
> > code
> > > > during the three-month period when no-one looked at it and Zeppelin
> > went
> > > > through a release?
> > > >
> > > > I cannot take it seriously when you say things like this.
> > > >
> > > > Having to “start from the beginning” cannot be a problem if you never
> > > > actually started the first time...
> > > >
> > > >
> > >
> > > You wanted coordination and cooperation. So i gave you suggestion that
> > > helping review process. For example, your code has been rebased since my
> > > comment and jongyoul's comment. that means committers will need to look
> > > from the beginning. That'll require more time. And maybe, i guess that's
> > > not what you want. But If you don't care, feel free to rebase.
> > >
> > > Thanks,
> > > moon
> > >
> > >
> > >
> > > >
> > > > From: moon soo Lee <mo...@apache.org>
> > > > Reply: moon soo Lee <mo...@apache.org>
> > > > Date: December 1, 2015 at 4:42:06 AM
> > > > To: Amos B. Elberg <am...@gmail.com>,
> > > > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > pull
> > > > request: R Interpreter for Zeppelin
> > > >
> > > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
> > > > wrote:
> > > > Thank you, Cos.
> > > >
> > > > I’d like to briefly address the issues raised by Moon:
> > > >
> > > > 1. This PR does not passes CI
> > > > The CI fails on core Zeppelin, *not* code in this PR.
> > > >
> > > > I’ve been seeking assistance on this since August.
> > > >
> > > > The most common reason is that SparkInterpreter is unable to launch
> > Spark
> > > > and open a Spark Backend. This is necessary to test the PR.
> > > >
> > > > 60+ hours, has been spent adapting and re-basing when the
> > > SparkInterpreter
> > > > architecture changed and broke the PR’s *tests.*
> > > >
> > > >
> > > > I'm sorry, but the other PRs are passing CI. If it's problem on
> > Zeppelin
> > > > core, why do you think other PRs are passing CI?
> > > >
> > > > And let's say Zeppelin core has problem and that makes your PR fails on
> > > CI
> > > > test. That's possible. But it still does not mean we can merge it with
> > CI
> > > > failing.
> > > >
> > > > If you think it's problem on Zeppelin core, then file an issue that
> > > > reproduce the problem on Zeppelin core, that might be more efficient
> > than
> > > > keep trying yourself.
> > > >
> > > >
> > > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > > What license problem *specifically* do you believe may exist?
> > > >
> > > > In preparing the PR, I:
> > > >
> > > > * Reviewed the Apache policy in detail.
> > > >
> > > > * Contacted the FSF to ask their interpretation of the “linking”
> > > > provisions of the GPL license.
> > > >
> > > > * Reviewed how other Apache software deals with this issue (e.g., Spark
> > > > talks to R, which is GPL'd).
> > > >
> > > > * No necessary *dependencies* of the PR have license conflicts. In
> > > > several cases, I contacted package authors who agreed to re-issue their
> > > > packages under Apache-compatible licenses. (Usually I had to do a bit
> > of
> > > > coding in exchange…)
> > > >
> > > > * Where the license had to stay GPL, the packages are *not necessary*
> > and
> > > > *not dependencies.* If the optional packages are present, they will be
> > > > used to enable additional functionality. Knitr is an example. The PR
> > will
> > > > compile and run fine without knitr. If knitr is available (it is part
> > of
> > > > the most common R distribution), the PR will enable the knitr
> > > interpreter.
> > > >
> > > > * This is exactly how this issue is addressed through the Apache
> > > > ecosystem.
> > > > The tl;dr is this: When Apache code is written to talk to libraries
> > that
> > > > may or may not be present on the user’s system, or where it talks to an
> > > API
> > > > but is agnostic about implementation, that is not “linking” in a way
> > that
> > > > implicate the anti-linking provision of the GPL.
> > > >
> > > > Otherwise, no Apache code could ever call a shell script! Let alone run
> > > > on Linux, or talk to R.
> > > >
> > > >
> > > > I'm not a legal expert. So following could be wrong.
> > > >
> > > > In my interpretation, KnitRInterpreter is not an optional feature while
> > > it
> > > > is always enabled when compiling Zeppelin and always enabled when
> > running
> > > > Zeppelin. And it requires dynamically linked GPL library on runtime.
> > (yes
> > > > it will fail when no KnitR is installed on the system)
> > > >
> > > > And of course, no Apache code can ever call a shell script, on the
> > > purpose
> > > > of dynamic linking with GPL library.
> > > >
> > > > I was guessing SparkR can be still in Apache License even if it is
> > > depends
> > > > on R. Because of GPL licensed compiler generated output is not GPL
> > > license.
> > > > and R is sort of compiler.
> > > >
> > > > If you can get answer from Spark community how SparkR get managed to
> > stay
> > > > in Apache License while R is GPL, the answer might help.
> > > >
> > > >
> > > > 3. Need more time to review.
> > > > Has any reviewer has downloaded the PR or run the demo notebook? (Which
> > > > is there for the benefit of reviewers, and isn’t intended to go in
> > final
> > > > distribution.)
> > > >
> > > > How many +1 comments from users would you like to see?
> > > >
> > > > How much time do you believe is required?
> > > >
> > > >
> > > > It all depends on when CI is going to pass, when license problem is
> > going
> > > > to be cleared, and when a committer willing to review and responsible
> > to
> > > > commit your contribution.
> > > >
> > > >
> > > > 1. Large code base change
> > > > Large code base changes require coordination and cooperation. This PR
> > > > necessarily implicates the build scripts, testing code, the
> > > > SparkInterpreter, etc.
> > > >
> > > > I have been seeking to coordinate since August.
> > > >
> > > > I continue to stand ready to do so.
> > > >
> > > > -Amos
> > > >
> > > >
> > > > If i give you one suggestion, Zeppelin committers sometimes ask rebase
> > > the
> > > > contribution branch for some reason. It is not the really the best
> > > > practice, but still okay while most contributions are not including
> > large
> > > > code base changes.
> > > >
> > > > However, your one, has more than 4000 lines of code change. If you
> > rebase
> > > > then review should be started from the beginning, again. So you might
> > > want
> > > > to minimize the rebase your branch.
> > > >
> > > > Thanks,
> > > > moon
> > > >
> > > >
> > > > From: moon soo Lee <mo...@apache.org>
> > > > Reply: dev@zeppelin.incubator.apache.org <
> > > > dev@zeppelin.incubator.apache.org>
> > > > Date: December 1, 2015 at 1:34:19 AM
> > > > To: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org
> > > >
> > > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > pull
> > > > request: R Interpreter for Zeppelin
> > > >
> > > > Hi Cos,
> > > >
> > > > Thanks for opening a discussion.
> > > > My answer about 'Why this PR is open for three months' is
> > > >
> > > > 1. This PR does not passes CI
> > > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > > 3. Need more time to review.
> > > >
> > > > It's my personal answer, other committers may have different opinion.
> > > >
> > > >
> > > > I think the question should be generalized. Because this PR is not the
> > > only
> > > > PR that is in impasse. There're more. For example
> > > >
> > > > Here's some examples that PR is not been merged.
> > > > https://github.com/apache/incubator-zeppelin/pull/53,
> > > > https://github.com/apache/incubator-zeppelin/pull/60
> > > > and so on.
> > > >
> > > > I can categorize the cases, based on experience of involving Zeppelin
> > > > community from the beginning.
> > > >
> > > > 1. Large code base change
> > > >
> > > > When contribution has large code base changes, it tend to take more
> > time
> > > to
> > > > review and merged. Normally, most contributions merged in 1day~1 week.
> > > > But some contribution has large code base changes take 2~4 weeks. Few
> > > > contribution that has very large code base change take months.
> > > >
> > > > 2. Communication lost
> > > >
> > > > Sometimes, committer is not responding, or contributor is not
> > responding.
> > > >
> > > > 3. Opinion diverges
> > > >
> > > > For some changes, included in contribution. When people have different
> > > > opinion and it continue to diverges, those PRs are not been merged.
> > > >
> > > >
> > > > I think having a guide such as ping other committer when a committer is
> > > not
> > > > responding, and divide contribution into small peaces if possible,
> > would
> > > > help most of the cases.
> > > > Of course committer have to pay attention more to the contribution and
> > > > helping people. That's the first one.
> > > >
> > > > Thanks,
> > > > moon
> > > >
> > > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org>
> > > wrote:
> > > >
> > > > > To make sure we're on the same page, here are two threads that I
> > found
> > > > > related
> > > > > to this topic.
> > > > >
> > > > > Thread 1:
> > > > > Subject: R?
> > > > > Started on: July 1, 2015
> > > > >
> > > > > Thread 2:
> > > > > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > > > > Zeppelin
> > > > > Started on: August 13, 2015
> > > > >
> > > > > If you want to fetch these from our archive send emails to
> > > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > > >
> > > > > Cos
> > > > >
> > > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > > > > Guys,
> > > > > >
> > > > > > While catching up on my emails from the last a couple of weeks,
> > this
> > > > > thread
> > > > > > caught my attention. I am not normally paying much attention to the
> > > > code
> > > > > > reviews traffic from GH, but this one is pretty different as it
> > spans
> > > > > three
> > > > > > months and counting.
> > > > > >
> > > > > > First, here are my five cents:
> > > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > > > > contributed to
> > > > > > an ASF project this file should simply contain ASL2 text, like in
> > [1]
> > > > > > - r/pom.xml perhaps shouldn't contain a separate <developers>
> > > section,
> > > > > but
> > > > > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > > > > > maintain this in the source code, but we have the list of all the
> > > > > > committers on the project's site [2] Every new committer's first
> > > > > commit is
> > > > > > to update the team page ;)
> > > > > > - comments like in
> > > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > > >
> > > > > > +/**
> > > > > > + * Created by aelberg on 7/28/15.
> > > > > > + */
> > > > > >
> > > > > > is better to be removed. It has been already discussed here [3].
> > And
> > > > > the
> > > > > > initial discussion on the topic [4] was linked as well
> > > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> > > > > R-specific
> > > > > > stuff - I have no idea about R, honestly, but
> > > > > >
> > > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > > > ...
> > > > > > +Author: David B. Dahl
> > > > > >
> > > > > > shouldn't be here, IMO. Normally, if some additional licenses are
> > > > > used,
> > > > > > they have to be listed in the top-level NOTICE file (already
> > there).
> > > > > >
> > > > > > - I am not going to offer any comments on the technical merits of
> > the
> > > > > patch,
> > > > > > as I haven't tried it personally. However, I don't see any serious
> > > > > > technical objections to the functionality in question.
> > > > > >
> > > > > > So, the question is - why the PR is open for three months? I hasn't
> > > > been
> > > > > able
> > > > > > to get a clear answer. What I found out though is pretty
> > unsettling,
> > > > > really.
> > > > > > The communication on the topic is almost non-existent, except for
> > > this
> > > > > sparse
> > > > > > and bitter thread in the GH.
> > > > > >
> > > > > > I would love to hear from the committers what's stopping the
> > > acceptance
> > > > > of the
> > > > > > code, besides of the minor issues I've mentioned earlier? What are
> > > the
> > > > > reasons for it?
> > > > > > Is there anything wrong with it?
> > > > > > One of the responsibilities of the committers is to make sure the
> > > > > > contributions are reviewed; new contributors are guided and do
> > > > > understand how
> > > > > > the project ticks. The easy feedback flow attracts new people,
> > > allowing
> > > > > the
> > > > > > community to grow and thrive.
> > > > > >
> > > > > > I am asking _explicitely_ not to re-start the bickering I have
> > > already
> > > > > > seen. At this point I am interested in the purely technical side of
> > > > this.
> > > > > >
> > > > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > > [3]
> > > > >
> > > >
> > >
> > http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > > [4] http://s.apache.org/iZl
> > > > > >
> > > > > > With regards,
> > > > > > Cos
> > > > > >
> > > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > > Github user elbamos commented on the pull request:
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > > >
> > > > > > > The current push should resolve some issues with changes in the
> > > > > > > Spark-Zeppelin interface that had created issues for users, as
> > > > > well as
> > > > > > > support for 1.5.1.
> > > > > > >
> > > > > > > Knitr support is improved, and the reason for a separate knitr
> > > > > interpreter may be clearer now.
> > > > > > >
> > > > > > > The amount of code borrowed from rscala is reduced.
> > > > > > >
> > > > > > > I did not address issues with @author tags, or files under the R/
> > > > > > > folder. The reason is, to be blunt, I don't understand what the
> > > > > precise
> > > > > > > concerns actually are.
> > > > > > >
> > > > > > > Please note that I changed .travis.yml to only use spark 1.4 and
> > > > > later.
> > > > > > > I'm sure there is a better way to do it, but I'm just not in a
> > > > > position
> > > > > > > to try to figure out the entire Zeppelin build process.
> > > > > > >
> > > > > > > Integrating this with the rest of zeppelin is going to take some
> > > > > work
> > > > > > > regarding pom's, travis, and so forth. I can do a lot of that,
> > > > > but I'm
> > > > > > > going to need to discuss it with the people who have been
> > "owning"
> > > > > those
> > > > > > > files. There are just too many moving pieces here.
> > > > > > >
> > > > > > > Because of the size of this (which is, unfortunately, necessary),
> > > > > > > posting here is probably not the most efficient way. That is also
> > > > > true
> > > > > > > because certain people chose to use this PR as a forum to air
> > other
> > > > > > > issues. Therefore, it would be better if reviewers e-mail me
> > > > > directly.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Thanks Amos for clarifying your interpretation. Then i can update
summarize, like

Amos's interpretation

- KnitR is optionally required by KnitRInterpreter.
- R dependency in SparkR has no problem. So KnitR should be the same.
- KnitRInterpreter is able to compile without KnitR dependency.
- All interpreters (includes KnitRInterpreter) in Zeppelin are optional.


Moon's interpretation

- KnitRInterpreter require KnitR library on runtime. And KniteRInterpreter
is not optional feature.
- R dependency in SparkR is exception of GPL. KnitR is not applied that
exception.
- KnitR is dynamically linked from Zeppelin through R.
- All interpreters in Zeppelin are enabled by default.


The reason i summarizing this is, not to make a decision, but to give a
short summary before start a discussion in legal-discuss@.

If i add one link about compiler exception and library linking through
interpreter, see
http://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL.

If summary seems have no big problem, i'm staring discussion on
legal-discuss@ by referencing this thread.

Thanks,
moon

On Wed, Dec 2, 2015 at 6:56 PM Corneau Damien <co...@gmail.com> wrote:

> I think that what moon means is that:
> If we merge the way it is now, the KnitRInterpreter will be part of the
> code base, so it isn't optional at code base level.
>
> To make it even more simple:
> * If the code has the right licensing -> that code can be part of the
> source code, and can be including in a binary release
> * If the code doesn't have the right licensing -> it can't be part of the
> source code, and can't be included in a binary release
> * If the code doesn't have the right licensing but is imported at build
> time (libraries for example) -> it is not in the source code, it can't be
> included in binary release
>
> So in the case of licensing issues, it would need to be fully optional
> (user bring the interpreter in his directory and build Zeppelin with it)
>
>
> On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <am...@gmail.com>
> wrote:
>
>> Moon let me clarify:
>>
>> Interpreted code doesn’t “link.”  The wiki article actually explains it
>> pretty well — https://en.wikipedia.org/wiki/GPL_linking_exception
>>  “Linking” against a library means compiling its headers into a binary, the
>> way a C compiler works.  The 2008 e-mail Moon distributed, called this the
>> “interpreter exception.”
>>
>> As for whether GPL’d code is a “mandatory dependency,” if knitr is
>> missing the PR will compile, run and test just fine.  The user is never
>> prompted to download it.  The only effect is, if the user types “knitr” and
>> knitr isn’t there, we display an InterpreterError that knitr isn’t there.
>>
>> KnitRInterpreter is not optionally required. so it does not matter KnitR
>> is
>> optionally required or not.
>> Aren’t all interpreters optional?  You can turn them on and off in the
>> config files.
>>
>> Do you mean that the KnitRInterpreter class gets compiled to bytecode
>> even if knitr is missing?  So what?  That isn't a mandatory dependency or a
>> link.
>>
>> From: moon soo Lee <mo...@apache.org>
>> Reply: dev@zeppelin.incubator.apache.org <
>> dev@zeppelin.incubator.apache.org>
>> Date: December 2, 2015 at 3:18:00 AM
>> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
>> Subject:  Re: License of KnitRInterpreter Was: Re: contributions impasse.
>> Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin
>>
>> Let me summarize license concern about KnitRInterpreter.
>>
>>
>> Amos's interpretation.
>>
>> KnitR is optionally required by KnitRInterpreter.
>> R dependency in SparkR has no problem. So KnitR should be the same.
>>
>>
>> Moon's interpretation.
>>
>> KnitRInterpreter is not optionally required. so it does not matter KnitR
>> is
>> optionally required or not.
>> R dependency in SparkR is exception of GPL. KnitR is not applied that
>> exception.
>>
>>
>> Amos, could you confirm my understanding to your interpretation is
>> correct?
>> If it is not could you clarify it?
>>
>> Thanks,
>> moon
>>
>> On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <am...@gmail.com>
>> wrote:
>>
>> > Just to put the final nail in this, I looked it up.
>> >
>> > I see no evidence of any “compiler exception.”
>> >
>> > There is an exception in the license for the runtime libraries that are
>> > bundled with GCC. See:
>> > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
>> >
>> > But no “compiler exception.”
>> >
>> > In fact, I believe this is part of the reason why LLVM was created.
>> >
>> > From: moon soo Lee <mo...@apache.org>
>> > Reply: moon soo Lee <mo...@apache.org>
>> > Date: December 1, 2015 at 8:16:36 PM
>> > To: Amos B. Elberg <am...@gmail.com>,
>> > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
>> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>> pull
>> > request: R Interpreter for Zeppelin
>> >
>> > Is knitR is commonly considered as a interpreter/compiler? or is it
>> > considered as a library routine?
>> >
>> > Thanks,
>> > moon
>> >
>> > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com>
>> > wrote:
>> > Moon - you give this as an explanation of the licensing issue:
>> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
>> >
>> > According to that, there is an exception in the GPL for interpreter
>> > languages. As long as you don’t distribute the code, its fine to talk to
>> > an interpreted language.
>> >
>> > Well, if that’s the case, then the PR plainly does not have a license
>> > issue. It doesn’t distribute any GPL’d R code.
>> >
>> > I’m not sure what’s confusing about this. It seems completely
>> > straightforward.
>> >
>> > Regarding this:
>> >
>> >
>> > --
>> > Amos Elberg
>> > Sent with Airmail
>> >
>> > From: moon soo Lee <mo...@apache.org>
>> > Reply: dev@zeppelin.incubator.apache.org <
>> > dev@zeppelin.incubator.apache.org>
>> > Date: December 1, 2015 at 6:48:47 PM
>> >
>> > To: dev@zeppelin.incubator.apache.org <
>> dev@zeppelin.incubator.apache.org>
>> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>> pull
>> > request: R Interpreter for Zeppelin
>> >
>> > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com>
>> > wrote:
>> >
>> > > I am going to try to minimize my reaction to Moon’s e-mail.
>> > >
>> > > The tl;dr is this:
>> > >
>> > > The reason we are having this discussion now is that active users of
>> the
>> > > PR — which now has its own user base — went public to complain about
>> > this.
>> >
>> >
>> > > The PR has been tested by an active user base for more than three
>> months.
>> > > No-one has been able to identify any specific actual licensing
>> problem,
>> > and
>> > > the PR was prepared based on an extensive, careful review of the
>> relevant
>> > > licensing issues and after contacting the relevant people.
>> > >
>> > >
>> >
>> > I admire every software that used by user and helping people. That
>> includes
>> > your work. But that's not the topic we're in discussion. Active user
>> does
>> > not mean your contribution can ignore the review.
>> >
>> >
>> >
>> > > It is not an explanation for someone who has been ignoring my “how
>> can I
>> > > move this forward…” emails for three months to point the finger and
>> say I
>> > > didn’t contact the right person or file the right report.
>> > >
>> > >
>> > This is also not the topic in this discussion.
>> >
>> >
>> > > The burden for providing an explanation for the inaction is on the
>> PMCC
>> > at
>> > > this point.
>> >
>> > I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
>> > > core, why do you think other PRs are passing CI?
>> > > They’re not! I often see comments on PRs to just ignore that CI is
>> > > failing.
>> > >
>> > > One of the most common reasons this PR fails CI, is CI times-out
>> > > downloading Spark to install. How could that possibly be caused by the
>> > PR?
>> > >
>> > > It looks to me like the only PRs with changes to the relevant parts of
>> > the
>> > > code — the SparkInterpreter — are being made by the person who wrote
>> the
>> > > testing suite.
>> > >
>> > > So, that would explain why some other PRs pass CI: Neither the
>> > > SparkInterpreter nor the testing suite are stable or robust, but since
>> > the
>> > > PRs are coming from the person who wrote both…
>> > >
>> > > And let's say Zeppelin core has problem and that makes your PR fails
>> on
>> > CI
>> > > test. That's possible. But it still does not mean we can merge it
>> with CI
>> > > failing.
>> > >
>> > > It means you should be working with me to figure out why the CI is
>> > failing.
>> > >
>> > > This PR has been tested by an active user base for the past three
>> months.
>> > > If CI is continuing to fail, and dozens of hours of effort have not
>> > > resolved the CI issues, then it is time to start considering whether
>> the
>> > > testing suite is part of the problem.
>> > >
>> > > The level of defensiveness about the CI and SparkInterpreter is not
>> > > helping to resolve these issues.
>> > >
>> > > If you think it's problem on Zeppelin core, then file an issue that
>> > > reproduce the problem on Zeppelin core, that might be more efficient
>> than
>> > > keep trying yourself.
>> > > I contacted you numerous times about such issues...
>> > >
>> >
>> >
>> > I remember i commented your issue about CI. but you just keep repeated
>> it's
>> > not your problem but Zeppelin core problem.
>> >
>> > Then please file an issue about the problem you found in Zeppelin Core.
>> > Then everyone will get into the problem.
>> >
>> >
>> >
>> > >
>> > > In my interpretation, KnitRInterpreter is not an optional feature
>> while
>> > it
>> > > is always enabled when compiling Zeppelin and always enabled when
>> running
>> > > Zeppelin. And it requires dynamically linked GPL library on runtime.
>> (yes
>> > > it will fail when no KnitR is installed on the system)
>> > >
>> > > Its not always enabled.
>> > > It is not dynamically linked at runtime.
>> > > It will not fail when knitr is missing. If knitr is not present, the
>> repl
>> > > interpreter starts and a note is written to to the log that the knitr
>> > > interpreter isn’t available because knitr is not present.
>> > >
>> > > no Apache code can ever call a shell script, on the purpose of dynamic
>> > > linking with GPL library.
>> > > You misunderstand.
>> > >
>> > > The *shell* is GPL'd.
>> > >
>> > > Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends
>> on
>> > a
>> > > shell script to launch?
>> > >
>> > Obviously not.
>> > >
>> > > The interaction with R in the PR is the same.
>> > >
>> > >
>> >
>> > Again, bash is one of exceptions of GPL, like other GPL licensed
>> > compiler/interpreter.
>> >
>> > Check here why Bash and R is okay with Apache License.
>> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
>> >
>> > I'm not sure we can apply the same exception for 'using' KnitR.
>> >
>> > My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter'
>> you
>> > wrote is not an optional feature. Which is clearly not optionally
>> enabled
>> > code and feature. And that depends on KnitR library which is GPL.
>> >
>> >
>> >
>> > > I was guessing SparkR can be still in Apache License even if it is
>> > depends
>> > > on R. Because of GPL licensed compiler generated output is not GPL
>> > license.
>> > > and R is sort of compiler. If you can get answer from Spark community
>> how
>> > > SparkR get managed to stay in Apache License while R is GPL, the
>> answer
>> > > might help.
>> > > The description of SparkR is not accurate in any respect. (Do you
>> think
>> > > SparkR is not talking to GPL-licensed libraries?)
>> > >
>> > > I don’t see that any genuine issue is being raised here.
>> > >
>> > > If there is an issue, the burden is on you to identify it.
>> > >
>> > > If i give you one suggestion, Zeppelin committers sometimes ask rebase
>> > the
>> > > contribution branch for some reason. It is not the really the best
>> > > practice, but still okay while most contributions are not including
>> large
>> > > code base changes
>> > > However, your one, has more than 4000 lines of code change. If you
>> rebase
>> > > then review should be started from the beginning, again. So you might
>> > want
>> > > to minimize the rebase your branch.
>> > >
>> > > Are you actually complaining that the problem is that I rebased the
>> code
>> > > during the three-month period when no-one looked at it and Zeppelin
>> went
>> > > through a release?
>> > >
>> > > I cannot take it seriously when you say things like this.
>> > >
>> > > Having to “start from the beginning” cannot be a problem if you never
>> > > actually started the first time...
>> > >
>> > >
>> >
>> > You wanted coordination and cooperation. So i gave you suggestion that
>> > helping review process. For example, your code has been rebased since my
>> > comment and jongyoul's comment. that means committers will need to look
>> > from the beginning. That'll require more time. And maybe, i guess that's
>> > not what you want. But If you don't care, feel free to rebase.
>> >
>> > Thanks,
>> > moon
>> >
>> >
>> >
>> > >
>> > > From: moon soo Lee <mo...@apache.org>
>> > > Reply: moon soo Lee <mo...@apache.org>
>> > > Date: December 1, 2015 at 4:42:06 AM
>> > > To: Amos B. Elberg <am...@gmail.com>,
>> > > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
>> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>> pull
>> > > request: R Interpreter for Zeppelin
>> > >
>> > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
>> > > wrote:
>> > > Thank you, Cos.
>> > >
>> > > I’d like to briefly address the issues raised by Moon:
>> > >
>> > > 1. This PR does not passes CI
>> > > The CI fails on core Zeppelin, *not* code in this PR.
>> > >
>> > > I’ve been seeking assistance on this since August.
>> > >
>> > > The most common reason is that SparkInterpreter is unable to launch
>> Spark
>> > > and open a Spark Backend. This is necessary to test the PR.
>> > >
>> > > 60+ hours, has been spent adapting and re-basing when the
>> > SparkInterpreter
>> > > architecture changed and broke the PR’s *tests.*
>> > >
>> > >
>> > > I'm sorry, but the other PRs are passing CI. If it's problem on
>> Zeppelin
>> > > core, why do you think other PRs are passing CI?
>> > >
>> > > And let's say Zeppelin core has problem and that makes your PR fails
>> on
>> > CI
>> > > test. That's possible. But it still does not mean we can merge it
>> with CI
>> > > failing.
>> > >
>> > > If you think it's problem on Zeppelin core, then file an issue that
>> > > reproduce the problem on Zeppelin core, that might be more efficient
>> than
>> > > keep trying yourself.
>> > >
>> > >
>> > > 2. Not 100% sure this PR has no license issue. (about KniteR)
>> > > What license problem *specifically* do you believe may exist?
>> > >
>> > > In preparing the PR, I:
>> > >
>> > > * Reviewed the Apache policy in detail.
>> > >
>> > > * Contacted the FSF to ask their interpretation of the “linking”
>> > > provisions of the GPL license.
>> > >
>> > > * Reviewed how other Apache software deals with this issue (e.g.,
>> Spark
>> > > talks to R, which is GPL'd).
>> > >
>> > > * No necessary *dependencies* of the PR have license conflicts. In
>> > > several cases, I contacted package authors who agreed to re-issue
>> their
>> > > packages under Apache-compatible licenses. (Usually I had to do a bit
>> of
>> > > coding in exchange…)
>> > >
>> > > * Where the license had to stay GPL, the packages are *not necessary*
>> and
>> > > *not dependencies.* If the optional packages are present, they will be
>> > > used to enable additional functionality. Knitr is an example. The PR
>> will
>> > > compile and run fine without knitr. If knitr is available (it is part
>> of
>> > > the most common R distribution), the PR will enable the knitr
>> > interpreter.
>> > >
>> > > * This is exactly how this issue is addressed through the Apache
>> > > ecosystem.
>> > > The tl;dr is this: When Apache code is written to talk to libraries
>> that
>> > > may or may not be present on the user’s system, or where it talks to
>> an
>> > API
>> > > but is agnostic about implementation, that is not “linking” in a way
>> that
>> > > implicate the anti-linking provision of the GPL.
>> > >
>> > > Otherwise, no Apache code could ever call a shell script! Let alone
>> run
>> > > on Linux, or talk to R.
>> > >
>> > >
>> > > I'm not a legal expert. So following could be wrong.
>> > >
>> > > In my interpretation, KnitRInterpreter is not an optional feature
>> while
>> > it
>> > > is always enabled when compiling Zeppelin and always enabled when
>> running
>> > > Zeppelin. And it requires dynamically linked GPL library on runtime.
>> (yes
>> > > it will fail when no KnitR is installed on the system)
>> > >
>> > > And of course, no Apache code can ever call a shell script, on the
>> > purpose
>> > > of dynamic linking with GPL library.
>> > >
>> > > I was guessing SparkR can be still in Apache License even if it is
>> > depends
>> > > on R. Because of GPL licensed compiler generated output is not GPL
>> > license.
>> > > and R is sort of compiler.
>> > >
>> > > If you can get answer from Spark community how SparkR get managed to
>> stay
>> > > in Apache License while R is GPL, the answer might help.
>> > >
>> > >
>> > > 3. Need more time to review.
>> > > Has any reviewer has downloaded the PR or run the demo notebook?
>> (Which
>> > > is there for the benefit of reviewers, and isn’t intended to go in
>> final
>> > > distribution.)
>> > >
>> > > How many +1 comments from users would you like to see?
>> > >
>> > > How much time do you believe is required?
>> > >
>> > >
>> > > It all depends on when CI is going to pass, when license problem is
>> going
>> > > to be cleared, and when a committer willing to review and responsible
>> to
>> > > commit your contribution.
>> > >
>> > >
>> > > 1. Large code base change
>> > > Large code base changes require coordination and cooperation. This PR
>> > > necessarily implicates the build scripts, testing code, the
>> > > SparkInterpreter, etc.
>> > >
>> > > I have been seeking to coordinate since August.
>> > >
>> > > I continue to stand ready to do so.
>> > >
>> > > -Amos
>> > >
>> > >
>> > > If i give you one suggestion, Zeppelin committers sometimes ask rebase
>> > the
>> > > contribution branch for some reason. It is not the really the best
>> > > practice, but still okay while most contributions are not including
>> large
>> > > code base changes.
>> > >
>> > > However, your one, has more than 4000 lines of code change. If you
>> rebase
>> > > then review should be started from the beginning, again. So you might
>> > want
>> > > to minimize the rebase your branch.
>> > >
>> > > Thanks,
>> > > moon
>> > >
>> > >
>> > > From: moon soo Lee <mo...@apache.org>
>> > > Reply: dev@zeppelin.incubator.apache.org <
>> > > dev@zeppelin.incubator.apache.org>
>> > > Date: December 1, 2015 at 1:34:19 AM
>> > > To: dev@zeppelin.incubator.apache.org <
>> dev@zeppelin.incubator.apache.org
>> > >
>> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
>> pull
>> > > request: R Interpreter for Zeppelin
>> > >
>> > > Hi Cos,
>> > >
>> > > Thanks for opening a discussion.
>> > > My answer about 'Why this PR is open for three months' is
>> > >
>> > > 1. This PR does not passes CI
>> > > 2. Not 100% sure this PR has no license issue. (about KniteR)
>> > > 3. Need more time to review.
>> > >
>> > > It's my personal answer, other committers may have different opinion.
>> > >
>> > >
>> > > I think the question should be generalized. Because this PR is not the
>> > only
>> > > PR that is in impasse. There're more. For example
>> > >
>> > > Here's some examples that PR is not been merged.
>> > > https://github.com/apache/incubator-zeppelin/pull/53,
>> > > https://github.com/apache/incubator-zeppelin/pull/60
>> > > and so on.
>> > >
>> > > I can categorize the cases, based on experience of involving Zeppelin
>> > > community from the beginning.
>> > >
>> > > 1. Large code base change
>> > >
>> > > When contribution has large code base changes, it tend to take more
>> time
>> > to
>> > > review and merged. Normally, most contributions merged in 1day~1 week.
>> > > But some contribution has large code base changes take 2~4 weeks. Few
>> > > contribution that has very large code base change take months.
>> > >
>> > > 2. Communication lost
>> > >
>> > > Sometimes, committer is not responding, or contributor is not
>> responding.
>> > >
>> > > 3. Opinion diverges
>> > >
>> > > For some changes, included in contribution. When people have different
>> > > opinion and it continue to diverges, those PRs are not been merged.
>> > >
>> > >
>> > > I think having a guide such as ping other committer when a committer
>> is
>> > not
>> > > responding, and divide contribution into small peaces if possible,
>> would
>> > > help most of the cases.
>> > > Of course committer have to pay attention more to the contribution and
>> > > helping people. That's the first one.
>> > >
>> > > Thanks,
>> > > moon
>> > >
>> > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org>
>> > wrote:
>> > >
>> > > > To make sure we're on the same page, here are two threads that I
>> found
>> > > > related
>> > > > to this topic.
>> > > >
>> > > > Thread 1:
>> > > > Subject: R?
>> > > > Started on: July 1, 2015
>> > > >
>> > > > Thread 2:
>> > > > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
>> > > > Zeppelin
>> > > > Started on: August 13, 2015
>> > > >
>> > > > If you want to fetch these from our archive send emails to
>> > > > dev-thread.1735@zeppelin.incubator.apache.org
>> > > > dev-thread.3573@zeppelin.incubator.apache.org
>> > > >
>> > > > Cos
>> > > >
>> > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
>> > > > > Guys,
>> > > > >
>> > > > > While catching up on my emails from the last a couple of weeks,
>> this
>> > > > thread
>> > > > > caught my attention. I am not normally paying much attention to
>> the
>> > > code
>> > > > > reviews traffic from GH, but this one is pretty different as it
>> spans
>> > > > three
>> > > > > months and counting.
>> > > > >
>> > > > > First, here are my five cents:
>> > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
>> > > > contributed to
>> > > > > an ASF project this file should simply contain ASL2 text, like in
>> [1]
>> > > > > - r/pom.xml perhaps shouldn't contain a separate <developers>
>> > section,
>> > > > but
>> > > > > Zeppelin might have different guidelines on it. Say, Bigtop
>> doesn't
>> > > > > maintain this in the source code, but we have the list of all the
>> > > > > committers on the project's site [2] Every new committer's first
>> > > > commit is
>> > > > > to update the team page ;)
>> > > > > - comments like in
>> > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
>> > > > >
>> > > > > +/**
>> > > > > + * Created by aelberg on 7/28/15.
>> > > > > + */
>> > > > >
>> > > > > is better to be removed. It has been already discussed here [3].
>> And
>> > > > the
>> > > > > initial discussion on the topic [4] was linked as well
>> > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
>> > > > R-specific
>> > > > > stuff - I have no idea about R, honestly, but
>> > > > >
>> > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
>> > > > > ...
>> > > > > +Author: David B. Dahl
>> > > > >
>> > > > > shouldn't be here, IMO. Normally, if some additional licenses are
>> > > > used,
>> > > > > they have to be listed in the top-level NOTICE file (already
>> there).
>> > > > >
>> > > > > - I am not going to offer any comments on the technical merits of
>> the
>> > > > patch,
>> > > > > as I haven't tried it personally. However, I don't see any serious
>> > > > > technical objections to the functionality in question.
>> > > > >
>> > > > > So, the question is - why the PR is open for three months? I
>> hasn't
>> > > been
>> > > > able
>> > > > > to get a clear answer. What I found out though is pretty
>> unsettling,
>> > > > really.
>> > > > > The communication on the topic is almost non-existent, except for
>> > this
>> > > > sparse
>> > > > > and bitter thread in the GH.
>> > > > >
>> > > > > I would love to hear from the committers what's stopping the
>> > acceptance
>> > > > of the
>> > > > > code, besides of the minor issues I've mentioned earlier? What are
>> > the
>> > > > reasons for it?
>> > > > > Is there anything wrong with it?
>> > > > > One of the responsibilities of the committers is to make sure the
>> > > > > contributions are reviewed; new contributors are guided and do
>> > > > understand how
>> > > > > the project ticks. The easy feedback flow attracts new people,
>> > allowing
>> > > > the
>> > > > > community to grow and thrive.
>> > > > >
>> > > > > I am asking _explicitely_ not to re-start the bickering I have
>> > already
>> > > > > seen. At this point I am interested in the purely technical side
>> of
>> > > this.
>> > > > >
>> > > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
>> > > > > [2] http://bigtop.apache.org/team-list.html
>> > > > > [3]
>> > > >
>> > >
>> >
>> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
>> > > > > [4] http://s.apache.org/iZl
>> > > > >
>> > > > > With regards,
>> > > > > Cos
>> > > > >
>> > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
>> > > > > > Github user elbamos commented on the pull request:
>> > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
>> > > > > >
>> > > > > > The current push should resolve some issues with changes in the
>> > > > > > Spark-Zeppelin interface that had created issues for users, as
>> > > > well as
>> > > > > > support for 1.5.1.
>> > > > > >
>> > > > > > Knitr support is improved, and the reason for a separate knitr
>> > > > interpreter may be clearer now.
>> > > > > >
>> > > > > > The amount of code borrowed from rscala is reduced.
>> > > > > >
>> > > > > > I did not address issues with @author tags, or files under the
>> R/
>> > > > > > folder. The reason is, to be blunt, I don't understand what the
>> > > > precise
>> > > > > > concerns actually are.
>> > > > > >
>> > > > > > Please note that I changed .travis.yml to only use spark 1.4 and
>> > > > later.
>> > > > > > I'm sure there is a better way to do it, but I'm just not in a
>> > > > position
>> > > > > > to try to figure out the entire Zeppelin build process.
>> > > > > >
>> > > > > > Integrating this with the rest of zeppelin is going to take some
>> > > > work
>> > > > > > regarding pom's, travis, and so forth. I can do a lot of that,
>> > > > but I'm
>> > > > > > going to need to discuss it with the people who have been
>> "owning"
>> > > > those
>> > > > > > files. There are just too many moving pieces here.
>> > > > > >
>> > > > > > Because of the size of this (which is, unfortunately,
>> necessary),
>> > > > > > posting here is probably not the most efficient way. That is
>> also
>> > > > true
>> > > > > > because certain people chose to use this PR as a forum to air
>> other
>> > > > > > issues. Therefore, it would be better if reviewers e-mail me
>> > > > directly.
>> > > >
>> > > >
>> > > >
>> > >
>> >
>>
>
>

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Corneau Damien <co...@gmail.com>]

I think that what moon means is that:
If we merge the way it is now, the KnitRInterpreter will be part of the
code base, so it isn't optional at code base level.

To make it even more simple:
* If the code has the right licensing -> that code can be part of the
source code, and can be including in a binary release
* If the code doesn't have the right licensing -> it can't be part of the
source code, and can't be included in a binary release
* If the code doesn't have the right licensing but is imported at build
time (libraries for example) -> it is not in the source code, it can't be
included in binary release

So in the case of licensing issues, it would need to be fully optional
(user bring the interpreter in his directory and build Zeppelin with it)


On Wed, Dec 2, 2015 at 6:33 PM, Amos B. Elberg <am...@gmail.com>
wrote:

> Moon let me clarify:
>
> Interpreted code doesn’t “link.”  The wiki article actually explains it
> pretty well — https://en.wikipedia.org/wiki/GPL_linking_exception
>  “Linking” against a library means compiling its headers into a binary, the
> way a C compiler works.  The 2008 e-mail Moon distributed, called this the
> “interpreter exception.”
>
> As for whether GPL’d code is a “mandatory dependency,” if knitr is missing
> the PR will compile, run and test just fine.  The user is never prompted to
> download it.  The only effect is, if the user types “knitr” and knitr isn’t
> there, we display an InterpreterError that knitr isn’t there.
>
> KnitRInterpreter is not optionally required. so it does not matter KnitR
> is
> optionally required or not.
> Aren’t all interpreters optional?  You can turn them on and off in the
> config files.
>
> Do you mean that the KnitRInterpreter class gets compiled to bytecode even
> if knitr is missing?  So what?  That isn't a mandatory dependency or a link.
>
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> Date: December 2, 2015 at 3:18:00 AM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: License of KnitRInterpreter Was: Re: contributions impasse.
> Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin
>
> Let me summarize license concern about KnitRInterpreter.
>
>
> Amos's interpretation.
>
> KnitR is optionally required by KnitRInterpreter.
> R dependency in SparkR has no problem. So KnitR should be the same.
>
>
> Moon's interpretation.
>
> KnitRInterpreter is not optionally required. so it does not matter KnitR is
> optionally required or not.
> R dependency in SparkR is exception of GPL. KnitR is not applied that
> exception.
>
>
> Amos, could you confirm my understanding to your interpretation is correct?
> If it is not could you clarify it?
>
> Thanks,
> moon
>
> On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <am...@gmail.com>
> wrote:
>
> > Just to put the final nail in this, I looked it up.
> >
> > I see no evidence of any “compiler exception.”
> >
> > There is an exception in the license for the runtime libraries that are
> > bundled with GCC. See:
> > http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
> >
> > But no “compiler exception.”
> >
> > In fact, I believe this is part of the reason why LLVM was created.
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: moon soo Lee <mo...@apache.org>
> > Date: December 1, 2015 at 8:16:36 PM
> > To: Amos B. Elberg <am...@gmail.com>,
> > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> > request: R Interpreter for Zeppelin
> >
> > Is knitR is commonly considered as a interpreter/compiler? or is it
> > considered as a library routine?
> >
> > Thanks,
> > moon
> >
> > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com>
> > wrote:
> > Moon - you give this as an explanation of the licensing issue:
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> >
> > According to that, there is an exception in the GPL for interpreter
> > languages. As long as you don’t distribute the code, its fine to talk to
> > an interpreted language.
> >
> > Well, if that’s the case, then the PR plainly does not have a license
> > issue. It doesn’t distribute any GPL’d R code.
> >
> > I’m not sure what’s confusing about this. It seems completely
> > straightforward.
> >
> > Regarding this:
> >
> >
> > --
> > Amos Elberg
> > Sent with Airmail
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > Date: December 1, 2015 at 6:48:47 PM
> >
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> > request: R Interpreter for Zeppelin
> >
> > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com>
> > wrote:
> >
> > > I am going to try to minimize my reaction to Moon’s e-mail.
> > >
> > > The tl;dr is this:
> > >
> > > The reason we are having this discussion now is that active users of
> the
> > > PR — which now has its own user base — went public to complain about
> > this.
> >
> >
> > > The PR has been tested by an active user base for more than three
> months.
> > > No-one has been able to identify any specific actual licensing problem,
> > and
> > > the PR was prepared based on an extensive, careful review of the
> relevant
> > > licensing issues and after contacting the relevant people.
> > >
> > >
> >
> > I admire every software that used by user and helping people. That
> includes
> > your work. But that's not the topic we're in discussion. Active user does
> > not mean your contribution can ignore the review.
> >
> >
> >
> > > It is not an explanation for someone who has been ignoring my “how can
> I
> > > move this forward…” emails for three months to point the finger and
> say I
> > > didn’t contact the right person or file the right report.
> > >
> > >
> > This is also not the topic in this discussion.
> >
> >
> > > The burden for providing an explanation for the inaction is on the PMCC
> > at
> > > this point.
> >
> > I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> > > core, why do you think other PRs are passing CI?
> > > They’re not! I often see comments on PRs to just ignore that CI is
> > > failing.
> > >
> > > One of the most common reasons this PR fails CI, is CI times-out
> > > downloading Spark to install. How could that possibly be caused by the
> > PR?
> > >
> > > It looks to me like the only PRs with changes to the relevant parts of
> > the
> > > code — the SparkInterpreter — are being made by the person who wrote
> the
> > > testing suite.
> > >
> > > So, that would explain why some other PRs pass CI: Neither the
> > > SparkInterpreter nor the testing suite are stable or robust, but since
> > the
> > > PRs are coming from the person who wrote both…
> > >
> > > And let's say Zeppelin core has problem and that makes your PR fails on
> > CI
> > > test. That's possible. But it still does not mean we can merge it with
> CI
> > > failing.
> > >
> > > It means you should be working with me to figure out why the CI is
> > failing.
> > >
> > > This PR has been tested by an active user base for the past three
> months.
> > > If CI is continuing to fail, and dozens of hours of effort have not
> > > resolved the CI issues, then it is time to start considering whether
> the
> > > testing suite is part of the problem.
> > >
> > > The level of defensiveness about the CI and SparkInterpreter is not
> > > helping to resolve these issues.
> > >
> > > If you think it's problem on Zeppelin core, then file an issue that
> > > reproduce the problem on Zeppelin core, that might be more efficient
> than
> > > keep trying yourself.
> > > I contacted you numerous times about such issues...
> > >
> >
> >
> > I remember i commented your issue about CI. but you just keep repeated
> it's
> > not your problem but Zeppelin core problem.
> >
> > Then please file an issue about the problem you found in Zeppelin Core.
> > Then everyone will get into the problem.
> >
> >
> >
> > >
> > > In my interpretation, KnitRInterpreter is not an optional feature while
> > it
> > > is always enabled when compiling Zeppelin and always enabled when
> running
> > > Zeppelin. And it requires dynamically linked GPL library on runtime.
> (yes
> > > it will fail when no KnitR is installed on the system)
> > >
> > > Its not always enabled.
> > > It is not dynamically linked at runtime.
> > > It will not fail when knitr is missing. If knitr is not present, the
> repl
> > > interpreter starts and a note is written to to the log that the knitr
> > > interpreter isn’t available because knitr is not present.
> > >
> > > no Apache code can ever call a shell script, on the purpose of dynamic
> > > linking with GPL library.
> > > You misunderstand.
> > >
> > > The *shell* is GPL'd.
> > >
> > > Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends
> on
> > a
> > > shell script to launch?
> > >
> > Obviously not.
> > >
> > > The interaction with R in the PR is the same.
> > >
> > >
> >
> > Again, bash is one of exceptions of GPL, like other GPL licensed
> > compiler/interpreter.
> >
> > Check here why Bash and R is okay with Apache License.
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> >
> > I'm not sure we can apply the same exception for 'using' KnitR.
> >
> > My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter'
> you
> > wrote is not an optional feature. Which is clearly not optionally enabled
> > code and feature. And that depends on KnitR library which is GPL.
> >
> >
> >
> > > I was guessing SparkR can be still in Apache License even if it is
> > depends
> > > on R. Because of GPL licensed compiler generated output is not GPL
> > license.
> > > and R is sort of compiler. If you can get answer from Spark community
> how
> > > SparkR get managed to stay in Apache License while R is GPL, the answer
> > > might help.
> > > The description of SparkR is not accurate in any respect. (Do you think
> > > SparkR is not talking to GPL-licensed libraries?)
> > >
> > > I don’t see that any genuine issue is being raised here.
> > >
> > > If there is an issue, the burden is on you to identify it.
> > >
> > > If i give you one suggestion, Zeppelin committers sometimes ask rebase
> > the
> > > contribution branch for some reason. It is not the really the best
> > > practice, but still okay while most contributions are not including
> large
> > > code base changes
> > > However, your one, has more than 4000 lines of code change. If you
> rebase
> > > then review should be started from the beginning, again. So you might
> > want
> > > to minimize the rebase your branch.
> > >
> > > Are you actually complaining that the problem is that I rebased the
> code
> > > during the three-month period when no-one looked at it and Zeppelin
> went
> > > through a release?
> > >
> > > I cannot take it seriously when you say things like this.
> > >
> > > Having to “start from the beginning” cannot be a problem if you never
> > > actually started the first time...
> > >
> > >
> >
> > You wanted coordination and cooperation. So i gave you suggestion that
> > helping review process. For example, your code has been rebased since my
> > comment and jongyoul's comment. that means committers will need to look
> > from the beginning. That'll require more time. And maybe, i guess that's
> > not what you want. But If you don't care, feel free to rebase.
> >
> > Thanks,
> > moon
> >
> >
> >
> > >
> > > From: moon soo Lee <mo...@apache.org>
> > > Reply: moon soo Lee <mo...@apache.org>
> > > Date: December 1, 2015 at 4:42:06 AM
> > > To: Amos B. Elberg <am...@gmail.com>,
> > > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull
> > > request: R Interpreter for Zeppelin
> > >
> > > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
> > > wrote:
> > > Thank you, Cos.
> > >
> > > I’d like to briefly address the issues raised by Moon:
> > >
> > > 1. This PR does not passes CI
> > > The CI fails on core Zeppelin, *not* code in this PR.
> > >
> > > I’ve been seeking assistance on this since August.
> > >
> > > The most common reason is that SparkInterpreter is unable to launch
> Spark
> > > and open a Spark Backend. This is necessary to test the PR.
> > >
> > > 60+ hours, has been spent adapting and re-basing when the
> > SparkInterpreter
> > > architecture changed and broke the PR’s *tests.*
> > >
> > >
> > > I'm sorry, but the other PRs are passing CI. If it's problem on
> Zeppelin
> > > core, why do you think other PRs are passing CI?
> > >
> > > And let's say Zeppelin core has problem and that makes your PR fails on
> > CI
> > > test. That's possible. But it still does not mean we can merge it with
> CI
> > > failing.
> > >
> > > If you think it's problem on Zeppelin core, then file an issue that
> > > reproduce the problem on Zeppelin core, that might be more efficient
> than
> > > keep trying yourself.
> > >
> > >
> > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > What license problem *specifically* do you believe may exist?
> > >
> > > In preparing the PR, I:
> > >
> > > * Reviewed the Apache policy in detail.
> > >
> > > * Contacted the FSF to ask their interpretation of the “linking”
> > > provisions of the GPL license.
> > >
> > > * Reviewed how other Apache software deals with this issue (e.g., Spark
> > > talks to R, which is GPL'd).
> > >
> > > * No necessary *dependencies* of the PR have license conflicts. In
> > > several cases, I contacted package authors who agreed to re-issue their
> > > packages under Apache-compatible licenses. (Usually I had to do a bit
> of
> > > coding in exchange…)
> > >
> > > * Where the license had to stay GPL, the packages are *not necessary*
> and
> > > *not dependencies.* If the optional packages are present, they will be
> > > used to enable additional functionality. Knitr is an example. The PR
> will
> > > compile and run fine without knitr. If knitr is available (it is part
> of
> > > the most common R distribution), the PR will enable the knitr
> > interpreter.
> > >
> > > * This is exactly how this issue is addressed through the Apache
> > > ecosystem.
> > > The tl;dr is this: When Apache code is written to talk to libraries
> that
> > > may or may not be present on the user’s system, or where it talks to an
> > API
> > > but is agnostic about implementation, that is not “linking” in a way
> that
> > > implicate the anti-linking provision of the GPL.
> > >
> > > Otherwise, no Apache code could ever call a shell script! Let alone run
> > > on Linux, or talk to R.
> > >
> > >
> > > I'm not a legal expert. So following could be wrong.
> > >
> > > In my interpretation, KnitRInterpreter is not an optional feature while
> > it
> > > is always enabled when compiling Zeppelin and always enabled when
> running
> > > Zeppelin. And it requires dynamically linked GPL library on runtime.
> (yes
> > > it will fail when no KnitR is installed on the system)
> > >
> > > And of course, no Apache code can ever call a shell script, on the
> > purpose
> > > of dynamic linking with GPL library.
> > >
> > > I was guessing SparkR can be still in Apache License even if it is
> > depends
> > > on R. Because of GPL licensed compiler generated output is not GPL
> > license.
> > > and R is sort of compiler.
> > >
> > > If you can get answer from Spark community how SparkR get managed to
> stay
> > > in Apache License while R is GPL, the answer might help.
> > >
> > >
> > > 3. Need more time to review.
> > > Has any reviewer has downloaded the PR or run the demo notebook? (Which
> > > is there for the benefit of reviewers, and isn’t intended to go in
> final
> > > distribution.)
> > >
> > > How many +1 comments from users would you like to see?
> > >
> > > How much time do you believe is required?
> > >
> > >
> > > It all depends on when CI is going to pass, when license problem is
> going
> > > to be cleared, and when a committer willing to review and responsible
> to
> > > commit your contribution.
> > >
> > >
> > > 1. Large code base change
> > > Large code base changes require coordination and cooperation. This PR
> > > necessarily implicates the build scripts, testing code, the
> > > SparkInterpreter, etc.
> > >
> > > I have been seeking to coordinate since August.
> > >
> > > I continue to stand ready to do so.
> > >
> > > -Amos
> > >
> > >
> > > If i give you one suggestion, Zeppelin committers sometimes ask rebase
> > the
> > > contribution branch for some reason. It is not the really the best
> > > practice, but still okay while most contributions are not including
> large
> > > code base changes.
> > >
> > > However, your one, has more than 4000 lines of code change. If you
> rebase
> > > then review should be started from the beginning, again. So you might
> > want
> > > to minimize the rebase your branch.
> > >
> > > Thanks,
> > > moon
> > >
> > >
> > > From: moon soo Lee <mo...@apache.org>
> > > Reply: dev@zeppelin.incubator.apache.org <
> > > dev@zeppelin.incubator.apache.org>
> > > Date: December 1, 2015 at 1:34:19 AM
> > > To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org
> > >
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull
> > > request: R Interpreter for Zeppelin
> > >
> > > Hi Cos,
> > >
> > > Thanks for opening a discussion.
> > > My answer about 'Why this PR is open for three months' is
> > >
> > > 1. This PR does not passes CI
> > > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > > 3. Need more time to review.
> > >
> > > It's my personal answer, other committers may have different opinion.
> > >
> > >
> > > I think the question should be generalized. Because this PR is not the
> > only
> > > PR that is in impasse. There're more. For example
> > >
> > > Here's some examples that PR is not been merged.
> > > https://github.com/apache/incubator-zeppelin/pull/53,
> > > https://github.com/apache/incubator-zeppelin/pull/60
> > > and so on.
> > >
> > > I can categorize the cases, based on experience of involving Zeppelin
> > > community from the beginning.
> > >
> > > 1. Large code base change
> > >
> > > When contribution has large code base changes, it tend to take more
> time
> > to
> > > review and merged. Normally, most contributions merged in 1day~1 week.
> > > But some contribution has large code base changes take 2~4 weeks. Few
> > > contribution that has very large code base change take months.
> > >
> > > 2. Communication lost
> > >
> > > Sometimes, committer is not responding, or contributor is not
> responding.
> > >
> > > 3. Opinion diverges
> > >
> > > For some changes, included in contribution. When people have different
> > > opinion and it continue to diverges, those PRs are not been merged.
> > >
> > >
> > > I think having a guide such as ping other committer when a committer is
> > not
> > > responding, and divide contribution into small peaces if possible,
> would
> > > help most of the cases.
> > > Of course committer have to pay attention more to the contribution and
> > > helping people. That's the first one.
> > >
> > > Thanks,
> > > moon
> > >
> > > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org>
> > wrote:
> > >
> > > > To make sure we're on the same page, here are two threads that I
> found
> > > > related
> > > > to this topic.
> > > >
> > > > Thread 1:
> > > > Subject: R?
> > > > Started on: July 1, 2015
> > > >
> > > > Thread 2:
> > > > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > > > Zeppelin
> > > > Started on: August 13, 2015
> > > >
> > > > If you want to fetch these from our archive send emails to
> > > > dev-thread.1735@zeppelin.incubator.apache.org
> > > > dev-thread.3573@zeppelin.incubator.apache.org
> > > >
> > > > Cos
> > > >
> > > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > > > Guys,
> > > > >
> > > > > While catching up on my emails from the last a couple of weeks,
> this
> > > > thread
> > > > > caught my attention. I am not normally paying much attention to the
> > > code
> > > > > reviews traffic from GH, but this one is pretty different as it
> spans
> > > > three
> > > > > months and counting.
> > > > >
> > > > > First, here are my five cents:
> > > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > > > contributed to
> > > > > an ASF project this file should simply contain ASL2 text, like in
> [1]
> > > > > - r/pom.xml perhaps shouldn't contain a separate <developers>
> > section,
> > > > but
> > > > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > > > > maintain this in the source code, but we have the list of all the
> > > > > committers on the project's site [2] Every new committer's first
> > > > commit is
> > > > > to update the team page ;)
> > > > > - comments like in
> > > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > > >
> > > > > +/**
> > > > > + * Created by aelberg on 7/28/15.
> > > > > + */
> > > > >
> > > > > is better to be removed. It has been already discussed here [3].
> And
> > > > the
> > > > > initial discussion on the topic [4] was linked as well
> > > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> > > > R-specific
> > > > > stuff - I have no idea about R, honestly, but
> > > > >
> > > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > > ...
> > > > > +Author: David B. Dahl
> > > > >
> > > > > shouldn't be here, IMO. Normally, if some additional licenses are
> > > > used,
> > > > > they have to be listed in the top-level NOTICE file (already
> there).
> > > > >
> > > > > - I am not going to offer any comments on the technical merits of
> the
> > > > patch,
> > > > > as I haven't tried it personally. However, I don't see any serious
> > > > > technical objections to the functionality in question.
> > > > >
> > > > > So, the question is - why the PR is open for three months? I hasn't
> > > been
> > > > able
> > > > > to get a clear answer. What I found out though is pretty
> unsettling,
> > > > really.
> > > > > The communication on the topic is almost non-existent, except for
> > this
> > > > sparse
> > > > > and bitter thread in the GH.
> > > > >
> > > > > I would love to hear from the committers what's stopping the
> > acceptance
> > > > of the
> > > > > code, besides of the minor issues I've mentioned earlier? What are
> > the
> > > > reasons for it?
> > > > > Is there anything wrong with it?
> > > > > One of the responsibilities of the committers is to make sure the
> > > > > contributions are reviewed; new contributors are guided and do
> > > > understand how
> > > > > the project ticks. The easy feedback flow attracts new people,
> > allowing
> > > > the
> > > > > community to grow and thrive.
> > > > >
> > > > > I am asking _explicitely_ not to re-start the bickering I have
> > already
> > > > > seen. At this point I am interested in the purely technical side of
> > > this.
> > > > >
> > > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > > > [2] http://bigtop.apache.org/team-list.html
> > > > > [3]
> > > >
> > >
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > > [4] http://s.apache.org/iZl
> > > > >
> > > > > With regards,
> > > > > Cos
> > > > >
> > > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > > Github user elbamos commented on the pull request:
> > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > > >
> > > > > > The current push should resolve some issues with changes in the
> > > > > > Spark-Zeppelin interface that had created issues for users, as
> > > > well as
> > > > > > support for 1.5.1.
> > > > > >
> > > > > > Knitr support is improved, and the reason for a separate knitr
> > > > interpreter may be clearer now.
> > > > > >
> > > > > > The amount of code borrowed from rscala is reduced.
> > > > > >
> > > > > > I did not address issues with @author tags, or files under the R/
> > > > > > folder. The reason is, to be blunt, I don't understand what the
> > > > precise
> > > > > > concerns actually are.
> > > > > >
> > > > > > Please note that I changed .travis.yml to only use spark 1.4 and
> > > > later.
> > > > > > I'm sure there is a better way to do it, but I'm just not in a
> > > > position
> > > > > > to try to figure out the entire Zeppelin build process.
> > > > > >
> > > > > > Integrating this with the rest of zeppelin is going to take some
> > > > work
> > > > > > regarding pom's, travis, and so forth. I can do a lot of that,
> > > > but I'm
> > > > > > going to need to discuss it with the people who have been
> "owning"
> > > > those
> > > > > > files. There are just too many moving pieces here.
> > > > > >
> > > > > > Because of the size of this (which is, unfortunately, necessary),
> > > > > > posting here is probably not the most efficient way. That is also
> > > > true
> > > > > > because certain people chose to use this PR as a forum to air
> other
> > > > > > issues. Therefore, it would be better if reviewers e-mail me
> > > > directly.
> > > >
> > > >
> > > >
> > >
> >
>

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Moon let me clarify:

Interpreted code doesn’t “link.”  The wiki article actually explains it pretty well — https://en.wikipedia.org/wiki/GPL_linking_exception   “Linking” against a library means compiling its headers into a binary, the way a C compiler works.  The 2008 e-mail Moon distributed, called this the “interpreter exception.”  

As for whether GPL’d code is a “mandatory dependency,” if knitr is missing the PR will compile, run and test just fine.  The user is never prompted to download it.  The only effect is, if the user types “knitr” and knitr isn’t there, we display an InterpreterError that knitr isn’t there.   

KnitRInterpreter is not optionally required. so it does not matter KnitR is 
optionally required or not. 
Aren’t all interpreters optional?  You can turn them on and off in the config files. 

Do you mean that the KnitRInterpreter class gets compiled to bytecode even if knitr is missing?  So what?  That isn't a mandatory dependency or a link.

From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 2, 2015 at 3:18:00 AM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

Let me summarize license concern about KnitRInterpreter.  


Amos's interpretation.  

KnitR is optionally required by KnitRInterpreter.  
R dependency in SparkR has no problem. So KnitR should be the same.  


Moon's interpretation.  

KnitRInterpreter is not optionally required. so it does not matter KnitR is  
optionally required or not.  
R dependency in SparkR is exception of GPL. KnitR is not applied that  
exception.  


Amos, could you confirm my understanding to your interpretation is correct?  
If it is not could you clarify it?  

Thanks,  
moon  

On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <am...@gmail.com>  
wrote:  

> Just to put the final nail in this, I looked it up.  
>  
> I see no evidence of any “compiler exception.”  
>  
> There is an exception in the license for the runtime libraries that are  
> bundled with GCC. See:  
> http://www.gnu.org/licenses/gcc-exception-3.1-faq.html  
>  
> But no “compiler exception.”  
>  
> In fact, I believe this is part of the reason why LLVM was created.  
>  
> From: moon soo Lee <mo...@apache.org>  
> Reply: moon soo Lee <mo...@apache.org>  
> Date: December 1, 2015 at 8:16:36 PM  
> To: Amos B. Elberg <am...@gmail.com>,  
> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> request: R Interpreter for Zeppelin  
>  
> Is knitR is commonly considered as a interpreter/compiler? or is it  
> considered as a library routine?  
>  
> Thanks,  
> moon  
>  
> On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com>  
> wrote:  
> Moon - you give this as an explanation of the licensing issue:  
> https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
>  
> According to that, there is an exception in the GPL for interpreter  
> languages. As long as you don’t distribute the code, its fine to talk to  
> an interpreted language.  
>  
> Well, if that’s the case, then the PR plainly does not have a license  
> issue. It doesn’t distribute any GPL’d R code.  
>  
> I’m not sure what’s confusing about this. It seems completely  
> straightforward.  
>  
> Regarding this:  
>  
>  
> --  
> Amos Elberg  
> Sent with Airmail  
>  
> From: moon soo Lee <mo...@apache.org>  
> Reply: dev@zeppelin.incubator.apache.org <  
> dev@zeppelin.incubator.apache.org>  
> Date: December 1, 2015 at 6:48:47 PM  
>  
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> request: R Interpreter for Zeppelin  
>  
> On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com>  
> wrote:  
>  
> > I am going to try to minimize my reaction to Moon’s e-mail.  
> >  
> > The tl;dr is this:  
> >  
> > The reason we are having this discussion now is that active users of the  
> > PR — which now has its own user base — went public to complain about  
> this.  
>  
>  
> > The PR has been tested by an active user base for more than three months.  
> > No-one has been able to identify any specific actual licensing problem,  
> and  
> > the PR was prepared based on an extensive, careful review of the relevant  
> > licensing issues and after contacting the relevant people.  
> >  
> >  
>  
> I admire every software that used by user and helping people. That includes  
> your work. But that's not the topic we're in discussion. Active user does  
> not mean your contribution can ignore the review.  
>  
>  
>  
> > It is not an explanation for someone who has been ignoring my “how can I  
> > move this forward…” emails for three months to point the finger and say I  
> > didn’t contact the right person or file the right report.  
> >  
> >  
> This is also not the topic in this discussion.  
>  
>  
> > The burden for providing an explanation for the inaction is on the PMCC  
> at  
> > this point.  
>  
> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin  
> > core, why do you think other PRs are passing CI?  
> > They’re not! I often see comments on PRs to just ignore that CI is  
> > failing.  
> >  
> > One of the most common reasons this PR fails CI, is CI times-out  
> > downloading Spark to install. How could that possibly be caused by the  
> PR?  
> >  
> > It looks to me like the only PRs with changes to the relevant parts of  
> the  
> > code — the SparkInterpreter — are being made by the person who wrote the  
> > testing suite.  
> >  
> > So, that would explain why some other PRs pass CI: Neither the  
> > SparkInterpreter nor the testing suite are stable or robust, but since  
> the  
> > PRs are coming from the person who wrote both…  
> >  
> > And let's say Zeppelin core has problem and that makes your PR fails on  
> CI  
> > test. That's possible. But it still does not mean we can merge it with CI  
> > failing.  
> >  
> > It means you should be working with me to figure out why the CI is  
> failing.  
> >  
> > This PR has been tested by an active user base for the past three months.  
> > If CI is continuing to fail, and dozens of hours of effort have not  
> > resolved the CI issues, then it is time to start considering whether the  
> > testing suite is part of the problem.  
> >  
> > The level of defensiveness about the CI and SparkInterpreter is not  
> > helping to resolve these issues.  
> >  
> > If you think it's problem on Zeppelin core, then file an issue that  
> > reproduce the problem on Zeppelin core, that might be more efficient than  
> > keep trying yourself.  
> > I contacted you numerous times about such issues...  
> >  
>  
>  
> I remember i commented your issue about CI. but you just keep repeated it's  
> not your problem but Zeppelin core problem.  
>  
> Then please file an issue about the problem you found in Zeppelin Core.  
> Then everyone will get into the problem.  
>  
>  
>  
> >  
> > In my interpretation, KnitRInterpreter is not an optional feature while  
> it  
> > is always enabled when compiling Zeppelin and always enabled when running  
> > Zeppelin. And it requires dynamically linked GPL library on runtime. (yes  
> > it will fail when no KnitR is installed on the system)  
> >  
> > Its not always enabled.  
> > It is not dynamically linked at runtime.  
> > It will not fail when knitr is missing. If knitr is not present, the repl  
> > interpreter starts and a note is written to to the log that the knitr  
> > interpreter isn’t available because knitr is not present.  
> >  
> > no Apache code can ever call a shell script, on the purpose of dynamic  
> > linking with GPL library.  
> > You misunderstand.  
> >  
> > The *shell* is GPL'd.  
> >  
> > Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends on  
> a  
> > shell script to launch?  
> >  
> Obviously not.  
> >  
> > The interaction with R in the PR is the same.  
> >  
> >  
>  
> Again, bash is one of exceptions of GPL, like other GPL licensed  
> compiler/interpreter.  
>  
> Check here why Bash and R is okay with Apache License.  
> https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
>  
> I'm not sure we can apply the same exception for 'using' KnitR.  
>  
> My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter' you  
> wrote is not an optional feature. Which is clearly not optionally enabled  
> code and feature. And that depends on KnitR library which is GPL.  
>  
>  
>  
> > I was guessing SparkR can be still in Apache License even if it is  
> depends  
> > on R. Because of GPL licensed compiler generated output is not GPL  
> license.  
> > and R is sort of compiler. If you can get answer from Spark community how  
> > SparkR get managed to stay in Apache License while R is GPL, the answer  
> > might help.  
> > The description of SparkR is not accurate in any respect. (Do you think  
> > SparkR is not talking to GPL-licensed libraries?)  
> >  
> > I don’t see that any genuine issue is being raised here.  
> >  
> > If there is an issue, the burden is on you to identify it.  
> >  
> > If i give you one suggestion, Zeppelin committers sometimes ask rebase  
> the  
> > contribution branch for some reason. It is not the really the best  
> > practice, but still okay while most contributions are not including large  
> > code base changes  
> > However, your one, has more than 4000 lines of code change. If you rebase  
> > then review should be started from the beginning, again. So you might  
> want  
> > to minimize the rebase your branch.  
> >  
> > Are you actually complaining that the problem is that I rebased the code  
> > during the three-month period when no-one looked at it and Zeppelin went  
> > through a release?  
> >  
> > I cannot take it seriously when you say things like this.  
> >  
> > Having to “start from the beginning” cannot be a problem if you never  
> > actually started the first time...  
> >  
> >  
>  
> You wanted coordination and cooperation. So i gave you suggestion that  
> helping review process. For example, your code has been rebased since my  
> comment and jongyoul's comment. that means committers will need to look  
> from the beginning. That'll require more time. And maybe, i guess that's  
> not what you want. But If you don't care, feel free to rebase.  
>  
> Thanks,  
> moon  
>  
>  
>  
> >  
> > From: moon soo Lee <mo...@apache.org>  
> > Reply: moon soo Lee <mo...@apache.org>  
> > Date: December 1, 2015 at 4:42:06 AM  
> > To: Amos B. Elberg <am...@gmail.com>,  
> > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> > request: R Interpreter for Zeppelin  
> >  
> > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>  
> > wrote:  
> > Thank you, Cos.  
> >  
> > I’d like to briefly address the issues raised by Moon:  
> >  
> > 1. This PR does not passes CI  
> > The CI fails on core Zeppelin, *not* code in this PR.  
> >  
> > I’ve been seeking assistance on this since August.  
> >  
> > The most common reason is that SparkInterpreter is unable to launch Spark  
> > and open a Spark Backend. This is necessary to test the PR.  
> >  
> > 60+ hours, has been spent adapting and re-basing when the  
> SparkInterpreter  
> > architecture changed and broke the PR’s *tests.*  
> >  
> >  
> > I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin  
> > core, why do you think other PRs are passing CI?  
> >  
> > And let's say Zeppelin core has problem and that makes your PR fails on  
> CI  
> > test. That's possible. But it still does not mean we can merge it with CI  
> > failing.  
> >  
> > If you think it's problem on Zeppelin core, then file an issue that  
> > reproduce the problem on Zeppelin core, that might be more efficient than  
> > keep trying yourself.  
> >  
> >  
> > 2. Not 100% sure this PR has no license issue. (about KniteR)  
> > What license problem *specifically* do you believe may exist?  
> >  
> > In preparing the PR, I:  
> >  
> > * Reviewed the Apache policy in detail.  
> >  
> > * Contacted the FSF to ask their interpretation of the “linking”  
> > provisions of the GPL license.  
> >  
> > * Reviewed how other Apache software deals with this issue (e.g., Spark  
> > talks to R, which is GPL'd).  
> >  
> > * No necessary *dependencies* of the PR have license conflicts. In  
> > several cases, I contacted package authors who agreed to re-issue their  
> > packages under Apache-compatible licenses. (Usually I had to do a bit of  
> > coding in exchange…)  
> >  
> > * Where the license had to stay GPL, the packages are *not necessary* and  
> > *not dependencies.* If the optional packages are present, they will be  
> > used to enable additional functionality. Knitr is an example. The PR will  
> > compile and run fine without knitr. If knitr is available (it is part of  
> > the most common R distribution), the PR will enable the knitr  
> interpreter.  
> >  
> > * This is exactly how this issue is addressed through the Apache  
> > ecosystem.  
> > The tl;dr is this: When Apache code is written to talk to libraries that  
> > may or may not be present on the user’s system, or where it talks to an  
> API  
> > but is agnostic about implementation, that is not “linking” in a way that  
> > implicate the anti-linking provision of the GPL.  
> >  
> > Otherwise, no Apache code could ever call a shell script! Let alone run  
> > on Linux, or talk to R.  
> >  
> >  
> > I'm not a legal expert. So following could be wrong.  
> >  
> > In my interpretation, KnitRInterpreter is not an optional feature while  
> it  
> > is always enabled when compiling Zeppelin and always enabled when running  
> > Zeppelin. And it requires dynamically linked GPL library on runtime. (yes  
> > it will fail when no KnitR is installed on the system)  
> >  
> > And of course, no Apache code can ever call a shell script, on the  
> purpose  
> > of dynamic linking with GPL library.  
> >  
> > I was guessing SparkR can be still in Apache License even if it is  
> depends  
> > on R. Because of GPL licensed compiler generated output is not GPL  
> license.  
> > and R is sort of compiler.  
> >  
> > If you can get answer from Spark community how SparkR get managed to stay  
> > in Apache License while R is GPL, the answer might help.  
> >  
> >  
> > 3. Need more time to review.  
> > Has any reviewer has downloaded the PR or run the demo notebook? (Which  
> > is there for the benefit of reviewers, and isn’t intended to go in final  
> > distribution.)  
> >  
> > How many +1 comments from users would you like to see?  
> >  
> > How much time do you believe is required?  
> >  
> >  
> > It all depends on when CI is going to pass, when license problem is going  
> > to be cleared, and when a committer willing to review and responsible to  
> > commit your contribution.  
> >  
> >  
> > 1. Large code base change  
> > Large code base changes require coordination and cooperation. This PR  
> > necessarily implicates the build scripts, testing code, the  
> > SparkInterpreter, etc.  
> >  
> > I have been seeking to coordinate since August.  
> >  
> > I continue to stand ready to do so.  
> >  
> > -Amos  
> >  
> >  
> > If i give you one suggestion, Zeppelin committers sometimes ask rebase  
> the  
> > contribution branch for some reason. It is not the really the best  
> > practice, but still okay while most contributions are not including large  
> > code base changes.  
> >  
> > However, your one, has more than 4000 lines of code change. If you rebase  
> > then review should be started from the beginning, again. So you might  
> want  
> > to minimize the rebase your branch.  
> >  
> > Thanks,  
> > moon  
> >  
> >  
> > From: moon soo Lee <mo...@apache.org>  
> > Reply: dev@zeppelin.incubator.apache.org <  
> > dev@zeppelin.incubator.apache.org>  
> > Date: December 1, 2015 at 1:34:19 AM  
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org  
> >  
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> > request: R Interpreter for Zeppelin  
> >  
> > Hi Cos,  
> >  
> > Thanks for opening a discussion.  
> > My answer about 'Why this PR is open for three months' is  
> >  
> > 1. This PR does not passes CI  
> > 2. Not 100% sure this PR has no license issue. (about KniteR)  
> > 3. Need more time to review.  
> >  
> > It's my personal answer, other committers may have different opinion.  
> >  
> >  
> > I think the question should be generalized. Because this PR is not the  
> only  
> > PR that is in impasse. There're more. For example  
> >  
> > Here's some examples that PR is not been merged.  
> > https://github.com/apache/incubator-zeppelin/pull/53,  
> > https://github.com/apache/incubator-zeppelin/pull/60  
> > and so on.  
> >  
> > I can categorize the cases, based on experience of involving Zeppelin  
> > community from the beginning.  
> >  
> > 1. Large code base change  
> >  
> > When contribution has large code base changes, it tend to take more time  
> to  
> > review and merged. Normally, most contributions merged in 1day~1 week.  
> > But some contribution has large code base changes take 2~4 weeks. Few  
> > contribution that has very large code base change take months.  
> >  
> > 2. Communication lost  
> >  
> > Sometimes, committer is not responding, or contributor is not responding.  
> >  
> > 3. Opinion diverges  
> >  
> > For some changes, included in contribution. When people have different  
> > opinion and it continue to diverges, those PRs are not been merged.  
> >  
> >  
> > I think having a guide such as ping other committer when a committer is  
> not  
> > responding, and divide contribution into small peaces if possible, would  
> > help most of the cases.  
> > Of course committer have to pay attention more to the contribution and  
> > helping people. That's the first one.  
> >  
> > Thanks,  
> > moon  
> >  
> > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org>  
> wrote:  
> >  
> > > To make sure we're on the same page, here are two threads that I found  
> > > related  
> > > to this topic.  
> > >  
> > > Thread 1:  
> > > Subject: R?  
> > > Started on: July 1, 2015  
> > >  
> > > Thread 2:  
> > > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for  
> > > Zeppelin  
> > > Started on: August 13, 2015  
> > >  
> > > If you want to fetch these from our archive send emails to  
> > > dev-thread.1735@zeppelin.incubator.apache.org  
> > > dev-thread.3573@zeppelin.incubator.apache.org  
> > >  
> > > Cos  
> > >  
> > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:  
> > > > Guys,  
> > > >  
> > > > While catching up on my emails from the last a couple of weeks, this  
> > > thread  
> > > > caught my attention. I am not normally paying much attention to the  
> > code  
> > > > reviews traffic from GH, but this one is pretty different as it spans  
> > > three  
> > > > months and counting.  
> > > >  
> > > > First, here are my five cents:  
> > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be  
> > > contributed to  
> > > > an ASF project this file should simply contain ASL2 text, like in [1]  
> > > > - r/pom.xml perhaps shouldn't contain a separate <developers>  
> section,  
> > > but  
> > > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't  
> > > > maintain this in the source code, but we have the list of all the  
> > > > committers on the project's site [2] Every new committer's first  
> > > commit is  
> > > > to update the team page ;)  
> > > > - comments like in  
> > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java  
> > > >  
> > > > +/**  
> > > > + * Created by aelberg on 7/28/15.  
> > > > + */  
> > > >  
> > > > is better to be removed. It has been already discussed here [3]. And  
> > > the  
> > > > initial discussion on the topic [4] was linked as well  
> > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is  
> > > R-specific  
> > > > stuff - I have no idea about R, honestly, but  
> > > >  
> > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE  
> > > > ...  
> > > > +Author: David B. Dahl  
> > > >  
> > > > shouldn't be here, IMO. Normally, if some additional licenses are  
> > > used,  
> > > > they have to be listed in the top-level NOTICE file (already there).  
> > > >  
> > > > - I am not going to offer any comments on the technical merits of the  
> > > patch,  
> > > > as I haven't tried it personally. However, I don't see any serious  
> > > > technical objections to the functionality in question.  
> > > >  
> > > > So, the question is - why the PR is open for three months? I hasn't  
> > been  
> > > able  
> > > > to get a clear answer. What I found out though is pretty unsettling,  
> > > really.  
> > > > The communication on the topic is almost non-existent, except for  
> this  
> > > sparse  
> > > > and bitter thread in the GH.  
> > > >  
> > > > I would love to hear from the committers what's stopping the  
> acceptance  
> > > of the  
> > > > code, besides of the minor issues I've mentioned earlier? What are  
> the  
> > > reasons for it?  
> > > > Is there anything wrong with it?  
> > > > One of the responsibilities of the committers is to make sure the  
> > > > contributions are reviewed; new contributors are guided and do  
> > > understand how  
> > > > the project ticks. The easy feedback flow attracts new people,  
> allowing  
> > > the  
> > > > community to grow and thrive.  
> > > >  
> > > > I am asking _explicitely_ not to re-start the bickering I have  
> already  
> > > > seen. At this point I am interested in the purely technical side of  
> > this.  
> > > >  
> > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE  
> > > > [2] http://bigtop.apache.org/team-list.html  
> > > > [3]  
> > >  
> >  
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html  
> > > > [4] http://s.apache.org/iZl  
> > > >  
> > > > With regards,  
> > > > Cos  
> > > >  
> > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:  
> > > > > Github user elbamos commented on the pull request:  
> > > > >  
> > > > >  
> > >  
> >  
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411  
> > > > >  
> > > > > The current push should resolve some issues with changes in the  
> > > > > Spark-Zeppelin interface that had created issues for users, as  
> > > well as  
> > > > > support for 1.5.1.  
> > > > >  
> > > > > Knitr support is improved, and the reason for a separate knitr  
> > > interpreter may be clearer now.  
> > > > >  
> > > > > The amount of code borrowed from rscala is reduced.  
> > > > >  
> > > > > I did not address issues with @author tags, or files under the R/  
> > > > > folder. The reason is, to be blunt, I don't understand what the  
> > > precise  
> > > > > concerns actually are.  
> > > > >  
> > > > > Please note that I changed .travis.yml to only use spark 1.4 and  
> > > later.  
> > > > > I'm sure there is a better way to do it, but I'm just not in a  
> > > position  
> > > > > to try to figure out the entire Zeppelin build process.  
> > > > >  
> > > > > Integrating this with the rest of zeppelin is going to take some  
> > > work  
> > > > > regarding pom's, travis, and so forth. I can do a lot of that,  
> > > but I'm  
> > > > > going to need to discuss it with the people who have been "owning"  
> > > those  
> > > > > files. There are just too many moving pieces here.  
> > > > >  
> > > > > Because of the size of this (which is, unfortunately, necessary),  
> > > > > posting here is probably not the most efficient way. That is also  
> > > true  
> > > > > because certain people chose to use this PR as a forum to air other  
> > > > > issues. Therefore, it would be better if reviewers e-mail me  
> > > directly.  
> > >  
> > >  
> > >  
> >  
>  

Re: License of KnitRInterpreter Was: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Let me summarize license concern about KnitRInterpreter.


Amos's interpretation.

KnitR is optionally required by KnitRInterpreter.
R dependency in SparkR has no problem. So KnitR should be the same.


Moon's interpretation.

KnitRInterpreter is not optionally required. so it does not matter KnitR is
optionally required or not.
R dependency in SparkR is exception of GPL. KnitR is not applied that
exception.


Amos, could you confirm my understanding to your interpretation is correct?
If it is not could you clarify it?

Thanks,
moon

On Wed, Dec 2, 2015 at 10:34 AM Amos B. Elberg <am...@gmail.com>
wrote:

> Just to put the final nail in this, I looked it up.
>
> I see no evidence of any “compiler exception.”
>
> There is an exception in the license for the runtime libraries that are
> bundled with GCC.  See:
> http://www.gnu.org/licenses/gcc-exception-3.1-faq.html
>
> But no “compiler exception.”
>
> In fact, I believe this is part of the reason why LLVM was created.
>
> From: moon soo Lee <mo...@apache.org>
> Reply: moon soo Lee <mo...@apache.org>
> Date: December 1, 2015 at 8:16:36 PM
> To: Amos B. Elberg <am...@gmail.com>,
> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> Is knitR is commonly considered as a interpreter/compiler? or is it
> considered as a library routine?
>
> Thanks,
> moon
>
> On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com>
> wrote:
> Moon - you give this as an explanation of the licensing issue:
> https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
>
> According to that, there is an exception in the GPL for interpreter
> languages.  As long as you don’t distribute the code, its fine to talk to
> an interpreted language.
>
> Well, if that’s the case, then the PR plainly does not have a license
> issue.  It doesn’t distribute any GPL’d R code.
>
> I’m not sure what’s confusing about this.  It seems completely
> straightforward.
>
> Regarding this:
>
>
> --
> Amos Elberg
> Sent with Airmail
>
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> Date: December 1, 2015 at 6:48:47 PM
>
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com>
> wrote:
>
> > I am going to try to minimize my reaction to Moon’s e-mail.
> >
> > The tl;dr is this:
> >
> > The reason we are having this discussion now is that active users of the
> > PR — which now has its own user base — went public to complain about
> this.
>
>
> > The PR has been tested by an active user base for more than three months.
> > No-one has been able to identify any specific actual licensing problem,
> and
> > the PR was prepared based on an extensive, careful review of the relevant
> > licensing issues and after contacting the relevant people.
> >
> >
>
> I admire every software that used by user and helping people. That includes
> your work. But that's not the topic we're in discussion. Active user does
> not mean your contribution can ignore the review.
>
>
>
> > It is not an explanation for someone who has been ignoring my “how can I
> > move this forward…” emails for three months to point the finger and say I
> > didn’t contact the right person or file the right report.
> >
> >
> This is also not the topic in this discussion.
>
>
> > The burden for providing an explanation for the inaction is on the PMCC
> at
> > this point.
>
> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> > core, why do you think other PRs are passing CI?
> > They’re not! I often see comments on PRs to just ignore that CI is
> > failing.
> >
> > One of the most common reasons this PR fails CI, is CI times-out
> > downloading Spark to install. How could that possibly be caused by the
> PR?
> >
> > It looks to me like the only PRs with changes to the relevant parts of
> the
> > code — the SparkInterpreter — are being made by the person who wrote the
> > testing suite.
> >
> > So, that would explain why some other PRs pass CI: Neither the
> > SparkInterpreter nor the testing suite are stable or robust, but since
> the
> > PRs are coming from the person who wrote both…
> >
> > And let's say Zeppelin core has problem and that makes your PR fails on
> CI
> > test. That's possible. But it still does not mean we can merge it with CI
> > failing.
> >
> > It means you should be working with me to figure out why the CI is
> failing.
> >
> > This PR has been tested by an active user base for the past three months.
> > If CI is continuing to fail, and dozens of hours of effort have not
> > resolved the CI issues, then it is time to start considering whether the
> > testing suite is part of the problem.
> >
> > The level of defensiveness about the CI and SparkInterpreter is not
> > helping to resolve these issues.
> >
> > If you think it's problem on Zeppelin core, then file an issue that
> > reproduce the problem on Zeppelin core, that might be more efficient than
> > keep trying yourself.
> > I contacted you numerous times about such issues...
> >
>
>
> I remember i commented your issue about CI. but you just keep repeated it's
> not your problem but Zeppelin core problem.
>
> Then please file an issue about the problem you found in Zeppelin Core.
> Then everyone will get into the problem.
>
>
>
> >
> > In my interpretation, KnitRInterpreter is not an optional feature while
> it
> > is always enabled when compiling Zeppelin and always enabled when running
> > Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
> > it will fail when no KnitR is installed on the system)
> >
> > Its not always enabled.
> > It is not dynamically linked at runtime.
> > It will not fail when knitr is missing. If knitr is not present, the repl
> > interpreter starts and a note is written to to the log that the knitr
> > interpreter isn’t available because knitr is not present.
> >
> > no Apache code can ever call a shell script, on the purpose of dynamic
> > linking with GPL library.
> > You misunderstand.
> >
> > The *shell* is GPL'd.
> >
> > Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends on
> a
> > shell script to launch?
> >
> Obviously not.
> >
> > The interaction with R in the PR is the same.
> >
> >
>
> Again, bash is one of exceptions of GPL, like other GPL licensed
> compiler/interpreter.
>
> Check here why Bash and R is okay with Apache License.
> https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
>
> I'm not sure we can apply the same exception for 'using' KnitR.
>
> My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter' you
> wrote is not an optional feature. Which is clearly not optionally enabled
> code and feature. And that depends on KnitR library which is GPL.
>
>
>
> > I was guessing SparkR can be still in Apache License even if it is
> depends
> > on R. Because of GPL licensed compiler generated output is not GPL
> license.
> > and R is sort of compiler. If you can get answer from Spark community how
> > SparkR get managed to stay in Apache License while R is GPL, the answer
> > might help.
> > The description of SparkR is not accurate in any respect. (Do you think
> > SparkR is not talking to GPL-licensed libraries?)
> >
> > I don’t see that any genuine issue is being raised here.
> >
> > If there is an issue, the burden is on you to identify it.
> >
> > If i give you one suggestion, Zeppelin committers sometimes ask rebase
> the
> > contribution branch for some reason. It is not the really the best
> > practice, but still okay while most contributions are not including large
> > code base changes
> > However, your one, has more than 4000 lines of code change. If you rebase
> > then review should be started from the beginning, again. So you might
> want
> > to minimize the rebase your branch.
> >
> > Are you actually complaining that the problem is that I rebased the code
> > during the three-month period when no-one looked at it and Zeppelin went
> > through a release?
> >
> > I cannot take it seriously when you say things like this.
> >
> > Having to “start from the beginning” cannot be a problem if you never
> > actually started the first time...
> >
> >
>
> You wanted coordination and cooperation. So i gave you suggestion that
> helping review process. For example, your code has been rebased since my
> comment and jongyoul's comment. that means committers will need to look
> from the beginning. That'll require more time. And maybe, i guess that's
> not what you want. But If you don't care, feel free to rebase.
>
> Thanks,
> moon
>
>
>
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: moon soo Lee <mo...@apache.org>
> > Date: December 1, 2015 at 4:42:06 AM
> > To: Amos B. Elberg <am...@gmail.com>,
> > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> > request: R Interpreter for Zeppelin
> >
> > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
> > wrote:
> > Thank you, Cos.
> >
> > I’d like to briefly address the issues raised by Moon:
> >
> > 1. This PR does not passes CI
> > The CI fails on core Zeppelin, *not* code in this PR.
> >
> > I’ve been seeking assistance on this since August.
> >
> > The most common reason is that SparkInterpreter is unable to launch Spark
> > and open a Spark Backend. This is necessary to test the PR.
> >
> > 60+ hours, has been spent adapting and re-basing when the
> SparkInterpreter
> > architecture changed and broke the PR’s *tests.*
> >
> >
> > I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> > core, why do you think other PRs are passing CI?
> >
> > And let's say Zeppelin core has problem and that makes your PR fails on
> CI
> > test. That's possible. But it still does not mean we can merge it with CI
> > failing.
> >
> > If you think it's problem on Zeppelin core, then file an issue that
> > reproduce the problem on Zeppelin core, that might be more efficient than
> > keep trying yourself.
> >
> >
> > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > What license problem *specifically* do you believe may exist?
> >
> > In preparing the PR, I:
> >
> > * Reviewed the Apache policy in detail.
> >
> > * Contacted the FSF to ask their interpretation of the “linking”
> > provisions of the GPL license.
> >
> > * Reviewed how other Apache software deals with this issue (e.g., Spark
> > talks to R, which is GPL'd).
> >
> > * No necessary *dependencies* of the PR have license conflicts. In
> > several cases, I contacted package authors who agreed to re-issue their
> > packages under Apache-compatible licenses. (Usually I had to do a bit of
> > coding in exchange…)
> >
> > * Where the license had to stay GPL, the packages are *not necessary* and
> > *not dependencies.* If the optional packages are present, they will be
> > used to enable additional functionality. Knitr is an example. The PR will
> > compile and run fine without knitr. If knitr is available (it is part of
> > the most common R distribution), the PR will enable the knitr
> interpreter.
> >
> > * This is exactly how this issue is addressed through the Apache
> > ecosystem.
> > The tl;dr is this: When Apache code is written to talk to libraries that
> > may or may not be present on the user’s system, or where it talks to an
> API
> > but is agnostic about implementation, that is not “linking” in a way that
> > implicate the anti-linking provision of the GPL.
> >
> > Otherwise, no Apache code could ever call a shell script! Let alone run
> > on Linux, or talk to R.
> >
> >
> > I'm not a legal expert. So following could be wrong.
> >
> > In my interpretation, KnitRInterpreter is not an optional feature while
> it
> > is always enabled when compiling Zeppelin and always enabled when running
> > Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
> > it will fail when no KnitR is installed on the system)
> >
> > And of course, no Apache code can ever call a shell script, on the
> purpose
> > of dynamic linking with GPL library.
> >
> > I was guessing SparkR can be still in Apache License even if it is
> depends
> > on R. Because of GPL licensed compiler generated output is not GPL
> license.
> > and R is sort of compiler.
> >
> > If you can get answer from Spark community how SparkR get managed to stay
> > in Apache License while R is GPL, the answer might help.
> >
> >
> > 3. Need more time to review.
> > Has any reviewer has downloaded the PR or run the demo notebook? (Which
> > is there for the benefit of reviewers, and isn’t intended to go in final
> > distribution.)
> >
> > How many +1 comments from users would you like to see?
> >
> > How much time do you believe is required?
> >
> >
> > It all depends on when CI is going to pass, when license problem is going
> > to be cleared, and when a committer willing to review and responsible to
> > commit your contribution.
> >
> >
> > 1. Large code base change
> > Large code base changes require coordination and cooperation. This PR
> > necessarily implicates the build scripts, testing code, the
> > SparkInterpreter, etc.
> >
> > I have been seeking to coordinate since August.
> >
> > I continue to stand ready to do so.
> >
> > -Amos
> >
> >
> > If i give you one suggestion, Zeppelin committers sometimes ask rebase
> the
> > contribution branch for some reason. It is not the really the best
> > practice, but still okay while most contributions are not including large
> > code base changes.
> >
> > However, your one, has more than 4000 lines of code change. If you rebase
> > then review should be started from the beginning, again. So you might
> want
> > to minimize the rebase your branch.
> >
> > Thanks,
> > moon
> >
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > Date: December 1, 2015 at 1:34:19 AM
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> > request: R Interpreter for Zeppelin
> >
> > Hi Cos,
> >
> > Thanks for opening a discussion.
> > My answer about 'Why this PR is open for three months' is
> >
> > 1. This PR does not passes CI
> > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > 3. Need more time to review.
> >
> > It's my personal answer, other committers may have different opinion.
> >
> >
> > I think the question should be generalized. Because this PR is not the
> only
> > PR that is in impasse. There're more. For example
> >
> > Here's some examples that PR is not been merged.
> > https://github.com/apache/incubator-zeppelin/pull/53,
> > https://github.com/apache/incubator-zeppelin/pull/60
> > and so on.
> >
> > I can categorize the cases, based on experience of involving Zeppelin
> > community from the beginning.
> >
> > 1. Large code base change
> >
> > When contribution has large code base changes, it tend to take more time
> to
> > review and merged. Normally, most contributions merged in 1day~1 week.
> > But some contribution has large code base changes take 2~4 weeks. Few
> > contribution that has very large code base change take months.
> >
> > 2. Communication lost
> >
> > Sometimes, committer is not responding, or contributor is not responding.
> >
> > 3. Opinion diverges
> >
> > For some changes, included in contribution. When people have different
> > opinion and it continue to diverges, those PRs are not been merged.
> >
> >
> > I think having a guide such as ping other committer when a committer is
> not
> > responding, and divide contribution into small peaces if possible, would
> > help most of the cases.
> > Of course committer have to pay attention more to the contribution and
> > helping people. That's the first one.
> >
> > Thanks,
> > moon
> >
> > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org>
> wrote:
> >
> > > To make sure we're on the same page, here are two threads that I found
> > > related
> > > to this topic.
> > >
> > > Thread 1:
> > > Subject: R?
> > > Started on: July 1, 2015
> > >
> > > Thread 2:
> > > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > > Zeppelin
> > > Started on: August 13, 2015
> > >
> > > If you want to fetch these from our archive send emails to
> > > dev-thread.1735@zeppelin.incubator.apache.org
> > > dev-thread.3573@zeppelin.incubator.apache.org
> > >
> > > Cos
> > >
> > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > > Guys,
> > > >
> > > > While catching up on my emails from the last a couple of weeks, this
> > > thread
> > > > caught my attention. I am not normally paying much attention to the
> > code
> > > > reviews traffic from GH, but this one is pretty different as it spans
> > > three
> > > > months and counting.
> > > >
> > > > First, here are my five cents:
> > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > > contributed to
> > > > an ASF project this file should simply contain ASL2 text, like in [1]
> > > > - r/pom.xml perhaps shouldn't contain a separate <developers>
> section,
> > > but
> > > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > > > maintain this in the source code, but we have the list of all the
> > > > committers on the project's site [2] Every new committer's first
> > > commit is
> > > > to update the team page ;)
> > > > - comments like in
> > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > >
> > > > +/**
> > > > + * Created by aelberg on 7/28/15.
> > > > + */
> > > >
> > > > is better to be removed. It has been already discussed here [3]. And
> > > the
> > > > initial discussion on the topic [4] was linked as well
> > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> > > R-specific
> > > > stuff - I have no idea about R, honestly, but
> > > >
> > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > ...
> > > > +Author: David B. Dahl
> > > >
> > > > shouldn't be here, IMO. Normally, if some additional licenses are
> > > used,
> > > > they have to be listed in the top-level NOTICE file (already there).
> > > >
> > > > - I am not going to offer any comments on the technical merits of the
> > > patch,
> > > > as I haven't tried it personally. However, I don't see any serious
> > > > technical objections to the functionality in question.
> > > >
> > > > So, the question is - why the PR is open for three months? I hasn't
> > been
> > > able
> > > > to get a clear answer. What I found out though is pretty unsettling,
> > > really.
> > > > The communication on the topic is almost non-existent, except for
> this
> > > sparse
> > > > and bitter thread in the GH.
> > > >
> > > > I would love to hear from the committers what's stopping the
> acceptance
> > > of the
> > > > code, besides of the minor issues I've mentioned earlier? What are
> the
> > > reasons for it?
> > > > Is there anything wrong with it?
> > > > One of the responsibilities of the committers is to make sure the
> > > > contributions are reviewed; new contributors are guided and do
> > > understand how
> > > > the project ticks. The easy feedback flow attracts new people,
> allowing
> > > the
> > > > community to grow and thrive.
> > > >
> > > > I am asking _explicitely_ not to re-start the bickering I have
> already
> > > > seen. At this point I am interested in the purely technical side of
> > this.
> > > >
> > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > > [2] http://bigtop.apache.org/team-list.html
> > > > [3]
> > >
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > [4] http://s.apache.org/iZl
> > > >
> > > > With regards,
> > > > Cos
> > > >
> > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > Github user elbamos commented on the pull request:
> > > > >
> > > > >
> > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > >
> > > > > The current push should resolve some issues with changes in the
> > > > > Spark-Zeppelin interface that had created issues for users, as
> > > well as
> > > > > support for 1.5.1.
> > > > >
> > > > > Knitr support is improved, and the reason for a separate knitr
> > > interpreter may be clearer now.
> > > > >
> > > > > The amount of code borrowed from rscala is reduced.
> > > > >
> > > > > I did not address issues with @author tags, or files under the R/
> > > > > folder. The reason is, to be blunt, I don't understand what the
> > > precise
> > > > > concerns actually are.
> > > > >
> > > > > Please note that I changed .travis.yml to only use spark 1.4 and
> > > later.
> > > > > I'm sure there is a better way to do it, but I'm just not in a
> > > position
> > > > > to try to figure out the entire Zeppelin build process.
> > > > >
> > > > > Integrating this with the rest of zeppelin is going to take some
> > > work
> > > > > regarding pom's, travis, and so forth. I can do a lot of that,
> > > but I'm
> > > > > going to need to discuss it with the people who have been "owning"
> > > those
> > > > > files. There are just too many moving pieces here.
> > > > >
> > > > > Because of the size of this (which is, unfortunately, necessary),
> > > > > posting here is probably not the most efficient way. That is also
> > > true
> > > > > because certain people chose to use this PR as a forum to air other
> > > > > issues. Therefore, it would be better if reviewers e-mail me
> > > directly.
> > >
> > >
> > >
> >
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Just to put the final nail in this, I looked it up. 

I see no evidence of any “compiler exception.” 

There is an exception in the license for the runtime libraries that are bundled with GCC.  See: http://www.gnu.org/licenses/gcc-exception-3.1-faq.html

But no “compiler exception.” 

In fact, I believe this is part of the reason why LLVM was created. 

From: moon soo Lee <mo...@apache.org>
Reply: moon soo Lee <mo...@apache.org>
Date: December 1, 2015 at 8:16:36 PM
To: Amos B. Elberg <am...@gmail.com>, dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

Is knitR is commonly considered as a interpreter/compiler? or is it considered as a library routine?

Thanks,
moon

On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com> wrote:
Moon - you give this as an explanation of the licensing issue:  https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html 

According to that, there is an exception in the GPL for interpreter languages.  As long as you don’t distribute the code, its fine to talk to an interpreted language. 

Well, if that’s the case, then the PR plainly does not have a license issue.  It doesn’t distribute any GPL’d R code. 

I’m not sure what’s confusing about this.  It seems completely straightforward. 

Regarding this:


-- 
Amos Elberg
Sent with Airmail

From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 1, 2015 at 6:48:47 PM

To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com> wrote:

> I am going to try to minimize my reaction to Moon’s e-mail.
>
> The tl;dr is this:
>
> The reason we are having this discussion now is that active users of the
> PR — which now has its own user base — went public to complain about this.


> The PR has been tested by an active user base for more than three months.
> No-one has been able to identify any specific actual licensing problem, and
> the PR was prepared based on an extensive, careful review of the relevant
> licensing issues and after contacting the relevant people.
>
>

I admire every software that used by user and helping people. That includes
your work. But that's not the topic we're in discussion. Active user does
not mean your contribution can ignore the review.



> It is not an explanation for someone who has been ignoring my “how can I
> move this forward…” emails for three months to point the finger and say I
> didn’t contact the right person or file the right report.
>
>
This is also not the topic in this discussion.


> The burden for providing an explanation for the inaction is on the PMCC at
> this point.

I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> core, why do you think other PRs are passing CI?
> They’re not! I often see comments on PRs to just ignore that CI is
> failing.
>
> One of the most common reasons this PR fails CI, is CI times-out
> downloading Spark to install. How could that possibly be caused by the PR?
>
> It looks to me like the only PRs with changes to the relevant parts of the
> code — the SparkInterpreter — are being made by the person who wrote the
> testing suite.
>
> So, that would explain why some other PRs pass CI: Neither the
> SparkInterpreter nor the testing suite are stable or robust, but since the
> PRs are coming from the person who wrote both…
>
> And let's say Zeppelin core has problem and that makes your PR fails on CI
> test. That's possible. But it still does not mean we can merge it with CI
> failing.
>
> It means you should be working with me to figure out why the CI is failing.
>
> This PR has been tested by an active user base for the past three months.
> If CI is continuing to fail, and dozens of hours of effort have not
> resolved the CI issues, then it is time to start considering whether the
> testing suite is part of the problem.
>
> The level of defensiveness about the CI and SparkInterpreter is not
> helping to resolve these issues.
>
> If you think it's problem on Zeppelin core, then file an issue that
> reproduce the problem on Zeppelin core, that might be more efficient than
> keep trying yourself.
> I contacted you numerous times about such issues...
>


I remember i commented your issue about CI. but you just keep repeated it's
not your problem but Zeppelin core problem.

Then please file an issue about the problem you found in Zeppelin Core.
Then everyone will get into the problem.



>
> In my interpretation, KnitRInterpreter is not an optional feature while it
> is always enabled when compiling Zeppelin and always enabled when running
> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
> it will fail when no KnitR is installed on the system)
>
> Its not always enabled.
> It is not dynamically linked at runtime.
> It will not fail when knitr is missing. If knitr is not present, the repl
> interpreter starts and a note is written to to the log that the knitr
> interpreter isn’t available because knitr is not present.
>
> no Apache code can ever call a shell script, on the purpose of dynamic
> linking with GPL library.
> You misunderstand.
>
> The *shell* is GPL'd.
>
> Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends on a
> shell script to launch?
>
Obviously not.
>
> The interaction with R in the PR is the same.
>
>

Again, bash is one of exceptions of GPL, like other GPL licensed
compiler/interpreter.

Check here why Bash and R is okay with Apache License.
https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html

I'm not sure we can apply the same exception for 'using' KnitR.

My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter' you
wrote is not an optional feature. Which is clearly not optionally enabled
code and feature. And that depends on KnitR library which is GPL.



> I was guessing SparkR can be still in Apache License even if it is depends
> on R. Because of GPL licensed compiler generated output is not GPL license.
> and R is sort of compiler. If you can get answer from Spark community how
> SparkR get managed to stay in Apache License while R is GPL, the answer
> might help.
> The description of SparkR is not accurate in any respect. (Do you think
> SparkR is not talking to GPL-licensed libraries?)
>
> I don’t see that any genuine issue is being raised here.
>
> If there is an issue, the burden is on you to identify it.
>
> If i give you one suggestion, Zeppelin committers sometimes ask rebase the
> contribution branch for some reason. It is not the really the best
> practice, but still okay while most contributions are not including large
> code base changes
> However, your one, has more than 4000 lines of code change. If you rebase
> then review should be started from the beginning, again. So you might want
> to minimize the rebase your branch.
>
> Are you actually complaining that the problem is that I rebased the code
> during the three-month period when no-one looked at it and Zeppelin went
> through a release?
>
> I cannot take it seriously when you say things like this.
>
> Having to “start from the beginning” cannot be a problem if you never
> actually started the first time...
>
>

You wanted coordination and cooperation. So i gave you suggestion that
helping review process. For example, your code has been rebased since my
comment and jongyoul's comment. that means committers will need to look
from the beginning. That'll require more time. And maybe, i guess that's
not what you want. But If you don't care, feel free to rebase.

Thanks,
moon



>
> From: moon soo Lee <mo...@apache.org>
> Reply: moon soo Lee <mo...@apache.org>
> Date: December 1, 2015 at 4:42:06 AM
> To: Amos B. Elberg <am...@gmail.com>,
> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
> wrote:
> Thank you, Cos.
>
> I’d like to briefly address the issues raised by Moon:
>
> 1. This PR does not passes CI
> The CI fails on core Zeppelin, *not* code in this PR.
>
> I’ve been seeking assistance on this since August.
>
> The most common reason is that SparkInterpreter is unable to launch Spark
> and open a Spark Backend. This is necessary to test the PR.
>
> 60+ hours, has been spent adapting and re-basing when the SparkInterpreter
> architecture changed and broke the PR’s *tests.*
>
>
> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> core, why do you think other PRs are passing CI?
>
> And let's say Zeppelin core has problem and that makes your PR fails on CI
> test. That's possible. But it still does not mean we can merge it with CI
> failing.
>
> If you think it's problem on Zeppelin core, then file an issue that
> reproduce the problem on Zeppelin core, that might be more efficient than
> keep trying yourself.
>
>
> 2. Not 100% sure this PR has no license issue. (about KniteR)
> What license problem *specifically* do you believe may exist?
>
> In preparing the PR, I:
>
> * Reviewed the Apache policy in detail.
>
> * Contacted the FSF to ask their interpretation of the “linking”
> provisions of the GPL license.
>
> * Reviewed how other Apache software deals with this issue (e.g., Spark
> talks to R, which is GPL'd).
>
> * No necessary *dependencies* of the PR have license conflicts. In
> several cases, I contacted package authors who agreed to re-issue their
> packages under Apache-compatible licenses. (Usually I had to do a bit of
> coding in exchange…)
>
> * Where the license had to stay GPL, the packages are *not necessary* and
> *not dependencies.* If the optional packages are present, they will be
> used to enable additional functionality. Knitr is an example. The PR will
> compile and run fine without knitr. If knitr is available (it is part of
> the most common R distribution), the PR will enable the knitr interpreter.
>
> * This is exactly how this issue is addressed through the Apache
> ecosystem.
> The tl;dr is this: When Apache code is written to talk to libraries that
> may or may not be present on the user’s system, or where it talks to an API
> but is agnostic about implementation, that is not “linking” in a way that
> implicate the anti-linking provision of the GPL.
>
> Otherwise, no Apache code could ever call a shell script! Let alone run
> on Linux, or talk to R.
>
>
> I'm not a legal expert. So following could be wrong.
>
> In my interpretation, KnitRInterpreter is not an optional feature while it
> is always enabled when compiling Zeppelin and always enabled when running
> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
> it will fail when no KnitR is installed on the system)
>
> And of course, no Apache code can ever call a shell script, on the purpose
> of dynamic linking with GPL library.
>
> I was guessing SparkR can be still in Apache License even if it is depends
> on R. Because of GPL licensed compiler generated output is not GPL license.
> and R is sort of compiler.
>
> If you can get answer from Spark community how SparkR get managed to stay
> in Apache License while R is GPL, the answer might help.
>
>
> 3. Need more time to review.
> Has any reviewer has downloaded the PR or run the demo notebook? (Which
> is there for the benefit of reviewers, and isn’t intended to go in final
> distribution.)
>
> How many +1 comments from users would you like to see?
>
> How much time do you believe is required?
>
>
> It all depends on when CI is going to pass, when license problem is going
> to be cleared, and when a committer willing to review and responsible to
> commit your contribution.
>
>
> 1. Large code base change
> Large code base changes require coordination and cooperation. This PR
> necessarily implicates the build scripts, testing code, the
> SparkInterpreter, etc.
>
> I have been seeking to coordinate since August.
>
> I continue to stand ready to do so.
>
> -Amos
>
>
> If i give you one suggestion, Zeppelin committers sometimes ask rebase the
> contribution branch for some reason. It is not the really the best
> practice, but still okay while most contributions are not including large
> code base changes.
>
> However, your one, has more than 4000 lines of code change. If you rebase
> then review should be started from the beginning, again. So you might want
> to minimize the rebase your branch.
>
> Thanks,
> moon
>
>
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> Date: December 1, 2015 at 1:34:19 AM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> Hi Cos,
>
> Thanks for opening a discussion.
> My answer about 'Why this PR is open for three months' is
>
> 1. This PR does not passes CI
> 2. Not 100% sure this PR has no license issue. (about KniteR)
> 3. Need more time to review.
>
> It's my personal answer, other committers may have different opinion.
>
>
> I think the question should be generalized. Because this PR is not the only
> PR that is in impasse. There're more. For example
>
> Here's some examples that PR is not been merged.
> https://github.com/apache/incubator-zeppelin/pull/53,
> https://github.com/apache/incubator-zeppelin/pull/60
> and so on.
>
> I can categorize the cases, based on experience of involving Zeppelin
> community from the beginning.
>
> 1. Large code base change
>
> When contribution has large code base changes, it tend to take more time to
> review and merged. Normally, most contributions merged in 1day~1 week.
> But some contribution has large code base changes take 2~4 weeks. Few
> contribution that has very large code base change take months.
>
> 2. Communication lost
>
> Sometimes, committer is not responding, or contributor is not responding.
>
> 3. Opinion diverges
>
> For some changes, included in contribution. When people have different
> opinion and it continue to diverges, those PRs are not been merged.
>
>
> I think having a guide such as ping other committer when a committer is not
> responding, and divide contribution into small peaces if possible, would
> help most of the cases.
> Of course committer have to pay attention more to the contribution and
> helping people. That's the first one.
>
> Thanks,
> moon
>
> On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org> wrote:
>
> > To make sure we're on the same page, here are two threads that I found
> > related
> > to this topic.
> >
> > Thread 1:
> > Subject: R?
> > Started on: July 1, 2015
> >
> > Thread 2:
> > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > Zeppelin
> > Started on: August 13, 2015
> >
> > If you want to fetch these from our archive send emails to
> > dev-thread.1735@zeppelin.incubator.apache.org
> > dev-thread.3573@zeppelin.incubator.apache.org
> >
> > Cos
> >
> > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > Guys,
> > >
> > > While catching up on my emails from the last a couple of weeks, this
> > thread
> > > caught my attention. I am not normally paying much attention to the
> code
> > > reviews traffic from GH, but this one is pretty different as it spans
> > three
> > > months and counting.
> > >
> > > First, here are my five cents:
> > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > contributed to
> > > an ASF project this file should simply contain ASL2 text, like in [1]
> > > - r/pom.xml perhaps shouldn't contain a separate <developers> section,
> > but
> > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > > maintain this in the source code, but we have the list of all the
> > > committers on the project's site [2] Every new committer's first
> > commit is
> > > to update the team page ;)
> > > - comments like in
> > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > >
> > > +/**
> > > + * Created by aelberg on 7/28/15.
> > > + */
> > >
> > > is better to be removed. It has been already discussed here [3]. And
> > the
> > > initial discussion on the topic [4] was linked as well
> > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> > R-specific
> > > stuff - I have no idea about R, honestly, but
> > >
> > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > ...
> > > +Author: David B. Dahl
> > >
> > > shouldn't be here, IMO. Normally, if some additional licenses are
> > used,
> > > they have to be listed in the top-level NOTICE file (already there).
> > >
> > > - I am not going to offer any comments on the technical merits of the
> > patch,
> > > as I haven't tried it personally. However, I don't see any serious
> > > technical objections to the functionality in question.
> > >
> > > So, the question is - why the PR is open for three months? I hasn't
> been
> > able
> > > to get a clear answer. What I found out though is pretty unsettling,
> > really.
> > > The communication on the topic is almost non-existent, except for this
> > sparse
> > > and bitter thread in the GH.
> > >
> > > I would love to hear from the committers what's stopping the acceptance
> > of the
> > > code, besides of the minor issues I've mentioned earlier? What are the
> > reasons for it?
> > > Is there anything wrong with it?
> > > One of the responsibilities of the committers is to make sure the
> > > contributions are reviewed; new contributors are guided and do
> > understand how
> > > the project ticks. The easy feedback flow attracts new people, allowing
> > the
> > > community to grow and thrive.
> > >
> > > I am asking _explicitely_ not to re-start the bickering I have already
> > > seen. At this point I am interested in the purely technical side of
> this.
> > >
> > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > [2] http://bigtop.apache.org/team-list.html
> > > [3]
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > [4] http://s.apache.org/iZl
> > >
> > > With regards,
> > > Cos
> > >
> > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > Github user elbamos commented on the pull request:
> > > >
> > > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > >
> > > > The current push should resolve some issues with changes in the
> > > > Spark-Zeppelin interface that had created issues for users, as
> > well as
> > > > support for 1.5.1.
> > > >
> > > > Knitr support is improved, and the reason for a separate knitr
> > interpreter may be clearer now.
> > > >
> > > > The amount of code borrowed from rscala is reduced.
> > > >
> > > > I did not address issues with @author tags, or files under the R/
> > > > folder. The reason is, to be blunt, I don't understand what the
> > precise
> > > > concerns actually are.
> > > >
> > > > Please note that I changed .travis.yml to only use spark 1.4 and
> > later.
> > > > I'm sure there is a better way to do it, but I'm just not in a
> > position
> > > > to try to figure out the entire Zeppelin build process.
> > > >
> > > > Integrating this with the rest of zeppelin is going to take some
> > work
> > > > regarding pom's, travis, and so forth. I can do a lot of that,
> > but I'm
> > > > going to need to discuss it with the people who have been "owning"
> > those
> > > > files. There are just too many moving pieces here.
> > > >
> > > > Because of the size of this (which is, unfortunately, necessary),
> > > > posting here is probably not the most efficient way. That is also
> > true
> > > > because certain people chose to use this PR as a forum to air other
> > > > issues. Therefore, it would be better if reviewers e-mail me
> > directly.
> >
> >
> >
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Konstantin Boudnik <co...@apache.org>]

The Foundation was around for 16 years and had worked our a lot of "best
practices" on how to deal with the situations like that. It's called "Apache
Way". So, you hardly need to search around to find out what to do when dealing
with interesting situations.

Cos

On Wed, Dec 02, 2015 at 07:48AM, Eran Witkon wrote:
> The more I read this thread and other thread about this topic the more I
> feel discomfort from the way we act as a community.
> I welcome Roman willingness to help here but suggest we all look and adopt
> the following way of thinking...
> 
> http://ecorner.stanford.edu/authorMaterialInfo.html?mid=1642
> 
> Eran
> On Wed, 2 Dec 2015 at 08:50 Amos B. Elberg <am...@gmail.com> wrote:
> 
> > Roman - thank you for taking a look at the PR!  The PR should compile
> > out-of-box if you have R and the evaluate package installed.
> >
> > I have been working on a “release” version rebased to match 5.5-release.
> > It has documentation, including what features are added with optional
> > (incompatible-license) R packages.  And how to build a seamless Spark
> > pipeline that mixes scala, python, and R without breaking lazy evaluation.
> >
> > I can send you a zip of that tomorrow, and will contact you directly to
> > provide it.
> >
> > From: Roman Shaposhnik <ro...@shaposhnik.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > Date: December 2, 2015 at 12:30:52 AM
> > To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> > CC: moon soo Lee <mo...@apache.org>
> > Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> > request: R Interpreter for Zeppelin
> >
> > Hi!
> >
> > reading through the thread made me realize that what Amos
> > is saying makes perfect sense to me. That said, I don't like to
> > take Moon's concerns lightly either.
> >
> > Hence, would it be possible to get the *bundle* of a binary
> > package built with the patch applied and ready to go? I'd
> > like to provide an extra pair of eyes to help settle this.
> >
> > Thanks,
> > Roman.
> >
> > P.S. I do a lot of this type of work around IP at my $DAYJOB
> >
> > On Tue, Dec 1, 2015 at 5:25 PM, Amos B. Elberg <am...@gmail.com>
> > wrote:
> > > Moon - If there were seriously a licensing issue, then you or someone
> > else would be able to say clearly and specifically what it is.
> > >
> > > There plainly is not. This idea you have about a “compiler exception”
> > has nothing to do with any of this. The link you sent doesn’t talk about
> > any “compiler exception.” (I’m not sure what a 2008 e-mail from a developer
> > who wanted to release a branded version of R has to do with Apache’s policy
> > concerning linking to GPL’d code anyway…)
> > >
> > > My e-mail was accidentally sent incomplete. This is the rest:
> > >
> > > You say this:
> > >
> > > My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter'
> > you
> > > wrote is not an optional feature. Which is clearly not optionally enabled
> > > code and feature. And that depends on KnitR library which is GPL.
> > >
> > > That’s false.
> > >
> > > ***
> > > I addressed the licensing issue only because waving the “license” flag
> > could scare people off.
> > >
> > > If someone really believed there was a licensing issue, they would be
> > able to say clearly what that issue is.
> > >
> > > I am not going to respond to the rest of Moon’s e-mail.
> > >
> > >
> > > From: moon soo Lee <mo...@apache.org>
> > > Reply: moon soo Lee <mo...@apache.org>
> > > Date: December 1, 2015 at 8:16:36 PM
> > > To: Amos B. Elberg <am...@gmail.com>,
> > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > pull request: R Interpreter for Zeppelin
> > >
> > > Is knitR is commonly considered as a interpreter/compiler? or is it
> > considered as a library routine?
> > >
> > > Thanks,
> > > moon
> > >
> > > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com>
> > wrote:
> > > Moon - you give this as an explanation of the licensing issue:
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > >
> > > According to that, there is an exception in the GPL for interpreter
> > languages. As long as you don’t distribute the code, its fine to talk to an
> > interpreted language.
> > >
> > > Well, if that’s the case, then the PR plainly does not have a license
> > issue. It doesn’t distribute any GPL’d R code.
> > >
> > > I’m not sure what’s confusing about this. It seems completely
> > straightforward.
> > >
> > > Regarding this:
> > >
> > >
> > > --
> > > Amos Elberg
> > > Sent with Airmail
> > >
> > > From: moon soo Lee <mo...@apache.org>
> > > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > > Date: December 1, 2015 at 6:48:47 PM
> > >
> > > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> > >
> > > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > pull request: R Interpreter for Zeppelin
> > >
> > > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com>
> > wrote:
> > >
> > >> I am going to try to minimize my reaction to Moon’s e-mail.
> > >>
> > >> The tl;dr is this:
> > >>
> > >> The reason we are having this discussion now is that active users of the
> > >> PR — which now has its own user base — went public to complain about
> > this.
> > >
> > >
> > >> The PR has been tested by an active user base for more than three
> > months.
> > >> No-one has been able to identify any specific actual licensing problem,
> > and
> > >> the PR was prepared based on an extensive, careful review of the
> > relevant
> > >> licensing issues and after contacting the relevant people.
> > >>
> > >>
> > >
> > > I admire every software that used by user and helping people. That
> > includes
> > > your work. But that's not the topic we're in discussion. Active user does
> > > not mean your contribution can ignore the review.
> > >
> > >
> > >
> > >> It is not an explanation for someone who has been ignoring my “how can I
> > >> move this forward…” emails for three months to point the finger and say
> > I
> > >> didn’t contact the right person or file the right report.
> > >>
> > >>
> > > This is also not the topic in this discussion.
> > >
> > >
> > >> The burden for providing an explanation for the inaction is on the PMCC
> > at
> > >> this point.
> > >
> > > I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> > >> core, why do you think other PRs are passing CI?
> > >> They’re not! I often see comments on PRs to just ignore that CI is
> > >> failing.
> > >>
> > >> One of the most common reasons this PR fails CI, is CI times-out
> > >> downloading Spark to install. How could that possibly be caused by the
> > PR?
> > >>
> > >> It looks to me like the only PRs with changes to the relevant parts of
> > the
> > >> code — the SparkInterpreter — are being made by the person who wrote the
> > >> testing suite.
> > >>
> > >> So, that would explain why some other PRs pass CI: Neither the
> > >> SparkInterpreter nor the testing suite are stable or robust, but since
> > the
> > >> PRs are coming from the person who wrote both…
> > >>
> > >> And let's say Zeppelin core has problem and that makes your PR fails on
> > CI
> > >> test. That's possible. But it still does not mean we can merge it with
> > CI
> > >> failing.
> > >>
> > >> It means you should be working with me to figure out why the CI is
> > failing.
> > >>
> > >> This PR has been tested by an active user base for the past three
> > months.
> > >> If CI is continuing to fail, and dozens of hours of effort have not
> > >> resolved the CI issues, then it is time to start considering whether the
> > >> testing suite is part of the problem.
> > >>
> > >> The level of defensiveness about the CI and SparkInterpreter is not
> > >> helping to resolve these issues.
> > >>
> > >> If you think it's problem on Zeppelin core, then file an issue that
> > >> reproduce the problem on Zeppelin core, that might be more efficient
> > than
> > >> keep trying yourself.
> > >> I contacted you numerous times about such issues...
> > >>
> > >
> > >
> > > I remember i commented your issue about CI. but you just keep repeated
> > it's
> > > not your problem but Zeppelin core problem.
> > >
> > > Then please file an issue about the problem you found in Zeppelin Core.
> > > Then everyone will get into the problem.
> > >
> > >
> > >
> > >>
> > >> In my interpretation, KnitRInterpreter is not an optional feature while
> > it
> > >> is always enabled when compiling Zeppelin and always enabled when
> > running
> > >> Zeppelin. And it requires dynamically linked GPL library on runtime.
> > (yes
> > >> it will fail when no KnitR is installed on the system)
> > >>
> > >> Its not always enabled.
> > >> It is not dynamically linked at runtime.
> > >> It will not fail when knitr is missing. If knitr is not present, the
> > repl
> > >> interpreter starts and a note is written to to the log that the knitr
> > >> interpreter isn’t available because knitr is not present.
> > >>
> > >> no Apache code can ever call a shell script, on the purpose of dynamic
> > >> linking with GPL library.
> > >> You misunderstand.
> > >>
> > >> The *shell* is GPL'd.
> > >>
> > >> Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends
> > on a
> > >> shell script to launch?
> > >>
> > > Obviously not.
> > >>
> > >> The interaction with R in the PR is the same.
> > >>
> > >>
> > >
> > > Again, bash is one of exceptions of GPL, like other GPL licensed
> > > compiler/interpreter.
> > >
> > > Check here why Bash and R is okay with Apache License.
> > > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> > >
> > > I'm not sure we can apply the same exception for 'using' KnitR.
> > >
> > > My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter'
> > you
> > > wrote is not an optional feature. Which is clearly not optionally enabled
> > > code and feature. And that depends on KnitR library which is GPL.
> > >
> > >
> > >
> > >> I was guessing SparkR can be still in Apache License even if it is
> > depends
> > >> on R. Because of GPL licensed compiler generated output is not GPL
> > license.
> > >> and R is sort of compiler. If you can get answer from Spark community
> > how
> > >> SparkR get managed to stay in Apache License while R is GPL, the answer
> > >> might help.
> > >> The description of SparkR is not accurate in any respect. (Do you think
> > >> SparkR is not talking to GPL-licensed libraries?)
> > >>
> > >> I don’t see that any genuine issue is being raised here.
> > >>
> > >> If there is an issue, the burden is on you to identify it.
> > >>
> > >> If i give you one suggestion, Zeppelin committers sometimes ask rebase
> > the
> > >> contribution branch for some reason. It is not the really the best
> > >> practice, but still okay while most contributions are not including
> > large
> > >> code base changes
> > >> However, your one, has more than 4000 lines of code change. If you
> > rebase
> > >> then review should be started from the beginning, again. So you might
> > want
> > >> to minimize the rebase your branch.
> > >>
> > >> Are you actually complaining that the problem is that I rebased the code
> > >> during the three-month period when no-one looked at it and Zeppelin went
> > >> through a release?
> > >>
> > >> I cannot take it seriously when you say things like this.
> > >>
> > >> Having to “start from the beginning” cannot be a problem if you never
> > >> actually started the first time...
> > >>
> > >>
> > >
> > > You wanted coordination and cooperation. So i gave you suggestion that
> > > helping review process. For example, your code has been rebased since my
> > > comment and jongyoul's comment. that means committers will need to look
> > > from the beginning. That'll require more time. And maybe, i guess that's
> > > not what you want. But If you don't care, feel free to rebase.
> > >
> > > Thanks,
> > > moon
> > >
> > >
> > >
> > >>
> > >> From: moon soo Lee <mo...@apache.org>
> > >> Reply: moon soo Lee <mo...@apache.org>
> > >> Date: December 1, 2015 at 4:42:06 AM
> > >> To: Amos B. Elberg <am...@gmail.com>,
> > >> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> > >> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > pull
> > >> request: R Interpreter for Zeppelin
> > >>
> > >> On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
> > >> wrote:
> > >> Thank you, Cos.
> > >>
> > >> I’d like to briefly address the issues raised by Moon:
> > >>
> > >> 1. This PR does not passes CI
> > >> The CI fails on core Zeppelin, *not* code in this PR.
> > >>
> > >> I’ve been seeking assistance on this since August.
> > >>
> > >> The most common reason is that SparkInterpreter is unable to launch
> > Spark
> > >> and open a Spark Backend. This is necessary to test the PR.
> > >>
> > >> 60+ hours, has been spent adapting and re-basing when the
> > SparkInterpreter
> > >> architecture changed and broke the PR’s *tests.*
> > >>
> > >>
> > >> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> > >> core, why do you think other PRs are passing CI?
> > >>
> > >> And let's say Zeppelin core has problem and that makes your PR fails on
> > CI
> > >> test. That's possible. But it still does not mean we can merge it with
> > CI
> > >> failing.
> > >>
> > >> If you think it's problem on Zeppelin core, then file an issue that
> > >> reproduce the problem on Zeppelin core, that might be more efficient
> > than
> > >> keep trying yourself.
> > >>
> > >>
> > >> 2. Not 100% sure this PR has no license issue. (about KniteR)
> > >> What license problem *specifically* do you believe may exist?
> > >>
> > >> In preparing the PR, I:
> > >>
> > >> * Reviewed the Apache policy in detail.
> > >>
> > >> * Contacted the FSF to ask their interpretation of the “linking”
> > >> provisions of the GPL license.
> > >>
> > >> * Reviewed how other Apache software deals with this issue (e.g., Spark
> > >> talks to R, which is GPL'd).
> > >>
> > >> * No necessary *dependencies* of the PR have license conflicts. In
> > >> several cases, I contacted package authors who agreed to re-issue their
> > >> packages under Apache-compatible licenses. (Usually I had to do a bit of
> > >> coding in exchange…)
> > >>
> > >> * Where the license had to stay GPL, the packages are *not necessary*
> > and
> > >> *not dependencies.* If the optional packages are present, they will be
> > >> used to enable additional functionality. Knitr is an example. The PR
> > will
> > >> compile and run fine without knitr. If knitr is available (it is part of
> > >> the most common R distribution), the PR will enable the knitr
> > interpreter.
> > >>
> > >> * This is exactly how this issue is addressed through the Apache
> > >> ecosystem.
> > >> The tl;dr is this: When Apache code is written to talk to libraries that
> > >> may or may not be present on the user’s system, or where it talks to an
> > API
> > >> but is agnostic about implementation, that is not “linking” in a way
> > that
> > >> implicate the anti-linking provision of the GPL.
> > >>
> > >> Otherwise, no Apache code could ever call a shell script! Let alone run
> > >> on Linux, or talk to R.
> > >>
> > >>
> > >> I'm not a legal expert. So following could be wrong.
> > >>
> > >> In my interpretation, KnitRInterpreter is not an optional feature while
> > it
> > >> is always enabled when compiling Zeppelin and always enabled when
> > running
> > >> Zeppelin. And it requires dynamically linked GPL library on runtime.
> > (yes
> > >> it will fail when no KnitR is installed on the system)
> > >>
> > >> And of course, no Apache code can ever call a shell script, on the
> > purpose
> > >> of dynamic linking with GPL library.
> > >>
> > >> I was guessing SparkR can be still in Apache License even if it is
> > depends
> > >> on R. Because of GPL licensed compiler generated output is not GPL
> > license.
> > >> and R is sort of compiler.
> > >>
> > >> If you can get answer from Spark community how SparkR get managed to
> > stay
> > >> in Apache License while R is GPL, the answer might help.
> > >>
> > >>
> > >> 3. Need more time to review.
> > >> Has any reviewer has downloaded the PR or run the demo notebook? (Which
> > >> is there for the benefit of reviewers, and isn’t intended to go in final
> > >> distribution.)
> > >>
> > >> How many +1 comments from users would you like to see?
> > >>
> > >> How much time do you believe is required?
> > >>
> > >>
> > >> It all depends on when CI is going to pass, when license problem is
> > going
> > >> to be cleared, and when a committer willing to review and responsible to
> > >> commit your contribution.
> > >>
> > >>
> > >> 1. Large code base change
> > >> Large code base changes require coordination and cooperation. This PR
> > >> necessarily implicates the build scripts, testing code, the
> > >> SparkInterpreter, etc.
> > >>
> > >> I have been seeking to coordinate since August.
> > >>
> > >> I continue to stand ready to do so.
> > >>
> > >> -Amos
> > >>
> > >>
> > >> If i give you one suggestion, Zeppelin committers sometimes ask rebase
> > the
> > >> contribution branch for some reason. It is not the really the best
> > >> practice, but still okay while most contributions are not including
> > large
> > >> code base changes.
> > >>
> > >> However, your one, has more than 4000 lines of code change. If you
> > rebase
> > >> then review should be started from the beginning, again. So you might
> > want
> > >> to minimize the rebase your branch.
> > >>
> > >> Thanks,
> > >> moon
> > >>
> > >>
> > >> From: moon soo Lee <mo...@apache.org>
> > >> Reply: dev@zeppelin.incubator.apache.org <
> > >> dev@zeppelin.incubator.apache.org>
> > >> Date: December 1, 2015 at 1:34:19 AM
> > >> To: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > >> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> > pull
> > >> request: R Interpreter for Zeppelin
> > >>
> > >> Hi Cos,
> > >>
> > >> Thanks for opening a discussion.
> > >> My answer about 'Why this PR is open for three months' is
> > >>
> > >> 1. This PR does not passes CI
> > >> 2. Not 100% sure this PR has no license issue. (about KniteR)
> > >> 3. Need more time to review.
> > >>
> > >> It's my personal answer, other committers may have different opinion.
> > >>
> > >>
> > >> I think the question should be generalized. Because this PR is not the
> > only
> > >> PR that is in impasse. There're more. For example
> > >>
> > >> Here's some examples that PR is not been merged.
> > >> https://github.com/apache/incubator-zeppelin/pull/53,
> > >> https://github.com/apache/incubator-zeppelin/pull/60
> > >> and so on.
> > >>
> > >> I can categorize the cases, based on experience of involving Zeppelin
> > >> community from the beginning.
> > >>
> > >> 1. Large code base change
> > >>
> > >> When contribution has large code base changes, it tend to take more
> > time to
> > >> review and merged. Normally, most contributions merged in 1day~1 week.
> > >> But some contribution has large code base changes take 2~4 weeks. Few
> > >> contribution that has very large code base change take months.
> > >>
> > >> 2. Communication lost
> > >>
> > >> Sometimes, committer is not responding, or contributor is not
> > responding.
> > >>
> > >> 3. Opinion diverges
> > >>
> > >> For some changes, included in contribution. When people have different
> > >> opinion and it continue to diverges, those PRs are not been merged.
> > >>
> > >>
> > >> I think having a guide such as ping other committer when a committer is
> > not
> > >> responding, and divide contribution into small peaces if possible, would
> > >> help most of the cases.
> > >> Of course committer have to pay attention more to the contribution and
> > >> helping people. That's the first one.
> > >>
> > >> Thanks,
> > >> moon
> > >>
> > >> On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org>
> > wrote:
> > >>
> > >> > To make sure we're on the same page, here are two threads that I found
> > >> > related
> > >> > to this topic.
> > >> >
> > >> > Thread 1:
> > >> > Subject: R?
> > >> > Started on: July 1, 2015
> > >> >
> > >> > Thread 2:
> > >> > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > >> > Zeppelin
> > >> > Started on: August 13, 2015
> > >> >
> > >> > If you want to fetch these from our archive send emails to
> > >> > dev-thread.1735@zeppelin.incubator.apache.org
> > >> > dev-thread.3573@zeppelin.incubator.apache.org
> > >> >
> > >> > Cos
> > >> >
> > >> > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > >> > > Guys,
> > >> > >
> > >> > > While catching up on my emails from the last a couple of weeks, this
> > >> > thread
> > >> > > caught my attention. I am not normally paying much attention to the
> > >> code
> > >> > > reviews traffic from GH, but this one is pretty different as it
> > spans
> > >> > three
> > >> > > months and counting.
> > >> > >
> > >> > > First, here are my five cents:
> > >> > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > >> > contributed to
> > >> > > an ASF project this file should simply contain ASL2 text, like in
> > [1]
> > >> > > - r/pom.xml perhaps shouldn't contain a separate <developers>
> > section,
> > >> > but
> > >> > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > >> > > maintain this in the source code, but we have the list of all the
> > >> > > committers on the project's site [2] Every new committer's first
> > >> > commit is
> > >> > > to update the team page ;)
> > >> > > - comments like in
> > >> > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > >> > >
> > >> > > +/**
> > >> > > + * Created by aelberg on 7/28/15.
> > >> > > + */
> > >> > >
> > >> > > is better to be removed. It has been already discussed here [3]. And
> > >> > the
> > >> > > initial discussion on the topic [4] was linked as well
> > >> > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> > >> > R-specific
> > >> > > stuff - I have no idea about R, honestly, but
> > >> > >
> > >> > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > >> > > ...
> > >> > > +Author: David B. Dahl
> > >> > >
> > >> > > shouldn't be here, IMO. Normally, if some additional licenses are
> > >> > used,
> > >> > > they have to be listed in the top-level NOTICE file (already there).
> > >> > >
> > >> > > - I am not going to offer any comments on the technical merits of
> > the
> > >> > patch,
> > >> > > as I haven't tried it personally. However, I don't see any serious
> > >> > > technical objections to the functionality in question.
> > >> > >
> > >> > > So, the question is - why the PR is open for three months? I hasn't
> > >> been
> > >> > able
> > >> > > to get a clear answer. What I found out though is pretty unsettling,
> > >> > really.
> > >> > > The communication on the topic is almost non-existent, except for
> > this
> > >> > sparse
> > >> > > and bitter thread in the GH.
> > >> > >
> > >> > > I would love to hear from the committers what's stopping the
> > acceptance
> > >> > of the
> > >> > > code, besides of the minor issues I've mentioned earlier? What are
> > the
> > >> > reasons for it?
> > >> > > Is there anything wrong with it?
> > >> > > One of the responsibilities of the committers is to make sure the
> > >> > > contributions are reviewed; new contributors are guided and do
> > >> > understand how
> > >> > > the project ticks. The easy feedback flow attracts new people,
> > allowing
> > >> > the
> > >> > > community to grow and thrive.
> > >> > >
> > >> > > I am asking _explicitely_ not to re-start the bickering I have
> > already
> > >> > > seen. At this point I am interested in the purely technical side of
> > >> this.
> > >> > >
> > >> > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > >> > > [2] http://bigtop.apache.org/team-list.html
> > >> > > [3]
> > >> >
> > >>
> > http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > >> > > [4] http://s.apache.org/iZl
> > >> > >
> > >> > > With regards,
> > >> > > Cos
> > >> > >
> > >> > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > >> > > > Github user elbamos commented on the pull request:
> > >> > > >
> > >> > > >
> > >> >
> > >>
> > https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > >> > > >
> > >> > > > The current push should resolve some issues with changes in the
> > >> > > > Spark-Zeppelin interface that had created issues for users, as
> > >> > well as
> > >> > > > support for 1.5.1.
> > >> > > >
> > >> > > > Knitr support is improved, and the reason for a separate knitr
> > >> > interpreter may be clearer now.
> > >> > > >
> > >> > > > The amount of code borrowed from rscala is reduced.
> > >> > > >
> > >> > > > I did not address issues with @author tags, or files under the R/
> > >> > > > folder. The reason is, to be blunt, I don't understand what the
> > >> > precise
> > >> > > > concerns actually are.
> > >> > > >
> > >> > > > Please note that I changed .travis.yml to only use spark 1.4 and
> > >> > later.
> > >> > > > I'm sure there is a better way to do it, but I'm just not in a
> > >> > position
> > >> > > > to try to figure out the entire Zeppelin build process.
> > >> > > >
> > >> > > > Integrating this with the rest of zeppelin is going to take some
> > >> > work
> > >> > > > regarding pom's, travis, and so forth. I can do a lot of that,
> > >> > but I'm
> > >> > > > going to need to discuss it with the people who have been "owning"
> > >> > those
> > >> > > > files. There are just too many moving pieces here.
> > >> > > >
> > >> > > > Because of the size of this (which is, unfortunately, necessary),
> > >> > > > posting here is probably not the most efficient way. That is also
> > >> > true
> > >> > > > because certain people chose to use this PR as a forum to air
> > other
> > >> > > > issues. Therefore, it would be better if reviewers e-mail me
> > >> > directly.
> > >> >
> > >> >
> > >> >
> > >>
> >

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Eran Witkon <er...@gmail.com>]

The more I read this thread and other thread about this topic the more I
feel discomfort from the way we act as a community.
I welcome Roman willingness to help here but suggest we all look and adopt
the following way of thinking...

http://ecorner.stanford.edu/authorMaterialInfo.html?mid=1642

Eran
On Wed, 2 Dec 2015 at 08:50 Amos B. Elberg <am...@gmail.com> wrote:

> Roman - thank you for taking a look at the PR!  The PR should compile
> out-of-box if you have R and the evaluate package installed.
>
> I have been working on a “release” version rebased to match 5.5-release.
> It has documentation, including what features are added with optional
> (incompatible-license) R packages.  And how to build a seamless Spark
> pipeline that mixes scala, python, and R without breaking lazy evaluation.
>
> I can send you a zip of that tomorrow, and will contact you directly to
> provide it.
>
> From: Roman Shaposhnik <ro...@shaposhnik.org>
> Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> Date: December 2, 2015 at 12:30:52 AM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> CC: moon soo Lee <mo...@apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> Hi!
>
> reading through the thread made me realize that what Amos
> is saying makes perfect sense to me. That said, I don't like to
> take Moon's concerns lightly either.
>
> Hence, would it be possible to get the *bundle* of a binary
> package built with the patch applied and ready to go? I'd
> like to provide an extra pair of eyes to help settle this.
>
> Thanks,
> Roman.
>
> P.S. I do a lot of this type of work around IP at my $DAYJOB
>
> On Tue, Dec 1, 2015 at 5:25 PM, Amos B. Elberg <am...@gmail.com>
> wrote:
> > Moon - If there were seriously a licensing issue, then you or someone
> else would be able to say clearly and specifically what it is.
> >
> > There plainly is not. This idea you have about a “compiler exception”
> has nothing to do with any of this. The link you sent doesn’t talk about
> any “compiler exception.” (I’m not sure what a 2008 e-mail from a developer
> who wanted to release a branded version of R has to do with Apache’s policy
> concerning linking to GPL’d code anyway…)
> >
> > My e-mail was accidentally sent incomplete. This is the rest:
> >
> > You say this:
> >
> > My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter'
> you
> > wrote is not an optional feature. Which is clearly not optionally enabled
> > code and feature. And that depends on KnitR library which is GPL.
> >
> > That’s false.
> >
> > ***
> > I addressed the licensing issue only because waving the “license” flag
> could scare people off.
> >
> > If someone really believed there was a licensing issue, they would be
> able to say clearly what that issue is.
> >
> > I am not going to respond to the rest of Moon’s e-mail.
> >
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: moon soo Lee <mo...@apache.org>
> > Date: December 1, 2015 at 8:16:36 PM
> > To: Amos B. Elberg <am...@gmail.com>,
> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull request: R Interpreter for Zeppelin
> >
> > Is knitR is commonly considered as a interpreter/compiler? or is it
> considered as a library routine?
> >
> > Thanks,
> > moon
> >
> > On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com>
> wrote:
> > Moon - you give this as an explanation of the licensing issue:
> https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> >
> > According to that, there is an exception in the GPL for interpreter
> languages. As long as you don’t distribute the code, its fine to talk to an
> interpreted language.
> >
> > Well, if that’s the case, then the PR plainly does not have a license
> issue. It doesn’t distribute any GPL’d R code.
> >
> > I’m not sure what’s confusing about this. It seems completely
> straightforward.
> >
> > Regarding this:
> >
> >
> > --
> > Amos Elberg
> > Sent with Airmail
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> > Date: December 1, 2015 at 6:48:47 PM
> >
> > To: dev@zeppelin.incubator.apache.org <dev@zeppelin.incubator.apache.org
> >
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull request: R Interpreter for Zeppelin
> >
> > On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com>
> wrote:
> >
> >> I am going to try to minimize my reaction to Moon’s e-mail.
> >>
> >> The tl;dr is this:
> >>
> >> The reason we are having this discussion now is that active users of the
> >> PR — which now has its own user base — went public to complain about
> this.
> >
> >
> >> The PR has been tested by an active user base for more than three
> months.
> >> No-one has been able to identify any specific actual licensing problem,
> and
> >> the PR was prepared based on an extensive, careful review of the
> relevant
> >> licensing issues and after contacting the relevant people.
> >>
> >>
> >
> > I admire every software that used by user and helping people. That
> includes
> > your work. But that's not the topic we're in discussion. Active user does
> > not mean your contribution can ignore the review.
> >
> >
> >
> >> It is not an explanation for someone who has been ignoring my “how can I
> >> move this forward…” emails for three months to point the finger and say
> I
> >> didn’t contact the right person or file the right report.
> >>
> >>
> > This is also not the topic in this discussion.
> >
> >
> >> The burden for providing an explanation for the inaction is on the PMCC
> at
> >> this point.
> >
> > I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> >> core, why do you think other PRs are passing CI?
> >> They’re not! I often see comments on PRs to just ignore that CI is
> >> failing.
> >>
> >> One of the most common reasons this PR fails CI, is CI times-out
> >> downloading Spark to install. How could that possibly be caused by the
> PR?
> >>
> >> It looks to me like the only PRs with changes to the relevant parts of
> the
> >> code — the SparkInterpreter — are being made by the person who wrote the
> >> testing suite.
> >>
> >> So, that would explain why some other PRs pass CI: Neither the
> >> SparkInterpreter nor the testing suite are stable or robust, but since
> the
> >> PRs are coming from the person who wrote both…
> >>
> >> And let's say Zeppelin core has problem and that makes your PR fails on
> CI
> >> test. That's possible. But it still does not mean we can merge it with
> CI
> >> failing.
> >>
> >> It means you should be working with me to figure out why the CI is
> failing.
> >>
> >> This PR has been tested by an active user base for the past three
> months.
> >> If CI is continuing to fail, and dozens of hours of effort have not
> >> resolved the CI issues, then it is time to start considering whether the
> >> testing suite is part of the problem.
> >>
> >> The level of defensiveness about the CI and SparkInterpreter is not
> >> helping to resolve these issues.
> >>
> >> If you think it's problem on Zeppelin core, then file an issue that
> >> reproduce the problem on Zeppelin core, that might be more efficient
> than
> >> keep trying yourself.
> >> I contacted you numerous times about such issues...
> >>
> >
> >
> > I remember i commented your issue about CI. but you just keep repeated
> it's
> > not your problem but Zeppelin core problem.
> >
> > Then please file an issue about the problem you found in Zeppelin Core.
> > Then everyone will get into the problem.
> >
> >
> >
> >>
> >> In my interpretation, KnitRInterpreter is not an optional feature while
> it
> >> is always enabled when compiling Zeppelin and always enabled when
> running
> >> Zeppelin. And it requires dynamically linked GPL library on runtime.
> (yes
> >> it will fail when no KnitR is installed on the system)
> >>
> >> Its not always enabled.
> >> It is not dynamically linked at runtime.
> >> It will not fail when knitr is missing. If knitr is not present, the
> repl
> >> interpreter starts and a note is written to to the log that the knitr
> >> interpreter isn’t available because knitr is not present.
> >>
> >> no Apache code can ever call a shell script, on the purpose of dynamic
> >> linking with GPL library.
> >> You misunderstand.
> >>
> >> The *shell* is GPL'd.
> >>
> >> Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends
> on a
> >> shell script to launch?
> >>
> > Obviously not.
> >>
> >> The interaction with R in the PR is the same.
> >>
> >>
> >
> > Again, bash is one of exceptions of GPL, like other GPL licensed
> > compiler/interpreter.
> >
> > Check here why Bash and R is okay with Apache License.
> > https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
> >
> > I'm not sure we can apply the same exception for 'using' KnitR.
> >
> > My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter'
> you
> > wrote is not an optional feature. Which is clearly not optionally enabled
> > code and feature. And that depends on KnitR library which is GPL.
> >
> >
> >
> >> I was guessing SparkR can be still in Apache License even if it is
> depends
> >> on R. Because of GPL licensed compiler generated output is not GPL
> license.
> >> and R is sort of compiler. If you can get answer from Spark community
> how
> >> SparkR get managed to stay in Apache License while R is GPL, the answer
> >> might help.
> >> The description of SparkR is not accurate in any respect. (Do you think
> >> SparkR is not talking to GPL-licensed libraries?)
> >>
> >> I don’t see that any genuine issue is being raised here.
> >>
> >> If there is an issue, the burden is on you to identify it.
> >>
> >> If i give you one suggestion, Zeppelin committers sometimes ask rebase
> the
> >> contribution branch for some reason. It is not the really the best
> >> practice, but still okay while most contributions are not including
> large
> >> code base changes
> >> However, your one, has more than 4000 lines of code change. If you
> rebase
> >> then review should be started from the beginning, again. So you might
> want
> >> to minimize the rebase your branch.
> >>
> >> Are you actually complaining that the problem is that I rebased the code
> >> during the three-month period when no-one looked at it and Zeppelin went
> >> through a release?
> >>
> >> I cannot take it seriously when you say things like this.
> >>
> >> Having to “start from the beginning” cannot be a problem if you never
> >> actually started the first time...
> >>
> >>
> >
> > You wanted coordination and cooperation. So i gave you suggestion that
> > helping review process. For example, your code has been rebased since my
> > comment and jongyoul's comment. that means committers will need to look
> > from the beginning. That'll require more time. And maybe, i guess that's
> > not what you want. But If you don't care, feel free to rebase.
> >
> > Thanks,
> > moon
> >
> >
> >
> >>
> >> From: moon soo Lee <mo...@apache.org>
> >> Reply: moon soo Lee <mo...@apache.org>
> >> Date: December 1, 2015 at 4:42:06 AM
> >> To: Amos B. Elberg <am...@gmail.com>,
> >> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> >> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull
> >> request: R Interpreter for Zeppelin
> >>
> >> On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
> >> wrote:
> >> Thank you, Cos.
> >>
> >> I’d like to briefly address the issues raised by Moon:
> >>
> >> 1. This PR does not passes CI
> >> The CI fails on core Zeppelin, *not* code in this PR.
> >>
> >> I’ve been seeking assistance on this since August.
> >>
> >> The most common reason is that SparkInterpreter is unable to launch
> Spark
> >> and open a Spark Backend. This is necessary to test the PR.
> >>
> >> 60+ hours, has been spent adapting and re-basing when the
> SparkInterpreter
> >> architecture changed and broke the PR’s *tests.*
> >>
> >>
> >> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> >> core, why do you think other PRs are passing CI?
> >>
> >> And let's say Zeppelin core has problem and that makes your PR fails on
> CI
> >> test. That's possible. But it still does not mean we can merge it with
> CI
> >> failing.
> >>
> >> If you think it's problem on Zeppelin core, then file an issue that
> >> reproduce the problem on Zeppelin core, that might be more efficient
> than
> >> keep trying yourself.
> >>
> >>
> >> 2. Not 100% sure this PR has no license issue. (about KniteR)
> >> What license problem *specifically* do you believe may exist?
> >>
> >> In preparing the PR, I:
> >>
> >> * Reviewed the Apache policy in detail.
> >>
> >> * Contacted the FSF to ask their interpretation of the “linking”
> >> provisions of the GPL license.
> >>
> >> * Reviewed how other Apache software deals with this issue (e.g., Spark
> >> talks to R, which is GPL'd).
> >>
> >> * No necessary *dependencies* of the PR have license conflicts. In
> >> several cases, I contacted package authors who agreed to re-issue their
> >> packages under Apache-compatible licenses. (Usually I had to do a bit of
> >> coding in exchange…)
> >>
> >> * Where the license had to stay GPL, the packages are *not necessary*
> and
> >> *not dependencies.* If the optional packages are present, they will be
> >> used to enable additional functionality. Knitr is an example. The PR
> will
> >> compile and run fine without knitr. If knitr is available (it is part of
> >> the most common R distribution), the PR will enable the knitr
> interpreter.
> >>
> >> * This is exactly how this issue is addressed through the Apache
> >> ecosystem.
> >> The tl;dr is this: When Apache code is written to talk to libraries that
> >> may or may not be present on the user’s system, or where it talks to an
> API
> >> but is agnostic about implementation, that is not “linking” in a way
> that
> >> implicate the anti-linking provision of the GPL.
> >>
> >> Otherwise, no Apache code could ever call a shell script! Let alone run
> >> on Linux, or talk to R.
> >>
> >>
> >> I'm not a legal expert. So following could be wrong.
> >>
> >> In my interpretation, KnitRInterpreter is not an optional feature while
> it
> >> is always enabled when compiling Zeppelin and always enabled when
> running
> >> Zeppelin. And it requires dynamically linked GPL library on runtime.
> (yes
> >> it will fail when no KnitR is installed on the system)
> >>
> >> And of course, no Apache code can ever call a shell script, on the
> purpose
> >> of dynamic linking with GPL library.
> >>
> >> I was guessing SparkR can be still in Apache License even if it is
> depends
> >> on R. Because of GPL licensed compiler generated output is not GPL
> license.
> >> and R is sort of compiler.
> >>
> >> If you can get answer from Spark community how SparkR get managed to
> stay
> >> in Apache License while R is GPL, the answer might help.
> >>
> >>
> >> 3. Need more time to review.
> >> Has any reviewer has downloaded the PR or run the demo notebook? (Which
> >> is there for the benefit of reviewers, and isn’t intended to go in final
> >> distribution.)
> >>
> >> How many +1 comments from users would you like to see?
> >>
> >> How much time do you believe is required?
> >>
> >>
> >> It all depends on when CI is going to pass, when license problem is
> going
> >> to be cleared, and when a committer willing to review and responsible to
> >> commit your contribution.
> >>
> >>
> >> 1. Large code base change
> >> Large code base changes require coordination and cooperation. This PR
> >> necessarily implicates the build scripts, testing code, the
> >> SparkInterpreter, etc.
> >>
> >> I have been seeking to coordinate since August.
> >>
> >> I continue to stand ready to do so.
> >>
> >> -Amos
> >>
> >>
> >> If i give you one suggestion, Zeppelin committers sometimes ask rebase
> the
> >> contribution branch for some reason. It is not the really the best
> >> practice, but still okay while most contributions are not including
> large
> >> code base changes.
> >>
> >> However, your one, has more than 4000 lines of code change. If you
> rebase
> >> then review should be started from the beginning, again. So you might
> want
> >> to minimize the rebase your branch.
> >>
> >> Thanks,
> >> moon
> >>
> >>
> >> From: moon soo Lee <mo...@apache.org>
> >> Reply: dev@zeppelin.incubator.apache.org <
> >> dev@zeppelin.incubator.apache.org>
> >> Date: December 1, 2015 at 1:34:19 AM
> >> To: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> >> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull
> >> request: R Interpreter for Zeppelin
> >>
> >> Hi Cos,
> >>
> >> Thanks for opening a discussion.
> >> My answer about 'Why this PR is open for three months' is
> >>
> >> 1. This PR does not passes CI
> >> 2. Not 100% sure this PR has no license issue. (about KniteR)
> >> 3. Need more time to review.
> >>
> >> It's my personal answer, other committers may have different opinion.
> >>
> >>
> >> I think the question should be generalized. Because this PR is not the
> only
> >> PR that is in impasse. There're more. For example
> >>
> >> Here's some examples that PR is not been merged.
> >> https://github.com/apache/incubator-zeppelin/pull/53,
> >> https://github.com/apache/incubator-zeppelin/pull/60
> >> and so on.
> >>
> >> I can categorize the cases, based on experience of involving Zeppelin
> >> community from the beginning.
> >>
> >> 1. Large code base change
> >>
> >> When contribution has large code base changes, it tend to take more
> time to
> >> review and merged. Normally, most contributions merged in 1day~1 week.
> >> But some contribution has large code base changes take 2~4 weeks. Few
> >> contribution that has very large code base change take months.
> >>
> >> 2. Communication lost
> >>
> >> Sometimes, committer is not responding, or contributor is not
> responding.
> >>
> >> 3. Opinion diverges
> >>
> >> For some changes, included in contribution. When people have different
> >> opinion and it continue to diverges, those PRs are not been merged.
> >>
> >>
> >> I think having a guide such as ping other committer when a committer is
> not
> >> responding, and divide contribution into small peaces if possible, would
> >> help most of the cases.
> >> Of course committer have to pay attention more to the contribution and
> >> helping people. That's the first one.
> >>
> >> Thanks,
> >> moon
> >>
> >> On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org>
> wrote:
> >>
> >> > To make sure we're on the same page, here are two threads that I found
> >> > related
> >> > to this topic.
> >> >
> >> > Thread 1:
> >> > Subject: R?
> >> > Started on: July 1, 2015
> >> >
> >> > Thread 2:
> >> > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> >> > Zeppelin
> >> > Started on: August 13, 2015
> >> >
> >> > If you want to fetch these from our archive send emails to
> >> > dev-thread.1735@zeppelin.incubator.apache.org
> >> > dev-thread.3573@zeppelin.incubator.apache.org
> >> >
> >> > Cos
> >> >
> >> > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> >> > > Guys,
> >> > >
> >> > > While catching up on my emails from the last a couple of weeks, this
> >> > thread
> >> > > caught my attention. I am not normally paying much attention to the
> >> code
> >> > > reviews traffic from GH, but this one is pretty different as it
> spans
> >> > three
> >> > > months and counting.
> >> > >
> >> > > First, here are my five cents:
> >> > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> >> > contributed to
> >> > > an ASF project this file should simply contain ASL2 text, like in
> [1]
> >> > > - r/pom.xml perhaps shouldn't contain a separate <developers>
> section,
> >> > but
> >> > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> >> > > maintain this in the source code, but we have the list of all the
> >> > > committers on the project's site [2] Every new committer's first
> >> > commit is
> >> > > to update the team page ;)
> >> > > - comments like in
> >> > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> >> > >
> >> > > +/**
> >> > > + * Created by aelberg on 7/28/15.
> >> > > + */
> >> > >
> >> > > is better to be removed. It has been already discussed here [3]. And
> >> > the
> >> > > initial discussion on the topic [4] was linked as well
> >> > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> >> > R-specific
> >> > > stuff - I have no idea about R, honestly, but
> >> > >
> >> > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> >> > > ...
> >> > > +Author: David B. Dahl
> >> > >
> >> > > shouldn't be here, IMO. Normally, if some additional licenses are
> >> > used,
> >> > > they have to be listed in the top-level NOTICE file (already there).
> >> > >
> >> > > - I am not going to offer any comments on the technical merits of
> the
> >> > patch,
> >> > > as I haven't tried it personally. However, I don't see any serious
> >> > > technical objections to the functionality in question.
> >> > >
> >> > > So, the question is - why the PR is open for three months? I hasn't
> >> been
> >> > able
> >> > > to get a clear answer. What I found out though is pretty unsettling,
> >> > really.
> >> > > The communication on the topic is almost non-existent, except for
> this
> >> > sparse
> >> > > and bitter thread in the GH.
> >> > >
> >> > > I would love to hear from the committers what's stopping the
> acceptance
> >> > of the
> >> > > code, besides of the minor issues I've mentioned earlier? What are
> the
> >> > reasons for it?
> >> > > Is there anything wrong with it?
> >> > > One of the responsibilities of the committers is to make sure the
> >> > > contributions are reviewed; new contributors are guided and do
> >> > understand how
> >> > > the project ticks. The easy feedback flow attracts new people,
> allowing
> >> > the
> >> > > community to grow and thrive.
> >> > >
> >> > > I am asking _explicitely_ not to re-start the bickering I have
> already
> >> > > seen. At this point I am interested in the purely technical side of
> >> this.
> >> > >
> >> > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> >> > > [2] http://bigtop.apache.org/team-list.html
> >> > > [3]
> >> >
> >>
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> >> > > [4] http://s.apache.org/iZl
> >> > >
> >> > > With regards,
> >> > > Cos
> >> > >
> >> > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> >> > > > Github user elbamos commented on the pull request:
> >> > > >
> >> > > >
> >> >
> >>
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> >> > > >
> >> > > > The current push should resolve some issues with changes in the
> >> > > > Spark-Zeppelin interface that had created issues for users, as
> >> > well as
> >> > > > support for 1.5.1.
> >> > > >
> >> > > > Knitr support is improved, and the reason for a separate knitr
> >> > interpreter may be clearer now.
> >> > > >
> >> > > > The amount of code borrowed from rscala is reduced.
> >> > > >
> >> > > > I did not address issues with @author tags, or files under the R/
> >> > > > folder. The reason is, to be blunt, I don't understand what the
> >> > precise
> >> > > > concerns actually are.
> >> > > >
> >> > > > Please note that I changed .travis.yml to only use spark 1.4 and
> >> > later.
> >> > > > I'm sure there is a better way to do it, but I'm just not in a
> >> > position
> >> > > > to try to figure out the entire Zeppelin build process.
> >> > > >
> >> > > > Integrating this with the rest of zeppelin is going to take some
> >> > work
> >> > > > regarding pom's, travis, and so forth. I can do a lot of that,
> >> > but I'm
> >> > > > going to need to discuss it with the people who have been "owning"
> >> > those
> >> > > > files. There are just too many moving pieces here.
> >> > > >
> >> > > > Because of the size of this (which is, unfortunately, necessary),
> >> > > > posting here is probably not the most efficient way. That is also
> >> > true
> >> > > > because certain people chose to use this PR as a forum to air
> other
> >> > > > issues. Therefore, it would be better if reviewers e-mail me
> >> > directly.
> >> >
> >> >
> >> >
> >>
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Roman - thank you for taking a look at the PR!  The PR should compile out-of-box if you have R and the evaluate package installed.  

I have been working on a “release” version rebased to match 5.5-release.  It has documentation, including what features are added with optional (incompatible-license) R packages.  And how to build a seamless Spark pipeline that mixes scala, python, and R without breaking lazy evaluation. 

I can send you a zip of that tomorrow, and will contact you directly to provide it.  

From: Roman Shaposhnik <ro...@shaposhnik.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 2, 2015 at 12:30:52 AM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
CC: moon soo Lee <mo...@apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

Hi!  

reading through the thread made me realize that what Amos  
is saying makes perfect sense to me. That said, I don't like to  
take Moon's concerns lightly either.  

Hence, would it be possible to get the *bundle* of a binary  
package built with the patch applied and ready to go? I'd  
like to provide an extra pair of eyes to help settle this.  

Thanks,  
Roman.  

P.S. I do a lot of this type of work around IP at my $DAYJOB  

On Tue, Dec 1, 2015 at 5:25 PM, Amos B. Elberg <am...@gmail.com> wrote:  
> Moon - If there were seriously a licensing issue, then you or someone else would be able to say clearly and specifically what it is.  
>  
> There plainly is not. This idea you have about a “compiler exception” has nothing to do with any of this. The link you sent doesn’t talk about any “compiler exception.” (I’m not sure what a 2008 e-mail from a developer who wanted to release a branded version of R has to do with Apache’s policy concerning linking to GPL’d code anyway…)  
>  
> My e-mail was accidentally sent incomplete. This is the rest:  
>  
> You say this:  
>  
> My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter' you  
> wrote is not an optional feature. Which is clearly not optionally enabled  
> code and feature. And that depends on KnitR library which is GPL.  
>  
> That’s false.  
>  
> ***  
> I addressed the licensing issue only because waving the “license” flag could scare people off.  
>  
> If someone really believed there was a licensing issue, they would be able to say clearly what that issue is.  
>  
> I am not going to respond to the rest of Moon’s e-mail.  
>  
>  
> From: moon soo Lee <mo...@apache.org>  
> Reply: moon soo Lee <mo...@apache.org>  
> Date: December 1, 2015 at 8:16:36 PM  
> To: Amos B. Elberg <am...@gmail.com>, dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  
>  
> Is knitR is commonly considered as a interpreter/compiler? or is it considered as a library routine?  
>  
> Thanks,  
> moon  
>  
> On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com> wrote:  
> Moon - you give this as an explanation of the licensing issue: https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
>  
> According to that, there is an exception in the GPL for interpreter languages. As long as you don’t distribute the code, its fine to talk to an interpreted language.  
>  
> Well, if that’s the case, then the PR plainly does not have a license issue. It doesn’t distribute any GPL’d R code.  
>  
> I’m not sure what’s confusing about this. It seems completely straightforward.  
>  
> Regarding this:  
>  
>  
> --  
> Amos Elberg  
> Sent with Airmail  
>  
> From: moon soo Lee <mo...@apache.org>  
> Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> Date: December 1, 2015 at 6:48:47 PM  
>  
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  
>  
> On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com> wrote:  
>  
>> I am going to try to minimize my reaction to Moon’s e-mail.  
>>  
>> The tl;dr is this:  
>>  
>> The reason we are having this discussion now is that active users of the  
>> PR — which now has its own user base — went public to complain about this.  
>  
>  
>> The PR has been tested by an active user base for more than three months.  
>> No-one has been able to identify any specific actual licensing problem, and  
>> the PR was prepared based on an extensive, careful review of the relevant  
>> licensing issues and after contacting the relevant people.  
>>  
>>  
>  
> I admire every software that used by user and helping people. That includes  
> your work. But that's not the topic we're in discussion. Active user does  
> not mean your contribution can ignore the review.  
>  
>  
>  
>> It is not an explanation for someone who has been ignoring my “how can I  
>> move this forward…” emails for three months to point the finger and say I  
>> didn’t contact the right person or file the right report.  
>>  
>>  
> This is also not the topic in this discussion.  
>  
>  
>> The burden for providing an explanation for the inaction is on the PMCC at  
>> this point.  
>  
> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin  
>> core, why do you think other PRs are passing CI?  
>> They’re not! I often see comments on PRs to just ignore that CI is  
>> failing.  
>>  
>> One of the most common reasons this PR fails CI, is CI times-out  
>> downloading Spark to install. How could that possibly be caused by the PR?  
>>  
>> It looks to me like the only PRs with changes to the relevant parts of the  
>> code — the SparkInterpreter — are being made by the person who wrote the  
>> testing suite.  
>>  
>> So, that would explain why some other PRs pass CI: Neither the  
>> SparkInterpreter nor the testing suite are stable or robust, but since the  
>> PRs are coming from the person who wrote both…  
>>  
>> And let's say Zeppelin core has problem and that makes your PR fails on CI  
>> test. That's possible. But it still does not mean we can merge it with CI  
>> failing.  
>>  
>> It means you should be working with me to figure out why the CI is failing.  
>>  
>> This PR has been tested by an active user base for the past three months.  
>> If CI is continuing to fail, and dozens of hours of effort have not  
>> resolved the CI issues, then it is time to start considering whether the  
>> testing suite is part of the problem.  
>>  
>> The level of defensiveness about the CI and SparkInterpreter is not  
>> helping to resolve these issues.  
>>  
>> If you think it's problem on Zeppelin core, then file an issue that  
>> reproduce the problem on Zeppelin core, that might be more efficient than  
>> keep trying yourself.  
>> I contacted you numerous times about such issues...  
>>  
>  
>  
> I remember i commented your issue about CI. but you just keep repeated it's  
> not your problem but Zeppelin core problem.  
>  
> Then please file an issue about the problem you found in Zeppelin Core.  
> Then everyone will get into the problem.  
>  
>  
>  
>>  
>> In my interpretation, KnitRInterpreter is not an optional feature while it  
>> is always enabled when compiling Zeppelin and always enabled when running  
>> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes  
>> it will fail when no KnitR is installed on the system)  
>>  
>> Its not always enabled.  
>> It is not dynamically linked at runtime.  
>> It will not fail when knitr is missing. If knitr is not present, the repl  
>> interpreter starts and a note is written to to the log that the knitr  
>> interpreter isn’t available because knitr is not present.  
>>  
>> no Apache code can ever call a shell script, on the purpose of dynamic  
>> linking with GPL library.  
>> You misunderstand.  
>>  
>> The *shell* is GPL'd.  
>>  
>> Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends on a  
>> shell script to launch?  
>>  
> Obviously not.  
>>  
>> The interaction with R in the PR is the same.  
>>  
>>  
>  
> Again, bash is one of exceptions of GPL, like other GPL licensed  
> compiler/interpreter.  
>  
> Check here why Bash and R is okay with Apache License.  
> https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  
>  
> I'm not sure we can apply the same exception for 'using' KnitR.  
>  
> My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter' you  
> wrote is not an optional feature. Which is clearly not optionally enabled  
> code and feature. And that depends on KnitR library which is GPL.  
>  
>  
>  
>> I was guessing SparkR can be still in Apache License even if it is depends  
>> on R. Because of GPL licensed compiler generated output is not GPL license.  
>> and R is sort of compiler. If you can get answer from Spark community how  
>> SparkR get managed to stay in Apache License while R is GPL, the answer  
>> might help.  
>> The description of SparkR is not accurate in any respect. (Do you think  
>> SparkR is not talking to GPL-licensed libraries?)  
>>  
>> I don’t see that any genuine issue is being raised here.  
>>  
>> If there is an issue, the burden is on you to identify it.  
>>  
>> If i give you one suggestion, Zeppelin committers sometimes ask rebase the  
>> contribution branch for some reason. It is not the really the best  
>> practice, but still okay while most contributions are not including large  
>> code base changes  
>> However, your one, has more than 4000 lines of code change. If you rebase  
>> then review should be started from the beginning, again. So you might want  
>> to minimize the rebase your branch.  
>>  
>> Are you actually complaining that the problem is that I rebased the code  
>> during the three-month period when no-one looked at it and Zeppelin went  
>> through a release?  
>>  
>> I cannot take it seriously when you say things like this.  
>>  
>> Having to “start from the beginning” cannot be a problem if you never  
>> actually started the first time...  
>>  
>>  
>  
> You wanted coordination and cooperation. So i gave you suggestion that  
> helping review process. For example, your code has been rebased since my  
> comment and jongyoul's comment. that means committers will need to look  
> from the beginning. That'll require more time. And maybe, i guess that's  
> not what you want. But If you don't care, feel free to rebase.  
>  
> Thanks,  
> moon  
>  
>  
>  
>>  
>> From: moon soo Lee <mo...@apache.org>  
>> Reply: moon soo Lee <mo...@apache.org>  
>> Date: December 1, 2015 at 4:42:06 AM  
>> To: Amos B. Elberg <am...@gmail.com>,  
>> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
>> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
>> request: R Interpreter for Zeppelin  
>>  
>> On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>  
>> wrote:  
>> Thank you, Cos.  
>>  
>> I’d like to briefly address the issues raised by Moon:  
>>  
>> 1. This PR does not passes CI  
>> The CI fails on core Zeppelin, *not* code in this PR.  
>>  
>> I’ve been seeking assistance on this since August.  
>>  
>> The most common reason is that SparkInterpreter is unable to launch Spark  
>> and open a Spark Backend. This is necessary to test the PR.  
>>  
>> 60+ hours, has been spent adapting and re-basing when the SparkInterpreter  
>> architecture changed and broke the PR’s *tests.*  
>>  
>>  
>> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin  
>> core, why do you think other PRs are passing CI?  
>>  
>> And let's say Zeppelin core has problem and that makes your PR fails on CI  
>> test. That's possible. But it still does not mean we can merge it with CI  
>> failing.  
>>  
>> If you think it's problem on Zeppelin core, then file an issue that  
>> reproduce the problem on Zeppelin core, that might be more efficient than  
>> keep trying yourself.  
>>  
>>  
>> 2. Not 100% sure this PR has no license issue. (about KniteR)  
>> What license problem *specifically* do you believe may exist?  
>>  
>> In preparing the PR, I:  
>>  
>> * Reviewed the Apache policy in detail.  
>>  
>> * Contacted the FSF to ask their interpretation of the “linking”  
>> provisions of the GPL license.  
>>  
>> * Reviewed how other Apache software deals with this issue (e.g., Spark  
>> talks to R, which is GPL'd).  
>>  
>> * No necessary *dependencies* of the PR have license conflicts. In  
>> several cases, I contacted package authors who agreed to re-issue their  
>> packages under Apache-compatible licenses. (Usually I had to do a bit of  
>> coding in exchange…)  
>>  
>> * Where the license had to stay GPL, the packages are *not necessary* and  
>> *not dependencies.* If the optional packages are present, they will be  
>> used to enable additional functionality. Knitr is an example. The PR will  
>> compile and run fine without knitr. If knitr is available (it is part of  
>> the most common R distribution), the PR will enable the knitr interpreter.  
>>  
>> * This is exactly how this issue is addressed through the Apache  
>> ecosystem.  
>> The tl;dr is this: When Apache code is written to talk to libraries that  
>> may or may not be present on the user’s system, or where it talks to an API  
>> but is agnostic about implementation, that is not “linking” in a way that  
>> implicate the anti-linking provision of the GPL.  
>>  
>> Otherwise, no Apache code could ever call a shell script! Let alone run  
>> on Linux, or talk to R.  
>>  
>>  
>> I'm not a legal expert. So following could be wrong.  
>>  
>> In my interpretation, KnitRInterpreter is not an optional feature while it  
>> is always enabled when compiling Zeppelin and always enabled when running  
>> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes  
>> it will fail when no KnitR is installed on the system)  
>>  
>> And of course, no Apache code can ever call a shell script, on the purpose  
>> of dynamic linking with GPL library.  
>>  
>> I was guessing SparkR can be still in Apache License even if it is depends  
>> on R. Because of GPL licensed compiler generated output is not GPL license.  
>> and R is sort of compiler.  
>>  
>> If you can get answer from Spark community how SparkR get managed to stay  
>> in Apache License while R is GPL, the answer might help.  
>>  
>>  
>> 3. Need more time to review.  
>> Has any reviewer has downloaded the PR or run the demo notebook? (Which  
>> is there for the benefit of reviewers, and isn’t intended to go in final  
>> distribution.)  
>>  
>> How many +1 comments from users would you like to see?  
>>  
>> How much time do you believe is required?  
>>  
>>  
>> It all depends on when CI is going to pass, when license problem is going  
>> to be cleared, and when a committer willing to review and responsible to  
>> commit your contribution.  
>>  
>>  
>> 1. Large code base change  
>> Large code base changes require coordination and cooperation. This PR  
>> necessarily implicates the build scripts, testing code, the  
>> SparkInterpreter, etc.  
>>  
>> I have been seeking to coordinate since August.  
>>  
>> I continue to stand ready to do so.  
>>  
>> -Amos  
>>  
>>  
>> If i give you one suggestion, Zeppelin committers sometimes ask rebase the  
>> contribution branch for some reason. It is not the really the best  
>> practice, but still okay while most contributions are not including large  
>> code base changes.  
>>  
>> However, your one, has more than 4000 lines of code change. If you rebase  
>> then review should be started from the beginning, again. So you might want  
>> to minimize the rebase your branch.  
>>  
>> Thanks,  
>> moon  
>>  
>>  
>> From: moon soo Lee <mo...@apache.org>  
>> Reply: dev@zeppelin.incubator.apache.org <  
>> dev@zeppelin.incubator.apache.org>  
>> Date: December 1, 2015 at 1:34:19 AM  
>> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
>> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
>> request: R Interpreter for Zeppelin  
>>  
>> Hi Cos,  
>>  
>> Thanks for opening a discussion.  
>> My answer about 'Why this PR is open for three months' is  
>>  
>> 1. This PR does not passes CI  
>> 2. Not 100% sure this PR has no license issue. (about KniteR)  
>> 3. Need more time to review.  
>>  
>> It's my personal answer, other committers may have different opinion.  
>>  
>>  
>> I think the question should be generalized. Because this PR is not the only  
>> PR that is in impasse. There're more. For example  
>>  
>> Here's some examples that PR is not been merged.  
>> https://github.com/apache/incubator-zeppelin/pull/53,  
>> https://github.com/apache/incubator-zeppelin/pull/60  
>> and so on.  
>>  
>> I can categorize the cases, based on experience of involving Zeppelin  
>> community from the beginning.  
>>  
>> 1. Large code base change  
>>  
>> When contribution has large code base changes, it tend to take more time to  
>> review and merged. Normally, most contributions merged in 1day~1 week.  
>> But some contribution has large code base changes take 2~4 weeks. Few  
>> contribution that has very large code base change take months.  
>>  
>> 2. Communication lost  
>>  
>> Sometimes, committer is not responding, or contributor is not responding.  
>>  
>> 3. Opinion diverges  
>>  
>> For some changes, included in contribution. When people have different  
>> opinion and it continue to diverges, those PRs are not been merged.  
>>  
>>  
>> I think having a guide such as ping other committer when a committer is not  
>> responding, and divide contribution into small peaces if possible, would  
>> help most of the cases.  
>> Of course committer have to pay attention more to the contribution and  
>> helping people. That's the first one.  
>>  
>> Thanks,  
>> moon  
>>  
>> On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org> wrote:  
>>  
>> > To make sure we're on the same page, here are two threads that I found  
>> > related  
>> > to this topic.  
>> >  
>> > Thread 1:  
>> > Subject: R?  
>> > Started on: July 1, 2015  
>> >  
>> > Thread 2:  
>> > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for  
>> > Zeppelin  
>> > Started on: August 13, 2015  
>> >  
>> > If you want to fetch these from our archive send emails to  
>> > dev-thread.1735@zeppelin.incubator.apache.org  
>> > dev-thread.3573@zeppelin.incubator.apache.org  
>> >  
>> > Cos  
>> >  
>> > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:  
>> > > Guys,  
>> > >  
>> > > While catching up on my emails from the last a couple of weeks, this  
>> > thread  
>> > > caught my attention. I am not normally paying much attention to the  
>> code  
>> > > reviews traffic from GH, but this one is pretty different as it spans  
>> > three  
>> > > months and counting.  
>> > >  
>> > > First, here are my five cents:  
>> > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be  
>> > contributed to  
>> > > an ASF project this file should simply contain ASL2 text, like in [1]  
>> > > - r/pom.xml perhaps shouldn't contain a separate <developers> section,  
>> > but  
>> > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't  
>> > > maintain this in the source code, but we have the list of all the  
>> > > committers on the project's site [2] Every new committer's first  
>> > commit is  
>> > > to update the team page ;)  
>> > > - comments like in  
>> > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java  
>> > >  
>> > > +/**  
>> > > + * Created by aelberg on 7/28/15.  
>> > > + */  
>> > >  
>> > > is better to be removed. It has been already discussed here [3]. And  
>> > the  
>> > > initial discussion on the topic [4] was linked as well  
>> > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is  
>> > R-specific  
>> > > stuff - I have no idea about R, honestly, but  
>> > >  
>> > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE  
>> > > ...  
>> > > +Author: David B. Dahl  
>> > >  
>> > > shouldn't be here, IMO. Normally, if some additional licenses are  
>> > used,  
>> > > they have to be listed in the top-level NOTICE file (already there).  
>> > >  
>> > > - I am not going to offer any comments on the technical merits of the  
>> > patch,  
>> > > as I haven't tried it personally. However, I don't see any serious  
>> > > technical objections to the functionality in question.  
>> > >  
>> > > So, the question is - why the PR is open for three months? I hasn't  
>> been  
>> > able  
>> > > to get a clear answer. What I found out though is pretty unsettling,  
>> > really.  
>> > > The communication on the topic is almost non-existent, except for this  
>> > sparse  
>> > > and bitter thread in the GH.  
>> > >  
>> > > I would love to hear from the committers what's stopping the acceptance  
>> > of the  
>> > > code, besides of the minor issues I've mentioned earlier? What are the  
>> > reasons for it?  
>> > > Is there anything wrong with it?  
>> > > One of the responsibilities of the committers is to make sure the  
>> > > contributions are reviewed; new contributors are guided and do  
>> > understand how  
>> > > the project ticks. The easy feedback flow attracts new people, allowing  
>> > the  
>> > > community to grow and thrive.  
>> > >  
>> > > I am asking _explicitely_ not to re-start the bickering I have already  
>> > > seen. At this point I am interested in the purely technical side of  
>> this.  
>> > >  
>> > > [1] https://github.com/apache/bigtop/blob/master/LICENSE  
>> > > [2] http://bigtop.apache.org/team-list.html  
>> > > [3]  
>> >  
>> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html  
>> > > [4] http://s.apache.org/iZl  
>> > >  
>> > > With regards,  
>> > > Cos  
>> > >  
>> > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:  
>> > > > Github user elbamos commented on the pull request:  
>> > > >  
>> > > >  
>> >  
>> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411  
>> > > >  
>> > > > The current push should resolve some issues with changes in the  
>> > > > Spark-Zeppelin interface that had created issues for users, as  
>> > well as  
>> > > > support for 1.5.1.  
>> > > >  
>> > > > Knitr support is improved, and the reason for a separate knitr  
>> > interpreter may be clearer now.  
>> > > >  
>> > > > The amount of code borrowed from rscala is reduced.  
>> > > >  
>> > > > I did not address issues with @author tags, or files under the R/  
>> > > > folder. The reason is, to be blunt, I don't understand what the  
>> > precise  
>> > > > concerns actually are.  
>> > > >  
>> > > > Please note that I changed .travis.yml to only use spark 1.4 and  
>> > later.  
>> > > > I'm sure there is a better way to do it, but I'm just not in a  
>> > position  
>> > > > to try to figure out the entire Zeppelin build process.  
>> > > >  
>> > > > Integrating this with the rest of zeppelin is going to take some  
>> > work  
>> > > > regarding pom's, travis, and so forth. I can do a lot of that,  
>> > but I'm  
>> > > > going to need to discuss it with the people who have been "owning"  
>> > those  
>> > > > files. There are just too many moving pieces here.  
>> > > >  
>> > > > Because of the size of this (which is, unfortunately, necessary),  
>> > > > posting here is probably not the most efficient way. That is also  
>> > true  
>> > > > because certain people chose to use this PR as a forum to air other  
>> > > > issues. Therefore, it would be better if reviewers e-mail me  
>> > directly.  
>> >  
>> >  
>> >  
>>  

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Roman Shaposhnik <ro...@shaposhnik.org>]

Hi!

reading through the thread made me realize that what Amos
is saying makes perfect sense to me. That said, I don't like to
take Moon's concerns lightly either.

Hence, would it be possible to get the *bundle* of a binary
package built with the patch applied and ready to go? I'd
like to provide an extra pair of eyes to help settle this.

Thanks,
Roman.

P.S. I do a lot of this type of work around IP at my $DAYJOB

On Tue, Dec 1, 2015 at 5:25 PM, Amos B. Elberg <am...@gmail.com> wrote:
> Moon - If there were seriously a licensing issue, then you or someone else would be able to say clearly and specifically what it is.
>
> There plainly is not.  This idea you have about a “compiler exception” has nothing to do with any of this. The link you sent doesn’t talk about any “compiler exception.”  (I’m not sure what a 2008 e-mail from a developer who wanted to release a branded version of R has to do with Apache’s policy concerning linking to GPL’d code anyway…)
>
> My e-mail was accidentally sent incomplete.  This is the rest:
>
> You say this:
>
> My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter' you
> wrote is not an optional feature. Which is clearly not optionally enabled
> code and feature. And that depends on KnitR library which is GPL.
>
> That’s false.
>
> ***
> I addressed the licensing issue only because waving the “license” flag could scare people off.
>
> If someone really believed there was a licensing issue, they would be able to say clearly what that issue is.
>
> I am not going to respond to the rest of Moon’s e-mail.
>
>
> From: moon soo Lee <mo...@apache.org>
> Reply: moon soo Lee <mo...@apache.org>
> Date: December 1, 2015 at 8:16:36 PM
> To: Amos B. Elberg <am...@gmail.com>, dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin
>
> Is knitR is commonly considered as a interpreter/compiler? or is it considered as a library routine?
>
> Thanks,
> moon
>
> On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com> wrote:
> Moon - you give this as an explanation of the licensing issue:  https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
>
> According to that, there is an exception in the GPL for interpreter languages.  As long as you don’t distribute the code, its fine to talk to an interpreted language.
>
> Well, if that’s the case, then the PR plainly does not have a license issue.  It doesn’t distribute any GPL’d R code.
>
> I’m not sure what’s confusing about this.  It seems completely straightforward.
>
> Regarding this:
>
>
> --
> Amos Elberg
> Sent with Airmail
>
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Date: December 1, 2015 at 6:48:47 PM
>
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin
>
> On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com> wrote:
>
>> I am going to try to minimize my reaction to Moon’s e-mail.
>>
>> The tl;dr is this:
>>
>> The reason we are having this discussion now is that active users of the
>> PR — which now has its own user base — went public to complain about this.
>
>
>> The PR has been tested by an active user base for more than three months.
>> No-one has been able to identify any specific actual licensing problem, and
>> the PR was prepared based on an extensive, careful review of the relevant
>> licensing issues and after contacting the relevant people.
>>
>>
>
> I admire every software that used by user and helping people. That includes
> your work. But that's not the topic we're in discussion. Active user does
> not mean your contribution can ignore the review.
>
>
>
>> It is not an explanation for someone who has been ignoring my “how can I
>> move this forward…” emails for three months to point the finger and say I
>> didn’t contact the right person or file the right report.
>>
>>
> This is also not the topic in this discussion.
>
>
>> The burden for providing an explanation for the inaction is on the PMCC at
>> this point.
>
> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
>> core, why do you think other PRs are passing CI?
>> They’re not! I often see comments on PRs to just ignore that CI is
>> failing.
>>
>> One of the most common reasons this PR fails CI, is CI times-out
>> downloading Spark to install. How could that possibly be caused by the PR?
>>
>> It looks to me like the only PRs with changes to the relevant parts of the
>> code — the SparkInterpreter — are being made by the person who wrote the
>> testing suite.
>>
>> So, that would explain why some other PRs pass CI: Neither the
>> SparkInterpreter nor the testing suite are stable or robust, but since the
>> PRs are coming from the person who wrote both…
>>
>> And let's say Zeppelin core has problem and that makes your PR fails on CI
>> test. That's possible. But it still does not mean we can merge it with CI
>> failing.
>>
>> It means you should be working with me to figure out why the CI is failing.
>>
>> This PR has been tested by an active user base for the past three months.
>> If CI is continuing to fail, and dozens of hours of effort have not
>> resolved the CI issues, then it is time to start considering whether the
>> testing suite is part of the problem.
>>
>> The level of defensiveness about the CI and SparkInterpreter is not
>> helping to resolve these issues.
>>
>> If you think it's problem on Zeppelin core, then file an issue that
>> reproduce the problem on Zeppelin core, that might be more efficient than
>> keep trying yourself.
>> I contacted you numerous times about such issues...
>>
>
>
> I remember i commented your issue about CI. but you just keep repeated it's
> not your problem but Zeppelin core problem.
>
> Then please file an issue about the problem you found in Zeppelin Core.
> Then everyone will get into the problem.
>
>
>
>>
>> In my interpretation, KnitRInterpreter is not an optional feature while it
>> is always enabled when compiling Zeppelin and always enabled when running
>> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
>> it will fail when no KnitR is installed on the system)
>>
>> Its not always enabled.
>> It is not dynamically linked at runtime.
>> It will not fail when knitr is missing. If knitr is not present, the repl
>> interpreter starts and a note is written to to the log that the knitr
>> interpreter isn’t available because knitr is not present.
>>
>> no Apache code can ever call a shell script, on the purpose of dynamic
>> linking with GPL library.
>> You misunderstand.
>>
>> The *shell* is GPL'd.
>>
>> Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends on a
>> shell script to launch?
>>
> Obviously not.
>>
>> The interaction with R in the PR is the same.
>>
>>
>
> Again, bash is one of exceptions of GPL, like other GPL licensed
> compiler/interpreter.
>
> Check here why Bash and R is okay with Apache License.
> https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
>
> I'm not sure we can apply the same exception for 'using' KnitR.
>
> My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter' you
> wrote is not an optional feature. Which is clearly not optionally enabled
> code and feature. And that depends on KnitR library which is GPL.
>
>
>
>> I was guessing SparkR can be still in Apache License even if it is depends
>> on R. Because of GPL licensed compiler generated output is not GPL license.
>> and R is sort of compiler. If you can get answer from Spark community how
>> SparkR get managed to stay in Apache License while R is GPL, the answer
>> might help.
>> The description of SparkR is not accurate in any respect. (Do you think
>> SparkR is not talking to GPL-licensed libraries?)
>>
>> I don’t see that any genuine issue is being raised here.
>>
>> If there is an issue, the burden is on you to identify it.
>>
>> If i give you one suggestion, Zeppelin committers sometimes ask rebase the
>> contribution branch for some reason. It is not the really the best
>> practice, but still okay while most contributions are not including large
>> code base changes
>> However, your one, has more than 4000 lines of code change. If you rebase
>> then review should be started from the beginning, again. So you might want
>> to minimize the rebase your branch.
>>
>> Are you actually complaining that the problem is that I rebased the code
>> during the three-month period when no-one looked at it and Zeppelin went
>> through a release?
>>
>> I cannot take it seriously when you say things like this.
>>
>> Having to “start from the beginning” cannot be a problem if you never
>> actually started the first time...
>>
>>
>
> You wanted coordination and cooperation. So i gave you suggestion that
> helping review process. For example, your code has been rebased since my
> comment and jongyoul's comment. that means committers will need to look
> from the beginning. That'll require more time. And maybe, i guess that's
> not what you want. But If you don't care, feel free to rebase.
>
> Thanks,
> moon
>
>
>
>>
>> From: moon soo Lee <mo...@apache.org>
>> Reply: moon soo Lee <mo...@apache.org>
>> Date: December 1, 2015 at 4:42:06 AM
>> To: Amos B. Elberg <am...@gmail.com>,
>> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
>> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
>> request: R Interpreter for Zeppelin
>>
>> On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
>> wrote:
>> Thank you, Cos.
>>
>> I’d like to briefly address the issues raised by Moon:
>>
>> 1. This PR does not passes CI
>> The CI fails on core Zeppelin, *not* code in this PR.
>>
>> I’ve been seeking assistance on this since August.
>>
>> The most common reason is that SparkInterpreter is unable to launch Spark
>> and open a Spark Backend. This is necessary to test the PR.
>>
>> 60+ hours, has been spent adapting and re-basing when the SparkInterpreter
>> architecture changed and broke the PR’s *tests.*
>>
>>
>> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
>> core, why do you think other PRs are passing CI?
>>
>> And let's say Zeppelin core has problem and that makes your PR fails on CI
>> test. That's possible. But it still does not mean we can merge it with CI
>> failing.
>>
>> If you think it's problem on Zeppelin core, then file an issue that
>> reproduce the problem on Zeppelin core, that might be more efficient than
>> keep trying yourself.
>>
>>
>> 2. Not 100% sure this PR has no license issue. (about KniteR)
>> What license problem *specifically* do you believe may exist?
>>
>> In preparing the PR, I:
>>
>> * Reviewed the Apache policy in detail.
>>
>> * Contacted the FSF to ask their interpretation of the “linking”
>> provisions of the GPL license.
>>
>> * Reviewed how other Apache software deals with this issue (e.g., Spark
>> talks to R, which is GPL'd).
>>
>> * No necessary *dependencies* of the PR have license conflicts. In
>> several cases, I contacted package authors who agreed to re-issue their
>> packages under Apache-compatible licenses. (Usually I had to do a bit of
>> coding in exchange…)
>>
>> * Where the license had to stay GPL, the packages are *not necessary* and
>> *not dependencies.* If the optional packages are present, they will be
>> used to enable additional functionality. Knitr is an example. The PR will
>> compile and run fine without knitr. If knitr is available (it is part of
>> the most common R distribution), the PR will enable the knitr interpreter.
>>
>> * This is exactly how this issue is addressed through the Apache
>> ecosystem.
>> The tl;dr is this: When Apache code is written to talk to libraries that
>> may or may not be present on the user’s system, or where it talks to an API
>> but is agnostic about implementation, that is not “linking” in a way that
>> implicate the anti-linking provision of the GPL.
>>
>> Otherwise, no Apache code could ever call a shell script! Let alone run
>> on Linux, or talk to R.
>>
>>
>> I'm not a legal expert. So following could be wrong.
>>
>> In my interpretation, KnitRInterpreter is not an optional feature while it
>> is always enabled when compiling Zeppelin and always enabled when running
>> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
>> it will fail when no KnitR is installed on the system)
>>
>> And of course, no Apache code can ever call a shell script, on the purpose
>> of dynamic linking with GPL library.
>>
>> I was guessing SparkR can be still in Apache License even if it is depends
>> on R. Because of GPL licensed compiler generated output is not GPL license.
>> and R is sort of compiler.
>>
>> If you can get answer from Spark community how SparkR get managed to stay
>> in Apache License while R is GPL, the answer might help.
>>
>>
>> 3. Need more time to review.
>> Has any reviewer has downloaded the PR or run the demo notebook? (Which
>> is there for the benefit of reviewers, and isn’t intended to go in final
>> distribution.)
>>
>> How many +1 comments from users would you like to see?
>>
>> How much time do you believe is required?
>>
>>
>> It all depends on when CI is going to pass, when license problem is going
>> to be cleared, and when a committer willing to review and responsible to
>> commit your contribution.
>>
>>
>> 1. Large code base change
>> Large code base changes require coordination and cooperation. This PR
>> necessarily implicates the build scripts, testing code, the
>> SparkInterpreter, etc.
>>
>> I have been seeking to coordinate since August.
>>
>> I continue to stand ready to do so.
>>
>> -Amos
>>
>>
>> If i give you one suggestion, Zeppelin committers sometimes ask rebase the
>> contribution branch for some reason. It is not the really the best
>> practice, but still okay while most contributions are not including large
>> code base changes.
>>
>> However, your one, has more than 4000 lines of code change. If you rebase
>> then review should be started from the beginning, again. So you might want
>> to minimize the rebase your branch.
>>
>> Thanks,
>> moon
>>
>>
>> From: moon soo Lee <mo...@apache.org>
>> Reply: dev@zeppelin.incubator.apache.org <
>> dev@zeppelin.incubator.apache.org>
>> Date: December 1, 2015 at 1:34:19 AM
>> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
>> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
>> request: R Interpreter for Zeppelin
>>
>> Hi Cos,
>>
>> Thanks for opening a discussion.
>> My answer about 'Why this PR is open for three months' is
>>
>> 1. This PR does not passes CI
>> 2. Not 100% sure this PR has no license issue. (about KniteR)
>> 3. Need more time to review.
>>
>> It's my personal answer, other committers may have different opinion.
>>
>>
>> I think the question should be generalized. Because this PR is not the only
>> PR that is in impasse. There're more. For example
>>
>> Here's some examples that PR is not been merged.
>> https://github.com/apache/incubator-zeppelin/pull/53,
>> https://github.com/apache/incubator-zeppelin/pull/60
>> and so on.
>>
>> I can categorize the cases, based on experience of involving Zeppelin
>> community from the beginning.
>>
>> 1. Large code base change
>>
>> When contribution has large code base changes, it tend to take more time to
>> review and merged. Normally, most contributions merged in 1day~1 week.
>> But some contribution has large code base changes take 2~4 weeks. Few
>> contribution that has very large code base change take months.
>>
>> 2. Communication lost
>>
>> Sometimes, committer is not responding, or contributor is not responding.
>>
>> 3. Opinion diverges
>>
>> For some changes, included in contribution. When people have different
>> opinion and it continue to diverges, those PRs are not been merged.
>>
>>
>> I think having a guide such as ping other committer when a committer is not
>> responding, and divide contribution into small peaces if possible, would
>> help most of the cases.
>> Of course committer have to pay attention more to the contribution and
>> helping people. That's the first one.
>>
>> Thanks,
>> moon
>>
>> On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org> wrote:
>>
>> > To make sure we're on the same page, here are two threads that I found
>> > related
>> > to this topic.
>> >
>> > Thread 1:
>> > Subject: R?
>> > Started on: July 1, 2015
>> >
>> > Thread 2:
>> > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
>> > Zeppelin
>> > Started on: August 13, 2015
>> >
>> > If you want to fetch these from our archive send emails to
>> > dev-thread.1735@zeppelin.incubator.apache.org
>> > dev-thread.3573@zeppelin.incubator.apache.org
>> >
>> > Cos
>> >
>> > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
>> > > Guys,
>> > >
>> > > While catching up on my emails from the last a couple of weeks, this
>> > thread
>> > > caught my attention. I am not normally paying much attention to the
>> code
>> > > reviews traffic from GH, but this one is pretty different as it spans
>> > three
>> > > months and counting.
>> > >
>> > > First, here are my five cents:
>> > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
>> > contributed to
>> > > an ASF project this file should simply contain ASL2 text, like in [1]
>> > > - r/pom.xml perhaps shouldn't contain a separate <developers> section,
>> > but
>> > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
>> > > maintain this in the source code, but we have the list of all the
>> > > committers on the project's site [2] Every new committer's first
>> > commit is
>> > > to update the team page ;)
>> > > - comments like in
>> > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
>> > >
>> > > +/**
>> > > + * Created by aelberg on 7/28/15.
>> > > + */
>> > >
>> > > is better to be removed. It has been already discussed here [3]. And
>> > the
>> > > initial discussion on the topic [4] was linked as well
>> > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
>> > R-specific
>> > > stuff - I have no idea about R, honestly, but
>> > >
>> > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
>> > > ...
>> > > +Author: David B. Dahl
>> > >
>> > > shouldn't be here, IMO. Normally, if some additional licenses are
>> > used,
>> > > they have to be listed in the top-level NOTICE file (already there).
>> > >
>> > > - I am not going to offer any comments on the technical merits of the
>> > patch,
>> > > as I haven't tried it personally. However, I don't see any serious
>> > > technical objections to the functionality in question.
>> > >
>> > > So, the question is - why the PR is open for three months? I hasn't
>> been
>> > able
>> > > to get a clear answer. What I found out though is pretty unsettling,
>> > really.
>> > > The communication on the topic is almost non-existent, except for this
>> > sparse
>> > > and bitter thread in the GH.
>> > >
>> > > I would love to hear from the committers what's stopping the acceptance
>> > of the
>> > > code, besides of the minor issues I've mentioned earlier? What are the
>> > reasons for it?
>> > > Is there anything wrong with it?
>> > > One of the responsibilities of the committers is to make sure the
>> > > contributions are reviewed; new contributors are guided and do
>> > understand how
>> > > the project ticks. The easy feedback flow attracts new people, allowing
>> > the
>> > > community to grow and thrive.
>> > >
>> > > I am asking _explicitely_ not to re-start the bickering I have already
>> > > seen. At this point I am interested in the purely technical side of
>> this.
>> > >
>> > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
>> > > [2] http://bigtop.apache.org/team-list.html
>> > > [3]
>> >
>> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
>> > > [4] http://s.apache.org/iZl
>> > >
>> > > With regards,
>> > > Cos
>> > >
>> > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
>> > > > Github user elbamos commented on the pull request:
>> > > >
>> > > >
>> >
>> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
>> > > >
>> > > > The current push should resolve some issues with changes in the
>> > > > Spark-Zeppelin interface that had created issues for users, as
>> > well as
>> > > > support for 1.5.1.
>> > > >
>> > > > Knitr support is improved, and the reason for a separate knitr
>> > interpreter may be clearer now.
>> > > >
>> > > > The amount of code borrowed from rscala is reduced.
>> > > >
>> > > > I did not address issues with @author tags, or files under the R/
>> > > > folder. The reason is, to be blunt, I don't understand what the
>> > precise
>> > > > concerns actually are.
>> > > >
>> > > > Please note that I changed .travis.yml to only use spark 1.4 and
>> > later.
>> > > > I'm sure there is a better way to do it, but I'm just not in a
>> > position
>> > > > to try to figure out the entire Zeppelin build process.
>> > > >
>> > > > Integrating this with the rest of zeppelin is going to take some
>> > work
>> > > > regarding pom's, travis, and so forth. I can do a lot of that,
>> > but I'm
>> > > > going to need to discuss it with the people who have been "owning"
>> > those
>> > > > files. There are just too many moving pieces here.
>> > > >
>> > > > Because of the size of this (which is, unfortunately, necessary),
>> > > > posting here is probably not the most efficient way. That is also
>> > true
>> > > > because certain people chose to use this PR as a forum to air other
>> > > > issues. Therefore, it would be better if reviewers e-mail me
>> > directly.
>> >
>> >
>> >
>>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Moon - If there were seriously a licensing issue, then you or someone else would be able to say clearly and specifically what it is. 

There plainly is not.  This idea you have about a “compiler exception” has nothing to do with any of this. The link you sent doesn’t talk about any “compiler exception.”  (I’m not sure what a 2008 e-mail from a developer who wanted to release a branded version of R has to do with Apache’s policy concerning linking to GPL’d code anyway…)

My e-mail was accidentally sent incomplete.  This is the rest:

You say this: 

My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter' you 
wrote is not an optional feature. Which is clearly not optionally enabled 
code and feature. And that depends on KnitR library which is GPL. 

That’s false. 

***
I addressed the licensing issue only because waving the “license” flag could scare people off.  

If someone really believed there was a licensing issue, they would be able to say clearly what that issue is.  

I am not going to respond to the rest of Moon’s e-mail.


From: moon soo Lee <mo...@apache.org>
Reply: moon soo Lee <mo...@apache.org>
Date: December 1, 2015 at 8:16:36 PM
To: Amos B. Elberg <am...@gmail.com>, dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

Is knitR is commonly considered as a interpreter/compiler? or is it considered as a library routine?

Thanks,
moon

On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com> wrote:
Moon - you give this as an explanation of the licensing issue:  https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html 

According to that, there is an exception in the GPL for interpreter languages.  As long as you don’t distribute the code, its fine to talk to an interpreted language. 

Well, if that’s the case, then the PR plainly does not have a license issue.  It doesn’t distribute any GPL’d R code. 

I’m not sure what’s confusing about this.  It seems completely straightforward. 

Regarding this:


-- 
Amos Elberg
Sent with Airmail

From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 1, 2015 at 6:48:47 PM

To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com> wrote:

> I am going to try to minimize my reaction to Moon’s e-mail.
>
> The tl;dr is this:
>
> The reason we are having this discussion now is that active users of the
> PR — which now has its own user base — went public to complain about this.


> The PR has been tested by an active user base for more than three months.
> No-one has been able to identify any specific actual licensing problem, and
> the PR was prepared based on an extensive, careful review of the relevant
> licensing issues and after contacting the relevant people.
>
>

I admire every software that used by user and helping people. That includes
your work. But that's not the topic we're in discussion. Active user does
not mean your contribution can ignore the review.



> It is not an explanation for someone who has been ignoring my “how can I
> move this forward…” emails for three months to point the finger and say I
> didn’t contact the right person or file the right report.
>
>
This is also not the topic in this discussion.


> The burden for providing an explanation for the inaction is on the PMCC at
> this point.

I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> core, why do you think other PRs are passing CI?
> They’re not! I often see comments on PRs to just ignore that CI is
> failing.
>
> One of the most common reasons this PR fails CI, is CI times-out
> downloading Spark to install. How could that possibly be caused by the PR?
>
> It looks to me like the only PRs with changes to the relevant parts of the
> code — the SparkInterpreter — are being made by the person who wrote the
> testing suite.
>
> So, that would explain why some other PRs pass CI: Neither the
> SparkInterpreter nor the testing suite are stable or robust, but since the
> PRs are coming from the person who wrote both…
>
> And let's say Zeppelin core has problem and that makes your PR fails on CI
> test. That's possible. But it still does not mean we can merge it with CI
> failing.
>
> It means you should be working with me to figure out why the CI is failing.
>
> This PR has been tested by an active user base for the past three months.
> If CI is continuing to fail, and dozens of hours of effort have not
> resolved the CI issues, then it is time to start considering whether the
> testing suite is part of the problem.
>
> The level of defensiveness about the CI and SparkInterpreter is not
> helping to resolve these issues.
>
> If you think it's problem on Zeppelin core, then file an issue that
> reproduce the problem on Zeppelin core, that might be more efficient than
> keep trying yourself.
> I contacted you numerous times about such issues...
>


I remember i commented your issue about CI. but you just keep repeated it's
not your problem but Zeppelin core problem.

Then please file an issue about the problem you found in Zeppelin Core.
Then everyone will get into the problem.



>
> In my interpretation, KnitRInterpreter is not an optional feature while it
> is always enabled when compiling Zeppelin and always enabled when running
> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
> it will fail when no KnitR is installed on the system)
>
> Its not always enabled.
> It is not dynamically linked at runtime.
> It will not fail when knitr is missing. If knitr is not present, the repl
> interpreter starts and a note is written to to the log that the knitr
> interpreter isn’t available because knitr is not present.
>
> no Apache code can ever call a shell script, on the purpose of dynamic
> linking with GPL library.
> You misunderstand.
>
> The *shell* is GPL'd.
>
> Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends on a
> shell script to launch?
>
Obviously not.
>
> The interaction with R in the PR is the same.
>
>

Again, bash is one of exceptions of GPL, like other GPL licensed
compiler/interpreter.

Check here why Bash and R is okay with Apache License.
https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html

I'm not sure we can apply the same exception for 'using' KnitR.

My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter' you
wrote is not an optional feature. Which is clearly not optionally enabled
code and feature. And that depends on KnitR library which is GPL.



> I was guessing SparkR can be still in Apache License even if it is depends
> on R. Because of GPL licensed compiler generated output is not GPL license.
> and R is sort of compiler. If you can get answer from Spark community how
> SparkR get managed to stay in Apache License while R is GPL, the answer
> might help.
> The description of SparkR is not accurate in any respect. (Do you think
> SparkR is not talking to GPL-licensed libraries?)
>
> I don’t see that any genuine issue is being raised here.
>
> If there is an issue, the burden is on you to identify it.
>
> If i give you one suggestion, Zeppelin committers sometimes ask rebase the
> contribution branch for some reason. It is not the really the best
> practice, but still okay while most contributions are not including large
> code base changes
> However, your one, has more than 4000 lines of code change. If you rebase
> then review should be started from the beginning, again. So you might want
> to minimize the rebase your branch.
>
> Are you actually complaining that the problem is that I rebased the code
> during the three-month period when no-one looked at it and Zeppelin went
> through a release?
>
> I cannot take it seriously when you say things like this.
>
> Having to “start from the beginning” cannot be a problem if you never
> actually started the first time...
>
>

You wanted coordination and cooperation. So i gave you suggestion that
helping review process. For example, your code has been rebased since my
comment and jongyoul's comment. that means committers will need to look
from the beginning. That'll require more time. And maybe, i guess that's
not what you want. But If you don't care, feel free to rebase.

Thanks,
moon



>
> From: moon soo Lee <mo...@apache.org>
> Reply: moon soo Lee <mo...@apache.org>
> Date: December 1, 2015 at 4:42:06 AM
> To: Amos B. Elberg <am...@gmail.com>,
> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
> wrote:
> Thank you, Cos.
>
> I’d like to briefly address the issues raised by Moon:
>
> 1. This PR does not passes CI
> The CI fails on core Zeppelin, *not* code in this PR.
>
> I’ve been seeking assistance on this since August.
>
> The most common reason is that SparkInterpreter is unable to launch Spark
> and open a Spark Backend. This is necessary to test the PR.
>
> 60+ hours, has been spent adapting and re-basing when the SparkInterpreter
> architecture changed and broke the PR’s *tests.*
>
>
> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> core, why do you think other PRs are passing CI?
>
> And let's say Zeppelin core has problem and that makes your PR fails on CI
> test. That's possible. But it still does not mean we can merge it with CI
> failing.
>
> If you think it's problem on Zeppelin core, then file an issue that
> reproduce the problem on Zeppelin core, that might be more efficient than
> keep trying yourself.
>
>
> 2. Not 100% sure this PR has no license issue. (about KniteR)
> What license problem *specifically* do you believe may exist?
>
> In preparing the PR, I:
>
> * Reviewed the Apache policy in detail.
>
> * Contacted the FSF to ask their interpretation of the “linking”
> provisions of the GPL license.
>
> * Reviewed how other Apache software deals with this issue (e.g., Spark
> talks to R, which is GPL'd).
>
> * No necessary *dependencies* of the PR have license conflicts. In
> several cases, I contacted package authors who agreed to re-issue their
> packages under Apache-compatible licenses. (Usually I had to do a bit of
> coding in exchange…)
>
> * Where the license had to stay GPL, the packages are *not necessary* and
> *not dependencies.* If the optional packages are present, they will be
> used to enable additional functionality. Knitr is an example. The PR will
> compile and run fine without knitr. If knitr is available (it is part of
> the most common R distribution), the PR will enable the knitr interpreter.
>
> * This is exactly how this issue is addressed through the Apache
> ecosystem.
> The tl;dr is this: When Apache code is written to talk to libraries that
> may or may not be present on the user’s system, or where it talks to an API
> but is agnostic about implementation, that is not “linking” in a way that
> implicate the anti-linking provision of the GPL.
>
> Otherwise, no Apache code could ever call a shell script! Let alone run
> on Linux, or talk to R.
>
>
> I'm not a legal expert. So following could be wrong.
>
> In my interpretation, KnitRInterpreter is not an optional feature while it
> is always enabled when compiling Zeppelin and always enabled when running
> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
> it will fail when no KnitR is installed on the system)
>
> And of course, no Apache code can ever call a shell script, on the purpose
> of dynamic linking with GPL library.
>
> I was guessing SparkR can be still in Apache License even if it is depends
> on R. Because of GPL licensed compiler generated output is not GPL license.
> and R is sort of compiler.
>
> If you can get answer from Spark community how SparkR get managed to stay
> in Apache License while R is GPL, the answer might help.
>
>
> 3. Need more time to review.
> Has any reviewer has downloaded the PR or run the demo notebook? (Which
> is there for the benefit of reviewers, and isn’t intended to go in final
> distribution.)
>
> How many +1 comments from users would you like to see?
>
> How much time do you believe is required?
>
>
> It all depends on when CI is going to pass, when license problem is going
> to be cleared, and when a committer willing to review and responsible to
> commit your contribution.
>
>
> 1. Large code base change
> Large code base changes require coordination and cooperation. This PR
> necessarily implicates the build scripts, testing code, the
> SparkInterpreter, etc.
>
> I have been seeking to coordinate since August.
>
> I continue to stand ready to do so.
>
> -Amos
>
>
> If i give you one suggestion, Zeppelin committers sometimes ask rebase the
> contribution branch for some reason. It is not the really the best
> practice, but still okay while most contributions are not including large
> code base changes.
>
> However, your one, has more than 4000 lines of code change. If you rebase
> then review should be started from the beginning, again. So you might want
> to minimize the rebase your branch.
>
> Thanks,
> moon
>
>
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> Date: December 1, 2015 at 1:34:19 AM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> Hi Cos,
>
> Thanks for opening a discussion.
> My answer about 'Why this PR is open for three months' is
>
> 1. This PR does not passes CI
> 2. Not 100% sure this PR has no license issue. (about KniteR)
> 3. Need more time to review.
>
> It's my personal answer, other committers may have different opinion.
>
>
> I think the question should be generalized. Because this PR is not the only
> PR that is in impasse. There're more. For example
>
> Here's some examples that PR is not been merged.
> https://github.com/apache/incubator-zeppelin/pull/53,
> https://github.com/apache/incubator-zeppelin/pull/60
> and so on.
>
> I can categorize the cases, based on experience of involving Zeppelin
> community from the beginning.
>
> 1. Large code base change
>
> When contribution has large code base changes, it tend to take more time to
> review and merged. Normally, most contributions merged in 1day~1 week.
> But some contribution has large code base changes take 2~4 weeks. Few
> contribution that has very large code base change take months.
>
> 2. Communication lost
>
> Sometimes, committer is not responding, or contributor is not responding.
>
> 3. Opinion diverges
>
> For some changes, included in contribution. When people have different
> opinion and it continue to diverges, those PRs are not been merged.
>
>
> I think having a guide such as ping other committer when a committer is not
> responding, and divide contribution into small peaces if possible, would
> help most of the cases.
> Of course committer have to pay attention more to the contribution and
> helping people. That's the first one.
>
> Thanks,
> moon
>
> On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org> wrote:
>
> > To make sure we're on the same page, here are two threads that I found
> > related
> > to this topic.
> >
> > Thread 1:
> > Subject: R?
> > Started on: July 1, 2015
> >
> > Thread 2:
> > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > Zeppelin
> > Started on: August 13, 2015
> >
> > If you want to fetch these from our archive send emails to
> > dev-thread.1735@zeppelin.incubator.apache.org
> > dev-thread.3573@zeppelin.incubator.apache.org
> >
> > Cos
> >
> > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > Guys,
> > >
> > > While catching up on my emails from the last a couple of weeks, this
> > thread
> > > caught my attention. I am not normally paying much attention to the
> code
> > > reviews traffic from GH, but this one is pretty different as it spans
> > three
> > > months and counting.
> > >
> > > First, here are my five cents:
> > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > contributed to
> > > an ASF project this file should simply contain ASL2 text, like in [1]
> > > - r/pom.xml perhaps shouldn't contain a separate <developers> section,
> > but
> > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > > maintain this in the source code, but we have the list of all the
> > > committers on the project's site [2] Every new committer's first
> > commit is
> > > to update the team page ;)
> > > - comments like in
> > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > >
> > > +/**
> > > + * Created by aelberg on 7/28/15.
> > > + */
> > >
> > > is better to be removed. It has been already discussed here [3]. And
> > the
> > > initial discussion on the topic [4] was linked as well
> > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> > R-specific
> > > stuff - I have no idea about R, honestly, but
> > >
> > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > ...
> > > +Author: David B. Dahl
> > >
> > > shouldn't be here, IMO. Normally, if some additional licenses are
> > used,
> > > they have to be listed in the top-level NOTICE file (already there).
> > >
> > > - I am not going to offer any comments on the technical merits of the
> > patch,
> > > as I haven't tried it personally. However, I don't see any serious
> > > technical objections to the functionality in question.
> > >
> > > So, the question is - why the PR is open for three months? I hasn't
> been
> > able
> > > to get a clear answer. What I found out though is pretty unsettling,
> > really.
> > > The communication on the topic is almost non-existent, except for this
> > sparse
> > > and bitter thread in the GH.
> > >
> > > I would love to hear from the committers what's stopping the acceptance
> > of the
> > > code, besides of the minor issues I've mentioned earlier? What are the
> > reasons for it?
> > > Is there anything wrong with it?
> > > One of the responsibilities of the committers is to make sure the
> > > contributions are reviewed; new contributors are guided and do
> > understand how
> > > the project ticks. The easy feedback flow attracts new people, allowing
> > the
> > > community to grow and thrive.
> > >
> > > I am asking _explicitely_ not to re-start the bickering I have already
> > > seen. At this point I am interested in the purely technical side of
> this.
> > >
> > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > [2] http://bigtop.apache.org/team-list.html
> > > [3]
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > [4] http://s.apache.org/iZl
> > >
> > > With regards,
> > > Cos
> > >
> > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > Github user elbamos commented on the pull request:
> > > >
> > > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > >
> > > > The current push should resolve some issues with changes in the
> > > > Spark-Zeppelin interface that had created issues for users, as
> > well as
> > > > support for 1.5.1.
> > > >
> > > > Knitr support is improved, and the reason for a separate knitr
> > interpreter may be clearer now.
> > > >
> > > > The amount of code borrowed from rscala is reduced.
> > > >
> > > > I did not address issues with @author tags, or files under the R/
> > > > folder. The reason is, to be blunt, I don't understand what the
> > precise
> > > > concerns actually are.
> > > >
> > > > Please note that I changed .travis.yml to only use spark 1.4 and
> > later.
> > > > I'm sure there is a better way to do it, but I'm just not in a
> > position
> > > > to try to figure out the entire Zeppelin build process.
> > > >
> > > > Integrating this with the rest of zeppelin is going to take some
> > work
> > > > regarding pom's, travis, and so forth. I can do a lot of that,
> > but I'm
> > > > going to need to discuss it with the people who have been "owning"
> > those
> > > > files. There are just too many moving pieces here.
> > > >
> > > > Because of the size of this (which is, unfortunately, necessary),
> > > > posting here is probably not the most efficient way. That is also
> > true
> > > > because certain people chose to use this PR as a forum to air other
> > > > issues. Therefore, it would be better if reviewers e-mail me
> > directly.
> >
> >
> >
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Is knitR is commonly considered as a interpreter/compiler? or is it
considered as a library routine?

Thanks,
moon

On Wed, Dec 2, 2015 at 10:12 AM Amos B. Elberg <am...@gmail.com>
wrote:

> Moon - you give this as an explanation of the licensing issue:
> https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
>
> According to that, there is an exception in the GPL for interpreter
> languages.  As long as you don’t distribute the code, its fine to talk to
> an interpreted language.
>
> Well, if that’s the case, then the PR plainly does not have a license
> issue.  It doesn’t distribute any GPL’d R code.
>
> I’m not sure what’s confusing about this.  It seems completely
> straightforward.
>
> Regarding this:
>
>
> --
> Amos Elberg
> Sent with Airmail
>
> From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org
> <de...@zeppelin.incubator.apache.org> <de...@zeppelin.incubator.apache.org>
> Date: December 1, 2015 at 6:48:47 PM
>
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull request: R Interpreter for Zeppelin
>
> On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com>
> wrote:
>
> > I am going to try to minimize my reaction to Moon’s e-mail.
> >
> > The tl;dr is this:
> >
> > The reason we are having this discussion now is that active users of the
> > PR — which now has its own user base — went public to complain about
> this.
>
>
> > The PR has been tested by an active user base for more than three
> months.
> > No-one has been able to identify any specific actual licensing problem,
> and
> > the PR was prepared based on an extensive, careful review of the
> relevant
> > licensing issues and after contacting the relevant people.
> >
> >
>
> I admire every software that used by user and helping people. That
> includes
> your work. But that's not the topic we're in discussion. Active user does
> not mean your contribution can ignore the review.
>
>
>
> > It is not an explanation for someone who has been ignoring my “how can I
> > move this forward…” emails for three months to point the finger and say
> I
> > didn’t contact the right person or file the right report.
> >
> >
> This is also not the topic in this discussion.
>
>
> > The burden for providing an explanation for the inaction is on the PMCC
> at
> > this point.
>
> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> > core, why do you think other PRs are passing CI?
> > They’re not! I often see comments on PRs to just ignore that CI is
> > failing.
> >
> > One of the most common reasons this PR fails CI, is CI times-out
> > downloading Spark to install. How could that possibly be caused by the
> PR?
> >
> > It looks to me like the only PRs with changes to the relevant parts of
> the
> > code — the SparkInterpreter — are being made by the person who wrote the
> > testing suite.
> >
> > So, that would explain why some other PRs pass CI: Neither the
> > SparkInterpreter nor the testing suite are stable or robust, but since
> the
> > PRs are coming from the person who wrote both…
> >
> > And let's say Zeppelin core has problem and that makes your PR fails on
> CI
> > test. That's possible. But it still does not mean we can merge it with
> CI
> > failing.
> >
> > It means you should be working with me to figure out why the CI is
> failing.
> >
> > This PR has been tested by an active user base for the past three
> months.
> > If CI is continuing to fail, and dozens of hours of effort have not
> > resolved the CI issues, then it is time to start considering whether the
> > testing suite is part of the problem.
> >
> > The level of defensiveness about the CI and SparkInterpreter is not
> > helping to resolve these issues.
> >
> > If you think it's problem on Zeppelin core, then file an issue that
> > reproduce the problem on Zeppelin core, that might be more efficient
> than
> > keep trying yourself.
> > I contacted you numerous times about such issues...
> >
>
>
> I remember i commented your issue about CI. but you just keep repeated
> it's
> not your problem but Zeppelin core problem.
>
> Then please file an issue about the problem you found in Zeppelin Core.
> Then everyone will get into the problem.
>
>
>
> >
> > In my interpretation, KnitRInterpreter is not an optional feature while
> it
> > is always enabled when compiling Zeppelin and always enabled when
> running
> > Zeppelin. And it requires dynamically linked GPL library on runtime.
> (yes
> > it will fail when no KnitR is installed on the system)
> >
> > Its not always enabled.
> > It is not dynamically linked at runtime.
> > It will not fail when knitr is missing. If knitr is not present, the
> repl
> > interpreter starts and a note is written to to the log that the knitr
> > interpreter isn’t available because knitr is not present.
> >
> > no Apache code can ever call a shell script, on the purpose of dynamic
> > linking with GPL library.
> > You misunderstand.
> >
> > The *shell* is GPL'd.
> >
> > Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends on
> a
> > shell script to launch?
> >
> Obviously not.
> >
> > The interaction with R in the PR is the same.
> >
> >
>
> Again, bash is one of exceptions of GPL, like other GPL licensed
> compiler/interpreter.
>
> Check here why Bash and R is okay with Apache License.
> https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html
>
> I'm not sure we can apply the same exception for 'using' KnitR.
>
> My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter'
> you
> wrote is not an optional feature. Which is clearly not optionally enabled
> code and feature. And that depends on KnitR library which is GPL.
>
>
>
> > I was guessing SparkR can be still in Apache License even if it is
> depends
> > on R. Because of GPL licensed compiler generated output is not GPL
> license.
> > and R is sort of compiler. If you can get answer from Spark community
> how
> > SparkR get managed to stay in Apache License while R is GPL, the answer
> > might help.
> > The description of SparkR is not accurate in any respect. (Do you think
> > SparkR is not talking to GPL-licensed libraries?)
> >
> > I don’t see that any genuine issue is being raised here.
> >
> > If there is an issue, the burden is on you to identify it.
> >
> > If i give you one suggestion, Zeppelin committers sometimes ask rebase
> the
> > contribution branch for some reason. It is not the really the best
> > practice, but still okay while most contributions are not including
> large
> > code base changes
> > However, your one, has more than 4000 lines of code change. If you
> rebase
> > then review should be started from the beginning, again. So you might
> want
> > to minimize the rebase your branch.
> >
> > Are you actually complaining that the problem is that I rebased the code
> > during the three-month period when no-one looked at it and Zeppelin went
> > through a release?
> >
> > I cannot take it seriously when you say things like this.
> >
> > Having to “start from the beginning” cannot be a problem if you never
> > actually started the first time...
> >
> >
>
> You wanted coordination and cooperation. So i gave you suggestion that
> helping review process. For example, your code has been rebased since my
> comment and jongyoul's comment. that means committers will need to look
> from the beginning. That'll require more time. And maybe, i guess that's
> not what you want. But If you don't care, feel free to rebase.
>
> Thanks,
> moon
>
>
>
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: moon soo Lee <mo...@apache.org>
> > Date: December 1, 2015 at 4:42:06 AM
> > To: Amos B. Elberg <am...@gmail.com>,
> > dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull
> > request: R Interpreter for Zeppelin
> >
> > On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
> > wrote:
> > Thank you, Cos.
> >
> > I’d like to briefly address the issues raised by Moon:
> >
> > 1. This PR does not passes CI
> > The CI fails on core Zeppelin, *not* code in this PR.
> >
> > I’ve been seeking assistance on this since August.
> >
> > The most common reason is that SparkInterpreter is unable to launch
> Spark
> > and open a Spark Backend. This is necessary to test the PR.
> >
> > 60+ hours, has been spent adapting and re-basing when the
> SparkInterpreter
> > architecture changed and broke the PR’s *tests.*
> >
> >
> > I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> > core, why do you think other PRs are passing CI?
> >
> > And let's say Zeppelin core has problem and that makes your PR fails on
> CI
> > test. That's possible. But it still does not mean we can merge it with
> CI
> > failing.
> >
> > If you think it's problem on Zeppelin core, then file an issue that
> > reproduce the problem on Zeppelin core, that might be more efficient
> than
> > keep trying yourself.
> >
> >
> > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > What license problem *specifically* do you believe may exist?
> >
> > In preparing the PR, I:
> >
> > * Reviewed the Apache policy in detail.
> >
> > * Contacted the FSF to ask their interpretation of the “linking”
> > provisions of the GPL license.
> >
> > * Reviewed how other Apache software deals with this issue (e.g., Spark
> > talks to R, which is GPL'd).
> >
> > * No necessary *dependencies* of the PR have license conflicts. In
> > several cases, I contacted package authors who agreed to re-issue their
> > packages under Apache-compatible licenses. (Usually I had to do a bit of
> > coding in exchange…)
> >
> > * Where the license had to stay GPL, the packages are *not necessary*
> and
> > *not dependencies.* If the optional packages are present, they will be
> > used to enable additional functionality. Knitr is an example. The PR
> will
> > compile and run fine without knitr. If knitr is available (it is part of
> > the most common R distribution), the PR will enable the knitr
> interpreter.
> >
> > * This is exactly how this issue is addressed through the Apache
> > ecosystem.
> > The tl;dr is this: When Apache code is written to talk to libraries that
> > may or may not be present on the user’s system, or where it talks to an
> API
> > but is agnostic about implementation, that is not “linking” in a way
> that
> > implicate the anti-linking provision of the GPL.
> >
> > Otherwise, no Apache code could ever call a shell script! Let alone run
> > on Linux, or talk to R.
> >
> >
> > I'm not a legal expert. So following could be wrong.
> >
> > In my interpretation, KnitRInterpreter is not an optional feature while
> it
> > is always enabled when compiling Zeppelin and always enabled when
> running
> > Zeppelin. And it requires dynamically linked GPL library on runtime.
> (yes
> > it will fail when no KnitR is installed on the system)
> >
> > And of course, no Apache code can ever call a shell script, on the
> purpose
> > of dynamic linking with GPL library.
> >
> > I was guessing SparkR can be still in Apache License even if it is
> depends
> > on R. Because of GPL licensed compiler generated output is not GPL
> license.
> > and R is sort of compiler.
> >
> > If you can get answer from Spark community how SparkR get managed to
> stay
> > in Apache License while R is GPL, the answer might help.
> >
> >
> > 3. Need more time to review.
> > Has any reviewer has downloaded the PR or run the demo notebook? (Which
> > is there for the benefit of reviewers, and isn’t intended to go in final
> > distribution.)
> >
> > How many +1 comments from users would you like to see?
> >
> > How much time do you believe is required?
> >
> >
> > It all depends on when CI is going to pass, when license problem is
> going
> > to be cleared, and when a committer willing to review and responsible to
> > commit your contribution.
> >
> >
> > 1. Large code base change
> > Large code base changes require coordination and cooperation. This PR
> > necessarily implicates the build scripts, testing code, the
> > SparkInterpreter, etc.
> >
> > I have been seeking to coordinate since August.
> >
> > I continue to stand ready to do so.
> >
> > -Amos
> >
> >
> > If i give you one suggestion, Zeppelin committers sometimes ask rebase
> the
> > contribution branch for some reason. It is not the really the best
> > practice, but still okay while most contributions are not including
> large
> > code base changes.
> >
> > However, your one, has more than 4000 lines of code change. If you
> rebase
> > then review should be started from the beginning, again. So you might
> want
> > to minimize the rebase your branch.
> >
> > Thanks,
> > moon
> >
> >
> > From: moon soo Lee <mo...@apache.org>
> > Reply: dev@zeppelin.incubator.apache.org <
> > dev@zeppelin.incubator.apache.org>
> > Date: December 1, 2015 at 1:34:19 AM
> > To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
>
> > Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull
> > request: R Interpreter for Zeppelin
> >
> > Hi Cos,
> >
> > Thanks for opening a discussion.
> > My answer about 'Why this PR is open for three months' is
> >
> > 1. This PR does not passes CI
> > 2. Not 100% sure this PR has no license issue. (about KniteR)
> > 3. Need more time to review.
> >
> > It's my personal answer, other committers may have different opinion.
> >
> >
> > I think the question should be generalized. Because this PR is not the
> only
> > PR that is in impasse. There're more. For example
> >
> > Here's some examples that PR is not been merged.
> > https://github.com/apache/incubator-zeppelin/pull/53,
> > https://github.com/apache/incubator-zeppelin/pull/60
> > and so on.
> >
> > I can categorize the cases, based on experience of involving Zeppelin
> > community from the beginning.
> >
> > 1. Large code base change
> >
> > When contribution has large code base changes, it tend to take more time
> to
> > review and merged. Normally, most contributions merged in 1day~1 week.
> > But some contribution has large code base changes take 2~4 weeks. Few
> > contribution that has very large code base change take months.
> >
> > 2. Communication lost
> >
> > Sometimes, committer is not responding, or contributor is not
> responding.
> >
> > 3. Opinion diverges
> >
> > For some changes, included in contribution. When people have different
> > opinion and it continue to diverges, those PRs are not been merged.
> >
> >
> > I think having a guide such as ping other committer when a committer is
> not
> > responding, and divide contribution into small peaces if possible, would
> > help most of the cases.
> > Of course committer have to pay attention more to the contribution and
> > helping people. That's the first one.
> >
> > Thanks,
> > moon
> >
> > On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org>
> wrote:
> >
> > > To make sure we're on the same page, here are two threads that I found
> > > related
> > > to this topic.
> > >
> > > Thread 1:
> > > Subject: R?
> > > Started on: July 1, 2015
> > >
> > > Thread 2:
> > > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > > Zeppelin
> > > Started on: August 13, 2015
> > >
> > > If you want to fetch these from our archive send emails to
> > > dev-thread.1735@zeppelin.incubator.apache.org
> > > dev-thread.3573@zeppelin.incubator.apache.org
> > >
> > > Cos
> > >
> > > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > > Guys,
> > > >
> > > > While catching up on my emails from the last a couple of weeks, this
> > > thread
> > > > caught my attention. I am not normally paying much attention to the
> > code
> > > > reviews traffic from GH, but this one is pretty different as it
> spans
> > > three
> > > > months and counting.
> > > >
> > > > First, here are my five cents:
> > > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > > contributed to
> > > > an ASF project this file should simply contain ASL2 text, like in
> [1]
> > > > - r/pom.xml perhaps shouldn't contain a separate <developers>
> section,
> > > but
> > > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > > > maintain this in the source code, but we have the list of all the
> > > > committers on the project's site [2] Every new committer's first
> > > commit is
> > > > to update the team page ;)
> > > > - comments like in
> > > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > > >
> > > > +/**
> > > > + * Created by aelberg on 7/28/15.
> > > > + */
> > > >
> > > > is better to be removed. It has been already discussed here [3]. And
> > > the
> > > > initial discussion on the topic [4] was linked as well
> > > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> > > R-specific
> > > > stuff - I have no idea about R, honestly, but
> > > >
> > > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > > ...
> > > > +Author: David B. Dahl
> > > >
> > > > shouldn't be here, IMO. Normally, if some additional licenses are
> > > used,
> > > > they have to be listed in the top-level NOTICE file (already there).
> > > >
> > > > - I am not going to offer any comments on the technical merits of
> the
> > > patch,
> > > > as I haven't tried it personally. However, I don't see any serious
> > > > technical objections to the functionality in question.
> > > >
> > > > So, the question is - why the PR is open for three months? I hasn't
> > been
> > > able
> > > > to get a clear answer. What I found out though is pretty unsettling,
> > > really.
> > > > The communication on the topic is almost non-existent, except for
> this
> > > sparse
> > > > and bitter thread in the GH.
> > > >
> > > > I would love to hear from the committers what's stopping the
> acceptance
> > > of the
> > > > code, besides of the minor issues I've mentioned earlier? What are
> the
> > > reasons for it?
> > > > Is there anything wrong with it?
> > > > One of the responsibilities of the committers is to make sure the
> > > > contributions are reviewed; new contributors are guided and do
> > > understand how
> > > > the project ticks. The easy feedback flow attracts new people,
> allowing
> > > the
> > > > community to grow and thrive.
> > > >
> > > > I am asking _explicitely_ not to re-start the bickering I have
> already
> > > > seen. At this point I am interested in the purely technical side of
> > this.
> > > >
> > > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > > [2] http://bigtop.apache.org/team-list.html
> > > > [3]
> > >
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > > [4] http://s.apache.org/iZl
> > > >
> > > > With regards,
> > > > Cos
> > > >
> > > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > > Github user elbamos commented on the pull request:
> > > > >
> > > > >
> > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > > >
> > > > > The current push should resolve some issues with changes in the
> > > > > Spark-Zeppelin interface that had created issues for users, as
> > > well as
> > > > > support for 1.5.1.
> > > > >
> > > > > Knitr support is improved, and the reason for a separate knitr
> > > interpreter may be clearer now.
> > > > >
> > > > > The amount of code borrowed from rscala is reduced.
> > > > >
> > > > > I did not address issues with @author tags, or files under the R/
> > > > > folder. The reason is, to be blunt, I don't understand what the
> > > precise
> > > > > concerns actually are.
> > > > >
> > > > > Please note that I changed .travis.yml to only use spark 1.4 and
> > > later.
> > > > > I'm sure there is a better way to do it, but I'm just not in a
> > > position
> > > > > to try to figure out the entire Zeppelin build process.
> > > > >
> > > > > Integrating this with the rest of zeppelin is going to take some
> > > work
> > > > > regarding pom's, travis, and so forth. I can do a lot of that,
> > > but I'm
> > > > > going to need to discuss it with the people who have been "owning"
> > > those
> > > > > files. There are just too many moving pieces here.
> > > > >
> > > > > Because of the size of this (which is, unfortunately, necessary),
> > > > > posting here is probably not the most efficient way. That is also
> > > true
> > > > > because certain people chose to use this PR as a forum to air
> other
> > > > > issues. Therefore, it would be better if reviewers e-mail me
> > > directly.
> > >
> > >
> > >
> >
>
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Moon - you give this as an explanation of the licensing issue:  https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html 

According to that, there is an exception in the GPL for interpreter languages.  As long as you don’t distribute the code, its fine to talk to an interpreted language. 

Well, if that’s the case, then the PR plainly does not have a license issue.  It doesn’t distribute any GPL’d R code. 

I’m not sure what’s confusing about this.  It seems completely straightforward. 

Regarding this:


-- 
Amos Elberg
Sent with Airmail

From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 1, 2015 at 6:48:47 PM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com> wrote:  

> I am going to try to minimize my reaction to Moon’s e-mail.  
>  
> The tl;dr is this:  
>  
> The reason we are having this discussion now is that active users of the  
> PR — which now has its own user base — went public to complain about this.  


> The PR has been tested by an active user base for more than three months.  
> No-one has been able to identify any specific actual licensing problem, and  
> the PR was prepared based on an extensive, careful review of the relevant  
> licensing issues and after contacting the relevant people.  
>  
>  

I admire every software that used by user and helping people. That includes  
your work. But that's not the topic we're in discussion. Active user does  
not mean your contribution can ignore the review.  



> It is not an explanation for someone who has been ignoring my “how can I  
> move this forward…” emails for three months to point the finger and say I  
> didn’t contact the right person or file the right report.  
>  
>  
This is also not the topic in this discussion.  


> The burden for providing an explanation for the inaction is on the PMCC at  
> this point.  

I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin  
> core, why do you think other PRs are passing CI?  
> They’re not! I often see comments on PRs to just ignore that CI is  
> failing.  
>  
> One of the most common reasons this PR fails CI, is CI times-out  
> downloading Spark to install. How could that possibly be caused by the PR?  
>  
> It looks to me like the only PRs with changes to the relevant parts of the  
> code — the SparkInterpreter — are being made by the person who wrote the  
> testing suite.  
>  
> So, that would explain why some other PRs pass CI: Neither the  
> SparkInterpreter nor the testing suite are stable or robust, but since the  
> PRs are coming from the person who wrote both…  
>  
> And let's say Zeppelin core has problem and that makes your PR fails on CI  
> test. That's possible. But it still does not mean we can merge it with CI  
> failing.  
>  
> It means you should be working with me to figure out why the CI is failing.  
>  
> This PR has been tested by an active user base for the past three months.  
> If CI is continuing to fail, and dozens of hours of effort have not  
> resolved the CI issues, then it is time to start considering whether the  
> testing suite is part of the problem.  
>  
> The level of defensiveness about the CI and SparkInterpreter is not  
> helping to resolve these issues.  
>  
> If you think it's problem on Zeppelin core, then file an issue that  
> reproduce the problem on Zeppelin core, that might be more efficient than  
> keep trying yourself.  
> I contacted you numerous times about such issues...  
>  


I remember i commented your issue about CI. but you just keep repeated it's  
not your problem but Zeppelin core problem.  

Then please file an issue about the problem you found in Zeppelin Core.  
Then everyone will get into the problem.  



>  
> In my interpretation, KnitRInterpreter is not an optional feature while it  
> is always enabled when compiling Zeppelin and always enabled when running  
> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes  
> it will fail when no KnitR is installed on the system)  
>  
> Its not always enabled.  
> It is not dynamically linked at runtime.  
> It will not fail when knitr is missing. If knitr is not present, the repl  
> interpreter starts and a note is written to to the log that the knitr  
> interpreter isn’t available because knitr is not present.  
>  
> no Apache code can ever call a shell script, on the purpose of dynamic  
> linking with GPL library.  
> You misunderstand.  
>  
> The *shell* is GPL'd.  
>  
> Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends on a  
> shell script to launch?  
>  
Obviously not.  
>  
> The interaction with R in the PR is the same.  
>  
>  

Again, bash is one of exceptions of GPL, like other GPL licensed  
compiler/interpreter.  

Check here why Bash and R is okay with Apache License.  
https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html  

I'm not sure we can apply the same exception for 'using' KnitR.  

My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter' you  
wrote is not an optional feature. Which is clearly not optionally enabled  
code and feature. And that depends on KnitR library which is GPL.  



> I was guessing SparkR can be still in Apache License even if it is depends  
> on R. Because of GPL licensed compiler generated output is not GPL license.  
> and R is sort of compiler. If you can get answer from Spark community how  
> SparkR get managed to stay in Apache License while R is GPL, the answer  
> might help.  
> The description of SparkR is not accurate in any respect. (Do you think  
> SparkR is not talking to GPL-licensed libraries?)  
>  
> I don’t see that any genuine issue is being raised here.  
>  
> If there is an issue, the burden is on you to identify it.  
>  
> If i give you one suggestion, Zeppelin committers sometimes ask rebase the  
> contribution branch for some reason. It is not the really the best  
> practice, but still okay while most contributions are not including large  
> code base changes  
> However, your one, has more than 4000 lines of code change. If you rebase  
> then review should be started from the beginning, again. So you might want  
> to minimize the rebase your branch.  
>  
> Are you actually complaining that the problem is that I rebased the code  
> during the three-month period when no-one looked at it and Zeppelin went  
> through a release?  
>  
> I cannot take it seriously when you say things like this.  
>  
> Having to “start from the beginning” cannot be a problem if you never  
> actually started the first time...  
>  
>  

You wanted coordination and cooperation. So i gave you suggestion that  
helping review process. For example, your code has been rebased since my  
comment and jongyoul's comment. that means committers will need to look  
from the beginning. That'll require more time. And maybe, i guess that's  
not what you want. But If you don't care, feel free to rebase.  

Thanks,  
moon  



>  
> From: moon soo Lee <mo...@apache.org>  
> Reply: moon soo Lee <mo...@apache.org>  
> Date: December 1, 2015 at 4:42:06 AM  
> To: Amos B. Elberg <am...@gmail.com>,  
> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> request: R Interpreter for Zeppelin  
>  
> On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>  
> wrote:  
> Thank you, Cos.  
>  
> I’d like to briefly address the issues raised by Moon:  
>  
> 1. This PR does not passes CI  
> The CI fails on core Zeppelin, *not* code in this PR.  
>  
> I’ve been seeking assistance on this since August.  
>  
> The most common reason is that SparkInterpreter is unable to launch Spark  
> and open a Spark Backend. This is necessary to test the PR.  
>  
> 60+ hours, has been spent adapting and re-basing when the SparkInterpreter  
> architecture changed and broke the PR’s *tests.*  
>  
>  
> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin  
> core, why do you think other PRs are passing CI?  
>  
> And let's say Zeppelin core has problem and that makes your PR fails on CI  
> test. That's possible. But it still does not mean we can merge it with CI  
> failing.  
>  
> If you think it's problem on Zeppelin core, then file an issue that  
> reproduce the problem on Zeppelin core, that might be more efficient than  
> keep trying yourself.  
>  
>  
> 2. Not 100% sure this PR has no license issue. (about KniteR)  
> What license problem *specifically* do you believe may exist?  
>  
> In preparing the PR, I:  
>  
> * Reviewed the Apache policy in detail.  
>  
> * Contacted the FSF to ask their interpretation of the “linking”  
> provisions of the GPL license.  
>  
> * Reviewed how other Apache software deals with this issue (e.g., Spark  
> talks to R, which is GPL'd).  
>  
> * No necessary *dependencies* of the PR have license conflicts. In  
> several cases, I contacted package authors who agreed to re-issue their  
> packages under Apache-compatible licenses. (Usually I had to do a bit of  
> coding in exchange…)  
>  
> * Where the license had to stay GPL, the packages are *not necessary* and  
> *not dependencies.* If the optional packages are present, they will be  
> used to enable additional functionality. Knitr is an example. The PR will  
> compile and run fine without knitr. If knitr is available (it is part of  
> the most common R distribution), the PR will enable the knitr interpreter.  
>  
> * This is exactly how this issue is addressed through the Apache  
> ecosystem.  
> The tl;dr is this: When Apache code is written to talk to libraries that  
> may or may not be present on the user’s system, or where it talks to an API  
> but is agnostic about implementation, that is not “linking” in a way that  
> implicate the anti-linking provision of the GPL.  
>  
> Otherwise, no Apache code could ever call a shell script! Let alone run  
> on Linux, or talk to R.  
>  
>  
> I'm not a legal expert. So following could be wrong.  
>  
> In my interpretation, KnitRInterpreter is not an optional feature while it  
> is always enabled when compiling Zeppelin and always enabled when running  
> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes  
> it will fail when no KnitR is installed on the system)  
>  
> And of course, no Apache code can ever call a shell script, on the purpose  
> of dynamic linking with GPL library.  
>  
> I was guessing SparkR can be still in Apache License even if it is depends  
> on R. Because of GPL licensed compiler generated output is not GPL license.  
> and R is sort of compiler.  
>  
> If you can get answer from Spark community how SparkR get managed to stay  
> in Apache License while R is GPL, the answer might help.  
>  
>  
> 3. Need more time to review.  
> Has any reviewer has downloaded the PR or run the demo notebook? (Which  
> is there for the benefit of reviewers, and isn’t intended to go in final  
> distribution.)  
>  
> How many +1 comments from users would you like to see?  
>  
> How much time do you believe is required?  
>  
>  
> It all depends on when CI is going to pass, when license problem is going  
> to be cleared, and when a committer willing to review and responsible to  
> commit your contribution.  
>  
>  
> 1. Large code base change  
> Large code base changes require coordination and cooperation. This PR  
> necessarily implicates the build scripts, testing code, the  
> SparkInterpreter, etc.  
>  
> I have been seeking to coordinate since August.  
>  
> I continue to stand ready to do so.  
>  
> -Amos  
>  
>  
> If i give you one suggestion, Zeppelin committers sometimes ask rebase the  
> contribution branch for some reason. It is not the really the best  
> practice, but still okay while most contributions are not including large  
> code base changes.  
>  
> However, your one, has more than 4000 lines of code change. If you rebase  
> then review should be started from the beginning, again. So you might want  
> to minimize the rebase your branch.  
>  
> Thanks,  
> moon  
>  
>  
> From: moon soo Lee <mo...@apache.org>  
> Reply: dev@zeppelin.incubator.apache.org <  
> dev@zeppelin.incubator.apache.org>  
> Date: December 1, 2015 at 1:34:19 AM  
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>  
> Subject: Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull  
> request: R Interpreter for Zeppelin  
>  
> Hi Cos,  
>  
> Thanks for opening a discussion.  
> My answer about 'Why this PR is open for three months' is  
>  
> 1. This PR does not passes CI  
> 2. Not 100% sure this PR has no license issue. (about KniteR)  
> 3. Need more time to review.  
>  
> It's my personal answer, other committers may have different opinion.  
>  
>  
> I think the question should be generalized. Because this PR is not the only  
> PR that is in impasse. There're more. For example  
>  
> Here's some examples that PR is not been merged.  
> https://github.com/apache/incubator-zeppelin/pull/53,  
> https://github.com/apache/incubator-zeppelin/pull/60  
> and so on.  
>  
> I can categorize the cases, based on experience of involving Zeppelin  
> community from the beginning.  
>  
> 1. Large code base change  
>  
> When contribution has large code base changes, it tend to take more time to  
> review and merged. Normally, most contributions merged in 1day~1 week.  
> But some contribution has large code base changes take 2~4 weeks. Few  
> contribution that has very large code base change take months.  
>  
> 2. Communication lost  
>  
> Sometimes, committer is not responding, or contributor is not responding.  
>  
> 3. Opinion diverges  
>  
> For some changes, included in contribution. When people have different  
> opinion and it continue to diverges, those PRs are not been merged.  
>  
>  
> I think having a guide such as ping other committer when a committer is not  
> responding, and divide contribution into small peaces if possible, would  
> help most of the cases.  
> Of course committer have to pay attention more to the contribution and  
> helping people. That's the first one.  
>  
> Thanks,  
> moon  
>  
> On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org> wrote:  
>  
> > To make sure we're on the same page, here are two threads that I found  
> > related  
> > to this topic.  
> >  
> > Thread 1:  
> > Subject: R?  
> > Started on: July 1, 2015  
> >  
> > Thread 2:  
> > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for  
> > Zeppelin  
> > Started on: August 13, 2015  
> >  
> > If you want to fetch these from our archive send emails to  
> > dev-thread.1735@zeppelin.incubator.apache.org  
> > dev-thread.3573@zeppelin.incubator.apache.org  
> >  
> > Cos  
> >  
> > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:  
> > > Guys,  
> > >  
> > > While catching up on my emails from the last a couple of weeks, this  
> > thread  
> > > caught my attention. I am not normally paying much attention to the  
> code  
> > > reviews traffic from GH, but this one is pretty different as it spans  
> > three  
> > > months and counting.  
> > >  
> > > First, here are my five cents:  
> > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be  
> > contributed to  
> > > an ASF project this file should simply contain ASL2 text, like in [1]  
> > > - r/pom.xml perhaps shouldn't contain a separate <developers> section,  
> > but  
> > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't  
> > > maintain this in the source code, but we have the list of all the  
> > > committers on the project's site [2] Every new committer's first  
> > commit is  
> > > to update the team page ;)  
> > > - comments like in  
> > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java  
> > >  
> > > +/**  
> > > + * Created by aelberg on 7/28/15.  
> > > + */  
> > >  
> > > is better to be removed. It has been already discussed here [3]. And  
> > the  
> > > initial discussion on the topic [4] was linked as well  
> > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is  
> > R-specific  
> > > stuff - I have no idea about R, honestly, but  
> > >  
> > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE  
> > > ...  
> > > +Author: David B. Dahl  
> > >  
> > > shouldn't be here, IMO. Normally, if some additional licenses are  
> > used,  
> > > they have to be listed in the top-level NOTICE file (already there).  
> > >  
> > > - I am not going to offer any comments on the technical merits of the  
> > patch,  
> > > as I haven't tried it personally. However, I don't see any serious  
> > > technical objections to the functionality in question.  
> > >  
> > > So, the question is - why the PR is open for three months? I hasn't  
> been  
> > able  
> > > to get a clear answer. What I found out though is pretty unsettling,  
> > really.  
> > > The communication on the topic is almost non-existent, except for this  
> > sparse  
> > > and bitter thread in the GH.  
> > >  
> > > I would love to hear from the committers what's stopping the acceptance  
> > of the  
> > > code, besides of the minor issues I've mentioned earlier? What are the  
> > reasons for it?  
> > > Is there anything wrong with it?  
> > > One of the responsibilities of the committers is to make sure the  
> > > contributions are reviewed; new contributors are guided and do  
> > understand how  
> > > the project ticks. The easy feedback flow attracts new people, allowing  
> > the  
> > > community to grow and thrive.  
> > >  
> > > I am asking _explicitely_ not to re-start the bickering I have already  
> > > seen. At this point I am interested in the purely technical side of  
> this.  
> > >  
> > > [1] https://github.com/apache/bigtop/blob/master/LICENSE  
> > > [2] http://bigtop.apache.org/team-list.html  
> > > [3]  
> >  
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html  
> > > [4] http://s.apache.org/iZl  
> > >  
> > > With regards,  
> > > Cos  
> > >  
> > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:  
> > > > Github user elbamos commented on the pull request:  
> > > >  
> > > >  
> >  
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411  
> > > >  
> > > > The current push should resolve some issues with changes in the  
> > > > Spark-Zeppelin interface that had created issues for users, as  
> > well as  
> > > > support for 1.5.1.  
> > > >  
> > > > Knitr support is improved, and the reason for a separate knitr  
> > interpreter may be clearer now.  
> > > >  
> > > > The amount of code borrowed from rscala is reduced.  
> > > >  
> > > > I did not address issues with @author tags, or files under the R/  
> > > > folder. The reason is, to be blunt, I don't understand what the  
> > precise  
> > > > concerns actually are.  
> > > >  
> > > > Please note that I changed .travis.yml to only use spark 1.4 and  
> > later.  
> > > > I'm sure there is a better way to do it, but I'm just not in a  
> > position  
> > > > to try to figure out the entire Zeppelin build process.  
> > > >  
> > > > Integrating this with the rest of zeppelin is going to take some  
> > work  
> > > > regarding pom's, travis, and so forth. I can do a lot of that,  
> > but I'm  
> > > > going to need to discuss it with the people who have been "owning"  
> > those  
> > > > files. There are just too many moving pieces here.  
> > > >  
> > > > Because of the size of this (which is, unfortunately, necessary),  
> > > > posting here is probably not the most efficient way. That is also  
> > true  
> > > > because certain people chose to use this PR as a forum to air other  
> > > > issues. Therefore, it would be better if reviewers e-mail me  
> > directly.  
> >  
> >  
> >  
>  

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

On Wed, Dec 2, 2015 at 1:09 AM Amos B. Elberg <am...@gmail.com> wrote:

> I am going to try to minimize my reaction to Moon’s e-mail.
>
> The tl;dr is this:
>
> The reason we are having this discussion now is that active users of the
> PR — which now has its own user base — went public to complain about this.


> The PR has been tested by an active user base for more than three months.
> No-one has been able to identify any specific actual licensing problem, and
> the PR was prepared based on an extensive, careful review of the relevant
> licensing issues and after contacting the relevant people.
>
>

I admire every software that used by user and helping people. That includes
your work. But that's not the topic we're in discussion. Active user does
not mean your contribution can ignore the review.



> It is not an explanation for someone who has been ignoring my “how can I
> move this forward…” emails for three months to point the finger and say I
> didn’t contact the right person or file the right report.
>
>
This is also not the topic in this discussion.


> The burden for providing an explanation for the inaction is on the PMCC at
> this point.

I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> core, why do you think other PRs are passing CI?
> They’re not!  I often see comments on PRs to just ignore that CI is
> failing.
>
> One of the most common reasons this PR fails CI, is CI times-out
> downloading Spark to install.  How could that possibly be caused by the PR?
>
> It looks to me like the only PRs with changes to the relevant parts of the
> code — the SparkInterpreter — are being made by the person who wrote the
> testing suite.
>
> So, that would explain why some other PRs pass CI:  Neither the
> SparkInterpreter nor the testing suite are stable or robust, but since the
> PRs are coming from the person who wrote both…
>
> And let's say Zeppelin core has problem and that makes your PR fails on CI
> test. That's possible. But it still does not mean we can merge it with CI
> failing.
>
> It means you should be working with me to figure out why the CI is failing.
>
> This PR has been tested by an active user base for the past three months.
> If CI is continuing to fail, and dozens of hours of effort have not
> resolved the CI issues, then it is time to start considering whether the
> testing suite is part of the problem.
>
> The level of defensiveness about the CI and SparkInterpreter is not
> helping to resolve these issues.
>
> If you think it's problem on Zeppelin core, then file an issue that
> reproduce the problem on Zeppelin core, that might be more efficient than
> keep trying yourself.
> I contacted you numerous times about such issues...
>


I remember i commented your issue about CI. but you just keep repeated it's
not your problem but Zeppelin core problem.

Then please file an issue about the problem you found in Zeppelin Core.
Then everyone will get into the problem.



>
> In my interpretation, KnitRInterpreter is not an optional feature while it
> is always enabled when compiling Zeppelin and always enabled when running
> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
> it will fail when no KnitR is installed on the system)
>
> Its not always enabled.
> It is not dynamically linked at runtime.
> It will not fail when knitr is missing.  If knitr is not present, the repl
> interpreter starts and a note is written to to the log that the knitr
> interpreter isn’t available because knitr is not present.
>
> no Apache code can ever call a shell script, on the purpose of dynamic
> linking with GPL library.
> You misunderstand.
>
> The *shell* is GPL'd.
>
> Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends on a
> shell script to launch?
>
Obviously not.
>
> The interaction with R in the PR is the same.
>
>

Again, bash is one of exceptions of GPL, like other GPL licensed
compiler/interpreter.

Check here why Bash and R is okay with Apache License.
https://stat.ethz.ch/pipermail/r-help/2008-July/169332.html

I'm not sure we can apply the same exception for 'using' KnitR.

My point is not 'KnitR' is optional or not. Point is 'KnitRInterpreter' you
wrote is not an optional feature. Which is clearly not optionally enabled
code and feature. And that depends on KnitR library which is GPL.



> I was guessing SparkR can be still in Apache License even if it is depends
> on R. Because of GPL licensed compiler generated output is not GPL license.
> and R is sort of compiler. If you can get answer from Spark community how
> SparkR get managed to stay in Apache License while R is GPL, the answer
> might help.
> The description of SparkR is not accurate in any respect.  (Do you think
> SparkR is not talking to GPL-licensed libraries?)
>
> I don’t see that any genuine issue is being raised here.
>
> If there is an issue, the burden is on you to identify it.
>
> If i give you one suggestion, Zeppelin committers sometimes ask rebase the
> contribution branch for some reason. It is not the really the best
> practice, but still okay while most contributions are not including large
> code base changes
> However, your one, has more than 4000 lines of code change. If you rebase
>  then review should be started from the beginning, again. So you might want
> to minimize the rebase your branch.
>
> Are you actually complaining that the problem is that I rebased the code
> during the three-month period when no-one looked at it and Zeppelin went
> through a release?
>
> I cannot take it seriously when you say things like this.
>
> Having to “start from the beginning” cannot be a problem if you never
> actually started the first time...
>
>

You wanted coordination and cooperation. So i gave you suggestion that
helping review process. For example, your code has been rebased since my
comment and jongyoul's comment. that means committers will need to look
from the beginning. That'll require more time. And maybe, i guess that's
not what you want. But If you don't care, feel free to rebase.

Thanks,
moon



>
> From: moon soo Lee <mo...@apache.org>
> Reply: moon soo Lee <mo...@apache.org>
> Date: December 1, 2015 at 4:42:06 AM
> To: Amos B. Elberg <am...@gmail.com>,
> dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com>
> wrote:
> Thank you, Cos.
>
> I’d like to briefly address the issues raised by Moon:
>
> 1. This PR does not passes CI
> The CI fails on core Zeppelin, *not* code in this PR.
>
> I’ve been seeking assistance on this since August.
>
> The most common reason is that SparkInterpreter is unable to launch Spark
> and open a Spark Backend.  This is necessary to test the PR.
>
> 60+ hours, has been spent adapting and re-basing when the SparkInterpreter
> architecture changed and broke the PR’s *tests.*
>
>
> I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
> core, why do you think other PRs are passing CI?
>
> And let's say Zeppelin core has problem and that makes your PR fails on CI
> test. That's possible. But it still does not mean we can merge it with CI
> failing.
>
> If you think it's problem on Zeppelin core, then file an issue that
> reproduce the problem on Zeppelin core, that might be more efficient than
> keep trying yourself.
>
>
> 2. Not 100% sure this PR has no license issue. (about KniteR)
> What license problem *specifically* do you believe may exist?
>
> In preparing the PR, I:
>
> * Reviewed the Apache policy in detail.
>
> * Contacted the FSF to ask their interpretation of the “linking”
> provisions of the GPL license.
>
> * Reviewed how other Apache software deals with this issue (e.g., Spark
> talks to R, which is GPL'd).
>
> * No necessary *dependencies* of the PR have license conflicts.  In
> several cases, I contacted package authors who agreed to re-issue their
> packages under Apache-compatible licenses.  (Usually I had to do a bit of
> coding in exchange…)
>
> * Where the license had to stay GPL, the packages are *not necessary* and
> *not dependencies.*  If the optional packages are present, they will be
> used to enable additional functionality.  Knitr is an example.  The PR will
> compile and run fine without knitr.  If knitr is available (it is part of
> the most common R distribution), the PR will enable the knitr interpreter.
>
> * This is exactly how this issue is addressed through the Apache
> ecosystem.
> The tl;dr is this:   When Apache code is written to talk to libraries that
> may or may not be present on the user’s system, or where it talks to an API
> but is agnostic about implementation, that is not “linking” in a way that
> implicate the anti-linking provision of the GPL.
>
> Otherwise, no Apache code could ever call a shell script!  Let alone run
> on Linux, or talk to R.
>
>
> I'm not a legal expert. So following could be wrong.
>
> In my interpretation, KnitRInterpreter is not an optional feature while it
> is always enabled when compiling Zeppelin and always enabled when running
> Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
> it will fail when no KnitR is installed on the system)
>
> And of course, no Apache code can ever call a shell script, on the purpose
> of dynamic linking with GPL library.
>
> I was guessing SparkR can be still in Apache License even if it is depends
> on R. Because of GPL licensed compiler generated output is not GPL license.
> and R is sort of compiler.
>
> If you can get answer from Spark community how SparkR get managed to stay
> in Apache License while R is GPL, the answer might help.
>
>
> 3. Need more time to review.
> Has any reviewer has downloaded the PR or run the demo notebook?  (Which
> is there for the benefit of reviewers, and isn’t intended to go in final
> distribution.)
>
> How many +1 comments from users would you like to see?
>
> How much time do you believe is required?
>
>
> It all depends on when CI is going to pass, when license problem is going
> to be cleared, and when a committer willing to review and responsible to
> commit your contribution.
>
>
> 1. Large code base change
> Large code base changes require coordination and cooperation.  This PR
> necessarily implicates the build scripts, testing code, the
> SparkInterpreter, etc.
>
> I have been seeking to coordinate since August.
>
> I continue to stand ready to do so.
>
> -Amos
>
>
> If i give you one suggestion, Zeppelin committers sometimes ask rebase the
> contribution branch for some reason. It is not the really the best
> practice, but still okay while most contributions are not including large
> code base changes.
>
> However, your one, has more than 4000 lines of code change. If you rebase
>  then review should be started from the beginning, again. So you might want
> to minimize the rebase your branch.
>
> Thanks,
> moon
>
>
> From: moon soo Lee <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org <
> dev@zeppelin.incubator.apache.org>
> Date: December 1, 2015 at 1:34:19 AM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull
> request: R Interpreter for Zeppelin
>
> Hi Cos,
>
> Thanks for opening a discussion.
> My answer about 'Why this PR is open for three months' is
>
> 1. This PR does not passes CI
> 2. Not 100% sure this PR has no license issue. (about KniteR)
> 3. Need more time to review.
>
> It's my personal answer, other committers may have different opinion.
>
>
> I think the question should be generalized. Because this PR is not the only
> PR that is in impasse. There're more. For example
>
> Here's some examples that PR is not been merged.
> https://github.com/apache/incubator-zeppelin/pull/53,
> https://github.com/apache/incubator-zeppelin/pull/60
> and so on.
>
> I can categorize the cases, based on experience of involving Zeppelin
> community from the beginning.
>
> 1. Large code base change
>
> When contribution has large code base changes, it tend to take more time to
> review and merged. Normally, most contributions merged in 1day~1 week.
> But some contribution has large code base changes take 2~4 weeks. Few
> contribution that has very large code base change take months.
>
> 2. Communication lost
>
> Sometimes, committer is not responding, or contributor is not responding.
>
> 3. Opinion diverges
>
> For some changes, included in contribution. When people have different
> opinion and it continue to diverges, those PRs are not been merged.
>
>
> I think having a guide such as ping other committer when a committer is not
> responding, and divide contribution into small peaces if possible, would
> help most of the cases.
> Of course committer have to pay attention more to the contribution and
> helping people. That's the first one.
>
> Thanks,
> moon
>
> On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org> wrote:
>
> > To make sure we're on the same page, here are two threads that I found
> > related
> > to this topic.
> >
> > Thread 1:
> > Subject: R?
> > Started on: July 1, 2015
> >
> > Thread 2:
> > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > Zeppelin
> > Started on: August 13, 2015
> >
> > If you want to fetch these from our archive send emails to
> > dev-thread.1735@zeppelin.incubator.apache.org
> > dev-thread.3573@zeppelin.incubator.apache.org
> >
> > Cos
> >
> > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > Guys,
> > >
> > > While catching up on my emails from the last a couple of weeks, this
> > thread
> > > caught my attention. I am not normally paying much attention to the
> code
> > > reviews traffic from GH, but this one is pretty different as it spans
> > three
> > > months and counting.
> > >
> > > First, here are my five cents:
> > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > contributed to
> > > an ASF project this file should simply contain ASL2 text, like in [1]
> > > - r/pom.xml perhaps shouldn't contain a separate <developers> section,
> > but
> > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > > maintain this in the source code, but we have the list of all the
> > > committers on the project's site [2] Every new committer's first
> > commit is
> > > to update the team page ;)
> > > - comments like in
> > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > >
> > > +/**
> > > + * Created by aelberg on 7/28/15.
> > > + */
> > >
> > > is better to be removed. It has been already discussed here [3]. And
> > the
> > > initial discussion on the topic [4] was linked as well
> > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> > R-specific
> > > stuff - I have no idea about R, honestly, but
> > >
> > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > ...
> > > +Author: David B. Dahl
> > >
> > > shouldn't be here, IMO. Normally, if some additional licenses are
> > used,
> > > they have to be listed in the top-level NOTICE file (already there).
> > >
> > > - I am not going to offer any comments on the technical merits of the
> > patch,
> > > as I haven't tried it personally. However, I don't see any serious
> > > technical objections to the functionality in question.
> > >
> > > So, the question is - why the PR is open for three months? I hasn't
> been
> > able
> > > to get a clear answer. What I found out though is pretty unsettling,
> > really.
> > > The communication on the topic is almost non-existent, except for this
> > sparse
> > > and bitter thread in the GH.
> > >
> > > I would love to hear from the committers what's stopping the acceptance
> > of the
> > > code, besides of the minor issues I've mentioned earlier? What are the
> > reasons for it?
> > > Is there anything wrong with it?
> > > One of the responsibilities of the committers is to make sure the
> > > contributions are reviewed; new contributors are guided and do
> > understand how
> > > the project ticks. The easy feedback flow attracts new people, allowing
> > the
> > > community to grow and thrive.
> > >
> > > I am asking _explicitely_ not to re-start the bickering I have already
> > > seen. At this point I am interested in the purely technical side of
> this.
> > >
> > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > [2] http://bigtop.apache.org/team-list.html
> > > [3]
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > [4] http://s.apache.org/iZl
> > >
> > > With regards,
> > > Cos
> > >
> > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > Github user elbamos commented on the pull request:
> > > >
> > > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > >
> > > > The current push should resolve some issues with changes in the
> > > > Spark-Zeppelin interface that had created issues for users, as
> > well as
> > > > support for 1.5.1.
> > > >
> > > > Knitr support is improved, and the reason for a separate knitr
> > interpreter may be clearer now.
> > > >
> > > > The amount of code borrowed from rscala is reduced.
> > > >
> > > > I did not address issues with @author tags, or files under the R/
> > > > folder. The reason is, to be blunt, I don't understand what the
> > precise
> > > > concerns actually are.
> > > >
> > > > Please note that I changed .travis.yml to only use spark 1.4 and
> > later.
> > > > I'm sure there is a better way to do it, but I'm just not in a
> > position
> > > > to try to figure out the entire Zeppelin build process.
> > > >
> > > > Integrating this with the rest of zeppelin is going to take some
> > work
> > > > regarding pom's, travis, and so forth. I can do a lot of that,
> > but I'm
> > > > going to need to discuss it with the people who have been "owning"
> > those
> > > > files. There are just too many moving pieces here.
> > > >
> > > > Because of the size of this (which is, unfortunately, necessary),
> > > > posting here is probably not the most efficient way. That is also
> > true
> > > > because certain people chose to use this PR as a forum to air other
> > > > issues. Therefore, it would be better if reviewers e-mail me
> > directly.
> >
> >
> >
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

I am going to try to minimize my reaction to Moon’s e-mail.  

The tl;dr is this:   

The reason we are having this discussion now is that active users of the PR — which now has its own user base — went public to complain about this. 

The PR has been tested by an active user base for more than three months.  No-one has been able to identify any specific actual licensing problem, and the PR was prepared based on an extensive, careful review of the relevant licensing issues and after contacting the relevant people.  

It is not an explanation for someone who has been ignoring my “how can I move this forward…” emails for three months to point the finger and say I didn’t contact the right person or file the right report. 

The burden for providing an explanation for the inaction is on the PMCC at this point. 


I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin core, why do you think other PRs are passing CI?
They’re not!  I often see comments on PRs to just ignore that CI is failing. 

One of the most common reasons this PR fails CI, is CI times-out downloading Spark to install.  How could that possibly be caused by the PR?

It looks to me like the only PRs with changes to the relevant parts of the code — the SparkInterpreter — are being made by the person who wrote the testing suite.

So, that would explain why some other PRs pass CI:  Neither the SparkInterpreter nor the testing suite are stable or robust, but since the PRs are coming from the person who wrote both… 

And let's say Zeppelin core has problem and that makes your PR fails on CI test. That's possible. But it still does not mean we can merge it with CI failing.

It means you should be working with me to figure out why the CI is failing.

This PR has been tested by an active user base for the past three months.  If CI is continuing to fail, and dozens of hours of effort have not resolved the CI issues, then it is time to start considering whether the testing suite is part of the problem. 

The level of defensiveness about the CI and SparkInterpreter is not helping to resolve these issues. 

If you think it's problem on Zeppelin core, then file an issue that reproduce the problem on Zeppelin core, that might be more efficient than keep trying yourself.
I contacted you numerous times about such issues...

In my interpretation, KnitRInterpreter is not an optional feature while it is always enabled when compiling Zeppelin and always enabled when running Zeppelin. And it requires dynamically linked GPL library on runtime. (yes it will fail when no KnitR is installed on the system)

Its not always enabled. 
It is not dynamically linked at runtime. 
It will not fail when knitr is missing.  If knitr is not present, the repl interpreter starts and a note is written to to the log that the knitr interpreter isn’t available because knitr is not present.  

no Apache code can ever call a shell script, on the purpose of dynamic linking with GPL library.
You misunderstand. 

The *shell* is GPL'd.  

Is Zeppelin “linked" against the GPL’d shell because Zeppelin depends on a shell script to launch?  

Obviously not.

The interaction with R in the PR is the same. 

I was guessing SparkR can be still in Apache License even if it is depends on R. Because of GPL licensed compiler generated output is not GPL license. and R is sort of compiler. If you can get answer from Spark community how SparkR get managed to stay in Apache License while R is GPL, the answer might help.
The description of SparkR is not accurate in any respect.  (Do you think SparkR is not talking to GPL-licensed libraries?)

I don’t see that any genuine issue is being raised here.   

If there is an issue, the burden is on you to identify it. 

If i give you one suggestion, Zeppelin committers sometimes ask rebase the contribution branch for some reason. It is not the really the best practice, but still okay while most contributions are not including large code base changes
However, your one, has more than 4000 lines of code change. If you rebase  then review should be started from the beginning, again. So you might want to minimize the rebase your branch.

Are you actually complaining that the problem is that I rebased the code during the three-month period when no-one looked at it and Zeppelin went through a release?

I cannot take it seriously when you say things like this.  

Having to “start from the beginning” cannot be a problem if you never actually started the first time...



From: moon soo Lee <mo...@apache.org>
Reply: moon soo Lee <mo...@apache.org>
Date: December 1, 2015 at 4:42:06 AM
To: Amos B. Elberg <am...@gmail.com>, dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com> wrote:
Thank you, Cos.  

I’d like to briefly address the issues raised by Moon:

1. This PR does not passes CI 
The CI fails on core Zeppelin, *not* code in this PR.  

I’ve been seeking assistance on this since August. 

The most common reason is that SparkInterpreter is unable to launch Spark and open a Spark Backend.  This is necessary to test the PR.

60+ hours, has been spent adapting and re-basing when the SparkInterpreter architecture changed and broke the PR’s *tests.* 


I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin core, why do you think other PRs are passing CI?

And let's say Zeppelin core has problem and that makes your PR fails on CI test. That's possible. But it still does not mean we can merge it with CI failing.

If you think it's problem on Zeppelin core, then file an issue that reproduce the problem on Zeppelin core, that might be more efficient than keep trying yourself.


2. Not 100% sure this PR has no license issue. (about KniteR) 
What license problem *specifically* do you believe may exist?  

In preparing the PR, I:  

* Reviewed the Apache policy in detail. 

* Contacted the FSF to ask their interpretation of the “linking” provisions of the GPL license.

* Reviewed how other Apache software deals with this issue (e.g., Spark talks to R, which is GPL'd). 

* No necessary *dependencies* of the PR have license conflicts.  In several cases, I contacted package authors who agreed to re-issue their packages under Apache-compatible licenses.  (Usually I had to do a bit of coding in exchange…)

* Where the license had to stay GPL, the packages are *not necessary* and *not dependencies.*  If the optional packages are present, they will be used to enable additional functionality.  Knitr is an example.  The PR will compile and run fine without knitr.  If knitr is available (it is part of the most common R distribution), the PR will enable the knitr interpreter.  

* This is exactly how this issue is addressed through the Apache ecosystem. 
The tl;dr is this:   When Apache code is written to talk to libraries that may or may not be present on the user’s system, or where it talks to an API but is agnostic about implementation, that is not “linking” in a way that implicate the anti-linking provision of the GPL.   

Otherwise, no Apache code could ever call a shell script!  Let alone run on Linux, or talk to R. 


I'm not a legal expert. So following could be wrong.

In my interpretation, KnitRInterpreter is not an optional feature while it is always enabled when compiling Zeppelin and always enabled when running Zeppelin. And it requires dynamically linked GPL library on runtime. (yes it will fail when no KnitR is installed on the system)

And of course, no Apache code can ever call a shell script, on the purpose of dynamic linking with GPL library.

I was guessing SparkR can be still in Apache License even if it is depends on R. Because of GPL licensed compiler generated output is not GPL license. and R is sort of compiler.

If you can get answer from Spark community how SparkR get managed to stay in Apache License while R is GPL, the answer might help.


3. Need more time to review. 
Has any reviewer has downloaded the PR or run the demo notebook?  (Which is there for the benefit of reviewers, and isn’t intended to go in final distribution.)  

How many +1 comments from users would you like to see? 

How much time do you believe is required?


It all depends on when CI is going to pass, when license problem is going to be cleared, and when a committer willing to review and responsible to commit your contribution.


1. Large code base change 
Large code base changes require coordination and cooperation.  This PR necessarily implicates the build scripts, testing code, the SparkInterpreter, etc.  

I have been seeking to coordinate since August. 

I continue to stand ready to do so. 

-Amos   


If i give you one suggestion, Zeppelin committers sometimes ask rebase the contribution branch for some reason. It is not the really the best practice, but still okay while most contributions are not including large code base changes.

However, your one, has more than 4000 lines of code change. If you rebase  then review should be started from the beginning, again. So you might want to minimize the rebase your branch.

Thanks,
moon

 
From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 1, 2015 at 1:34:19 AM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

Hi Cos,

Thanks for opening a discussion.
My answer about 'Why this PR is open for three months' is

1. This PR does not passes CI
2. Not 100% sure this PR has no license issue. (about KniteR)
3. Need more time to review.

It's my personal answer, other committers may have different opinion.


I think the question should be generalized. Because this PR is not the only
PR that is in impasse. There're more. For example

Here's some examples that PR is not been merged.
https://github.com/apache/incubator-zeppelin/pull/53,
https://github.com/apache/incubator-zeppelin/pull/60
and so on.

I can categorize the cases, based on experience of involving Zeppelin
community from the beginning.

1. Large code base change

When contribution has large code base changes, it tend to take more time to
review and merged. Normally, most contributions merged in 1day~1 week.
But some contribution has large code base changes take 2~4 weeks. Few
contribution that has very large code base change take months.

2. Communication lost

Sometimes, committer is not responding, or contributor is not responding.

3. Opinion diverges

For some changes, included in contribution. When people have different
opinion and it continue to diverges, those PRs are not been merged.


I think having a guide such as ping other committer when a committer is not
responding, and divide contribution into small peaces if possible, would
help most of the cases.
Of course committer have to pay attention more to the contribution and
helping people. That's the first one.

Thanks,
moon

On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org> wrote:

> To make sure we're on the same page, here are two threads that I found
> related
> to this topic.
>
> Thread 1:
> Subject: R?
> Started on: July 1, 2015
>
> Thread 2:
> Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> Zeppelin
> Started on: August 13, 2015
>
> If you want to fetch these from our archive send emails to
> dev-thread.1735@zeppelin.incubator.apache.org
> dev-thread.3573@zeppelin.incubator.apache.org
>
> Cos
>
> On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > Guys,
> >
> > While catching up on my emails from the last a couple of weeks, this
> thread
> > caught my attention. I am not normally paying much attention to the code
> > reviews traffic from GH, but this one is pretty different as it spans
> three
> > months and counting.
> >
> > First, here are my five cents:
> > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> contributed to
> > an ASF project this file should simply contain ASL2 text, like in [1]
> > - r/pom.xml perhaps shouldn't contain a separate <developers> section,
> but
> > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > maintain this in the source code, but we have the list of all the
> > committers on the project's site [2] Every new committer's first
> commit is
> > to update the team page ;)
> > - comments like in
> r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> >
> > +/**
> > + * Created by aelberg on 7/28/15.
> > + */
> >
> > is better to be removed. It has been already discussed here [3]. And
> the
> > initial discussion on the topic [4] was linked as well
> > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> R-specific
> > stuff - I have no idea about R, honestly, but
> >
> > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > ...
> > +Author: David B. Dahl
> >
> > shouldn't be here, IMO. Normally, if some additional licenses are
> used,
> > they have to be listed in the top-level NOTICE file (already there).
> >
> > - I am not going to offer any comments on the technical merits of the
> patch,
> > as I haven't tried it personally. However, I don't see any serious
> > technical objections to the functionality in question.
> >
> > So, the question is - why the PR is open for three months? I hasn't been
> able
> > to get a clear answer. What I found out though is pretty unsettling,
> really.
> > The communication on the topic is almost non-existent, except for this
> sparse
> > and bitter thread in the GH.
> >
> > I would love to hear from the committers what's stopping the acceptance
> of the
> > code, besides of the minor issues I've mentioned earlier? What are the
> reasons for it?
> > Is there anything wrong with it?
> > One of the responsibilities of the committers is to make sure the
> > contributions are reviewed; new contributors are guided and do
> understand how
> > the project ticks. The easy feedback flow attracts new people, allowing
> the
> > community to grow and thrive.
> >
> > I am asking _explicitely_ not to re-start the bickering I have already
> > seen. At this point I am interested in the purely technical side of this.
> >
> > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > [2] http://bigtop.apache.org/team-list.html
> > [3]
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > [4] http://s.apache.org/iZl
> >
> > With regards,
> > Cos
> >
> > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > Github user elbamos commented on the pull request:
> > >
> > >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > >
> > > The current push should resolve some issues with changes in the
> > > Spark-Zeppelin interface that had created issues for users, as
> well as
> > > support for 1.5.1.
> > >
> > > Knitr support is improved, and the reason for a separate knitr
> interpreter may be clearer now.
> > >
> > > The amount of code borrowed from rscala is reduced.
> > >
> > > I did not address issues with @author tags, or files under the R/
> > > folder. The reason is, to be blunt, I don't understand what the
> precise
> > > concerns actually are.
> > >
> > > Please note that I changed .travis.yml to only use spark 1.4 and
> later.
> > > I'm sure there is a better way to do it, but I'm just not in a
> position
> > > to try to figure out the entire Zeppelin build process.
> > >
> > > Integrating this with the rest of zeppelin is going to take some
> work
> > > regarding pom's, travis, and so forth. I can do a lot of that,
> but I'm
> > > going to need to discuss it with the people who have been "owning"
> those
> > > files. There are just too many moving pieces here.
> > >
> > > Because of the size of this (which is, unfortunately, necessary),
> > > posting here is probably not the most efficient way. That is also
> true
> > > because certain people chose to use this PR as a forum to air other
> > > issues. Therefore, it would be better if reviewers e-mail me
> directly.
>
>
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

On Tue, Dec 1, 2015 at 4:40 PM Amos B. Elberg <am...@gmail.com> wrote:

> Thank you, Cos.
>
> I’d like to briefly address the issues raised by Moon:
>
> 1. This PR does not passes CI
>
> The CI fails on core Zeppelin, *not* code in this PR.
>
> I’ve been seeking assistance on this since August.
>
> The most common reason is that SparkInterpreter is unable to launch Spark
> and open a Spark Backend.  This is necessary to test the PR.
>
> 60+ hours, has been spent adapting and re-basing when the SparkInterpreter
> architecture changed and broke the PR’s *tests.*
>

I'm sorry, but the other PRs are passing CI. If it's problem on Zeppelin
core, why do you think other PRs are passing CI?

And let's say Zeppelin core has problem and that makes your PR fails on CI
test. That's possible. But it still does not mean we can merge it with CI
failing.

If you think it's problem on Zeppelin core, then file an issue that
reproduce the problem on Zeppelin core, that might be more efficient than
keep trying yourself.


2. Not 100% sure this PR has no license issue. (about KniteR)
>
> What license problem *specifically* do you believe may exist?
>
> In preparing the PR, I:
>
> * Reviewed the Apache policy in detail.
>
> * Contacted the FSF to ask their interpretation of the “linking”
> provisions of the GPL license.
>
> * Reviewed how other Apache software deals with this issue (e.g., Spark
> talks to R, which is GPL'd).
>
> * No necessary *dependencies* of the PR have license conflicts.  In
> several cases, I contacted package authors who agreed to re-issue their
> packages under Apache-compatible licenses.  (Usually I had to do a bit of
> coding in exchange…)
>
> * Where the license had to stay GPL, the packages are *not necessary* and
> *not dependencies.*  If the optional packages are present, they will be
> used to enable additional functionality.  Knitr is an example.  The PR will
> compile and run fine without knitr.  If knitr is available (it is part of
> the most common R distribution), the PR will enable the knitr interpreter.
> * This is exactly how this issue is addressed through the Apache
> ecosystem.
>
> The tl;dr is this:   When Apache code is written to talk to libraries that
> may or may not be present on the user’s system, or where it talks to an API
> but is agnostic about implementation, that is not “linking” in a way that
> implicate the anti-linking provision of the GPL.
>
> Otherwise, no Apache code could ever call a shell script!  Let alone run
> on Linux, or talk to R.
>

I'm not a legal expert. So following could be wrong.

In my interpretation, KnitRInterpreter is not an optional feature while it
is always enabled when compiling Zeppelin and always enabled when running
Zeppelin. And it requires dynamically linked GPL library on runtime. (yes
it will fail when no KnitR is installed on the system)

And of course, no Apache code can ever call a shell script, on the purpose
of dynamic linking with GPL library.

I was guessing SparkR can be still in Apache License even if it is depends
on R. Because of GPL licensed compiler generated output is not GPL license.
and R is sort of compiler.

If you can get answer from Spark community how SparkR get managed to stay
in Apache License while R is GPL, the answer might help.


3. Need more time to review.
>
> Has any reviewer has downloaded the PR or run the demo notebook?  (Which
> is there for the benefit of reviewers, and isn’t intended to go in final
> distribution.)
>
> How many +1 comments from users would you like to see?
>
> How much time do you believe is required?
>

It all depends on when CI is going to pass, when license problem is going
to be cleared, and when a committer willing to review and responsible to
commit your contribution.


1. Large code base change
>
> Large code base changes require coordination and cooperation.  This PR
> necessarily implicates the build scripts, testing code, the
> SparkInterpreter, etc.
>
> I have been seeking to coordinate since August.
>
> I continue to stand ready to do so.
>
> -Amos
>

If i give you one suggestion, Zeppelin committers sometimes ask rebase the
contribution branch for some reason. It is not the really the best
practice, but still okay while most contributions are not including large
code base changes.

However, your one, has more than 4000 lines of code change. If you rebase
 then review should be started from the beginning, again. So you might want
to minimize the rebase your branch.

Thanks,
moon



> From: moon soo Lee <mo...@apache.org> <mo...@apache.org>
> Reply: dev@zeppelin.incubator.apache.org
> <de...@zeppelin.incubator.apache.org> <de...@zeppelin.incubator.apache.org>
> Date: December 1, 2015 at 1:34:19 AM
> To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
> <de...@zeppelin.incubator.apache.org>
> Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin
> pull request: R Interpreter for Zeppelin
>
> Hi Cos,
>
> Thanks for opening a discussion.
> My answer about 'Why this PR is open for three months' is
>
> 1. This PR does not passes CI
> 2. Not 100% sure this PR has no license issue. (about KniteR)
> 3. Need more time to review.
>
> It's my personal answer, other committers may have different opinion.
>
>
> I think the question should be generalized. Because this PR is not the
> only
> PR that is in impasse. There're more. For example
>
> Here's some examples that PR is not been merged.
> https://github.com/apache/incubator-zeppelin/pull/53,
> https://github.com/apache/incubator-zeppelin/pull/60
> and so on.
>
> I can categorize the cases, based on experience of involving Zeppelin
> community from the beginning.
>
> 1. Large code base change
>
> When contribution has large code base changes, it tend to take more time
> to
> review and merged. Normally, most contributions merged in 1day~1 week.
> But some contribution has large code base changes take 2~4 weeks. Few
> contribution that has very large code base change take months.
>
> 2. Communication lost
>
> Sometimes, committer is not responding, or contributor is not responding.
>
> 3. Opinion diverges
>
> For some changes, included in contribution. When people have different
> opinion and it continue to diverges, those PRs are not been merged.
>
>
> I think having a guide such as ping other committer when a committer is
> not
> responding, and divide contribution into small peaces if possible, would
> help most of the cases.
> Of course committer have to pay attention more to the contribution and
> helping people. That's the first one.
>
> Thanks,
> moon
>
> On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org> wrote:
>
> > To make sure we're on the same page, here are two threads that I found
> > related
> > to this topic.
> >
> > Thread 1:
> > Subject: R?
> > Started on: July 1, 2015
> >
> > Thread 2:
> > Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> > Zeppelin
> > Started on: August 13, 2015
> >
> > If you want to fetch these from our archive send emails to
> > dev-thread.1735@zeppelin.incubator.apache.org
> > dev-thread.3573@zeppelin.incubator.apache.org
> >
> > Cos
> >
> > On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > > Guys,
> > >
> > > While catching up on my emails from the last a couple of weeks, this
> > thread
> > > caught my attention. I am not normally paying much attention to the
> code
> > > reviews traffic from GH, but this one is pretty different as it spans
> > three
> > > months and counting.
> > >
> > > First, here are my five cents:
> > > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> > contributed to
> > > an ASF project this file should simply contain ASL2 text, like in [1]
> > > - r/pom.xml perhaps shouldn't contain a separate <developers> section,
> > but
> > > Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> > > maintain this in the source code, but we have the list of all the
> > > committers on the project's site [2] Every new committer's first
> > commit is
> > > to update the team page ;)
> > > - comments like in
> > r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> > >
> > > +/**
> > > + * Created by aelberg on 7/28/15.
> > > + */
> > >
> > > is better to be removed. It has been already discussed here [3]. And
> > the
> > > initial discussion on the topic [4] was linked as well
> > > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> > R-specific
> > > stuff - I have no idea about R, honestly, but
> > >
> > > +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> > > ...
> > > +Author: David B. Dahl
> > >
> > > shouldn't be here, IMO. Normally, if some additional licenses are
> > used,
> > > they have to be listed in the top-level NOTICE file (already there).
> > >
> > > - I am not going to offer any comments on the technical merits of the
> > patch,
> > > as I haven't tried it personally. However, I don't see any serious
> > > technical objections to the functionality in question.
> > >
> > > So, the question is - why the PR is open for three months? I hasn't
> been
> > able
> > > to get a clear answer. What I found out though is pretty unsettling,
> > really.
> > > The communication on the topic is almost non-existent, except for this
> > sparse
> > > and bitter thread in the GH.
> > >
> > > I would love to hear from the committers what's stopping the
> acceptance
> > of the
> > > code, besides of the minor issues I've mentioned earlier? What are the
> > reasons for it?
> > > Is there anything wrong with it?
> > > One of the responsibilities of the committers is to make sure the
> > > contributions are reviewed; new contributors are guided and do
> > understand how
> > > the project ticks. The easy feedback flow attracts new people,
> allowing
> > the
> > > community to grow and thrive.
> > >
> > > I am asking _explicitely_ not to re-start the bickering I have already
> > > seen. At this point I am interested in the purely technical side of
> this.
> > >
> > > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > > [2] http://bigtop.apache.org/team-list.html
> > > [3]
> >
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > > [4] http://s.apache.org/iZl
> > >
> > > With regards,
> > > Cos
> > >
> > > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > > Github user elbamos commented on the pull request:
> > > >
> > > >
> >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > > >
> > > > The current push should resolve some issues with changes in the
> > > > Spark-Zeppelin interface that had created issues for users, as
> > well as
> > > > support for 1.5.1.
> > > >
> > > > Knitr support is improved, and the reason for a separate knitr
> > interpreter may be clearer now.
> > > >
> > > > The amount of code borrowed from rscala is reduced.
> > > >
> > > > I did not address issues with @author tags, or files under the R/
> > > > folder. The reason is, to be blunt, I don't understand what the
> > precise
> > > > concerns actually are.
> > > >
> > > > Please note that I changed .travis.yml to only use spark 1.4 and
> > later.
> > > > I'm sure there is a better way to do it, but I'm just not in a
> > position
> > > > to try to figure out the entire Zeppelin build process.
> > > >
> > > > Integrating this with the rest of zeppelin is going to take some
> > work
> > > > regarding pom's, travis, and so forth. I can do a lot of that,
> > but I'm
> > > > going to need to discuss it with the people who have been "owning"
> > those
> > > > files. There are just too many moving pieces here.
> > > >
> > > > Because of the size of this (which is, unfortunately, necessary),
> > > > posting here is probably not the most efficient way. That is also
> > true
> > > > because certain people chose to use this PR as a forum to air other
> > > > issues. Therefore, it would be better if reviewers e-mail me
> > directly.
> >
> >
> >
>
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

["Amos B. Elberg" <am...@gmail.com>]

Thank you, Cos.  

I’d like to briefly address the issues raised by Moon:

1. This PR does not passes CI 
The CI fails on core Zeppelin, *not* code in this PR.  

I’ve been seeking assistance on this since August. 

The most common reason is that SparkInterpreter is unable to launch Spark and open a Spark Backend.  This is necessary to test the PR.

60+ hours, has been spent adapting and re-basing when the SparkInterpreter architecture changed and broke the PR’s *tests.* 

2. Not 100% sure this PR has no license issue. (about KniteR) 
What license problem *specifically* do you believe may exist?  

In preparing the PR, I:  

* Reviewed the Apache policy in detail. 

* Contacted the FSF to ask their interpretation of the “linking” provisions of the GPL license.

* Reviewed how other Apache software deals with this issue (e.g., Spark talks to R, which is GPL'd). 

* No necessary *dependencies* of the PR have license conflicts.  In several cases, I contacted package authors who agreed to re-issue their packages under Apache-compatible licenses.  (Usually I had to do a bit of coding in exchange…)

* Where the license had to stay GPL, the packages are *not necessary* and *not dependencies.*  If the optional packages are present, they will be used to enable additional functionality.  Knitr is an example.  The PR will compile and run fine without knitr.  If knitr is available (it is part of the most common R distribution), the PR will enable the knitr interpreter.  

* This is exactly how this issue is addressed through the Apache ecosystem. 
The tl;dr is this:   When Apache code is written to talk to libraries that may or may not be present on the user’s system, or where it talks to an API but is agnostic about implementation, that is not “linking” in a way that implicate the anti-linking provision of the GPL.   

Otherwise, no Apache code could ever call a shell script!  Let alone run on Linux, or talk to R. 

3. Need more time to review. 
Has any reviewer has downloaded the PR or run the demo notebook?  (Which is there for the benefit of reviewers, and isn’t intended to go in final distribution.)  

How many +1 comments from users would you like to see? 

How much time do you believe is required?

1. Large code base change 
Large code base changes require coordination and cooperation.  This PR necessarily implicates the build scripts, testing code, the SparkInterpreter, etc.  

I have been seeking to coordinate since August. 

I continue to stand ready to do so. 

-Amos


From: moon soo Lee <mo...@apache.org>
Reply: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Date: December 1, 2015 at 1:34:19 AM
To: dev@zeppelin.incubator.apache.org <de...@zeppelin.incubator.apache.org>
Subject:  Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin  

Hi Cos,  

Thanks for opening a discussion.  
My answer about 'Why this PR is open for three months' is  

1. This PR does not passes CI  
2. Not 100% sure this PR has no license issue. (about KniteR)  
3. Need more time to review.  

It's my personal answer, other committers may have different opinion.  


I think the question should be generalized. Because this PR is not the only  
PR that is in impasse. There're more. For example  

Here's some examples that PR is not been merged.  
https://github.com/apache/incubator-zeppelin/pull/53,  
https://github.com/apache/incubator-zeppelin/pull/60  
and so on.  

I can categorize the cases, based on experience of involving Zeppelin  
community from the beginning.  

1. Large code base change  

When contribution has large code base changes, it tend to take more time to  
review and merged. Normally, most contributions merged in 1day~1 week.  
But some contribution has large code base changes take 2~4 weeks. Few  
contribution that has very large code base change take months.  

2. Communication lost  

Sometimes, committer is not responding, or contributor is not responding.  

3. Opinion diverges  

For some changes, included in contribution. When people have different  
opinion and it continue to diverges, those PRs are not been merged.  


I think having a guide such as ping other committer when a committer is not  
responding, and divide contribution into small peaces if possible, would  
help most of the cases.  
Of course committer have to pay attention more to the contribution and  
helping people. That's the first one.  

Thanks,  
moon  

On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org> wrote:  

> To make sure we're on the same page, here are two threads that I found  
> related  
> to this topic.  
>  
> Thread 1:  
> Subject: R?  
> Started on: July 1, 2015  
>  
> Thread 2:  
> Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for  
> Zeppelin  
> Started on: August 13, 2015  
>  
> If you want to fetch these from our archive send emails to  
> dev-thread.1735@zeppelin.incubator.apache.org  
> dev-thread.3573@zeppelin.incubator.apache.org  
>  
> Cos  
>  
> On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:  
> > Guys,  
> >  
> > While catching up on my emails from the last a couple of weeks, this  
> thread  
> > caught my attention. I am not normally paying much attention to the code  
> > reviews traffic from GH, but this one is pretty different as it spans  
> three  
> > months and counting.  
> >  
> > First, here are my five cents:  
> > - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be  
> contributed to  
> > an ASF project this file should simply contain ASL2 text, like in [1]  
> > - r/pom.xml perhaps shouldn't contain a separate <developers> section,  
> but  
> > Zeppelin might have different guidelines on it. Say, Bigtop doesn't  
> > maintain this in the source code, but we have the list of all the  
> > committers on the project's site [2] Every new committer's first  
> commit is  
> > to update the team page ;)  
> > - comments like in  
> r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java  
> >  
> > +/**  
> > + * Created by aelberg on 7/28/15.  
> > + */  
> >  
> > is better to be removed. It has been already discussed here [3]. And  
> the  
> > initial discussion on the topic [4] was linked as well  
> > - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is  
> R-specific  
> > stuff - I have no idea about R, honestly, but  
> >  
> > +License: GPL (>= 2) | BSD_3_clause + file LICENSE  
> > ...  
> > +Author: David B. Dahl  
> >  
> > shouldn't be here, IMO. Normally, if some additional licenses are  
> used,  
> > they have to be listed in the top-level NOTICE file (already there).  
> >  
> > - I am not going to offer any comments on the technical merits of the  
> patch,  
> > as I haven't tried it personally. However, I don't see any serious  
> > technical objections to the functionality in question.  
> >  
> > So, the question is - why the PR is open for three months? I hasn't been  
> able  
> > to get a clear answer. What I found out though is pretty unsettling,  
> really.  
> > The communication on the topic is almost non-existent, except for this  
> sparse  
> > and bitter thread in the GH.  
> >  
> > I would love to hear from the committers what's stopping the acceptance  
> of the  
> > code, besides of the minor issues I've mentioned earlier? What are the  
> reasons for it?  
> > Is there anything wrong with it?  
> > One of the responsibilities of the committers is to make sure the  
> > contributions are reviewed; new contributors are guided and do  
> understand how  
> > the project ticks. The easy feedback flow attracts new people, allowing  
> the  
> > community to grow and thrive.  
> >  
> > I am asking _explicitely_ not to re-start the bickering I have already  
> > seen. At this point I am interested in the purely technical side of this.  
> >  
> > [1] https://github.com/apache/bigtop/blob/master/LICENSE  
> > [2] http://bigtop.apache.org/team-list.html  
> > [3]  
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html  
> > [4] http://s.apache.org/iZl  
> >  
> > With regards,  
> > Cos  
> >  
> > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:  
> > > Github user elbamos commented on the pull request:  
> > >  
> > >  
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411  
> > >  
> > > The current push should resolve some issues with changes in the  
> > > Spark-Zeppelin interface that had created issues for users, as  
> well as  
> > > support for 1.5.1.  
> > >  
> > > Knitr support is improved, and the reason for a separate knitr  
> interpreter may be clearer now.  
> > >  
> > > The amount of code borrowed from rscala is reduced.  
> > >  
> > > I did not address issues with @author tags, or files under the R/  
> > > folder. The reason is, to be blunt, I don't understand what the  
> precise  
> > > concerns actually are.  
> > >  
> > > Please note that I changed .travis.yml to only use spark 1.4 and  
> later.  
> > > I'm sure there is a better way to do it, but I'm just not in a  
> position  
> > > to try to figure out the entire Zeppelin build process.  
> > >  
> > > Integrating this with the rest of zeppelin is going to take some  
> work  
> > > regarding pom's, travis, and so forth. I can do a lot of that,  
> but I'm  
> > > going to need to discuss it with the people who have been "owning"  
> those  
> > > files. There are just too many moving pieces here.  
> > >  
> > > Because of the size of this (which is, unfortunately, necessary),  
> > > posting here is probably not the most efficient way. That is also  
> true  
> > > because certain people chose to use this PR as a forum to air other  
> > > issues. Therefore, it would be better if reviewers e-mail me  
> directly.  
>  
>  
>  

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[moon soo Lee <mo...@apache.org>]

Hi Cos,

Thanks for opening a discussion.
My answer about 'Why this PR is open for three months' is

1. This PR does not passes CI
2. Not 100% sure this PR has no license issue. (about KniteR)
3. Need more time to review.

It's my personal answer, other committers may have different opinion.


I think the question should be generalized. Because this PR is not the only
PR that is in impasse. There're more. For example

Here's some examples that PR is not been merged.
https://github.com/apache/incubator-zeppelin/pull/53,
https://github.com/apache/incubator-zeppelin/pull/60
and so on.

I can categorize the cases, based on experience of involving Zeppelin
community from the beginning.

1. Large code base change

When contribution has large code base changes, it tend to take more time to
review and merged. Normally, most contributions merged in 1day~1 week.
But some contribution has large code base changes take 2~4 weeks. Few
contribution that has very large code base change take months.

2. Communication lost

Sometimes, committer is not responding, or contributor is not responding.

3. Opinion diverges

For some changes, included in contribution. When people have different
opinion and it continue to diverges, those PRs are not been merged.


I think having a guide such as ping other committer when a committer is not
responding, and divide contribution into small peaces if possible, would
help most of the cases.
Of course committer have to pay attention more to the contribution and
helping people. That's the first one.

Thanks,
moon

On Tue, Dec 1, 2015 at 1:54 PM Konstantin Boudnik <co...@apache.org> wrote:

> To make sure we're on the same page, here are two threads that I found
> related
> to this topic.
>
> Thread 1:
>  Subject: R?
>  Started on: July 1, 2015
>
> Thread 2:
>  Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for
> Zeppelin
>  Started on: August 13, 2015
>
> If you want to fetch these from our archive send emails to
>         dev-thread.1735@zeppelin.incubator.apache.org
>         dev-thread.3573@zeppelin.incubator.apache.org
>
> Cos
>
> On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> > Guys,
> >
> > While catching up on my emails from the last a couple of weeks, this
> thread
> > caught my attention. I am not normally paying much attention to the code
> > reviews traffic from GH, but this one is pretty different as it spans
> three
> > months and counting.
> >
> > First, here are my five cents:
> >  - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be
> contributed to
> >    an ASF project this file should simply contain ASL2 text, like in [1]
> >  - r/pom.xml perhaps shouldn't contain a separate <developers> section,
> but
> >    Zeppelin might have different guidelines on it. Say, Bigtop doesn't
> >    maintain this in the source code, but we have the list of all the
> >    committers on the project's site [2] Every new committer's first
> commit is
> >    to update the team page ;)
> >  - comments like in
> r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> >
> >  +/**
> >  + * Created by aelberg on 7/28/15.
> >  + */
> >
> >    is better to be removed. It has been already discussed here [3]. And
> the
> >    initial discussion on the topic [4] was linked as well
> >  - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is
> R-specific
> >    stuff - I have no idea about R, honestly, but
> >
> >     +License: GPL (>= 2) | BSD_3_clause + file LICENSE
> >     ...
> >     +Author: David B. Dahl
> >
> >    shouldn't be here, IMO. Normally, if some additional licenses are
> used,
> >    they have to be listed in the top-level NOTICE file (already there).
> >
> >  - I am not going to offer any comments on the technical merits of the
> patch,
> >    as I haven't tried it personally. However, I don't see any serious
> >    technical objections to the functionality in question.
> >
> > So, the question is - why the PR is open for three months? I hasn't been
> able
> > to get a clear answer. What I found out though is pretty unsettling,
> really.
> > The communication on the topic is almost non-existent, except for this
> sparse
> > and bitter thread in the GH.
> >
> > I would love to hear from the committers what's stopping the acceptance
> of the
> > code, besides of the minor issues I've mentioned earlier? What are the
> reasons for it?
> > Is there anything wrong with it?
> > One of the responsibilities of the committers is to make sure the
> > contributions are reviewed; new contributors are guided and do
> understand how
> > the project ticks. The easy feedback flow attracts new people, allowing
> the
> > community to grow and thrive.
> >
> > I am asking _explicitely_ not to re-start the bickering I have already
> > seen. At this point I am interested in the purely technical side of this.
> >
> > [1] https://github.com/apache/bigtop/blob/master/LICENSE
> > [2] http://bigtop.apache.org/team-list.html
> > [3]
> http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> > [4] http://s.apache.org/iZl
> >
> > With regards,
> >   Cos
> >
> > On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > > Github user elbamos commented on the pull request:
> > >
> > >
> https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> > >
> > >     The current push should resolve some issues with changes in the
> > >     Spark-Zeppelin interface that had created issues for users, as
> well as
> > >     support for 1.5.1.
> > >
> > >     Knitr support is improved, and the reason for a separate knitr
> interpreter may be clearer now.
> > >
> > >     The amount of code borrowed from rscala is reduced.
> > >
> > >     I did not address issues with @author tags, or files under the R/
> > >     folder.  The reason is, to be blunt, I don't understand what the
> precise
> > >     concerns actually are.
> > >
> > >     Please note that I changed .travis.yml to only use spark 1.4 and
> later.
> > >     I'm sure there is a better way to do it, but I'm just not in a
> position
> > >     to try to figure out the entire Zeppelin build process.
> > >
> > >     Integrating this with the rest of zeppelin is going to take some
> work
> > >     regarding pom's, travis, and so forth.  I can do a lot of that,
> but I'm
> > >     going to need to discuss it with the people who have been "owning"
> those
> > >     files.  There are just too many moving pieces here.
> > >
> > >     Because of the size of this (which is, unfortunately, necessary),
> > >     posting here is probably not the most efficient way. That is also
> true
> > >     because certain people chose to use this PR as a forum to air other
> > >     issues.  Therefore, it would be better if reviewers e-mail me
> directly.
>
>
>

Re: contributions impasse. Was: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin

[Konstantin Boudnik <co...@apache.org>]

To make sure we're on the same page, here are two threads that I found related
to this topic. 

Thread 1:
 Subject: R?
 Started on: July 1, 2015

Thread 2:
 Subject: [GitHub] incubator-zeppelin pull request: R Interpreter for Zeppelin
 Started on: August 13, 2015

If you want to fetch these from our archive send emails to
        dev-thread.1735@zeppelin.incubator.apache.org
        dev-thread.3573@zeppelin.incubator.apache.org

Cos

On Mon, Nov 30, 2015 at 06:27PM, Konstantin Boudnik wrote:
> Guys,
> 
> While catching up on my emails from the last a couple of weeks, this thread
> caught my attention. I am not normally paying much attention to the code
> reviews traffic from GH, but this one is pretty different as it spans three
> months and counting.
> 
> First, here are my five cents:
>  - r/R/rzeppelin/LICENSE is wrong: if the code is aimed to be contributed to
>    an ASF project this file should simply contain ASL2 text, like in [1]
>  - r/pom.xml perhaps shouldn't contain a separate <developers> section, but
>    Zeppelin might have different guidelines on it. Say, Bigtop doesn't
>    maintain this in the source code, but we have the list of all the
>    committers on the project's site [2] Every new committer's first commit is
>    to update the team page ;)
>  - comments like in r/src/main/java/org/apache/zeppelin/rinterpreter/KnitR.java
> 
>  +/**
>  + * Created by aelberg on 7/28/15.
>  + */ 
>    
>    is better to be removed. It has been already discussed here [3]. And the
>    initial discussion on the topic [4] was linked as well
>  - same goes to r/R/rzeppelin/DESCRIPTION. I am not sure if this is R-specific
>    stuff - I have no idea about R, honestly, but 
> 
>     +License: GPL (>= 2) | BSD_3_clause + file LICENSE     
>     ...
>     +Author: David B. Dahl
>    
>    shouldn't be here, IMO. Normally, if some additional licenses are used,
>    they have to be listed in the top-level NOTICE file (already there).
> 
>  - I am not going to offer any comments on the technical merits of the patch,
>    as I haven't tried it personally. However, I don't see any serious
>    technical objections to the functionality in question. 
> 
> So, the question is - why the PR is open for three months? I hasn't been able
> to get a clear answer. What I found out though is pretty unsettling, really.
> The communication on the topic is almost non-existent, except for this sparse
> and bitter thread in the GH.
> 
> I would love to hear from the committers what's stopping the acceptance of the
> code, besides of the minor issues I've mentioned earlier? What are the reasons for it?
> Is there anything wrong with it?
> One of the responsibilities of the committers is to make sure the
> contributions are reviewed; new contributors are guided and do understand how
> the project ticks. The easy feedback flow attracts new people, allowing the
> community to grow and thrive.
> 
> I am asking _explicitely_ not to re-start the bickering I have already
> seen. At this point I am interested in the purely technical side of this.
> 
> [1] https://github.com/apache/bigtop/blob/master/LICENSE
> [2] http://bigtop.apache.org/team-list.html
> [3] http://apache-nifi-developer-list.39713.n7.nabble.com/author-tags-td1335.html
> [4] http://s.apache.org/iZl
> 
> With regards,
>   Cos
> 
> On Mon, Nov 16, 2015 at 11:06PM, elbamos wrote:
> > Github user elbamos commented on the pull request:
> > 
> >     https://github.com/apache/incubator-zeppelin/pull/208#issuecomment-157203411
> >   
> >     The current push should resolve some issues with changes in the
> >     Spark-Zeppelin interface that had created issues for users, as well as
> >     support for 1.5.1.
> >     
> >     Knitr support is improved, and the reason for a separate knitr interpreter may be clearer now. 
> >     
> >     The amount of code borrowed from rscala is reduced. 
> >     
> >     I did not address issues with @author tags, or files under the R/
> >     folder.  The reason is, to be blunt, I don't understand what the precise
> >     concerns actually are.  
> >     
> >     Please note that I changed .travis.yml to only use spark 1.4 and later.
> >     I'm sure there is a better way to do it, but I'm just not in a position
> >     to try to figure out the entire Zeppelin build process.  
> >     
> >     Integrating this with the rest of zeppelin is going to take some work
> >     regarding pom's, travis, and so forth.  I can do a lot of that, but I'm
> >     going to need to discuss it with the people who have been "owning" those
> >     files.  There are just too many moving pieces here.  
> >     
> >     Because of the size of this (which is, unfortunately, necessary),
> >     posting here is probably not the most efficient way. That is also true
> >     because certain people chose to use this PR as a forum to air other
> >     issues.  Therefore, it would be better if reviewers e-mail me directly.