You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@climate.apache.org by Omkar Reddy <om...@gmail.com> on 2020/09/03 17:56:34 UTC
Re: [VOTE] Board Resolution to move Apache OCW to the Attic WAS Re:
[DISCUSS] Future of Apache OCW
+1, apologies for the late response.
On Mon, 3 Aug 2020 at 10:50, Michael Joyce <ml...@gmail.com> wrote:
> +1
>
>
> On Fri, Jul 31, 2020 at 2:55 PM lewis john mcgibbney <le...@apache.org>
> wrote:
>
> > Hi dev@climate.a.o,
> >
> > This thread opens a VOTE thread for the creation of a Board Resolution
> > to move Apache OCW to the Attic.
> >
> > I will leave this VOTE open for a minimum of 72 hours before
> > attempting to insert the resolution into the next Board agenda.
> >
> >
> > ?. Terminate the Apache Open Climate Workbench (OCW) Project
> >
> > WHEREAS, the Project Management Committee of the Apache OCW
> > project has chosen by vote to recommend moving the project to the
> > Attic; and
> >
> > WHEREAS, the Board of Directors deems it no longer in the best
> > interest of the Foundation to continue the Apache OCW project
> > due to inactivity;
> >
> > NOW, THEREFORE, BE IT RESOLVED, that the Apache OCW
> > project is hereby terminated; and be it further
> >
> > RESOLVED, that the Attic PMC be and hereby is tasked with
> > oversight over the software developed by the Apache OCW
> > Project; and be it further
> >
> > RESOLVED, that the office of "Vice President, Apache OCW" is
> > hereby terminated; and be it further
> >
> > RESOLVED, that the Apache OCW PMC is hereby terminated.
> >
> >
> > Thanks in advance for VOTE'ing Here is my +1
> > lewismc
> >
>
Re: [EXTERNAL] Re: [VOTE] Board Resolution to move Apache OCW to the Attic WAS Re: [DISCUSS] Future of Apache OCW
Posted by Lewis John McGibbney <le...@apache.org>.
Hi Kyo,
Thanks for your input however I am puzzled with your tone!
On 2020/09/03 22:36:34, "Lee, Kyo (US 398L)" <hu...@jpl.nasa.gov.INVALID> wrote:
> 'No' from me.
> I do not answer to any angry, emotional email suddenly written by members who had not given any constructive resolution.
>
Can you elaborate? I am not able to infer anything useful from this.
Thank you
lewismc
Re: RE: [EXTERNAL] Re: [VOTE] Board Resolution to move Apache OCW to the Attic WAS Re: [DISCUSS] Future of Apache OCW
Posted by Lewis John McGibbney <le...@apache.org>.
Hi Alex,
Thanks for your input. Please see my responses inline
On 2020/09/04 00:39:54, "Goodman, Alexander (US 398K)" <al...@jpl.nasa.gov.INVALID> wrote:
> I am sorry for not responding to this sooner. I am tentatively voting -1 against retirement (even though I think my vote makes little difference at this point if it still counts).
It absolutely does make a difference as it enables you express your opinion. That always counts :)
> Frankly, when I initially saw this lengthy series of emails I wasn’t sure how I should respond since I am pretty torn between both sides. I am not really sure if my vote holds weight anymore since there are now already 3 votes in favor of retirement, but I want to clarify on some points that Lewis and Jimmy have previously stated as reasons for voting in favor retirement, since I think they do not cover the full story. This will be lengthy so you may not feel the need to read through all of this, but in summary I agree that activity is an issue, but I disagree with the characterization of some that we have not been transparent enough about our future plans.
> First, I want to elaborate more on the refactoring that Kyo has mentioned. This was an effort I had initially started in early 2018. This was originally started by internal funding opportunities at JPL to create a replacement for OCW which would use more modern libraries (xarray and dask), since it would not only make it easy for us to handle big data use cases as we had long desired but it would also make it easier for us to maintain the codebase in the long run since it would make it much smaller. At JPL we have internally called this prototype the “Big Climate Data Pipeline” (bcdp), and per JPL policy it was not initially open source. We had to formally go through a process (applying for a New Technology Release (NTR) and receiving approval. We did not receive the formal approval to release BCDP to open source until late last year. In the end, since I am the only person who is actively developing it, I decided it would be more convenient to release it in its own github repos
itory separate from the main one (apache/climate) as this would allow me to make new releases very quickly which was necessary for the work I was doing in testing it on a Kubernetes cluster. Maybe it’s my misunderstanding, but some of the previous conversation seems to imply that we were not being transparent and keeping everything a secret which is not the case. We discussed BCDP many times on the mailing list and the plan to eventually transition the BCDP codebase was also made into a formal JIRA issue right around the time BCDP was released to open source for the first time.
> https://github.com/bcdp/bcdp
> If taking this approach goes against ASF principles, then I apologize, but I do not recall being told that this was the case.
There si no issue with any of the above Alex. The issue is the Apache OCW community and codebase is extremely quiet and we are therefore proposing it for retirement.
This entire 'thing' is about community. Without community there is no real project. It's as simple as that.
Nothing you see to have done is 'wrong' or intentionally going 'against' anything.
> So with all that out of the way, let’s talk about the other issue which is our low activity. This is an undeniable truth and reality that I don’t see changing anytime soon whether BCDP gets integrated into OCW or not. My stance from our very discussion on this last year hasn’t changed, and I encourage anyone who has read this far to also see what I have said back then:
> http://mail-archives.apache.org/mod_mbox/climate-dev/201911.mbox/browser
> The amount of discussion on this mailing list is being used as one metric of activity which I think is understandable given that this is an open source project, but in practice we have only two core developers (me and Kyo) who both work at the same place. We have tried our best to make our plans transparent by making the appropriate JIRA issues, but at the end of the day we have also had many offline discussions with each other about the project out of convenience. We’ve had one non-JPL person (Michael Anderson) contribute every now and then with a few bug fixes, but otherwise this has been the reality of this project for a very long time. I don’t know if the expectation for activity is for us to be as vibrant as we were in 2012-2015, but that’s pretty unrealistic given that we just don’t have sufficient time or funding, and that is also one of the reasons BCDP integration has been so slow, and I don’t see this changing anytime soon.
This happens Alex. People move on, some projects survive and other don't. Software sustainability is a huge issue. It's no ones fault here. Proposing OCW for retirement has nothing to do with anyone doing anything wrong. It is simply a realization that the community has seen a sustained period of inactivity which indicates that no-one is using the software or actively developing it.
> So with all that said, although I currently am voting against retirement, I would like to stress that I am just a PMC member and am ignorant on many of the policies for ASF projects, and am unsure if OCW is the right fit given that we are ultimately a small project with a niche userbase, and I can’t really argue the fact that our activity is low. I would appreciate if Lewis could provide his take on this question one more time.
There are multiple metrics one can use to gauge community health. Low mailing list/comms activity is just one. At the ASF, typically the addition of Committers and/or PMC members is another. In the case of OCW the facts are as follows
- No new PMC members. Last addition was Ibrahim Jarif on 2016-04-25.
- No new committers. Last addition was Christopher Douglas on 2016-04-26.
To many of us, this indicates that the project has not been attracting new committers or PMC members for atleast 4 years.
Let's also look at project activity in terms of software releases. 1.3.0 was released on 2018-04-23. This has over 2 years ago and we've seen very little activity since then.
At this stage, even with the work you are doing on BCDP, I still don't think that this justifies OCW continuing the way it has.
Does this make sense? Do you agree/disagree with any of this?
Thank you
lewismc
Re: [EXTERNAL] Re: [VOTE] Board Resolution to move Apache OCW to the
Attic WAS Re: [DISCUSS] Future of Apache OCW
Posted by Michael Joyce <jo...@apache.org>.
Hey Alex,
First off, your vote always matters. You're a member of this community, a
committer, and a PMC member. Thank you for chiming in with information,
your thoughts and, most importantly, actually participating in the
governance of the project in a constructive way.
It's cool that y'all want to push changes from BCDP out to OCW and I
understand completely why development of it occured the way it did. As you
noted, it doesn't really change the issues that OCW faces. In many ways it
is the problem. The project moving to the Attic doesn't mean we did
something wrong, or we've failed in our endeavors, or anything negative.
It's an acknowledgement that the ASF project has reached its end of life
and gives a place for the project to live without further development (at
the ASF), without PMC oversight, and without impact to users of the
existing code base. Moving to the attic doesn't mean we rm -rf the code
base and pretend OCW never existed and it doesn't change the licensing of
the current code base.
I'd encourage you to read the AFS info on the Attic.
https://attic.apache.org/
Here's an example. Apache Wink went to the Attic in 2017
https://attic.apache.org/projects/wink.html
You can still view and fork the code off Github if you want to continue
development
https://github.com/apache/attic-wink
"""
When should a project move to the Attic?
Projects whose PMC are unable to muster 3 votes for a release, who have no
active committers or are unable to fulfill their reporting duties to the
board are all good candidates for the Attic.
"""
- OCW hasn't had a new committer / PMC member since ~ 2016-04-25
- OCW hasn't had a release since 2018-04-23
- OCW has had no development activity in ~10 months and very little for the
last 1-2 years.
- A thread discussing the project's health issues and ultimately a vote to
move to the Attic couldn't garner enough PMC votes so the board had to be
asked to step in and offer assistance. Note that this means you'd have the
same problem if you want to tag a release as well ...
- OCW has, at best, 2 active members who work at the same place and whose
involvement in the task seems entirely dependent on funding from their
employer (I'll openly admit this is my interpretation of your words and
it's possible I'm misrepresenting this. If so I apologize and please feel
free to correct me). When that funding stops what happens?
The project state isn't healthy. There isn't a community growing here. It's
been like this for a long time and I can't imagine this is going to change.
Again, I want to reiterate, this isn't something that we caused recently.
This has been an issue for a looooong time (certainly while I was the PMC
Chair).
That's ok though. The project can move to the Attic, you can fork it, etc.
That's what the Attic is for after all. Note, this isn't invalidating any
of the -1 votes against going to the Attic. I'm not the BDFL of OCW. But if
we're going to keep OCW out of the Attic there needs to be something done.
The community needs to exist outside of internal JPL funding of 2 people.
Concerted effort needs to be made to pull new developers in and build a
community. Development should be actually occurring, releases need made,
communication should be happening on the list (if it doesn't happen there
it doesn't matter), etc. If that can't or won't happen then why aren't we
retiring OCW? If we can't even get enough votes to release the code base
then why not retire it?
As an aside, sometimes a project might not be a good fit for the ASF model.
I manage JPL projects which are open source and fall under that umbrella
and it looks like BCDP may be taking that approach as well. There's nothing
wrong with that.
On Thu, Sep 3, 2020 at 5:40 PM Goodman, Alexander (US 398K)
<al...@jpl.nasa.gov.invalid> wrote:
> I am sorry for not responding to this sooner. I am tentatively voting -1
> against retirement (even though I think my vote makes little difference at
> this point if it still counts).
> Frankly, when I initially saw this lengthy series of emails I wasn’t sure
> how I should respond since I am pretty torn between both sides. I am not
> really sure if my vote holds weight anymore since there are now already 3
> votes in favor of retirement, but I want to clarify on some points that
> Lewis and Jimmy have previously stated as reasons for voting in favor
> retirement, since I think they do not cover the full story. This will be
> lengthy so you may not feel the need to read through all of this, but in
> summary I agree that activity is an issue, but I disagree with the
> characterization of some that we have not been transparent enough about our
> future plans.
> First, I want to elaborate more on the refactoring that Kyo has mentioned.
> This was an effort I had initially started in early 2018. This was
> originally started by internal funding opportunities at JPL to create a
> replacement for OCW which would use more modern libraries (xarray and
> dask), since it would not only make it easy for us to handle big data use
> cases as we had long desired but it would also make it easier for us to
> maintain the codebase in the long run since it would make it much smaller.
> At JPL we have internally called this prototype the “Big Climate Data
> Pipeline” (bcdp), and per JPL policy it was not initially open source. We
> had to formally go through a process (applying for a New Technology Release
> (NTR) and receiving approval. We did not receive the formal approval to
> release BCDP to open source until late last year. In the end, since I am
> the only person who is actively developing it, I decided it would be more
> convenient to release it in its own github repository separate from the
> main one (apache/climate) as this would allow me to make new releases very
> quickly which was necessary for the work I was doing in testing it on a
> Kubernetes cluster. Maybe it’s my misunderstanding, but some of the
> previous conversation seems to imply that we were not being transparent and
> keeping everything a secret which is not the case. We discussed BCDP many
> times on the mailing list and the plan to eventually transition the BCDP
> codebase was also made into a formal JIRA issue right around the time BCDP
> was released to open source for the first time.
> https://github.com/bcdp/bcdp
> If taking this approach goes against ASF principles, then I apologize, but
> I do not recall being told that this was the case.
> So with all that out of the way, let’s talk about the other issue which is
> our low activity. This is an undeniable truth and reality that I don’t see
> changing anytime soon whether BCDP gets integrated into OCW or not. My
> stance from our very discussion on this last year hasn’t changed, and I
> encourage anyone who has read this far to also see what I have said back
> then:
> http://mail-archives.apache.org/mod_mbox/climate-dev/201911.mbox/browser
> The amount of discussion on this mailing list is being used as one metric
> of activity which I think is understandable given that this is an open
> source project, but in practice we have only two core developers (me and
> Kyo) who both work at the same place. We have tried our best to make our
> plans transparent by making the appropriate JIRA issues, but at the end of
> the day we have also had many offline discussions with each other about the
> project out of convenience. We’ve had one non-JPL person (Michael Anderson)
> contribute every now and then with a few bug fixes, but otherwise this has
> been the reality of this project for a very long time. I don’t know if the
> expectation for activity is for us to be as vibrant as we were in
> 2012-2015, but that’s pretty unrealistic given that we just don’t have
> sufficient time or funding, and that is also one of the reasons BCDP
> integration has been so slow, and I don’t see this changing anytime soon.
> So with all that said, although I currently am voting against retirement,
> I would like to stress that I am just a PMC member and am ignorant on many
> of the policies for ASF projects, and am unsure if OCW is the right fit
> given that we are ultimately a small project with a niche userbase, and I
> can’t really argue the fact that our activity is low. I would appreciate if
> Lewis could provide his take on this question one more time.
> Thanks,
> Alex
> -----Original Message-----
> From: Lee, Kyo (US 398L) <hu...@jpl.nasa.gov.INVALID>
> Sent: Thursday, September 3, 2020 4:37 PM
> To: dev@climate.apache.org
> Subject: Re: [EXTERNAL] Re: [VOTE] Board Resolution to move Apache OCW to
> the Attic WAS Re: [DISCUSS] Future of Apache OCW
>
> 'No' from me.
> I do not answer to any angry, emotional email suddenly written by members
> who had not given any constructive resolution.
>
> Best regards,
> Kyo
>
> On 9/3/20, 10:57 AM, "Omkar Reddy" <om...@gmail.com> wrote:
>
> +1, apologies for the late response.
>
> On Mon, 3 Aug 2020 at 10:50, Michael Joyce <ml...@gmail.com> wrote:
>
> > +1
> >
> >
> > On Fri, Jul 31, 2020 at 2:55 PM lewis john mcgibbney <
> lewismc@apache.org>
> > wrote:
> >
> > > Hi dev@climate.a.o,
> > >
> > > This thread opens a VOTE thread for the creation of a Board
> Resolution
> > > to move Apache OCW to the Attic.
> > >
> > > I will leave this VOTE open for a minimum of 72 hours before
> > > attempting to insert the resolution into the next Board agenda.
> > >
> > >
> > > ?. Terminate the Apache Open Climate Workbench (OCW) Project
> > >
> > > WHEREAS, the Project Management Committee of the Apache OCW
> > > project has chosen by vote to recommend moving the project
> to the
> > > Attic; and
> > >
> > > WHEREAS, the Board of Directors deems it no longer in the
> best
> > > interest of the Foundation to continue the Apache OCW
> project
> > > due to inactivity;
> > >
> > > NOW, THEREFORE, BE IT RESOLVED, that the Apache OCW
> > > project is hereby terminated; and be it further
> > >
> > > RESOLVED, that the Attic PMC be and hereby is tasked with
> > > oversight over the software developed by the Apache OCW
> > > Project; and be it further
> > >
> > > RESOLVED, that the office of "Vice President, Apache OCW" is
> > > hereby terminated; and be it further
> > >
> > > RESOLVED, that the Apache OCW PMC is hereby terminated.
> > >
> > >
> > > Thanks in advance for VOTE'ing Here is my +1
> > > lewismc
> > >
> >
>
RE: [EXTERNAL] Re: [VOTE] Board Resolution to move Apache OCW to the
Attic WAS Re: [DISCUSS] Future of Apache OCW
Posted by "Goodman, Alexander (US 398K)" <al...@jpl.nasa.gov.INVALID>.
I am sorry for not responding to this sooner. I am tentatively voting -1 against retirement (even though I think my vote makes little difference at this point if it still counts).
Frankly, when I initially saw this lengthy series of emails I wasn’t sure how I should respond since I am pretty torn between both sides. I am not really sure if my vote holds weight anymore since there are now already 3 votes in favor of retirement, but I want to clarify on some points that Lewis and Jimmy have previously stated as reasons for voting in favor retirement, since I think they do not cover the full story. This will be lengthy so you may not feel the need to read through all of this, but in summary I agree that activity is an issue, but I disagree with the characterization of some that we have not been transparent enough about our future plans.
First, I want to elaborate more on the refactoring that Kyo has mentioned. This was an effort I had initially started in early 2018. This was originally started by internal funding opportunities at JPL to create a replacement for OCW which would use more modern libraries (xarray and dask), since it would not only make it easy for us to handle big data use cases as we had long desired but it would also make it easier for us to maintain the codebase in the long run since it would make it much smaller. At JPL we have internally called this prototype the “Big Climate Data Pipeline” (bcdp), and per JPL policy it was not initially open source. We had to formally go through a process (applying for a New Technology Release (NTR) and receiving approval. We did not receive the formal approval to release BCDP to open source until late last year. In the end, since I am the only person who is actively developing it, I decided it would be more convenient to release it in its own github repository separate from the main one (apache/climate) as this would allow me to make new releases very quickly which was necessary for the work I was doing in testing it on a Kubernetes cluster. Maybe it’s my misunderstanding, but some of the previous conversation seems to imply that we were not being transparent and keeping everything a secret which is not the case. We discussed BCDP many times on the mailing list and the plan to eventually transition the BCDP codebase was also made into a formal JIRA issue right around the time BCDP was released to open source for the first time.
https://github.com/bcdp/bcdp
If taking this approach goes against ASF principles, then I apologize, but I do not recall being told that this was the case.
So with all that out of the way, let’s talk about the other issue which is our low activity. This is an undeniable truth and reality that I don’t see changing anytime soon whether BCDP gets integrated into OCW or not. My stance from our very discussion on this last year hasn’t changed, and I encourage anyone who has read this far to also see what I have said back then:
http://mail-archives.apache.org/mod_mbox/climate-dev/201911.mbox/browser
The amount of discussion on this mailing list is being used as one metric of activity which I think is understandable given that this is an open source project, but in practice we have only two core developers (me and Kyo) who both work at the same place. We have tried our best to make our plans transparent by making the appropriate JIRA issues, but at the end of the day we have also had many offline discussions with each other about the project out of convenience. We’ve had one non-JPL person (Michael Anderson) contribute every now and then with a few bug fixes, but otherwise this has been the reality of this project for a very long time. I don’t know if the expectation for activity is for us to be as vibrant as we were in 2012-2015, but that’s pretty unrealistic given that we just don’t have sufficient time or funding, and that is also one of the reasons BCDP integration has been so slow, and I don’t see this changing anytime soon.
So with all that said, although I currently am voting against retirement, I would like to stress that I am just a PMC member and am ignorant on many of the policies for ASF projects, and am unsure if OCW is the right fit given that we are ultimately a small project with a niche userbase, and I can’t really argue the fact that our activity is low. I would appreciate if Lewis could provide his take on this question one more time.
Thanks,
Alex
-----Original Message-----
From: Lee, Kyo (US 398L) <hu...@jpl.nasa.gov.INVALID>
Sent: Thursday, September 3, 2020 4:37 PM
To: dev@climate.apache.org
Subject: Re: [EXTERNAL] Re: [VOTE] Board Resolution to move Apache OCW to the Attic WAS Re: [DISCUSS] Future of Apache OCW
'No' from me.
I do not answer to any angry, emotional email suddenly written by members who had not given any constructive resolution.
Best regards,
Kyo
On 9/3/20, 10:57 AM, "Omkar Reddy" <om...@gmail.com> wrote:
+1, apologies for the late response.
On Mon, 3 Aug 2020 at 10:50, Michael Joyce <ml...@gmail.com> wrote:
> +1
>
>
> On Fri, Jul 31, 2020 at 2:55 PM lewis john mcgibbney <le...@apache.org>
> wrote:
>
> > Hi dev@climate.a.o,
> >
> > This thread opens a VOTE thread for the creation of a Board Resolution
> > to move Apache OCW to the Attic.
> >
> > I will leave this VOTE open for a minimum of 72 hours before
> > attempting to insert the resolution into the next Board agenda.
> >
> >
> > ?. Terminate the Apache Open Climate Workbench (OCW) Project
> >
> > WHEREAS, the Project Management Committee of the Apache OCW
> > project has chosen by vote to recommend moving the project to the
> > Attic; and
> >
> > WHEREAS, the Board of Directors deems it no longer in the best
> > interest of the Foundation to continue the Apache OCW project
> > due to inactivity;
> >
> > NOW, THEREFORE, BE IT RESOLVED, that the Apache OCW
> > project is hereby terminated; and be it further
> >
> > RESOLVED, that the Attic PMC be and hereby is tasked with
> > oversight over the software developed by the Apache OCW
> > Project; and be it further
> >
> > RESOLVED, that the office of "Vice President, Apache OCW" is
> > hereby terminated; and be it further
> >
> > RESOLVED, that the Apache OCW PMC is hereby terminated.
> >
> >
> > Thanks in advance for VOTE'ing Here is my +1
> > lewismc
> >
>
Re: [EXTERNAL] Re: [VOTE] Board Resolution to move Apache OCW to the
Attic WAS Re: [DISCUSS] Future of Apache OCW
Posted by "Lee, Kyo (US 398L)" <hu...@jpl.nasa.gov.INVALID>.
'No' from me.
I do not answer to any angry, emotional email suddenly written by members who had not given any constructive resolution.
Best regards,
Kyo
On 9/3/20, 10:57 AM, "Omkar Reddy" <om...@gmail.com> wrote:
+1, apologies for the late response.
On Mon, 3 Aug 2020 at 10:50, Michael Joyce <ml...@gmail.com> wrote:
> +1
>
>
> On Fri, Jul 31, 2020 at 2:55 PM lewis john mcgibbney <le...@apache.org>
> wrote:
>
> > Hi dev@climate.a.o,
> >
> > This thread opens a VOTE thread for the creation of a Board Resolution
> > to move Apache OCW to the Attic.
> >
> > I will leave this VOTE open for a minimum of 72 hours before
> > attempting to insert the resolution into the next Board agenda.
> >
> >
> > ?. Terminate the Apache Open Climate Workbench (OCW) Project
> >
> > WHEREAS, the Project Management Committee of the Apache OCW
> > project has chosen by vote to recommend moving the project to the
> > Attic; and
> >
> > WHEREAS, the Board of Directors deems it no longer in the best
> > interest of the Foundation to continue the Apache OCW project
> > due to inactivity;
> >
> > NOW, THEREFORE, BE IT RESOLVED, that the Apache OCW
> > project is hereby terminated; and be it further
> >
> > RESOLVED, that the Attic PMC be and hereby is tasked with
> > oversight over the software developed by the Apache OCW
> > Project; and be it further
> >
> > RESOLVED, that the office of "Vice President, Apache OCW" is
> > hereby terminated; and be it further
> >
> > RESOLVED, that the Apache OCW PMC is hereby terminated.
> >
> >
> > Thanks in advance for VOTE'ing Here is my +1
> > lewismc
> >
>