You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flex.apache.org by Bertrand Delacretaz <bd...@apache.org> on 2012/04/05 16:12:44 UTC
Board report, "Apache internals are starting to become a problem" ?
Hi,
Thanks very much to Alex for submitting our board report in time at
http://wiki.apache.org/incubator/April2012 - I'd like to briefly
discuss the following two paragraphs that start with "Apache internals
are starting to become a problem", with my mentor hat on.
There might still be time to amend these statements if we want to, but
please notify general@incubator.apache.org if that happens, as I have
already signed off the report.
Quoting from that report:
> We submitted the software grant and code on Friday
> PM, but the secretary was unable to record the software grant before
> the weekend. That meant that Infra couldn't import the code since they
> can only do that on weekends, and that delayed the project by a whole
> week...
I don't think it's fair to blame the secretary or infra for that - a
few hours delay in recording a code grant is certainly acceptable, and
infra requiring big svn imports to happen on weekends for minimal
disruption sounds totally reasonable to me as well.
Those big svn imports do not happen every day, so it seems ok to have
to plan them a bit in advance.
I agree that the overall delays in getting the code in svn are
frustrating, but secretary/infra is only a minor part in that IMO. And
in the meantime people can play with the copy that Carol committed
anyway IIUC.
My suggestion: tone that down a bit, and remove the secretary bit completely.
>...The same holds true for the bug database as well. We have been waiting
> for the bug database since Feb 1. A gating concern is that our import
> of 30000 bugs might take down the main JIRA instance. Having distributed
> JIRA instances would scale better...
Here I agree that INFRA-4380 seems to be stalled. At some point IIRC
we discussed the alternative of importing issues via jira's RESTful or
other API, throttling to avoid putting too much load on the instance.
Has that been pursued?
-Bertrand (mostly offline in the next few days, might be slow to follow up)
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Omar Gonzalez <om...@gmail.com>.
The delay issues with getting code into SVN and getting bugs into JIRA are
the single biggest detriment to this project at the moment and it is very
much affecting this community as it is trying to move on from the events
that landed this project in Apache's hands to begin with. I feel like it's
putting a huge damper on our ability to both move this project forward and
keep the interest of the community. I understand Apache Infra has a lot of
projects to attend, but that is precisely the problem. Apache projects
should have more autonomy around their databases, months of waiting for a
JIRA import is just unacceptable, specifically for a project with a
fractured community that is trying to build momentum and its community in
tact.
-omar@apache.org
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Fri, Apr 6, 2012 at 6:08 AM, Alex Harui <ah...@adobe.com> wrote:
> ...BTW, I modified the Flex Podling Report. Take a look and see if you think
> it is better....
Thanks very much, I agree with the revised report, exposes the
problems while avoiding pointing fingers. I have removed my signing
statement.
We can discuss the rest after the holiday as far as I'm concerned.
-Bertrand
Re: Board report, "Apache internals are starting to become a
problem" ?
Posted by Alex Harui <ah...@adobe.com>.
On 4/5/12 6:50 PM, "Greg Reddin" <gr...@gmail.com> wrote:
>
> Perhaps companies like Adobe could donate the money for an extra FT infra
> contractor along with their projects? Just a thought.
Interesting, what kind of $ is that?
BTW, I modified the Flex Podling Report. Take a look and see if you think
it is better.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Greg Reddin <gr...@gmail.com>.
Sent from my mobile device.
On Apr 5, 2012, at 7:19 PM, Igor Costa <ig...@gmail.com> wrote:
> Alex words are not bound directly to any person, but the lack of infra
> people who are in charge of doing supposed stuff.
While I agree to an extent it is imperative for us to understand how apache works. The problem with lodging complaints toward anything at apache is we don't really have a voice unless we are doing something to fix the problem. If someone comes on this list and starts complaining about what we don't have done in 3 months we point them to svn and Jira and say "patches welcome". You're mistaken if you think infra is much different. The difference with infra us that it takes a lot longer to build enough merit to gain trust. This is not because they are a closed club, but because they've had too many people come in and start something then leave then to support it. It's part of the culture around here. The fact that it's a mostly volunteer org instead of a corporate org is part of what makes apache great. Unfortunately the way infra is run is a side effect of that.
Perhaps companies like Adobe could donate the money for an extra FT infra contractor along with their projects? Just a thought.
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Igor Costa <ig...@gmail.com>.
I do agree with Alex Harui report.
Apache should solve this problem soon, we have to focus on product
development, theses infra problems, should not exists.
Alex words are not bound directly to any person, but the lack of infra
people who are in charge of doing supposed stuff.
Regards
----------------------------
Igor Costa
www.igorcosta.com
www.igorcosta.org
On Thu, Apr 5, 2012 at 6:51 PM, Greg Reddin <gr...@gmail.com> wrote:
> On Thu, Apr 5, 2012 at 4:35 PM, Alex Harui <ah...@adobe.com> wrote:
> > I will try to re-word the report to make that argument without targeting
> > secretary and infra.
>
> Thanks, I think that will be good.
>
> > I believe the next step was to try to make the "parity" release. Carol
> said
> > she has the build script ready to go on her whiteboard. We can't really
> cut
> > a release from her whiteboard can we? That's why I think we're waiting
> on
> > SVN import to trunk.
>
> Yeah, I think we should wait to make the release when everything is in
> trunk.
>
> > Is there something else we need to do before trying to make this release?
>
> I'd be interested in seeing what came out if we did a dry run release
> from trunk. Then we'd at least know what we'd have to change before we
> create a real release.
>
> Greg
>
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Greg Reddin <gr...@gmail.com>.
On Thu, Apr 5, 2012 at 4:35 PM, Alex Harui <ah...@adobe.com> wrote:
> I will try to re-word the report to make that argument without targeting
> secretary and infra.
Thanks, I think that will be good.
> I believe the next step was to try to make the "parity" release. Carol said
> she has the build script ready to go on her whiteboard. We can't really cut
> a release from her whiteboard can we? That's why I think we're waiting on
> SVN import to trunk.
Yeah, I think we should wait to make the release when everything is in trunk.
> Is there something else we need to do before trying to make this release?
I'd be interested in seeing what came out if we did a dry run release
from trunk. Then we'd at least know what we'd have to change before we
create a real release.
Greg
Re: Board report, "Apache internals are starting to become a
problem" ?
Posted by Alex Harui <ah...@adobe.com>.
On 4/5/12 2:39 PM, "Omar Gonzalez" <om...@gmail.com> wrote:
> Are you saying you're ok with trying to cut a release without having
> Mustella tests to run against the SDK? Just making sure.
Carol and/or I will take the package and run mustella against it.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Omar Gonzalez <om...@gmail.com>.
>
> Is there something else we need to do before trying to make this release?
> I
> know there are things we need to get done before a release would ever get
> signed off (like settle the trademark issue), but it seemed like everything
> is waiting on being able to run the script from trunk.
>
> --
> Alex Harui
> Flex SDK Team
> Adobe Systems, Inc.
> http://blogs.adobe.com/aharui
>
Are you saying you're ok with trying to cut a release without having
Mustella tests to run against the SDK? Just making sure.
-omar
Re: Board report, "Apache internals are starting to become a
problem" ?
Posted by Alex Harui <ah...@adobe.com>.
On 4/5/12 2:20 PM, "Greg Reddin" <gr...@gmail.com> wrote:
> On Thu, Apr 5, 2012 at 4:03 PM, Alex Harui <ah...@adobe.com> wrote:
>> That's a good point, but I would have to think that there are distributed
>> server installations at plenty of other places in the world where they have
>> ways of auto-updating the "world" quickly.
>
> That's probably true, and I suspect you could make an argument that
> our infra should be set up that way. So, as far as our board report is
> concerned (sorry I haven't read it yet), we should make sure we state
> this in terms of our needs or expectations. Something like: "We're
> concerned that the delays in getting our infra setup are hurting the
> project."
I will try to re-word the report to make that argument without targeting
secretary and infra.
> At the same time, we should probably be looking at ways to
> work around the current situation and not let it hold us back.
I believe the next step was to try to make the "parity" release. Carol said
she has the build script ready to go on her whiteboard. We can't really cut
a release from her whiteboard can we? That's why I think we're waiting on
SVN import to trunk.
Is there something else we need to do before trying to make this release? I
know there are things we need to get done before a release would ever get
signed off (like settle the trademark issue), but it seemed like everything
is waiting on being able to run the script from trunk.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Greg Reddin <gr...@gmail.com>.
On Thu, Apr 5, 2012 at 4:03 PM, Alex Harui <ah...@adobe.com> wrote:
> That's a good point, but I would have to think that there are distributed
> server installations at plenty of other places in the world where they have
> ways of auto-updating the "world" quickly.
That's probably true, and I suspect you could make an argument that
our infra should be set up that way. So, as far as our board report is
concerned (sorry I haven't read it yet), we should make sure we state
this in terms of our needs or expectations. Something like: "We're
concerned that the delays in getting our infra setup are hurting the
project." At the same time, we should probably be looking at ways to
work around the current situation and not let it hold us back.
Re: Board report, "Apache internals are starting to become a
problem" ?
Posted by Alex Harui <ah...@adobe.com>.
On 4/5/12 1:46 PM, "Greg Reddin" <gr...@gmail.com> wrote:
>
>> My half-baked thinking (and I don't really understand server-side issues) is
>> that you would assign a server to flex.apache.org at it would have enough
>> horsepower to host SVN and JIRA and a web-site. Infrastructure's job would
>> be narrowed to backups and maintenance of the servers and answering
>> questions, but administration would be mostly pushed to the podling.
>
> The problem with that approach is that source control and issue
> tracking are probably Apache's 2 most critical services. So there's an
> SLA associated with them (probably not an official SLA, but an
> expected one). Infra must be able to respond when svn or Jira goes
> belly up - or when a critical security issue pops up like happened a
> couple years ago. They also need to know the services are built and
> managed using best practices, etc. Imagine if each of the 150 or so
> projects had its own Jira. When a critical security bug comes up 150
> Jiras need to be patched ASAP.
That's a good point, but I would have to think that there are distributed
server installations at plenty of other places in the world where they have
ways of auto-updating the "world" quickly. My understanding is for clients,
IT departments have standards of what can and cannot be installed on a
desktop. Every time a security issue is found in Flash, those IT
departments have to push a player to every desktop.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Greg Reddin <gr...@gmail.com>.
On Thu, Apr 5, 2012 at 3:36 PM, Alex Harui <ah...@adobe.com> wrote:
> I did not give the secretary advanced warning. I will in the future. Being
> new to Apache, it isn't clear what turnaround times you can expect. I got
> quick response from the secretary when filing my ICLA.
Yep, it's a learning process. When I filed my ICLA several years ago
it took about a month to get my account. So they are constantly
improving things.
> My half-baked thinking (and I don't really understand server-side issues) is
> that you would assign a server to flex.apache.org at it would have enough
> horsepower to host SVN and JIRA and a web-site. Infrastructure's job would
> be narrowed to backups and maintenance of the servers and answering
> questions, but administration would be mostly pushed to the podling.
The problem with that approach is that source control and issue
tracking are probably Apache's 2 most critical services. So there's an
SLA associated with them (probably not an official SLA, but an
expected one). Infra must be able to respond when svn or Jira goes
belly up - or when a critical security issue pops up like happened a
couple years ago. They also need to know the services are built and
managed using best practices, etc. Imagine if each of the 150 or so
projects had its own Jira. When a critical security bug comes up 150
Jiras need to be patched ASAP.
I'm frustrated with the lag as well. I do think maybe we need to
consider our options for moving forward despite the lag.
Greg
Re: Board report, "Apache internals are starting to become a
problem" ?
Posted by Alex Harui <ah...@adobe.com>.
On 4/5/12 10:05 AM, "Dave Fisher" <da...@comcast.net> wrote:
>
> Please, Apache is a volunteer organization. The secretary is a volunteer. Did
> you give the ASF secretary a heads up that the signature was coming and you
> wanted an expedited approval?
I did not give the secretary advanced warning. I will in the future. Being
new to Apache, it isn't clear what turnaround times you can expect. I got
quick response from the secretary when filing my ICLA.
>
> How would 1000 separate SVN servers work?
>
My half-baked thinking (and I don't really understand server-side issues) is
that you would assign a server to flex.apache.org at it would have enough
horsepower to host SVN and JIRA and a web-site. Infrastructure's job would
be narrowed to backups and maintenance of the servers and answering
questions, but administration would be mostly pushed to the podling. Carol
practiced the import on her own SVN instance, I practiced the JIRA import on
my own JIRA instance. We did so in order to be comfortable that what we
sent to Apache would work. We learned enough to be able to do the imports
ourselves if we could just have access.
Thinking about it more, how are backups managed? If you have to restore
something can you do it without blowing away changes in another project? Is
it really a good idea for an SVN server failure to halt every Apache
project?
>
> I think that you can certainly reword the report to indicate that delays in
> importing the large JIRA database are a concern.
>
> The main scalability is the number of paid Infrastructure consultants. AFAIK
> this is now 3.5 people in addition to numerous volunteers. This is for 100+
> TLPs and over 50 podlings. I don't think that it helps to call into question
> scalability of volunteers. It helps to understand that it will always be an
> issue.
>
> The report should focus on the podling's concerns. The IPMC can then note that
> you need attention that will help.
>
I really think if Infra could promote Carol and/or I to be volunteers for
our portion of the SVN and JIRA servers, that would be more scalable. Of
course they won't do that because of the risk given that there are single
instances and a mistake could take down the entire foundation. That's why I
want the message to be to consider having each podling have its own server.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Dave Fisher <da...@comcast.net>.
Alex,
On Apr 5, 2012, at 9:12 AM, Alex Harui wrote:
>
>
>
> On 4/5/12 7:12 AM, "Bertrand Delacretaz" <bd...@apache.org> wrote:
>
>>
>> I don't think it's fair to blame the secretary or infra for that - a
>> few hours delay in recording a code grant is certainly acceptable, and
>> infra requiring big svn imports to happen on weekends for minimal
>> disruption sounds totally reasonable to me as well.
>>
>> Those big svn imports do not happen every day, so it seems ok to have
>> to plan them a bit in advance.
>>
>> I agree that the overall delays in getting the code in svn are
>> frustrating, but secretary/infra is only a minor part in that IMO. And
>> in the meantime people can play with the copy that Carol committed
>> anyway IIUC.
>>
>> My suggestion: tone that down a bit, and remove the secretary bit completely.
> My goal isn't to blame any set of individuals, but I think the process is
> cumbersome given this is 2012. If the actual deadline to submit a grant in
> order to make a weekend SVN import is Thursday and not Friday, that really
> needs to be known to all future podlings. It is no fun to ask a VP to come
> in on his vacation to sign the grant and then have it be for naught.
Please, Apache is a volunteer organization. The secretary is a volunteer. Did you give the ASF secretary a heads up that the signature was coming and you wanted an expedited approval?
>
> I also don't get how SVN scales for 10 years down the road with 1000 more
> podlings and 1000000 of files. The maintenance and overhead and maybe
> performance of the server might be prohibitive. Why not assign podlings
> their own server and give import rights to a few select folks?
How would 1000 separate SVN servers work?
>
> Folks know the code is on the whiteboard, but there is a tendency to wait
> until it gets into the trunk. We want to try to start building and
> packaging a release from the trunk and now we are waiting another week.
Personally I don't understand the waiting. JFDI.
>
> I'll think about how to re-word this section, but I still think there is a
> scalability problem that is impeding us. The podling doesn't have enough
> autonomy around its own databases.
I think that you can certainly reword the report to indicate that delays in importing the large JIRA database are a concern.
The main scalability is the number of paid Infrastructure consultants. AFAIK this is now 3.5 people in addition to numerous volunteers. This is for 100+ TLPs and over 50 podlings. I don't think that it helps to call into question scalability of volunteers. It helps to understand that it will always be an issue.
The report should focus on the podling's concerns. The IPMC can then note that you need attention that will help.
>
>>
>>> ...The same holds true for the bug database as well. We have been waiting
>>> for the bug database since Feb 1. A gating concern is that our import
>>> of 30000 bugs might take down the main JIRA instance. Having distributed
>>> JIRA instances would scale better...
>>
>> Here I agree that INFRA-4380 seems to be stalled. At some point IIRC
>> we discussed the alternative of importing issues via jira's RESTful or
>> other API, throttling to avoid putting too much load on the instance.
>> Has that been pursued?
> Infra refuses to let us try to import this many issues, even throttled.
> Again, another example of the scalability issue. If we had our own JIRA
> instance, we could try things without taking down everybody else. I don't
> know how many issues are in the rest of Apache JIRA, but we're about to jam
> in 30000 issues.
When did you have your last discussion with Infrastructure about JIRA? Was it the conversation from a few weeks ago that I tried to kick start? We'll have to get back to that. Sorry if I've been focused on my $job and Apache OpenOffice and stopped pushing that issue with Infra. I'll get back on it later today.
Deep breaths, have patience.
Regards,
Dave
>
> --
> Alex Harui
> Flex SDK Team
> Adobe Systems, Inc.
> http://blogs.adobe.com/aharui
>
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Greg Reddin <gr...@gmail.com>.
On Thu, Apr 5, 2012 at 12:29 PM, Carol Frampton <cf...@adobe.com> wrote:
> If I read the recent note from Tony in INFRA-4380 correctly, Apache is
> running JIRA in an unsupported configuration. To me that means it is
> unlikely there will be a bug fix forthcoming from Atlassian.
I think infra's opinion is that the bug is not related to the
unsupported configuration. They are looking at options to set up a
supported configuration and verify that the bug is repeatable. I can't
remember where I read that - probably on the infra list.
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Dave Fisher <da...@comcast.net>.
On Apr 5, 2012, at 11:35 AM, Bertrand Delacretaz wrote:
> On Thu, Apr 5, 2012 at 7:43 PM, Dave Fisher <da...@comcast.net> wrote:
>> ...If there are any JIRA / Tomcat folks on the PPMC who can help then it would be good
>> to know it now. The more volunteers to help with Project specific Infrastructure requests
>> the better....
>
> I don't have experience administering jira servers, but I have some
> experience writing simple jira plugins and addressing it via its http
> interfaces. I could probably help create and/or test an alternate way
> of importing the flex data if that's desired.
Great. Tony and I are conversing on infrastructure@. There is a key custom plugin.
Step by step ...
Regards,
Dave
>
> -Bertrand
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Thu, Apr 5, 2012 at 7:43 PM, Dave Fisher <da...@comcast.net> wrote:
> ...If there are any JIRA / Tomcat folks on the PPMC who can help then it would be good
> to know it now. The more volunteers to help with Project specific Infrastructure requests
> the better....
I don't have experience administering jira servers, but I have some
experience writing simple jira plugins and addressing it via its http
interfaces. I could probably help create and/or test an alternate way
of importing the flex data if that's desired.
-Bertrand
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Dave Fisher <da...@comcast.net>.
On Apr 5, 2012, at 10:29 AM, Carol Frampton wrote:
>
>
> On 4/5/12 1 :21PM, "Dave Fisher" <da...@comcast.net> wrote:
>
>>
>> On Apr 5, 2012, at 9:12 AM, Alex Harui wrote:
>>
>>>
>>>
>>>
>>> On 4/5/12 7:12 AM, "Bertrand Delacretaz" <bd...@apache.org> wrote:
>>>
>>>>
>>>> I don't think it's fair to blame the secretary or infra for that - a
>>>> few hours delay in recording a code grant is certainly acceptable, and
>>>> infra requiring big svn imports to happen on weekends for minimal
>>>> disruption sounds totally reasonable to me as well.
>>>>
>>>> Those big svn imports do not happen every day, so it seems ok to have
>>>> to plan them a bit in advance.
>>>>
>>>> I agree that the overall delays in getting the code in svn are
>>>> frustrating, but secretary/infra is only a minor part in that IMO. And
>>>> in the meantime people can play with the copy that Carol committed
>>>> anyway IIUC.
>>>>
>>>> My suggestion: tone that down a bit, and remove the secretary bit
>>>> completely.
>>> My goal isn't to blame any set of individuals, but I think the process
>>> is
>>> cumbersome given this is 2012. If the actual deadline to submit a
>>> grant in
>>> order to make a weekend SVN import is Thursday and not Friday, that
>>> really
>>> needs to be known to all future podlings. It is no fun to ask a VP to
>>> come
>>> in on his vacation to sign the grant and then have it be for naught.
>>>
>>> I also don't get how SVN scales for 10 years down the road with 1000
>>> more
>>> podlings and 1000000 of files. The maintenance and overhead and maybe
>>> performance of the server might be prohibitive. Why not assign podlings
>>> their own server and give import rights to a few select folks?
>>>
>>> Folks know the code is on the whiteboard, but there is a tendency to
>>> wait
>>> until it gets into the trunk. We want to try to start building and
>>> packaging a release from the trunk and now we are waiting another week.
>>>
>>> I'll think about how to re-word this section, but I still think there
>>> is a
>>> scalability problem that is impeding us. The podling doesn't have
>>> enough
>>> autonomy around its own databases.
>>>
>>>>
>>>>> ...The same holds true for the bug database as well. We have been
>>>>> waiting
>>>>> for the bug database since Feb 1. A gating concern is that our import
>>>>> of 30000 bugs might take down the main JIRA instance. Having
>>>>> distributed
>>>>> JIRA instances would scale better...
>>>>
>>>> Here I agree that INFRA-4380 seems to be stalled. At some point IIRC
>>>> we discussed the alternative of importing issues via jira's RESTful or
>>>> other API, throttling to avoid putting too much load on the instance.
>>>> Has that been pursued?
>>> Infra refuses to let us try to import this many issues, even throttled.
>>> Again, another example of the scalability issue. If we had our own JIRA
>>> instance, we could try things without taking down everybody else. I
>>> don't
>>> know how many issues are in the rest of Apache JIRA, but we're about to
>>> jam
>>> in 30000 issues.
>>
>> Please recall that part of the trouble is that JIRA is not open source
>> and Apache is waiting on Atlassian for a bug fix.
>>
>> Please see the notes from Monday on the issue:
>> https://issues.apache.org/jira/browse/INFRA-4380
>
> If I read the recent note from Tony in INFRA-4380 correctly, Apache is
> running JIRA in an unsupported configuration. To me that means it is
> unlikely there will be a bug fix forthcoming from Atlassian.
I sent an email to infrastructure suggesting we start exploring options like a project specific VM and separate JIRA instance. I've experience with JIRA configuration at work, so I can help.
There could be two goals that Infra could accomplish:
(1) Get Apache Flex it's JIRA instance.
(2) Explore a supported configuration.
But then it isn't clear what part of the configuration Atlassian does not support. JIRA runs on Tomcat and a Database. Apache is likely to run the most advanced and secure Tomcat. Likely before almost anyone else. Or it could be the Database. Or, the JVM. Or, the OS.
If there are any JIRA / Tomcat folks on the PPMC who can help then it would be good to know it now. The more volunteers to help with Project specific Infrastructure requests the better.
Regards,
Dave
>
> Carol
>
>>
>> It is not forgotten. If it is time to discuss alternative approaches with
>> Infrastructure.
>>
>> Regards,
>> Dave
>>
>>
>>
>>>
>>> --
>>> Alex Harui
>>> Flex SDK Team
>>> Adobe Systems, Inc.
>>> http://blogs.adobe.com/aharui
>>>
>>
>
Re: Board report, "Apache internals are starting to become a
problem" ?
Posted by Carol Frampton <cf...@adobe.com>.
On 4/5/12 1 :21PM, "Dave Fisher" <da...@comcast.net> wrote:
>
>On Apr 5, 2012, at 9:12 AM, Alex Harui wrote:
>
>>
>>
>>
>> On 4/5/12 7:12 AM, "Bertrand Delacretaz" <bd...@apache.org> wrote:
>>
>>>
>>> I don't think it's fair to blame the secretary or infra for that - a
>>> few hours delay in recording a code grant is certainly acceptable, and
>>> infra requiring big svn imports to happen on weekends for minimal
>>> disruption sounds totally reasonable to me as well.
>>>
>>> Those big svn imports do not happen every day, so it seems ok to have
>>> to plan them a bit in advance.
>>>
>>> I agree that the overall delays in getting the code in svn are
>>> frustrating, but secretary/infra is only a minor part in that IMO. And
>>> in the meantime people can play with the copy that Carol committed
>>> anyway IIUC.
>>>
>>> My suggestion: tone that down a bit, and remove the secretary bit
>>>completely.
>> My goal isn't to blame any set of individuals, but I think the process
>>is
>> cumbersome given this is 2012. If the actual deadline to submit a
>>grant in
>> order to make a weekend SVN import is Thursday and not Friday, that
>>really
>> needs to be known to all future podlings. It is no fun to ask a VP to
>>come
>> in on his vacation to sign the grant and then have it be for naught.
>>
>> I also don't get how SVN scales for 10 years down the road with 1000
>>more
>> podlings and 1000000 of files. The maintenance and overhead and maybe
>> performance of the server might be prohibitive. Why not assign podlings
>> their own server and give import rights to a few select folks?
>>
>> Folks know the code is on the whiteboard, but there is a tendency to
>>wait
>> until it gets into the trunk. We want to try to start building and
>> packaging a release from the trunk and now we are waiting another week.
>>
>> I'll think about how to re-word this section, but I still think there
>>is a
>> scalability problem that is impeding us. The podling doesn't have
>>enough
>> autonomy around its own databases.
>>
>>>
>>>> ...The same holds true for the bug database as well. We have been
>>>>waiting
>>>> for the bug database since Feb 1. A gating concern is that our import
>>>> of 30000 bugs might take down the main JIRA instance. Having
>>>>distributed
>>>> JIRA instances would scale better...
>>>
>>> Here I agree that INFRA-4380 seems to be stalled. At some point IIRC
>>> we discussed the alternative of importing issues via jira's RESTful or
>>> other API, throttling to avoid putting too much load on the instance.
>>> Has that been pursued?
>> Infra refuses to let us try to import this many issues, even throttled.
>> Again, another example of the scalability issue. If we had our own JIRA
>> instance, we could try things without taking down everybody else. I
>>don't
>> know how many issues are in the rest of Apache JIRA, but we're about to
>>jam
>> in 30000 issues.
>
>Please recall that part of the trouble is that JIRA is not open source
>and Apache is waiting on Atlassian for a bug fix.
>
>Please see the notes from Monday on the issue:
>https://issues.apache.org/jira/browse/INFRA-4380
If I read the recent note from Tony in INFRA-4380 correctly, Apache is
running JIRA in an unsupported configuration. To me that means it is
unlikely there will be a bug fix forthcoming from Atlassian.
Carol
>
>It is not forgotten. If it is time to discuss alternative approaches with
>Infrastructure.
>
>Regards,
>Dave
>
>
>
>>
>> --
>> Alex Harui
>> Flex SDK Team
>> Adobe Systems, Inc.
>> http://blogs.adobe.com/aharui
>>
>
Re: Board report, "Apache internals are starting to become a problem" ?
Posted by Dave Fisher <da...@comcast.net>.
On Apr 5, 2012, at 9:12 AM, Alex Harui wrote:
>
>
>
> On 4/5/12 7:12 AM, "Bertrand Delacretaz" <bd...@apache.org> wrote:
>
>>
>> I don't think it's fair to blame the secretary or infra for that - a
>> few hours delay in recording a code grant is certainly acceptable, and
>> infra requiring big svn imports to happen on weekends for minimal
>> disruption sounds totally reasonable to me as well.
>>
>> Those big svn imports do not happen every day, so it seems ok to have
>> to plan them a bit in advance.
>>
>> I agree that the overall delays in getting the code in svn are
>> frustrating, but secretary/infra is only a minor part in that IMO. And
>> in the meantime people can play with the copy that Carol committed
>> anyway IIUC.
>>
>> My suggestion: tone that down a bit, and remove the secretary bit completely.
> My goal isn't to blame any set of individuals, but I think the process is
> cumbersome given this is 2012. If the actual deadline to submit a grant in
> order to make a weekend SVN import is Thursday and not Friday, that really
> needs to be known to all future podlings. It is no fun to ask a VP to come
> in on his vacation to sign the grant and then have it be for naught.
>
> I also don't get how SVN scales for 10 years down the road with 1000 more
> podlings and 1000000 of files. The maintenance and overhead and maybe
> performance of the server might be prohibitive. Why not assign podlings
> their own server and give import rights to a few select folks?
>
> Folks know the code is on the whiteboard, but there is a tendency to wait
> until it gets into the trunk. We want to try to start building and
> packaging a release from the trunk and now we are waiting another week.
>
> I'll think about how to re-word this section, but I still think there is a
> scalability problem that is impeding us. The podling doesn't have enough
> autonomy around its own databases.
>
>>
>>> ...The same holds true for the bug database as well. We have been waiting
>>> for the bug database since Feb 1. A gating concern is that our import
>>> of 30000 bugs might take down the main JIRA instance. Having distributed
>>> JIRA instances would scale better...
>>
>> Here I agree that INFRA-4380 seems to be stalled. At some point IIRC
>> we discussed the alternative of importing issues via jira's RESTful or
>> other API, throttling to avoid putting too much load on the instance.
>> Has that been pursued?
> Infra refuses to let us try to import this many issues, even throttled.
> Again, another example of the scalability issue. If we had our own JIRA
> instance, we could try things without taking down everybody else. I don't
> know how many issues are in the rest of Apache JIRA, but we're about to jam
> in 30000 issues.
Please recall that part of the trouble is that JIRA is not open source and Apache is waiting on Atlassian for a bug fix.
Please see the notes from Monday on the issue: https://issues.apache.org/jira/browse/INFRA-4380
It is not forgotten. If it is time to discuss alternative approaches with Infrastructure.
Regards,
Dave
>
> --
> Alex Harui
> Flex SDK Team
> Adobe Systems, Inc.
> http://blogs.adobe.com/aharui
>
Re: Board report, "Apache internals are starting to become a
problem" ?
Posted by Alex Harui <ah...@adobe.com>.
On 4/5/12 7:12 AM, "Bertrand Delacretaz" <bd...@apache.org> wrote:
>
> I don't think it's fair to blame the secretary or infra for that - a
> few hours delay in recording a code grant is certainly acceptable, and
> infra requiring big svn imports to happen on weekends for minimal
> disruption sounds totally reasonable to me as well.
>
> Those big svn imports do not happen every day, so it seems ok to have
> to plan them a bit in advance.
>
> I agree that the overall delays in getting the code in svn are
> frustrating, but secretary/infra is only a minor part in that IMO. And
> in the meantime people can play with the copy that Carol committed
> anyway IIUC.
>
> My suggestion: tone that down a bit, and remove the secretary bit completely.
My goal isn't to blame any set of individuals, but I think the process is
cumbersome given this is 2012. If the actual deadline to submit a grant in
order to make a weekend SVN import is Thursday and not Friday, that really
needs to be known to all future podlings. It is no fun to ask a VP to come
in on his vacation to sign the grant and then have it be for naught.
I also don't get how SVN scales for 10 years down the road with 1000 more
podlings and 1000000 of files. The maintenance and overhead and maybe
performance of the server might be prohibitive. Why not assign podlings
their own server and give import rights to a few select folks?
Folks know the code is on the whiteboard, but there is a tendency to wait
until it gets into the trunk. We want to try to start building and
packaging a release from the trunk and now we are waiting another week.
I'll think about how to re-word this section, but I still think there is a
scalability problem that is impeding us. The podling doesn't have enough
autonomy around its own databases.
>
>> ...The same holds true for the bug database as well. We have been waiting
>> for the bug database since Feb 1. A gating concern is that our import
>> of 30000 bugs might take down the main JIRA instance. Having distributed
>> JIRA instances would scale better...
>
> Here I agree that INFRA-4380 seems to be stalled. At some point IIRC
> we discussed the alternative of importing issues via jira's RESTful or
> other API, throttling to avoid putting too much load on the instance.
> Has that been pursued?
Infra refuses to let us try to import this many issues, even throttled.
Again, another example of the scalability issue. If we had our own JIRA
instance, we could try things without taking down everybody else. I don't
know how many issues are in the rest of Apache JIRA, but we're about to jam
in 30000 issues.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui