Graduation ?

[Ian Boston <ie...@tfd.co.uk>]

I notice on general@incubator there is a thread starting on criteria  
for graduation, initially stated as

- when the community has performed at least one Apache release
- when the community has cleaned the code from all license issues
- when the community is diverse and open

The "diverse and open" is in the process of being defined in more  
detail.

My question:

Is Shindig ready for graduation ?

Ian 

Re: Graduation ?

[Chris Chabot <ch...@google.com>]

On Thu, Jun 25, 2009 at 1:14 PM, Upayavira <uv...@odoko.co.uk> wrote:

> I presume you are not a Google employee. I'd appreciate it if you would
> speak about Shindig from a non-google perspective.
>

He's not, so he couldn't :)

This is because, understandably, Google does dominate Shindig. For
> graduation, the major concern is that sufficient space has been created
> in the project for non-Google contributors to participate on an equal
> footing.
>
> What I do notice is that activity here is fast. I read very little of
> the traffic on this list. As a mentor, I simply cannot keep up. Does
> this form a barrier for someone who would like to work on/with Shindig,
> but cannot do so (a) full time (b) on the West Coast USA (c) based from
> a Google office (d) etc?
>

Ian in this example is from the UK, my self I'm from the Netherlands and we
have some chaps from for instance the BBC who are from the UK as well .. I'm
not going to go guess percentages, but at least some of that heavy traffic
you refer to is generated from non Googlers, non full time shindig
developers, who are in fact not located on the US West Coast :)

While it is a partially valid point you are making, it's not nearly as much
as it used to be anymore; Geographically we're much more spread out as
Shindig was at it's inception, and also the initial-company vs independent
contributors has vastly improved too, I might be cheating a little bit by
looking at 'currently most active committers' instead of the full committers
list, but if you'd allow for that slightly positive perspective on things we
have 4 very active Googlers and 4 'independent' committers. So while the
presence might still be construed as relatively initial-company-heavy, we've
also past the point where the project was dependent only on that company.

Also from a contributions point of view we're seeing very regular patches
from lots of independent parties, which to me also indicates good project
health and interest.

As the discussion on general@incubator noted, most projects only attract one
new contributor during it's incubation, so with 4 new independent committers
(it was actually 5, until i went and joined Google), we're doing quite well
and I think this does say something about us being open and accessible as a
project.

Well.. if you don't mind reading lots of mail, but the amount of mail never
has been a measure for the openness of a community, else the LKM would've
surely meant the demise of linux :) I would even go so far as to say it's a
sign of health, if all those discussions and decisions were made off-list,
then we'd have a problem

   -- Chris

Re: Graduation ?

[Vincent Siveton <vs...@apache.org>]

2009/6/25 Upayavira <uv...@odoko.co.uk>:
> I presume you are not a Google employee. I'd appreciate it if you would
> speak about Shindig from a non-google perspective.

I am involved in this project from its beginning. Now as a committer,
I think, like others, the community is open.
I am also not affiliate with Google, and like Ian, I never feel that
technical decisions are decided behind the ML or just pushed as a
fact.
The code review is an excellent thing and encourages discussions. From
what I have time to read, discussions are franks between both
committers and non commiters.
I find that the ML activities are between 9AM-5PM PST, mon-friday, but
all messages have generally responses during next days.
So yes I think the community seems mature for a graduation.

For new people, I think the learning curve is not easy. I committed
technical docs for the website but due to recent changes, they are
probably not up to date (a good news is that the javadoc is!).
Also, the open social world is not easy for new comers. There are
shindig ML, opensocial foundation ML, several websites... brief, a
tons of informations, and as a committer, I am not involved in all
part of theses informations...
I think for a graduation perspective we need to concentrate our
efforts to improve the documentation and design choices (think to JUL
questions recently).

Cheers,

Vincent

Re: Graduation ?

[Ian Boston <ie...@tfd.co.uk>]

On 25 Jun 2009, at 12:14, Upayavira wrote:

> On Wed, 2009-06-24 at 16:10 +0100, Ian Boston wrote:
>> On 24 Jun 2009, at 10:04, Chris Chabot wrote:
>>
>>> If anyone (from mentors to contributers and especially mailing list
>>> lurkers)
>>> feels differently, then please let us know so we know to fix it ...
>>> but from
>>> where I'm standing I would say we look about ready!
>>
>> I agree with everything that Chris has said (I would not have dared
>> ask the question otherwise :) ).
>>
>> With a Lurker hat on:
>> Sometime I feel that we dont discuss motivation or design of a change
>> as much as we might, which makes it hard for new commers to  
>> understand
>> what is going on, or about to happen. In some respects this is
>> unnecessary since we are implementing the Gadges and OS specs.  
>> However
>> there are plenty of TLP's that are hard to work in as a result of
>> being a) complex and b) fast moving. IMHO its someting we have to  
>> work
>> at. That would be my only critisism.
>>
>> With a committer hat on:
>> The review of submissions from commiters and contributers alike, and
>> rapid resolution of differences of opinion without hurting personal
>> possitions or emotions typifies Shindig. (IMHO)
>
> I presume you are not a Google employee. I'd appreciate it if you  
> would
> speak about Shindig from a non-google perspective.
>
> This is because, understandably, Google does dominate Shindig. For
> graduation, the major concern is that sufficient space has been  
> created
> in the project for non-Google contributors to participate on an equal
> footing.
>
> What I do notice is that activity here is fast. I read very little of
> the traffic on this list. As a mentor, I simply cannot keep up. Does
> this form a barrier for someone who would like to work on/with  
> Shindig,
> but cannot do so (a) full time (b) on the West Coast USA (c) based  
> from
> a Google office (d) etc?
>
> Upayavira
>


With a non google employee hat on, a personal view:

I Started getting involved in Shindig about 16 months ago, and found  
the community immediately open and friendly. In spite of being Google  
dominated, more so in those days, there was an certainly an openness  
to talk about things on list. Its hard for me to judge if the onlist  
activity is the tip on icebrerg representing all the communication,  
but I cant remember of any instances over the past year where I have  
felt that I was watching a replay of a previously agreed outcome.

Shortly after joining I noticed a number of potential improvements in  
the areas of the Social Model, the Service Provider Interfaces and the  
JSON serialization. All of which I felt able to contribute to,  
refactoring the service layer into an SPI with seperate sample  
implementation, tweaking the model and providing an alternative  
implementation JSON serialization (although this was not that great a  
piece of work, subsiquently fixed by Paul L). At that time I certainly  
was not working full time on Shindig and I live in the UK, but in  
general then and now I feel I am able to keep up with the pace of  
change. If I compare shindig-dev to Jackrabbit dev which I also follow  
I think they are quite simular in rate of change and complexity. So  
does the pace of change form a barrier to contributions to a), b), c)  
and d) ?  I think the answer is no, with the caveat that the attention  
to detail that is reqiured is high, which requires carefull thought on  
the part of anyone contributing.

I suspect that this is driven by the unusual circumstnaces that mean  
the code at trunk/head is appearing in significant production, as I  
understand it, shortly after being committed. Shortly when compared to  
many other Apache projects. This has real benefits and significant  
drawbacks. The real benefits are that the code base is driven by  
production evidence, the drawbacks is that changes represent risk, and  
the likelyhood is that those not running in heavy production will make  
contributions without a detailed knowledge of the impact.

I my mind this is a teathing problem for the community that could be  
addressed by decoupling the production deployments from trunk/head and  
greater use of a sandbox space for experimentation. Having said all  
that shindig-dev is a list that I try and read daily, even though I  
dont have enough time at the moment to make significant contributions,  
or review the patches of others.

I hope that helps put a GMT, non Google perspective on it.
On the whole I think that there a good ballance of Google/non Google  
committers, but slightly less production risk for contributions would  
reduce barriers.

All IMVHO, others mileage may vary, but I would like to hear from  
others.
Ian









>

Re: Graduation ?

[Upayavira <uv...@odoko.co.uk>]

On Wed, 2009-06-24 at 16:10 +0100, Ian Boston wrote:
> On 24 Jun 2009, at 10:04, Chris Chabot wrote:
> 
> > If anyone (from mentors to contributers and especially mailing list  
> > lurkers)
> > feels differently, then please let us know so we know to fix it ...  
> > but from
> > where I'm standing I would say we look about ready!
> 
> I agree with everything that Chris has said (I would not have dared  
> ask the question otherwise :) ).
> 
> With a Lurker hat on:
> Sometime I feel that we dont discuss motivation or design of a change  
> as much as we might, which makes it hard for new commers to understand  
> what is going on, or about to happen. In some respects this is  
> unnecessary since we are implementing the Gadges and OS specs. However  
> there are plenty of TLP's that are hard to work in as a result of  
> being a) complex and b) fast moving. IMHO its someting we have to work  
> at. That would be my only critisism.
> 
> With a committer hat on:
> The review of submissions from commiters and contributers alike, and  
> rapid resolution of differences of opinion without hurting personal  
> possitions or emotions typifies Shindig. (IMHO)

I presume you are not a Google employee. I'd appreciate it if you would
speak about Shindig from a non-google perspective.

This is because, understandably, Google does dominate Shindig. For
graduation, the major concern is that sufficient space has been created
in the project for non-Google contributors to participate on an equal
footing.

What I do notice is that activity here is fast. I read very little of
the traffic on this list. As a mentor, I simply cannot keep up. Does
this form a barrier for someone who would like to work on/with Shindig,
but cannot do so (a) full time (b) on the West Coast USA (c) based from
a Google office (d) etc?

Upayavira

Re: Graduation ?

[Ian Boston <ie...@tfd.co.uk>]

On 24 Jun 2009, at 10:04, Chris Chabot wrote:

> If anyone (from mentors to contributers and especially mailing list  
> lurkers)
> feels differently, then please let us know so we know to fix it ...  
> but from
> where I'm standing I would say we look about ready!

I agree with everything that Chris has said (I would not have dared  
ask the question otherwise :) ).

With a Lurker hat on:
Sometime I feel that we dont discuss motivation or design of a change  
as much as we might, which makes it hard for new commers to understand  
what is going on, or about to happen. In some respects this is  
unnecessary since we are implementing the Gadges and OS specs. However  
there are plenty of TLP's that are hard to work in as a result of  
being a) complex and b) fast moving. IMHO its someting we have to work  
at. That would be my only critisism.

With a committer hat on:
The review of submissions from commiters and contributers alike, and  
rapid resolution of differences of opinion without hurting personal  
possitions or emotions typifies Shindig. (IMHO)

Ian

Re: Graduation ?

[Ian Boston <ie...@tfd.co.uk>]

 From [1] It looks like timing is important.
I would be happy to start a thread on Sunday.
I understand there is a Members meeting imminent or in progress (not  
being a member), so before I start a vote I want to check that our  
Mentors are
a) available and
b) supportive
c) have had enough time to think and discuss this issue.

Mentors ?
Ian

On 2 Jul 2009, at 20:32, Lane LiaBraaten wrote:

> There's some great feedback in this thread and it's especially good  
> to hear
> from the non-Google, non-PST folks.  The general takeaway I get is  
> that
> while there are still ways in which we can grow as a project and  
> community,
> we're looking good w.r.t. the basic requirements for graduation.
>
> In short, the graduation process is [1]:
> - Hold community graduation vote
> - Create a charter/resolution
> - Hold IPMC recommendation vote
> - Submit resolution to the board
>
> So in order to pursue this, the first step is  to start a '[VOTE]  
> Graduate
> Apache Shindig as Top Level Project' thread and cc general@incubator
>
> Another thing we'll need to address is the status page [2] as it looks
> generally out of date.  I'm happy to help update it if there is no  
> current
> owner.
>
> Cheers,
> Lane
>
> [1] http://incubator.apache.org/guides/graduation.html#toplevel
> [2] http://incubator.apache.org/projects/shindig.html
>
> On Sat, Jun 27, 2009 at 3:25 AM, Chris Chabot <ch...@google.com>  
> wrote:
>
>> Hey Gabriel, thanks for the feedback!
>>
>> On Sat, Jun 27, 2009 at 12:20 AM, Gabriel Barros <gbarros@yahoo-inc.com
>>> wrote:
>>
>>> 1. involvement faq: I was going to talk about not much info on how  
>>> to
>>> get involved. But the new website is much clearer! Now i can only  
>>> push a
>>> little further and ask for more clarity on the steps to contribute
>>> patches. Or is the email-the-list the preferable one anyways? (never
>>> worked on any project at Jira, and i've already seen people using
>>> another tool for code reviews)
>>>
>>
>>
>> There is some generic Apache text on this matter:
>> http://www.apache.org/foundation/getinvolved.html However I agree  
>> this
>> could
>> be expanded on, and link to things like our code guidelines etc.
>>
>> The process is that someone should submit a .patch file to a Jira  
>> issue,
>> and
>> when doing so check the "I grant the apache foundation full rights  
>> to this
>> work" checkbox.. that's an important legal bit that makes sure we can
>> consume the code without polluting the legal standing of the  
>> project. After
>> that a committer will look at the patch, give feedback if need be,  
>> and
>> after
>> that's all done, it'll get committed to svn.
>>
>>
>>
>>> 2. PHP vs Java vs X: I got the impression that the java version  
>>> has more
>>> contributors and a more complete/tested code base. I may be  
>>> getting the
>>> wrong impression (I only used the bleeding edge code, I serve my
>>> development environment directly from the svn trunk). But if it is  
>>> so,
>>> then being more open about this would be  positive.
>>
>>
>> The difference isn't as big as you might think, we have only one  
>> committer
>> that's dedicated to the php part (ie: me), however we have 3 people  
>> who
>> contribute regularly (one even full time) next to that, and 8+  
>> people who
>> contribute patches and improvements through Jira regularly whenever
>> something broke, and a good assortment of people who contribute one  
>> patch
>> before going into lurker mode again.
>>
>> Now it's true that some of those contributes are a bit more quiet  
>> on the
>> shindig-dev list, which I think has something to do with culture and
>> language (the majority of the people mentioned above are not US or  
>> western
>> Europe based) which is why it might seem less then it is; However  
>> that
>> doesn't reduce anything of the value of the community and code.
>>
>> Next to that the php version reaches ~450 million registered users  
>> on over
>> 15 live sites, is being implemented on some 9 more currently (and  
>> growing
>> almost daily it seems!), and found it's way into > 6 open source  
>> projects,
>> and that's just the part I know off, many people don't report back  
>> to us so
>> the actual number is probably a lot higher.
>>
>> That being said, the trunk was unstable for a little while during the
>> implementation of a whole new gadget rendering pipeline to fit in  
>> the new
>> proxied content, data pipelining and templating bits into it  
>> properly, so
>> some instability has happened, they don't call it the bleeding edge  
>> for
>> nothing :) However if you do run into any issues, please report  
>> them &
>> patches are always welcome! :)
>>
>> Also, i failed to see a integration test suit that i can run when
>>> changing code, or starting an implementation in another language for
>>> example.
>>>
>>
>> That might be an unfair measurement since no open source project I  
>> know of
>> has such a integration test suite, usually it comes down to "Try  
>> all the
>> code that's meant to work with this and see what breaks" :)
>>
>> However we do have a pretty good 0.7 and 0.8.1 based compliance  
>> test suite
>> that can go a long way in helping with this:
>> http://code.google.com/p/opensocial-resources/wiki/ComplianceTests
>>
>> And on the spec list there's some talk about how to build a better  
>> and more
>> extensible test suite for 0.9, see the thread
>>
>> http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thread/thread/132347d7c4cd0ca8?hl=enfor
>> more info.
>>
>> 3. Google issue raised on the thread: My 2cents. The main open social
>>> player (at least in Brazil, India) is Orkut. Changes happens first  
>>> in
>>> orkut, maybe that gives out the impression that google is steering
>>> shindig direction. Most people came to shindig as a way to have a  
>>> local
>>> sandbox.  Even though I've never been able to run any stable orkut  
>>> app
>>> in shindig.
>>>
>>
>> The whole basis for open source is to 'scratch an itch', ie people  
>> will
>> always work on what's important to them, and if something else is  
>> important
>> to you, then you contribute a patch for it :)
>>
>> In this case, sure some of the changes might make it to the orkut or
>> iGoogle
>> sandbox first, and other changes might make it to netlog or hi5  
>> first,
>> completely depending on what priorities exist and what resources  
>> people put
>> into it; However any container could do exactly the same, you do an  
>> svn
>> checkout, deploy it to your sandbox and/or production setup, and  
>> presto
>> you're running the latest version. The only reason why not everyone  
>> does
>> that is due to time and budget constraints, but there's nothing  
>> baring them
>> from doing so if they felt it was important enough to invest in it.
>>
>> On the topic of running orkut apps on your local setup.. Not every
>> container
>> is 100% compliant, so sometimes a compatibility issue will get in  
>> the way,
>> but more often it'll simply be the case that the back-end of the  
>> gadget
>> only
>> accepts <some containers> name and their specific certificates for  
>> signed
>> requests.
>>
>> So in pretty much all cases where a gadget has a DB per site they  
>> support,
>> and use signed requests to make calls to their back-end (as they  
>> should!),
>> you'll have to contact the gadget developer and ask him to add  
>> support for
>> your container too. Gadget dev's also prefer it that way since it  
>> allows
>> them to customize the gadget's look and feel for the specific site  
>> (color,
>> size, etc), see if it matches their target audience and there's the  
>> issue
>> of
>> language and culture too of course.
>>
>>  -- Chris
>>

Re: Graduation ?

[Upayavira <uv...@odoko.co.uk>]

On Sat, 2009-07-04 at 01:04 +0200, Chris Chabot wrote:
> On Sat, Jul 4, 2009 at 12:33 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> 
> >
> > Scroll down :)
> > <snip>
> >
> 
> 
> > I hear what you are saying about getting trunk into production to allow the
> > Opensocial and Gadget Developer community to review spec changes, but surely
> > anyone pushing to production should be a branch and merging commits to
> > stable points rather than expecting head of trunk to be stable at all times?
> > If it cant be done with svn because of the rate of change, then perhaps
> > cloning the Git mirror of shindig will make a branch viable as it *is*
> > (IMHO) better at merging high volume patch streams.
> >
> 
> 
> I kind of tried to cover that in my lengthy rambling too, a lot of the
> current contributions are either people developing:
> 
>    -  spec suggestions
>    - prototypes for voted in spec changes
>    - developing new spec items, which they deploy to a sandbox to get
>    feedback (please do note the *sandbox* part, it's not always directly live:)
>    - Someone deploying a recent version of shindig to their platform (you
>    can easily tell when hi5, linked-in, ning, etc do their upgrade cycle since
>    that's when we get a storm of patches from them)
> 
> So there is no real 'pie in the sky' type development going on, ie long term
> "hey wouldn't it be nice if we did FOO in a couple of months, lets try it",
> everything that is being developed is to form the new spec, implement the
> new spec, or to fix some issues that came up during deployment on someone's
> platform.
> 
> Now while we're implementing a new spec (as we're working on 0.9 support
> right now), there's definitely an opportunity for people to jump in and pick
> one of the items, especially since the spec is the documentation of what
> needs to be implemented; However since we like to see how these things work
> in practise so we don't develop a theoretical spec that doesn't work, we
> probably do want to keep these iteration cycles relatively short (1 or 2
> months max).

> And while I'm writing this, it also occurs to me that maybe the barrier of
> entry for shindig is higher since we implement a spec (so half of the good
> stuff, the design, happens in the spec process), and don't really allow wild
> changes in shindig that would break away from the spec, or new features that
> are not part of the spec (for obvious reasons, if we had 10^99 versions of
> OpenSocial, it wouldn't be a usable platform anymore).
> 
> That's probably not so exciting and inviting as some other open source
> projects where people have much more of a free hand. (though do note: by
> participating in the spec process, you do actually have the opportunity to
> help determine the design, it just happens on a different list since we like
> the spec to be implementation neutral and not tied to just one vendor or
> implementation)

Two suggestions:

 * anyone interested in helping, just come and ask. If shy, feel free 
   to ask me privately 
 * make frequent releases. With frequent releases, the pressure to use 
   trunk is vastly reduced. If there are releases every two weeks, that 
   at least gives me the chance to break things at the beginning of the 
   next release cycle. Folks are more likely to make their release 
   cycles match those of Shindig if it means they get slightly better 
   tested code as a result.

Upayavira

Re: Graduation ?

[Chris Chabot <ch...@google.com>]

On Sat, Jul 4, 2009 at 12:33 AM, Ian Boston <ie...@tfd.co.uk> wrote:

>
> Scroll down :)
> <snip>
>


> I hear what you are saying about getting trunk into production to allow the
> Opensocial and Gadget Developer community to review spec changes, but surely
> anyone pushing to production should be a branch and merging commits to
> stable points rather than expecting head of trunk to be stable at all times?
> If it cant be done with svn because of the rate of change, then perhaps
> cloning the Git mirror of shindig will make a branch viable as it *is*
> (IMHO) better at merging high volume patch streams.
>


I kind of tried to cover that in my lengthy rambling too, a lot of the
current contributions are either people developing:

   -  spec suggestions
   - prototypes for voted in spec changes
   - developing new spec items, which they deploy to a sandbox to get
   feedback (please do note the *sandbox* part, it's not always directly live:)
   - Someone deploying a recent version of shindig to their platform (you
   can easily tell when hi5, linked-in, ning, etc do their upgrade cycle since
   that's when we get a storm of patches from them)

So there is no real 'pie in the sky' type development going on, ie long term
"hey wouldn't it be nice if we did FOO in a couple of months, lets try it",
everything that is being developed is to form the new spec, implement the
new spec, or to fix some issues that came up during deployment on someone's
platform.

Now while we're implementing a new spec (as we're working on 0.9 support
right now), there's definitely an opportunity for people to jump in and pick
one of the items, especially since the spec is the documentation of what
needs to be implemented; However since we like to see how these things work
in practise so we don't develop a theoretical spec that doesn't work, we
probably do want to keep these iteration cycles relatively short (1 or 2
months max).

And while I'm writing this, it also occurs to me that maybe the barrier of
entry for shindig is higher since we implement a spec (so half of the good
stuff, the design, happens in the spec process), and don't really allow wild
changes in shindig that would break away from the spec, or new features that
are not part of the spec (for obvious reasons, if we had 10^99 versions of
OpenSocial, it wouldn't be a usable platform anymore).

That's probably not so exciting and inviting as some other open source
projects where people have much more of a free hand. (though do note: by
participating in the spec process, you do actually have the opportunity to
help determine the design, it just happens on a different list since we like
the spec to be implementation neutral and not tied to just one vendor or
implementation)

   -- Chris

Re: Graduation ?

[Ian Boston <ie...@tfd.co.uk>]

Scroll down :)


On 3 Jul 2009, at 22:56, Chris Chabot wrote:

> Hey All,
>
> We've also had some discussion about the committer balance status and
> barrier-to-entry for shindig and how the trunk is used and I'd like to
> re-post my response since it contained a few insights in how the  
> OpenSocial
> spec process works and how shindig is used by some:
>
> I do see plenty of random small contributions, but so far no one  
> sticks
> around longer then their OpenSocial/Shindig implementation phase.  
> These are
> mostly people who are paid to implement shindig and then move on to
> something else; Which in a way is easy to imagine because as you  
> know the
> social web is fast moving, margins are incredibly tight, and most  
> social
> networks are more concerned about competing against the big other  
> player in
> the market then they are about contributing to open source. A short  
> sighted
> vision perhaps for some of them, but also something I can kind of
> understand.
>
> So my hope is more for someone who thinks 'Shindig is cool and I  
> want to use
> this in imaginative ways" in a university or free time kind of  
> environment,
> or a social site that is making enough money or has the right person  
> to
> convince them that they should allocate time for it.
>
> As far as the release cycle and the stability of the branch goes, I  
> can't
> speak for the teams who work on Orkut and iGoogle, but my  
> speculation is
> that like for any social site, their main priority is to have  
> features out
> there that attract developers that will make social apps that  
> attract people
> to the site, make sure that things are secure and stable and fast  
> enough, so
> that their site can function and grow in a very competitive market.
>
> OpenSocial is a bit of a different animal then most open source  
> projects, we
> have an OpenSocial specification process that is completely separate  
> from
> Apache Shindig, and in it's current state it evolves very quickly; Not
> because the people involved in it (which a much wider range of  
> companies
> represented as Shindig has) like new and shiny things but because  
> the social
> web is evolving at an incredibly rapid rate.. and there's plenty of
> competitive pressure at the same time too, for many it's a situation  
> where
> OpenSocial represents the 'web like' way of doing things, open  
> protocols and
> standards, with data portability and a place where any one can  
> innovate,
> against a platform that doesn't have any of these things.
>
> So the pressure to keep the trunk stable not from Google or any  
> affiliate or
> any of the consumers of shindig directly (if it were it wouldn't be  
> such a
> big problem:), but more of the market and innovation pace in  
> general, and
> from the desire to create an open and social web that anyone,  
> anywhere in
> the world can use to develop their ideas on, and making sure that  
> end users
> can own their own data and identity by giving them tools and  
> protocols with
> which they can. (a great way to learn more about this topic is the  
> archives
> from the http://www.thesocialweb.tv/ episodes. Those guys do a lot  
> for the
> open social web)
>
> Now as a sanity check we've build in the requirement that any new spec
> proposal should have a prototype build before we call it final.. that
> prevents great ideas that don't work in practice ending up in the spec
> (which is why we now have 0.8.1 instead of the original 0.8:), and  
> we also
> love it if we can get some OpenSocial gadget developer feedback and  
> live
> experiences before we call it completely 'done'. To facilitate this  
> you *do*
> need a stable experimentation tree, a place where you can code up  
> these new
> features and put in production on large scale sites
>
> And that, well that very much that conflicts with the idea of having  
> an
> unstable trunk.. trunk is where new code should go, but we need the  
> new code
> to be stable to complete the feedback loop that is an important  
> factor in
> the OpenSocial spec forming process.
>
> How you could combine those 2 concepts, well that's a riddle that I  
> don't
> have an answer for either. If that coding was done on an alternative  
> branch,
> then the trunk wouldn't see any development (and thus become the new  
> 'stable
> branch'), and the experimentation branch would become the new  
> trunk.. in
> other words that's only moving the problem around and creating more
> confusion for everyone involved.
>
> Now for PHP Shindig these pressures are slightly less (though far from
> non-existent), I can afford to have an unstable trunk for a few  
> weeks here
> and there since there's no main stream sites driving sandboxes  
> directly from
> it, however in practice when ever I do have some instability (like  
> during
> the gadget rendering refactoring & templating implementation), I get  
> lots of
> complaining and upset mails in various flavors and colors.. since we  
> haven't
> had releases and people are used to the trunk being stable, well  
> they are a
> bit taken aback when it's not, so there's a cultural problem we're  
> facing as
> well.
>
> ---- (end of original post to shindig-private)
>
> Since writing that I've chatted with a couple of people about how we  
> might
> be able to address this, and one of the ideas I've heard is that  
> maybe we
> can move the OpenSocial new spec prototype development to experimental
> branches, this should relieve the pressure on the trunk a bit, and  
> might
> have the added benefit of no old prototype code that didn't make it  
> into the
> spec to still sneak into the release :)
>
> Also, there's some people who would like to see the next OpenSocial  
> spec to
> be more of a bug fix with a few small additions rather then a huge  
> major
> pile of changes again (it's up to the OpenSocial community as a  
> whole though
> to decide), so perhaps some less churn there could make shindig  
> easier to
> follow as well.
>
> Anyhow, hope this post provides some insights, and if anyone has any  
> ideas
> or opinions, please share them!


Chris,
I hear what you are saying about getting trunk into production to  
allow the Opensocial and Gadget Developer community to review spec  
changes, but surely anyone pushing to production should be a branch  
and merging commits to stable points rather than expecting head of  
trunk to be stable at all times?
If it cant be done with svn because of the rate of change, then  
perhaps cloning the Git mirror of shindig will make a branch viable as  
it *is* (IMHO) better at merging high volume patch streams.

Also I am not certain that the suggestion of new features in branches  
(end of the post) helps, since those deploying will be deploying those  
features.... unless you mean that trunk should carry on as the normal  
development thread at a measured, community friendly pace, and the new  
features branches are really production branches... changes accepted  
back as contributions.

Ian

>
>   -- Chris

Re: Graduation ?

[Chris Chabot <ch...@google.com>]

Hey All,

We've also had some discussion about the committer balance status and
barrier-to-entry for shindig and how the trunk is used and I'd like to
re-post my response since it contained a few insights in how the OpenSocial
spec process works and how shindig is used by some:

I do see plenty of random small contributions, but so far no one sticks
around longer then their OpenSocial/Shindig implementation phase. These are
mostly people who are paid to implement shindig and then move on to
something else; Which in a way is easy to imagine because as you know the
social web is fast moving, margins are incredibly tight, and most social
networks are more concerned about competing against the big other player in
the market then they are about contributing to open source. A short sighted
vision perhaps for some of them, but also something I can kind of
understand.

So my hope is more for someone who thinks 'Shindig is cool and I want to use
this in imaginative ways" in a university or free time kind of environment,
or a social site that is making enough money or has the right person to
convince them that they should allocate time for it.

As far as the release cycle and the stability of the branch goes, I can't
speak for the teams who work on Orkut and iGoogle, but my speculation is
that like for any social site, their main priority is to have features out
there that attract developers that will make social apps that attract people
to the site, make sure that things are secure and stable and fast enough, so
that their site can function and grow in a very competitive market.

OpenSocial is a bit of a different animal then most open source projects, we
have an OpenSocial specification process that is completely separate from
Apache Shindig, and in it's current state it evolves very quickly; Not
because the people involved in it (which a much wider range of companies
represented as Shindig has) like new and shiny things but because the social
web is evolving at an incredibly rapid rate.. and there's plenty of
competitive pressure at the same time too, for many it's a situation where
OpenSocial represents the 'web like' way of doing things, open protocols and
standards, with data portability and a place where any one can innovate,
against a platform that doesn't have any of these things.

So the pressure to keep the trunk stable not from Google or any affiliate or
any of the consumers of shindig directly (if it were it wouldn't be such a
big problem:), but more of the market and innovation pace in general, and
from the desire to create an open and social web that anyone, anywhere in
the world can use to develop their ideas on, and making sure that end users
can own their own data and identity by giving them tools and protocols with
which they can. (a great way to learn more about this topic is the archives
from the http://www.thesocialweb.tv/ episodes. Those guys do a lot for the
open social web)

Now as a sanity check we've build in the requirement that any new spec
proposal should have a prototype build before we call it final.. that
prevents great ideas that don't work in practice ending up in the spec
(which is why we now have 0.8.1 instead of the original 0.8:), and we also
love it if we can get some OpenSocial gadget developer feedback and live
experiences before we call it completely 'done'. To facilitate this you *do*
need a stable experimentation tree, a place where you can code up these new
features and put in production on large scale sites

And that, well that very much that conflicts with the idea of having an
unstable trunk.. trunk is where new code should go, but we need the new code
to be stable to complete the feedback loop that is an important factor in
the OpenSocial spec forming process.

How you could combine those 2 concepts, well that's a riddle that I don't
have an answer for either. If that coding was done on an alternative branch,
then the trunk wouldn't see any development (and thus become the new 'stable
branch'), and the experimentation branch would become the new trunk.. in
other words that's only moving the problem around and creating more
confusion for everyone involved.

Now for PHP Shindig these pressures are slightly less (though far from
non-existent), I can afford to have an unstable trunk for a few weeks here
and there since there's no main stream sites driving sandboxes directly from
it, however in practice when ever I do have some instability (like during
the gadget rendering refactoring & templating implementation), I get lots of
complaining and upset mails in various flavors and colors.. since we haven't
had releases and people are used to the trunk being stable, well they are a
bit taken aback when it's not, so there's a cultural problem we're facing as
well.

---- (end of original post to shindig-private)

Since writing that I've chatted with a couple of people about how we might
be able to address this, and one of the ideas I've heard is that maybe we
can move the OpenSocial new spec prototype development to experimental
branches, this should relieve the pressure on the trunk a bit, and might
have the added benefit of no old prototype code that didn't make it into the
spec to still sneak into the release :)

Also, there's some people who would like to see the next OpenSocial spec to
be more of a bug fix with a few small additions rather then a huge major
pile of changes again (it's up to the OpenSocial community as a whole though
to decide), so perhaps some less churn there could make shindig easier to
follow as well.

Anyhow, hope this post provides some insights, and if anyone has any ideas
or opinions, please share them!

   -- Chris

Re: Graduation ?

[Upayavira <uv...@odoko.co.uk>]

On Thu, 2009-07-02 at 12:32 -0700, Lane LiaBraaten wrote:
> There's some great feedback in this thread and it's especially good to hear
> from the non-Google, non-PST folks.  The general takeaway I get is that
> while there are still ways in which we can grow as a project and community,
> we're looking good w.r.t. the basic requirements for graduation.
> 
> In short, the graduation process is [1]:
>  - Hold community graduation vote
>  - Create a charter/resolution
>  - Hold IPMC recommendation vote
>  - Submit resolution to the board
> 
> So in order to pursue this, the first step is  to start a '[VOTE] Graduate
> Apache Shindig as Top Level Project' thread and cc general@incubator
> 
> Another thing we'll need to address is the status page [2] as it looks
> generally out of date.  I'm happy to help update it if there is no current
> owner.

Hey, I think we need more time, and I'd vote -1 if a vote came up right
now.

I think we have identified some good stuff, but I don't think we're
there yet.

Here are two issues that come to my mind:

 * we currently only have one committer on the PHP side. Not much by 
   way of diversity
 * I've heard mention of how some users are using trunk in production, 
   and that this constrains how the project can function - e.g. trunk
   breakages are not really acceptable. This needs discussing as a 
   community dynamic, as (a) breaking things is an important part of 
   learning, so not being able to makes it much harder for newcomers
   to learn (b) SVN is not a public product of the ASF. It is internal
   to developers. It is public only to allow newcomers to join in the
   effort. The public product of the ASF is releases. Thus, our end
   users should be using releases. If our latest release is too old,
   we should make a new one.

The first of these is probably just a matter of time (if you're a PHP
dev who wants to participate more, start sending us your patches, we're
watching).

The second I'd appreciate hearing more comments on from among the
community - it is an unusual dynamic, and I hope with some discussion we
can come to greater clarity over it.

Regards, Upayavira

Re: Graduation ?

[Lane LiaBraaten <ll...@google.com>]

There's some great feedback in this thread and it's especially good to hear
from the non-Google, non-PST folks.  The general takeaway I get is that
while there are still ways in which we can grow as a project and community,
we're looking good w.r.t. the basic requirements for graduation.

In short, the graduation process is [1]:
 - Hold community graduation vote
 - Create a charter/resolution
 - Hold IPMC recommendation vote
 - Submit resolution to the board

So in order to pursue this, the first step is  to start a '[VOTE] Graduate
Apache Shindig as Top Level Project' thread and cc general@incubator

Another thing we'll need to address is the status page [2] as it looks
generally out of date.  I'm happy to help update it if there is no current
owner.

Cheers,
Lane

[1] http://incubator.apache.org/guides/graduation.html#toplevel
[2] http://incubator.apache.org/projects/shindig.html

On Sat, Jun 27, 2009 at 3:25 AM, Chris Chabot <ch...@google.com> wrote:

> Hey Gabriel, thanks for the feedback!
>
> On Sat, Jun 27, 2009 at 12:20 AM, Gabriel Barros <gbarros@yahoo-inc.com
> >wrote:
>
> > 1. involvement faq: I was going to talk about not much info on how to
> > get involved. But the new website is much clearer! Now i can only push a
> > little further and ask for more clarity on the steps to contribute
> > patches. Or is the email-the-list the preferable one anyways? (never
> > worked on any project at Jira, and i've already seen people using
> > another tool for code reviews)
> >
>
>
> There is some generic Apache text on this matter:
> http://www.apache.org/foundation/getinvolved.html However I agree this
> could
> be expanded on, and link to things like our code guidelines etc.
>
> The process is that someone should submit a .patch file to a Jira issue,
> and
> when doing so check the "I grant the apache foundation full rights to this
> work" checkbox.. that's an important legal bit that makes sure we can
> consume the code without polluting the legal standing of the project. After
> that a committer will look at the patch, give feedback if need be, and
> after
> that's all done, it'll get committed to svn.
>
>
>
> > 2. PHP vs Java vs X: I got the impression that the java version has more
> > contributors and a more complete/tested code base. I may be getting the
> > wrong impression (I only used the bleeding edge code, I serve my
> > development environment directly from the svn trunk). But if it is so,
> > then being more open about this would be  positive.
>
>
> The difference isn't as big as you might think, we have only one committer
> that's dedicated to the php part (ie: me), however we have 3 people who
> contribute regularly (one even full time) next to that, and 8+ people who
> contribute patches and improvements through Jira regularly whenever
> something broke, and a good assortment of people who contribute one patch
> before going into lurker mode again.
>
> Now it's true that some of those contributes are a bit more quiet on the
> shindig-dev list, which I think has something to do with culture and
> language (the majority of the people mentioned above are not US or western
> Europe based) which is why it might seem less then it is; However that
> doesn't reduce anything of the value of the community and code.
>
> Next to that the php version reaches ~450 million registered users on over
> 15 live sites, is being implemented on some 9 more currently (and growing
> almost daily it seems!), and found it's way into > 6 open source projects,
> and that's just the part I know off, many people don't report back to us so
> the actual number is probably a lot higher.
>
> That being said, the trunk was unstable for a little while during the
> implementation of a whole new gadget rendering pipeline to fit in the new
> proxied content, data pipelining and templating bits into it properly, so
> some instability has happened, they don't call it the bleeding edge for
> nothing :) However if you do run into any issues, please report them &
> patches are always welcome! :)
>
> Also, i failed to see a integration test suit that i can run when
> > changing code, or starting an implementation in another language for
> > example.
> >
>
> That might be an unfair measurement since no open source project I know of
> has such a integration test suite, usually it comes down to "Try all the
> code that's meant to work with this and see what breaks" :)
>
> However we do have a pretty good 0.7 and 0.8.1 based compliance test suite
> that can go a long way in helping with this:
> http://code.google.com/p/opensocial-resources/wiki/ComplianceTests
>
> And on the spec list there's some talk about how to build a better and more
> extensible test suite for 0.9, see the thread
>
> http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thread/thread/132347d7c4cd0ca8?hl=enfor
> more info.
>
> 3. Google issue raised on the thread: My 2cents. The main open social
> > player (at least in Brazil, India) is Orkut. Changes happens first in
> > orkut, maybe that gives out the impression that google is steering
> > shindig direction. Most people came to shindig as a way to have a local
> > sandbox.  Even though I've never been able to run any stable orkut app
> > in shindig.
> >
>
> The whole basis for open source is to 'scratch an itch', ie people will
> always work on what's important to them, and if something else is important
> to you, then you contribute a patch for it :)
>
> In this case, sure some of the changes might make it to the orkut or
> iGoogle
> sandbox first, and other changes might make it to netlog or hi5 first,
> completely depending on what priorities exist and what resources people put
> into it; However any container could do exactly the same, you do an svn
> checkout, deploy it to your sandbox and/or production setup, and presto
> you're running the latest version. The only reason why not everyone does
> that is due to time and budget constraints, but there's nothing baring them
> from doing so if they felt it was important enough to invest in it.
>
> On the topic of running orkut apps on your local setup.. Not every
> container
> is 100% compliant, so sometimes a compatibility issue will get in the way,
> but more often it'll simply be the case that the back-end of the gadget
> only
> accepts <some containers> name and their specific certificates for signed
> requests.
>
> So in pretty much all cases where a gadget has a DB per site they support,
> and use signed requests to make calls to their back-end (as they should!),
> you'll have to contact the gadget developer and ask him to add support for
> your container too. Gadget dev's also prefer it that way since it allows
> them to customize the gadget's look and feel for the specific site (color,
> size, etc), see if it matches their target audience and there's the issue
> of
> language and culture too of course.
>
>   -- Chris
>

Re: Graduation ?

[Chris Chabot <ch...@google.com>]

Hey Gabriel, thanks for the feedback!

On Sat, Jun 27, 2009 at 12:20 AM, Gabriel Barros <gb...@yahoo-inc.com>wrote:

> 1. involvement faq: I was going to talk about not much info on how to
> get involved. But the new website is much clearer! Now i can only push a
> little further and ask for more clarity on the steps to contribute
> patches. Or is the email-the-list the preferable one anyways? (never
> worked on any project at Jira, and i've already seen people using
> another tool for code reviews)
>


There is some generic Apache text on this matter:
http://www.apache.org/foundation/getinvolved.html However I agree this could
be expanded on, and link to things like our code guidelines etc.

The process is that someone should submit a .patch file to a Jira issue, and
when doing so check the "I grant the apache foundation full rights to this
work" checkbox.. that's an important legal bit that makes sure we can
consume the code without polluting the legal standing of the project. After
that a committer will look at the patch, give feedback if need be, and after
that's all done, it'll get committed to svn.



> 2. PHP vs Java vs X: I got the impression that the java version has more
> contributors and a more complete/tested code base. I may be getting the
> wrong impression (I only used the bleeding edge code, I serve my
> development environment directly from the svn trunk). But if it is so,
> then being more open about this would be  positive.


The difference isn't as big as you might think, we have only one committer
that's dedicated to the php part (ie: me), however we have 3 people who
contribute regularly (one even full time) next to that, and 8+ people who
contribute patches and improvements through Jira regularly whenever
something broke, and a good assortment of people who contribute one patch
before going into lurker mode again.

Now it's true that some of those contributes are a bit more quiet on the
shindig-dev list, which I think has something to do with culture and
language (the majority of the people mentioned above are not US or western
Europe based) which is why it might seem less then it is; However that
doesn't reduce anything of the value of the community and code.

Next to that the php version reaches ~450 million registered users on over
15 live sites, is being implemented on some 9 more currently (and growing
almost daily it seems!), and found it's way into > 6 open source projects,
and that's just the part I know off, many people don't report back to us so
the actual number is probably a lot higher.

That being said, the trunk was unstable for a little while during the
implementation of a whole new gadget rendering pipeline to fit in the new
proxied content, data pipelining and templating bits into it properly, so
some instability has happened, they don't call it the bleeding edge for
nothing :) However if you do run into any issues, please report them &
patches are always welcome! :)

Also, i failed to see a integration test suit that i can run when
> changing code, or starting an implementation in another language for
> example.
>

That might be an unfair measurement since no open source project I know of
has such a integration test suite, usually it comes down to "Try all the
code that's meant to work with this and see what breaks" :)

However we do have a pretty good 0.7 and 0.8.1 based compliance test suite
that can go a long way in helping with this:
http://code.google.com/p/opensocial-resources/wiki/ComplianceTests

And on the spec list there's some talk about how to build a better and more
extensible test suite for 0.9, see the thread
http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thread/thread/132347d7c4cd0ca8?hl=enfor
more info.

3. Google issue raised on the thread: My 2cents. The main open social
> player (at least in Brazil, India) is Orkut. Changes happens first in
> orkut, maybe that gives out the impression that google is steering
> shindig direction. Most people came to shindig as a way to have a local
> sandbox.  Even though I've never been able to run any stable orkut app
> in shindig.
>

The whole basis for open source is to 'scratch an itch', ie people will
always work on what's important to them, and if something else is important
to you, then you contribute a patch for it :)

In this case, sure some of the changes might make it to the orkut or iGoogle
sandbox first, and other changes might make it to netlog or hi5 first,
completely depending on what priorities exist and what resources people put
into it; However any container could do exactly the same, you do an svn
checkout, deploy it to your sandbox and/or production setup, and presto
you're running the latest version. The only reason why not everyone does
that is due to time and budget constraints, but there's nothing baring them
from doing so if they felt it was important enough to invest in it.

On the topic of running orkut apps on your local setup.. Not every container
is 100% compliant, so sometimes a compatibility issue will get in the way,
but more often it'll simply be the case that the back-end of the gadget only
accepts <some containers> name and their specific certificates for signed
requests.

So in pretty much all cases where a gadget has a DB per site they support,
and use signed requests to make calls to their back-end (as they should!),
you'll have to contact the gadget developer and ask him to add support for
your container too. Gadget dev's also prefer it that way since it allows
them to customize the gadget's look and feel for the specific site (color,
size, etc), see if it matches their target audience and there's the issue of
language and culture too of course.

   -- Chris

Re: Graduation ?

[Gabriel Barros <gb...@yahoo-inc.com>]

Le mercredi 24 juin 2009 à 11:04 +0200, Chris Chabot a écrit :
> If anyone (from mentors to contributers and especially mailing list lurkers)
> feels differently, then please let us know so we know to fix it ... but from
> where I'm standing I would say we look about ready!

Agreed. but even what's good can improve :)

1. involvement faq: I was going to talk about not much info on how to
get involved. But the new website is much clearer! Now i can only push a
little further and ask for more clarity on the steps to contribute
patches. Or is the email-the-list the preferable one anyways? (never
worked on any project at Jira, and i've already seen people using
another tool for code reviews)

2. PHP vs Java vs X: I got the impression that the java version has more
contributors and a more complete/tested code base. I may be getting the
wrong impression (I only used the bleeding edge code, I serve my
development environment directly from the svn trunk). But if it is so,
then being more open about this would be  positive.
Also, i failed to see a integration test suit that i can run when
changing code, or starting an implementation in another language for
example.

3. Google issue raised on the thread: My 2cents. The main open social
player (at least in Brazil, India) is Orkut. Changes happens first in
orkut, maybe that gives out the impression that google is steering
shindig direction. Most people came to shindig as a way to have a local
sandbox.  Even though I've never been able to run any stable orkut app
in shindig.


--gabriel barros, a newcomer lurker.

Re: Graduation ?

[Chris Chabot <ch...@google.com>]

ps thanks for that feedback.

In many open source projects the lurkers make for the silent majority, so
getting feedback from that perspective is very useful and valuable!

    -- Chris

On Wed, Jun 24, 2009 at 1:37 PM, Tim Wintle <ti...@teamrubber.com>wrote:

> On Wed, 2009-06-24 at 11:04 +0200, Chris Chabot wrote:
> > We've also had our technical disagreements (quite a bit more then
> > one:) which, since we're all here still, we seem to have survived just
> > fine; And I would like to think that we do have an open and inclusive
> > atmosphere.
> >
> > If anyone (from mentors to contributers and especially mailing list
> > lurkers) feels differently, then please let us know so we know to fix
> > it ... but from where I'm standing I would say we look about ready!
>
> Guess I'm fairly much in the "mailing list lurkers" section having only
> submitted one (minor) patch - and I'd agree with your sentiment about
> having an open and inclusive atmosphere.
>
> Tim Wintle
>
>

Re: Graduation ?

[Tim Wintle <ti...@teamrubber.com>]

On Wed, 2009-06-24 at 11:04 +0200, Chris Chabot wrote:
> We've also had our technical disagreements (quite a bit more then
> one:) which, since we're all here still, we seem to have survived just
> fine; And I would like to think that we do have an open and inclusive
> atmosphere.
> 
> If anyone (from mentors to contributers and especially mailing list
> lurkers) feels differently, then please let us know so we know to fix
> it ... but from where I'm standing I would say we look about ready!

Guess I'm fairly much in the "mailing list lurkers" section having only
submitted one (minor) patch - and I'd agree with your sentiment about
having an open and inclusive atmosphere.

Tim Wintle

Re: Graduation ?

[Chris Chabot <ch...@google.com>]

There's been a bit more clarification posted, and on the point that has
concerned us the most in the past (diversity), this was said:

*Often podlings have to add at least one new committer just to meet the
diversity requirement of at least 3 'independent' committers.....
<snip>
As of a year or two ago when I last looked at the statistics, most
podlings only ever add one committer during incubation.
*
Since we've started we've added Ian, Vincent, Paul and Chico, so well over
the '1 new committer' that most graduating podlings see.

We've also had our technical disagreements (quite a bit more then one:)
which, since we're all here still, we seem to have survived just fine; And I
would like to think that we do have an open and inclusive atmosphere.

If anyone (from mentors to contributers and especially mailing list lurkers)
feels differently, then please let us know so we know to fix it ... but from
where I'm standing I would say we look about ready!

   -- Chris

On Wed, Jun 24, 2009 at 12:54 AM, Ian Boston <ie...@tfd.co.uk> wrote:

> I notice on general@incubator there is a thread starting on criteria for
> graduation, initially stated as
>
> - when the community has performed at least one Apache release
> - when the community has cleaned the code from all license issues
> - when the community is diverse and open
>
> The "diverse and open" is in the process of being defined in more detail.
>
> My question:
>
> Is Shindig ready for graduation ?
>
> Ian
>