Format for project data: DOAP/JSON/other (was: Is https://projects-new.apache.org/ ready for prime time?)

[sebb <se...@gmail.com>]

[Starting thread with new subject]

On 21 June 2015 at 16:03, Hervé BOUTEMY <he...@free.fr> wrote:
> Le dimanche 21 juin 2015 15:54:29 jan i a écrit :
>> On 21 June 2015 at 15:48, Daniel Gruno <hu...@apache.org> wrote:
>> > I think the site is ready for a more prominent role, but I find this
>> > discussion confusing, and I find it somewhat sad that we're gonna stick
>> > with something as arcane as DOAP.
>>
>> +100 !!
>>
>> DOAP == Dead On Arrival Permanently :-) JSON == Jump Simply On New
>> (but I know I am only 1 voice).
> step by step, please: this will avoid confusion between independant topics
>
> switching without disturbing current conventions/knowledge is something that
> already takes a long time and energy: I know it because I put a lot of energy
> on it for a few monthes now!
>
> We started a discussion on this source format topic during april, and AFAIK
> nobody worked on it.
>
> What I'd like now is to switch: we can discuss later on what we want to change
> (and communication to every comittees this requires).
> With the new site, we'll be able to change formats if we want, the only
> requirement is to have json files for the visualization

Some PMCs are very responsive to requests to maintain DOAP files;
others take months and multiple reminders even for a simple fix. This
does not directly affect the format used to hold the data. However it
does mean that changing to a new format is likely to take a lot of
time and involve a lot of work. Meanwhile the code will need to
continue to support the old format.

It would however be an opportunity to make some improvements. For example:
- we could be more specific about the data that really needs to be
maintained by PMCs
- we could require that the files are stored in a well-known place,
rather than requiring an index file to find them.

Re: Format for project data: DOAP/JSON/other (was: Is https://projects-new.apache.org/ ready for prime time?)

[Christopher <ct...@apache.org>]

On Sun, Jun 21, 2015 at 11:29 AM, sebb <se...@gmail.com> wrote:
> [Starting thread with new subject]
>
> On 21 June 2015 at 16:03, Hervé BOUTEMY <he...@free.fr> wrote:
>> Le dimanche 21 juin 2015 15:54:29 jan i a écrit :
>>> On 21 June 2015 at 15:48, Daniel Gruno <hu...@apache.org> wrote:
>>> > I think the site is ready for a more prominent role, but I find this
>>> > discussion confusing, and I find it somewhat sad that we're gonna stick
>>> > with something as arcane as DOAP.
>>>
>>> +100 !!
>>>
>>> DOAP == Dead On Arrival Permanently :-) JSON == Jump Simply On New
>>> (but I know I am only 1 voice).
>> step by step, please: this will avoid confusion between independant topics
>>
>> switching without disturbing current conventions/knowledge is something that
>> already takes a long time and energy: I know it because I put a lot of energy
>> on it for a few monthes now!
>>
>> We started a discussion on this source format topic during april, and AFAIK
>> nobody worked on it.
>>
>> What I'd like now is to switch: we can discuss later on what we want to change
>> (and communication to every comittees this requires).
>> With the new site, we'll be able to change formats if we want, the only
>> requirement is to have json files for the visualization
>
> Some PMCs are very responsive to requests to maintain DOAP files;
> others take months and multiple reminders even for a simple fix. This
> does not directly affect the format used to hold the data. However it
> does mean that changing to a new format is likely to take a lot of
> time and involve a lot of work. Meanwhile the code will need to
> continue to support the old format.
>
> It would however be an opportunity to make some improvements. For example:
> - we could be more specific about the data that really needs to be
> maintained by PMCs
> - we could require that the files are stored in a well-known place,
> rather than requiring an index file to find them.

Personally, I'd like to see a nice web interface for a project's
"profile", akin to social media profiles or one's personal profile at
nearly every site on the web. Whatever file format is used behind the
scenes shouldn't matter for PMCs. I think projects-new started us off
in the right direction, essentially doing this for release
information, but really everything about a project could be contained
in a profile managed by a web interface of this type.

I don't think adding requirements to these files (like a well-known
location) are going to make them more maintainable for PMCs. It's just
going to be one more thing for them to remember to get right for
something which is relatively rarely edited. It'd be much easier to
make projects.apache.org be for projects what id.apache.org is for
committers.

For comparison, Fedora seems to do a decent job of providing a
consistent web view of all aspects of a project:
https://apps.fedoraproject.org/packages/java-1.8.0-openjdk . There's
actually several different views, but in general, there's links to
contributors, upstream project page, builds, bug tracker, "updates"
(analogous to releases), source, etc. I realize Apache projects aren't
as homogeneous as package management in Fedora, which makes things a
bit difficult for us, but I do think there's probably some lessons to
be gained. For one, most of those web views are auto-generated, based
on maintainer and user actions (like triggering an update, filing a
bug, applying a patch, or retiring a package). As such, these
interfaces tend not to be editable. However, one can pretty easily
imagine similar project profile pages which are editable by a PMC.

However, I'm not really all that familiar with the historical need for
DOAP files, what tools actually use them, or the historical
preferences for maintaining them. So, I admittedly may not fully grasp
the scope of the problem/changes being discussed. I'm just thinking
that if we can get rid of the requirement for PMCs to maintain them
(in favor of a web-interface way for PMCs to describe all aspects of
their projects), then I'd like that.

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii

Re: Format for project data: DOAP/JSON/other (was: Is https://projects-new.apache.org/ ready for prime time?)

[Hervé BOUTEMY <he...@free.fr>]

from my experience when working on the topic, what was very confusing was PMC 
descriptors vs projects DOAP files, and the fact that everything was mixed in 
code [1]

Then I just copied content in projects-new data directory and renamed things 
to be clear about these 2 types of descriptors [2]: IMHO, this will already 
help a lot


Another thing I did is that Python scripts that parse descriptors (projects 
and committees) to generate json files also copy .rdf files in 
/doap/{committeeId} [3]


Last thing at the moment: once again, I'm not a DOAP expert, and there seem to 
be semantic web experts in ASF who should help others better understand how to 
use that. But at the moment, the format seems really not well understood by a 
vast majority of people, even if some advanced rdf files [4] show what can be 
done

Regards,

Hervé

[1] http://svn.apache.org/viewvc/infrastructure/site-tools/trunk/projects/

[2] http://svn.apache.org/viewvc/comdev/projects.apache.org/data/

[3] https://projects-new.apache.org/doap/
http://svn.apache.org/viewvc/comdev/projects.apache.org/site/doap/

[4] http://svn.apache.org/repos/asf/oltu/site/doap_org.apache.oltu.parent.rdf

Le dimanche 21 juin 2015 16:29:19 sebb a écrit :
> [Starting thread with new subject]
> 
> On 21 June 2015 at 16:03, Hervé BOUTEMY <he...@free.fr> wrote:
> > Le dimanche 21 juin 2015 15:54:29 jan i a écrit :
> >> On 21 June 2015 at 15:48, Daniel Gruno <hu...@apache.org> wrote:
> >> > I think the site is ready for a more prominent role, but I find this
> >> > discussion confusing, and I find it somewhat sad that we're gonna stick
> >> > with something as arcane as DOAP.
> >> 
> >> +100 !!
> >> 
> >> DOAP == Dead On Arrival Permanently :-) JSON == Jump Simply On New
> >> (but I know I am only 1 voice).
> > 
> > step by step, please: this will avoid confusion between independant topics
> > 
> > switching without disturbing current conventions/knowledge is something
> > that already takes a long time and energy: I know it because I put a lot
> > of energy on it for a few monthes now!
> > 
> > We started a discussion on this source format topic during april, and
> > AFAIK
> > nobody worked on it.
> > 
> > What I'd like now is to switch: we can discuss later on what we want to
> > change (and communication to every comittees this requires).
> > With the new site, we'll be able to change formats if we want, the only
> > requirement is to have json files for the visualization
> 
> Some PMCs are very responsive to requests to maintain DOAP files;
> others take months and multiple reminders even for a simple fix. This
> does not directly affect the format used to hold the data. However it
> does mean that changing to a new format is likely to take a lot of
> time and involve a lot of work. Meanwhile the code will need to
> continue to support the old format.
> 
> It would however be an opportunity to make some improvements. For example:
> - we could be more specific about the data that really needs to be
> maintained by PMCs
> - we could require that the files are stored in a well-known place,
> rather than requiring an index file to find them.