You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@community.apache.org by James Carman <ja...@carmanconsulting.com> on 2015/04/16 23:12:17 UTC
Project Visualization Tool...
At ApacheCon, we discussed creating a project visualization tool to
help folks navigation the ever-growing number of projects we have here
at the ASF. The idea would be to allow folks to see some form of "tag
cloud" or something (with the tags being the projects themselves), but
the cloud is interactive, allowing filtering by various dimensions
(size of project, age, relationships to other projects, programming
language, etc.).
We already have a new projects page in the works:
https://projects-new.apache.org/
which displays quite a bit of information. Where do we get that
information? Do folks have any other ideas about different ways of
browsing/exploring the projects? One idea we have is to lean on
TinkerPop (currently incubating) to load the data into a graph
structure to allow the data to be easily manipulated (the "gremlin"
language allows you to traverse the graph in this way very easily).
Thoughts?
James Carman
Re: Project Visualization Tool...
Posted by jan i <ja...@apache.org>.
On Saturday, April 18, 2015, Shane Curcuru <as...@shanecurcuru.org> wrote:
> We had a great session, and a lot of energy, hopefully we can make some
> progress. One note: this needs to be a comdev PMC project, and we need
> to really plan the data part out if we want to be successful.
Yeah we had a great discussion, but the original goal got lost in all kind
of technical details (which was actually interesting)
the original discussion was something like "how do I find related projects
to contribute to?" and I believe the nice graph is part of that solution
but not the whole solution. I believe we could in addition use a "ebay"
page, where projects can make small advertisements about a problem or sub
project they need solved. There are probably committers in other projects
who fancy solving it and an advert might make them interested in the
project.
> Note that projects-new.a.o is the planned future replacement for
> projects.a.o - there are *significant* differences, so you need to look
> at the About page and the source repo. In particular, the new site uses
> it's own new JSON generated sources which (I think) will no longer use
> the DOAPs.
>
> In particular, Infra currently does *not* consider either the data
> gathering (i.e. populating the JSON behind the projects-new site) nor
> the visualizations (current or ones we want to build) as core supported
> services. So whatever we build needs to be maintained by this PMC to
> start with.
which is similar to today.
rgds
jan i
>
> Also, Link dump of useful related bits: ----------------
>
> Old service, based on crappy cron jobs and DOAP files from projects:
> https://projects.apache.org/
>
> New service, soon to be infra supported, relying on JSON data generated
> by infra on a regular schedule:
> https://projects-new.apache.org/
>
> Useful PMC chair report helper, that surfaces a number of different
> statistics about your PMC(s), including mailing list stats,
> PMC/committer changes, some software releases, etc. etc. (Members have
> visibility to all PMCs):
> https://reporter.apache.org
>
> Rob Weir (AOO, Member) used to do some visualization stuff and might
> have code ideas:
> http://www.robweir.com/blog/2013/05/mapping-apache.html
>
> Ken Coar's old mailing list stats page:
>
> https://people.apache.org/~coar/mlists.html
>
> The AOO project wrote a mailing list visualizer for who talks to whom:
> https://blogs.apache.org/OOo/entry/visualizing_the_aoo_dev_list
>
> Some outside statistics FLOSSmole generated about Apache communities and
> lists:
> http://flossmole.org/category/tags/apache
>
> Random other interesting analytics:
> The Subversion project has the "contribulyzer"
>
>
>
> - Shane
>
--
Sent from My iPad, sorry for any misspellings.
Re : Re: Project Visualization Tool...
Posted by he...@free.fr.
It was told the new site would use native json, instead of doap
But I'm not convinced at all, since Doap is an invaluable source of info, documented, and so on
then imho it would be better to generate json from doap
I disabled the json edit feature recently since it will cause problems
regards
Hervé
----- Mail d'origine -----
De: Shane Curcuru <as...@shanecurcuru.org>
À: dev@community.apache.org
Envoyé: Sat, 18 Apr 2015 06:43:37 +0200 (CEST)
Objet: Re: Project Visualization Tool...
We had a great session, and a lot of energy, hopefully we can make some
progress. One note: this needs to be a comdev PMC project, and we need
to really plan the data part out if we want to be successful.
Note that projects-new.a.o is the planned future replacement for
projects.a.o - there are *significant* differences, so you need to look
at the About page and the source repo. In particular, the new site uses
it's own new JSON generated sources which (I think) will no longer use
the DOAPs.
In particular, Infra currently does *not* consider either the data
gathering (i.e. populating the JSON behind the projects-new site) nor
the visualizations (current or ones we want to build) as core supported
services. So whatever we build needs to be maintained by this PMC to
start with.
Also, Link dump of useful related bits: ----------------
Old service, based on crappy cron jobs and DOAP files from projects:
https://projects.apache.org/
New service, soon to be infra supported, relying on JSON data generated
by infra on a regular schedule:
https://projects-new.apache.org/
Useful PMC chair report helper, that surfaces a number of different
statistics about your PMC(s), including mailing list stats,
PMC/committer changes, some software releases, etc. etc. (Members have
visibility to all PMCs):
https://reporter.apache.org
Rob Weir (AOO, Member) used to do some visualization stuff and might
have code ideas:
http://www.robweir.com/blog/2013/05/mapping-apache.html
Ken Coar's old mailing list stats page:
https://people.apache.org/~coar/mlists.html
The AOO project wrote a mailing list visualizer for who talks to whom:
https://blogs.apache.org/OOo/entry/visualizing_the_aoo_dev_list
Some outside statistics FLOSSmole generated about Apache communities and
lists:
http://flossmole.org/category/tags/apache
Random other interesting analytics:
The Subversion project has the "contribulyzer"
- Shane
Re: Project Visualization Tool...
Posted by Shane Curcuru <as...@shanecurcuru.org>.
We had a great session, and a lot of energy, hopefully we can make some
progress. One note: this needs to be a comdev PMC project, and we need
to really plan the data part out if we want to be successful.
Note that projects-new.a.o is the planned future replacement for
projects.a.o - there are *significant* differences, so you need to look
at the About page and the source repo. In particular, the new site uses
it's own new JSON generated sources which (I think) will no longer use
the DOAPs.
In particular, Infra currently does *not* consider either the data
gathering (i.e. populating the JSON behind the projects-new site) nor
the visualizations (current or ones we want to build) as core supported
services. So whatever we build needs to be maintained by this PMC to
start with.
Also, Link dump of useful related bits: ----------------
Old service, based on crappy cron jobs and DOAP files from projects:
https://projects.apache.org/
New service, soon to be infra supported, relying on JSON data generated
by infra on a regular schedule:
https://projects-new.apache.org/
Useful PMC chair report helper, that surfaces a number of different
statistics about your PMC(s), including mailing list stats,
PMC/committer changes, some software releases, etc. etc. (Members have
visibility to all PMCs):
https://reporter.apache.org
Rob Weir (AOO, Member) used to do some visualization stuff and might
have code ideas:
http://www.robweir.com/blog/2013/05/mapping-apache.html
Ken Coar's old mailing list stats page:
https://people.apache.org/~coar/mlists.html
The AOO project wrote a mailing list visualizer for who talks to whom:
https://blogs.apache.org/OOo/entry/visualizing_the_aoo_dev_list
Some outside statistics FLOSSmole generated about Apache communities and
lists:
http://flossmole.org/category/tags/apache
Random other interesting analytics:
The Subversion project has the "contribulyzer"
- Shane
Fwd: Re: Project Visualization Tool...
Posted by Marlon Pierce <ma...@iu.edu>.
Interesting little graphic--
Marlon
-------- Forwarded Message --------
Subject: Re: Project Visualization Tool...
Date: Thu, 16 Apr 2015 16:15:02 -0500
From: Roman Shaposhnik <ro...@shaposhnik.org>
Reply-To: dev@community.apache.org
To: ComDev <de...@community.apache.org>
Btw, this is the visualization that I showed to everybody
at BarCamp:
http://www.robweir.com/blog/2013/05/mapping-apache.html
Thanks,
Roman.
On Thu, Apr 16, 2015 at 4:12 PM, James Carman
<ja...@carmanconsulting.com> wrote:
> At ApacheCon, we discussed creating a project visualization tool to
> help folks navigation the ever-growing number of projects we have here
> at the ASF. The idea would be to allow folks to see some form of "tag
> cloud" or something (with the tags being the projects themselves), but
> the cloud is interactive, allowing filtering by various dimensions
> (size of project, age, relationships to other projects, programming
> language, etc.).
>
> We already have a new projects page in the works:
>
> https://projects-new.apache.org/
>
> which displays quite a bit of information. Where do we get that
> information? Do folks have any other ideas about different ways of
> browsing/exploring the projects? One idea we have is to lean on
> TinkerPop (currently incubating) to load the data into a graph
> structure to allow the data to be easily manipulated (the "gremlin"
> language allows you to traverse the graph in this way very easily).
>
> Thoughts?
>
> James Carman
Re: Project Visualization Tool...
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
Btw, this is the visualization that I showed to everybody
at BarCamp:
http://www.robweir.com/blog/2013/05/mapping-apache.html
Thanks,
Roman.
On Thu, Apr 16, 2015 at 4:12 PM, James Carman
<ja...@carmanconsulting.com> wrote:
> At ApacheCon, we discussed creating a project visualization tool to
> help folks navigation the ever-growing number of projects we have here
> at the ASF. The idea would be to allow folks to see some form of "tag
> cloud" or something (with the tags being the projects themselves), but
> the cloud is interactive, allowing filtering by various dimensions
> (size of project, age, relationships to other projects, programming
> language, etc.).
>
> We already have a new projects page in the works:
>
> https://projects-new.apache.org/
>
> which displays quite a bit of information. Where do we get that
> information? Do folks have any other ideas about different ways of
> browsing/exploring the projects? One idea we have is to lean on
> TinkerPop (currently incubating) to load the data into a graph
> structure to allow the data to be easily manipulated (the "gremlin"
> language allows you to traverse the graph in this way very easily).
>
> Thoughts?
>
> James Carman
Re: Project Visualization Tool...
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
This is all taken from our DOAP file
On Thu, Apr 16, 2015 at 5:45 PM, Konstantin Boudnik <co...@apache.org> wrote:
> Wow. great stuff! I was wondering how do you get
> the projects-per-language stats? E.g. as a Groovy aficionado I looked at
>
> https://projects-new.apache.org/projects.html?language#Groovy
>
> and don't see Apache Bigtop which uses Groovy and Gradle heavily.
>
> Thanks!
> Cos
>
>
> On Thu, Apr 16, 2015 at 04:12PM, James Carman wrote:
>> At ApacheCon, we discussed creating a project visualization tool to
>> help folks navigation the ever-growing number of projects we have here
>> at the ASF. The idea would be to allow folks to see some form of "tag
>> cloud" or something (with the tags being the projects themselves), but
>> the cloud is interactive, allowing filtering by various dimensions
>> (size of project, age, relationships to other projects, programming
>> language, etc.).
>>
>> We already have a new projects page in the works:
>>
>> https://projects-new.apache.org/
>>
>> which displays quite a bit of information. Where do we get that
>> information? Do folks have any other ideas about different ways of
>> browsing/exploring the projects? One idea we have is to lean on
>> TinkerPop (currently incubating) to load the data into a graph
>> structure to allow the data to be easily manipulated (the "gremlin"
>> language allows you to traverse the graph in this way very easily).
>>
>> Thoughts?
>>
>> James Carman
Re: Project Visualization Tool...
Posted by Konstantin Boudnik <co...@apache.org>.
Wow. great stuff! I was wondering how do you get
the projects-per-language stats? E.g. as a Groovy aficionado I looked at
https://projects-new.apache.org/projects.html?language#Groovy
and don't see Apache Bigtop which uses Groovy and Gradle heavily.
Thanks!
Cos
On Thu, Apr 16, 2015 at 04:12PM, James Carman wrote:
> At ApacheCon, we discussed creating a project visualization tool to
> help folks navigation the ever-growing number of projects we have here
> at the ASF. The idea would be to allow folks to see some form of "tag
> cloud" or something (with the tags being the projects themselves), but
> the cloud is interactive, allowing filtering by various dimensions
> (size of project, age, relationships to other projects, programming
> language, etc.).
>
> We already have a new projects page in the works:
>
> https://projects-new.apache.org/
>
> which displays quite a bit of information. Where do we get that
> information? Do folks have any other ideas about different ways of
> browsing/exploring the projects? One idea we have is to lean on
> TinkerPop (currently incubating) to load the data into a graph
> structure to allow the data to be easily manipulated (the "gremlin"
> language allows you to traverse the graph in this way very easily).
>
> Thoughts?
>
> James Carman
Re : Re: Project Visualization Tool...
Posted by he...@free.fr.
If you look at sources, part of it in a crontab
But not everything: I'm trying to improve the automated extractions, eventually fixing source data, to be able to do the full extracts through cron
the code is open to every committer: don't hesitate to modify it :)
Regards
Hervé
----- Mail d'origine -----
De: David Crossley <cr...@apache.org>
À: dev@community.apache.org
Envoyé: Fri, 17 Apr 2015 08:46:10 +0200 (CEST)
Objet: Re: Project Visualization Tool...
On Thu, Apr 16, 2015 at 04:12:17PM -0500, James Carman wrote:
> At ApacheCon, we discussed creating a project visualization tool to
> help folks navigation the ever-growing number of projects we have here
> at the ASF. The idea would be to allow folks to see some form of "tag
> cloud" or something (with the tags being the projects themselves), but
> the cloud is interactive, allowing filtering by various dimensions
> (size of project, age, relationships to other projects, programming
> language, etc.).
>
> We already have a new projects page in the works:
>
> https://projects-new.apache.org/
>
> which displays quite a bit of information. Where do we get that
> information?
Herve added an "About" page recently:
https://projects-new.apache.org/about.html
Each project manages their own DOAP file.
Those files are listed at the old projects.a.o
Watching these mails lists recently, i gather that someone
needs to run a script manually to re-populate the project-new.a.o
-David
> Do folks have any other ideas about different ways of
> browsing/exploring the projects? One idea we have is to lean on
> TinkerPop (currently incubating) to load the data into a graph
> structure to allow the data to be easily manipulated (the "gremlin"
> language allows you to traverse the graph in this way very easily).
>
> Thoughts?
>
> James Carman
Re: Project Visualization Tool...
Posted by David Crossley <cr...@apache.org>.
On Thu, Apr 16, 2015 at 04:12:17PM -0500, James Carman wrote:
> At ApacheCon, we discussed creating a project visualization tool to
> help folks navigation the ever-growing number of projects we have here
> at the ASF. The idea would be to allow folks to see some form of "tag
> cloud" or something (with the tags being the projects themselves), but
> the cloud is interactive, allowing filtering by various dimensions
> (size of project, age, relationships to other projects, programming
> language, etc.).
>
> We already have a new projects page in the works:
>
> https://projects-new.apache.org/
>
> which displays quite a bit of information. Where do we get that
> information?
Herve added an "About" page recently:
https://projects-new.apache.org/about.html
Each project manages their own DOAP file.
Those files are listed at the old projects.a.o
Watching these mails lists recently, i gather that someone
needs to run a script manually to re-populate the project-new.a.o
-David
> Do folks have any other ideas about different ways of
> browsing/exploring the projects? One idea we have is to lean on
> TinkerPop (currently incubating) to load the data into a graph
> structure to allow the data to be easily manipulated (the "gremlin"
> language allows you to traverse the graph in this way very easily).
>
> Thoughts?
>
> James Carman