automating history charts and Clutch

[David Crossley <cr...@apache.org>]

sebb wrote:
> David Crossley wrote:
> > Jukka Zitting wrote:
> >> crossley wrote:
> >> > r1386462
> >> > Modified:
> >> >     incubator/public/trunk/content/history/current.txt
> >>
> >> Thanks for keeping that file up to date! The graph at
> >> http://incubator.apache.org/history/ is quite useful.
> >>
> >> However, I'm wondering if there really is need to manually maintain a
> >> separate text file when all the required information should already be
> >> present in podlings.xml. Can we generate the current.txt file from
> >> podlings.xml, or (even better) have the client create the history
> >> graph directly from podlings.xml?
> >
> > It would be fantastic if someone could automate that.
> > At the moment, the current.txt (yellow) is maintained manually
> > and the entry.txt (green) is automatically handled by Clutch.
> > Ideally both would be automated independently of Clutch.
> 
> Perhaps add a task to build.xml that generates current.txt and
> entry.txt from podlings.xml?
> Could extract the entry.txt code from clutch and extend it to handle current.txt
> 
> If that sounds reasonable, I'm happy to give that a try.

That seems like a good solution, now that we have the podlings.xml file.

> Maybe at some point clutch itself could be run automatically whenever
> podlings.xml is updated.

Perhaps just run it twice per day regardless. There are other
things that it assesses too.

Yesterday the podlings.xml was updated a number of times
in quick succession to catch up with laggard graduates.
So would not want to run cumbersome Clutch on each update.
Or perhaps it could be triggered by Ant, say 30 minutes after
the update, otherwise once per day.

Clutch seems reasonably robust now. The main thing that
used to trip it up was project names with inconsistent
spellings for different resources. The recent addition of
the "resourceAliases" attribute in podlings.xml does assist.

-David

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: automating history charts and Clutch

[David Crossley <cr...@apache.org>]

sebb wrote:
> >
> > Not sure why the order is currently reversed; the Javascript does not
> > seem to care about ordering.
> >
> > Is there a reason for the reverse data order?

There was no real reason, just that manual editing was easier at the top.

> > I've started looking at current.txt.
> 
> Done, though the process requires xslt and Perl, as I could not work
> out how to create a running total in XSLT.
> 
> Also the output is in increasing date order and includes one entry for
> each distinct start/end date.
> Not sure what dates were use originally as they don't all seem to
> appear in podlings.xml.

The dates were approximate around when they sent their vote result email.
Many graduates do not bother to tidy up their resources, so i made it easy for me.

> The final total is 1 different from before, but agrees with podlings.xml

I interpret it differently. There are 43 "current" in podlings.xml
and that agrees with what Clutch gets from that file. Also there are 43
in the generated target/current.ent file. In the new history/current.txt
there are 44.

Ah, doing some grep and diff stuff:
Composer retired but has no enddate.

-David

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: automating history charts and Clutch

[sebb <se...@gmail.com>]

On 18 September 2012 12:07, sebb <se...@gmail.com> wrote:
> On 18 September 2012 03:32, David Crossley <cr...@apache.org> wrote:
>> sebb wrote:
>>> David Crossley wrote:
>>> > Jukka Zitting wrote:
>>> >> crossley wrote:
>>> >> > r1386462
>>> >> > Modified:
>>> >> >     incubator/public/trunk/content/history/current.txt
>>> >>
>>> >> Thanks for keeping that file up to date! The graph at
>>> >> http://incubator.apache.org/history/ is quite useful.
>>> >>
>>> >> However, I'm wondering if there really is need to manually maintain a
>>> >> separate text file when all the required information should already be
>>> >> present in podlings.xml. Can we generate the current.txt file from
>>> >> podlings.xml, or (even better) have the client create the history
>>> >> graph directly from podlings.xml?
>>> >
>>> > It would be fantastic if someone could automate that.
>>> > At the moment, the current.txt (yellow) is maintained manually
>>> > and the entry.txt (green) is automatically handled by Clutch.
>>> > Ideally both would be automated independently of Clutch.
>>>
>>> Perhaps add a task to build.xml that generates current.txt and
>>> entry.txt from podlings.xml?
>>> Could extract the entry.txt code from clutch and extend it to handle current.txt
>>>
>>> If that sounds reasonable, I'm happy to give that a try.
>>
>> That seems like a good solution, now that we have the podlings.xml file.
>
> I've created an XSLT script that generates a version of entry.txt.
> It currently generates the output in date order rather than reverse
> date order as that is easier - one can use position() for the number.

Eventually went with reverse data order as it was not much more difficult.

> Not sure why the order is currently reversed; the Javascript does not
> seem to care about ordering.
>
> Is there a reason for the reverse data order?
>
> I've started looking at current.txt.

Done, though the process requires xslt and Perl, as I could not work
out how to create a running total in XSLT.

Also the output is in increasing date order and includes one entry for
each distinct start/end date.
Not sure what dates were use originally as they don't all seem to
appear in podlings.xml.

The final total is 1 different from before, but agrees with podlings.xml

>>> Maybe at some point clutch itself could be run automatically whenever
>>> podlings.xml is updated.
>>
>> Perhaps just run it twice per day regardless. There are other
>> things that it assesses too.
>
> I'd forgotten that.
>
>> Yesterday the podlings.xml was updated a number of times
>> in quick succession to catch up with laggard graduates.
>> So would not want to run cumbersome Clutch on each update.
>
> Very true, it does take quite a while.
>
>> Or perhaps it could be triggered by Ant, say 30 minutes after
>> the update, otherwise once per day.
>
> Tricky to get that right.
>
>> Clutch seems reasonably robust now. The main thing that
>> used to trip it up was project names with inconsistent
>> spellings for different resources. The recent addition of
>> the "resourceAliases" attribute in podlings.xml does assist.
>
> Yes, though we would need to ensure that the log file was available
> for inspection.
>
> Could perhaps be run by the apsite login on minotaur. This runs the
> people and projects builds.

Just realised that apsite won't be able to commit the generated output.

>> -David
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: automating history charts and Clutch

[sebb <se...@gmail.com>]

On 18 September 2012 03:32, David Crossley <cr...@apache.org> wrote:
> sebb wrote:
>> David Crossley wrote:
>> > Jukka Zitting wrote:
>> >> crossley wrote:
>> >> > r1386462
>> >> > Modified:
>> >> >     incubator/public/trunk/content/history/current.txt
>> >>
>> >> Thanks for keeping that file up to date! The graph at
>> >> http://incubator.apache.org/history/ is quite useful.
>> >>
>> >> However, I'm wondering if there really is need to manually maintain a
>> >> separate text file when all the required information should already be
>> >> present in podlings.xml. Can we generate the current.txt file from
>> >> podlings.xml, or (even better) have the client create the history
>> >> graph directly from podlings.xml?
>> >
>> > It would be fantastic if someone could automate that.
>> > At the moment, the current.txt (yellow) is maintained manually
>> > and the entry.txt (green) is automatically handled by Clutch.
>> > Ideally both would be automated independently of Clutch.
>>
>> Perhaps add a task to build.xml that generates current.txt and
>> entry.txt from podlings.xml?
>> Could extract the entry.txt code from clutch and extend it to handle current.txt
>>
>> If that sounds reasonable, I'm happy to give that a try.
>
> That seems like a good solution, now that we have the podlings.xml file.

I've created an XSLT script that generates a version of entry.txt.
It currently generates the output in date order rather than reverse
date order as that is easier - one can use position() for the number.
Not sure why the order is currently reversed; the Javascript does not
seem to care about ordering.

Is there a reason for the reverse data order?

I've started looking at current.txt.

>> Maybe at some point clutch itself could be run automatically whenever
>> podlings.xml is updated.
>
> Perhaps just run it twice per day regardless. There are other
> things that it assesses too.

I'd forgotten that.

> Yesterday the podlings.xml was updated a number of times
> in quick succession to catch up with laggard graduates.
> So would not want to run cumbersome Clutch on each update.

Very true, it does take quite a while.

> Or perhaps it could be triggered by Ant, say 30 minutes after
> the update, otherwise once per day.

Tricky to get that right.

> Clutch seems reasonably robust now. The main thing that
> used to trip it up was project names with inconsistent
> spellings for different resources. The recent addition of
> the "resourceAliases" attribute in podlings.xml does assist.

Yes, though we would need to ensure that the log file was available
for inspection.

Could perhaps be run by the apsite login on minotaur. This runs the
people and projects builds.

> -David
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org