You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@forrest.apache.org by Vadim Gritsenko <va...@verizon.net> on 2002/11/09 01:59:29 UTC
[ANNOUNCE] New site stats
Hi Forresters,
We discussed a bit (with Steven) about building apache
projects/hosts/etc stats, which is one of the tasks for the Forrest, so
I guess this announcement is not completely offtopic on this list.
I'd like to announce a page I put together which to show off some of the
stats which I was able to extract from the log files (with minimum
memory consumption). These stats in no sence are complete or correct,
but an attemp to build complete and correct stats.
URL for the page is:
http://www.apache.org/~vgritsenko/stats/index.html
Images are updated daily (and some of them - weekly), at night. Site is
statically generated using Forrest.
Please take a look and send your suggestions, opinions, flames.
Regards,
Vadim
Steven Noels wrote:
> Vadim Gritsenko wrote:
>
> > This also can be done. My question is is this interesting to forrest
> > community or somebody else (you said , Sam?)...... or it's better for
> > me to drop all this nonsense and go and work on Cocoon bugs? :)
>
> It was one of the original plans for Forrest, to put up some stats up
> the different projects (downloads, mails sent on the lists, etc etc)
>
> What would be good is to have a low impact way of extracting whatever
> kind of data we can make sense of from the server logs (like
> user agents, top downloads, busiest website, etc etc).
>
> Now that we start playing with the idea of going independent with
> cocoon.apache.org, that should take into account aggregating several
> logs, too... Hmmmpff.
>
> I don't know ATM, maybe we should just take your data as a test stream
> and see how we can work with it in Forrest (for the xml.apache.org
> front site). And if people like it, they will add more streams and
> we'll finally have that 'Apache project dashboard' displaying all
> kinds of interesting (and other) data.
>
> My point was that finite steps are difficult to aggregate starting
> from XML, so maybe cumulative would be better. This all depends on the
> frequency of sampling of course - for weekly stats, you still can show
> them in a trend chart and make sense of it.
>
> But we'd better move this to forrest-dev so that others can tell their
> opinion too.
>
> </Steven>
Re: [ANNOUNCE] New site stats
Posted by Vadim Gritsenko <va...@verizon.net>.
Jeff Turner wrote:
>On Fri, Nov 08, 2002 at 07:59:29PM -0500, Vadim Gritsenko wrote:
>
>
...
>>URL for the page is:
>> http://www.apache.org/~vgritsenko/stats/index.html
>>
>>
>
>Very nice :)
>
>
>
>>Images are updated daily (and some of them - weekly), at night. Site is
>>statically generated using Forrest.
>>
>>
>
>Are the charts generated by Cocoon too?
>
Charts are generated from the SVG using Batik and shell script. I was
not sure that infrastructure@ won't jump on me if i to regenerate the
whole site.
OTOH, regeneration of the whole site allows to brings much more
potential: add tables with actual figures, etc.
Vadim
>--Jeff
>
>
>
>>Please take a look and send your suggestions, opinions, flames.
>>
>>
>>Regards,
>>Vadim
>>
>>
Re: [ANNOUNCE] New site stats
Posted by Jeff Turner <je...@apache.org>.
On Fri, Nov 08, 2002 at 07:59:29PM -0500, Vadim Gritsenko wrote:
> Hi Forresters,
>
> We discussed a bit (with Steven) about building apache
> projects/hosts/etc stats, which is one of the tasks for the Forrest, so
> I guess this announcement is not completely offtopic on this list.
>
> I'd like to announce a page I put together which to show off some of the
> stats which I was able to extract from the log files (with minimum
> memory consumption). These stats in no sence are complete or correct,
> but an attemp to build complete and correct stats.
>
> URL for the page is:
> http://www.apache.org/~vgritsenko/stats/index.html
Very nice :)
> Images are updated daily (and some of them - weekly), at night. Site is
> statically generated using Forrest.
Are the charts generated by Cocoon too?
--Jeff
> Please take a look and send your suggestions, opinions, flames.
>
>
> Regards,
> Vadim
Re: [ANNOUNCE] New site stats
Posted by Vadim Gritsenko <va...@verizon.net>.
David Crossley wrote:
>Vadim Gritsenko wrote:
>
>
>>Hi Forresters,
>>
>>We discussed a bit (with Steven) about building apache
>>projects/hosts/etc stats, which is one of the tasks for the Forrest, so
>>I guess this announcement is not completely offtopic on this list.
>>
>>I'd like to announce a page I put together which to show off some of the
>>stats which I was able to extract from the log files (with minimum
>>memory consumption). These stats in no sence are complete or correct,
>>but an attemp to build complete and correct stats.
>>
>>
>
>Brilliant stuff Vadim.
>
>
>
>>URL for the page is:
>> http://www.apache.org/~vgritsenko/stats/index.html
>>
>>Images are updated daily (and some of them - weekly), at night. Site is
>>statically generated using Forrest.
>>
>>Please take a look and send your suggestions, opinions, flames.
>>
>>
>
>I would like the Warning to be extended to say something like:
>"As with any statistics, you must take care with interpretation."
>
:)
>Is it possible while the stats are being generated, that
>some other information can be accumulated and stored as daily
>summaries? e.g. project downloads.
>
Yes.
>There was some discussion on this in Forrest's early days
>and a DTD was partially developed.
>
I see the main problem now is to cleanup/check quality of the numbers
generated, and to discuss what kind of data to be extracted in addition
to whatever we have now, and to decide what other grpahs to generate.
Vadim
> Here is one pointer
> Graph data
> http://marc.theaimsgroup.com/?l=forrest-dev&m=101432422003463
>
>--David
>
>
>
Re: [ANNOUNCE] New site stats
Posted by David Crossley <cr...@indexgeo.com.au>.
Vadim Gritsenko wrote:
> Hi Forresters,
>
> We discussed a bit (with Steven) about building apache
> projects/hosts/etc stats, which is one of the tasks for the Forrest, so
> I guess this announcement is not completely offtopic on this list.
>
> I'd like to announce a page I put together which to show off some of the
> stats which I was able to extract from the log files (with minimum
> memory consumption). These stats in no sence are complete or correct,
> but an attemp to build complete and correct stats.
Brilliant stuff Vadim.
> URL for the page is:
> http://www.apache.org/~vgritsenko/stats/index.html
>
> Images are updated daily (and some of them - weekly), at night. Site is
> statically generated using Forrest.
>
> Please take a look and send your suggestions, opinions, flames.
I would like the Warning to be extended to say something like:
"As with any statistics, you must take care with interpretation."
Is it possible while the stats are being generated, that
some other information can be accumulated and stored as daily
summaries? e.g. project downloads.
There was some discussion on this in Forrest's early days
and a DTD was partially developed. Here is one pointer
Graph data
http://marc.theaimsgroup.com/?l=forrest-dev&m=101432422003463
--David
Re: [ANNOUNCE] New site stats
Posted by Stefano Mazzocchi <st...@apache.org>.
Vadim Gritsenko wrote:
> Hi Forresters,
>
> We discussed a bit (with Steven) about building apache
> projects/hosts/etc stats, which is one of the tasks for the Forrest, so
> I guess this announcement is not completely offtopic on this list.
>
> I'd like to announce a page I put together which to show off some of the
> stats which I was able to extract from the log files (with minimum
> memory consumption). These stats in no sence are complete or correct,
> but an attemp to build complete and correct stats.
>
> URL for the page is:
> http://www.apache.org/~vgritsenko/stats/index.html
>
> Images are updated daily (and some of them - weekly), at night. Site is
> statically generated using Forrest.
>
> Please take a look and send your suggestions, opinions, flames.
I *LOVE* it!
I *LOVE* it!
I *LOVE* it!
I'm forwarding this to community@ right away.
--
Stefano Mazzocchi <st...@apache.org>
--------------------------------------------------------------------
Re: [ANNOUNCE] New site stats
Posted by Vadim Gritsenko <va...@verizon.net>.
Steven Noels wrote:
> Vadim Gritsenko wrote:
>
>> Hi Forresters,
>>
>> We discussed a bit (with Steven) about building apache
>> projects/hosts/etc stats, which is one of the tasks for the Forrest,
>> so I guess this announcement is not completely offtopic on this list.
>
>
> I don't feel like I should be credited, since you did all the hard work.
>
> You kick *ss, Vadim - thanks for putting this up. Could the sources
> for this application be made part of Forrest, or at least some common
> infrastructure? That way, we can take a look & augment.
The code sucks badly atm, and there are bugs in the script (like project
"icons"), which has to be fixed... Written in Perl (thanks to my
girlfriend for the exaplanations of the perl syntax and data
structures!). The main part is:
http://www.apache.org/~vgritsenko/stats/log-analyzer-daily.pl
Output is:
http://www.apache.org/~vgritsenko/stats/day-2002-11-08.txt
If somebody wants to tweak SVGs:
http://www.apache.org/~vgritsenko/stats/pie-daily-projects-requests.svg
http://www.apache.org/~vgritsenko/stats/trend-daily-projects-requests.svg
> I'm aware of the fact that these scripts need to run on Daedalus, but
> at least three of us have access, too.
>
> Again, many thanks. Can we call you Lord of Users and Stats from now?
>
> A toothbrush medal for private Vadim!
:)
Vadim
> </Steven>
Re: [ANNOUNCE] New site stats
Posted by Steven Noels <st...@outerthought.org>.
Vadim Gritsenko wrote:
> Hi Forresters,
>
> We discussed a bit (with Steven) about building apache
> projects/hosts/etc stats, which is one of the tasks for the Forrest, so
> I guess this announcement is not completely offtopic on this list.
I don't feel like I should be credited, since you did all the hard work.
You kick *ss, Vadim - thanks for putting this up. Could the sources for
this application be made part of Forrest, or at least some common
infrastructure? That way, we can take a look & augment.
I'm aware of the fact that these scripts need to run on Daedalus, but at
least three of us have access, too.
Again, many thanks. Can we call you Lord of Users and Stats from now?
A toothbrush medal for private Vadim!
</Steven>
--
Steven Noels http://outerthought.org/
Outerthought - Open Source, Java & XML Competence Support Center
stevenn@outerthought.org stevenn@apache.org
Re: [ANNOUNCE] New site stats
Posted by Nick Chalko <ni...@chalko.com>.
>
> URL for the page is:
> http://www.apache.org/~vgritsenko/stats/index.html
Nice looking page.
I noticed a Forrest quirk
When I click on the stabe the Vladim's image moves.
It appears that the Breadcrumbs are effecting the placemnet of the
project logo.
R,
Nick
ps
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.2b) Gecko/20021016
Re: [ANNOUNCE] New site stats
Posted by David Crossley <cr...@indexgeo.com.au>.
Ivelin Ivanov wrote:
>
> Good stuff. This should be a publicly available page (off of the main site)
> for anyone to look.
As Vadim said, it has been on the Forrest task list for a long
time, to do this sort of thing and more. This is one of the
first community resources for Forrest to deliver, apart from the
brilliant document generation facilities.
I presume that the empty directory at
http://xml.apache.org/forrest/community/
is where this stuff belongs. So i am adding a basic page there
as we speak.
Thanks Vadim, it is great to see this. Dreams come true.
--David
> I am surprise to find 2 things:
>
> 1) traffic from xml. vs jakarta. is proportionate to the number of hits.
> Something which used to be out of balanse. Is this due to the lightweight
> nature of Forrest?
>
> 2) Cocoon is almost as popular as Struts. Why didn't I see Turbine or
> Velocity. Thought they are much more popular than Cocoon?
>
>
> Ivlein
>
>
> ----- Original Message -----
> From: "Vadim Gritsenko" <va...@verizon.net>
> To: <fo...@xml.apache.org>
> Cc: "Steven Noels" <st...@outerthought.org>
> Sent: Friday, November 08, 2002 6:59 PM
> Subject: [ANNOUNCE] New site stats
>
>
> > Hi Forresters,
> >
> > We discussed a bit (with Steven) about building apache
> > projects/hosts/etc stats, which is one of the tasks for the Forrest, so
> > I guess this announcement is not completely offtopic on this list.
> >
> > I'd like to announce a page I put together which to show off some of the
> > stats which I was able to extract from the log files (with minimum
> > memory consumption). These stats in no sence are complete or correct,
> > but an attemp to build complete and correct stats.
> >
> > URL for the page is:
> > http://www.apache.org/~vgritsenko/stats/index.html
> >
> > Images are updated daily (and some of them - weekly), at night. Site is
> > statically generated using Forrest.
> >
> > Please take a look and send your suggestions, opinions, flames.
> >
> >
> > Regards,
> > Vadim
> >
> >
> >
> > Steven Noels wrote:
> >
> > > Vadim Gritsenko wrote:
> > >
> > > > This also can be done. My question is is this interesting to forrest
> > > > community or somebody else (you said , Sam?)...... or it's better for
> > > > me to drop all this nonsense and go and work on Cocoon bugs? :)
> > >
> > > It was one of the original plans for Forrest, to put up some stats up
> > > the different projects (downloads, mails sent on the lists, etc etc)
> > >
> > > What would be good is to have a low impact way of extracting whatever
> > > kind of data we can make sense of from the server logs (like
> > > user agents, top downloads, busiest website, etc etc).
> > >
> > > Now that we start playing with the idea of going independent with
> > > cocoon.apache.org, that should take into account aggregating several
> > > logs, too... Hmmmpff.
> > >
> > > I don't know ATM, maybe we should just take your data as a test stream
> > > and see how we can work with it in Forrest (for the xml.apache.org
> > > front site). And if people like it, they will add more streams and
> > > we'll finally have that 'Apache project dashboard' displaying all
> > > kinds of interesting (and other) data.
> > >
> > > My point was that finite steps are difficult to aggregate starting
> > > from XML, so maybe cumulative would be better. This all depends on the
> > > frequency of sampling of course - for weekly stats, you still can show
> > > them in a trend chart and make sense of it.
> > >
> > > But we'd better move this to forrest-dev so that others can tell their
> > > opinion too.
> > >
> > > </Steven>
> >
> >
>
Re: [ANNOUNCE] New site stats
Posted by Vadim Gritsenko <va...@verizon.net>.
Ivelin Ivanov wrote:
>Good stuff. This should be a publicly available page (off of the main site)
>for anyone to look.
>
>I am surprise to find 2 things:
>
>1) traffic from xml. vs jakarta. is proportionate to the number of hits.
>Something which used to be out of balanse. Is this due to the lightweight
>nature of Forrest?
>
>2) Cocoon is almost as popular as Struts. Why didn't I see Turbine or
>Velocity. Thought they are much more popular than Cocoon?
>
They did not fit into the graphs. I decided to show only top performers
- to have more readable graphs.
Whole data can be shown in html tables.
Vadim
>Ivlein
>
>
>----- Original Message -----
>From: "Vadim Gritsenko" <va...@verizon.net>
>To: <fo...@xml.apache.org>
>Cc: "Steven Noels" <st...@outerthought.org>
>Sent: Friday, November 08, 2002 6:59 PM
>Subject: [ANNOUNCE] New site stats
>
>
>
>
>>Hi Forresters,
>>
>>We discussed a bit (with Steven) about building apache
>>projects/hosts/etc stats, which is one of the tasks for the Forrest, so
>>I guess this announcement is not completely offtopic on this list.
>>
>>I'd like to announce a page I put together which to show off some of the
>>stats which I was able to extract from the log files (with minimum
>>memory consumption). These stats in no sence are complete or correct,
>>but an attemp to build complete and correct stats.
>>
>>URL for the page is:
>> http://www.apache.org/~vgritsenko/stats/index.html
>>
>>Images are updated daily (and some of them - weekly), at night. Site is
>>statically generated using Forrest.
>>
>>Please take a look and send your suggestions, opinions, flames.
>>
>>
>>Regards,
>>Vadim
>>
>>
>>
>>Steven Noels wrote:
>>
>>
>>
>>>Vadim Gritsenko wrote:
>>>
>>>
>>>
>>>>This also can be done. My question is is this interesting to forrest
>>>>community or somebody else (you said , Sam?)...... or it's better for
>>>>me to drop all this nonsense and go and work on Cocoon bugs? :)
>>>>
>>>>
>>>It was one of the original plans for Forrest, to put up some stats up
>>>the different projects (downloads, mails sent on the lists, etc etc)
>>>
>>>What would be good is to have a low impact way of extracting whatever
>>>kind of data we can make sense of from the server logs (like
>>>user agents, top downloads, busiest website, etc etc).
>>>
>>>Now that we start playing with the idea of going independent with
>>>cocoon.apache.org, that should take into account aggregating several
>>>logs, too... Hmmmpff.
>>>
>>>I don't know ATM, maybe we should just take your data as a test stream
>>>and see how we can work with it in Forrest (for the xml.apache.org
>>>front site). And if people like it, they will add more streams and
>>>we'll finally have that 'Apache project dashboard' displaying all
>>>kinds of interesting (and other) data.
>>>
>>>My point was that finite steps are difficult to aggregate starting
>>>from XML, so maybe cumulative would be better. This all depends on the
>>>frequency of sampling of course - for weekly stats, you still can show
>>>them in a trend chart and make sense of it.
>>>
>>>But we'd better move this to forrest-dev so that others can tell their
>>>opinion too.
>>>
>>></Steven>
>>>
>>>
>>
>>
>
>
>
>
Re: [ANNOUNCE] New site stats
Posted by Ivelin Ivanov <iv...@apache.org>.
Good stuff. This should be a publicly available page (off of the main site)
for anyone to look.
I am surprise to find 2 things:
1) traffic from xml. vs jakarta. is proportionate to the number of hits.
Something which used to be out of balanse. Is this due to the lightweight
nature of Forrest?
2) Cocoon is almost as popular as Struts. Why didn't I see Turbine or
Velocity. Thought they are much more popular than Cocoon?
Ivlein
----- Original Message -----
From: "Vadim Gritsenko" <va...@verizon.net>
To: <fo...@xml.apache.org>
Cc: "Steven Noels" <st...@outerthought.org>
Sent: Friday, November 08, 2002 6:59 PM
Subject: [ANNOUNCE] New site stats
> Hi Forresters,
>
> We discussed a bit (with Steven) about building apache
> projects/hosts/etc stats, which is one of the tasks for the Forrest, so
> I guess this announcement is not completely offtopic on this list.
>
> I'd like to announce a page I put together which to show off some of the
> stats which I was able to extract from the log files (with minimum
> memory consumption). These stats in no sence are complete or correct,
> but an attemp to build complete and correct stats.
>
> URL for the page is:
> http://www.apache.org/~vgritsenko/stats/index.html
>
> Images are updated daily (and some of them - weekly), at night. Site is
> statically generated using Forrest.
>
> Please take a look and send your suggestions, opinions, flames.
>
>
> Regards,
> Vadim
>
>
>
> Steven Noels wrote:
>
> > Vadim Gritsenko wrote:
> >
> > > This also can be done. My question is is this interesting to forrest
> > > community or somebody else (you said , Sam?)...... or it's better for
> > > me to drop all this nonsense and go and work on Cocoon bugs? :)
> >
> > It was one of the original plans for Forrest, to put up some stats up
> > the different projects (downloads, mails sent on the lists, etc etc)
> >
> > What would be good is to have a low impact way of extracting whatever
> > kind of data we can make sense of from the server logs (like
> > user agents, top downloads, busiest website, etc etc).
> >
> > Now that we start playing with the idea of going independent with
> > cocoon.apache.org, that should take into account aggregating several
> > logs, too... Hmmmpff.
> >
> > I don't know ATM, maybe we should just take your data as a test stream
> > and see how we can work with it in Forrest (for the xml.apache.org
> > front site). And if people like it, they will add more streams and
> > we'll finally have that 'Apache project dashboard' displaying all
> > kinds of interesting (and other) data.
> >
> > My point was that finite steps are difficult to aggregate starting
> > from XML, so maybe cumulative would be better. This all depends on the
> > frequency of sampling of course - for weekly stats, you still can show
> > them in a trend chart and make sense of it.
> >
> > But we'd better move this to forrest-dev so that others can tell their
> > opinion too.
> >
> > </Steven>
>
>