You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jetspeed-user@portals.apache.org by Jeff Pierce <ha...@gmail.com> on 2011/03/08 16:00:08 UTC
Resetting portal statistics
How do you reset the portal statistics that are being collected?
I checked on a couple of pages but didn't see it.
Also, are the flush parms the "frequency" of data collected? By that I mean
if I change these values will it reduce the size of the database file?
Is there a way to set the "limit" on size? Ours has grown to be quite
large.
How do you disable stats completely from being collected?
http://portals.apache.org/jetspeed-2/devguide/dev-statistics.html
Thanks so much.
Re: Resetting portal statistics
Posted by Jeff Pierce <ha...@gmail.com>.
I agree it would be nice if there was a purge method on the stats service.
It would also be nice if some additional configuration options were
available that applied to retention of data (all, 1 week, 1 day, 1 hour -
those kind of options).
We did some testing with a simple service. We injected the JetspeedDS data
source into our service and configured the Spring init-method. In the init
method we deleted all the rows from the PAGE_STATISTICS, PORTLET_STATISTICS,
and USER_STATISTICS tables using SQL. Doing the deletes didn't result in
the physical file size shrinking since Derby doesn't reclaim used space. We
added the necessary SP Derby calls to SYSCS_UTIL.SYSCS_COMPRESS_TABLE for
each table. This is a Derby-only solution but that's OK with us for now.
This worked fine with smaller stats tables. However, the larger the tables
the less reliable the init-method became. As the table grew we'd get
different exceptions doing the delete in the init-method, probably due to
different timing issues and colliding with things Jetspeed was trying to do
as well. We saw concurrent update exceptions and Container was opened in
read-only mode exceptions.
If someone has a suggestion on how to get our service called in a more
reliable, predictable manner after or even before Jetspeed has done its
thing that'd be great. For now we've solved the problem by not calling a
service but shrinking the database using the Derby ij tool from the command
line. I've shared the script below that we are using.
connect 'jdbc:derby:c:/temp/db/productiondb';
delete from APP.PAGE_STATISTICS;
delete from APP.PORTLET_STATISTICS;
delete from APP.USER_STATISTICS;
call SYSCS_UTIL.SYSCS_COMPRESS_TABLE('APP', 'PAGE_STATISTICS', 0);
call SYSCS_UTIL.SYSCS_COMPRESS_TABLE('APP', 'PORTLET_STATISTICS', 0);
call SYSCS_UTIL.SYSCS_COMPRESS_TABLE('APP', 'USER_STATISTICS', 0);
commit;
exit;
On Tue, Mar 8, 2011 at 2:03 PM, David Taylor <da...@gmail.com>wrote:
> On Tue, Mar 8, 2011 at 8:29 AM, Jeff Pierce <ha...@gmail.com> wrote:
> > Thanks for the quick response everyone.
> >
> > I think we'll look to implement a JS service that cleans up the data on a
> > regular basis. I'm assuming even a user-defined service can get access
> to
> > the JS database or possible the stats service has methods to call that we
> > can utilize.
> >
> Yes you can get access to the data source from a service. Take a look
> at the Statistics impl, since it uses JDBC batch updates, it needs
> access to the JDBC data source. In fact it would be a nice addition to
> the API to have a method to purge the stats db
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jetspeed-user-unsubscribe@portals.apache.org
> For additional commands, e-mail: jetspeed-user-help@portals.apache.org
>
>
Re: Resetting portal statistics
Posted by David Taylor <da...@gmail.com>.
On Tue, Mar 8, 2011 at 8:29 AM, Jeff Pierce <ha...@gmail.com> wrote:
> Thanks for the quick response everyone.
>
> I think we'll look to implement a JS service that cleans up the data on a
> regular basis. I'm assuming even a user-defined service can get access to
> the JS database or possible the stats service has methods to call that we
> can utilize.
>
Yes you can get access to the data source from a service. Take a look
at the Statistics impl, since it uses JDBC batch updates, it needs
access to the JDBC data source. In fact it would be a nice addition to
the API to have a method to purge the stats db
---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-user-unsubscribe@portals.apache.org
For additional commands, e-mail: jetspeed-user-help@portals.apache.org
Re: Resetting portal statistics
Posted by Jeff Pierce <ha...@gmail.com>.
Thanks for the quick response everyone.
I think we'll look to implement a JS service that cleans up the data on a
regular basis. I'm assuming even a user-defined service can get access to
the JS database or possible the stats service has methods to call that we
can utilize.
On Tue, Mar 8, 2011 at 10:17 AM, David Taylor <da...@gmail.com>wrote:
> On Tue, Mar 8, 2011 at 7:00 AM, Jeff Pierce <ha...@gmail.com> wrote:
> > How do you reset the portal statistics that are being collected?
> >
> > I checked on a couple of pages but didn't see it.
> >
> > Also, are the flush parms the "frequency" of data collected? By that I
> mean
> > if I change these values will it reduce the size of the database file?
> >
> Statistics are not written to the database immediately. Instead the
> stats are queued up in memory to be committed in batch to the database
> at the configured interval per in the system properties:
>
> # Flush time in in Millisecond
> portal.statistics.MaxTimePortalToFlush=300000
> portal.statistics.MaxTimeUserToFlush=5000
> portal.statistics.MaxTimePageToFlush=60000
>
>
> > Is there a way to set the "limit" on size? Ours has grown to be quite
> > large.
> >
> No. It just keeps growing. You could issue a SQL statement to delete
> or archive records older than a certain date
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jetspeed-user-unsubscribe@portals.apache.org
> For additional commands, e-mail: jetspeed-user-help@portals.apache.org
>
>
Re: Resetting portal statistics
Posted by David Taylor <da...@gmail.com>.
On Tue, Mar 8, 2011 at 7:00 AM, Jeff Pierce <ha...@gmail.com> wrote:
> How do you reset the portal statistics that are being collected?
>
> I checked on a couple of pages but didn't see it.
>
> Also, are the flush parms the "frequency" of data collected? By that I mean
> if I change these values will it reduce the size of the database file?
>
Statistics are not written to the database immediately. Instead the
stats are queued up in memory to be committed in batch to the database
at the configured interval per in the system properties:
# Flush time in in Millisecond
portal.statistics.MaxTimePortalToFlush=300000
portal.statistics.MaxTimeUserToFlush=5000
portal.statistics.MaxTimePageToFlush=60000
> Is there a way to set the "limit" on size? Ours has grown to be quite
> large.
>
No. It just keeps growing. You could issue a SQL statement to delete
or archive records older than a certain date
---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-user-unsubscribe@portals.apache.org
For additional commands, e-mail: jetspeed-user-help@portals.apache.org
Re: Resetting portal statistics
Posted by Frank Otto <ot...@delta-barth.de>.
hi, you can disable statistics by setting the second argument of
PortalStatisticsImpl constructor to false in statistic.xml
(WEB-INF\assembly) .
f.e.:
<bean id="PortalStatistics"
class="org.apache.jetspeed.statistics.impl.PortalStatisticsImpl"
init-method="springInit"
destroy-method="springDestroy">
<meta key="j2:cat" value="default" />
<!-- logToCLF -->
<constructor-arg index='0' type="boolean">
<value>${portal.statistics.logToLogger}</value>
</constructor-arg>
<!-- logToDatabase -->
<constructor-arg index='1' type="boolean">
*<value>false</value>*
</constructor-arg>
...
kind regards,
Frank
Am 08.03.2011 16:00, schrieb Jeff Pierce:
> How do you reset the portal statistics that are being collected?
>
> I checked on a couple of pages but didn't see it.
>
> Also, are the flush parms the "frequency" of data collected? By that I mean
> if I change these values will it reduce the size of the database file?
>
> Is there a way to set the "limit" on size? Ours has grown to be quite
> large.
>
> How do you disable stats completely from being collected?
>
> http://portals.apache.org/jetspeed-2/devguide/dev-statistics.html
>
> Thanks so much.
>