You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@continuum.apache.org by Wendy Smoak <ws...@gmail.com> on 2008/03/28 16:44:52 UTC
New feature: Scheduled purge of old build results
On Fri, Mar 28, 2008 at 8:31 AM, Emmanuel Venisse
<em...@gmail.com> wrote:
>> URL: http://jira.codehaus.org/browse/CONTINUUM-1696
> We must discuss about it.
Okay. :) Do you have a concern about adding this feature, or ... ?
--
Wendy
Re: New feature: Scheduled purge of old build results
Posted by Emmanuel Venisse <em...@gmail.com>.
On Fri, Mar 28, 2008 at 6:05 PM, Olivier Lamy <ol...@apache.org> wrote:
> 2008/3/28, Emmanuel Venisse <em...@gmail.com>:
> > On Fri, Mar 28, 2008 at 4:44 PM, Wendy Smoak <ws...@gmail.com> wrote:
> >
> > > On Fri, Mar 28, 2008 at 8:31 AM, Emmanuel Venisse
> > > <em...@gmail.com> wrote:
> > >
> >
> > > >> URL: http://jira.codehaus.org/browse/CONTINUUM-1696<
> http://jira.codehaus.org/browse/CONTINUUM-1696>
> >
> > >
> > > > We must discuss about it.
> > >
> > > Okay. :) Do you have a concern about adding this feature, or ... ?
> > >
> >
> >
> > :)
> >
> > I'd want to know what you mean with a purge.
> > If you just delete old build results in the db, I don't think it will
> be
> > enough.
> > It will be better to keep somewhere (in a file) some datas about
> deleted
> > build results so they will can be included in some historical graphs
> when
> > we'll add them.
> >
> > At the same time, I'd like to move build results out of the db because
> build
> > results use lot of memory with all scm informations. If we want to keep
> some
> > build results informations in the db for some requests, we need only
> some
> > basic datas ( start/end dates, duration, build number, state) and
> > dependencies changes. All others informations are statics and we don't
> need
> > them for requests, only for the build result page. Before to implement
> the
> > purge, I think it would be better to implement this to save memory
>
> totally Agree
>
> >. Then the purge process will be:
> > - remove the build result information file
> > - remove the basic build result record in the db
> > - create a new file for the history with data that was in the record.
> >
> > WDYT?
> >
> >
> > Emmanuel
> >
> Fine (but we will create some files too which need a purge system too ?
> ;-))
Maybe we'll can choose in the purge system to purge history or not
Re: New feature: Scheduled purge of old build results
Posted by Olivier Lamy <ol...@apache.org>.
2008/3/28, Emmanuel Venisse <em...@gmail.com>:
> On Fri, Mar 28, 2008 at 4:44 PM, Wendy Smoak <ws...@gmail.com> wrote:
>
> > On Fri, Mar 28, 2008 at 8:31 AM, Emmanuel Venisse
> > <em...@gmail.com> wrote:
> >
>
> > >> URL: http://jira.codehaus.org/browse/CONTINUUM-1696<http://jira.codehaus.org/browse/CONTINUUM-1696>
>
> >
> > > We must discuss about it.
> >
> > Okay. :) Do you have a concern about adding this feature, or ... ?
> >
>
>
> :)
>
> I'd want to know what you mean with a purge.
> If you just delete old build results in the db, I don't think it will be
> enough.
> It will be better to keep somewhere (in a file) some datas about deleted
> build results so they will can be included in some historical graphs when
> we'll add them.
>
> At the same time, I'd like to move build results out of the db because build
> results use lot of memory with all scm informations. If we want to keep some
> build results informations in the db for some requests, we need only some
> basic datas ( start/end dates, duration, build number, state) and
> dependencies changes. All others informations are statics and we don't need
> them for requests, only for the build result page. Before to implement the
> purge, I think it would be better to implement this to save memory
totally Agree
>. Then the purge process will be:
> - remove the build result information file
> - remove the basic build result record in the db
> - create a new file for the history with data that was in the record.
>
> WDYT?
>
>
> Emmanuel
>
Fine (but we will create some files too which need a purge system too ? ;-))
Re: New feature: Scheduled purge of old build results
Posted by Rahul Thakur <ra...@gmail.com>.
> I'd want to know what you mean with a purge.
> If you just delete old build results in the db, I don't think it will be
> enough.
> It will be better to keep somewhere (in a file) some datas about deleted
> build results so they will can be included in some historical graphs when
> we'll add them.
>
> At the same time, I'd like to move build results out of the db because build
> results use lot of memory with all scm informations. If we want to keep some
>
I am not really inclined to move the build results out of DB. May be
lazy-load the build results when needed. I like the ability to keep all
data/info for a given Continuum instance consolidated in one place.
Rahul
Re: New feature: Scheduled purge of old build results
Posted by Emmanuel Venisse <em...@gmail.com>.
On Fri, Mar 28, 2008 at 6:13 PM, Wendy Smoak <ws...@gmail.com> wrote:
> On Fri, Mar 28, 2008 at 9:50 AM, Emmanuel Venisse
> <em...@gmail.com> wrote:
>
> > I'd want to know what you mean with a purge.
> > If you just delete old build results in the db, I don't think it will
> be
> > enough.
> > It will be better to keep somewhere (in a file) some datas about
> deleted
> > build results so they will can be included in some historical graphs
> when
> > we'll add them.
>
> It's less a memory issue for me than a space issue, both on the
> filesystem (all the old build output text files) and in the database,
> where they have to be dumped to XML and re-imported when you upgrade.
> I want the option to completely remove them from the system.
>
> A staged approach and saving historical data as you have described is
> fine with me though, it turns the process into 'archiving' and
> 'purging'. I still want the ability to completely get rid of it,
> preferably with a scheduled task.
>
agreed. A purge will delete current and archived datas.
ok for a scheduled task too.
Re: New feature: Scheduled purge of old build results
Posted by Wendy Smoak <ws...@gmail.com>.
On Fri, Mar 28, 2008 at 9:50 AM, Emmanuel Venisse
<em...@gmail.com> wrote:
> I'd want to know what you mean with a purge.
> If you just delete old build results in the db, I don't think it will be
> enough.
> It will be better to keep somewhere (in a file) some datas about deleted
> build results so they will can be included in some historical graphs when
> we'll add them.
It's less a memory issue for me than a space issue, both on the
filesystem (all the old build output text files) and in the database,
where they have to be dumped to XML and re-imported when you upgrade.
I want the option to completely remove them from the system.
A staged approach and saving historical data as you have described is
fine with me though, it turns the process into 'archiving' and
'purging'. I still want the ability to completely get rid of it,
preferably with a scheduled task.
--
Wendy
Re: New feature: Scheduled purge of old build results
Posted by Emmanuel Venisse <em...@gmail.com>.
On Fri, Mar 28, 2008 at 4:44 PM, Wendy Smoak <ws...@gmail.com> wrote:
> On Fri, Mar 28, 2008 at 8:31 AM, Emmanuel Venisse
> <em...@gmail.com> wrote:
>
> >> URL: http://jira.codehaus.org/browse/CONTINUUM-1696<http://jira.codehaus.org/browse/CONTINUUM-1696>
>
> > We must discuss about it.
>
> Okay. :) Do you have a concern about adding this feature, or ... ?
>
:)
I'd want to know what you mean with a purge.
If you just delete old build results in the db, I don't think it will be
enough.
It will be better to keep somewhere (in a file) some datas about deleted
build results so they will can be included in some historical graphs when
we'll add them.
At the same time, I'd like to move build results out of the db because build
results use lot of memory with all scm informations. If we want to keep some
build results informations in the db for some requests, we need only some
basic datas ( start/end dates, duration, build number, state) and
dependencies changes. All others informations are statics and we don't need
them for requests, only for the build result page. Before to implement the
purge, I think it would be better to implement this to save memory. Then the
purge process will be:
- remove the build result information file
- remove the basic build result record in the db
- create a new file for the history with data that was in the record.
WDYT?
Emmanuel