You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Jabir Ahmed <ja...@gmail.com> on 2012/05/22 21:49:55 UTC
oozie database cleanup+maintenance tools
is there a nice way to cleanup the oozie database instead of writing delete
SQL's ourselves since the database has grown over a period of time , and as
part of regular maintenance we would want to delete old job information
from the tables.
Re: oozie database cleanup+maintenance tools
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Jabir,
if you don't do anything 30 days is the purging time. no need to see
anything in oozie-site.xml
keep in mind that the clean up is 30 days (by default) from the completion
time of the job, if the job is more than 30 days but still running won't be
cleaned up
thx
On Wed, May 23, 2012 at 12:49 AM, Jabir Ahmed <ja...@gmail.com> wrote:
> we already have this in the oozie-site.xml but still i see old entries
> in mysql db
>
> >
> > <property>
> > <name>oozie.service.PurgeService.older.than</name>
> > <value>30</value>
> > <description>
> > Jobs older than this value, in days, will be purged by the
> PurgeService.
> > </description>
> > </property>
> >
> > <property>
> > <name>oozie.service.PurgeService.purge.interval</name>
> > <value>3600</value>
> > <description>
> > Interval at which the purge service will run, in seconds.
> > </description>
> > </property>
> >
>
> mysql> select id,created_time from COORD_JOBS limit 1;
> +--------------------------------------+---------------------+
> | id | created_time |
> +--------------------------------------+---------------------+
> | 0000012-120213195549749-oozie-oozi-C | 2012-02-14 08:55:11 |
> +--------------------------------------+---------------------+
> 1 row in set (0.00 sec)
>
> mysql> select now();
> +---------------------+
> | now() |
> +---------------------+
> | 2012-05-23 07:46:02 |
> +---------------------+
> 1 row in set (0.00 sec)
>
> should we have org.apache.oozie.service.PurgeService also defined
> as below in oozie-site.xml
>
> <property>
> <name>oozie.services</name>
> <value>
>
>
> org.apache.oozie.service.PurgeService,
>
> </value>
> </property>
>
>
>
>
>
>
> On Wed, May 23, 2012 at 11:12 AM, Jabir Ahmed <ja...@gmail.com>
> wrote:
> >
> > hi Alejandro
> >
> > we are using 3.1.x
> >
> > <property>
> > <name>oozie.service.PurgeService.older.than</name>
> > <value>30</value>
> > <description>
> > Jobs older than this value, in days, will be purged by the
> PurgeService.
> > </description>
> > </property>
> >
> > <property>
> > <name>oozie.service.PurgeService.purge.interval</name>
> > <value>3600</value>
> > <description>
> > Interval at which the purge service will run, in seconds.
> > </description>
> > </property>
> >
> >
> > Do these properties have a default value or the purge service is
> disabled if the properties/values are not defined in oozie-site.xml.
> >
> > Thanks
> >
> > jabir
> >
> >
> >
> > On Wed, May 23, 2012 at 2:32 AM, Alejandro Abdelnur <tu...@cloudera.com>
> wrote:
> >>
> >> Hi Jaber,
> >>
> >> what version of Oozie are you using?
> >>
> >> Oozie has a purge service that cleans up completed jobs older than 60
> days
> >> (default), this purge used to have some issue in earlier versions.
> >>
> >> thx
> >>
> >> On Tue, May 22, 2012 at 1:18 PM, Jabir Ahmed <ja...@gmail.com>
> wrote:
> >>
> >> > is there a nice way to cleanup the oozie database instead of writing
> >> > delete SQL's ourselves since the database has grown over a period of
> time ,
> >> > and as part of regular maintenance we would want to delete old job
> >> > information from the mysql tables.
> >> >
> >>
> >>
> >>
> >> --
> >> Alejandro
> >
> >
> >
> >
> > --
> > "The best way to make your dreams come true is to wake up." - Paul Valery
> >
> > --
> > Phone: +91 99162 93063
> > IM/e-mail: jabirahmed@yahoo.com , jabirahmed@gmail.com
>
>
>
>
> --
> "The best way to make your dreams come true is to wake up." - Paul Valery
>
> --
> Phone: +91 99162 93063
> IM/e-mail: jabirahmed@yahoo.com , jabirahmed@gmail.com
>
--
Alejandro
Re: oozie database cleanup+maintenance tools
Posted by Jabir Ahmed <ja...@gmail.com>.
we already have this in the oozie-site.xml but still i see old entries
in mysql db
>
> <property>
> <name>oozie.service.PurgeService.older.than</name>
> <value>30</value>
> <description>
> Jobs older than this value, in days, will be purged by the PurgeService.
> </description>
> </property>
>
> <property>
> <name>oozie.service.PurgeService.purge.interval</name>
> <value>3600</value>
> <description>
> Interval at which the purge service will run, in seconds.
> </description>
> </property>
>
mysql> select id,created_time from COORD_JOBS limit 1;
+--------------------------------------+---------------------+
| id | created_time |
+--------------------------------------+---------------------+
| 0000012-120213195549749-oozie-oozi-C | 2012-02-14 08:55:11 |
+--------------------------------------+---------------------+
1 row in set (0.00 sec)
mysql> select now();
+---------------------+
| now() |
+---------------------+
| 2012-05-23 07:46:02 |
+---------------------+
1 row in set (0.00 sec)
should we have org.apache.oozie.service.PurgeService also defined
as below in oozie-site.xml
<property>
<name>oozie.services</name>
<value>
org.apache.oozie.service.PurgeService,
</value>
</property>
On Wed, May 23, 2012 at 11:12 AM, Jabir Ahmed <ja...@gmail.com> wrote:
>
> hi Alejandro
>
> we are using 3.1.x
>
> <property>
> <name>oozie.service.PurgeService.older.than</name>
> <value>30</value>
> <description>
> Jobs older than this value, in days, will be purged by the PurgeService.
> </description>
> </property>
>
> <property>
> <name>oozie.service.PurgeService.purge.interval</name>
> <value>3600</value>
> <description>
> Interval at which the purge service will run, in seconds.
> </description>
> </property>
>
>
> Do these properties have a default value or the purge service is disabled if the properties/values are not defined in oozie-site.xml.
>
> Thanks
>
> jabir
>
>
>
> On Wed, May 23, 2012 at 2:32 AM, Alejandro Abdelnur <tu...@cloudera.com> wrote:
>>
>> Hi Jaber,
>>
>> what version of Oozie are you using?
>>
>> Oozie has a purge service that cleans up completed jobs older than 60 days
>> (default), this purge used to have some issue in earlier versions.
>>
>> thx
>>
>> On Tue, May 22, 2012 at 1:18 PM, Jabir Ahmed <ja...@gmail.com> wrote:
>>
>> > is there a nice way to cleanup the oozie database instead of writing
>> > delete SQL's ourselves since the database has grown over a period of time ,
>> > and as part of regular maintenance we would want to delete old job
>> > information from the mysql tables.
>> >
>>
>>
>>
>> --
>> Alejandro
>
>
>
>
> --
> "The best way to make your dreams come true is to wake up." - Paul Valery
>
> --
> Phone: +91 99162 93063
> IM/e-mail: jabirahmed@yahoo.com , jabirahmed@gmail.com
--
"The best way to make your dreams come true is to wake up." - Paul Valery
--
Phone: +91 99162 93063
IM/e-mail: jabirahmed@yahoo.com , jabirahmed@gmail.com
Re: oozie database cleanup+maintenance tools
Posted by Jabir Ahmed <ja...@gmail.com>.
hi Alejandro
we are using 3.1.x
<property>
<name>oozie.service.PurgeService.older.than</name>
<value>30</value>
<description>
Jobs older than this value, in days, will be purged by the
PurgeService.
</description>
</property>
<property>
<name>oozie.service.PurgeService.purge.interval</name>
<value>3600</value>
<description>
Interval at which the purge service will run, in seconds.
</description>
</property>
Do these properties have a default value or the purge service is
disabled if the properties/values are not defined in oozie-site.xml.
Thanks
jabir
On Wed, May 23, 2012 at 2:32 AM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
> Hi Jaber,
>
> what version of Oozie are you using?
>
> Oozie has a purge service that cleans up completed jobs older than 60 days
> (default), this purge used to have some issue in earlier versions.
>
> thx
>
> On Tue, May 22, 2012 at 1:18 PM, Jabir Ahmed <ja...@gmail.com> wrote:
>
> > is there a nice way to cleanup the oozie database instead of writing
> > delete SQL's ourselves since the database has grown over a period of
> time ,
> > and as part of regular maintenance we would want to delete old job
> > information from the mysql tables.
> >
>
>
>
> --
> Alejandro
>
--
"The best way to make your dreams come true is to wake up." - Paul Valery
--
Phone: +91 99162 93063
IM/e-mail: jabirahmed@yahoo.com , jabirahmed@gmail.com
Re: oozie database cleanup+maintenance tools
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Hi Jaber,
what version of Oozie are you using?
Oozie has a purge service that cleans up completed jobs older than 60 days
(default), this purge used to have some issue in earlier versions.
thx
On Tue, May 22, 2012 at 1:18 PM, Jabir Ahmed <ja...@gmail.com> wrote:
> is there a nice way to cleanup the oozie database instead of writing
> delete SQL's ourselves since the database has grown over a period of time ,
> and as part of regular maintenance we would want to delete old job
> information from the mysql tables.
>
--
Alejandro
oozie database cleanup+maintenance tools
Posted by Jabir Ahmed <ja...@gmail.com>.
is there a nice way to cleanup the oozie database instead of writing
delete SQL's ourselves since the database has grown over a period of time ,
and as part of regular maintenance we would want to delete old job
information from the mysql tables.