Re: Review Request 19855: OOZIE-1761 Improve sharelib purging logic

[Rohini Palaniswamy <ro...@gmail.com>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19855/#review39327
-----------------------------------------------------------



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/19855/#comment71671>

    This was not part of official Oozie release. Only need it in our internal branch.



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/19855/#comment71673>

    Where is current defined?



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/19855/#comment71676>

    Can you remove all these variables, simplify and do it directly in expireTs



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/19855/#comment71678>

    long expiryTime = TimeUnit.MILLIS.convert(services.getConf().getInt(ShareLibService.LAUNCHERJAR_LIB_RETENTION, 7)), TimeUnit.DAYS)
    
    String expireTs = new SimpleDateFormat("yyyyMMddHHmmss").format(new Date(expiryTime + TimeUnit.MILLIS.convert(1, TimeUnit.DAYS));
    
    String expireTs2 = new SimpleDateFormat("yyyyMMddHHmmss").format(new Date(expiryTime + TimeUnit.MILLIS.convert(2, TimeUnit.DAYS));
    
    



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/19855/#comment71681>

    private void createDirs(Filesystem fs, Path... paths) {
      for (Path path : paths) {
         fs.mkdirs(path); 
      }
    }


- Rohini Palaniswamy


On March 31, 2014, 7:03 p.m., Purshotam Shah wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19855/
> -----------------------------------------------------------
> 
> (Updated March 31, 2014, 7:03 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1761
>     https://issues.apache.org/jira/browse/OOZIE-1761
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Currently oozie server purges anything older then 7 (oozie.ShareLibService.temp.sharelib.retention.days) days, keeping 2 latest dir.
> Which might be faulty.
> Take an example.
> Currently used sharelib is sharelib-1, which is created a month before.
> If I restart Oozie server 2 times, it will create directory sharelib-2 and sharelib-3 and delete anything older than 7 days( keeping two latest).
> In this case oozie server will delete sharelib-1, which is wrong and all running jobs will start failing.
> Better logic is to keep all share-lib between current timestamp and 7days old + 1 latest sharelib older than 7 days.
> 
> 
> Diffs
> -----
> 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java 89939f1 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java e7d7a26 
> 
> Diff: https://reviews.apache.org/r/19855/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>