[jira] [Commented] (AMBARI-24731) Infra Manager: scheduled cleanup of old jobs

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMBARI-24731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646198#comment-16646198 ] 

ASF GitHub Bot commented on AMBARI-24731:
-----------------------------------------

kasakrisz opened a new pull request #4: AMBARI-24731 - Infra Manager: scheduled cleanup of old jobs
URL: https://github.com/apache/ambari-infra/pull/4
 
 
   ## What changes were proposed in this pull request?
   
   - Create clean up job for deleting expired job data from Infra Manager local database
   - Replace aws sdk to minio because aws sdk can not be modularized and it was not worked with localstack which we use for mock s3 in IT
   - Refactor job configuration: prior the refactor only job groups could be defined from now single jobs too
   - Extract Solr specific functionality from JobScheduler to get a more general scheduler. 
   - Use immutable configuration and parameter objects for jobs
   - Adjust integration test for the separate infra repo
   
   ## How was this patch tested?
   
   1. run Integration test 
   2. manually:
   - Deploy Ambari and a cluster with infra solr, infra manager, logsearch, hdfs
   - enable archive_service_job set the destination to hdfs and run it using infra manager API
   - check hdfs contains files in the archives destination directory. Check the content of the files
   3. manually:
   - start infra manager in a docker environment
   - set cleanup schedule to run the job in every minute and set the ttl for a small duration of time
   - check infra managers sqlite db contains job data and their creation date
    

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Infra Manager: scheduled cleanup of old jobs
> --------------------------------------------
>
>                 Key: AMBARI-24731
>                 URL: https://issues.apache.org/jira/browse/AMBARI-24731
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-infra
>    Affects Versions: 2.8.0
>            Reporter: Krisztian Kasa
>            Assignee: Krisztian Kasa
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 2.8.0
>
>
> Infra manager should have a mechanism, which can be scheduled (like every day once), to check against old job history and get rid of them (if the job already finished about 7 days, it can be deleted -> all references in the db)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)