[jira] [Comment Edited] (SLING-3028) Support for progress tracking of jobs and keeping jobs

["Carsten Ziegeler (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SLING-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13793608#comment-13793608 ] 

Carsten Ziegeler edited comment on SLING-3028 at 10/14/13 4:43 PM:
-------------------------------------------------------------------

Many thanks for your comments [~sseifert@pro-vision.de] - it's more than appreciated. For the most issues, I've created a separate one for further discussion, except those which I'm atm unsure if we should address them now. Feel free to open additional issues :)
1. Scheduling options : I've created SLING-3166 to further discuss this
2. DAYLY vs DAILY: Yes, I noticed this as well and corrected it some days ago
3. Logging: I've created SLING-3167 for this
4. JobUtil class is deprecated: I've created SLING-3168 for this
5. and 6. Names of enumeration: SLING-3169
7a "how type/state is set if the job sets a state/result FAILED? SUCCEEDED or CANCELLED? both is not correct". - FAILED means it's retried again so it is either QUEUED or ACTIVE - unless the max retries is reached, then it goes into CANCELLED.
7b. "btw. CANCELLED is british englisch, CANCELED is american english" - thanks, I'll leave it like this as we already have released api using this variant.
7c. "in our own job implementation with have states for "transitions" as well like "CANCELING". this is useful because sometimes a job that is canceled cannot cancel at once, but takes some seconds or minutes to wait for the best time when the job can be canceled. if this job is displayed in a management GUI it would be helpful to show this transition state as well. to make it complete a transition state "STARTING" should be added as well. more enhance features like PAUSING/PAUSED/RESTARTING could be added in the feature - too much for now"
The current job model does not have the support for such transition states as the states are regarded from the engine pov: this simply starts the job processing (calling the consumer) and gets a result. I assume that the transition states are handled/set by the consumer? If you think we should support this directly via the API please make an additional issue.
8. Job Name: Yes, this is an old relict we probably should rather drop it for the new API - the job name is a user given unique id - while the job id is the internal unique job id. The job name is only relevant if there is the chance that addJob is called more than once for creating the same job. For example in a clustered installation if you have an observation listener creating jobs  based on repository changes, you need to make sure that this is only done once cluster wide. Either you code this into your observation listeners or you let everyone use the same job name when creating the job (well the same name for the same job) - in this case the job eventing takes care of it and such a job is created only once. I'll think about it once more and probably drop it (set it to deprecated)
9. Schedule Name: each schedule gets a unique name to identify it and have a way to update the schedule or unschedule it. The other option I see would be to create a unique scheduling id when a new schedule is created and then use this to reference it. But in that case the client needs to store this id somewhere. If you think that this needs further enhancements or changes, please open a separate ticket.
10. Own queue implementations: these are not possible at all. For your use case of a time window queue, you maybe could use the scheduled jobs? If you think it's worth exploring this, please create a separate issue.
11. Complex result structures: SLING-3170



was (Author: cziegeler):
Many thanks for your comments [~sseifert@pro-vision.de] - it's more than appreciated. For the most issues, I've created a separate one for further discussion, except those which I'm atm unsure if we should address them now. Feel free to open additional issues :)
1. Scheduling options : I've created SLING-3166 to further discuss this
2. DAYLY vs DAILY: Yes, I noticed this as well and corrected it some days ago
3. Logging: I've created SLING-3167 for this
4. JobUtil class is deprecated: I've created SLING-3168 for this
5. and 6. Names of enumeration: SLING-3169
7a "how type/state is set if the job sets a state/result FAILED? SUCCEEDED or CANCELLED? both is not correct". - FAILED means it's retried again so it is either QUEUED or ACTIVE - unless the max retries is reached, then it goes into CANCELLED.
7b. "btw. CANCELLED is british englisch, CANCELED is american english" - thanks, I'll correct this.
7c. "in our own job implementation with have states for "transitions" as well like "CANCELING". this is useful because sometimes a job that is canceled cannot cancel at once, but takes some seconds or minutes to wait for the best time when the job can be canceled. if this job is displayed in a management GUI it would be helpful to show this transition state as well. to make it complete a transition state "STARTING" should be added as well. more enhance features like PAUSING/PAUSED/RESTARTING could be added in the feature - too much for now"
The current job model does not have the support for such transition states as the states are regarded from the engine pov: this simply starts the job processing (calling the consumer) and gets a result. I assume that the transition states are handled/set by the consumer? If you think we should support this directly via the API please make an additional issue.
8. Job Name: Yes, this is an old relict we probably should rather drop it for the new API - the job name is a user given unique id - while the job id is the internal unique job id. The job name is only relevant if there is the chance that addJob is called more than once for creating the same job. For example in a clustered installation if you have an observation listener creating jobs  based on repository changes, you need to make sure that this is only done once cluster wide. Either you code this into your observation listeners or you let everyone use the same job name when creating the job (well the same name for the same job) - in this case the job eventing takes care of it and such a job is created only once. I'll think about it once more and probably drop it (set it to deprecated)
9. Schedule Name: each schedule gets a unique name to identify it and have a way to update the schedule or unschedule it. The other option I see would be to create a unique scheduling id when a new schedule is created and then use this to reference it. But in that case the client needs to store this id somewhere. If you think that this needs further enhancements or changes, please open a separate ticket.
10. Own queue implementations: these are not possible at all. For your use case of a time window queue, you maybe could use the scheduled jobs? If you think it's worth exploring this, please create a separate issue.
11. Complex result structures: SLING-3170


> Support for progress tracking of jobs and keeping jobs
> ------------------------------------------------------
>
>                 Key: SLING-3028
>                 URL: https://issues.apache.org/jira/browse/SLING-3028
>             Project: Sling
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Victor Saar
>            Assignee: Carsten Ziegeler
>              Labels: jobs
>             Fix For: Extensions Event 3.3.0
>
>         Attachments: SLING-3028.patch
>
>
> For long-running jobs, it would be useful to have some means to track progress, which can be shown in a console for the user. This should include the following:
> * ETA
> * Completeness value computed from (optional, defaults to 1.0) max and current value (e.g. 42% or 23/100)
> * Log output stream for detailed progress information
> * Failure reason in case job failed
> AFAICS this requires a few changes to the existing implementation:
> * Jobs need additional support for setting properties, e.g. max and current progress value
> * Jobs need to be kept at least for a while after they completed/failed to give access to failure information/log stream



--
This message was sent by Atlassian JIRA
(v6.1#6144)