[GitHub] [shardingsphere-elasticjob] Lucas-307 opened a new issue #1856: ElasticJob add static sharding strategy, add completed flag and split job create trigger API plan

[GitBox <gi...@apache.org>]

Lucas-307 opened a new issue #1856:
URL: https://github.com/apache/shardingsphere-elasticjob/issues/1856


   
   ## Reason
   
   Scaling module dependent on ElasticJob module to sharding and execute scaling job, there are some issues like:
   
   1. OneOff job can't do average sharding even if one-off job start in a short interval.
   2. Sharding item sometimes exexute in different server, which may cause some problems.
   3. OneOff job don't have a complete flag, which may cause job execute more than once.
   
   
   
   ## Plan
   
   So, we decide to refactor it with this plan:
   
   1. add static sharding strategy
   2. add completed flag
   3. split job create trigger API
   
   
   
   ## Steps
   
   1. Before refactor
   
      - Optimize JobInstance 
        - add server field(value is current server ip), and replace it with split by jobInstanceId.
        - add labels field which match JobConfiguration's label field(one to more).
      - Optimize JobConfiguration
        - add label field which match JobInstance's labels field.
   
   2. Add static sharding
   
      - Optimize JobConfiguration
        - add boolean staticSharding field to mark current JobConfiguration is use static sharding strategy or not.
   
      - ShardingListenerManager and ReconcileService
        - if staticSharding is true and have been done sharding before, do not create necessary flag again.
   
   3. Add completed flag
   
      - for one-off job, if any item completed, create a completed flag in `sharding/item/completed`
   
      - failover or reboot JobInstance do not do it again.
   
   4. Register job instance
   
      Add `JobInstanceRegistry.register(jobInstanceId, labels)` API to start a JobInstance and execute:
   
      - listen `/jobs/*/config` and start all uncompleted job
   
        uncompleted job check by
   
        - `config.enable` = true
        - `config.label` match JobInstance's labels
        - all schedule job
        - one-off job which `sharding/item/completed` < `config.shardingTotalCount` 
   
      - start job by old API
        - new ScheduleJobBootstrap().schedule()
        - new OneOffJobBootstrap().execute()
   
   5. Add trigger API
   
      add `JobOperateAPI.trigger(jobName, class, config)` API to create a job
   
      - create `/jobs/jobName` node to persist job class.
   
      - create `/jobs/jobName/config` to persist job config.
   
   
   
   ## API changes
   
   1. new API: split register and trigger 
   
      - register job instance
   
      ```
      JobInstanceRegistry.register(jobInstanceId, labels)
      ```
   
      - trigger job
   
      ```
      JobOperateAPI.trigger(jobName, class, jobConfig)
      ```
   
   2. old API: create and execute
   
      - one-off job
   
      ```
      new OneOffJobBootstrap(regCenter, elasticJob, jobConfig).execute()
      ```
   
      - schedule job
   
      ```
      new ScheduleJobBootstrap(regCenter, elasticJob, jobConfig).schedule()
      ```
   
      


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org