[jira] [Created] (GRIFFIN-232) Add support for running batch jobs when an external event occurs

["Aleksandr Borgatin (JIRA)" <ji...@apache.org>]

Aleksandr Borgatin created GRIFFIN-232:
------------------------------------------

             Summary: Add support for running batch jobs when an external event occurs
                 Key: GRIFFIN-232
                 URL: https://issues.apache.org/jira/browse/GRIFFIN-232
             Project: Griffin (Incubating)
          Issue Type: Improvement
            Reporter: Aleksandr Borgatin


Presently we can only add batch jobs that run on a cron expression.

Proposal is to add support starting batch jobs by external events, for example create/update hive table, create/update file in HDFS.

Moreover, so that everyone can create their own events without changing the code of the Griffin Service, we can make this feature pluggable.

In connection with, I propose to add new object of model - ListenerJob. The example of request for adding ListenerJob:

 
{code:java}
{
  "job.name": "listener_job",
  "cron.expression": "0 * * * * ?",
  "cron.time.zone": "GMT+8:00",
  "job.class": "org.apache.griffin.listener.FileScanJob",
  "job.config": {
    "FILE_NAME": "/Users/test/file.txt”
  }
}
{code}
As a result, this object will be saved in the database and a listener in quartz is scheduled with the following parameters from the query: the type from the “job.class”, the schedule from the “cron.expression”, and the JobDataMap from the “job.config”.

After that we can add a batch job and instead of a cron expression write in the request the listener identifier to subscribe to the previously created listener.

When the desired event occurs, quartz job will find all the batch jobs that are subscribed to this listener, and call them to run once.

The class from the “job.class” must be child of AbstractListenerJob, which provides service for running jobs by listener identifier and has one abstract method:
{code:java}
@DisallowConcurrentExecution
@PersistJobDataAfterExecution
public abstract class AbstractListenerJob implements Job {

    public static final String JOB_ID_CONFIG_MAP_PROPERTY = "JOB_ID_CONFIG_MAP_PROPERTY";

    @Autowired
    private ListenerJobService listenerJobService;

    @Override
    public void execute(JobExecutionContext context) throws JobExecutionException {
        if (needNotify(context)) {
            long jobId = context.getMergedJobDataMap().getLong(JOB_ID_CONFIG_MAP_PROPERTY);
            listenerJobService.triggerJobsByListenerJobId(jobId);
        }

    }

    protected abstract boolean needNotify(JobExecutionContext context) throws JobExecutionException;
}
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)