You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@oozie.apache.org by purna pradeep <pu...@gmail.com> on 2018/05/13 22:47:11 UTC

Workflow S3 listener

Hi,
Hi,

I’m very new to oozie ,actually I would like to run Spark 2.3 jobs on oozie
based on file arrival on aws s3 which is a dependency for the job


I see some examples which uses s3 as input event datasets as below


<datasets>
<dataset name="ix" frequency="${coord:days(1)}"
initial-instance="2015-05-10T15:25Z" timezone="Europe/Zurich">
<uri-template>s3n://mybucket/a/b/${YEAR}/${MONTH}/${DAY}</uri-template>
</dataset>
</datasets>

So my question is does oozie listenes to file arrival on aws s3 to check
for dependency before kicking off spark job ??

Thanks