Adding custom engine in Apache Falcon

[Aniket R More <An...@bitwiseglobal.com>]

Hi ,

Falcon currently supports 4 engines such as oozie,pig,hive,spark. Is it possible to add another engine ? I know i can run my own scripts in oozie. But what I want is to add my own custom engine in apache falcon. Can anyone please guide me on this.

Thanks,

Aniket

**************************************Disclaimer****************************************** This e-mail message and any attachments may contain confidential information and is for the sole use of the intended recipient(s) only. Any views or opinions presented or implied are solely those of the author and do not necessarily represent the views of BitWise. If you are not the intended recipient(s), you are hereby notified that disclosure, printing, copying, forwarding, distribution, or the taking of any action whatsoever in reliance on the contents of this electronic information is strictly prohibited. If you have received this e-mail message in error, please immediately notify the sender and delete the electronic message and any attachments.BitWise does not accept liability for any virus introduced by this e-mail or any attachments. ********************************************************************************************

Re: Adding custom engine in Apache Falcon

[Sandeep Samudrala <sa...@gmail.com>]

Aniket,
Below mentioned jira should give some idea on to the flow and changes.

https://issues.apache.org/jira/browse/FALCON-1763

On Tue, Jun 27, 2017 at 11:11 PM, Seetharam Venkatesh <vs...@gmail.com>
wrote:

> Aniket, look at how existing engines are handled as a template and
> replicate the same. Look at either hive or pig for an example. I suspect
> there's any more documentation around this.
>
> Sent from my iPhone,
> Venkatesh
>
> > On Jun 27, 2017, at 6:30 AM, Aniket R More <
> Aniket.More@bitwiseglobal.com> wrote:
> >
> > Hi ,
> >
> > Falcon currently supports 4 engines such as oozie,pig,hive,spark. Is it
> possible to add another engine ? I know i can run my own scripts in oozie.
> But what I want is to add my own custom engine in apache falcon. Can anyone
> please guide me on this.
> >
> > Thanks,
> >
> > Aniket
> >
> > **************************************Disclaimer******************************************
> This e-mail message and any attachments may contain confidential
> information and is for the sole use of the intended recipient(s) only. Any
> views or opinions presented or implied are solely those of the author and
> do not necessarily represent the views of BitWise. If you are not the
> intended recipient(s), you are hereby notified that disclosure, printing,
> copying, forwarding, distribution, or the taking of any action whatsoever
> in reliance on the contents of this electronic information is strictly
> prohibited. If you have received this e-mail message in error, please
> immediately notify the sender and delete the electronic message and any
> attachments.BitWise does not accept liability for any virus introduced by
> this e-mail or any attachments. ******************************
> **************************************************************
>

Re: Adding custom engine in Apache Falcon

[Seetharam Venkatesh <vs...@gmail.com>]

Aniket, look at how existing engines are handled as a template and replicate the same. Look at either hive or pig for an example. I suspect there's any more documentation around this. 

Sent from my iPhone,
Venkatesh

> On Jun 27, 2017, at 6:30 AM, Aniket R More <An...@bitwiseglobal.com> wrote:
> 
> Hi ,
> 
> Falcon currently supports 4 engines such as oozie,pig,hive,spark. Is it possible to add another engine ? I know i can run my own scripts in oozie. But what I want is to add my own custom engine in apache falcon. Can anyone please guide me on this.
> 
> Thanks,
> 
> Aniket
> 
> **************************************Disclaimer****************************************** This e-mail message and any attachments may contain confidential information and is for the sole use of the intended recipient(s) only. Any views or opinions presented or implied are solely those of the author and do not necessarily represent the views of BitWise. If you are not the intended recipient(s), you are hereby notified that disclosure, printing, copying, forwarding, distribution, or the taking of any action whatsoever in reliance on the contents of this electronic information is strictly prohibited. If you have received this e-mail message in error, please immediately notify the sender and delete the electronic message and any attachments.BitWise does not accept liability for any virus introduced by this e-mail or any attachments. ********************************************************************************************