[jira] [Comment Edited] (FALCON-1109) Rest API support for recipe

["Sowmya Ramesh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FALCON-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368201#comment-14368201 ] 

Sowmya Ramesh edited comment on FALCON-1109 at 3/19/15 12:21 AM:
-----------------------------------------------------------------

Under the hood recipe is just another regular Falcon process. Most of the entity and instance Rest API's provided for falcon process entity can be reused for recipe. We can either introduce corresponding recipe Rest API and map it to falcon process Rest API's or document to use the Falcon process rest API's; I don't have strong opinion here.

Recipe schedule and listing are little different from the falcon process entity operations and existing Falcon process API's cannot be used.

* Recipe prepare and schedule*
{noformat}
POST http://localhost:15000/api/recipe/prepareAndSchedule/:recipe-name <recipePropertyFile>

Client is expected to get the recipePropertyFile for the corresponding recipe from the recipe repository and update the values before scheduling the recipe
{noformat}

*Recipe Listing*
There can be an use case to list only recipe entities or instances. To facilitate that under the hood during recipe processing, Falcon System specific tag [Keyword] can be added to identify the recipes. Later, the tag can be used for filtering to list only recipe entities.


Above API's provided will be generic and work for all the recipes as recipe property file is unique for each recipe. RecipeTool, based on each recipe can do the validation and error handling.

[~sureshms] had some concerns about sending <recipe property> file. He mentioned that instead of sending the recipe property file all the required properties can be passed as arguments/query params.

I am not totally convinced with that approach as if the arguments increase it will be cumbersome for the user to remember each argument. From ease of use point of view I think passing a property file is better as keys are already provided and the user has to just update the values.

Also [~sureshms] didn't want to use generic rest API's. Today recipe can be used for HDFS data replication (HDFS DR). He mentioned that Falcon should provide DR as a service, and user should be abstracted from the recipe concept. If this approach is followed we will have to add new Rest API for each kind of DR or any new recipe functionality we add.

[~sriksun], [~svenkat], [~venkatnrangan], [[~sureshms]: Thoughts?! Let's please use this jira to finalize Rest API functionality. Thanks!


was (Author: sowmyaramesh):
Under the hood recipe is just another regular Falcon process. Most of the entity and instance Rest API's provided for falcon process entity can be reused for recipe. We can either introduce corresponding recipe Rest API and map it to falcon Rest API's or document to use the Falcon process rest API's; I don't have strong opinion here.

Recipe schedule and listing are little different from the falcon process entity operations and existing Falcon process API's cannot be used.

* Recipe prepare and schedule*
{noformat}
POST http://localhost:15000/api/recipe/prepareAndSchedule/:recipe-name <recipePropertyFile>

Client is expected to get the recipePropertyFile for the corresponding recipe from the recipe repository and update the values before scheduling the recipe
{noformat}

*Recipe Listing*
There can be an use case to list only recipe entities or instances. To facilitate that under the hood during recipe processing, Falcon System specific tag [Keyword] can be added to identify the recipes. Later tag can be used for filtering to list only recipe entities.

Above API's provided will be generic and work for all the recipes as recipe property file is unique for each recipe. RecipeTool based on each recipe can do the validation and error handling.

[~sureshms] had some concerns about sending <recipe property> file. He mentioned that instead of sending the recipe property file all the required properties can be passed as arguments/query params.

I am not totally convinced with that approach as if the arguments increase it will be cumbersome for the user to remember each argument. From ease of use point of view I think passing a property file is better as keys are already provided and the user has to just update the values.

Also [~sureshms] didn't want to use generic rest API's. Today recipe can be used to for HDFS data replicaiton (HDFS DR). He mentioned that DR Falcon should provide DR as a service and user should be abstracted from the recipe concept. If this approach is followed we will have to add new Rest API for each kind of DR.

[~sriksun], [~svenkat], [~venkatnrangan], [[~sureshms]: Thoughts?! Let's please use this jira to finalize Rest API functionality. Thanks!

> Rest API support for recipe
> ---------------------------
>
>                 Key: FALCON-1109
>                 URL: https://issues.apache.org/jira/browse/FALCON-1109
>             Project: Falcon
>          Issue Type: Sub-task
>          Components: client
>    Affects Versions: 0.6
>            Reporter: Sowmya Ramesh
>              Labels: Recipe
>             Fix For: 0.7
>
>
> Recipe processing will be done on the server side. Rest API functionality should be provided to prepare and schedule recipe, list recipe, delete and other operations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)