[jira] [Commented] (OOZIE-3372) Adding credentials to multiple workflows after enabling Kerberos

["Andras Piros (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16661958#comment-16661958 ] 

Andras Piros commented on OOZIE-3372:
-------------------------------------

Interesting idea [~kuldeepkulkarni09@gmail.com] having a default set of {{CredentialProvider}} classes that would be always given to each workflow.

What can be done today is by using the [Fluent Job API|https://github.com/apache/oozie/blob/master/docs/src/site/markdown/DG_FluentJobAPI.md] to define a common {{Credential}} object w/ a {{CredentialBuilder}} and use it multiple times [as in the example|https://github.com/apache/oozie/blob/master/examples/src/main/java/org/apache/oozie/example/fluentjob/CredentialsRetrying.java]. This gives the user a possibility to reuse the same Fluent Job {{Credential}} across different {{Workflow}} instances. It doesn't have an effect for all the workflows, though - only for the ones you'll be injecting.

I see following caveats for having one single place adding a {{Credential}} to all the {{Workflow}} instances of an Oozie service:
 * all the {{Workflow}} instances would be affected with no exception. This can be problematic if some workflows don't need e.g. {{HbaseCredential}} instances (because these don't need to talk to HBase), for performance reasons
 * {{HCatCredentials}} and {{Hive2Credentials}} use parameters that are specific to the HCatalog or HiveServer2 instances these workflow actions will be connecting. There are scenarios where there are multiple HiveServer2 instances w/ different {{Hive2Credentials}} parameters, needed for two different workflow actions for the same workflow job. In case we'd have a default {{Hive2Credentials}} created and supplied for both actions (but necessary only for the first one), and a special one (necessary for only the other action), I can imagine the second action wouldn't be able to authenticate correctly, based on the incorrect {{Hive2Credentials}} parameters supplied by the default one

To summarize, even if we'd implement a way to inject special {{CredentialsProvider}} instances into each workflow action by default, we still would want some kind of intelligent blacklisting, based on e.g. action type / default providers mapping (e.g. {{Hive2Credentials}} wouldn't be injected to Sqoop actions but for Hive2 and Spark actions), and explicit definition (the default {{Hive2Credentials}} wouldn't be injected for Hive2 actions that have an explicit setting). [~kmarton] [~asalamon74] what do you think?

> Adding credentials to multiple workflows after enabling Kerberos
> ----------------------------------------------------------------
>
>                 Key: OOZIE-3372
>                 URL: https://issues.apache.org/jira/browse/OOZIE-3372
>             Project: Oozie
>          Issue Type: Improvement
>          Components: workflow
>            Reporter: Kuldeep Kulkarni
>            Priority: Minor
>
> If I have thousands of running workflows in a cluster(unsecured) and I configure Kerberos authentication, I will have to edit each workflow to add a credential section which can be difficult and time consuming. Is there any option to avoid manual edit? If not, can we add a feature using which we can configure credentials at one place (maybe in Oozie configs) and can be referred by respective action in the workflow? 
> I know it can be complex to do this however just wanted to check if there is any better way or this can be considered as a feature request.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)