[jira] [Updated] (OOZIE-3112) SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction

["Peter Cseh (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OOZIE-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Peter Cseh updated OOZIE-3112:
------------------------------
    Attachment: OOZIE-3112.01.patch

Attaching patch. The precedence is in the properties-file:
- user-defined properites file via the --properties-file option
- localized spark-defaults.conf file in the container of the SparkAction
- Properties propagated via SparkConfigurationService.

This changes the earlier behavior. Earlier it was:
- if there was a --properties-file option and a spark-defaults.conf file, the user-defined file was ignored
- properties propagated via SparkConfigurationService had precedence over the --properties-file option or the localized spark-defaults.conf file

> SparkConfigrationService overwrites properties provided via --properties-file option in SparkAction
> ---------------------------------------------------------------------------------------------------
>
>                 Key: OOZIE-3112
>                 URL: https://issues.apache.org/jira/browse/OOZIE-3112
>             Project: Oozie
>          Issue Type: Bug
>            Reporter: Peter Cseh
>            Assignee: Peter Cseh
>         Attachments: OOZIE-3112.01.patch
>
>
> SparkConfigurationService injects the config values as --conf to SparkSubmit. This will overwrite propties provided in the --properies-file option which is not the expected behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)