Helpers for MAPREDUCE-3727 scenario

[Jonathan Eagles <je...@gmail.com>]

MapReduce (MAPREDUCE-3727) and Pig on Tez (PIG-4333) have implemented
a work around to unset the mapreduce.job.credentials.binary property.
Now hive has hit the same scenario this week (no jira yet that I see).
While Tez dones generally handle mapreduce properties, should we
provide a config cleaning helper that will help unify the config
cleaning that will protect our users?

Jon

Re: Helpers for MAPREDUCE-3727 scenario

[Siddharth Seth <ss...@apache.org>]

Tez does provide some config helpers - to translate configs between MR and
Tez. I'm not sure which downstream projects actually make use of these. We
could add some cleanup code here / or a new helper - but it'll be up to the
users to actually make use of this in the correct place.
For the specific property (mapreduce.job.credentials.binary) - Tez uses
it's own property for this - which I believe Pig / Hive setup. If I
understand the usage correctly - Oozie should only be loading this property
in the launcher, which should have a separate configuration from the actual
job. Does Oozie leak these configs into the configs for the main job ?

On Tue, Mar 10, 2015 at 8:32 AM, Jonathan Eagles <je...@gmail.com> wrote:

> MapReduce (MAPREDUCE-3727) and Pig on Tez (PIG-4333) have implemented
> a work around to unset the mapreduce.job.credentials.binary property.
> Now hive has hit the same scenario this week (no jira yet that I see).
> While Tez dones generally handle mapreduce properties, should we
> provide a config cleaning helper that will help unify the config
> cleaning that will protect our users?
>
> Jon
>