You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@pig.apache.org by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org> on 2012/09/28 05:51:07 UTC

[jira] [Updated] (PIG-2925) Extremely long JobConf values should not be added to Streaming environment

     [ https://issues.apache.org/jira/browse/PIG-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy V. Ryaboy updated PIG-2925:
-----------------------------------

    Attachment: PIG-2925.patch

Decided it's easier to set the max length, and allow users to bump it up or down as needed, than to mess with explicit blacklists. Too hard to control what everyone might decide to shove into their config.
                
> Extremely long JobConf values should not be added to Streaming environment
> --------------------------------------------------------------------------
>
>                 Key: PIG-2925
>                 URL: https://issues.apache.org/jira/browse/PIG-2925
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Dmitriy V. Ryaboy
>            Assignee: Dmitriy V. Ryaboy
>         Attachments: PIG-2925.patch
>
>
> PIG-2900 made Pig match Hadoop's practice of making all JobConf properties available as environment variables for pig streaming. 
> As it happens, ProcessBuilder creates a long string with all environment variables and their values on it, then runs it; having some of the weightier properties like serialized plans in the environment can make the total command length exceed OS limits. 
> We propose adding a blacklist of properties which should not be added to streaming environment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira