You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Yuri Pradkin (JIRA)" <ji...@apache.org> on 2009/12/09 19:52:18 UTC
[jira] Created: (MAPREDUCE-1286) Quotes in environment
HADOOP_CLIENT_OPTS confuse parsing if this env is concatenated with
something else
Quotes in environment HADOOP_CLIENT_OPTS confuse parsing if this env is concatenated with something else
--------------------------------------------------------------------------------------------------------
Key: MAPREDUCE-1286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1286
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: task
Affects Versions: 0.20.2
Environment: Linux F12; streaming
Reporter: Yuri Pradkin
Assignee: Yuri Pradkin
I use streaming and in the perl-reducer I write to hdfs using a pipe to hdfs -put - .... It turns out that because TaskRunner sets the environment HADOOP_CLIENT_OPTS in double quotes, when hdfs shell script concatenates these with something else, the command fails: .e.g java -Dblah=x -Dfoo=y "-Dhadoop.tasklog.taskid=z -Dhadoop.tasklog.totalLogFileSize=s"...
Since I don't see any reason to have these double quotes in the original code; I propose they're removed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (MAPREDUCE-1286) Quotes in environment
HADOOP_CLIENT_OPTS confuse parsing if this env is concatenated with
something else
Posted by "Christian Kunz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christian Kunz reopened MAPREDUCE-1286:
---------------------------------------
I would vote to declare this as blocker for 0.20.
It not only affects hadoop streaming, but also system calls using the hadoop command line utility will fail like:
Exception in thread "main" java.lang.NoClassDefFoundError: "-Dhadoop/tasklog/taskid=attempt_201006181908_0498_m_000000_0
Caused by: java.lang.ClassNotFoundException: "-Dhadoop.tasklog.taskid=attempt_201006181908_0498_m_000000_0
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class: "-Dhadoop.tasklog.taskid=attempt_201006181908_0498_m_000000_0. Program will exit.
> Quotes in environment HADOOP_CLIENT_OPTS confuse parsing if this env is concatenated with something else
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1286
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1286
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
> Affects Versions: 0.20.2
> Environment: Linux F12; streaming
> Reporter: Yuri Pradkin
> Assignee: Yuri Pradkin
> Attachments: mr-1286
>
>
> I use streaming and in the perl-reducer I write to hdfs using a pipe to hdfs -put - .... It turns out that because TaskRunner sets the environment HADOOP_CLIENT_OPTS in double quotes, when hdfs shell script concatenates these with something else, the command fails: .e.g java -Dblah=x -Dfoo=y "-Dhadoop.tasklog.taskid=z -Dhadoop.tasklog.totalLogFileSize=s"...
> Since I don't see any reason to have these double quotes in the original code; I propose they're removed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.