You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Christian Kunz (JIRA)" <ji...@apache.org> on 2008/10/18 21:50:44 UTC
[jira] Created: (HADOOP-4456) distributed filecache:
mapred.create.symlink=yes not effective
distributed filecache: mapred.create.symlink=yes not effective
--------------------------------------------------------------
Key: HADOOP-4456
URL: https://issues.apache.org/jira/browse/HADOOP-4456
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Affects Versions: 0.17.2
Reporter: Christian Kunz
I wanted to run a pipes job with an executable that has to load libhdfs.so.1 dynamically and I tried to achieve this by using mapred.create.symlink=yes.
But I did not see any symbolic links created.
I might do something wrong, but on the other hand, when looking at the code, it seems that in mapred/TaskRunner.java the workDir is created after the attempts to create symbolic links.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4456) distributed filecache:
mapred.create.symlink=yes not effective
Posted by "Christian Kunz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640920#action_12640920 ]
Christian Kunz commented on HADOOP-4456:
----------------------------------------
Amareshwari looked at my job file and noticed that my setting was:
mapred.create.symlink=true
instead of
mapred.create.symlink=yes
My bad. Somehow I did not think this would make any difference.
Thanks, Amareshwari, for looking into it.
> distributed filecache: mapred.create.symlink=yes not effective
> --------------------------------------------------------------
>
> Key: HADOOP-4456
> URL: https://issues.apache.org/jira/browse/HADOOP-4456
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.2
> Reporter: Christian Kunz
> Assignee: Amareshwari Sriramadasu
>
> I wanted to run a pipes job with an executable that has to load libhdfs.so.1 dynamically and I tried to achieve this by using mapred.create.symlink=yes.
> But I did not see any symbolic links created.
> I might do something wrong, but on the other hand, when looking at the code, it seems that in mapred/TaskRunner.java the workDir is created after the attempts to create symbolic links.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-4456) distributed filecache:
mapred.create.symlink=yes not effective
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu reassigned HADOOP-4456:
-----------------------------------------------
Assignee: Amareshwari Sriramadasu
> distributed filecache: mapred.create.symlink=yes not effective
> --------------------------------------------------------------
>
> Key: HADOOP-4456
> URL: https://issues.apache.org/jira/browse/HADOOP-4456
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.2
> Reporter: Christian Kunz
> Assignee: Amareshwari Sriramadasu
>
> I wanted to run a pipes job with an executable that has to load libhdfs.so.1 dynamically and I tried to achieve this by using mapred.create.symlink=yes.
> But I did not see any symbolic links created.
> I might do something wrong, but on the other hand, when looking at the code, it seems that in mapred/TaskRunner.java the workDir is created after the attempts to create symbolic links.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4456) distributed filecache:
mapred.create.symlink=yes not effective
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640861#action_12640861 ]
Arun C Murthy commented on HADOOP-4456:
---------------------------------------
Christian, can you please confirm that the URI added to the DistributedCache was the form: hdfs//<namenode>:<port>/source#target?
The DistributedCache uses the 'URI fragment' as the target for the symlink...
> distributed filecache: mapred.create.symlink=yes not effective
> --------------------------------------------------------------
>
> Key: HADOOP-4456
> URL: https://issues.apache.org/jira/browse/HADOOP-4456
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.2
> Reporter: Christian Kunz
>
> I wanted to run a pipes job with an executable that has to load libhdfs.so.1 dynamically and I tried to achieve this by using mapred.create.symlink=yes.
> But I did not see any symbolic links created.
> I might do something wrong, but on the other hand, when looking at the code, it seems that in mapred/TaskRunner.java the workDir is created after the attempts to create symbolic links.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4456) distributed filecache:
mapred.create.symlink=yes not effective
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640907#action_12640907 ]
Amareshwari Sriramadasu commented on HADOOP-4456:
-------------------------------------------------
I tried to reproduce this. But I couldnt.
I did mapred.create.symlink=yes and gave file to be symlinked as mapred.cache.files=hdfs://localhost:19000/user/amarsri/file.txt#file. And I see symlinks created on branch 0.17 and also trunk.
> distributed filecache: mapred.create.symlink=yes not effective
> --------------------------------------------------------------
>
> Key: HADOOP-4456
> URL: https://issues.apache.org/jira/browse/HADOOP-4456
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.2
> Reporter: Christian Kunz
>
> I wanted to run a pipes job with an executable that has to load libhdfs.so.1 dynamically and I tried to achieve this by using mapred.create.symlink=yes.
> But I did not see any symbolic links created.
> I might do something wrong, but on the other hand, when looking at the code, it seems that in mapred/TaskRunner.java the workDir is created after the attempts to create symbolic links.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-4456) distributed filecache:
mapred.create.symlink=yes not effective
Posted by "Christian Kunz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christian Kunz resolved HADOOP-4456.
------------------------------------
Resolution: Invalid
> distributed filecache: mapred.create.symlink=yes not effective
> --------------------------------------------------------------
>
> Key: HADOOP-4456
> URL: https://issues.apache.org/jira/browse/HADOOP-4456
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.2
> Reporter: Christian Kunz
> Assignee: Amareshwari Sriramadasu
>
> I wanted to run a pipes job with an executable that has to load libhdfs.so.1 dynamically and I tried to achieve this by using mapred.create.symlink=yes.
> But I did not see any symbolic links created.
> I might do something wrong, but on the other hand, when looking at the code, it seems that in mapred/TaskRunner.java the workDir is created after the attempts to create symbolic links.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4456) distributed filecache:
mapred.create.symlink=yes not effective
Posted by "Christian Kunz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640908#action_12640908 ]
Christian Kunz commented on HADOOP-4456:
----------------------------------------
Did you try this with a pipes job?
> distributed filecache: mapred.create.symlink=yes not effective
> --------------------------------------------------------------
>
> Key: HADOOP-4456
> URL: https://issues.apache.org/jira/browse/HADOOP-4456
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.2
> Reporter: Christian Kunz
>
> I wanted to run a pipes job with an executable that has to load libhdfs.so.1 dynamically and I tried to achieve this by using mapred.create.symlink=yes.
> But I did not see any symbolic links created.
> I might do something wrong, but on the other hand, when looking at the code, it seems that in mapred/TaskRunner.java the workDir is created after the attempts to create symbolic links.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4456) distributed filecache:
mapred.create.symlink=yes not effective
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640910#action_12640910 ]
Amareshwari Sriramadasu commented on HADOOP-4456:
-------------------------------------------------
Yes. I tried with the pipes job and also normal java job.
> distributed filecache: mapred.create.symlink=yes not effective
> --------------------------------------------------------------
>
> Key: HADOOP-4456
> URL: https://issues.apache.org/jira/browse/HADOOP-4456
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.2
> Reporter: Christian Kunz
> Assignee: Amareshwari Sriramadasu
>
> I wanted to run a pipes job with an executable that has to load libhdfs.so.1 dynamically and I tried to achieve this by using mapred.create.symlink=yes.
> But I did not see any symbolic links created.
> I might do something wrong, but on the other hand, when looking at the code, it seems that in mapred/TaskRunner.java the workDir is created after the attempts to create symbolic links.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4456) distributed filecache:
mapred.create.symlink=yes not effective
Posted by "Christian Kunz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640865#action_12640865 ]
Christian Kunz commented on HADOOP-4456:
----------------------------------------
Confirmed
> distributed filecache: mapred.create.symlink=yes not effective
> --------------------------------------------------------------
>
> Key: HADOOP-4456
> URL: https://issues.apache.org/jira/browse/HADOOP-4456
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.2
> Reporter: Christian Kunz
>
> I wanted to run a pipes job with an executable that has to load libhdfs.so.1 dynamically and I tried to achieve this by using mapred.create.symlink=yes.
> But I did not see any symbolic links created.
> I might do something wrong, but on the other hand, when looking at the code, it seems that in mapred/TaskRunner.java the workDir is created after the attempts to create symbolic links.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.