You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Alejandro Abdelnur (JIRA)" <ji...@apache.org> on 2007/07/10 16:31:04 UTC
[jira] Created: (HADOOP-1587) Tasks run by MiniMRCluster don't get
sysprops from TestCases
Tasks run by MiniMRCluster don't get sysprops from TestCases
------------------------------------------------------------
Key: HADOOP-1587
URL: https://issues.apache.org/jira/browse/HADOOP-1587
Project: Hadoop
Issue Type: Bug
Components: test
Environment: unix
Reporter: Alejandro Abdelnur
Priority: Blocker
While it seems a general problem it is surfacing in streaming TestSymLink testcase with patch for HADOOP-1558
The contrib testcases use src/contrib/test/hadoop-site.xml.
The property 'mapred.system.dir' is this file is defined as with a variable '${contrib.name}'.
The src/build/build-contrib.xml ant file sets the sysproperty 'contrib.name' to the name of the contrib component for the JVM running the testcase.
The problem is that when a testcase uses MiniMRCluster the TaskRunner forks a JVM for the task and in this JVM (which uses the above hadoop-site.xml) the variable 'contrib.name' is undefined.
If I hardcode 'streaming' in the hadoop-site.xml for the TestSymLink the testcase works fine.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1587) Tasks run by MiniMRCluster don't get
sysprops from TestCases
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-1587:
--------------------------------
Attachment: 1587.patch
Ok the patch fixes the issue by passing the sysprops in the JobConf property mapred.child.java.opts from TestSymLink.
> Tasks run by MiniMRCluster don't get sysprops from TestCases
> ------------------------------------------------------------
>
> Key: HADOOP-1587
> URL: https://issues.apache.org/jira/browse/HADOOP-1587
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Environment: unix
> Reporter: Alejandro Abdelnur
> Assignee: Devaraj Das
> Priority: Blocker
> Attachments: 1587.patch
>
>
> While it seems a general problem it is surfacing in streaming TestSymLink testcase with patch for HADOOP-1558
> The contrib testcases use src/contrib/test/hadoop-site.xml.
> The property 'mapred.system.dir' is this file is defined as with a variable '${contrib.name}'.
> The src/build/build-contrib.xml ant file sets the sysproperty 'contrib.name' to the name of the contrib component for the JVM running the testcase.
> The problem is that when a testcase uses MiniMRCluster the TaskRunner forks a JVM for the task and in this JVM (which uses the above hadoop-site.xml) the variable 'contrib.name' is undefined.
> If I hardcode 'streaming' in the hadoop-site.xml for the TestSymLink the testcase works fine.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1587) Tasks run by MiniMRCluster don't get
sysprops from TestCases
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1587:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I committed this. Thanks, Devaraj!
> Tasks run by MiniMRCluster don't get sysprops from TestCases
> ------------------------------------------------------------
>
> Key: HADOOP-1587
> URL: https://issues.apache.org/jira/browse/HADOOP-1587
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Environment: unix
> Reporter: Alejandro Abdelnur
> Assignee: Devaraj Das
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: 1587.patch
>
>
> While it seems a general problem it is surfacing in streaming TestSymLink testcase with patch for HADOOP-1558
> The contrib testcases use src/contrib/test/hadoop-site.xml.
> The property 'mapred.system.dir' is this file is defined as with a variable '${contrib.name}'.
> The src/build/build-contrib.xml ant file sets the sysproperty 'contrib.name' to the name of the contrib component for the JVM running the testcase.
> The problem is that when a testcase uses MiniMRCluster the TaskRunner forks a JVM for the task and in this JVM (which uses the above hadoop-site.xml) the variable 'contrib.name' is undefined.
> If I hardcode 'streaming' in the hadoop-site.xml for the TestSymLink the testcase works fine.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
RE: Jetty 6.x upgrade?
Posted by Devaraj Das <dd...@yahoo-inc.com>.
Thanks! Please note that this has not been thoroughly tested :)
-----Original Message-----
From: James Kennedy [mailto:james.kennedy@troove.net]
Sent: Wednesday, July 25, 2007 7:07 PM
To: hadoop-dev@lucene.apache.org
Subject: Re: Jetty 6.x upgrade?
>> I have a patch and set of jars for 6.1.4. I haven't tried things with
>> 6.1.2 though. I will upload those on jira against hadoop-1273. Is that
ok?
Thats great. I'd rather upgrade to 6.1.4 than downgrade to 5.1.4.
> Hi James, Pls take a look at hadoop-1650.
>
Thanks for addressing this issue so quickly, i'll try the patch out later
today.
> -----Original Message-----
> From: James Kennedy [mailto:james.kennedy@troove.net]
> Sent: Wednesday, July 25, 2007 12:19 AM
> To: hadoop-dev@lucene.apache.org
> Subject: Jetty 6.x upgrade?
>
> Is there any chance we can reopen the Jetty 6.x upgrade task
> (https://issues.apache.org/jira/browse/HADOOP-565) now that the Jetty
> peformance prob appears to be fixed?
>
> I'm having trouble incorporating MapReduce into my app which is using
> Jetty
> 6.1.1 because Hadoop is still using 5.1.4. I'm getting all kinds of
> compatibility issues and its a real pain.
>
>
Re: Jetty 6.x upgrade?
Posted by James Kennedy <ja...@troove.net>.
>> I have a patch and set of jars for 6.1.4. I haven't tried things with 6.1.2
>> though. I will upload those on jira against hadoop-1273. Is that ok?
Thats great. I'd rather upgrade to 6.1.4 than downgrade to 5.1.4.
> Hi James, Pls take a look at hadoop-1650.
>
Thanks for addressing this issue so quickly, i'll try the patch out
later today.
> -----Original Message-----
> From: James Kennedy [mailto:james.kennedy@troove.net]
> Sent: Wednesday, July 25, 2007 12:19 AM
> To: hadoop-dev@lucene.apache.org
> Subject: Jetty 6.x upgrade?
>
> Is there any chance we can reopen the Jetty 6.x upgrade task
> (https://issues.apache.org/jira/browse/HADOOP-565) now that the Jetty
> peformance prob appears to be fixed?
>
> I'm having trouble incorporating MapReduce into my app which is using Jetty
> 6.1.1 because Hadoop is still using 5.1.4. I'm getting all kinds of
> compatibility issues and its a real pain.
>
>
RE: Jetty 6.x upgrade?
Posted by Devaraj Das <dd...@yahoo-inc.com>.
Hi James, Pls take a look at hadoop-1650.
-----Original Message-----
From: James Kennedy [mailto:james.kennedy@troove.net]
Sent: Wednesday, July 25, 2007 12:19 AM
To: hadoop-dev@lucene.apache.org
Subject: Jetty 6.x upgrade?
Is there any chance we can reopen the Jetty 6.x upgrade task
(https://issues.apache.org/jira/browse/HADOOP-565) now that the Jetty
peformance prob appears to be fixed?
I'm having trouble incorporating MapReduce into my app which is using Jetty
6.1.1 because Hadoop is still using 5.1.4. I'm getting all kinds of
compatibility issues and its a real pain.
RE: Jetty 6.x upgrade?
Posted by Devaraj Das <dd...@yahoo-inc.com>.
I have a patch and set of jars for 6.1.4. I haven't tried things with 6.1.2
though. I will upload those on jira against hadoop-1273. Is that ok?
-----Original Message-----
From: James Kennedy [mailto:james.kennedy@troove.net]
Sent: Wednesday, July 25, 2007 12:19 AM
To: hadoop-dev@lucene.apache.org
Subject: Jetty 6.x upgrade?
Is there any chance we can reopen the Jetty 6.x upgrade task
(https://issues.apache.org/jira/browse/HADOOP-565) now that the Jetty
peformance prob appears to be fixed?
I'm having trouble incorporating MapReduce into my app which is using Jetty
6.1.1 because Hadoop is still using 5.1.4. I'm getting all kinds of
compatibility issues and its a real pain.
Jetty 6.x upgrade?
Posted by James Kennedy <ja...@troove.net>.
Is there any chance we can reopen the Jetty 6.x upgrade task
(https://issues.apache.org/jira/browse/HADOOP-565) now that the Jetty
peformance prob appears to be fixed?
I'm having trouble incorporating MapReduce into my app which is using
Jetty 6.1.1 because Hadoop is still using 5.1.4. I'm getting all kinds
of compatibility issues and its a real pain.
[jira] Commented: (HADOOP-1587) Tasks run by MiniMRCluster don't
get sysprops from TestCases
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12515056 ]
Devaraj Das commented on HADOOP-1587:
-------------------------------------
I could reproduce the problem and i will comment in more detail on the problem shortly..
> Tasks run by MiniMRCluster don't get sysprops from TestCases
> ------------------------------------------------------------
>
> Key: HADOOP-1587
> URL: https://issues.apache.org/jira/browse/HADOOP-1587
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Environment: unix
> Reporter: Alejandro Abdelnur
> Assignee: Devaraj Das
> Priority: Blocker
>
> While it seems a general problem it is surfacing in streaming TestSymLink testcase with patch for HADOOP-1558
> The contrib testcases use src/contrib/test/hadoop-site.xml.
> The property 'mapred.system.dir' is this file is defined as with a variable '${contrib.name}'.
> The src/build/build-contrib.xml ant file sets the sysproperty 'contrib.name' to the name of the contrib component for the JVM running the testcase.
> The problem is that when a testcase uses MiniMRCluster the TaskRunner forks a JVM for the task and in this JVM (which uses the above hadoop-site.xml) the variable 'contrib.name' is undefined.
> If I hardcode 'streaming' in the hadoop-site.xml for the TestSymLink the testcase works fine.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1587) Tasks run by MiniMRCluster don't
get sysprops from TestCases
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12513201 ]
Devaraj Das commented on HADOOP-1587:
-------------------------------------
I could not reproduce this problem with the current trunk. Could you pls try to do the same with the current trunk?
> Tasks run by MiniMRCluster don't get sysprops from TestCases
> ------------------------------------------------------------
>
> Key: HADOOP-1587
> URL: https://issues.apache.org/jira/browse/HADOOP-1587
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Environment: unix
> Reporter: Alejandro Abdelnur
> Priority: Blocker
>
> While it seems a general problem it is surfacing in streaming TestSymLink testcase with patch for HADOOP-1558
> The contrib testcases use src/contrib/test/hadoop-site.xml.
> The property 'mapred.system.dir' is this file is defined as with a variable '${contrib.name}'.
> The src/build/build-contrib.xml ant file sets the sysproperty 'contrib.name' to the name of the contrib component for the JVM running the testcase.
> The problem is that when a testcase uses MiniMRCluster the TaskRunner forks a JVM for the task and in this JVM (which uses the above hadoop-site.xml) the variable 'contrib.name' is undefined.
> If I hardcode 'streaming' in the hadoop-site.xml for the TestSymLink the testcase works fine.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1587) Tasks run by MiniMRCluster don't get
sysprops from TestCases
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sameer Paranjpye reassigned HADOOP-1587:
----------------------------------------
Assignee: Devaraj Das
> Tasks run by MiniMRCluster don't get sysprops from TestCases
> ------------------------------------------------------------
>
> Key: HADOOP-1587
> URL: https://issues.apache.org/jira/browse/HADOOP-1587
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Environment: unix
> Reporter: Alejandro Abdelnur
> Assignee: Devaraj Das
> Priority: Blocker
>
> While it seems a general problem it is surfacing in streaming TestSymLink testcase with patch for HADOOP-1558
> The contrib testcases use src/contrib/test/hadoop-site.xml.
> The property 'mapred.system.dir' is this file is defined as with a variable '${contrib.name}'.
> The src/build/build-contrib.xml ant file sets the sysproperty 'contrib.name' to the name of the contrib component for the JVM running the testcase.
> The problem is that when a testcase uses MiniMRCluster the TaskRunner forks a JVM for the task and in this JVM (which uses the above hadoop-site.xml) the variable 'contrib.name' is undefined.
> If I hardcode 'streaming' in the hadoop-site.xml for the TestSymLink the testcase works fine.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1587) Tasks run by MiniMRCluster don't
get sysprops from TestCases
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12515281 ]
Hadoop QA commented on HADOOP-1587:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12362518/1587.patch applied and successfully tested against trunk revision r559068.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/461/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/461/console
> Tasks run by MiniMRCluster don't get sysprops from TestCases
> ------------------------------------------------------------
>
> Key: HADOOP-1587
> URL: https://issues.apache.org/jira/browse/HADOOP-1587
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Environment: unix
> Reporter: Alejandro Abdelnur
> Assignee: Devaraj Das
> Priority: Blocker
> Attachments: 1587.patch
>
>
> While it seems a general problem it is surfacing in streaming TestSymLink testcase with patch for HADOOP-1558
> The contrib testcases use src/contrib/test/hadoop-site.xml.
> The property 'mapred.system.dir' is this file is defined as with a variable '${contrib.name}'.
> The src/build/build-contrib.xml ant file sets the sysproperty 'contrib.name' to the name of the contrib component for the JVM running the testcase.
> The problem is that when a testcase uses MiniMRCluster the TaskRunner forks a JVM for the task and in this JVM (which uses the above hadoop-site.xml) the variable 'contrib.name' is undefined.
> If I hardcode 'streaming' in the hadoop-site.xml for the TestSymLink the testcase works fine.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1587) Tasks run by MiniMRCluster don't get
sysprops from TestCases
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-1587:
--------------------------------
Status: Patch Available (was: Open)
> Tasks run by MiniMRCluster don't get sysprops from TestCases
> ------------------------------------------------------------
>
> Key: HADOOP-1587
> URL: https://issues.apache.org/jira/browse/HADOOP-1587
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Environment: unix
> Reporter: Alejandro Abdelnur
> Assignee: Devaraj Das
> Priority: Blocker
> Attachments: 1587.patch
>
>
> While it seems a general problem it is surfacing in streaming TestSymLink testcase with patch for HADOOP-1558
> The contrib testcases use src/contrib/test/hadoop-site.xml.
> The property 'mapred.system.dir' is this file is defined as with a variable '${contrib.name}'.
> The src/build/build-contrib.xml ant file sets the sysproperty 'contrib.name' to the name of the contrib component for the JVM running the testcase.
> The problem is that when a testcase uses MiniMRCluster the TaskRunner forks a JVM for the task and in this JVM (which uses the above hadoop-site.xml) the variable 'contrib.name' is undefined.
> If I hardcode 'streaming' in the hadoop-site.xml for the TestSymLink the testcase works fine.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1587) Tasks run by MiniMRCluster don't get
sysprops from TestCases
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1587:
----------------------------------
Fix Version/s: 0.14.0
> Tasks run by MiniMRCluster don't get sysprops from TestCases
> ------------------------------------------------------------
>
> Key: HADOOP-1587
> URL: https://issues.apache.org/jira/browse/HADOOP-1587
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Environment: unix
> Reporter: Alejandro Abdelnur
> Assignee: Devaraj Das
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: 1587.patch
>
>
> While it seems a general problem it is surfacing in streaming TestSymLink testcase with patch for HADOOP-1558
> The contrib testcases use src/contrib/test/hadoop-site.xml.
> The property 'mapred.system.dir' is this file is defined as with a variable '${contrib.name}'.
> The src/build/build-contrib.xml ant file sets the sysproperty 'contrib.name' to the name of the contrib component for the JVM running the testcase.
> The problem is that when a testcase uses MiniMRCluster the TaskRunner forks a JVM for the task and in this JVM (which uses the above hadoop-site.xml) the variable 'contrib.name' is undefined.
> If I hardcode 'streaming' in the hadoop-site.xml for the TestSymLink the testcase works fine.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.