You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Randy Clayton (JIRA)" <ji...@apache.org> on 2012/05/08 22:35:49 UTC
[jira] [Created] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Randy Clayton created MAPREDUCE-4234:
----------------------------------------
Summary: SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
Key: MAPREDUCE-4234
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: examples
Affects Versions: 0.23.3
Reporter: Randy Clayton
Priority: Minor
The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426572#comment-13426572 ]
Hudson commented on MAPREDUCE-4234:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #331 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/331/])
svn merge -c 1367789 FIXES: MAPREDUCE-4234. SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name) (Robert Evans via jeagles) (Revision 1367795)
Result = SUCCESS
jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367795
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/SortValidator.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/SortValidator.java
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426647#comment-13426647 ]
Hudson commented on MAPREDUCE-4234:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #1154 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1154/])
MAPREDUCE-4234. SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name) (Robert Evans via jeagles) (Revision 1367789)
Result = FAILURE
jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367789
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/SortValidator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/SortValidator.java
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Jonathan Eagles (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-4234:
---------------------------------------
Resolution: Fixed
Target Version/s: 0.23.3, 3.0.0, 2.2.0-alpha
Status: Resolved (was: Patch Available)
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284751#comment-13284751 ]
Harsh J commented on MAPREDUCE-4234:
------------------------------------
Can the file not be created in the regular target/ or the working directory itself?
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425811#comment-13425811 ]
Robert Joseph Evans commented on MAPREDUCE-4234:
------------------------------------------------
Yes the test failure is unrelated. MAPREDUCE-4475
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Jonathan Eagles (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425910#comment-13425910 ]
Jonathan Eagles commented on MAPREDUCE-4234:
--------------------------------------------
The coding changes look good at first glance. I will take a deeper look at the usage.
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "madhukara phatak (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
madhukara phatak updated MAPREDUCE-4234:
----------------------------------------
Affects Version/s: trunk
Status: Patch Available (was: Open)
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425326#comment-13425326 ]
Hadoop QA commented on MAPREDUCE-4234:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12538433/MR-4234.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2677//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2677//console
This message is automatically generated.
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-4234:
-------------------------------------------
Attachment: MR-4234.txt
This patch updates the validator to include a UUID and then cleans up afterwards.
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426164#comment-13426164 ]
Hudson commented on MAPREDUCE-4234:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2609 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2609/])
MAPREDUCE-4234. SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name) (Robert Evans via jeagles) (Revision 1367789)
Result = SUCCESS
jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367789
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/SortValidator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/SortValidator.java
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426589#comment-13426589 ]
Hudson commented on MAPREDUCE-4234:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #1122 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1122/])
MAPREDUCE-4234. SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name) (Robert Evans via jeagles) (Revision 1367789)
Result = FAILURE
jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367789
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/SortValidator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/SortValidator.java
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419277#comment-13419277 ]
Robert Joseph Evans commented on MAPREDUCE-4234:
------------------------------------------------
Thanks for the review Harsh, I will update it to use something like
{code}
localDir = new File(System.getProperty("test.build.dir", "target/test-dir"),
+ SortValidator.class.getName());
{code}
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-4234:
-------------------------------------------
Attachment: MR-4234.txt
The failure looks spurious to me. I have not been able to reproduce it. I have upmerged and am posting the updated patch.
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423486#comment-13423486 ]
Hadoop QA commented on MAPREDUCE-4234:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12538073/MR-4234.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2664//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2664//console
This message is automatically generated.
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278838#comment-13278838 ]
Hadoop QA commented on MAPREDUCE-4234:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12528056/MAPREDUCE-4234.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 javadoc. The javadoc tool appears to have generated 2 warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2394//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2394//console
This message is automatically generated.
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419284#comment-13419284 ]
Robert Joseph Evans commented on MAPREDUCE-4234:
------------------------------------------------
Never mind, I thought that it was a regular unit test, but it is not. This is used to validate that sort worked as expected, and is a stand alone executable so we do want to throw it in hadoop.tmp.dir, with the addition of a UUID, we probably want to clean up after we are done anyways.
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426189#comment-13426189 ]
Hudson commented on MAPREDUCE-4234:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2562 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2562/])
MAPREDUCE-4234. SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name) (Robert Evans via jeagles) (Revision 1367789)
Result = FAILURE
jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367789
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/SortValidator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/SortValidator.java
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Jonathan Eagles (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426130#comment-13426130 ]
Jonathan Eagles commented on MAPREDUCE-4234:
--------------------------------------------
+1. Checking in the changes.
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426160#comment-13426160 ]
Hudson commented on MAPREDUCE-4234:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #2544 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2544/])
MAPREDUCE-4234. SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name) (Robert Evans via jeagles) (Revision 1367789)
Result = SUCCESS
jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367789
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/SortValidator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/SortValidator.java
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch, MR-4234.txt, MR-4234.txt
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "madhukara phatak (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
madhukara phatak updated MAPREDUCE-4234:
----------------------------------------
Attachment: MAPREDUCE-4234.patch
added UUID to get an unique output name
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-4234) SortValidator.java is
incompatible with multi-user or parallel use (due to a /tmp file with
static name)
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans reassigned MAPREDUCE-4234:
----------------------------------------------
Assignee: Robert Joseph Evans
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
> --------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4234
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
> Affects Versions: 0.23.3, trunk
> Reporter: Randy Clayton
> Assignee: Robert Joseph Evans
> Priority: Minor
> Attachments: MAPREDUCE-4234.patch
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira