You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Chris K Wensel (JIRA)" <ji...@apache.org> on 2008/10/23 00:41:44 UTC
[jira] Created: (HADOOP-4498) JobHistory does not escape regex
chars in jobName
JobHistory does not escape regex chars in jobName
-------------------------------------------------
Key: HADOOP-4498
URL: https://issues.apache.org/jira/browse/HADOOP-4498
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.19.0
Reporter: Chris K Wensel
Priority: Blocker
JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
Trace:
Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
....
at java.util.regex.Pattern.error(Pattern.java:1713)
at java.util.regex.Pattern.clazz(Pattern.java:2254)
at java.util.regex.Pattern.clazz(Pattern.java:2210)
at java.util.regex.Pattern.sequence(Pattern.java:1818)
at java.util.regex.Pattern.expr(Pattern.java:1752)
at java.util.regex.Pattern.compile(Pattern.java:1460)
at java.util.regex.Pattern.<init>(Pattern.java:1133)
at java.util.regex.Pattern.compile(Pattern.java:823)
at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644092#action_12644092 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4498:
------------------------------------------------
The following seems not working well.
{noformat}
jobName pattern
------------------------------------------------
abc\\Exyz \Qabc\\E\\E\Qxyz\E
{noformat}
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644298#action_12644298 ]
Hudson commented on HADOOP-4498:
--------------------------------
Integrated in Hadoop-trunk #648 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/648/])
. Ensure that JobHistory correctly escapes the job name so that regex patterns work. Contributed by Chris Wensel.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Assignee: Chris K Wensel
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-4498:
----------------------------------
Resolution: Fixed
Fix Version/s: 0.19.0
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Chris!
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Assignee: Chris K Wensel
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris K Wensel updated HADOOP-4498:
-----------------------------------
Status: Patch Available (was: Open)
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris K Wensel updated HADOOP-4498:
-----------------------------------
Attachment: hadoop-4498-v2.patch
escapes \E properly. includes two unit tests as suggested
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-4498:
------------------------------------
Release Note: (was: Stuffs jobName between \E and \Q so it is treated as a literal and will not interfere with the pattern parsing.)
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Assignee: Chris K Wensel
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12643458#action_12643458 ]
Amareshwari Sriramadasu commented on HADOOP-4498:
-------------------------------------------------
+1
Latest patch looks good.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644166#action_12644166 ]
Hadoop QA commented on HADOOP-4498:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12392867/hadoop-4498-v3.patch
against trunk revision 709040.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3513/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3513/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3513/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3513/console
This message is automatically generated.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Assignee: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4498:
-------------------------------------------
Status: Patch Available (was: Open)
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Assignee: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris K Wensel updated HADOOP-4498:
-----------------------------------
Status: Patch Available (was: Open)
Thank for being patient with this issues.
the first test now reflects the issue in the same manner it presented.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644104#action_12644104 ]
Chris K Wensel commented on HADOOP-4498:
----------------------------------------
great news its working for your test case. thanks for testing this.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Assignee: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642376#action_12642376 ]
Amar Kamat commented on HADOOP-4498:
------------------------------------
Chris,
Can you help us reproduce this bug? What should be the jobname which can lead to this issue? Plz include a testcase which
{code}
1) Submits a job with the *special* job name
2) Check if the job submission succeeds
{code}
This test should fail without the fix.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642199#action_12642199 ]
Chris K Wensel commented on HADOOP-4498:
----------------------------------------
All but org.apache.hadoop.fs.TestLocalDirAllocator tests pass.
Did not include a unit test since most of these methods are private to JobHistory..
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642373#action_12642373 ]
Amareshwari Sriramadasu commented on HADOOP-4498:
-------------------------------------------------
If I'm not missing something, the following String replaceAll is not useful.
{noformat}
+ private static String escapeRegexChars( String string ) {
+ return "\\Q"+string.replaceAll("\\\\E", "\\\\E")+"\\E";
+ }
+
{noformat}
If possible can you add unit test with jobNames having metacharacters and \E etc.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644098#action_12644098 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4498:
------------------------------------------------
> The following seems not working well...
Oops, I was wrong about that. The replacement is perfect.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4498:
-------------------------------------------
Assignee: Chris K Wensel
Status: Open (was: Patch Available)
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Assignee: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape regex
chars in jobName
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris K Wensel updated HADOOP-4498:
-----------------------------------
Component/s: mapred
changed component affected
> JobHistory does not escape regex chars in jobName
> -------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642892#action_12642892 ]
Amareshwari Sriramadasu commented on HADOOP-4498:
-------------------------------------------------
I submitted the job with name : [[name] value], without the patch and the job was successful.
I was able to parse the history of the jobs with complex names too, both from the web ui and history viewer.
Also the Testcase: testComplexName passes without the patch, though the Testcase: testComplexNameWithRegex failed
Can you tell us how to reproduce the issue?
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642385#action_12642385 ]
Chris K Wensel commented on HADOOP-4498:
----------------------------------------
I'll add a unit test for this.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12643713#action_12643713 ]
Chris K Wensel commented on HADOOP-4498:
----------------------------------------
anyone willing to commit this so we can get into the 0.19.x release?
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris K Wensel updated HADOOP-4498:
-----------------------------------
Attachment: hadoop-4498.patch
Stuffs jobName between \Q and \E so it is treated as a literal value.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642507#action_12642507 ]
Chris K Wensel commented on HADOOP-4498:
----------------------------------------
all tests pass except TestLocalDirAllocator, which always fails for me (even with a clean tree).
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris K Wensel updated HADOOP-4498:
-----------------------------------
Status: Open (was: Patch Available)
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris K Wensel updated HADOOP-4498:
-----------------------------------
Summary: JobHistory does not escape literal jobName when used in a regex pattern (was: JobHistory does not escape regex chars in jobName)
changed title
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris K Wensel updated HADOOP-4498:
-----------------------------------
Status: Open (was: Patch Available)
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris K Wensel updated HADOOP-4498:
-----------------------------------
Attachment: hadoop-4498-v3.patch
both tests now properly exercise the bug
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498-v2.patch, hadoop-4498-v3.patch, hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642394#action_12642394 ]
Chris K Wensel commented on HADOOP-4498:
----------------------------------------
good catch on the regex, this works much better
{noformat}
private static String escapeRegexChars( String string ) {
return "\\Q"+string.replaceAll("\\\\E", "\\\\E\\\\\\\\E\\\\Q")+"\\E";
}
{noformat}
via a println of the pattern..
{noformat}
jobName pattern
------------------------------------------------
[[name] value] \Q[[name] value]\E
name \Evalue] \Qname \E\\E\Qvalue]\E
{noformat}
let me know if this doesn't seem right. i'll run the full test suite in the morning.
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4498) JobHistory does not escape literal
jobName when used in a regex pattern
Posted by "Chris K Wensel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris K Wensel updated HADOOP-4498:
-----------------------------------
Release Note: Stuffs jobName between \E and \Q so it is treated as a literal and will not interfere with the pattern parsing.
Status: Patch Available (was: Open)
> JobHistory does not escape literal jobName when used in a regex pattern
> -----------------------------------------------------------------------
>
> Key: HADOOP-4498
> URL: https://issues.apache.org/jira/browse/HADOOP-4498
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Chris K Wensel
> Priority: Blocker
> Attachments: hadoop-4498.patch
>
>
> JobHistory#getJobHistoryFileName throws a PatternSyntaxException if the jobName includes characters that could be regex metacharacters or metasequences and are subsequently considered malformed.
> Trace:
> Exception in thread "initJobs" java.util.regex.PatternSyntaxException: Unclosed character class near index 97
> ....
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.clazz(Pattern.java:2210)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:823)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:632)
> at org.apache.hadoop.mapred.JobHistory$JobInfo.finalizeRecovery(JobHistory.java:740)
> at org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:1532)
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2232)
> at org.apache.hadoop.mapred.JobInProgress.terminateJob(JobInProgress.java:1938)
> at org.apache.hadoop.mapred.JobInProgress.terminate(JobInProgress.java:1953)
> at org.apache.hadoop.mapred.JobInProgress.fail(JobInProgress.java:2012)
> at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:62)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.