[jira] [Updated] (MAPREDUCE-5626) TaskLogServlet could not get syslog

["yangjun (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

yangjun updated MAPREDUCE-5626:
-------------------------------

       Fix Version/s: 1.2.1
              Labels: patch  (was: )
    Target Version/s: 1.2.1
        Release Note: modify TaskLogServlet haveTaskLog method
              Status: Patch Available  (was: Open)

modify TaskLogServlet haveTaskLog method
private boolean haveTaskLog(TaskAttemptID taskId, boolean isCleanup, 
TaskLog.LogName type) throws IOException { 
File f = TaskLog.getTaskLogFile(taskId, isCleanup, type); 
if (f.exists() && f.canRead())
{ return true; }
else { 
File indexFile = TaskLog.getIndexFile(taskId, isCleanup); 
if (!indexFile.exists())
{ return false; }
BufferedReader fis; 
try
{ fis = new BufferedReader(new InputStreamReader( SecureIOUtils.openForRead(indexFile, TaskLog.obtainLogDirOwner(taskId)))); }
catch (FileNotFoundException ex) { 
LOG.warn("Index file for the log of " + taskId 
+ " does not exist.");
// Assume no task reuse is used and files exist on attemptdir 
StringBuffer input = new StringBuffer(); 
input.append(LogFileDetail.LOCATION 
+ TaskLog.getAttemptDir(taskId, isCleanup) + "\n"); 
for (LogName logName : TaskLog.LOGS_TRACKED_BY_INDEX_FILES)
{ input.append(logName + ":0 -1\n"); }

fis = new BufferedReader(new StringReader(input.toString())); 
}
try { 
String str = fis.readLine(); 
if (str == null)
{ // thefile doesn't have anything throw new IOException("Index file for the log of " + taskId + "is empty."); }

String loc = str.substring(str.indexOf(LogFileDetail.LOCATION) 
+ LogFileDetail.LOCATION.length()); 
File tf = new File(loc, type.toString()); 
return tf.exists() && tf.canRead();
} finally
{ if (fis != null) fis.close(); }

}
}

> TaskLogServlet could not get syslog
> -----------------------------------
>
>                 Key: MAPREDUCE-5626
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5626
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 1.2.1
>         Environment: Linux version 2.6.18-238.9.1.el5
> Java(TM) SE Runtime Environment (build 1.6.0_43-b01)
> hadoop-1.2.1
>            Reporter: yangjun
>            Assignee: yangjun
>            Priority: Minor
>              Labels: patch
>             Fix For: 1.2.1
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> When multiply tasks use one jvm and generated logs.
> eg.
> ./attempt_201211220735_0001_m_000000_0:
> log.index
> ./attempt_201211220735_0001_m_000001_0:
> log.index
> ./attempt_201211220735_0001_m_000002_0:
> log.index  stderr  stdout  syslog
> get from http://xxxxxxxx:50060/tasklog?attemptid= attempt_201211220735_0001_m_000000_0 
> could get stderr,stdout,but not the others,include syslog.
> see TaskLogServlet.haveTaskLog() method, not check from local && log.index, but check the original path.
> resolve:
> modify TaskLogServlet haveTaskLog method
>     private boolean haveTaskLog(TaskAttemptID taskId, boolean isCleanup,  
>             TaskLog.LogName type) throws IOException {  
>         File f = TaskLog.getTaskLogFile(taskId, isCleanup, type);  
>         if (f.exists() && f.canRead()) {  
>             return true;  
>         } else {  
>             File indexFile = TaskLog.getIndexFile(taskId, isCleanup);  
>             if (!indexFile.exists()) {  
>                 return false;  
>             }  
>        
>        
>             BufferedReader fis;  
>             try {  
>                 fis = new BufferedReader(new InputStreamReader(  
>                         SecureIOUtils.openForRead(indexFile,  
>                                 TaskLog.obtainLogDirOwner(taskId))));  
>             } catch (FileNotFoundException ex) {  
>                 LOG.warn("Index file for the log of " + taskId  
>                         + " does not exist.");  
>        
>        
>                 // Assume no task reuse is used and files exist on attemptdir  
>                 StringBuffer input = new StringBuffer();  
>                 input.append(LogFileDetail.LOCATION  
>                         + TaskLog.getAttemptDir(taskId, isCleanup) + "\n");  
>                 for (LogName logName : TaskLog.LOGS_TRACKED_BY_INDEX_FILES) {  
>                     input.append(logName + ":0 -1\n");  
>                 }  
>                 fis = new BufferedReader(new StringReader(input.toString()));  
>             }  
>        
>        
>             try {  
>                 String str = fis.readLine();  
>                 if (str == null) { // thefile doesn't have anything  
>                     throw new IOException("Index file for the log of " + taskId  
>                             + "is empty.");  
>                 }  
>                 String loc = str.substring(str.indexOf(LogFileDetail.LOCATION)  
>                         + LogFileDetail.LOCATION.length());  
>                 File tf = new File(loc, type.toString());  
>                 return tf.exists() && tf.canRead();  
>        
>        
>             } finally {  
>                 if (fis != null)  
>                     fis.close();  
>             }  
>         }  
>        
>        
>     }  
> workaround:
> url add filter=SYSLOG could print syslog also.



--
This message was sent by Atlassian JIRA
(v6.1#6144)