You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hemanth Yamijala (JIRA)" <ji...@apache.org> on 2008/03/31 06:56:24 UTC
[jira] Created: (HADOOP-3133) [HOD] hodring.log-destination-uri
should be set to a default value
[HOD] hodring.log-destination-uri should be set to a default value
------------------------------------------------------------------
Key: HADOOP-3133
URL: https://issues.apache.org/jira/browse/HADOOP-3133
Project: Hadoop Core
Issue Type: Improvement
Components: contrib/hod
Affects Versions: 0.16.0
Reporter: Hemanth Yamijala
Priority: Minor
In the current setup, we do not insist a value for hodring.log-destination-uri which is used to store tarred hadoop logs. Now, if there is a hadoop start up problem, the cluster could get automatically deallocated. However, the hadoop logs would be lost too, as the above configuration parameter does not have a default value.
Possibly this should be set to atleast file:///<local file system> path so that the logs will be saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3133) [HOD] hodring.log-destination-uri
should be set to a default value
Posted by "Luca Telloli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12592434#action_12592434 ]
Luca Telloli commented on HADOOP-3133:
--------------------------------------
Additionally, log-destination-uri has a different behavior where set on hdfs:// or file:// .
If used like hdfs://destination, the final tars will be under: destination/$USER/hod-logs/$JOBID
while on a file-system it would be just under destination.
I think that the syntax for the [hodring] section should be enriched in such a way that different parameters/variables can be accessed for storing logs, like $USER or $JOBID, especially on a non HDFS filesystem (where they'd take much less space). for instance by specifying a path like file://dir/$USER/$JOBID.
I made a test creating manually the dir/$USER directory, then passing the variable $USER with log-destination-uri=file:///dir/$USER, and it worked, but since the job id is not visible to bash, it is not possible to have a directory user/job.
> [HOD] hodring.log-destination-uri should be set to a default value
> ------------------------------------------------------------------
>
> Key: HADOOP-3133
> URL: https://issues.apache.org/jira/browse/HADOOP-3133
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Priority: Minor
> Fix For: 0.18.0
>
>
> In the current setup, we do not insist a value for hodring.log-destination-uri which is used to store tarred hadoop logs. Now, if there is a hadoop start up problem, the cluster could get automatically deallocated. However, the hadoop logs would be lost too, as the above configuration parameter does not have a default value.
> Possibly this should be set to atleast file:///<local file system> path so that the logs will be saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3133) [HOD] hodring.log-destination-uri
should be set to a default value
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-3133:
-------------------------------------
Fix Version/s: (was: 0.18.0)
Moving out of the 0.18 queue as these could not make the feature freeze date.
> [HOD] hodring.log-destination-uri should be set to a default value
> ------------------------------------------------------------------
>
> Key: HADOOP-3133
> URL: https://issues.apache.org/jira/browse/HADOOP-3133
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Priority: Minor
>
> In the current setup, we do not insist a value for hodring.log-destination-uri which is used to store tarred hadoop logs. Now, if there is a hadoop start up problem, the cluster could get automatically deallocated. However, the hadoop logs would be lost too, as the above configuration parameter does not have a default value.
> Possibly this should be set to atleast file:///<local file system> path so that the logs will be saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-3133) [HOD]
hodring.log-destination-uri should be set to a default value
Posted by "Luca Telloli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12592434#action_12592434 ]
tuka edited comment on HADOOP-3133 at 4/25/08 10:18 AM:
----------------------------------------------------------------
Additionally, log-destination-uri has a different behavior where set on hdfs:// or file:// , because it considers different variables.
I think that the syntax for the [hodring] section should be enriched in such a way that different parameters/variables can be accessed for storing logs, like $USER or $JOBID, especially on a non HDFS filesystem (where they'd take much less space). for instance by specifying a path like file://dir/$USER/$JOBID.
I made a test creating manually the dir/$USER directory, then passing the variable $USER with log-destination-uri=file:///dir/$USER, and it worked, but since the job id is not visible to bash, it is not possible to have a directory user/job.
was (Author: tuka):
Additionally, log-destination-uri has a different behavior where set on hdfs:// or file:// .
If used like hdfs://destination, the final tars will be under: destination/$USER/hod-logs/$JOBID
while on a file-system it would be just under destination.
I think that the syntax for the [hodring] section should be enriched in such a way that different parameters/variables can be accessed for storing logs, like $USER or $JOBID, especially on a non HDFS filesystem (where they'd take much less space). for instance by specifying a path like file://dir/$USER/$JOBID.
I made a test creating manually the dir/$USER directory, then passing the variable $USER with log-destination-uri=file:///dir/$USER, and it worked, but since the job id is not visible to bash, it is not possible to have a directory user/job.
> [HOD] hodring.log-destination-uri should be set to a default value
> ------------------------------------------------------------------
>
> Key: HADOOP-3133
> URL: https://issues.apache.org/jira/browse/HADOOP-3133
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Priority: Minor
> Fix For: 0.18.0
>
>
> In the current setup, we do not insist a value for hodring.log-destination-uri which is used to store tarred hadoop logs. Now, if there is a hadoop start up problem, the cluster could get automatically deallocated. However, the hadoop logs would be lost too, as the above configuration parameter does not have a default value.
> Possibly this should be set to atleast file:///<local file system> path so that the logs will be saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3133) [HOD] hodring.log-destination-uri
should be set to a default value
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-3133:
-------------------------------------
Fix Version/s: 0.18.0
> [HOD] hodring.log-destination-uri should be set to a default value
> ------------------------------------------------------------------
>
> Key: HADOOP-3133
> URL: https://issues.apache.org/jira/browse/HADOOP-3133
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/hod
> Affects Versions: 0.16.0
> Reporter: Hemanth Yamijala
> Priority: Minor
> Fix For: 0.18.0
>
>
> In the current setup, we do not insist a value for hodring.log-destination-uri which is used to store tarred hadoop logs. Now, if there is a hadoop start up problem, the cluster could get automatically deallocated. However, the hadoop logs would be lost too, as the above configuration parameter does not have a default value.
> Possibly this should be set to atleast file:///<local file system> path so that the logs will be saved.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.