You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Mac Yang (JIRA)" <ji...@apache.org> on 2009/04/20 23:14:47 UTC
[jira] Created: (CHUKWA-162) ChukwaConfiguration should handle
agent and backend configuration separately
ChukwaConfiguration should handle agent and backend configuration separately
----------------------------------------------------------------------------
Key: CHUKWA-162
URL: https://issues.apache.org/jira/browse/CHUKWA-162
Project: Hadoop Chukwa
Issue Type: Bug
Reporter: Mac Yang
Currently all components (both front end and back end) are forced to consume all config files together. As a result, different components can not have different settings for the same config parameter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-162) ChukwaConfiguration should handle
agent and backend configuration separately
Posted by "Mac Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700951#action_12700951 ]
Mac Yang commented on CHUKWA-162:
---------------------------------
And that's why it's not a blocker.
It's current forcing all these config files to be meshed together and that seems like a bad design.
* hadoop-site.xml
* chukwa-agent-conf.xml
* chukwa-collector-conf.xml
* chukwa-demux-conf.xml
What happens when the map reduced based aggregation becomes available? I don't think it makes sense to limit that to use the same settings as the demux jobs.
> ChukwaConfiguration should handle agent and backend configuration separately
> ----------------------------------------------------------------------------
>
> Key: CHUKWA-162
> URL: https://issues.apache.org/jira/browse/CHUKWA-162
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Mac Yang
>
> Currently all components (both front end and back end) are forced to consume all config files together. As a result, different components can not have different settings for the same config parameter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-162) ChukwaConfiguration should handle
agent and backend configuration separately
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701050#action_12701050 ]
Eric Yang commented on CHUKWA-162:
----------------------------------
Why collector needs hadoop-site.xml? This dependency didn't exist before. demux pipeline could pick up hadoop configuration by sourcing HADOOP_CONF_DIR in the class path. For now, it is required to place hadoop-site.xml in CHUKWA_CONF_DIR, but this could be fixed by fixing the demux.sh shell script.
> ChukwaConfiguration should handle agent and backend configuration separately
> ----------------------------------------------------------------------------
>
> Key: CHUKWA-162
> URL: https://issues.apache.org/jira/browse/CHUKWA-162
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Mac Yang
>
> Currently all components (both front end and back end) are forced to consume all config files together. As a result, different components can not have different settings for the same config parameter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-162) ChukwaConfiguration should handle
agent and backend configuration separately
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700957#action_12700957 ]
Jerome Boulon commented on CHUKWA-162:
--------------------------------------
We should be able to initialized all components by giving an existing Configuration object or by specifying a key that will add specific resource(s).
+1 on getting a specific configuration
- agent - chukwa-agent-conf.xml only
- collector - hadoop-site.xml + chukwa-collector.xml
- demux pipeline - hadoop-site.xml + chukwa-demux-conf.xml
Also it's not totally true that there's no sharing between data collection and data processors, writer.hdfs.filesystem is one example.
> ChukwaConfiguration should handle agent and backend configuration separately
> ----------------------------------------------------------------------------
>
> Key: CHUKWA-162
> URL: https://issues.apache.org/jira/browse/CHUKWA-162
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Mac Yang
>
> Currently all components (both front end and back end) are forced to consume all config files together. As a result, different components can not have different settings for the same config parameter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-162) ChukwaConfiguration should handle
agent and backend configuration separately
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700945#action_12700945 ]
Eric Yang commented on CHUKWA-162:
----------------------------------
Please be more specific on the components that has this problem. There is no parameter sharing between data collection and data processors to my knowledge.
> ChukwaConfiguration should handle agent and backend configuration separately
> ----------------------------------------------------------------------------
>
> Key: CHUKWA-162
> URL: https://issues.apache.org/jira/browse/CHUKWA-162
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Mac Yang
>
> Currently all components (both front end and back end) are forced to consume all config files together. As a result, different components can not have different settings for the same config parameter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.