You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Paolo Castagna (Commented) (JIRA)" <ji...@apache.org> on 2011/11/02 17:09:33 UTC

[jira] [Commented] (WHIRR-425) Define a distinct mechanism that handles the work of transfering artefacts on the remote machines

    [ https://issues.apache.org/jira/browse/WHIRR-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142243#comment-13142243 ] 

Paolo Castagna commented on WHIRR-425:
--------------------------------------

I experienced an issue which is probably related to this, details here: http://markmail.org/message/3ldqgbl5ppj7rg5c

I started an Hadoop cluster of 17 nodes, but the NameNode reported only 8 live nodes.
Trying to connect to one of the instances not listed by the NameNode, I verified no Java process was running and there were errors in the /tmp/logs/stderr.log file.
                
> Define a distinct mechanism that handles the work of transfering artefacts on the remote machines
> -------------------------------------------------------------------------------------------------
>
>                 Key: WHIRR-425
>                 URL: https://issues.apache.org/jira/browse/WHIRR-425
>             Project: Whirr
>          Issue Type: Improvement
>          Components: core
>            Reporter: Andrei Savu
>
> As we begin to start larger clusters we need to start to think about a different, more reliable and more scalable way of bringing the required artefacts on the remote machines. I suggest we should create some abstraction so that we can use different policies based on needs (e.g. for small clusters direct download for apache.org, for larger clusters build a cache in s3 or download from machines inside the same cluster in a peer to peer fashion). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira