You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Tom White (JIRA)" <ji...@apache.org> on 2011/06/03 19:42:47 UTC
[jira] [Created] (WHIRR-325) Reduce cloud provider-specific code in
scripts
Reduce cloud provider-specific code in scripts
----------------------------------------------
Key: WHIRR-325
URL: https://issues.apache.org/jira/browse/WHIRR-325
Project: Whirr
Issue Type: Improvement
Reporter: Tom White
Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-325) Reduce cloud provider-specific code
in scripts
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045057#comment-13045057 ]
Andrei Savu commented on WHIRR-325:
-----------------------------------
> We could use this along with runScriptOnNode() to pass IP addresses, etc. Is this what you had in mind?
Exactly.
> One possible downside to this approach is that using "whirr run-script" wouldn't be able to set these environment variables
We could rewrite "whirr run-script" in terms of runScriptOnNode.
Actually I think it would be useful to have a runScriptOnNode method on the ClusterController that would set all the required environment variables and we could call that by using an executor service.
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-325) Reduce cloud provider-specific code
in scripts
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13079248#comment-13079248 ]
Andrei Savu commented on WHIRR-325:
-----------------------------------
We can resume work on this one as soon as we upgrade to jclouds 1.1.0.
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
> Attachments: WHIRR-325-rewrite-run-script.patch
>
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-325) Reduce cloud provider-specific code in
scripts
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated WHIRR-325:
----------------------------
Attachment: WHIRR-325.patch
This patch show the changes needed to StatementBuilder to set environment variables with the instance data (such as IP address). It still needs work to update the service scripts to use the variables.
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
> Attachments: WHIRR-325-rewrite-run-script.patch, WHIRR-325.patch
>
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-325) Reduce cloud provider-specific code
in scripts
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043906#comment-13043906 ]
Tom White commented on WHIRR-325:
---------------------------------
Passing node-specific scripts (which is what is needed to pass each node its IP addresses) would need a modification to jclouds, since ComputeService#runScriptOnNodesMatching runs the same script on all nodes. Calling this API for each node in turn with a different script is inefficient since jclouds queries for all the running nodes and filters each time.
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-325) Reduce cloud provider-specific code
in scripts
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044030#comment-13044030 ]
Tom White commented on WHIRR-325:
---------------------------------
Perfect. I think this is the way to go - using an executor service to run things concurrently.
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-325) Reduce cloud provider-specific code
in scripts
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044704#comment-13044704 ]
Tom White commented on WHIRR-325:
---------------------------------
> I would use ScriptBuilder.addEnvironmentVariableScope for this.
I hadn't seen this before. We could use this along with runScriptOnNode() to pass IP addresses, etc. Is this what you had in mind?
One possible downside to this approach is that using "whirr run-script" wouldn't be able to set these environment variables, so any scripts that take advantage of them would could not be run like this.
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-325) Reduce cloud provider-specific code
in scripts
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045452#comment-13045452 ]
Andrei Savu commented on WHIRR-325:
-----------------------------------
I have create an issue for solving this: http://code.google.com/p/jclouds/issues/detail?id=597
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
> Attachments: WHIRR-325-rewrite-run-script.patch
>
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-325) Reduce cloud provider-specific code
in scripts
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045563#comment-13045563 ]
Tom White commented on WHIRR-325:
---------------------------------
> We could rewrite "whirr run-script" in terms of runScriptOnNode
Yes, this would work.
Rather than adding methods to ClusterController, I think we need a RunScriptClusterAction which encapsulates this logic. This would also address WHIRR-324.
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
> Attachments: WHIRR-325-rewrite-run-script.patch
>
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-325) Reduce cloud provider-specific code in
scripts
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-325:
------------------------------
Attachment: WHIRR-325-rewrite-run-script.patch
I've done some work on this one (trying to rewrite "whirr run-script") and I believe I've discovered a bug in jclouds.
It seems like {{runScriptOnNode}} ignores the new credentials provided as RunScriptOptions.
I believe the error should be fixed in {{BaseComputeService.runScriptOnNode}} and the code should be similar to {{BaseComputeService.TransformNodesIntoInitializedScriptRunners.apply}}
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
> Attachments: WHIRR-325-rewrite-run-script.patch
>
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-325) Reduce cloud provider-specific code
in scripts
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085831#comment-13085831 ]
Andrei Savu commented on WHIRR-325:
-----------------------------------
Looks good to me.
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
> Attachments: WHIRR-325-rewrite-run-script.patch, WHIRR-325.patch
>
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-325) Reduce cloud provider-specific code
in scripts
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044617#comment-13044617 ]
Andrei Savu commented on WHIRR-325:
-----------------------------------
I would use ScriptBuilder.addEnvironmentVariableScope for this.
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-325) Reduce cloud provider-specific code
in scripts
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044014#comment-13044014 ]
Adrian Cole commented on WHIRR-325:
-----------------------------------
when we move to jclouds 1.0.0, we can use the single-node feature avoiding the overhead you mention.
ExecResponse runScriptOnNode(String id, String runScript);
Cheers!
> Reduce cloud provider-specific code in scripts
> ----------------------------------------------
>
> Key: WHIRR-325
> URL: https://issues.apache.org/jira/browse/WHIRR-325
> Project: Whirr
> Issue Type: Improvement
> Reporter: Tom White
>
> Many of the bash functions have a cloud provider option (-c) that they use to determine how to find the local public/private IP address, or the local data directory (/mnt vs /data, for example).
> We should pass what is needed (e.g. IP addresses, directories), or at least put the code in a common function.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira