jclouds-labs-google and nodes not created by jclouds

[Rick Wright <ri...@google.com>]

Hi all,
I'm running into an issue with the jclouds-labs-google code and I'm trying
to figure out if it's a bug, working-as-intended, or if I'm just missing
something fundamental.

I'm getting errors whenever I try to list or create nodes if I have any
instances in my project that *were not* created by jclouds (like through
the web interface, for example).  After troubleshooting for a while, the
problem seems to be this:

In InstanceZoneToNodeMetadata.java [1] on line 87, it attempts to get the
group information from the metadata on the node.  It appears to be looking
for an attribute called 'jclouds-group' and uses the value of that to set
the value for the group.  However, for instances that are created outside
of jclouds, this key-value pair does not exist, and so group ends up being
null.

The null group causes an error (NullPointerException)
from org.jclouds.compute.internal.FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.checkGroup
which prevents nodes from being listed and prevents new nodes from being
created.

As a workaround, setting the metadata (e.g. jclouds-group=nogroup) for the
other instances keeps the error from occurring, but that's obviously not
ideal.

Does this make sense?  Am I missing something obvious, or is there a
problem here?

Thanks,
Rick

[1] -
https://github.com/jclouds/jclouds-labs-google/blob/master/google-compute-engine/src/main/java/org/jclouds/googlecomputeengine/compute/functions/InstanceInZoneToNodeMetadata.java

Re: jclouds-labs-google and nodes not created by jclouds

[Andrew Phillips <ap...@qrmedia.com>]

> in https://issues.apache.org/jira/browse/JCLOUDS-442 my intent was to avoid
> the creation of too many firewalls as the current implementation is doing.

Looking briefly at the code in the PR, I don't think so. As far as I  
can see, the code still expects to be able to get a non-null group and  
will still fail if the group is null, which is the problem Rick is  
talking about.

Or am I missing something here..?

ap

Re: jclouds-labs-google and nodes not created by jclouds

[Andrea Turli <an...@gmail.com>]

Andrew P.,

in https://issues.apache.org/jira/browse/JCLOUDS-442 my intent was to avoid
the creation of too many firewalls as the current implementation is doing.
I'm not sure if this will solve (incidentally) also the issue discussed
here.

Andrea


On Wed, Mar 19, 2014 at 12:47 AM, Andrew Phillips <ap...@qrmedia.com>wrote:

> In the case of non-jclouds created nodes, you can get the valid firewall
>> rules from a combination of the network and the tags on the
>> instance/firewall.
>>
>
> Sounds like we might need a pull request to update that portion of the
> code? You might also want to look at:
>
> https://issues.apache.org/jira/browse/JCLOUDS-381 and
> https://issues.apache.org/jira/browse/JCLOUDS-442
>
> first, as they both touch this portion.
>
> Andrea, abayer...any thoughts on this?
>
> ap
>

Re: jclouds-labs-google and nodes not created by jclouds

[Andrew Phillips <ap...@qrmedia.com>]

> In the case of non-jclouds created nodes, you can get the valid firewall
> rules from a combination of the network and the tags on the
> instance/firewall.

Sounds like we might need a pull request to update that portion of the  
code? You might also want to look at:

https://issues.apache.org/jira/browse/JCLOUDS-381 and
https://issues.apache.org/jira/browse/JCLOUDS-442

first, as they both touch this portion.

Andrea, abayer...any thoughts on this?

ap

Re: jclouds-labs-google and nodes not created by jclouds

[Rick Wright <ri...@google.com>]

Each instance (node) is associated with a "network"  (The project usually
comes with a "default" network already defined, and if you create an
instance from the web interface without changing the network, the instance
will generally be associated with that network.)

Firewalls are a property of the network, so you can define the firewall
rules on the network and then any instances associated with that network
get those rules (although the firewall rules can also be associated with
tags, so you can have the specific rules only apply to a subset of the
instances on the network by setting the appropriate tags).

So, for example, you can have a firewall rule called "web" that allows
connections on TCP port 80 and set the target tags to "webserver".  If this
firewall is associated with the default network and then you create an
instance also associated with the default network, then setting the tag
"webserver" on the instance will open TCP:80.

In the case of non-jclouds created nodes, you can get the valid firewall
rules from a combination of the network and the tags on the
instance/firewall.

-Rick


On Tue, Mar 18, 2014 at 4:06 PM, Andrew Phillips <ap...@qrmedia.com>wrote:

> Does this make sense?  Am I missing something obvious, or is there a
>> problem here?
>>
>
> Looks like your troubleshooting is pretty spot on ;-) And I'd say (as an
> outsider - I didn't implement the code so don't know the exact intention)
> it looks like a bug to me. "groupFromMapOrName" definitely can return null,
> and InstanceInZoneToNodeMetadata doesn't seem to take that into account.
>
> It seems that the code requires the name to get the appropriate firewall.
> In the case of a non-jclouds created node, what should the firewall logic
> do?
>
> ap
>

Re: jclouds-labs-google and nodes not created by jclouds

[Andrew Phillips <ap...@qrmedia.com>]

> Does this make sense?  Am I missing something obvious, or is there a
> problem here?

Looks like your troubleshooting is pretty spot on ;-) And I'd say (as  
an outsider - I didn't implement the code so don't know the exact  
intention) it looks like a bug to me. "groupFromMapOrName" definitely  
can return null, and InstanceInZoneToNodeMetadata doesn't seem to take  
that into account.

It seems that the code requires the name to get the appropriate  
firewall. In the case of a non-jclouds created node, what should the  
firewall logic do?

ap