Re: Default endpoint via Zookeeper (D-OSGi) (repost to get its own thread, sorry all)

[David Bosschaert <da...@gmail.com>]

Hi Thom,

Great to hear that this is resolved.

Copying users@cxf.apache.org, for the record :)

WRT to releasing 1.1, I'm hoping we can get it out soon, before the
end of the month I would imagine. CXF PMC folks, when do you think
you'd be able to do a DOSGi 1.1 release?

Thanks,

David

2009/9/9 Shulok, Thomas <th...@lmco.com>:
> Hi David,
>
> That did the trick.  I'm also no longer seeing the InterfaceDataMonitorListenerImpl getting pinged every second.
>
> Any indication when 1.1 will actually get buffed up and released?
>
> Thanks for all your help,
> Thom
>
> -----Original Message-----
> From: David Bosschaert [mailto:david.bosschaert@gmail.com]
> Sent: Wednesday, September 09, 2009 7:50 AM
> To: Shulok, Thomas
> Subject: Re: Default endpoint via Zookeeper (D-OSGi) (repost to get its own thread, sorry all)
>
> Hi Thom,
>
> That client-side behaviour was added in response to your problem. I added in on August 21st.
>
>> Other wrinkle is I'm using the single bundle distro on the client side, just in case that makes a difference...
>
> that should work just fine, or are you seeing any problems with it?
>
> So is your problem resolved now?
>
> Cheers,
>
> David
>
> 2009/9/9 Shulok, Thomas <th...@lmco.com>:
>> Hi David,
>>
>> Thanks for the update.  The client does find Zookeeper okay, and it even finds the node in question.  The one thing my client doesn't have, though, is the latest 1.1 snapshot (I figured if the property wasn't even showing up in the node, there wasn't much point in working the client side of it).  Was the "auto-add" capability added to the client recently?  The way it behaves right now if it doesn't find the explicit org.apache.cxf.ws.address, the client defaults back to localhost for the  ws address, if it does find it in the node, it all works great.  Other wrinkle is I'm using the single bundle distro on the client side, just in case that makes a difference...
>>
>> Thanks,
>> Thom
>>
>> -----Original Message-----
>> From: David Bosschaert [mailto:david.bosschaert@gmail.com]
>> Sent: Wednesday, September 09, 2009 7:25 AM
>> To: Shulok, Thomas
>> Subject: Re: Default endpoint via Zookeeper (D-OSGi) (repost to get
>> its own thread, sorry all)
>>
>> Hi Thomas,
>>
>> Looking at your issue again and running a version of the discovery
>> demo that has been made similarly, I'm getting the same information in
>> the zookeeper registry, but it still works fine for me :) You see the
>> org.apache.cxf.ws.address isn't really needed. It is automatically
>> added by the client side of the zookeeper discovery if it's missing
>> (see line 76 of
>> https://svn.apache.org/repos/asf/cxf/dosgi/trunk/discovery/distributed
>> /cxf-discovery/src/main/java/org/apache/cxf/dosgi/discovery/zookeeper/
>> InterfaceDataMonitorListenerImpl.java)
>>
>> However, the reason why your client isn't finding the server might be
>> caused by the fact that the zookeeper client needs to be told about
>> the location of the zookeeper server. You can do this by placing a
>> file like the attached (put in the proper host name instead of
>> 127.0.0.1) in the "load" directory of felix (this is automatically picked up by the config admin that's part of the zookeeper single-bundle distro) or by otherwise putting the infor in the config admin service. I realise that this may not be mentioned explicitly enough in the docs (http://cxf.apache.org/dosgi-discovery.html)...
>>
>> Oneway to figure out whether this is set up correctly is by doing X =
>> the number of the zookeeper-based discovery bundle
>> -> services X
>> ...
>> objectClass = org.osgi.service.cm.ManagedService
>> service.id = 39
>> service.pid = org.apache.cxf.dosgi.discovery.zookeeper
>> zookeeper.host = nbdubdbosscha2.emea.progress.com zookeeper.port =
>> 2181 zookeeper.timeout = 3000
>>
>> You need to make sure that the zookeeper.host in this Config Admin managed service is properly set.
>>
>> Hope this helps,
>>
>> David
>>
>> 2009/9/8 Shulok, Thomas <th...@lmco.com>:
>>> Hi David,
>>>
>>> This might be a quicker way to track it down.  I've attached a simple bundle that demonstrates the problem (source included).  If you install and start it, you should see a node entry like the following:
>>>
>>>
>>> [zk: localhost:2181(CONNECTED) 8] get
>>> /osgi/service_registry/zooey/Zoo/129.218.156.29#9000##zooey#Zoo
>>> #
>>> #Tue Sep 08 16:17:17 CDT 2009
>>> osgi.remote.endpoint.location=http\://129.218.156.29\:9000/zooey/Zoo
>>> service.id=50
>>> objectClass=[Ljava.lang.String;@115870
>>> service.exported.interfaces=*
>>> osgi.remote.endpoint.id=a20d6940-0d9e-4896-9b55-cf1b87aa78b5
>>>
>>> cZxid = 326
>>> ctime = Tue Sep 08 16:17:17 CDT 2009
>>> mZxid = 326
>>> mtime = Tue Sep 08 16:17:17 CDT 2009
>>> pZxid = 326
>>> cversion = 0
>>> dataVersion = 0
>>> aclVersion = 0
>>> ephemeralOwner = 82079853410516998
>>> dataLength = 245
>>> numChildren = 0
>>>
>>> If you see the above, the problem must be on your side, if you see a org.apache.cxf.ws.address entry, then it must be on my side.
>>>
>>> Thanks,
>>> Thom
>>>
>>> PS you may need to start the bundle twice since the service decorator is inside the bundle.  Sometimes it doesn't seem to hook it on the first start.
>>>
>>>
>>> -----Original Message-----
>>> From: David Bosschaert [mailto:david.bosschaert@gmail.com]
>>> Sent: Friday, September 04, 2009 7:35 AM
>>> To: Shulok, Thomas
>>> Subject: Re: Default endpoint via Zookeeper (D-OSGi) (repost to get
>>> its own thread, sorry all)
>>>
>>> Ok, let me see if I can reproduce this over the weekend...
>>> Could you maybe send me the following information:
>>> * the list of bundles on the client side (i.e. a felix ps command)
>>> * the list of bundles on the server side
>>> * your service properties as registered on the server side (i.e. a
>>> felix Services x command output)
>>> * the services registered by the CXF bundle DSW on the consumer side
>>> * the data in your zookeeper directory instance (e.g. a zkCli> get
>>> /osgi/service_registry/your/package/yourService/xxx)
>>>
>>> Thanks,
>>>
>>> David
>>>
>>> 2009/9/4 Shulok, Thomas <th...@lmco.com>:
>>>> Hi David,
>>>>
>>>> I'm running with the clean option, so it should be flushing the cache.  When I get a chance, I'll construct a brain-dead simple example from scratch, though.  A couple questions in the meantime...
>>>>
>>>> Do I have all the right updated bundles? (listed in the prior post)
>>>>
>>>> Are you actually seeing the org.apache.cxf.ws.address property or the osgi.remote.configuration.pojo.address property.  I noticed you referenced the latter in a prior response, maybe there's an impedance mismatch between the old and new approaches?
>>>>
>>>> Any chance someone stepped on your commit?
>>>>
>>>> Thanks,
>>>> Thom
>>>>
>>>> -----Original Message-----
>>>> From: David Bosschaert [mailto:david.bosschaert@gmail.com]
>>>> Sent: Friday, September 04, 2009 4:46 AM
>>>> To: Shulok, Thomas
>>>> Subject: Re: Default endpoint via Zookeeper (D-OSGi) (repost to get
>>>> its own thread, sorry all)
>>>>
>>>> Hi Thomas,
>>>>
>>>> One thought I just had is that the old bundles might be cached by
>>>> the OSGi container. It might be worth starting with completely empty
>>>> containers from scratch.
>>>>
>>>> David
>>>>
>>>> 2009/9/4 David Bosschaert <da...@gmail.com>:
>>>>> Hi Thomas,
>>>>>
>>>>> Replying privately.
>>>>> Have you actually tried it out? I can look into it again, but it
>>>>> was definitely working for me.
>>>>>
>>>>> David
>>>>>
>>>>> 2009/9/4 Shulok, Thomas <th...@lmco.com>:
>>>>>> Closer, but not quite...
>>>>>>
>>>>>> The osgi.remote.endpoint.location does show up in the Zookeeper entry with the IP address properly externalized, but there is no org.apache.cxf.ws.address entry.  That appears to be what the client will bind against, not the enpoint location.
>>>>>>
>>>>>> It's also more than possible I didn't update all the requisite bundles.  I updated these:
>>>>>> cxf-dosgi-ri-dsw-cxf-1.1-SNAPSHOT.jar
>>>>>> cxf-dosgi-ri-discovery-local-1.1-SNAPSHOT.jar
>>>>>> cxf-dosgi-ri-discovery-distributed-zookeeper-wrapper-1.1-SNAPSHOT.
>>>>>> j ar cxf-dosgi-ri-discovery-distributed-1.1-SNAPSHOT.jar
>>>>>>
>>>>>> Missing any?
>>>>>>
>>>>>> Thanks,
>>>>>> Thom
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: David Bosschaert [mailto:david.bosschaert@gmail.com]
>>>>>> Sent: Friday, August 21, 2009 5:22 AM
>>>>>> To: users@cxf.apache.org
>>>>>> Subject: Re: Default endpoint via Zookeeper (D-OSGi) (repost to
>>>>>> get its own thread, sorry all)
>>>>>>
>>>>>> I've just committed a fix in r806505. It should be pushed to 1.1-SNAPSHOT in maven overnight.
>>>>>> Please let me know if it works for you.
>>>>>>
>>>>>> David
>>>>>>
>>>>>> 2009/8/21 David Bosschaert <da...@gmail.com>:
>>>>>>> Hi Thom,
>>>>>>>
>>>>>>> Just to let you know that using two machines I have reproduced your problem.
>>>>>>> I'm currently working on a fix.
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>> 2009/8/18 David Bosschaert <da...@gmail.com>:
>>>>>>>> Ah, right, and it did work for me because I was trying it out on
>>>>>>>> a single machine.
>>>>>>>> Let me go back and try to fix this issue.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>> 2009/8/13 Shulok, Thomas <th...@lmco.com>:
>>>>>>>>> Hi David,
>>>>>>>>>
>>>>>>>>> I don't think I explained the problem clearly enough.  The node does indeed get registered, but it's missing a property.  Instead of doing an ls on the entry, do a get.  Inside the entry, I think you'll see that it's missing the  org.apache.cxf.ws.address property.  Without that, the remote client will bind to localhost (itself) because it can't find the server.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Thom
>>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: David Bosschaert [mailto:david.bosschaert@gmail.com]
>>>>>>>>> Sent: Thursday, August 13, 2009 8:37 AM
>>>>>>>>> To: users@cxf.apache.org
>>>>>>>>> Subject: Re: Default endpoint via Zookeeper (D-OSGi) (repost to
>>>>>>>>> get its own thread, sorry all)
>>>>>>>>>
>>>>>>>>> That works for me too.
>>>>>>>>>
>>>>>>>>> I'm now using the following code:
>>>>>>>>>  Dictionary props = new Hashtable();
>>>>>>>>>
>>>>>>>>>  props.put("service.exported.interfaces", "*");
>>>>>>>>>
>>>>>>>>>  reg = bc.registerService(DisplayService.class.getName(), new
>>>>>>>>> DisplayServiceImpl(""), props);
>>>>>>>>>
>>>>>>>>> Its working fine for me. Could you give it a try?
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>> 2009/8/13 Shulok, Thomas <th...@lmco.com>:
>>>>>>>>>>
>>>>>>>>>> Hi David,
>>>>>>>>>>
>>>>>>>>>> Still one difference...I don't specify (the equivalent of)
>>>>>>>>>>
>>>>>>>>>>  props.put("service.exported.configs", "org.apache.cxf.ws");
>>>>>>>>>>
>>>>>>>>>> in the service decorator.  Maybe I've been leveraging some undocumented default behavior, but if you just specify the service.exported.interfaces property, the current D-OSGi implementation will assume you want the web service config and set it up for you.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Thom
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: David Bosschaert [mailto:david.bosschaert@gmail.com]
>>>>>>>>>> Sent: Thursday, August 13, 2009 3:11 AM
>>>>>>>>>> To: users@cxf.apache.org
>>>>>>>>>> Subject: Re: Default endpoint via Zookeeper (D-OSGi) (repost
>>>>>>>>>> to get its own thread, sorry all)
>>>>>>>>>>
>>>>>>>>>> Hi Thomas,
>>>>>>>>>>
>>>>>>>>>> I changed my copy of the Discovery demo to reflect your situation, and it seems to work for me.
>>>>>>>>>> In the file
>>>>>>>>>> http://svn.apache.org/repos/asf/cxf/dosgi/trunk/samples/discov
>>>>>>>>>> e
>>>>>>>>>> ry/i
>>>>>>>>>> mpl
>>>>>>>>>> /src/main/java/org/apache/cxf/dosgi/samples/discovery/impl/Act
>>>>>>>>>> i vato r.j ava I'm registering my services as follows:
>>>>>>>>>>  Dictionary props = new Hashtable();
>>>>>>>>>>
>>>>>>>>>>  props.put("service.exported.interfaces", "*");
>>>>>>>>>>  props.put("service.exported.configs", "org.apache.cxf.ws");
>>>>>>>>>>
>>>>>>>>>>  reg = bc.registerService(DisplayService.class.getName(), new DisplayServiceImpl(""), props); This registers the service under the default name in ZooKeeper, in this case the DefaultService interface name, so if I look in the location for org.apache.cxf.dosgi.samples.discovery.DisplayService in the ZooKeeper registry I can see my service there:
>>>>>>>>>>  zkCli> ls
>>>>>>>>>> /osgi/service_registry/org/apache/cxf/dosgi/samples/discovery/
>>>>>>>>>> D
>>>>>>>>>> ispl
>>>>>>>>>> ayS
>>>>>>>>>> ervice
>>>>>>>>>>
>>>>>>>>>> [10.2.4.18#9000##org#apache#cxf#dosgi#samples#discovery#Displa
>>>>>>>>>> y
>>>>>>>>>> Serv
>>>>>>>>>> ice
>>>>>>>>>> ]
>>>>>>>>>>
>>>>>>>>>> running the client that is part of this demo works, it can find the service that is exposed using the defaults.
>>>>>>>>>>
>>>>>>>>>> Is this what you're trying to do or am I missing something?
>>>>>>>>>>
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>> 2009/8/12 Shulok, Thomas <th...@lmco.com>:
>>>>>>>>>>> One bug leads to another... :-)
>>>>>>>>>>>
>>>>>>>>>>> Your fix puts the correct endpoint location into the Zookeeper node.  Works great, thanks!  But...
>>>>>>>>>>>
>>>>>>>>>>> If I don't specify org.apache.cxf.ws.address in the service decorator, I would expect it to default to http://localhost:9000/fully/qualified/ClassName as described in the reference guide (http://cxf.apache.org/distributed-osgi-reference.html) under service provider properties.  The default would then get transmorgrified into the actual IP of localhost just like the endpoint and inserted into the node.  Unfortunately, if I don't specify org.apache.cxf.ws.address, it doesn't show up in the Zookeeper node at all, which means the remote client thinks the web service address is localhost (i.e. bad).
>>>>>>>>>>>
>>>>>>>>>>> Any hope/plan to incorporate the default behavior with the Zookeeper integration?
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>> Thom
>>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: David Bosschaert [mailto:david.bosschaert@gmail.com]
>>>>>>>>>>> Sent: Monday, August 10, 2009 6:49 AM
>>>>>>>>>>> To: users@cxf.apache.org
>>>>>>>>>>> Subject: Re: Default endpoint via Zookeeper (D-OSGi) (repost
>>>>>>>>>>> to get its own thread, sorry all)
>>>>>>>>>>>
>>>>>>>>>>> Hi Thomas,
>>>>>>>>>>>
>>>>>>>>>>> Just to let you know I've just committed a fix for this to trunk (1.1-SNAPSHOT). The builds should publish a new snapshot of this tonight.
>>>>>>>>>>>
>>>>>>>>>>> With this fix you can set the org.apache.cxf.ws.address property to a localhost value, like this:
>>>>>>>>>>>  props.put("org.apache.cxf.ws.address",
>>>>>>>>>>> "http://localhost:9100/display");
>>>>>>>>>>>
>>>>>>>>>>> Once it's pushed into the zookeeper server, the localhost and
>>>>>>>>>>> 127.0.0.1 URLs are changed to contain the real IP address of the host, like this:
>>>>>>>>>>>
>>>>>>>>>>> Hope this works for you too.
>>>>>>>>>>> zkCli> get
>>>>>>>>>>> zkCli> /osgi/service_registry/org/apache/cxf/dosgi/samples/di
>>>>>>>>>>> zkCli> s cove ry/ D is playService/10.2.4.18#9100##display
>>>>>>>>>>> #
>>>>>>>>>>> #Mon Aug 10 14:37:47 BST 2009
>>>>>>>>>>> osgi.remote.endpoint.location=http\://10.2.4.18\:9100/display
>>>>>>>>>>> service.id=40
>>>>>>>>>>> objectClass=[Ljava.lang.String;@188f506
>>>>>>>>>>> osgi.remote.interfaces=*
>>>>>>>>>>> osgi.remote.configuration.pojo.address=http\://10.2.4.18\:910
>>>>>>>>>>> 0
>>>>>>>>>>> /dis
>>>>>>>>>>> pla
>>>>>>>>>>> y
>>>>>>>>>>> osgi.remote.configuration.type=pojo
>>>>>>>>>>> osgi.remote.endpoint.id=f5a3d0ef-b1f6-4fe4-b03f-33190ec28dab
>>>>>>>>>>> ...
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> David
>>>>>>>>>>>
>>>>>>>>>>> 2009/8/10 David Bosschaert <da...@gmail.com>:
>>>>>>>>>>>> Hi Thomas,
>>>>>>>>>>>>
>>>>>>>>>>>> Here are a couple of pointers.
>>>>>>>>>>>>
>>>>>>>>>>>> If you look at the Discovery demo, the Service
>>>>>>>>>>>> Implementation Activator, you can see that in this activator
>>>>>>>>>>>> it figures out the host name and sets that in the "osgi.remote.configuration.pojo.address"
>>>>>>>>>>>> property:
>>>>>>>>>>>> https://svn.apache.org/repos/asf/cxf/dosgi/trunk/samples/dis
>>>>>>>>>>>> c
>>>>>>>>>>>> over
>>>>>>>>>>>> y/i
>>>>>>>>>>>> m
>>>>>>>>>>>> p
>>>>>>>>>>>> l/src/main/java/org/apache/cxf/dosgi/samples/discovery/impl/Activator.
>>>>>>>>>>>> java So you could do something similar when setting the property.
>>>>>>>>>>>>
>>>>>>>>>>>> However, if you are setting the property from a static file,
>>>>>>>>>>>> this isn't really convenient. Therefore there is some
>>>>>>>>>>>> additional support for translating 'localhost' into the
>>>>>>>>>>>> machine name/IP addr when its registered in Discovery. If
>>>>>>>>>>>> you specify the host in your address property as
>>>>>>>>>>>> 'localhost', the zookeeper code will do a lookup on your
>>>>>>>>>>>> machine name and register it under that (or the IP address). So in my case I'm getting the following registration in ZooKeeper:
>>>>>>>>>>>> zkCli> ls
>>>>>>>>>>>> zkCli> /osgi/service_registry/org/apache/cxf/dosgi/samples/d
>>>>>>>>>>>> zkCli> i
>>>>>>>>>>>> zkCli> scov
>>>>>>>>>>>> zkCli> ery
>>>>>>>>>>>> zkCli> /
>>>>>>>>>>>> zkCli> D
>>>>>>>>>>>> zkCli> isplayService
>>>>>>>>>>>>  [10.2.4.18#9100##display]
>>>>>>>>>>>> So it translated 'localhost' into 10.2.4.18
>>>>>>>>>>>>
>>>>>>>>>>>> However, looking at the properties inside the registration,
>>>>>>>>>>>> I see that 'localhost' is still in there:
>>>>>>>>>>>> zkCli> get
>>>>>>>>>>>> zkCli> /osgi/service_registry/org/apache/cxf/dosgi/samples/d
>>>>>>>>>>>> zkCli> i scov ery / D isplayService/10.2.4.18#9100##display
>>>>>>>>>>>> #
>>>>>>>>>>>> #Mon Aug 10 10:50:22 BST 2009
>>>>>>>>>>>> osgi.remote.endpoint.location=http\://localhost\:9100/displa
>>>>>>>>>>>> y
>>>>>>>>>>>> service.id=40
>>>>>>>>>>>> objectClass=[Ljava.lang.String;@1292ba7
>>>>>>>>>>>> osgi.remote.interfaces=*
>>>>>>>>>>>> osgi.remote.configuration.pojo.address=http\://localhost\:91
>>>>>>>>>>>> 0
>>>>>>>>>>>> 0/di
>>>>>>>>>>>> spl
>>>>>>>>>>>> a
>>>>>>>>>>>> y
>>>>>>>>>>>> osgi.remote.configuration.type=pojo
>>>>>>>>>>>> osgi.remote.endpoint.id=464a4c2f-205c-4a06-a5e2-2f5127dc0894
>>>>>>>>>>>> ...
>>>>>>>>>>>>
>>>>>>>>>>>> So that's clearly a bug, I've filed
>>>>>>>>>>>> https://issues.apache.org/jira/browse/CXF-2385 and will try
>>>>>>>>>>>> to fix this over the coming few days.
>>>>>>>>>>>>
>>>>>>>>>>>> Hope this helps,
>>>>>>>>>>>>
>>>>>>>>>>>> David
>>>>>>>>>>>>
>>>>>>>>>>>> 2009/8/7 Shulok, Thomas <th...@lmco.com>:
>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Working with the Zookeeper discovery setup, and things are
>>>>>>>>>>>>> working as long as I explicitly specify
>>>>>>>>>>>>> org.apache.cxf.ws.address in my service decoration xml file.
>>>>>>>>>>>>> If I don't specify it, it will default to localhost. Not
>>>>>>>>>>>>> unreasonable, but the problem is, localhost will be the
>>>>>>>>>>>>> endpoint that gets registered with Zookeeper. So, when another box looks for the interface, Zookeeper will tell it that it is on localhost (i.e.
>>>>>>>>>>>>> the client itself), and that will fail. With this setup, do
>>>>>>>>>>>>> you have to explicitly specify the endpoint, or is there
>>>>>>>>>>>>> some other default behavior to leverage that will actually
>>>>>>>>>>>>> use the real IP of the registering localhost when it gets
>>>>>>>>>>>>> entered into Zookeeper so other clients can find it?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Many thanks,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thom
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>