You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@celix.apache.org by Björn Petri <bj...@sundevil.de> on 2013/09/15 10:02:01 UTC
Re: Remote service using shared mem
Hello,
I just created JIRA issue CELIX-81, including a patch which adds several
components allowing CELIX to use Shared Memory for remote services
(discovery_shm, remote_service_admin_shm and example_proxy_shm).
Besides, two new deployment targets are added, which can be used to
locally test the shared memory implementation using the already existing
example_service.
Netstring support (see JIRA issue CELIX-80 for according patch) is
required to get the this shm implementation working. Also note that
shared memory segments or semaphores might be left in the system and
needs to be removed manually (see ipcs, ipcrm) when CELIX does not
shutdown as expected.
Regards,
Bjoern
On 08/21/2013 09:09 PM, Alexander Broekhuis wrote:
> Hi,
>
>
>> Thanks for the suggestion, I think it is a useful option. I will propose
>> it.
>>
> I agree, having a shared memory transport in the Amdatu Remote Services
> would be a nice addition. Feel free to open a request for it and start some
> discussion on the Amdatu list! ;).
>
> First I'd like to make the current transport and discovery working between
> Java and C, but different transports in both languages would be a good
> thing.
>
Re: Remote service using shared mem
Posted by Gerrit Binnenmars <ge...@gmail.com>.
Hello Bjorn and Alexander,
Both of you, thanks for your work and cooperation to bring the remote
shm patch to Celix.
Greetings Gerrit
> Am 2013-12-05 20:25, schrieb Alexander Broekhuis:
>> Hi,
>>
>> 2013/12/5 Björn Petri <bj...@sundevil.de>
>>
>>>
>>> Hi Alexander,
>>>
>>> thanks for your feedback. Concerning your issues:
>>>
>>> * I am fine with replacing semtimedop again through semop, as the
>>> function
>>> calls are almost identical, I can do that right away.
>>>
>>
>> Would be great :). This doesn't introduce any problems?
>
> Not as long as all discovery bundles are working fine. As soon as one
> discovery bundle is not shutdown properly, the semtimedop still allows
> all other discovery bundles to shutdown.
>
> But as I do not expect the discovery bundle to 'not shutdown
> properly', replacing semtimedop with semop will not bring in any
> changed behavior.
>
> I've put a new patch in place, which has also been updated concerning
> the latest svn changes and the export of the RSA-Headers (CELIX-99).
>
> Regards,
> Bjoern
Re: Remote service using shared mem
Posted by Björn Petri <bj...@sundevil.de>.
Am 2013-12-05 20:25, schrieb Alexander Broekhuis:
> Hi,
>
> 2013/12/5 Björn Petri <bj...@sundevil.de>
>
>>
>> Hi Alexander,
>>
>> thanks for your feedback. Concerning your issues:
>>
>> * I am fine with replacing semtimedop again through semop, as the
>> function
>> calls are almost identical, I can do that right away.
>>
>
> Would be great :). This doesn't introduce any problems?
Not as long as all discovery bundles are working fine. As soon as one
discovery bundle is not shutdown properly, the semtimedop still allows
all other discovery bundles to shutdown.
But as I do not expect the discovery bundle to 'not shutdown properly',
replacing semtimedop with semop will not bring in any changed behavior.
I've put a new patch in place, which has also been updated concerning
the latest svn changes and the export of the RSA-Headers (CELIX-99).
Regards,
Bjoern
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
>
> I've pushed a fix for the proxy callback methods with the correct json
> handling. If all is ok now you should be able to use the calculator example
> as is. If there are any changes needed, please say so, so we can keep the
> HTTP and SHM RSA aligned.
>
I've done an additional commit which might be useful for the SHM RSA. I've
renamed the current HTTP implementation to a more descriptive directory
(remote_service_admin_http) and reused the existing directory to place all
common headers.
These headers include the public (service) and private headers needed to
implement an RSA.
Specific details can easily be placed in an implementation specific header
(see the http implementation as an example).
It might be useful to use these common headers for the SHM RSA as well,
there is no need anymore to have duplicates.
For both issues you can take a look at [1] and [2].
[1]: https://issues.apache.org/jira/browse/CELIX-98
[2]: https://issues.apache.org/jira/browse/CELIX-99
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
>
> * Same for the JSON format changes. As far as I remember, there is only a
>> line per function which specifies the JSON format.
>>
>
> I am already working on those locally, if you have a moment I'll push some
> changes so that the proxy/endpoint can be used by both the HTTP and the SHM
> RSA.
>
I've pushed a fix for the proxy callback methods with the correct json
handling. If all is ok now you should be able to use the calculator example
as is. If there are any changes needed, please say so, so we can keep the
HTTP and SHM RSA aligned.
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
Hi,
2013/12/5 Björn Petri <bj...@sundevil.de>
>
> Hi Alexander,
>
> thanks for your feedback. Concerning your issues:
>
> * I am fine with replacing semtimedop again through semop, as the function
> calls are almost identical, I can do that right away.
>
Would be great :). This doesn't introduce any problems?
>
> * Same for the JSON format changes. As far as I remember, there is only a
> line per function which specifies the JSON format.
>
I am already working on those locally, if you have a moment I'll push some
changes so that the proxy/endpoint can be used by both the HTTP and the SHM
RSA.
>
> I'll provide you with the changes asap. Let me know when something else
> should be done.
>
I'll do, thx for the quick repsonse!
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Björn Petri <bj...@sundevil.de>.
Hi Alexander,
thanks for your feedback. Concerning your issues:
* I am fine with replacing semtimedop again through semop, as the
function calls are almost identical, I can do that right away.
* Same for the JSON format changes. As far as I remember, there is only
a line per function which specifies the JSON format.
I'll provide you with the changes asap. Let me know when something else
should be done.
Regards,
Björn
Am 2013-12-05 17:17, schrieb Alexander Broekhuis:
> Hi Bjoern,
>
> I'm now trying to apply the patch and have some new issues, I've
> added some
> remarks in between:
>
> * It is created against an older revision, some small changes that
> easily
> can be fixed.
> For a final patch I'd like this to be solved, but isn't a big
> issue.
> * It now uses "semtimedop", which is a GLibC function (and not posix)
> so
> now it doesn't work on OSX.
> While I think we should work towards a broader support with these
> kind of
> things, I can live with the first version being Linux (GLibC) only.
> But
> this does mean I cannot test it at this moment.. So someone should
> definitely still do this (Pepijn?)
> * Calculator Endpoint/Proxy uses the older format for the JSON data
> (which
> is incompatible with the Amdatu RSA).
> This is a small change in the encoding/decoding, but isn't a big
> issue.
> * The SHM RSA uses setHandler/setCallback instead of
> setEndpointProperties.
> The existing HTTP RSA doesn't use these yet.
> At this moment this breaks the Calculator example when used with
> the HTTP
> RSA. Probably not that difficult to port back to the HTTP RSA. I
> prefer the
> newer method, so definitely in favour of applying those changes to
> the HTTP
> RSA as well.
>
> Some other remarks/updates not yet mentioned on this list:
> Yesterday we already talked about how to donate this code. Since this
> code
> introduces new bundles it makes the most sense to handle this using a
> Code
> Grant and IP Clearance.
> We already discussed that Pepijn is willing to do the paper work, so
> hopefully it al doesn't take that much time.
>
> So to summarize (in order):
> 1) We need a working set of bundles (at least tested on Linux,
> preferably
> on OSX as well). Existing code should still work as well.
> 2) After that a code grant with Issue number and MD5 sum needs to be
> filled
> in and filed.
> 3) Finally the IP Clearance should be handled.
>
> I have some time to do some work, so I can help out a bit. For now
> I'll
> work on getting the setHandler/setCallback method in the HTTP RSA as
> well,
> if that is in place, ideally you shouldn't need to update those for
> the SHM
> RSA.
>
> Anything I forgot or overlooking?
>
>
>
> 2013/12/4 Björn Petri <bj...@sundevil.de>
>
>>
>>
>> I updated the functionality of the discovery component to ensure
>> that all
>> threads perform its update routine, when a service was added/stopped
>> or the
>> discovery bundle is stopped.
>>
>> I would be happy if you find the time to give it a short check.
>>
>> Regards,
>> Bjoern
>>
>>
>>
>>
>>
>> Am 2013-10-03 14:07, schrieb Björn Petri:
>>
>> Hi Alexander,
>>>
>>> introducing the lock in the discovery_stop function was done with
>>> purpose. The unlocking and locking should enable all
>>> discovery-polling-threads to update their local list of services
>>> and
>>> in the case of the stopping discovery bundle it should allow the
>>> thread to leave it's loop (the loop control variable should be set
>>> to
>>> false before performing the unlock and lock). So, it should
>>> actually
>>> not wait on the join - I'll take a look at the code and check what
>>> is
>>> going on there.
>>>
>>> Regards,
>>> Björn
>>>
>>>
>>>
>>>
>>> Am 2013-10-03 11:59, schrieb Alexander Broekhuis:
>>>
>>>> Hi Bjoern,
>>>>
>>>> I have some problems with the discovery now. If I try to stop the
>>>> discovery
>>>> bundle, it tries to join the discovery thread, but somehow the
>>>> lock isn't
>>>> released and the thread doesn't exit.
>>>>
>>>> Looking in the code, the discovery_stop now has an unlock directly
>>>> followed
>>>> by a lock. This seems to be the problem. Did a small error sneak
>>>> in? Or
>>>> is
>>>> there some reason for the unlock/lock?
>>>>
>>>> I'll do some more testing later on this week, so I'll get back to
>>>> you :)
>>>>
>>>>
>>>> 2013/10/2 Björn Petri <bj...@sundevil.de>
>>>>
>>>>
>>>>> Alexander,
>>>>>
>>>>> I updated the rs shared memory implementation to
>>>>>
>>>>> (1) use System-V shared memory routines instead of ones from APR,
>>>>> (2) make use of your install_bundles macro,
>>>>> (3) perform some necessary cleanup when the exported service is
>>>>> stopped
>>>>> (This is also missing in the "standard"-rsa implementation). See
>>>>> remoteServiceAdmin_**removeExportedService function for details,
>>>>> (4) get rid of the linking of the example_proxy against the
>>>>> rsa_shm,
>>>>> (5) and fixed some minor bugs.
>>>>>
>>>>> looking forward to your feedback,
>>>>> Bjoern
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Am 2013-09-27 16:11, schrieb Alexander Broekhuis:
>>>>>
>>>>> 2013/9/27 Björn Petri <bj...@sundevil.de>
>>>>>
>>>>>>
>>>>>> On 09/18/2013 06:29 PM, Alexander Broekhuis wrote:
>>>>>>
>>>>>>>
>>>>>>> I checked it, and there are some small leftovers in the code.
>>>>>>> The
>>>>>>> proxy
>>>>>>>
>>>>>>>> still includes the rsa and also links with the library.
>>>>>>>>
>>>>>>>>
>>>>>>>> How do you want to get rid of this dependency? The proxy
>>>>>>>> needs to
>>>>>>> include
>>>>>>> definition of the remote_proxy_service as well as the rsa does.
>>>>>>> Do I
>>>>>>> miss
>>>>>>> something?
>>>>>>>
>>>>>>>
>>>>>>> Since the proxy now uses the function pointer and only the
>>>>>>> service
>>>>>> struct
>>>>>> of the RSA it only needs the header files (as far as I can
>>>>>> tell). Also,
>>>>>> with the concept of bundles, and Celix not supporting
>>>>>> code-sharing at
>>>>>> this
>>>>>> point, bundles (actually the library in it) should never link to
>>>>>> a
>>>>>> library
>>>>>> of any other bundle.
>>>>>> Libraries are loaded locally, so the symbols aren't shared with
>>>>>> others. So
>>>>>> this means that if you link with another bundle, at runtime
>>>>>> there will
>>>>>> be
>>>>>> unresolved references. Hence the need for services (structs with
>>>>>> function
>>>>>> pointers) etc.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
Hi Bjoern,
I'm now trying to apply the patch and have some new issues, I've added some
remarks in between:
* It is created against an older revision, some small changes that easily
can be fixed.
For a final patch I'd like this to be solved, but isn't a big issue.
* It now uses "semtimedop", which is a GLibC function (and not posix) so
now it doesn't work on OSX.
While I think we should work towards a broader support with these kind of
things, I can live with the first version being Linux (GLibC) only. But
this does mean I cannot test it at this moment.. So someone should
definitely still do this (Pepijn?)
* Calculator Endpoint/Proxy uses the older format for the JSON data (which
is incompatible with the Amdatu RSA).
This is a small change in the encoding/decoding, but isn't a big issue.
* The SHM RSA uses setHandler/setCallback instead of setEndpointProperties.
The existing HTTP RSA doesn't use these yet.
At this moment this breaks the Calculator example when used with the HTTP
RSA. Probably not that difficult to port back to the HTTP RSA. I prefer the
newer method, so definitely in favour of applying those changes to the HTTP
RSA as well.
Some other remarks/updates not yet mentioned on this list:
Yesterday we already talked about how to donate this code. Since this code
introduces new bundles it makes the most sense to handle this using a Code
Grant and IP Clearance.
We already discussed that Pepijn is willing to do the paper work, so
hopefully it al doesn't take that much time.
So to summarize (in order):
1) We need a working set of bundles (at least tested on Linux, preferably
on OSX as well). Existing code should still work as well.
2) After that a code grant with Issue number and MD5 sum needs to be filled
in and filed.
3) Finally the IP Clearance should be handled.
I have some time to do some work, so I can help out a bit. For now I'll
work on getting the setHandler/setCallback method in the HTTP RSA as well,
if that is in place, ideally you shouldn't need to update those for the SHM
RSA.
Anything I forgot or overlooking?
2013/12/4 Björn Petri <bj...@sundevil.de>
>
>
> I updated the functionality of the discovery component to ensure that all
> threads perform its update routine, when a service was added/stopped or the
> discovery bundle is stopped.
>
> I would be happy if you find the time to give it a short check.
>
> Regards,
> Bjoern
>
>
>
>
>
> Am 2013-10-03 14:07, schrieb Björn Petri:
>
> Hi Alexander,
>>
>> introducing the lock in the discovery_stop function was done with
>> purpose. The unlocking and locking should enable all
>> discovery-polling-threads to update their local list of services and
>> in the case of the stopping discovery bundle it should allow the
>> thread to leave it's loop (the loop control variable should be set to
>> false before performing the unlock and lock). So, it should actually
>> not wait on the join - I'll take a look at the code and check what is
>> going on there.
>>
>> Regards,
>> Björn
>>
>>
>>
>>
>> Am 2013-10-03 11:59, schrieb Alexander Broekhuis:
>>
>>> Hi Bjoern,
>>>
>>> I have some problems with the discovery now. If I try to stop the
>>> discovery
>>> bundle, it tries to join the discovery thread, but somehow the lock isn't
>>> released and the thread doesn't exit.
>>>
>>> Looking in the code, the discovery_stop now has an unlock directly
>>> followed
>>> by a lock. This seems to be the problem. Did a small error sneak in? Or
>>> is
>>> there some reason for the unlock/lock?
>>>
>>> I'll do some more testing later on this week, so I'll get back to you :)
>>>
>>>
>>> 2013/10/2 Björn Petri <bj...@sundevil.de>
>>>
>>>
>>>> Alexander,
>>>>
>>>> I updated the rs shared memory implementation to
>>>>
>>>> (1) use System-V shared memory routines instead of ones from APR,
>>>> (2) make use of your install_bundles macro,
>>>> (3) perform some necessary cleanup when the exported service is stopped
>>>> (This is also missing in the "standard"-rsa implementation). See
>>>> remoteServiceAdmin_**removeExportedService function for details,
>>>> (4) get rid of the linking of the example_proxy against the rsa_shm,
>>>> (5) and fixed some minor bugs.
>>>>
>>>> looking forward to your feedback,
>>>> Bjoern
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Am 2013-09-27 16:11, schrieb Alexander Broekhuis:
>>>>
>>>> 2013/9/27 Björn Petri <bj...@sundevil.de>
>>>>
>>>>>
>>>>> On 09/18/2013 06:29 PM, Alexander Broekhuis wrote:
>>>>>
>>>>>>
>>>>>> I checked it, and there are some small leftovers in the code. The
>>>>>> proxy
>>>>>>
>>>>>>> still includes the rsa and also links with the library.
>>>>>>>
>>>>>>>
>>>>>>> How do you want to get rid of this dependency? The proxy needs to
>>>>>> include
>>>>>> definition of the remote_proxy_service as well as the rsa does. Do I
>>>>>> miss
>>>>>> something?
>>>>>>
>>>>>>
>>>>>> Since the proxy now uses the function pointer and only the service
>>>>> struct
>>>>> of the RSA it only needs the header files (as far as I can tell). Also,
>>>>> with the concept of bundles, and Celix not supporting code-sharing at
>>>>> this
>>>>> point, bundles (actually the library in it) should never link to a
>>>>> library
>>>>> of any other bundle.
>>>>> Libraries are loaded locally, so the symbols aren't shared with
>>>>> others. So
>>>>> this means that if you link with another bundle, at runtime there will
>>>>> be
>>>>> unresolved references. Hence the need for services (structs with
>>>>> function
>>>>> pointers) etc.
>>>>>
>>>>>
>>>>
>>>>
>
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Björn Petri <bj...@sundevil.de>.
I updated the functionality of the discovery component to ensure that
all threads perform its update routine, when a service was added/stopped
or the discovery bundle is stopped.
I would be happy if you find the time to give it a short check.
Regards,
Bjoern
Am 2013-10-03 14:07, schrieb Björn Petri:
> Hi Alexander,
>
> introducing the lock in the discovery_stop function was done with
> purpose. The unlocking and locking should enable all
> discovery-polling-threads to update their local list of services and
> in the case of the stopping discovery bundle it should allow the
> thread to leave it's loop (the loop control variable should be set to
> false before performing the unlock and lock). So, it should actually
> not wait on the join - I'll take a look at the code and check what is
> going on there.
>
> Regards,
> Björn
>
>
>
>
> Am 2013-10-03 11:59, schrieb Alexander Broekhuis:
>> Hi Bjoern,
>>
>> I have some problems with the discovery now. If I try to stop the
>> discovery
>> bundle, it tries to join the discovery thread, but somehow the lock
>> isn't
>> released and the thread doesn't exit.
>>
>> Looking in the code, the discovery_stop now has an unlock directly
>> followed
>> by a lock. This seems to be the problem. Did a small error sneak in?
>> Or is
>> there some reason for the unlock/lock?
>>
>> I'll do some more testing later on this week, so I'll get back to
>> you :)
>>
>>
>> 2013/10/2 Björn Petri <bj...@sundevil.de>
>>
>>>
>>> Alexander,
>>>
>>> I updated the rs shared memory implementation to
>>>
>>> (1) use System-V shared memory routines instead of ones from APR,
>>> (2) make use of your install_bundles macro,
>>> (3) perform some necessary cleanup when the exported service is
>>> stopped
>>> (This is also missing in the "standard"-rsa implementation). See
>>> remoteServiceAdmin_**removeExportedService function for details,
>>> (4) get rid of the linking of the example_proxy against the
>>> rsa_shm,
>>> (5) and fixed some minor bugs.
>>>
>>> looking forward to your feedback,
>>> Bjoern
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Am 2013-09-27 16:11, schrieb Alexander Broekhuis:
>>>
>>> 2013/9/27 Björn Petri <bj...@sundevil.de>
>>>>
>>>> On 09/18/2013 06:29 PM, Alexander Broekhuis wrote:
>>>>>
>>>>> I checked it, and there are some small leftovers in the code.
>>>>> The proxy
>>>>>> still includes the rsa and also links with the library.
>>>>>>
>>>>>>
>>>>> How do you want to get rid of this dependency? The proxy needs to
>>>>> include
>>>>> definition of the remote_proxy_service as well as the rsa does.
>>>>> Do I miss
>>>>> something?
>>>>>
>>>>>
>>>> Since the proxy now uses the function pointer and only the service
>>>> struct
>>>> of the RSA it only needs the header files (as far as I can tell).
>>>> Also,
>>>> with the concept of bundles, and Celix not supporting code-sharing
>>>> at this
>>>> point, bundles (actually the library in it) should never link to a
>>>> library
>>>> of any other bundle.
>>>> Libraries are loaded locally, so the symbols aren't shared with
>>>> others. So
>>>> this means that if you link with another bundle, at runtime there
>>>> will be
>>>> unresolved references. Hence the need for services (structs with
>>>> function
>>>> pointers) etc.
>>>>
>>>
>>>
Re: Remote service using shared mem
Posted by Björn Petri <bj...@sundevil.de>.
Hi Alexander,
introducing the lock in the discovery_stop function was done with
purpose. The unlocking and locking should enable all
discovery-polling-threads to update their local list of services and in
the case of the stopping discovery bundle it should allow the thread to
leave it's loop (the loop control variable should be set to false before
performing the unlock and lock). So, it should actually not wait on the
join - I'll take a look at the code and check what is going on there.
Regards,
Björn
Am 2013-10-03 11:59, schrieb Alexander Broekhuis:
> Hi Bjoern,
>
> I have some problems with the discovery now. If I try to stop the
> discovery
> bundle, it tries to join the discovery thread, but somehow the lock
> isn't
> released and the thread doesn't exit.
>
> Looking in the code, the discovery_stop now has an unlock directly
> followed
> by a lock. This seems to be the problem. Did a small error sneak in?
> Or is
> there some reason for the unlock/lock?
>
> I'll do some more testing later on this week, so I'll get back to you
> :)
>
>
> 2013/10/2 Björn Petri <bj...@sundevil.de>
>
>>
>> Alexander,
>>
>> I updated the rs shared memory implementation to
>>
>> (1) use System-V shared memory routines instead of ones from APR,
>> (2) make use of your install_bundles macro,
>> (3) perform some necessary cleanup when the exported service is
>> stopped
>> (This is also missing in the "standard"-rsa implementation). See
>> remoteServiceAdmin_**removeExportedService function for details,
>> (4) get rid of the linking of the example_proxy against the rsa_shm,
>> (5) and fixed some minor bugs.
>>
>> looking forward to your feedback,
>> Bjoern
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Am 2013-09-27 16:11, schrieb Alexander Broekhuis:
>>
>> 2013/9/27 Björn Petri <bj...@sundevil.de>
>>>
>>> On 09/18/2013 06:29 PM, Alexander Broekhuis wrote:
>>>>
>>>> I checked it, and there are some small leftovers in the code. The
>>>> proxy
>>>>> still includes the rsa and also links with the library.
>>>>>
>>>>>
>>>> How do you want to get rid of this dependency? The proxy needs to
>>>> include
>>>> definition of the remote_proxy_service as well as the rsa does. Do
>>>> I miss
>>>> something?
>>>>
>>>>
>>> Since the proxy now uses the function pointer and only the service
>>> struct
>>> of the RSA it only needs the header files (as far as I can tell).
>>> Also,
>>> with the concept of bundles, and Celix not supporting code-sharing
>>> at this
>>> point, bundles (actually the library in it) should never link to a
>>> library
>>> of any other bundle.
>>> Libraries are loaded locally, so the symbols aren't shared with
>>> others. So
>>> this means that if you link with another bundle, at runtime there
>>> will be
>>> unresolved references. Hence the need for services (structs with
>>> function
>>> pointers) etc.
>>>
>>
>>
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
Hi Bjoern,
I have some problems with the discovery now. If I try to stop the discovery
bundle, it tries to join the discovery thread, but somehow the lock isn't
released and the thread doesn't exit.
Looking in the code, the discovery_stop now has an unlock directly followed
by a lock. This seems to be the problem. Did a small error sneak in? Or is
there some reason for the unlock/lock?
I'll do some more testing later on this week, so I'll get back to you :)
2013/10/2 Björn Petri <bj...@sundevil.de>
>
> Alexander,
>
> I updated the rs shared memory implementation to
>
> (1) use System-V shared memory routines instead of ones from APR,
> (2) make use of your install_bundles macro,
> (3) perform some necessary cleanup when the exported service is stopped
> (This is also missing in the "standard"-rsa implementation). See
> remoteServiceAdmin_**removeExportedService function for details,
> (4) get rid of the linking of the example_proxy against the rsa_shm,
> (5) and fixed some minor bugs.
>
> looking forward to your feedback,
> Bjoern
>
>
>
>
>
>
>
>
>
> Am 2013-09-27 16:11, schrieb Alexander Broekhuis:
>
> 2013/9/27 Björn Petri <bj...@sundevil.de>
>>
>> On 09/18/2013 06:29 PM, Alexander Broekhuis wrote:
>>>
>>> I checked it, and there are some small leftovers in the code. The proxy
>>>> still includes the rsa and also links with the library.
>>>>
>>>>
>>> How do you want to get rid of this dependency? The proxy needs to include
>>> definition of the remote_proxy_service as well as the rsa does. Do I miss
>>> something?
>>>
>>>
>> Since the proxy now uses the function pointer and only the service struct
>> of the RSA it only needs the header files (as far as I can tell). Also,
>> with the concept of bundles, and Celix not supporting code-sharing at this
>> point, bundles (actually the library in it) should never link to a library
>> of any other bundle.
>> Libraries are loaded locally, so the symbols aren't shared with others. So
>> this means that if you link with another bundle, at runtime there will be
>> unresolved references. Hence the need for services (structs with function
>> pointers) etc.
>>
>
>
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Björn Petri <bj...@sundevil.de>.
Alexander,
I updated the rs shared memory implementation to
(1) use System-V shared memory routines instead of ones from APR,
(2) make use of your install_bundles macro,
(3) perform some necessary cleanup when the exported service is stopped
(This is also missing in the "standard"-rsa implementation). See
remoteServiceAdmin_removeExportedService function for details,
(4) get rid of the linking of the example_proxy against the rsa_shm,
(5) and fixed some minor bugs.
looking forward to your feedback,
Bjoern
Am 2013-09-27 16:11, schrieb Alexander Broekhuis:
> 2013/9/27 Björn Petri <bj...@sundevil.de>
>
>> On 09/18/2013 06:29 PM, Alexander Broekhuis wrote:
>>
>>> I checked it, and there are some small leftovers in the code. The
>>> proxy
>>> still includes the rsa and also links with the library.
>>>
>>
>> How do you want to get rid of this dependency? The proxy needs to
>> include
>> definition of the remote_proxy_service as well as the rsa does. Do I
>> miss
>> something?
>>
>
> Since the proxy now uses the function pointer and only the service
> struct
> of the RSA it only needs the header files (as far as I can tell).
> Also,
> with the concept of bundles, and Celix not supporting code-sharing at
> this
> point, bundles (actually the library in it) should never link to a
> library
> of any other bundle.
> Libraries are loaded locally, so the symbols aren't shared with
> others. So
> this means that if you link with another bundle, at runtime there
> will be
> unresolved references. Hence the need for services (structs with
> function
> pointers) etc.
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
2013/9/27 Björn Petri <bj...@sundevil.de>
> On 09/18/2013 06:29 PM, Alexander Broekhuis wrote:
>
>> I checked it, and there are some small leftovers in the code. The proxy
>> still includes the rsa and also links with the library.
>>
>
> How do you want to get rid of this dependency? The proxy needs to include
> definition of the remote_proxy_service as well as the rsa does. Do I miss
> something?
>
Since the proxy now uses the function pointer and only the service struct
of the RSA it only needs the header files (as far as I can tell). Also,
with the concept of bundles, and Celix not supporting code-sharing at this
point, bundles (actually the library in it) should never link to a library
of any other bundle.
Libraries are loaded locally, so the symbols aren't shared with others. So
this means that if you link with another bundle, at runtime there will be
unresolved references. Hence the need for services (structs with function
pointers) etc.
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Björn Petri <bj...@sundevil.de>.
On 09/18/2013 06:29 PM, Alexander Broekhuis wrote:
> I checked it, and there are some small leftovers in the code. The
> proxy still includes the rsa and also links with the library.
How do you want to get rid of this dependency? The proxy needs to
include definition of the remote_proxy_service as well as the rsa does.
Do I miss something?
- Bjoern
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
Hi
2013/9/18 Björn Petri <bj...@sundevil.de>
>
> Hi Alexander,
>
> Proposed changes are included and a new patch is available within
> CELIX-81.
I checked it, and there are some small leftovers in the code. The proxy
still includes the rsa and also links with the library.
> But I stumbled over another problem: when importing or exporting a
> service, the according reference is saved in a hashMaps within the RSA and
> the topology manager. But there is no functionality yet to remove those.
> This leads to a segfault (at least in the RSA_SHM) when issuing a
> (add/sub/sqrt-) command after re-starting the example-service. A possible
> solution has been already discussed with Pepijn and I'll try to provide an
> according patch within the next weeks.
>
If you need some help and/or thoughts, feel free to share the solution!
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Björn Petri <bj...@sundevil.de>.
Hi Alexander,
Proposed changes are included and a new patch is available within
CELIX-81. But I stumbled over another problem: when importing or
exporting a service, the according reference is saved in a hashMaps
within the RSA and the topology manager. But there is no functionality
yet to remove those. This leads to a segfault (at least in the RSA_SHM)
when issuing a (add/sub/sqrt-) command after re-starting the
example-service. A possible solution has been already discussed with
Pepijn and I'll try to provide an according patch within the next weeks.
Regards,
Bjoern
Am 2013-09-17 19:50, schrieb Alexander Broekhuis:
> I did some more testing/debugging and have some changes :)
>
> 1) In remote_services/CMakeLists.txt you re-enabled the inclusion of
> the
> utils directory.
>
> 2) The thread in Discovery and the RSA itself do not have a return
> value,
> also the thread_exit is only called if a precondition is met,
> otherwise the
> method just return without an exit. This can be solved by always
> calling
> apr_thread_exit and returning NULL at the end.
>
> 3) As mentioned in the other mail, the RSA needs to be updated for
> CELIX-82. Simple fix, take a look at the other RSA for the code :).
>
> 4) When creating a thread in the RSA you add a NULL pointer to the
> hashmap
> and only later on create an actual pointer. This results in a NULL
> pointer
> in the hashmap and the pointer to the thread being "lost". Simply
> putting
> the thread pointer in the hashmap after thread creation is enough to
> fix
> this. This one was also the reason for the segfault when the
> framework
> stops and not the use of shmem etc.
>
> Ps: I can make those changes and add the code, but if you prefer to
> verify
> my remarks and make a new patch I can wait a bit.
>
> Pps: The code looks good! And together with a few simple fixes Pepijn
> committed today I think not a lot of extra work is needed before
> committing
> this.
>
>
>
>
>
> 2013/9/17 Alexander Broekhuis <a....@gmail.com>
>
>> Hi,
>>
>> Unrelated to the previous problems, Pepijn today fixed issue
>> CELIX-82 [1].
>> This also requires an update in the SHM code. Could you also port
>> that
>> change to your code?
>>
>> 2013/9/17 Björn Petri <bj...@sundevil.de>
>>
>>>
>>> Hi Alexander,
>>>
>>> I already updated the issues we spoke about, so stopping the
>>> example-service does not segfault any more.
>>
>>
>> I tried a bit with the new patch, and still have a problem. I
>> haven't
>> tested it in detail, but during the stop of the RSA something
>> unexpected
>> happens.
>>
>> The problem reported is: celix(82209,0x10ff7a180) malloc: *** error
>> for
>> object 0x7f9ff48dda00: pointer being freed was not allocated
>> *** set a breakpoint in malloc_error_break to debug
>>
>> A reason for such errors can be the use of pools and still freeing
>> pointers. If a pointer which is in a pool is being freed, during the
>> apr
>> shutdown it is freed again.
>> In this case I suspect it has to do with the use of APR for some
>> shmem
>> functions, while other parts are done using standard (low level) api
>> calls.
>>
>>
>> [1]: https://issues.apache.org/jira/browse/CELIX-82
>>
>>
>> --
>> Met vriendelijke groet,
>>
>> Alexander Broekhuis
>>
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
Hi Gerrit,
Thanks for the reply, but lets keep the list communication in English
please ;). One new patch would be great!
Regarding the code:
I have found another (the last) issue as well. In the example_proxy_shm a
direct method of the RSA is used for sending data. Since the proxy and the
rsa are in 2 bundles this doesn't work on a mac. Funny thing is, there is
already some code for setting a callback method. Simply enabling this code
made everything work for me...
Is there a reason why it isn't used?
2013/9/17 Gerrit Binnenmars <ge...@gmail.com>
> Op 17-9-2013 19:50, Alexander Broekhuis schreef:
>
> I did some more testing/debugging and have some changes :)
>>
>> 1) In remote_services/CMakeLists.txt you re-enabled the inclusion of the
>> utils directory.
>>
>> 2) The thread in Discovery and the RSA itself do not have a return value,
>> also the thread_exit is only called if a precondition is met, otherwise
>> the
>> method just return without an exit. This can be solved by always calling
>> apr_thread_exit and returning NULL at the end.
>>
>> 3) As mentioned in the other mail, the RSA needs to be updated for
>> CELIX-82. Simple fix, take a look at the other RSA for the code :).
>>
>> 4) When creating a thread in the RSA you add a NULL pointer to the hashmap
>> and only later on create an actual pointer. This results in a NULL pointer
>> in the hashmap and the pointer to the thread being "lost". Simply putting
>> the thread pointer in the hashmap after thread creation is enough to fix
>> this. This one was also the reason for the segfault when the framework
>> stops and not the use of shmem etc.
>>
>> Ps: I can make those changes and add the code, but if you prefer to verify
>> my remarks and make a new patch I can wait a bit.
>>
>> Pps: The code looks good! And together with a few simple fixes Pepijn
>> committed today I think not a lot of extra work is needed before
>> committing
>> this.
>>
>>
>>
>>
>>
>> 2013/9/17 Alexander Broekhuis <a....@gmail.com>
>>
>> Hi,
>>>
>>> Unrelated to the previous problems, Pepijn today fixed issue CELIX-82
>>> [1].
>>> This also requires an update in the SHM code. Could you also port that
>>> change to your code?
>>>
>>> 2013/9/17 Björn Petri <bj...@sundevil.de>
>>>
>>> Hi Alexander,
>>>>
>>>> I already updated the issues we spoke about, so stopping the
>>>> example-service does not segfault any more.
>>>>
>>>
>>> I tried a bit with the new patch, and still have a problem. I haven't
>>> tested it in detail, but during the stop of the RSA something unexpected
>>> happens.
>>>
>>> The problem reported is: celix(82209,0x10ff7a180) malloc: *** error for
>>> object 0x7f9ff48dda00: pointer being freed was not allocated
>>> *** set a breakpoint in malloc_error_break to debug
>>>
>>> A reason for such errors can be the use of pools and still freeing
>>> pointers. If a pointer which is in a pool is being freed, during the apr
>>> shutdown it is freed again.
>>> In this case I suspect it has to do with the use of APR for some shmem
>>> functions, while other parts are done using standard (low level) api
>>> calls.
>>>
>>>
>>> [1]: https://issues.apache.org/**jira/browse/CELIX-82<https://issues.apache.org/jira/browse/CELIX-82>
>>>
>>>
>>> --
>>> Met vriendelijke groet,
>>>
>>> Alexander Broekhuis
>>>
>>>
>>
>> Hallo Alexander,
>
> Bedankt voor de snelle review. Het handigste is het denk ik als we 1
> nieuwe patch maken voor alles.
> Zal ik morgen vragen aan Bjorn.
>
> Groeten Gerrit
>
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Gerrit Binnenmars <ge...@gmail.com>.
Op 17-9-2013 19:50, Alexander Broekhuis schreef:
> I did some more testing/debugging and have some changes :)
>
> 1) In remote_services/CMakeLists.txt you re-enabled the inclusion of the
> utils directory.
>
> 2) The thread in Discovery and the RSA itself do not have a return value,
> also the thread_exit is only called if a precondition is met, otherwise the
> method just return without an exit. This can be solved by always calling
> apr_thread_exit and returning NULL at the end.
>
> 3) As mentioned in the other mail, the RSA needs to be updated for
> CELIX-82. Simple fix, take a look at the other RSA for the code :).
>
> 4) When creating a thread in the RSA you add a NULL pointer to the hashmap
> and only later on create an actual pointer. This results in a NULL pointer
> in the hashmap and the pointer to the thread being "lost". Simply putting
> the thread pointer in the hashmap after thread creation is enough to fix
> this. This one was also the reason for the segfault when the framework
> stops and not the use of shmem etc.
>
> Ps: I can make those changes and add the code, but if you prefer to verify
> my remarks and make a new patch I can wait a bit.
>
> Pps: The code looks good! And together with a few simple fixes Pepijn
> committed today I think not a lot of extra work is needed before committing
> this.
>
>
>
>
>
> 2013/9/17 Alexander Broekhuis <a....@gmail.com>
>
>> Hi,
>>
>> Unrelated to the previous problems, Pepijn today fixed issue CELIX-82 [1].
>> This also requires an update in the SHM code. Could you also port that
>> change to your code?
>>
>> 2013/9/17 Björn Petri <bj...@sundevil.de>
>>
>>> Hi Alexander,
>>>
>>> I already updated the issues we spoke about, so stopping the
>>> example-service does not segfault any more.
>>
>> I tried a bit with the new patch, and still have a problem. I haven't
>> tested it in detail, but during the stop of the RSA something unexpected
>> happens.
>>
>> The problem reported is: celix(82209,0x10ff7a180) malloc: *** error for
>> object 0x7f9ff48dda00: pointer being freed was not allocated
>> *** set a breakpoint in malloc_error_break to debug
>>
>> A reason for such errors can be the use of pools and still freeing
>> pointers. If a pointer which is in a pool is being freed, during the apr
>> shutdown it is freed again.
>> In this case I suspect it has to do with the use of APR for some shmem
>> functions, while other parts are done using standard (low level) api calls.
>>
>>
>> [1]: https://issues.apache.org/jira/browse/CELIX-82
>>
>>
>> --
>> Met vriendelijke groet,
>>
>> Alexander Broekhuis
>>
>
>
Hallo Alexander,
Bedankt voor de snelle review. Het handigste is het denk ik als we 1
nieuwe patch maken voor alles.
Zal ik morgen vragen aan Bjorn.
Groeten Gerrit
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
I did some more testing/debugging and have some changes :)
1) In remote_services/CMakeLists.txt you re-enabled the inclusion of the
utils directory.
2) The thread in Discovery and the RSA itself do not have a return value,
also the thread_exit is only called if a precondition is met, otherwise the
method just return without an exit. This can be solved by always calling
apr_thread_exit and returning NULL at the end.
3) As mentioned in the other mail, the RSA needs to be updated for
CELIX-82. Simple fix, take a look at the other RSA for the code :).
4) When creating a thread in the RSA you add a NULL pointer to the hashmap
and only later on create an actual pointer. This results in a NULL pointer
in the hashmap and the pointer to the thread being "lost". Simply putting
the thread pointer in the hashmap after thread creation is enough to fix
this. This one was also the reason for the segfault when the framework
stops and not the use of shmem etc.
Ps: I can make those changes and add the code, but if you prefer to verify
my remarks and make a new patch I can wait a bit.
Pps: The code looks good! And together with a few simple fixes Pepijn
committed today I think not a lot of extra work is needed before committing
this.
2013/9/17 Alexander Broekhuis <a....@gmail.com>
> Hi,
>
> Unrelated to the previous problems, Pepijn today fixed issue CELIX-82 [1].
> This also requires an update in the SHM code. Could you also port that
> change to your code?
>
> 2013/9/17 Björn Petri <bj...@sundevil.de>
>
>>
>> Hi Alexander,
>>
>> I already updated the issues we spoke about, so stopping the
>> example-service does not segfault any more.
>
>
> I tried a bit with the new patch, and still have a problem. I haven't
> tested it in detail, but during the stop of the RSA something unexpected
> happens.
>
> The problem reported is: celix(82209,0x10ff7a180) malloc: *** error for
> object 0x7f9ff48dda00: pointer being freed was not allocated
> *** set a breakpoint in malloc_error_break to debug
>
> A reason for such errors can be the use of pools and still freeing
> pointers. If a pointer which is in a pool is being freed, during the apr
> shutdown it is freed again.
> In this case I suspect it has to do with the use of APR for some shmem
> functions, while other parts are done using standard (low level) api calls.
>
>
> [1]: https://issues.apache.org/jira/browse/CELIX-82
>
>
> --
> Met vriendelijke groet,
>
> Alexander Broekhuis
>
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
Hi,
Unrelated to the previous problems, Pepijn today fixed issue CELIX-82 [1].
This also requires an update in the SHM code. Could you also port that
change to your code?
2013/9/17 Björn Petri <bj...@sundevil.de>
>
> Hi Alexander,
>
> I already updated the issues we spoke about, so stopping the
> example-service does not segfault any more.
I tried a bit with the new patch, and still have a problem. I haven't
tested it in detail, but during the stop of the RSA something unexpected
happens.
The problem reported is: celix(82209,0x10ff7a180) malloc: *** error for
object 0x7f9ff48dda00: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
A reason for such errors can be the use of pools and still freeing
pointers. If a pointer which is in a pool is being freed, during the apr
shutdown it is freed again.
In this case I suspect it has to do with the use of APR for some shmem
functions, while other parts are done using standard (low level) api calls.
[1]: https://issues.apache.org/jira/browse/CELIX-82
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Björn Petri <bj...@sundevil.de>.
Hi Alexander,
I already updated the issues we spoke about, so stopping the
example-service does not segfault any more. Besides, I will also perform
some more testing and see what else needs to be changed.
Greetings,
Bjoern
On 09/16/2013 10:13 PM, Alexander Broekhuis wrote:
> 2013/9/16 Björn Petri <bj...@sundevil.de>
>
>> I agree, Pepijn told me that Erik Jasman has already done quite a lot of
>> work to achieve this.
>> Maybe we could contine with his approach?!
>
> Yeah, I'd like to see that code. Not sure if Erik has already shared
> something.
>
>>
>> When testing "remote-services-shm" and stopping the "example" bundle, the
>>> discovery tries to deregister an entry, this results in a call to register
>>> which somehow fails with an segfault. Maybe I am missing something, but
>>> there is no removal anywhere. How is this supposed to work? Btw, before
>>> the
>>> segfault the following error is printed:
>>> DISCOVERY: Endpoint for example, with filter "(null)" removed
>>> DISCOVERY : discovery_registerSHMService : encoding data from HashMap
>>> failed
>>>
>> As the registered services are held as a hashmap (based on netstring) in
>> the shared memory, the plan was to do a remove instead of a put if the
>> value is set to NULL. Probably it is a good idea to rename the
>> discovery_registerSHMService when I use it for registering and
>> deregistering.
>
> While at some points I get that some method like this might make sense... I
> always prefer code readability. Using a register method to do an
> unregister... Just doesn't make sense to me. So please do make a
> unregister/deregister method for the deregister use case. Code wise you'd
> end up with almost the same code, but there is no NULL check and 2 flows in
> one method.
>
>
>> I have problems getting the code to run on my Mac. I run into several
>>
>>> segfaults, so this needs some more testing before I can commit it.
>>>
>> I will take care about the issues mentioned above and provide an updated
>> patch.
>
> I think there are some more problems beyond the ones mentioned, but I need
> to test this a bit more. Looking forward to a new patch!
>
>
Re: Remote service using shared mem
Posted by er...@jansman.eu.
> 2013/9/16 Björn Petri <bj...@sundevil.de>
>
>>
>> I agree, Pepijn told me that Erik Jasman has already done quite a lot of
>> work to achieve this.
>> Maybe we could contine with his approach?!
>
>
> Yeah, I'd like to see that code. Not sure if Erik has already shared
> something.
>
I think Pepijn has continued the work on that part so I'll ask him about
the status and check if I can do some more work as soon as GSOC is done.
As far as I can remember the transport service was disconnected from the
remote service admin but I can't recall the details at this moment.
>
> --
> Met vriendelijke groet,
>
> Alexander Broekhuis
>
Regards,
Erik
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
2013/9/16 Björn Petri <bj...@sundevil.de>
>
> I agree, Pepijn told me that Erik Jasman has already done quite a lot of
> work to achieve this.
> Maybe we could contine with his approach?!
Yeah, I'd like to see that code. Not sure if Erik has already shared
something.
>
>
> When testing "remote-services-shm" and stopping the "example" bundle, the
>> discovery tries to deregister an entry, this results in a call to register
>> which somehow fails with an segfault. Maybe I am missing something, but
>> there is no removal anywhere. How is this supposed to work? Btw, before
>> the
>> segfault the following error is printed:
>> DISCOVERY: Endpoint for example, with filter "(null)" removed
>> DISCOVERY : discovery_registerSHMService : encoding data from HashMap
>> failed
>>
>
> As the registered services are held as a hashmap (based on netstring) in
> the shared memory, the plan was to do a remove instead of a put if the
> value is set to NULL. Probably it is a good idea to rename the
> discovery_registerSHMService when I use it for registering and
> deregistering.
While at some points I get that some method like this might make sense... I
always prefer code readability. Using a register method to do an
unregister... Just doesn't make sense to me. So please do make a
unregister/deregister method for the deregister use case. Code wise you'd
end up with almost the same code, but there is no NULL check and 2 flows in
one method.
> I have problems getting the code to run on my Mac. I run into several
>
>> segfaults, so this needs some more testing before I can commit it.
>>
>
> I will take care about the issues mentioned above and provide an updated
> patch.
I think there are some more problems beyond the ones mentioned, but I need
to test this a bit more. Looking forward to a new patch!
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Björn Petri <bj...@sundevil.de>.
Am 2013-09-16 13:16, schrieb Alexander Broekhuis:
>
> Some other remarks/questions:
>
> I noticed you created a new proxy for shm usage. While currently this
> is
> ok, I think we need to do some refactoring to take out the transport
> knowledge from the endpoints, they should only handle
> encoding/decoding.
> Some generic callback method in the RSA should be called, which then
> takes
> care of the transport.
I agree, Pepijn told me that Erik Jasman has already done quite a lot
of work to achieve this.
Maybe we could contine with his approach?!
> Naming of the shmem files etc is currently fixed (/tmp/...), this
> should be
> made an property. Now it simply won't work on Win32 at all, although
> I am
> not sure how shmem is solved on windows, trying to keep the code as
> platform independent as possible is always a good thing.
>
> In the RSA (line 543) itself there is a hardcoded reference to
> /services/example, why is this?
Hmm .. that's a leftover from the original rsa and indeed superfluous.
> When testing "remote-services-shm" and stopping the "example" bundle,
> the
> discovery tries to deregister an entry, this results in a call to
> register
> which somehow fails with an segfault. Maybe I am missing something,
> but
> there is no removal anywhere. How is this supposed to work? Btw,
> before the
> segfault the following error is printed:
> DISCOVERY: Endpoint for example, with filter "(null)" removed
> DISCOVERY : discovery_registerSHMService : encoding data from HashMap
> failed
As the registered services are held as a hashmap (based on netstring)
in the shared memory, the plan was to do a remove instead of a put if
the value is set to NULL. Probably it is a good idea to rename the
discovery_registerSHMService when I use it for registering and
deregistering.
> I have problems getting the code to run on my Mac. I run into several
> segfaults, so this needs some more testing before I can commit it.
I will take care about the issues mentioned above and provide an
updated patch.
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
Hi
The whole netstring spec is quite straightforward (http://cr.yp.to/proto/**
> netstrings.txt <http://cr.yp.to/proto/netstrings.txt>). Hence it should
> not be that difficult to use it in Java. Probably there are already some
> implementations available.
Thanks, I was also talking about the whole shmem usage. But I already took
a look, and it fairly straightforward. I need to doe some research wrt Java
and shmem.
>
> It does check and although tries to attach to the shared memory if
> available. But it is a little bit more tricky for the semaphores as there
> might be the need to re-initalize them. Especially for the discovery part -
> using the current implementation it is not possible to determine whether
> there is already another celix instance running which has initialized those
> semaphores or whether they have been left from an segfaulting celix.
>
Makes sense.
Some other remarks/questions:
I noticed you created a new proxy for shm usage. While currently this is
ok, I think we need to do some refactoring to take out the transport
knowledge from the endpoints, they should only handle encoding/decoding.
Some generic callback method in the RSA should be called, which then takes
care of the transport.
Naming of the shmem files etc is currently fixed (/tmp/...), this should be
made an property. Now it simply won't work on Win32 at all, although I am
not sure how shmem is solved on windows, trying to keep the code as
platform independent as possible is always a good thing.
In the RSA (line 543) itself there is a hardcoded reference to
/services/example, why is this?
When testing "remote-services-shm" and stopping the "example" bundle, the
discovery tries to deregister an entry, this results in a call to register
which somehow fails with an segfault. Maybe I am missing something, but
there is no removal anywhere. How is this supposed to work? Btw, before the
segfault the following error is printed:
DISCOVERY: Endpoint for example, with filter "(null)" removed
DISCOVERY : discovery_registerSHMService : encoding data from HashMap failed
I have problems getting the code to run on my Mac. I run into several
segfaults, so this needs some more testing before I can commit it.
--
Met vriendelijke groet,
Alexander Broekhuis
Re: Remote service using shared mem
Posted by Björn Petri <bj...@sundevil.de>.
Am 2013-09-16 10:00, schrieb Alexander Broekhuis:
[..]
>
> Out of curiosity and for compatibility with Java, is there also some
> java
> support for it? I am still interested in writing a java version as
> well.
> Maybe you could provide some technical details on the working of it.
The whole netstring spec is quite straightforward
(http://cr.yp.to/proto/netstrings.txt). Hence it should not be that
difficult to use it in Java. Probably there are already some
implementations available.
>> Also note that shared memory segments or semaphores might be left in
>> the
>> system and needs to be removed manually (see ipcs, ipcrm) when CELIX
>> does
>> not shutdown as expected.
>
> Did you use APR for this? Or doesn't APR support semaphores etc? I
> haven't
> looked at it at all, but using APR would be great.
I used the apr_shm routines, but unfortunately I could not use
apr_global_mutex routines as the re-opening of the mutex in a child
process only works if you have the pointer to the original created mutex
available, which in turn could not be saved in the shared memory as
apr_shm does not support fixed addresses for the shared memory
attachment.
> Also, does the code check if there is anything left when starting
> again?
It does check and although tries to attach to the shared memory if
available. But it is a little bit more tricky for the semaphores as
there might be the need to re-initalize them. Especially for the
discovery part - using the current implementation it is not possible to
determine whether there is already another celix instance running which
has initialized those semaphores or whether they have been left from an
segfaulting celix.
Greetings,
Bjoern
Re: Remote service using shared mem
Posted by Alexander Broekhuis <a....@gmail.com>.
Hi,
I just created JIRA issue CELIX-81, including a patch which adds several
> components allowing CELIX to use Shared Memory for remote services
> (discovery_shm, remote_service_admin_shm and example_proxy_shm). Besides,
> two new deployment targets are added, which can be used to locally test the
> shared memory implementation using the already existing example_service.
>
Thanks for this work! If Pepijn hasn't started working on this, I will test
and add it to the build tomorrow.
>
> Netstring support (see JIRA issue CELIX-80 for according patch) is
> required to get the this shm implementation working.
Out of curiosity and for compatibility with Java, is there also some java
support for it? I am still interested in writing a java version as well.
Maybe you could provide some technical details on the working of it.
> Also note that shared memory segments or semaphores might be left in the
> system and needs to be removed manually (see ipcs, ipcrm) when CELIX does
> not shutdown as expected.
>
Did you use APR for this? Or doesn't APR support semaphores etc? I haven't
looked at it at all, but using APR would be great.
Also, does the code check if there is anything left when starting again?
Either way, making Celix robust so that is doesn't shutdown as expected is
the best solution ;).
--
Met vriendelijke groet,
Alexander Broekhuis