You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@felix.apache.org by Dirk Hogan <di...@forgerock.com> on 2017/04/27 16:33:33 UTC

memory visibility of @Reference service references

I have been puzzled about the visibility of references written/read across
threads in Felix.

Scenario:

public class myOSGiComponent {

@Reference

SomeOtherOSGiService myServiceReference;

public void activate(ComponentContext cc) {

// myServiceReference will be set - use it to initialize other
dependencies, etc

}

public void clientRequestViaEmbeddedJetty() {

myServiceReference.foo();

}

The bottom line is that a STATIC reference does not have to be volatile. I
can see that the write to myServiceReference is guaranteed to be visible in
the activate method, provided that same Felix thread which set
myServiceReference invokes activate, and thus the visibility is guaranteed
by program order (17.4.3 of the Java Language Spec).

However, in the absence of the synchronization action provided by e.g. a
volatile @Reference declaration, I do not see how Felix can provide a Java
Memory Model compliant guarantee that myServiceReference will be visible in
clientRequestViaEmbeddedJetty, as the thread which set the reference, and
the thread which invokes clientRequestViaEmbeddedJetty, will be distinct.
In the absence of a volatile @Reference declaration, there is no
synchronization action (17.4.2 of JLS) which would guarantee that the
reference write is visible to the reference read, as they are being
performed by distinct threads.

Perhaps an explanation as to why a DYNAMIC @Reference does need to be
volatile would be helpful - e.g. why does component
de-activation/re-activation provide memory visibility guarantees for STATIC
references?


Thanks


Dirk

Re: memory visibility of @Reference service references

Posted by Julian Sedding <js...@gmail.com>.
Hi David

On Fri, Apr 28, 2017 at 5:19 PM, David Jencks <da...@gmail.com> wrote:
> Step 5 happens before step 1.

You are right. SCR registers a ServiceFactory with the framework
before step 1 (at least for a delayed component). And on top of that,
I got the terminology pretty wrong.

Let me try again (the following assumes a delayed component):

1. the component becomes satisfied
2. DS (or rather SCR) registers a ServiceFactory with the framework's
service registry
3. the service is requested from the service registry by thread T1
3.1. DS creates a new instances of the implementation object S1 in thread T1
3.2. DS injects S1 with the (static) @Reference service in thread T1
3.3. DS calls activate() in thread T1
3.4. at this time no other thread has a reference to service S1
3.5. DS makes S1 accessible and returns it from the
ServiceFactory#getService() method
4. at this time other threads can get a fully initialized reference to
service S1

So regarding Java's memory model, I still assume that full activation
of the implementation object S1 before it leaves the scope of
ServiceFactory#getService() is the key. FWIW, the Felix implementation
keeps the reference to S1 in a volatile field that can only be
accessed once a volatile boolean is set to true.

Does this sound better?

Regards
Julian

>
> I’m not sure that java by itself has sufficient synchronization to assure that a requesting thread gets the fully correct memory set up by thread T1.  However I Think that with the synchronization in Felix DS requesting threads will get correct information.
>
> thanks
> david jencks
>
>> On Apr 28, 2017, at 1:13 AM, Julian Sedding <js...@gmail.com> wrote:
>>
>> My theory is as follows:
>>
>> 1. DS creates a new instances of the service S1 in thread T1
>> 2. DS injects the (static) @Reference service in thread T1
>> 3. DS calls activate() in thread T1
>> 4. at this time no other thread has a reference to service S1
>> 5. DS publishes the service in the OSGi service registry
>> 6. at this time other thread can get a fully initialized reference to service S1
>>
>> I believe that because up to 4. the reference to S1 is "private" to
>> thread T1. In 5. S1 is "published" and thus first allowed to cross a
>> thread boundary (I assume Java internally creates some sort of copy of
>> S1's memory for a new thread T2). However, at that point S1 is fully
>> initialized and immutable. Should the static @Reference on S1 change,
>> it is deactivated and a new instance S1' is created, activated and
>> registered.
>>
>> In contrast, with a dynamic @Reference, the field's value may change
>> after it is published. Thus it needs to be volatile.
>>
>> As I said, this is a theory ;) I have no references to back me up.
>>
>> Regards
>> Julian
>>
>>
>> On Thu, Apr 27, 2017 at 7:06 PM, David Jencks <da...@gmail.com> wrote:
>>> My theory about why this is ok is as follows, perhaps you can see something wrong with it.
>>>
>>> Felix DS has some synchronization after activate is called, therefore the state of the static reference is definitely written to memory.  This reference field won’t change after that, since it’s (SCR) static.
>>> Any getService call on the service reference also has sufficient synchronization that it happens after the above write.  Therefore any other thread which obtained the referenced service will get an object with the correct field value.  Therefore any access to this possibly cached object by any thread will have the correct value.
>>>
>>> thanks
>>>
>>> david jencks
>>>
>>>> On Apr 27, 2017, at 9:33 AM, Dirk Hogan <di...@forgerock.com> wrote:
>>>>
>>>> I have been puzzled about the visibility of references written/read across
>>>> threads in Felix.
>>>>
>>>> Scenario:
>>>>
>>>> public class myOSGiComponent {
>>>>
>>>> @Reference
>>>>
>>>> SomeOtherOSGiService myServiceReference;
>>>>
>>>> public void activate(ComponentContext cc) {
>>>>
>>>> // myServiceReference will be set - use it to initialize other
>>>> dependencies, etc
>>>>
>>>> }
>>>>
>>>> public void clientRequestViaEmbeddedJetty() {
>>>>
>>>> myServiceReference.foo();
>>>>
>>>> }
>>>>
>>>> The bottom line is that a STATIC reference does not have to be volatile. I
>>>> can see that the write to myServiceReference is guaranteed to be visible in
>>>> the activate method, provided that same Felix thread which set
>>>> myServiceReference invokes activate, and thus the visibility is guaranteed
>>>> by program order (17.4.3 of the Java Language Spec).
>>>>
>>>> However, in the absence of the synchronization action provided by e.g. a
>>>> volatile @Reference declaration, I do not see how Felix can provide a Java
>>>> Memory Model compliant guarantee that myServiceReference will be visible in
>>>> clientRequestViaEmbeddedJetty, as the thread which set the reference, and
>>>> the thread which invokes clientRequestViaEmbeddedJetty, will be distinct.
>>>> In the absence of a volatile @Reference declaration, there is no
>>>> synchronization action (17.4.2 of JLS) which would guarantee that the
>>>> reference write is visible to the reference read, as they are being
>>>> performed by distinct threads.
>>>>
>>>> Perhaps an explanation as to why a DYNAMIC @Reference does need to be
>>>> volatile would be helpful - e.g. why does component
>>>> de-activation/re-activation provide memory visibility guarantees for STATIC
>>>> references?
>>>>
>>>>
>>>> Thanks
>>>>
>>>>
>>>> Dirk
>>>
>

Re: memory visibility of @Reference service references

Posted by David Jencks <da...@gmail.com>.
Step 5 happens before step 1.

I’m not sure that java by itself has sufficient synchronization to assure that a requesting thread gets the fully correct memory set up by thread T1.  However I Think that with the synchronization in Felix DS requesting threads will get correct information.

thanks
david jencks

> On Apr 28, 2017, at 1:13 AM, Julian Sedding <js...@gmail.com> wrote:
> 
> My theory is as follows:
> 
> 1. DS creates a new instances of the service S1 in thread T1
> 2. DS injects the (static) @Reference service in thread T1
> 3. DS calls activate() in thread T1
> 4. at this time no other thread has a reference to service S1
> 5. DS publishes the service in the OSGi service registry
> 6. at this time other thread can get a fully initialized reference to service S1
> 
> I believe that because up to 4. the reference to S1 is "private" to
> thread T1. In 5. S1 is "published" and thus first allowed to cross a
> thread boundary (I assume Java internally creates some sort of copy of
> S1's memory for a new thread T2). However, at that point S1 is fully
> initialized and immutable. Should the static @Reference on S1 change,
> it is deactivated and a new instance S1' is created, activated and
> registered.
> 
> In contrast, with a dynamic @Reference, the field's value may change
> after it is published. Thus it needs to be volatile.
> 
> As I said, this is a theory ;) I have no references to back me up.
> 
> Regards
> Julian
> 
> 
> On Thu, Apr 27, 2017 at 7:06 PM, David Jencks <da...@gmail.com> wrote:
>> My theory about why this is ok is as follows, perhaps you can see something wrong with it.
>> 
>> Felix DS has some synchronization after activate is called, therefore the state of the static reference is definitely written to memory.  This reference field won’t change after that, since it’s (SCR) static.
>> Any getService call on the service reference also has sufficient synchronization that it happens after the above write.  Therefore any other thread which obtained the referenced service will get an object with the correct field value.  Therefore any access to this possibly cached object by any thread will have the correct value.
>> 
>> thanks
>> 
>> david jencks
>> 
>>> On Apr 27, 2017, at 9:33 AM, Dirk Hogan <di...@forgerock.com> wrote:
>>> 
>>> I have been puzzled about the visibility of references written/read across
>>> threads in Felix.
>>> 
>>> Scenario:
>>> 
>>> public class myOSGiComponent {
>>> 
>>> @Reference
>>> 
>>> SomeOtherOSGiService myServiceReference;
>>> 
>>> public void activate(ComponentContext cc) {
>>> 
>>> // myServiceReference will be set - use it to initialize other
>>> dependencies, etc
>>> 
>>> }
>>> 
>>> public void clientRequestViaEmbeddedJetty() {
>>> 
>>> myServiceReference.foo();
>>> 
>>> }
>>> 
>>> The bottom line is that a STATIC reference does not have to be volatile. I
>>> can see that the write to myServiceReference is guaranteed to be visible in
>>> the activate method, provided that same Felix thread which set
>>> myServiceReference invokes activate, and thus the visibility is guaranteed
>>> by program order (17.4.3 of the Java Language Spec).
>>> 
>>> However, in the absence of the synchronization action provided by e.g. a
>>> volatile @Reference declaration, I do not see how Felix can provide a Java
>>> Memory Model compliant guarantee that myServiceReference will be visible in
>>> clientRequestViaEmbeddedJetty, as the thread which set the reference, and
>>> the thread which invokes clientRequestViaEmbeddedJetty, will be distinct.
>>> In the absence of a volatile @Reference declaration, there is no
>>> synchronization action (17.4.2 of JLS) which would guarantee that the
>>> reference write is visible to the reference read, as they are being
>>> performed by distinct threads.
>>> 
>>> Perhaps an explanation as to why a DYNAMIC @Reference does need to be
>>> volatile would be helpful - e.g. why does component
>>> de-activation/re-activation provide memory visibility guarantees for STATIC
>>> references?
>>> 
>>> 
>>> Thanks
>>> 
>>> 
>>> Dirk
>> 


Re: memory visibility of @Reference service references

Posted by Julian Sedding <js...@gmail.com>.
My theory is as follows:

1. DS creates a new instances of the service S1 in thread T1
2. DS injects the (static) @Reference service in thread T1
3. DS calls activate() in thread T1
4. at this time no other thread has a reference to service S1
5. DS publishes the service in the OSGi service registry
6. at this time other thread can get a fully initialized reference to service S1

I believe that because up to 4. the reference to S1 is "private" to
thread T1. In 5. S1 is "published" and thus first allowed to cross a
thread boundary (I assume Java internally creates some sort of copy of
S1's memory for a new thread T2). However, at that point S1 is fully
initialized and immutable. Should the static @Reference on S1 change,
it is deactivated and a new instance S1' is created, activated and
registered.

In contrast, with a dynamic @Reference, the field's value may change
after it is published. Thus it needs to be volatile.

As I said, this is a theory ;) I have no references to back me up.

Regards
Julian


On Thu, Apr 27, 2017 at 7:06 PM, David Jencks <da...@gmail.com> wrote:
> My theory about why this is ok is as follows, perhaps you can see something wrong with it.
>
> Felix DS has some synchronization after activate is called, therefore the state of the static reference is definitely written to memory.  This reference field won’t change after that, since it’s (SCR) static.
> Any getService call on the service reference also has sufficient synchronization that it happens after the above write.  Therefore any other thread which obtained the referenced service will get an object with the correct field value.  Therefore any access to this possibly cached object by any thread will have the correct value.
>
> thanks
>
> david jencks
>
>> On Apr 27, 2017, at 9:33 AM, Dirk Hogan <di...@forgerock.com> wrote:
>>
>> I have been puzzled about the visibility of references written/read across
>> threads in Felix.
>>
>> Scenario:
>>
>> public class myOSGiComponent {
>>
>> @Reference
>>
>> SomeOtherOSGiService myServiceReference;
>>
>> public void activate(ComponentContext cc) {
>>
>> // myServiceReference will be set - use it to initialize other
>> dependencies, etc
>>
>> }
>>
>> public void clientRequestViaEmbeddedJetty() {
>>
>> myServiceReference.foo();
>>
>> }
>>
>> The bottom line is that a STATIC reference does not have to be volatile. I
>> can see that the write to myServiceReference is guaranteed to be visible in
>> the activate method, provided that same Felix thread which set
>> myServiceReference invokes activate, and thus the visibility is guaranteed
>> by program order (17.4.3 of the Java Language Spec).
>>
>> However, in the absence of the synchronization action provided by e.g. a
>> volatile @Reference declaration, I do not see how Felix can provide a Java
>> Memory Model compliant guarantee that myServiceReference will be visible in
>> clientRequestViaEmbeddedJetty, as the thread which set the reference, and
>> the thread which invokes clientRequestViaEmbeddedJetty, will be distinct.
>> In the absence of a volatile @Reference declaration, there is no
>> synchronization action (17.4.2 of JLS) which would guarantee that the
>> reference write is visible to the reference read, as they are being
>> performed by distinct threads.
>>
>> Perhaps an explanation as to why a DYNAMIC @Reference does need to be
>> volatile would be helpful - e.g. why does component
>> de-activation/re-activation provide memory visibility guarantees for STATIC
>> references?
>>
>>
>> Thanks
>>
>>
>> Dirk
>

Re: memory visibility of @Reference service references

Posted by David Jencks <da...@gmail.com>.
My theory about why this is ok is as follows, perhaps you can see something wrong with it.

Felix DS has some synchronization after activate is called, therefore the state of the static reference is definitely written to memory.  This reference field won’t change after that, since it’s (SCR) static.
Any getService call on the service reference also has sufficient synchronization that it happens after the above write.  Therefore any other thread which obtained the referenced service will get an object with the correct field value.  Therefore any access to this possibly cached object by any thread will have the correct value.

thanks

david jencks

> On Apr 27, 2017, at 9:33 AM, Dirk Hogan <di...@forgerock.com> wrote:
> 
> I have been puzzled about the visibility of references written/read across
> threads in Felix.
> 
> Scenario:
> 
> public class myOSGiComponent {
> 
> @Reference
> 
> SomeOtherOSGiService myServiceReference;
> 
> public void activate(ComponentContext cc) {
> 
> // myServiceReference will be set - use it to initialize other
> dependencies, etc
> 
> }
> 
> public void clientRequestViaEmbeddedJetty() {
> 
> myServiceReference.foo();
> 
> }
> 
> The bottom line is that a STATIC reference does not have to be volatile. I
> can see that the write to myServiceReference is guaranteed to be visible in
> the activate method, provided that same Felix thread which set
> myServiceReference invokes activate, and thus the visibility is guaranteed
> by program order (17.4.3 of the Java Language Spec).
> 
> However, in the absence of the synchronization action provided by e.g. a
> volatile @Reference declaration, I do not see how Felix can provide a Java
> Memory Model compliant guarantee that myServiceReference will be visible in
> clientRequestViaEmbeddedJetty, as the thread which set the reference, and
> the thread which invokes clientRequestViaEmbeddedJetty, will be distinct.
> In the absence of a volatile @Reference declaration, there is no
> synchronization action (17.4.2 of JLS) which would guarantee that the
> reference write is visible to the reference read, as they are being
> performed by distinct threads.
> 
> Perhaps an explanation as to why a DYNAMIC @Reference does need to be
> volatile would be helpful - e.g. why does component
> de-activation/re-activation provide memory visibility guarantees for STATIC
> references?
> 
> 
> Thanks
> 
> 
> Dirk


Re: memory visibility of @Reference service references

Posted by Neil Bartlett <nj...@gmail.com>.
The Declarative Services spec requires that there is a happens-before relationship between SCR setting the field and activating the component instance — see OSGi Compendium 6.0.0 section 112.3.8.1.

There are various ways of achieving this under the Java Memory Model. For example the SCR implementation can use internal synchronized blocks.

Regards,
Neil

> On 27 Apr 2017, at 17:33, Dirk Hogan <di...@forgerock.com> wrote:
> 
> I have been puzzled about the visibility of references written/read across
> threads in Felix.
> 
> Scenario:
> 
> public class myOSGiComponent {
> 
> @Reference
> 
> SomeOtherOSGiService myServiceReference;
> 
> public void activate(ComponentContext cc) {
> 
> // myServiceReference will be set - use it to initialize other
> dependencies, etc
> 
> }
> 
> public void clientRequestViaEmbeddedJetty() {
> 
> myServiceReference.foo();
> 
> }
> 
> The bottom line is that a STATIC reference does not have to be volatile. I
> can see that the write to myServiceReference is guaranteed to be visible in
> the activate method, provided that same Felix thread which set
> myServiceReference invokes activate, and thus the visibility is guaranteed
> by program order (17.4.3 of the Java Language Spec).
> 
> However, in the absence of the synchronization action provided by e.g. a
> volatile @Reference declaration, I do not see how Felix can provide a Java
> Memory Model compliant guarantee that myServiceReference will be visible in
> clientRequestViaEmbeddedJetty, as the thread which set the reference, and
> the thread which invokes clientRequestViaEmbeddedJetty, will be distinct.
> In the absence of a volatile @Reference declaration, there is no
> synchronization action (17.4.2 of JLS) which would guarantee that the
> reference write is visible to the reference read, as they are being
> performed by distinct threads.
> 
> Perhaps an explanation as to why a DYNAMIC @Reference does need to be
> volatile would be helpful - e.g. why does component
> de-activation/re-activation provide memory visibility guarantees for STATIC
> references?
> 
> 
> Thanks
> 
> 
> Dirk