A solution to avoid ConcurrentModificationException

[Ramkumar R <ra...@gmail.com>]

Hi All,
Looking at the history of JIRA's (2170, 2085, 2092) raised due to
ConcurrentModificationException, its evident that our runtime would throw
more such exceptions in the coming days, such problem occurs if two threads
try to add a contribution simultaneously. Also the above mentioned JIRA's
gives us a clue that mostly these issues are noticed for the ExtensionPoint
implementation classes.

Current Implementation of ExtensionPoint classes use either ArrayList OR
HashMap to maintain the list of processors, listeners, factories and
providers. Basically the Implementation of ArrayList & HashMap are not
synchronized, hence If multiple threads try to access an ArrayList/HashMap
instance concurrently, and at least one of the threads modifies the list/map
structurally, it must be synchronized externally. Hence a fix is required
for these classes, *please have a look at JIRA-2170 comments for more
details on the implemented solution*.

As a precautionary measure, I like to raise a JIRA for all the
ExtensionPoint classes to provide this fix for the above said issue, before
we encounter them one by one. Before I go ahead, I like to hear from people
about their thought on this regard. Thanks.

-- 
Thanks & Regards,
Ramkumar Ramalingam

Re: A solution to avoid ConcurrentModificationException

[Ramkumar R <ra...@gmail.com>]

Thanks Simon, for your recommendation on synchronizing the access to
loadListeners() method for TUSCANY-2192, i agree with you and have reopened
the JIRA to apply the fix.

Thanks Raymond and Scott for your inputs on this regard. For the scenario
explained by Scott it looks like synchronizing the access to
loadDataBindings() method resolves the issue.

To get the better understanding on JIRA's TUSCANY-2192 and TUSCANY-2170, it
would be of great help if Greg can share the scenario for which he is facing
this issue. In the mean time i am planning to put an itest (as suggested by
Simon) in place to replicate these issues.

Thanks,
Ramkumar Ramalingam

Re: A solution to avoid ConcurrentModificationException

[Raymond Feng <en...@gmail.com>]

Hi,

I buy in this case as it's an issue around the lazy loading.

Thanks,
Raymond
--------------------------------------------------
From: "Scott Kurz" <sc...@gmail.com>
Sent: Thursday, April 10, 2008 8:02 AM
To: <tu...@ws.apache.org>
Subject: Re: A solution to avoid ConcurrentModificationException

> Raymond,
>
> So, for 2085 for example, we have multiple different modules that, when
> activated, may do an addDatabinding on the defaultDBExtensionPoint.
>
> Eventually.. later...someone comes along and uses this to do an
> introspectType.   At that point the loadDataBindings() is done.
>
> That all seems to make sense to me... it just needs to be done in a
> thread-safe way in case two initial introspectType's are done at once on
> different threads.
>
>
> On Thu, Apr 10, 2008 at 10:59 AM, Simon Laws <si...@googlemail.com>
> wrote:
>
>> On Thu, Apr 10, 2008 at 3:29 PM, Raymond Feng <en...@gmail.com> 
>> wrote:
>>
>> > Hi,
>> >
>> > Thanks Ramkumar for looking into these concurrent issues.
>> >
>> > Before we jump into a solution/fix, I suggest that we step back and
>> agree
>> > on whether the Tuscany core and extensions should be bootstrapped
>> > sequentially or concurrently. A few use cases would be helpful. Can 
>> > some
>> of
>> > the JIRA reporters describe their needs to make these extension points
>> > thread-safe?
>> >
>> > Raymond
>> > --------------------------------------------------
>> > From: "Ramkumar R" <ra...@gmail.com>
>> > Sent: Thursday, April 10, 2008 3:59 AM
>> > To: <tu...@ws.apache.org>
>> > Subject: A solution to avoid ConcurrentModificationException
>> >
>> >
>> >  Hi All,
>> > > Looking at the history of JIRA's (2170, 2085, 2092) raised due to
>> > > ConcurrentModificationException, its evident that our runtime would
>> > > throw
>> > > more such exceptions in the coming days, such problem occurs if two
>> > > threads
>> > > try to add a contribution simultaneously. Also the above mentioned
>> > > JIRA's
>> > > gives us a clue that mostly these issues are noticed for the
>> > > ExtensionPoint
>> > > implementation classes.
>> > >
>> > > Current Implementation of ExtensionPoint classes use either ArrayList
>> OR
>> > > HashMap to maintain the list of processors, listeners, factories and
>> > > providers. Basically the Implementation of ArrayList & HashMap are 
>> > > not
>> > > synchronized, hence If multiple threads try to access an
>> > > ArrayList/HashMap
>> > > instance concurrently, and at least one of the threads modifies the
>> > > list/map
>> > > structurally, it must be synchronized externally. Hence a fix is
>> > > required
>> > > for these classes, *please have a look at JIRA-2170 comments for more
>> > > details on the implemented solution*.
>> > >
>> > > As a precautionary measure, I like to raise a JIRA for all the
>> > > ExtensionPoint classes to provide this fix for the above said issue,
>> > > before
>> > > we encounter them one by one. Before I go ahead, I like to hear from
>> > > people
>> > > about their thought on this regard. Thanks.
>> > >
>> > > --
>> > > Thanks & Regards,
>> > > Ramkumar Ramalingam
>> > >
>> > >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>> >
>> >
>> I don't think people are consciously bootstrapping Tuscany concurrently
>> but
>> doing things concurrently like adding contributions.
>>
>> Simon
>>
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org

Re: A solution to avoid ConcurrentModificationException

[Scott Kurz <sc...@gmail.com>]

Raymond,

So, for 2085 for example, we have multiple different modules that, when
activated, may do an addDatabinding on the defaultDBExtensionPoint.

Eventually.. later...someone comes along and uses this to do an
introspectType.   At that point the loadDataBindings() is done.

That all seems to make sense to me... it just needs to be done in a
thread-safe way in case two initial introspectType's are done at once on
different threads.


On Thu, Apr 10, 2008 at 10:59 AM, Simon Laws <si...@googlemail.com>
wrote:

> On Thu, Apr 10, 2008 at 3:29 PM, Raymond Feng <en...@gmail.com> wrote:
>
> > Hi,
> >
> > Thanks Ramkumar for looking into these concurrent issues.
> >
> > Before we jump into a solution/fix, I suggest that we step back and
> agree
> > on whether the Tuscany core and extensions should be bootstrapped
> > sequentially or concurrently. A few use cases would be helpful. Can some
> of
> > the JIRA reporters describe their needs to make these extension points
> > thread-safe?
> >
> > Raymond
> > --------------------------------------------------
> > From: "Ramkumar R" <ra...@gmail.com>
> > Sent: Thursday, April 10, 2008 3:59 AM
> > To: <tu...@ws.apache.org>
> > Subject: A solution to avoid ConcurrentModificationException
> >
> >
> >  Hi All,
> > > Looking at the history of JIRA's (2170, 2085, 2092) raised due to
> > > ConcurrentModificationException, its evident that our runtime would
> > > throw
> > > more such exceptions in the coming days, such problem occurs if two
> > > threads
> > > try to add a contribution simultaneously. Also the above mentioned
> > > JIRA's
> > > gives us a clue that mostly these issues are noticed for the
> > > ExtensionPoint
> > > implementation classes.
> > >
> > > Current Implementation of ExtensionPoint classes use either ArrayList
> OR
> > > HashMap to maintain the list of processors, listeners, factories and
> > > providers. Basically the Implementation of ArrayList & HashMap are not
> > > synchronized, hence If multiple threads try to access an
> > > ArrayList/HashMap
> > > instance concurrently, and at least one of the threads modifies the
> > > list/map
> > > structurally, it must be synchronized externally. Hence a fix is
> > > required
> > > for these classes, *please have a look at JIRA-2170 comments for more
> > > details on the implemented solution*.
> > >
> > > As a precautionary measure, I like to raise a JIRA for all the
> > > ExtensionPoint classes to provide this fix for the above said issue,
> > > before
> > > we encounter them one by one. Before I go ahead, I like to hear from
> > > people
> > > about their thought on this regard. Thanks.
> > >
> > > --
> > > Thanks & Regards,
> > > Ramkumar Ramalingam
> > >
> > >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
> >
> >
> I don't think people are consciously bootstrapping Tuscany concurrently
> but
> doing things concurrently like adding contributions.
>
> Simon
>

Re: A solution to avoid ConcurrentModificationException

[Simon Laws <si...@googlemail.com>]

On Thu, Apr 10, 2008 at 3:29 PM, Raymond Feng <en...@gmail.com> wrote:

> Hi,
>
> Thanks Ramkumar for looking into these concurrent issues.
>
> Before we jump into a solution/fix, I suggest that we step back and agree
> on whether the Tuscany core and extensions should be bootstrapped
> sequentially or concurrently. A few use cases would be helpful. Can some of
> the JIRA reporters describe their needs to make these extension points
> thread-safe?
>
> Raymond
> --------------------------------------------------
> From: "Ramkumar R" <ra...@gmail.com>
> Sent: Thursday, April 10, 2008 3:59 AM
> To: <tu...@ws.apache.org>
> Subject: A solution to avoid ConcurrentModificationException
>
>
>  Hi All,
> > Looking at the history of JIRA's (2170, 2085, 2092) raised due to
> > ConcurrentModificationException, its evident that our runtime would
> > throw
> > more such exceptions in the coming days, such problem occurs if two
> > threads
> > try to add a contribution simultaneously. Also the above mentioned
> > JIRA's
> > gives us a clue that mostly these issues are noticed for the
> > ExtensionPoint
> > implementation classes.
> >
> > Current Implementation of ExtensionPoint classes use either ArrayList OR
> > HashMap to maintain the list of processors, listeners, factories and
> > providers. Basically the Implementation of ArrayList & HashMap are not
> > synchronized, hence If multiple threads try to access an
> > ArrayList/HashMap
> > instance concurrently, and at least one of the threads modifies the
> > list/map
> > structurally, it must be synchronized externally. Hence a fix is
> > required
> > for these classes, *please have a look at JIRA-2170 comments for more
> > details on the implemented solution*.
> >
> > As a precautionary measure, I like to raise a JIRA for all the
> > ExtensionPoint classes to provide this fix for the above said issue,
> > before
> > we encounter them one by one. Before I go ahead, I like to hear from
> > people
> > about their thought on this regard. Thanks.
> >
> > --
> > Thanks & Regards,
> > Ramkumar Ramalingam
> >
> >
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
>
I don't think people are consciously bootstrapping Tuscany concurrently but
doing things concurrently like adding contributions.

Simon

Re: A solution to avoid ConcurrentModificationException

[Raymond Feng <en...@gmail.com>]

Hi,

Thanks Ramkumar for looking into these concurrent issues.

Before we jump into a solution/fix, I suggest that we step back and agree on 
whether the Tuscany core and extensions should be bootstrapped sequentially 
or concurrently. A few use cases would be helpful. Can some of the JIRA 
reporters describe their needs to make these extension points thread-safe?

Raymond
--------------------------------------------------
From: "Ramkumar R" <ra...@gmail.com>
Sent: Thursday, April 10, 2008 3:59 AM
To: <tu...@ws.apache.org>
Subject: A solution to avoid ConcurrentModificationException

> Hi All,
> Looking at the history of JIRA's (2170, 2085, 2092) raised due to
> ConcurrentModificationException, its evident that our runtime would throw
> more such exceptions in the coming days, such problem occurs if two 
> threads
> try to add a contribution simultaneously. Also the above mentioned JIRA's
> gives us a clue that mostly these issues are noticed for the 
> ExtensionPoint
> implementation classes.
>
> Current Implementation of ExtensionPoint classes use either ArrayList OR
> HashMap to maintain the list of processors, listeners, factories and
> providers. Basically the Implementation of ArrayList & HashMap are not
> synchronized, hence If multiple threads try to access an ArrayList/HashMap
> instance concurrently, and at least one of the threads modifies the 
> list/map
> structurally, it must be synchronized externally. Hence a fix is required
> for these classes, *please have a look at JIRA-2170 comments for more
> details on the implemented solution*.
>
> As a precautionary measure, I like to raise a JIRA for all the
> ExtensionPoint classes to provide this fix for the above said issue, 
> before
> we encounter them one by one. Before I go ahead, I like to hear from 
> people
> about their thought on this regard. Thanks.
>
> -- 
> Thanks & Regards,
> Ramkumar Ramalingam
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org

Re: A solution to avoid ConcurrentModificationException

[Simon Laws <si...@googlemail.com>]

On Thu, Apr 10, 2008 at 11:59 AM, Ramkumar R <ra...@gmail.com> wrote:

> Hi All,
> Looking at the history of JIRA's (2170, 2085, 2092) raised due to
> ConcurrentModificationException, its evident that our runtime would throw
> more such exceptions in the coming days, such problem occurs if two
> threads
> try to add a contribution simultaneously. Also the above mentioned JIRA's
> gives us a clue that mostly these issues are noticed for the
> ExtensionPoint
> implementation classes.
>
> Current Implementation of ExtensionPoint classes use either ArrayList OR
> HashMap to maintain the list of processors, listeners, factories and
> providers. Basically the Implementation of ArrayList & HashMap are not
> synchronized, hence If multiple threads try to access an ArrayList/HashMap
> instance concurrently, and at least one of the threads modifies the
> list/map
> structurally, it must be synchronized externally. Hence a fix is required
> for these classes, *please have a look at JIRA-2170 comments for more
> details on the implemented solution*.
>
> As a precautionary measure, I like to raise a JIRA for all the
> ExtensionPoint classes to provide this fix for the above said issue,
> before
> we encounter them one by one. Before I go ahead, I like to hear from
> people
> about their thought on this regard. Thanks.
>
> --
> Thanks & Regards,
> Ramkumar Ramalingam
>


Hi Ram

We need to be a little careful here. In general it's good to ensure thread
safety of the underlying collections but we also need to consider the wider
context and make sure we aren't masking other problems by fixing the
underlying errors.

For example, in TUSCANY-2092, the wider issue seems to be that the
loadListeners() method is not synchronized and hence runs twice when it
should only run once. Synchronizing the collection will ensure that the
error isn't thrown but we will likely end up with a list that is twice as
long as we expect it to be. This is because the test at the start of
loadListeners(),

        if (loadedListeners)
            return;

won't prevent multiple concurrent threads getting in here. In which case
each thread will load the listeners and we will end up with a list that is
twice as long as we expect it to be. There may be similar wider implications
in the other JIRA and certainly 2085 looks like it could suffer from the
same problem if the first two messages through the runtime required access
to databindings concurrently. It would be worth double checking these JIRA
to ensure that we aren't masking other issues by fixing the collections.

I would suggest setting up an itest to do some simple concurrency testing
by, in the first instance, loading two contributions concurrently and
sending multiple messages concurrently. We can then extend in the future if
required. I don't think there is a test for this already. There is a test
that sends many OneWay messages but that's not quite the same thing.

Regards

Simon