You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/06/18 14:51:05 UTC
[GitHub] [pulsar] merlimat opened a new pull request, #16125: Use non-reentrant lock for GrowableArrayBlockingQueue tail
merlimat opened a new pull request, #16125:
URL: https://github.com/apache/pulsar/pull/16125
### Motivation
`GrowableArrayBlockingQueue`, as the name implies, doesn't have a max size and will never block the producers. Because of that, we don't need a reentrant lock to guard the tail of the queue, instead, we can use the lighter-weight StampedLock.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #16125: Use non-reentrant lock for GrowableArrayBlockingQueue tail
Posted by GitBox <gi...@apache.org>.
lhotari commented on PR #16125:
URL: https://github.com/apache/pulsar/pull/16125#issuecomment-1189326054
> @lhotari OK, I get your point. It's the array elements, not the array object. This concern make sense. My guess on this is that there are happen-before relations here:
> - `put()` ->
> - `data[tailIndex.value] = e;//write Ops` -->
> - `SIZE_UPDATER.getAndIncrement(...) in put()` -->
> - `SIZE_UPDATER.get() in poll()` -->
> - `T item = data[headIndex.value];//Read Ops`.
> So the array element written by `put` is always visible to `poll`.
>
> Not 100% sure about this :)
Good analysis @Jason918 . Yes it seems that SIZE_UPDATER saves it in this case, however there's a risk that there's an access path which isn't thread safe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on pull request #16125: Use non-reentrant lock for GrowableArrayBlockingQueue tail
Posted by GitBox <gi...@apache.org>.
Jason918 commented on PR #16125:
URL: https://github.com/apache/pulsar/pull/16125#issuecomment-1188858989
> `data` array elements are mutated and read with 2 different locks in the methods in `GrowableArrayBlockingQueue` class. For example, `poll` is guarded by `headLock` and `put` is guarded by `tailLock`. I don't see how that could be thread safe. There's a related StackOverflow answer https://stackoverflow.com/a/8978397 which states:
>
> > "This means that you can safely write to two different indexes concurrently. However you need to synchronize a write/read to the same index if you want to make sure the consumer thread sees the last value written by the producer thread."
@lhotari OK, I get your point. The race condition refers to array elements. This make sense.
I can try to create a bad case for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on pull request #16125: Use non-reentrant lock for GrowableArrayBlockingQueue tail
Posted by GitBox <gi...@apache.org>.
Jason918 commented on PR #16125:
URL: https://github.com/apache/pulsar/pull/16125#issuecomment-1190096730
> > @lhotari OK, I get your point. It's the array elements, not the array object. This concern make sense. My guess on this is that there are happen-before relations here:
> >
> > * `put()` ->
> > * `data[tailIndex.value] = e;//write Ops` -->
> > * `SIZE_UPDATER.getAndIncrement(...) in put()` -->
> > * `SIZE_UPDATER.get() in poll()` -->
> > * `T item = data[headIndex.value];//Read Ops`.
> > So the array element written by `put` is always visible to `poll`.
> >
> > Not 100% sure about this :)
>
> Good analysis @Jason918 . Yes it seems that SIZE_UPDATER saves it in this case, however there's a risk that there's an access path which isn't thread safe.
Sure, we can try to construct a unit test to verify this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on pull request #16125: Use non-reentrant lock for GrowableArrayBlockingQueue tail
Posted by GitBox <gi...@apache.org>.
Jason918 commented on PR #16125:
URL: https://github.com/apache/pulsar/pull/16125#issuecomment-1188866996
> > @lhotari The `data` array object mutation only happens in the code path `put() -> expandArray()`, where `put()` acquires the `tailLock`, and `expandArray()` acquires the `headLock`. So the `data` array object should be thread safe if you acquires either `tailLock` or `headLock` before accessing the object.
>
> `data` array elements are mutated and read with 2 different locks in the methods in `GrowableArrayBlockingQueue` class. For example, `poll` is guarded by `headLock` and `put` is guarded by `tailLock`. I don't see how that could be thread safe. There's a related StackOverflow answer https://stackoverflow.com/a/8978397 which states:
>
> > "This means that you can safely write to two different indexes concurrently. However you need to synchronize a write/read to the same index if you want to make sure the consumer thread sees the last value written by the producer thread."
@lhotari OK, I get your point. It's the array elements, not the array object. This concern make sense. My guess on this is that there are happen-before relations here:
- `put()` ->
- `data[tailIndex.value] = e;//write Ops` -->
- `SIZE_UPDATER.getAndIncrement(...) in put()` -->
- `SIZE_UPDATER.get() in poll()` -->
- `T item = data[headIndex.value];//Read Ops`.
So the array element written by `put` is always visible to `poll`.
Not 100% sure about this :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #16125: Use non-reentrant lock for GrowableArrayBlockingQueue tail
Posted by GitBox <gi...@apache.org>.
lhotari commented on PR #16125:
URL: https://github.com/apache/pulsar/pull/16125#issuecomment-1188296636
> @lhotari The `data` array object mutation only happens in the code path `put() -> expandArray()`, where `put()` acquires the `tailLock`, and `expandArray()` acquires the `headLock`. So the `data` array object should be thread safe if you acquires either `tailLock` or `headLock` before accessing the object.
`data` array elements are mutated and read with 2 different locks in the methods in `GrowableArrayBlockingQueue` class.
For example, `poll` is guarded by `headLock` and `put` is guarded by `tailLock`. I don't see how that could be thread safe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #16125: Use non-reentrant lock for GrowableArrayBlockingQueue tail
Posted by GitBox <gi...@apache.org>.
lhotari commented on PR #16125:
URL: https://github.com/apache/pulsar/pull/16125#issuecomment-1185456898
> > The data array is mutated and read under 2 separate locks which doesn't seem right.
>
> @lhotari All guarded by `headLock`?
@Jason918 it's not all guarded by `headLock`. for example, `put` isn't guarded by `headLock`.
There's a thread safety issue in `GrowableArrayBlockingQueue` since the `data` array is mutated and read under 2 separate locks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat merged pull request #16125: Use non-reentrant lock for GrowableArrayBlockingQueue tail
Posted by GitBox <gi...@apache.org>.
merlimat merged PR #16125:
URL: https://github.com/apache/pulsar/pull/16125
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on pull request #16125: Use non-reentrant lock for GrowableArrayBlockingQueue tail
Posted by GitBox <gi...@apache.org>.
Jason918 commented on PR #16125:
URL: https://github.com/apache/pulsar/pull/16125#issuecomment-1186699630
> > > The data array is mutated and read under 2 separate locks which doesn't seem right.
> >
> >
> > @lhotari All guarded by `headLock`?
>
> @Jason918 it's not the case that everything is guarded by `headLock`. for example, `put` isn't guarded by `headLock`. There's a thread safety issue in `GrowableArrayBlockingQueue` since the `data` array is mutated and read under 2 separate locks.
@lhotari The `data` array object mutation only happens in the code path `put() -> expandArray()`, where `put()` acquires the `tailLock`, and `expandArray()` acquires the `headLock`. So the `data` array object should be thread safe if you acquires either `tailLock` or `headLock` before accessing the object.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on pull request #16125: Use non-reentrant lock for GrowableArrayBlockingQueue tail
Posted by GitBox <gi...@apache.org>.
Jason918 commented on PR #16125:
URL: https://github.com/apache/pulsar/pull/16125#issuecomment-1159905076
> The data array is mutated and read under 2 separate locks which doesn't seem right.
@lhotari All guarded by `headLock`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org