[PR] [fix][broker] Serialize metadata-store topic policy notifications [pulsar]

["void-ptr974 (via GitHub)" <gi...@apache.org>]

void-ptr974 opened a new pull request, #26974:
URL: https://github.com/apache/pulsar/pull/26974

   ### Motivation
   
   Metadata-store topic policy updates currently call listeners directly from cache completion and metadata notification threads. Local and global policy callbacks for the same topic can run concurrently and bypass the topic notification thread used by the system-topic implementation. Moving every event directly onto that executor would also allow a blocked callback to accumulate duplicate work.
   
   ### Modifications
   
   - Dispatch local, global, and deletion callbacks through `BrokerService.getTopicPoliciesNotifyThread` for the normalized topic.
   - Keep one refresh chain per metadata path through reading, queued completion, and listener invocation. Coalesce newer events and reread the latest policies when needed; discard superseded reads and queued work from a previous listener registration or a closed service.
   - Coalesce consecutive pending deletions across scopes, while preserving deletion batches separated by a queued scoped completion.
   - Handle executor rejection without invoking listeners inline. A rejected deletion reloads the latest policies; a rejected ordinary completion clears its pending state so a later event can refresh again.
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   
   Added deterministic tests using controlled futures, latches, and a real ordered executor. They cover thread affinity and serialization, update storms during reads/queued completions/active callbacks, deletion storms, deletion ordering, listener replacement, stale results, closure, and executor rejection.
   
   Local validation (automatic test retries disabled):
   
   ```text
   ./gradlew quickCheck :pulsar-broker:test -PtestRetryCount=0 -PtestFailFast=false \
     --tests 'org.apache.pulsar.broker.service.MetadataStoreTopicPoliciesServiceTest' \
     --tests 'org.apache.pulsar.broker.admin.MetadataStoreTopicPoliciesTest.testPriorityOfGlobalPolicies' \
     --tests 'org.apache.pulsar.broker.admin.MetadataStoreTopicPoliciesTest.testPolicyOverwrittenByNamespaceLevel' \
     --offline
   ```
   
   All 18 new regression scenarios and 3 existing integration test executions passed. `quickCheck` passed. Seven selected regression scenarios also failed against the unpatched `origin/master` implementation for the expected inline-callback or duplicate-read behavior, and passed with the fix.
   
   ### Does this pull request potentially affect one of the following parts:
   
   - [ ] Dependencies (add or upgrade a dependency)
   - [ ] The public API
   - [ ] The schema
   - [ ] The default values of configurations
   - [x] The threading model: metadata-store topic policy listeners now use the existing per-topic notification executor.
   - [ ] The binary protocol
   - [ ] The REST endpoints
   - [ ] The admin CLI options
   - [ ] The metrics
   - [ ] Anything that affects deployment
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [fix][broker] Fix metadata-store topic policy callbacks running on the wrong thread [pulsar]

["void-ptr974 (via GitHub)" <gi...@apache.org>]

void-ptr974 closed pull request #26974: [fix][broker] Fix metadata-store topic policy callbacks running on the wrong thread
URL: https://github.com/apache/pulsar/pull/26974


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org