You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/12/22 02:34:12 UTC

[GitHub] [pulsar] Demogorgon314 opened a new pull request, #19023: [improve][broker] PIP-192: Implement broker version filter for new load manager

Demogorgon314 opened a new pull request, #19023:
URL: https://github.com/apache/pulsar/pull/19023

   ### Motivation
   We will start raising PRs to implement PIP-192, https://github.com/apache/pulsar/issues/16691
   
   ### Modifications
   
   This PR Added `BrokerVersionFilter` to the load balancer extensions and its unit test.
   
   For the pip-192 project, this `BrokerVersionFilter`
   - Copy of the existing broker version filter, `org.apache.pulsar.broker.loadbalance.impl.BrokerVersionFilter`.
   - Change the interface base on https://github.com/apache/pulsar/pull/18810#discussion_r1053937530
   
   ### Documentation
   
   <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
   
   - [ ] `doc` <!-- Your PR contains doc changes. Please attach the local preview screenshots (run `sh start.sh` at `pulsar/site2/website`) to your PR description, or else your PR might not get merged. -->
   - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
   - [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
   - [ ] `doc-complete` <!-- Docs have been already added -->
   
   ### Matching PR in forked repository
   
   PR in forked repository: https://github.com/Demogorgon314/pulsar/pull/9<!-- ENTER URL HERE -->
   
   <!--
   After opening this PR, the build in apache/pulsar will fail and instructions will
   be provided for opening a PR in the PR author's forked repository.
   
   apache/pulsar pull requests should be first tested in your own fork since the 
   apache/pulsar CI based on GitHub Actions has constrained resources and quota.
   GitHub Actions provides separate quota for pull requests that are executed in 
   a forked repository.
   
   The tests will be run in the forked repository until all PR review comments have
   been handled, the tests pass and the PR is approved by a reviewer.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] Demogorgon314 commented on a diff in pull request #19023: [improve][broker] PIP-192: Implement broker version filter for new load manager

Posted by GitBox <gi...@apache.org>.
Demogorgon314 commented on code in PR #19023:
URL: https://github.com/apache/pulsar/pull/19023#discussion_r1058136911


##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/filter/BrokerVersionFilter.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.loadbalance.extensions.filter;
+
+import com.github.zafarkhaja.semver.Version;
+import java.util.Iterator;
+import java.util.Map;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.pulsar.broker.ServiceConfiguration;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterBadVersionException;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterException;
+import org.apache.pulsar.broker.loadbalance.extensions.LoadManagerContext;
+import org.apache.pulsar.broker.loadbalance.extensions.data.BrokerLookupData;
+
+/**
+ * Filter by broker version.
+ */
+@Slf4j
+public class BrokerVersionFilter implements BrokerFilter {
+
+    public static final String FILTER_NAME = "broker_version_filter";
+
+
+    /**
+     * From the given set of available broker candidates, filter those old brokers using the version numbers.
+     *
+     * @param brokers The currently available brokers that have not already been filtered.
+     * @param context The load manager context.
+     *
+     */
+    @Override
+    public Map<String, BrokerLookupData> filter(Map<String, BrokerLookupData> brokers, LoadManagerContext context)
+            throws BrokerFilterException {
+        ServiceConfiguration conf = context.brokerConfiguration();
+        if (!conf.isPreferLaterVersions() || brokers.isEmpty()) {
+            return brokers;
+        }
+
+        Version latestVersion;
+        try {
+            latestVersion = getLatestVersionNumber(brokers);
+            log.info("Latest broker version found was [{}]", latestVersion);
+        } catch (Exception ex) {
+            log.warn("Disabling PreferLaterVersions feature; reason: " + ex.getMessage());
+            throw new BrokerFilterBadVersionException("Cannot determine newest broker version: " + ex.getMessage());
+        }
+
+        int numBrokersLatestVersion = 0;
+        int numBrokersOlderVersion = 0;
+
+        Iterator<String> brokerIterator = brokers.keySet().iterator();

Review Comment:
   Good point!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] codelipenghui commented on pull request #19023: [improve][broker] PIP-192: Implement broker version filter for new load manager

Posted by GitBox <gi...@apache.org>.
codelipenghui commented on PR #19023:
URL: https://github.com/apache/pulsar/pull/19023#issuecomment-1367812287

   /pulsarbot run-failure-checks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] Technoboy- merged pull request #19023: [improve][broker] PIP-192: Implement broker version filter for new load manager

Posted by "Technoboy- (via GitHub)" <gi...@apache.org>.
Technoboy- merged PR #19023:
URL: https://github.com/apache/pulsar/pull/19023


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] codecov-commenter commented on pull request #19023: [improve][broker] PIP-192: Implement broker version filter for new load manager

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #19023:
URL: https://github.com/apache/pulsar/pull/19023#issuecomment-1367820207

   # [Codecov](https://codecov.io/gh/apache/pulsar/pull/19023?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#19023](https://codecov.io/gh/apache/pulsar/pull/19023?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a40e190) into [master](https://codecov.io/gh/apache/pulsar/commit/b36e012a36bba82a7303490e6b8d8f7b50929ce0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b36e012) will **decrease** coverage by `10.46%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pulsar/pull/19023/graphs/tree.svg?width=650&height=150&src=pr&token=acYqCpsK9J&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/19023?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #19023       +/-   ##
   =============================================
   - Coverage     47.74%   37.27%   -10.47%     
   + Complexity    10623     1990     -8633     
   =============================================
     Files           703      209      -494     
     Lines         68828    14452    -54376     
     Branches       7381     1577     -5804     
   =============================================
   - Hits          32861     5387    -27474     
   + Misses        32307     8479    -23828     
   + Partials       3660      586     -3074     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `37.27% <ø> (-10.47%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pulsar/pull/19023?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...he/pulsar/client/impl/PartitionedProducerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19023/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1BhcnRpdGlvbmVkUHJvZHVjZXJJbXBsLmphdmE=) | `30.34% <0.00%> (-5.13%)` | :arrow_down: |
   | [.../apache/pulsar/client/impl/BatchMessageIdImpl.java](https://codecov.io/gh/apache/pulsar/pull/19023/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0JhdGNoTWVzc2FnZUlkSW1wbC5qYXZh) | `67.50% <0.00%> (-4.73%)` | :arrow_down: |
   | [.../pulsar/client/impl/BatchMessageContainerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19023/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0JhdGNoTWVzc2FnZUNvbnRhaW5lckltcGwuamF2YQ==) | `55.95% <0.00%> (-1.02%)` | :arrow_down: |
   | [...va/org/apache/pulsar/client/impl/ProducerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19023/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VySW1wbC5qYXZh) | `16.25% <0.00%> (-0.75%)` | :arrow_down: |
   | [.../pulsar/client/impl/ProducerStatsRecorderImpl.java](https://codecov.io/gh/apache/pulsar/pull/19023/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VyU3RhdHNSZWNvcmRlckltcGwuamF2YQ==) | `84.04% <0.00%> (-0.62%)` | :arrow_down: |
   | [...va/org/apache/pulsar/client/impl/ConsumerBase.java](https://codecov.io/gh/apache/pulsar/pull/19023/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0NvbnN1bWVyQmFzZS5qYXZh) | `21.51% <0.00%> (-0.42%)` | :arrow_down: |
   | [...he/pulsar/client/impl/MultiTopicsConsumerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19023/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL011bHRpVG9waWNzQ29uc3VtZXJJbXBsLmphdmE=) | `22.80% <0.00%> (-0.07%)` | :arrow_down: |
   | [...va/org/apache/pulsar/client/impl/ConsumerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19023/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0NvbnN1bWVySW1wbC5qYXZh) | `15.06% <0.00%> (-0.06%)` | :arrow_down: |
   | [.../java/org/apache/pulsar/client/impl/ClientCnx.java](https://codecov.io/gh/apache/pulsar/pull/19023/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0NsaWVudENueC5qYXZh) | `29.97% <0.00%> (ø)` | |
   | [...ache/pulsar/client/impl/ZeroQueueConsumerImpl.java](https://codecov.io/gh/apache/pulsar/pull/19023/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1plcm9RdWV1ZUNvbnN1bWVySW1wbC5qYXZh) | `0.00% <0.00%> (ø)` | |
   | ... and [503 more](https://codecov.io/gh/apache/pulsar/pull/19023/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] Demogorgon314 commented on a diff in pull request #19023: [improve][broker] PIP-192: Implement broker version filter for new load manager

Posted by GitBox <gi...@apache.org>.
Demogorgon314 commented on code in PR #19023:
URL: https://github.com/apache/pulsar/pull/19023#discussion_r1058137020


##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/filter/BrokerVersionFilter.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.loadbalance.extensions.filter;
+
+import com.github.zafarkhaja.semver.Version;
+import java.util.Iterator;
+import java.util.Map;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.pulsar.broker.ServiceConfiguration;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterBadVersionException;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterException;
+import org.apache.pulsar.broker.loadbalance.extensions.LoadManagerContext;
+import org.apache.pulsar.broker.loadbalance.extensions.data.BrokerLookupData;
+
+/**
+ * Filter by broker version.
+ */
+@Slf4j
+public class BrokerVersionFilter implements BrokerFilter {
+
+    public static final String FILTER_NAME = "broker_version_filter";
+
+
+    /**
+     * From the given set of available broker candidates, filter those old brokers using the version numbers.
+     *
+     * @param brokers The currently available brokers that have not already been filtered.
+     * @param context The load manager context.
+     *
+     */
+    @Override
+    public Map<String, BrokerLookupData> filter(Map<String, BrokerLookupData> brokers, LoadManagerContext context)
+            throws BrokerFilterException {
+        ServiceConfiguration conf = context.brokerConfiguration();
+        if (!conf.isPreferLaterVersions() || brokers.isEmpty()) {
+            return brokers;
+        }
+
+        Version latestVersion;
+        try {
+            latestVersion = getLatestVersionNumber(brokers);
+            log.info("Latest broker version found was [{}]", latestVersion);

Review Comment:
   Updated



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] heesung-sn commented on a diff in pull request #19023: [improve][broker] PIP-192: Implement broker version filter for new load manager

Posted by GitBox <gi...@apache.org>.
heesung-sn commented on code in PR #19023:
URL: https://github.com/apache/pulsar/pull/19023#discussion_r1057840848


##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/filter/BrokerVersionFilter.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.loadbalance.extensions.filter;
+
+import com.github.zafarkhaja.semver.Version;
+import java.util.Iterator;
+import java.util.Map;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.pulsar.broker.ServiceConfiguration;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterBadVersionException;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterException;
+import org.apache.pulsar.broker.loadbalance.extensions.LoadManagerContext;
+import org.apache.pulsar.broker.loadbalance.extensions.data.BrokerLookupData;
+
+/**
+ * Filter by broker version.
+ */
+@Slf4j
+public class BrokerVersionFilter implements BrokerFilter {
+
+    public static final String FILTER_NAME = "broker_version_filter";
+
+
+    /**
+     * From the given set of available broker candidates, filter those old brokers using the version numbers.
+     *
+     * @param brokers The currently available brokers that have not already been filtered.
+     * @param context The load manager context.
+     *
+     */
+    @Override
+    public Map<String, BrokerLookupData> filter(Map<String, BrokerLookupData> brokers, LoadManagerContext context)
+            throws BrokerFilterException {
+        ServiceConfiguration conf = context.brokerConfiguration();
+        if (!conf.isPreferLaterVersions() || brokers.isEmpty()) {
+            return brokers;
+        }
+
+        Version latestVersion;
+        try {
+            latestVersion = getLatestVersionNumber(brokers);
+            log.info("Latest broker version found was [{}]", latestVersion);
+        } catch (Exception ex) {
+            log.warn("Disabling PreferLaterVersions feature; reason: " + ex.getMessage());
+            throw new BrokerFilterBadVersionException("Cannot determine newest broker version: " + ex.getMessage());
+        }
+
+        int numBrokersLatestVersion = 0;
+        int numBrokersOlderVersion = 0;
+
+        Iterator<String> brokerIterator = brokers.keySet().iterator();

Review Comment:
   entrySet().iterator() to reduce the brokers.get(brokerId) lookup cost?
   
   ref: https://findbugs.sourceforge.net/bugDescriptions.html#WMI_WRONG_MAP_ITERATOR



##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/filter/BrokerVersionFilter.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.loadbalance.extensions.filter;
+
+import com.github.zafarkhaja.semver.Version;
+import java.util.Iterator;
+import java.util.Map;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.pulsar.broker.ServiceConfiguration;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterBadVersionException;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterException;
+import org.apache.pulsar.broker.loadbalance.extensions.LoadManagerContext;
+import org.apache.pulsar.broker.loadbalance.extensions.data.BrokerLookupData;
+
+/**
+ * Filter by broker version.
+ */
+@Slf4j
+public class BrokerVersionFilter implements BrokerFilter {
+
+    public static final String FILTER_NAME = "broker_version_filter";
+
+
+    /**
+     * From the given set of available broker candidates, filter those old brokers using the version numbers.
+     *
+     * @param brokers The currently available brokers that have not already been filtered.
+     * @param context The load manager context.
+     *
+     */
+    @Override
+    public Map<String, BrokerLookupData> filter(Map<String, BrokerLookupData> brokers, LoadManagerContext context)
+            throws BrokerFilterException {
+        ServiceConfiguration conf = context.brokerConfiguration();
+        if (!conf.isPreferLaterVersions() || brokers.isEmpty()) {
+            return brokers;
+        }
+
+        Version latestVersion;
+        try {
+            latestVersion = getLatestVersionNumber(brokers);
+            log.info("Latest broker version found was [{}]", latestVersion);

Review Comment:
   Do this print log too much? Shouldn't it be log.debug?



##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/filter/BrokerVersionFilter.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.loadbalance.extensions.filter;
+
+import com.github.zafarkhaja.semver.Version;
+import java.util.Iterator;
+import java.util.Map;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.pulsar.broker.ServiceConfiguration;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterBadVersionException;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterException;
+import org.apache.pulsar.broker.loadbalance.extensions.LoadManagerContext;
+import org.apache.pulsar.broker.loadbalance.extensions.data.BrokerLookupData;
+
+/**
+ * Filter by broker version.
+ */
+@Slf4j
+public class BrokerVersionFilter implements BrokerFilter {
+
+    public static final String FILTER_NAME = "broker_version_filter";
+
+
+    /**
+     * From the given set of available broker candidates, filter those old brokers using the version numbers.
+     *
+     * @param brokers The currently available brokers that have not already been filtered.
+     * @param context The load manager context.
+     *
+     */
+    @Override
+    public Map<String, BrokerLookupData> filter(Map<String, BrokerLookupData> brokers, LoadManagerContext context)
+            throws BrokerFilterException {
+        ServiceConfiguration conf = context.brokerConfiguration();
+        if (!conf.isPreferLaterVersions() || brokers.isEmpty()) {
+            return brokers;
+        }
+
+        Version latestVersion;
+        try {
+            latestVersion = getLatestVersionNumber(brokers);
+            log.info("Latest broker version found was [{}]", latestVersion);
+        } catch (Exception ex) {
+            log.warn("Disabling PreferLaterVersions feature; reason: " + ex.getMessage());
+            throw new BrokerFilterBadVersionException("Cannot determine newest broker version: " + ex.getMessage());

Review Comment:
   If an exception is thrown here, do we skip this filter?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] Demogorgon314 commented on a diff in pull request #19023: [improve][broker] PIP-192: Implement broker version filter for new load manager

Posted by GitBox <gi...@apache.org>.
Demogorgon314 commented on code in PR #19023:
URL: https://github.com/apache/pulsar/pull/19023#discussion_r1058130912


##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/filter/BrokerVersionFilter.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.loadbalance.extensions.filter;
+
+import com.github.zafarkhaja.semver.Version;
+import java.util.Iterator;
+import java.util.Map;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.pulsar.broker.ServiceConfiguration;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterBadVersionException;
+import org.apache.pulsar.broker.loadbalance.BrokerFilterException;
+import org.apache.pulsar.broker.loadbalance.extensions.LoadManagerContext;
+import org.apache.pulsar.broker.loadbalance.extensions.data.BrokerLookupData;
+
+/**
+ * Filter by broker version.
+ */
+@Slf4j
+public class BrokerVersionFilter implements BrokerFilter {
+
+    public static final String FILTER_NAME = "broker_version_filter";
+
+
+    /**
+     * From the given set of available broker candidates, filter those old brokers using the version numbers.
+     *
+     * @param brokers The currently available brokers that have not already been filtered.
+     * @param context The load manager context.
+     *
+     */
+    @Override
+    public Map<String, BrokerLookupData> filter(Map<String, BrokerLookupData> brokers, LoadManagerContext context)
+            throws BrokerFilterException {
+        ServiceConfiguration conf = context.brokerConfiguration();
+        if (!conf.isPreferLaterVersions() || brokers.isEmpty()) {
+            return brokers;
+        }
+
+        Version latestVersion;
+        try {
+            latestVersion = getLatestVersionNumber(brokers);
+            log.info("Latest broker version found was [{}]", latestVersion);
+        } catch (Exception ex) {
+            log.warn("Disabling PreferLaterVersions feature; reason: " + ex.getMessage());
+            throw new BrokerFilterBadVersionException("Cannot determine newest broker version: " + ex.getMessage());

Review Comment:
   Yes, we should handle the exception when using the filter.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org