You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/09/01 10:04:20 UTC
[GitHub] [pulsar] andrasbeni opened a new pull request, #17396: [improve][test] Add integration test for entry filters
andrasbeni opened a new pull request, #17396:
URL: https://github.com/apache/pulsar/pull/17396
Master Issue: #17132
### Motivation
Add tests to prevent unintended changes to how the broker uses Entry Filters.
### Modifications
- Created an entry filter, `PatternEntryFilter`.
- Created a test that configures the entry filter for the brokers and verifies that the entry filter functions correctly
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
This change added tests and can be verified as follows:
*Added integration tests for entry filters*
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (no)
- The public API: (no)
- The schema: (no)
- The default values of configurations: (no)
- The wire protocol: (no)
- The rest endpoints: (no)
- The admin cli options: (no)
- Anything that affects deployment: (no)
### Documentation
- [x] `doc-not-needed`
This change only affects tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] andrasbeni commented on pull request #17396: [improve][test] Add integration test for entry filters
Posted by GitBox <gi...@apache.org>.
andrasbeni commented on PR #17396:
URL: https://github.com/apache/pulsar/pull/17396#issuecomment-1237910010
/pulsarbot rerun-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] andrasbeni commented on a diff in pull request #17396: [improve][test] Add integration test for entry filters
Posted by GitBox <gi...@apache.org>.
andrasbeni commented on code in PR #17396:
URL: https://github.com/apache/pulsar/pull/17396#discussion_r965747819
##########
tests/integration/src/test/java/org/apache/pulsar/tests/integration/plugins/TestEntryFilters.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.tests.integration.plugins;
+
+import static org.testng.Assert.assertNotNull;
+import lombok.Cleanup;
+import org.apache.pulsar.client.api.Consumer;
+import org.apache.pulsar.client.api.ConsumerBuilder;
+import org.apache.pulsar.client.api.MessageId;
+import org.apache.pulsar.client.api.Producer;
+import org.apache.pulsar.client.api.PulsarClient;
+import org.apache.pulsar.client.api.Schema;
+import org.apache.pulsar.client.api.SubscriptionInitialPosition;
+import org.apache.pulsar.client.api.SubscriptionType;
+import org.apache.pulsar.tests.integration.messaging.TopicMessagingBase;
+import org.testng.annotations.Test;
+import java.util.Collections;
+import java.util.function.Supplier;
+
+public class TestEntryFilters extends TopicMessagingBase {
+
+ private static final String PREFIX = "PULSAR_PREFIX_";
+
+ @Override
+ public void setupCluster() throws Exception {
+ brokerEnvs.put(PREFIX + "entryFilterNames", "pattern_filter");
+ brokerEnvs.put(PREFIX + "entryFiltersDirectory", "/pulsar/examples");
+ super.setupCluster();
+ }
+
+ @Test(dataProvider = "ServiceUrls")
+ public void test(Supplier<String> serviceUrlSupplier) throws Exception {
+ String serviceUrl = serviceUrlSupplier.get();
+
+ final String topicName = getNonPartitionedTopic("filtered-topic", true);
+ @Cleanup
+ final PulsarClient client = PulsarClient.builder()
+ .serviceUrl(serviceUrl)
+ .build();
+
+ String evenPattern = "^[a-z]+-\\d*[02468]$";
+
+ @Cleanup
+ final Producer<String> producer = client.newProducer(Schema.STRING)
+ .topic(topicName)
+ .enableBatching(false)
+ .producerName("producer")
+ .create();
+ int messagesToSend = 20;
+ for (int i = 0; i < messagesToSend; i++) {
+ String messageValue = producer.getProducerName() + "-" + i;
+ MessageId messageId = producer.newMessage()
+ .value(messageValue)
+ .property("filter_property", messageValue)
Review Comment:
I agree it would be cleaner, but I don't think this alone is a strong enough reason to introduce a dependency between the two maven modules these classes are in, so I decided to duplicate this string instead. Let me know what you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] andrasbeni commented on pull request #17396: [improve][test] Add integration test for entry filters
Posted by GitBox <gi...@apache.org>.
andrasbeni commented on PR #17396:
URL: https://github.com/apache/pulsar/pull/17396#issuecomment-1242740557
/pulsarbot rerun pulsar-ci-checks-completed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] andrasbeni commented on pull request #17396: [improve][test] Add integration test for entry filters
Posted by GitBox <gi...@apache.org>.
andrasbeni commented on PR #17396:
URL: https://github.com/apache/pulsar/pull/17396#issuecomment-1236073047
/pulsarbot rerun-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] andrasbeni commented on pull request #17396: [improve][test] Add integration test for entry filters
Posted by GitBox <gi...@apache.org>.
andrasbeni commented on PR #17396:
URL: https://github.com/apache/pulsar/pull/17396#issuecomment-1248069713
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] andrasbeni commented on pull request #17396: [improve][test] Add integration test for entry filters
Posted by GitBox <gi...@apache.org>.
andrasbeni commented on PR #17396:
URL: https://github.com/apache/pulsar/pull/17396#issuecomment-1244114298
/pulsarbot rerun pulsar-ci-checks-completed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui merged pull request #17396: [improve][test] Add integration test for entry filters
Posted by GitBox <gi...@apache.org>.
codelipenghui merged PR #17396:
URL: https://github.com/apache/pulsar/pull/17396
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] andrasbeni commented on pull request #17396: [improve][test] Add integration test for entry filters
Posted by GitBox <gi...@apache.org>.
andrasbeni commented on PR #17396:
URL: https://github.com/apache/pulsar/pull/17396#issuecomment-1244119747
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on a diff in pull request #17396: [improve][test] Add integration test for entry filters
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on code in PR #17396:
URL: https://github.com/apache/pulsar/pull/17396#discussion_r965733347
##########
tests/integration/src/test/java/org/apache/pulsar/tests/integration/plugins/TestEntryFilters.java:
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.tests.integration.plugins;
+
+import static org.testng.Assert.assertNotNull;
+import lombok.Cleanup;
+import org.apache.pulsar.client.api.Consumer;
+import org.apache.pulsar.client.api.ConsumerBuilder;
+import org.apache.pulsar.client.api.MessageId;
+import org.apache.pulsar.client.api.Producer;
+import org.apache.pulsar.client.api.PulsarClient;
+import org.apache.pulsar.client.api.Schema;
+import org.apache.pulsar.client.api.SubscriptionInitialPosition;
+import org.apache.pulsar.client.api.SubscriptionType;
+import org.apache.pulsar.tests.integration.messaging.TopicMessagingBase;
+import org.testng.annotations.Test;
+import java.util.Collections;
+import java.util.function.Supplier;
+
+public class TestEntryFilters extends TopicMessagingBase {
+
+ private static final String PREFIX = "PULSAR_PREFIX_";
+
+ @Override
+ public void setupCluster() throws Exception {
+ brokerEnvs.put(PREFIX + "entryFilterNames", "pattern_filter");
+ brokerEnvs.put(PREFIX + "entryFiltersDirectory", "/pulsar/examples");
+ super.setupCluster();
+ }
+
+ @Test(dataProvider = "ServiceUrls")
+ public void test(Supplier<String> serviceUrlSupplier) throws Exception {
+ String serviceUrl = serviceUrlSupplier.get();
+
+ final String topicName = getNonPartitionedTopic("filtered-topic", true);
+ @Cleanup
+ final PulsarClient client = PulsarClient.builder()
+ .serviceUrl(serviceUrl)
+ .build();
+
+ String evenPattern = "^[a-z]+-\\d*[02468]$";
+
+ @Cleanup
+ final Producer<String> producer = client.newProducer(Schema.STRING)
+ .topic(topicName)
+ .enableBatching(false)
+ .producerName("producer")
+ .create();
+ int messagesToSend = 20;
+ for (int i = 0; i < messagesToSend; i++) {
+ String messageValue = producer.getProducerName() + "-" + i;
+ MessageId messageId = producer.newMessage()
+ .value(messageValue)
+ .property("filter_property", messageValue)
Review Comment:
Maybe we can use the static param `FILTER_PROPERTY` of the class `PatternEntryFilter.java` as the key here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org