You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/12/08 05:55:54 UTC
[GitHub] [pulsar] mattisonchao opened a new pull request, #18816: [fix][common] Fix unexpected behaviour by invoke `PulsarConfigurationLoader#convertFrom`
mattisonchao opened a new pull request, #18816:
URL: https://github.com/apache/pulsar/pull/18816
### Motivation
This regression was introduced by #16234; I found it by setting `proxyAdditionalServlets ` on the proxy, but it didn't work. the reason is the `proxyConfig.getProperties().getProperty("proxyAdditionalServlets")` returns `null` after invoking `PulsarConfigurationLoader#convertFrom`.
The RC is the source config and the new config uses the same properties then we set the unknown field to the new config without any type check. I'm not sure if it's better to add the check here, but ignore to use the properties field can fix the regression.
### Modifications
- Don't use the same properties object when converting.
### Verifying this change
- [x] Make sure that the change passes the CI checks.
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] mattisonchao commented on a diff in pull request #18816: [fix][broker] Fix unexpected behaviour by invoke `PulsarConfigurationLoader#convertFrom`
Posted by GitBox <gi...@apache.org>.
mattisonchao commented on code in PR #18816:
URL: https://github.com/apache/pulsar/pull/18816#discussion_r1043032253
##########
pulsar-broker-common/src/main/java/org/apache/pulsar/common/configuration/PulsarConfigurationLoader.java:
##########
@@ -187,7 +187,8 @@ public static ServiceConfiguration convertFrom(PulsarConfiguration conf, boolean
try {
confField.setAccessible(true);
Field convertedConfField = ServiceConfiguration.class.getDeclaredField(confField.getName());
- if (!Modifier.isStatic(convertedConfField.getModifiers())) {
+ if (!Modifier.isStatic(convertedConfField.getModifiers())
+ && convertedConfField.getDeclaredAnnotation(FieldContext.class) != null) {
Review Comment:
fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codecov-commenter commented on pull request #18816: [fix][broker] Fix unexpected behaviour by invoke `PulsarConfigurationLoader#convertFrom`
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #18816:
URL: https://github.com/apache/pulsar/pull/18816#issuecomment-1342114230
# [Codecov](https://codecov.io/gh/apache/pulsar/pull/18816?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#18816](https://codecov.io/gh/apache/pulsar/pull/18816?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2e4511e) into [master](https://codecov.io/gh/apache/pulsar/commit/68ca60cea7665e6a4bd9f07518b038c17893fe02?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (68ca60c) will **decrease** coverage by `12.98%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/pulsar/pull/18816?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #18816 +/- ##
=============================================
- Coverage 50.05% 37.07% -12.99%
+ Complexity 11024 1969 -9055
=============================================
Files 703 209 -494
Lines 68814 14425 -54389
Branches 7378 1574 -5804
=============================================
- Hits 34446 5348 -29098
+ Misses 30621 8492 -22129
+ Partials 3747 585 -3162
```
| Flag | Coverage Δ | |
|---|---|---|
| unittests | `37.07% <ø> (-12.99%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pulsar/pull/18816?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../org/apache/pulsar/client/impl/ConnectionPool.java](https://codecov.io/gh/apache/pulsar/pull/18816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0Nvbm5lY3Rpb25Qb29sLmphdmE=) | `37.43% <0.00%> (-1.03%)` | :arrow_down: |
| [...rg/apache/pulsar/client/impl/PulsarClientImpl.java](https://codecov.io/gh/apache/pulsar/pull/18816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1B1bHNhckNsaWVudEltcGwuamF2YQ==) | `43.85% <0.00%> (-0.95%)` | :arrow_down: |
| [...ersistentStickyKeyDispatcherMultipleConsumers.java](https://codecov.io/gh/apache/pulsar/pull/18816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3BlcnNpc3RlbnQvUGVyc2lzdGVudFN0aWNreUtleURpc3BhdGNoZXJNdWx0aXBsZUNvbnN1bWVycy5qYXZh) | | |
| [...r/stats/prometheus/PrometheusMetricsGenerator.java](https://codecov.io/gh/apache/pulsar/pull/18816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL1Byb21ldGhldXNNZXRyaWNzR2VuZXJhdG9yLmphdmE=) | | |
| [...org/apache/pulsar/broker/service/Subscription.java](https://codecov.io/gh/apache/pulsar/pull/18816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1N1YnNjcmlwdGlvbi5qYXZh) | | |
| [...sar/broker/loadbalance/impl/PulsarServiceUnit.java](https://codecov.io/gh/apache/pulsar/pull/18816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9pbXBsL1B1bHNhclNlcnZpY2VVbml0LmphdmE=) | | |
| [...r/service/schema/AvroSchemaCompatibilityCheck.java](https://codecov.io/gh/apache/pulsar/pull/18816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3NjaGVtYS9BdnJvU2NoZW1hQ29tcGF0aWJpbGl0eUNoZWNrLmphdmE=) | | |
| [...ion/pendingack/impl/MLPendingAckStoreProvider.java](https://codecov.io/gh/apache/pulsar/pull/18816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci90cmFuc2FjdGlvbi9wZW5kaW5nYWNrL2ltcGwvTUxQZW5kaW5nQWNrU3RvcmVQcm92aWRlci5qYXZh) | | |
| [...kkeeper/mledger/impl/ManagedLedgerFactoryImpl.java](https://codecov.io/gh/apache/pulsar/pull/18816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bWFuYWdlZC1sZWRnZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2Jvb2trZWVwZXIvbWxlZGdlci9pbXBsL01hbmFnZWRMZWRnZXJGYWN0b3J5SW1wbC5qYXZh) | | |
| [.../pulsar/broker/systopic/SystemTopicClientBase.java](https://codecov.io/gh/apache/pulsar/pull/18816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zeXN0b3BpYy9TeXN0ZW1Ub3BpY0NsaWVudEJhc2UuamF2YQ==) | | |
| ... and [487 more](https://codecov.io/gh/apache/pulsar/pull/18816/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- closed pull request #18816: [fix][common] Fix unexpected behaviour by invoke `PulsarConfigurationLoader#convertFrom`
Posted by GitBox <gi...@apache.org>.
Technoboy- closed pull request #18816: [fix][common] Fix unexpected behaviour by invoke `PulsarConfigurationLoader#convertFrom`
URL: https://github.com/apache/pulsar/pull/18816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui merged pull request #18816: [fix][broker] Fix unexpected behaviour by invoke `PulsarConfigurationLoader#convertFrom`
Posted by GitBox <gi...@apache.org>.
codelipenghui merged PR #18816:
URL: https://github.com/apache/pulsar/pull/18816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] mattisonchao commented on a diff in pull request #18816: [fix][broker] Fix unexpected behaviour by invoke `PulsarConfigurationLoader#convertFrom`
Posted by GitBox <gi...@apache.org>.
mattisonchao commented on code in PR #18816:
URL: https://github.com/apache/pulsar/pull/18816#discussion_r1043011267
##########
pulsar-broker-common/src/main/java/org/apache/pulsar/common/configuration/PulsarConfigurationLoader.java:
##########
@@ -187,7 +187,8 @@ public static ServiceConfiguration convertFrom(PulsarConfiguration conf, boolean
try {
confField.setAccessible(true);
Field convertedConfField = ServiceConfiguration.class.getDeclaredField(confField.getName());
- if (!Modifier.isStatic(convertedConfField.getModifiers())) {
+ if (!Modifier.isStatic(convertedConfField.getModifiers())
+ && convertedConfField.getDeclaredAnnotation(FieldContext.class) != null) {
Review Comment:
Sorry, I'm fixing it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on a diff in pull request #18816: [fix][broker] Fix unexpected behaviour by invoke `PulsarConfigurationLoader#convertFrom`
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on code in PR #18816:
URL: https://github.com/apache/pulsar/pull/18816#discussion_r1042991922
##########
pulsar-proxy/src/test/java/org/apache/pulsar/proxy/server/ProxyConfigurationTest.java:
##########
@@ -110,4 +110,26 @@ public void testBackwardCompatibility() throws IOException {
assertEquals(serviceConfig.getMetadataStoreSessionTimeoutMillis(), 100);
assertEquals(serviceConfig.getMetadataStoreCacheExpirySeconds(), 300);
}
+
+
+ @Test
+ public void testConvert() throws IOException {
+ File testConfigFile = new File("tmp." + System.currentTimeMillis() + ".properties");
+ if (testConfigFile.exists()) {
+ testConfigFile.delete();
+ }
+ try (PrintWriter printWriter = new PrintWriter(new OutputStreamWriter(new FileOutputStream(testConfigFile)))) {
+ printWriter.println("proxyAdditionalServlets=a,b,c");
+ }
+ testConfigFile.deleteOnExit();
+ try(InputStream stream = new FileInputStream(testConfigFile)) {
+ ProxyConfiguration proxyConfig = PulsarConfigurationLoader.create(stream, ProxyConfiguration.class);
+ assertEquals(proxyConfig.getProperties().getProperty("proxyAdditionalServlets"), "a,b,c");
+ assertEquals(proxyConfig.getProxyAdditionalServlets().size(), 3);
+ PulsarConfigurationLoader.convertFrom(proxyConfig);
Review Comment:
Do we need to check the return value of this method?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] mattisonchao commented on a diff in pull request #18816: [fix][broker] Fix unexpected behaviour by invoke `PulsarConfigurationLoader#convertFrom`
Posted by GitBox <gi...@apache.org>.
mattisonchao commented on code in PR #18816:
URL: https://github.com/apache/pulsar/pull/18816#discussion_r1042993394
##########
pulsar-proxy/src/test/java/org/apache/pulsar/proxy/server/ProxyConfigurationTest.java:
##########
@@ -110,4 +110,26 @@ public void testBackwardCompatibility() throws IOException {
assertEquals(serviceConfig.getMetadataStoreSessionTimeoutMillis(), 100);
assertEquals(serviceConfig.getMetadataStoreCacheExpirySeconds(), 300);
}
+
+
+ @Test
+ public void testConvert() throws IOException {
+ File testConfigFile = new File("tmp." + System.currentTimeMillis() + ".properties");
+ if (testConfigFile.exists()) {
+ testConfigFile.delete();
+ }
+ try (PrintWriter printWriter = new PrintWriter(new OutputStreamWriter(new FileOutputStream(testConfigFile)))) {
+ printWriter.println("proxyAdditionalServlets=a,b,c");
+ }
+ testConfigFile.deleteOnExit();
+ try(InputStream stream = new FileInputStream(testConfigFile)) {
+ ProxyConfiguration proxyConfig = PulsarConfigurationLoader.create(stream, ProxyConfiguration.class);
+ assertEquals(proxyConfig.getProperties().getProperty("proxyAdditionalServlets"), "a,b,c");
+ assertEquals(proxyConfig.getProxyAdditionalServlets().size(), 3);
+ PulsarConfigurationLoader.convertFrom(proxyConfig);
Review Comment:
No, This PR just want to check the regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on a diff in pull request #18816: [fix][broker] Fix unexpected behaviour by invoke `PulsarConfigurationLoader#convertFrom`
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on code in PR #18816:
URL: https://github.com/apache/pulsar/pull/18816#discussion_r1042994631
##########
pulsar-proxy/src/test/java/org/apache/pulsar/proxy/server/ProxyConfigurationTest.java:
##########
@@ -110,4 +110,26 @@ public void testBackwardCompatibility() throws IOException {
assertEquals(serviceConfig.getMetadataStoreSessionTimeoutMillis(), 100);
assertEquals(serviceConfig.getMetadataStoreCacheExpirySeconds(), 300);
}
+
+
+ @Test
+ public void testConvert() throws IOException {
+ File testConfigFile = new File("tmp." + System.currentTimeMillis() + ".properties");
+ if (testConfigFile.exists()) {
+ testConfigFile.delete();
+ }
+ try (PrintWriter printWriter = new PrintWriter(new OutputStreamWriter(new FileOutputStream(testConfigFile)))) {
+ printWriter.println("proxyAdditionalServlets=a,b,c");
+ }
+ testConfigFile.deleteOnExit();
+ try(InputStream stream = new FileInputStream(testConfigFile)) {
+ ProxyConfiguration proxyConfig = PulsarConfigurationLoader.create(stream, ProxyConfiguration.class);
+ assertEquals(proxyConfig.getProperties().getProperty("proxyAdditionalServlets"), "a,b,c");
+ assertEquals(proxyConfig.getProxyAdditionalServlets().size(), 3);
+ PulsarConfigurationLoader.convertFrom(proxyConfig);
Review Comment:
ok, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #18816: [fix][common] Fix unexpected value by invoke `PulsarConfigurationLoader#convertFrom`
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #18816:
URL: https://github.com/apache/pulsar/pull/18816#issuecomment-1342084773
@mattisonchao Please add the following content to your PR description and select a checkbox:
```
- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on a diff in pull request #18816: [fix][broker] Fix unexpected behaviour by invoke `PulsarConfigurationLoader#convertFrom`
Posted by GitBox <gi...@apache.org>.
Jason918 commented on code in PR #18816:
URL: https://github.com/apache/pulsar/pull/18816#discussion_r1043002514
##########
pulsar-broker-common/src/main/java/org/apache/pulsar/common/configuration/PulsarConfigurationLoader.java:
##########
@@ -187,7 +187,8 @@ public static ServiceConfiguration convertFrom(PulsarConfiguration conf, boolean
try {
confField.setAccessible(true);
Field convertedConfField = ServiceConfiguration.class.getDeclaredField(confField.getName());
- if (!Modifier.isStatic(convertedConfField.getModifiers())) {
+ if (!Modifier.isStatic(convertedConfField.getModifiers())
+ && convertedConfField.getDeclaredAnnotation(FieldContext.class) != null) {
Review Comment:
Does this mean that the `properties` in the `conf` won't be appeared in the returned ServiceConfiguration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org