You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/02/10 04:40:12 UTC
[GitHub] [pulsar] zymap opened a new pull request #9551: Avoid introduce bookkeeper-common into the pulsar-common
zymap opened a new pull request #9551:
URL: https://github.com/apache/pulsar/pull/9551
---
*Motivation*
Direct using jackson to parse json to avoid introduce the bookkeeper-common
into the pulsar-common. That will make other modules which are using pulsar-common
has some unused bookkeeper dependencies.
<!--
### Contribution Checklist
- Name the pull request in the form "[Issue XYZ][component] Title of the pull request", where *XYZ* should be replaced by the actual issue number.
Skip *Issue XYZ* if there is no associated github issue for this pull request.
Skip *component* if you are unsure about which is the best component. E.g. `[docs] Fix typo in produce method`.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message
- Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
*(If this PR fixes a github issue, please add `Fixes #<xyz>`.)*
Fixes #<xyz>
*(or if this PR is one task of a github issue, please add `Master Issue: #<xyz>` to link to the master issue.)*
Master Issue: #<xyz>
### Motivation
*Explain here the context, and why you're making that change. What is the problem you're trying to solve.*
### Modifications
*Describe the modifications you've done.*
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads (10MB)*
- *Extended integration test for recovery after broker failure*
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API: (yes / no)
- The schema: (yes / no / don't know)
- The default values of configurations: (yes / no)
- The wire protocol: (yes / no)
- The rest endpoints: (yes / no)
- The admin cli options: (yes / no)
- Anything that affects deployment: (yes / no / don't know)
### Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
- If a feature is not applicable for documentation, explain why?
- If a feature is not documented yet in this PR, please create a followup issue for adding the documentation
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on a change in pull request #9551: Avoid introducing bookkeeper-common into the pulsar-common
Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #9551:
URL: https://github.com/apache/pulsar/pull/9551#discussion_r573521848
##########
File path: pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/EnsemblePlacementPolicyConfig.java
##########
@@ -30,6 +31,8 @@
private final Class policyClass;
private final Map<String, Object> properties;
+ private static ObjectMapper mapper = new ObjectMapper();
Review comment:
btw using consistently ObjectMapperFactory.getThreadLocal() is good to me
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on a change in pull request #9551: Avoid introducing bookkeeper-common into the pulsar-common
Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #9551:
URL: https://github.com/apache/pulsar/pull/9551#discussion_r573521447
##########
File path: pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/EnsemblePlacementPolicyConfig.java
##########
@@ -30,6 +31,8 @@
private final Class policyClass;
private final Map<String, Object> properties;
+ private static ObjectMapper mapper = new ObjectMapper();
Review comment:
@merlimat I am not sure I get your point.
AFAIK it is very common to have static fields holding ObjectMapper if you do not need to change the configuration.
Usually you create one ObjectMapper per class or per group of classes (internally it holds a cache)
https://stackoverflow.com/questions/3907929/should-i-declare-jacksons-objectmapper-as-a-static-field
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on a change in pull request #9551: Avoid introducing bookkeeper-common into the pulsar-common
Posted by GitBox <gi...@apache.org>.
merlimat commented on a change in pull request #9551:
URL: https://github.com/apache/pulsar/pull/9551#discussion_r573495082
##########
File path: pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/EnsemblePlacementPolicyConfig.java
##########
@@ -30,6 +31,8 @@
private final Class policyClass;
private final Map<String, Object> properties;
+ private static ObjectMapper mapper = new ObjectMapper();
Review comment:
This is not thread safe and also is not used the preferred configuration for Jackson. `pulsar-common` has already `ObjectMapperFactory.getThreadLocal()` which is used for this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on a change in pull request #9551: Avoid introducing bookkeeper-common into the pulsar-common
Posted by GitBox <gi...@apache.org>.
merlimat commented on a change in pull request #9551:
URL: https://github.com/apache/pulsar/pull/9551#discussion_r573927273
##########
File path: pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/EnsemblePlacementPolicyConfig.java
##########
@@ -30,6 +31,8 @@
private final Class policyClass;
private final Map<String, Object> properties;
+ private static ObjectMapper mapper = new ObjectMapper();
Review comment:
We've seen concurrency issues in the past resulting in random exceptions. If I'm not remembering wrong, they were related to the caching of the classes introspection in the object mapper.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat merged pull request #9551: Avoid introducing bookkeeper-common into the pulsar-common
Posted by GitBox <gi...@apache.org>.
merlimat merged pull request #9551:
URL: https://github.com/apache/pulsar/pull/9551
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #9551: Avoid introducing bookkeeper-common into the pulsar-common
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #9551:
URL: https://github.com/apache/pulsar/pull/9551#issuecomment-778033371
/pulsarbot run-failure-tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on a change in pull request #9551: Avoid introducing bookkeeper-common into the pulsar-common
Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #9551:
URL: https://github.com/apache/pulsar/pull/9551#discussion_r573504656
##########
File path: pulsar-common/src/test/java/org/apache/pulsar/common/policies/data/EnsemblePlacementPolicyConfigTest.java
##########
@@ -0,0 +1,54 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.common.policies.data;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import org.testng.Assert;
+import org.testng.annotations.Test;
+
+import java.util.Collections;
+
+public class EnsemblePlacementPolicyConfigTest {
+
+ static class MockedEnsemblePlacementPolicy {}
+
+ @Test
+ public void testEncodeDecodeSuccessfully()
+ throws EnsemblePlacementPolicyConfig.ParseEnsemblePlacementPolicyConfigException {
+
+ EnsemblePlacementPolicyConfig originalConfig =
+ new EnsemblePlacementPolicyConfig(MockedEnsemblePlacementPolicy.class, Collections.EMPTY_MAP);
+ byte[] encodedConfig = originalConfig.encode();
Review comment:
in order to confirm backward compatibility in the future I feel it may have a good value to try to also deserialise a fixed string constant, I mean a value that has not been produced but current version of the code
what about adding such a test ?
- String encoded = "{.....}"
- decode
- assertEquals
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on a change in pull request #9551: Avoid introducing bookkeeper-common into the pulsar-common
Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #9551:
URL: https://github.com/apache/pulsar/pull/9551#discussion_r575034981
##########
File path: pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/EnsemblePlacementPolicyConfig.java
##########
@@ -30,6 +31,8 @@
private final Class policyClass;
private final Map<String, Object> properties;
+ private static ObjectMapper mapper = new ObjectMapper();
Review comment:
@merlimat that makes sense to me.
In fact usually you create a `static` ObjectMapper when you are sure that it will deal only with a limited and fixed set classes .
in this case the ObjectMapper will deal only with this class, and thus it is safe to use a static instance.,
As just said before, I prefer to have a consistent way of coding, so here in Pulsar is it fine to me to use ObjectManagerFactory. The overhead is to create a ObjectMapper (and the internal cache) per thread instead of having just one per target class.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zymap commented on a change in pull request #9551: Avoid introducing bookkeeper-common into the pulsar-common
Posted by GitBox <gi...@apache.org>.
zymap commented on a change in pull request #9551:
URL: https://github.com/apache/pulsar/pull/9551#discussion_r573497607
##########
File path: pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/EnsemblePlacementPolicyConfig.java
##########
@@ -30,6 +31,8 @@
private final Class policyClass;
private final Map<String, Object> properties;
+ private static ObjectMapper mapper = new ObjectMapper();
Review comment:
Thanks for the reminder. I change to use the `ObjectMapperFactory.getThreadLocal()`. Please take a look again.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org