You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2020/11/10 17:48:04 UTC

[GitHub] [helix] alirezazamani opened a new pull request #1523: Fix redundant context writes to ZK

alirezazamani opened a new pull request #1523:
URL: https://github.com/apache/helix/pull/1523


   ### Issues
   
   - [X] My PR addresses the following Helix issues and references them in the PR description:
   Fixes #1522  
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI changes:
   In this PR, the redundant and excessive writes to ZK with regard to workflow and job context have been eliminated.
   
   ### Tests
   
   - [x] The following tests are written for this issue:
   TestContextUpdateIfNoChange
   
   - [x] The following is the result of the "mvn test" command on the appropriate module:
   Running the tests
   Helix-core:
   ```
   ```
   Helix-rest:
   ```
   ```
   
   ### Commits
   
   - My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
     1. Subject is separated from body by a blank line
     1. Subject is limited to 50 characters (not including Jira issue reference)
     1. Subject does not end with a period
     1. Subject uses the imperative mood ("add", not "adding")
     1. Body wraps at 72 characters
     1. Body explains "what" and "why", not "how"
   
   ### Code Quality
   
   - My diff has been formatted using helix-style.xml 
   (helix-style-intellij.xml if IntelliJ IDE is used)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on pull request #1523:
URL: https://github.com/apache/helix/pull/1523#issuecomment-725644470


   Based on our discussion, we decided to close this PR and come back to it later.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] dasahcc commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
dasahcc commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520808899



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -174,6 +178,13 @@ private void refreshContexts(HelixDataAccessor accessor) {
       }
     }
 
+    for (Map.Entry<String, ZNRecord> entry : _contextMap.entrySet()) {
+      if (entry.getValue() != null) {
+        _initialContextMapHashcode.put(entry.getKey(),
+            Objects.hashCode(entry.getValue().toString()));

Review comment:
       Do we need to use Objects.hashCode? Since we already converted it to String, we can call the String's function hashCode();

##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +278,11 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {

Review comment:
       I think this may be an overkill by checking every update. We can let it keep updating but just one check for all the context before doing writes to ZK. Otherwise, say I am trying to update one context 10 times, I need to convert to String every time. 
   
   Also we can leverage Java8 stream to do the checks before write to ZK, it could be efficient instead of every update check.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520930640



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       That is not possible or costly. We do in place change the content. If we want to check the content we need to do a deep copy of the context caches at the refresh phase which increases the memory footprint a lot.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] kaisun2000 commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r521152549



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       I think MD5 is probably ok while hashcode is not. The thing is that even in you doc you quote, it has around 'An aggregate collision rate of 8 per 10,000 still isn’t bad in an absolute sense.'  0.1% is still too high for our usage. 
   
   MD5 etc secure hash is the foundation of digital signature. The have much much lower false rate. Basically people can claim for all practical purpose, they can replace equality test. 
   
   We can talk about it tomorrow.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520930640



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       That is not possible or will be very costly. We do in place change the content. If we want to check the content we need to do a deep copy of the context caches at the refresh phase which increases the memory footprint a lot.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520904922



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       Yeah, your point is true since the possible input of hashcode is bigger than the possible outputs. However, since we convert the input to a string, the chance is pretty low (note that it will not be random strings and input string is derived from the ZNRecord). Also, hash collisions usually happen in short size strings. We had other options like deep copy and etc. but that options could cause memory and GC issues.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r521030372



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       We can also use different algorithms such as md5. I feel it is kind of overkill but if there is a strong preference for that we can use different algorithms.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520932675



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       @dasahcc @jiajunwang  what do you think about this discussion? I think the concern is valid in theory but not in practice. http://sigpwned.com/2018/08/10/string-hashcode-is-plenty-unique/




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] kaisun2000 commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520929697



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       Can we do something like if hash equals, let to do a further check of content equality before skipping it?  otherwise, this code has the potential of losing update, which would be hard to debug in the field.
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520932675



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       @dasahcc @jiajunwang  what do you think about this discussion? I think the concern is valid in theory but not in practice. I think the fact that the input is not random and is a String created based on ZNRecord protects us against this issue. http://sigpwned.com/2018/08/10/string-hashcode-is-plenty-unique/




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520904922



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       Yeah, your point is true since the possible input of hashcode is bigger than the possible outputs. However, since we convert the input to a string, the chance is pretty low (note that it will not be random strings and input string is derived from the ZNRecord). We had other options like deep copy and etc. but that options could cause memory and GC issues.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] kaisun2000 commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520931481



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       Then what is the solution here? 
   
   Is it possible to calculate security hash using say md5?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520884510



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -174,6 +178,13 @@ private void refreshContexts(HelixDataAccessor accessor) {
       }
     }
 
+    for (Map.Entry<String, ZNRecord> entry : _contextMap.entrySet()) {
+      if (entry.getValue() != null) {
+        _initialContextMapHashcode.put(entry.getKey(),
+            Objects.hashCode(entry.getValue().toString()));

Review comment:
       I changed it to String.hashcode()




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520932675



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       @dasahcc @jiajunwang  what do you think about this discussion (This PR follows what we discussed to address this issue)? I think the concern is valid in theory but not in practice. I think the fact that the input is not random and is a String created based on ZNRecord protects us against this issue. http://sigpwned.com/2018/08/10/string-hashcode-is-plenty-unique/




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] kaisun2000 commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r521152549



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       I think MD5 is probably ok while hashcode is not. The thing is that even in the doc you quoted, it has around 'An aggregate collision rate of 8 per 10,000 still isn’t bad in an absolute sense.'  0.1% is still too high for our usage. 
   
   MD5 etc secure hash is the foundation of digital signature. The have much much lower false rate. Basically people can claim for all practical purpose, they can replace equality test. 
   
   We can talk about it tomorrow.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520930737



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +278,11 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {

Review comment:
       Addressed.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani closed pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani closed pull request #1523:
URL: https://github.com/apache/helix/pull/1523


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r521507951



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       I added the MD5 commit now. Both of them work for me and IMO both would work considering our input is not a random string. Anyway, we can go with either of them. If we decided to go with MD5 we can keep the last commit. If not we can revert it.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] alirezazamani commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
alirezazamani commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520884903



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +278,11 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {

Review comment:
       Thanks for the suggestion. I will see if we can do the check at the update part




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


[GitHub] [helix] kaisun2000 commented on a change in pull request #1523: Fix redundant context writes to ZK

Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #1523:
URL: https://github.com/apache/helix/pull/1523#discussion_r520897642



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -267,6 +277,10 @@ public void updateWorkflowContext(String resourceName, WorkflowContext workflowC
    * Update context of the Workflow or Job
    */
   private void updateContext(String resourceName, ZNRecord record) {
+    if (record != null && _initialContextMapHashcode.containsKey(resourceName)
+        && _initialContextMapHashcode.get(resourceName).equals((record.toString().hashCode()))) {
+      return;
+    }

Review comment:
       The problem I see here is that hash code equals does not mean the content is equal, right? 
   
   hashcode1 ! = hashcode2 ==> content1 != content2: this is fine
   
   hashcode1 == hashcode2  ==> content1 == content2: this is not true.  Unless you use security hash, the probability is very high that this is true.
   
   Or is it that I did not fully understand the intention here?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org