You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2020/07/05 07:11:35 UTC
[GitHub] [iceberg] jun-he opened a new pull request #1169: Reset temporary states inside thread local objects
jun-he opened a new pull request #1169:
URL: https://github.com/apache/iceberg/pull/1169
Similar to #1139, this change reset temporary states inside thread local objects to avoid possible memory leak issues.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jun-he commented on a change in pull request #1169: Remove thread local objects and use new visitors
Posted by GitBox <gi...@apache.org>.
jun-he commented on a change in pull request #1169:
URL: https://github.com/apache/iceberg/pull/1169#discussion_r450008898
##########
File path: api/src/main/java/org/apache/iceberg/expressions/InclusiveMetricsEvaluator.java
##########
@@ -102,7 +102,15 @@ private boolean eval(ContentFile<?> file) {
this.lowerBounds = file.lowerBounds();
this.upperBounds = file.upperBounds();
- return ExpressionVisitors.visitEvaluator(expr, this);
+ try {
+ return ExpressionVisitors.visitEvaluator(expr, this);
+ } finally {
+ // allow temporary state to be collected because this is in a thread-local
+ this.valueCounts = null;
Review comment:
Thanks @rdsr and @rdblue for the comments. Updated the PR accordingly to remove the thread local objects and use new visitors to keep the code straightforward.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdsr commented on pull request #1169: Remove thread local objects and use new visitors
Posted by GitBox <gi...@apache.org>.
rdsr commented on pull request #1169:
URL: https://github.com/apache/iceberg/pull/1169#issuecomment-654376804
+1. Looks great now!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdsr merged pull request #1169: Remove thread local objects and use new visitors
Posted by GitBox <gi...@apache.org>.
rdsr merged pull request #1169:
URL: https://github.com/apache/iceberg/pull/1169
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdsr commented on a change in pull request #1169: Reset temporary states inside thread local objects
Posted by GitBox <gi...@apache.org>.
rdsr commented on a change in pull request #1169:
URL: https://github.com/apache/iceberg/pull/1169#discussion_r449850015
##########
File path: api/src/main/java/org/apache/iceberg/expressions/InclusiveMetricsEvaluator.java
##########
@@ -102,7 +102,15 @@ private boolean eval(ContentFile<?> file) {
this.lowerBounds = file.lowerBounds();
this.upperBounds = file.upperBounds();
- return ExpressionVisitors.visitEvaluator(expr, this);
+ try {
+ return ExpressionVisitors.visitEvaluator(expr, this);
+ } finally {
+ // allow temporary state to be collected because this is in a thread-local
+ this.valueCounts = null;
Review comment:
I think this is subtle and hard to understand on why this is needed. I prefer the the new visitor approach as you guys discussed in #1139.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #1169: Remove thread local objects and use new visitors
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #1169:
URL: https://github.com/apache/iceberg/pull/1169#issuecomment-654320001
Looks good to me. What do you think, @rdsr?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jun-he commented on pull request #1169: Remove thread local objects and use new visitors
Posted by GitBox <gi...@apache.org>.
jun-he commented on pull request #1169:
URL: https://github.com/apache/iceberg/pull/1169#issuecomment-654527458
Thanks for the review!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a change in pull request #1169: Reset temporary states inside thread local objects
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #1169:
URL: https://github.com/apache/iceberg/pull/1169#discussion_r449917364
##########
File path: api/src/main/java/org/apache/iceberg/expressions/InclusiveMetricsEvaluator.java
##########
@@ -102,7 +102,15 @@ private boolean eval(ContentFile<?> file) {
this.lowerBounds = file.lowerBounds();
this.upperBounds = file.upperBounds();
- return ExpressionVisitors.visitEvaluator(expr, this);
+ try {
+ return ExpressionVisitors.visitEvaluator(expr, this);
+ } finally {
+ // allow temporary state to be collected because this is in a thread-local
+ this.valueCounts = null;
Review comment:
That sounds good to me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org