You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2020/09/23 10:31:20 UTC

[GitHub] [iceberg] kbendick opened a new pull request #1491: [API][Types] Add a precondition null check for the constructor argument of PruneColumns Schema Visitor

kbendick opened a new pull request #1491:
URL: https://github.com/apache/iceberg/pull/1491


   The class PruneColumns has a single constructor, which takes in a Set<Integer>, `PruneColumns(Set<Integer> selected)`.
   
   Later on in this class, there are no null checks before calling methods on the value `selected` such as `contains`, etc.
   
   Admittedly, the only place that PruneColumns is _currently_ used, `org.apache.iceberg.types.TypeUtils#select(Types.StructType struct, Set<Integer> fieldIds)`, has a precondition check for null on fieldIds - which is passed into PruneColumns. Thus, it is arguably not necessary to add this additional check.
   
   However, because the class `PruneColumns` is part of `api` and might get used elsewhere, it seems sane to add this precondition check given that we do no null checking before accessing it and so it would be up to the caller to add the preconditions check if they were to reuse it.
   
   Lastly, it's important to note that `PruneColumns`,  while part of `api`, is not declared as Public and so one could argue that this is not needed given that it's internal to iceberg and anybody using it should just know that its constructor argument must be non-null. I'm not a personal believer in tribal knowledge like that, and so I'm opening this PR.
   
   If we don't think this PR is necessary, given the precondition check in the one place that it is currently used, then we can go ahead and close this PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


[GitHub] [iceberg] kbendick commented on a change in pull request #1491: [API][Types] Add a precondition null check for the constructor arg of PruneColumns, a SchemaVisitor

Posted by GitBox <gi...@apache.org>.
kbendick commented on a change in pull request #1491:
URL: https://github.com/apache/iceberg/pull/1491#discussion_r495518748



##########
File path: api/src/main/java/org/apache/iceberg/types/PruneColumns.java
##########
@@ -22,12 +22,14 @@
 import java.util.List;
 import java.util.Set;
 import org.apache.iceberg.Schema;
+import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
 import org.apache.iceberg.relocated.com.google.common.collect.Lists;
 
 class PruneColumns extends TypeUtil.SchemaVisitor<Type> {
   private final Set<Integer> selected;
 
   PruneColumns(Set<Integer> selected) {
+    Preconditions.checkNotNull(selected, "Selected cannot be null");

Review comment:
       I agree with you on this. I was following what I thought was somewhat of a convention - what I should really refer to as something I've seen previously in the code base - of using the argument name directly. However, I also just today saw some instances that are definitely much more explicit.
   
   I will update to be `Selected field ids` as its inline with our definitions of `Select` and `Project` as well as it still contains the argument name in it. Please correct me if I'm incorrect about `select` vs `project`. @rdblue 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


[GitHub] [iceberg] rdblue merged pull request #1491: [API][Core - Types/Avro][Parquet] Add a precondition null check for the constructor arg of PruneColumns

Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #1491:
URL: https://github.com/apache/iceberg/pull/1491


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


[GitHub] [iceberg] kbendick commented on a change in pull request #1491: [API][Types] Add a precondition null check for the constructor arg of PruneColumns, a SchemaVisitor

Posted by GitBox <gi...@apache.org>.
kbendick commented on a change in pull request #1491:
URL: https://github.com/apache/iceberg/pull/1491#discussion_r495519829



##########
File path: core/src/main/java/org/apache/iceberg/avro/PruneColumns.java
##########
@@ -39,6 +39,7 @@
   private final NameMapping nameMapping;
 
   PruneColumns(Set<Integer> selectedIds, NameMapping nameMapping) {
+    Preconditions.checkNotNull(selectedIds, "Selected field ids cannot be null");

Review comment:
       I will also check if `nameMapping` needs a precondition null check.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


[GitHub] [iceberg] rdblue commented on a change in pull request #1491: [API][Types] Add a precondition null check for the constructor arg of PruneColumns, a SchemaVisitor

Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #1491:
URL: https://github.com/apache/iceberg/pull/1491#discussion_r494513496



##########
File path: api/src/main/java/org/apache/iceberg/types/PruneColumns.java
##########
@@ -22,12 +22,14 @@
 import java.util.List;
 import java.util.Set;
 import org.apache.iceberg.Schema;
+import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
 import org.apache.iceberg.relocated.com.google.common.collect.Lists;
 
 class PruneColumns extends TypeUtil.SchemaVisitor<Type> {
   private final Set<Integer> selected;
 
   PruneColumns(Set<Integer> selected) {
+    Preconditions.checkNotNull(selected, "Selected cannot be null");

Review comment:
       Rather than just using the name of the variable, I think this should give more context to users when it happens. "Field id set" or "Selected field ids" would be better.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


[GitHub] [iceberg] rdblue commented on pull request #1491: [API][Core - Types/Avro][Parquet] Add a precondition null check for the constructor arg of PruneColumns

Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #1491:
URL: https://github.com/apache/iceberg/pull/1491#issuecomment-700353804


   Looks good. Thanks, @kbendick!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


[GitHub] [iceberg] rdblue commented on a change in pull request #1491: [API][Types] Add a precondition null check for the constructor arg of PruneColumns, a SchemaVisitor

Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #1491:
URL: https://github.com/apache/iceberg/pull/1491#discussion_r494513496



##########
File path: api/src/main/java/org/apache/iceberg/types/PruneColumns.java
##########
@@ -22,12 +22,14 @@
 import java.util.List;
 import java.util.Set;
 import org.apache.iceberg.Schema;
+import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
 import org.apache.iceberg.relocated.com.google.common.collect.Lists;
 
 class PruneColumns extends TypeUtil.SchemaVisitor<Type> {
   private final Set<Integer> selected;
 
   PruneColumns(Set<Integer> selected) {
+    Preconditions.checkNotNull(selected, "Selected cannot be null");

Review comment:
       Rather than just using the name of the variable, I think this should give more context to users when it happens. "Field id set" or "Selected field ids" would be better.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


[GitHub] [iceberg] kbendick commented on pull request #1491: [API][Types] Add a precondition null check for the constructor arg of PruneColumns, a SchemaVisitor

Posted by GitBox <gi...@apache.org>.
kbendick commented on pull request #1491:
URL: https://github.com/apache/iceberg/pull/1491#issuecomment-699576667


   I found a few other places where there are `PruneColumns` classes that have the same behavior, i.e. not checking for null inside of the code in the class and not having a null precondition check in the constructor on the same field.
   
   I've updated those, but I need to take a closer look at a Python class of the same name as well as some classes related to Spark named `PruneColumnsWithoutReordering` and `PruneColumnsWithReordering`.
   
   Do you happen to know if Python typically null checks or is it possible that we assume things have gone through JVM code in certain cases and therefore don't add them? I'll check myself for cases, but I'd love it if you or someone could shed some insight onto that for Python inter-op. Possibly something I should ask on users@ or dev@? cc @rdblue 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org