You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2020/05/29 21:44:44 UTC

[GitHub] [iceberg] rdblue edited a comment on pull request #1077: Fix NPE when counting entries

rdblue edited a comment on pull request #1077:
URL: https://github.com/apache/iceberg/pull/1077#issuecomment-636205153


   Looks like there are two problems. First, `findType` throws a `NullPointerException` if the type isn't found. I think that should check whether the name was found, like this:
   
   ```diff
      public Type findType(String name) {
        Preconditions.checkArgument(!name.isEmpty(), "Invalid column name: (empty)");
   -    return findType(lazyNameToId().get(name));
   +    Integer id = lazyNameToId().get(name);
   +    if (id != null) {
   +      return findType(id);
   +    }
   +    return null;
      }
   ```
   
   Second, if the file projection is null, it is still dereferenced. So we need to fix that as well in `ManifestEntriesTable`:
   
   ```diff
        protected CloseableIterable<FileScanTask> planFiles(
            TableOperations ops, Snapshot snapshot, Expression rowFilter, boolean caseSensitive, boolean colStats) {
          CloseableIterable<ManifestFile> manifests = CloseableIterable.withNoopClose(snapshot.manifests());
   -      Schema fileSchema = new Schema(schema().findType("data_file").asStructType().fields());
   +      Type fileProjection = schema().findType("data_file");
   +      Schema fileSchema = fileProjection != null ? new Schema(fileProjection.asStructType().fields()) : new Schema();
          String schemaString = SchemaParser.toJson(schema());
          String specString = PartitionSpecParser.toJson(PartitionSpec.unpartitioned());
          ResidualEvaluator residuals = ResidualEvaluator.unpartitioned(rowFilter);
   ```
   
   After that, adding this to the entries table test case works:
   
   ```
       Assert.assertEquals("Count should return 1",
           1, spark.read().format("iceberg").load(loadLocation(tableIdentifier, "entries")).count());
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org