You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kenneth Knowles (JIRA)" <ji...@apache.org> on 2019/02/01 01:10:00 UTC

[jira] [Commented] (BEAM-6545) NPE when decoding null base 64 strings

    [ https://issues.apache.org/jira/browse/BEAM-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757848#comment-16757848 ] 

Kenneth Knowles commented on BEAM-6545:
---------------------------------------

I have done a manual audit of the remaining uses in our codebase. Relevant occurrences listed below:

{code}
$ grep -r decodeBase64 *
...
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/UngroupedShuffleReaderFactory.java:        decodeBase64(getString(spec, WorkerPropertyNames.SHUFFLE_READER_CONFIG)),
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/GroupingShuffleReaderFactory.java:          decodeBase64(getString(spec, WorkerPropertyNames.SHUFFLE_READER_CONFIG)),
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/GroupingShuffleReaderFactory.java:        decodeBase64(getString(spec, WorkerPropertyNames.SHUFFLE_READER_CONFIG)),
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/ShuffleSinkFactory.java:        decodeBase64(getString(spec, WorkerPropertyNames.SHUFFLE_WRITER_CONFIG, null)),
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/util/common/worker/ByteArrayShufflePosition.java:    return ByteArrayShufflePosition.of(decodeBase64(position));
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WorkerCustomSources.java:              decodeBase64(serializedSplits.get(index)), "UnboundedSource split");
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WorkerCustomSources.java:                Base64.decodeBase64(getString(spec, SERIALIZED_SOURCE)), "Source");
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/PartitioningShuffleReaderFactory.java:        decodeBase64(getString(spec, WorkerPropertyNames.SHUFFLE_READER_CONFIG)),
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/FakeBigQueryServices.java:    ByteArrayInputStream input = new ByteArrayInputStream(Base64.decodeBase64(query));
{code}

In each case, a misplaced {{null}} would crash elsewhere so it cannot regress due to decodeBase64 enforcing it.

> NPE when decoding null base 64 strings
> --------------------------------------
>
>                 Key: BEAM-6545
>                 URL: https://issues.apache.org/jira/browse/BEAM-6545
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-dataflow
>    Affects Versions: 2.9.0
>            Reporter: Ahmet Altay
>            Assignee: Kenneth Knowles
>            Priority: Major
>             Fix For: 2.10.0
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> **ByteArrayShufflePosition.fromBase64 is marked with a @Nullable argument, however it does not properly handle null inputs resulting in NPE.
> This seems like an unintended change we picked up from the dependency: google-http-java-client/ switched from apache commons to guava ([https://github.com/googleapis/google-http-java-client/commit/990c534f0e5103a142b0639c12c90cb990a00cfd#diff-97264fba16d690a26d63fbbc992af937)] 
>  
> and decodeBase64 behaves differently in both cases. Former can handle null by returning null, latter will throw NPE.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)