You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@streampark.apache.org by "88fantasy (via GitHub)" <gi...@apache.org> on 2026/08/12 23:11:59 UTC
[I] [Bug] BuildPipeline.execStep throws NullPointerException for any step that legitimately returns null [streampark]
88fantasy opened a new issue, #4481:
URL: https://github.com/apache/streampark/issues/4481
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/streampark/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues referencing `BuildPipeline`/`execStep`.
### Java Version
Temurin 21.0.11 (console), also reproduced building with Microsoft OpenJDK 11.0.28
### Scala Version
2.12.x
### StreamPark Version
3.0.0-SNAPSHOT (`dev` branch, commit `f89652b67`)
### Flink Version
2.2.1 (official binary distribution, standalone/remote cluster)
### Deploy mode
remote
### What happened
Building/releasing a Flink SQL application always fails at step 1/2 ("Create building workspace") with an uncaught `NullPointerException`, even though the workspace directory is genuinely created successfully on disk. The failure is **not visible anywhere in the normal log files** (`logs/error.*.log`) — it only shows up in the raw stdout redirect (`logs/streampark.out`), because it's thrown from a background thread pool task before the console's `GlobalExceptionHandler` is ever involved, and the persisted `ApplicationBuildPipeline.hasError`/`errorSummary`/`errorStack` fields stay `null`/`false`, making the failure look silent from the REST API (`POST /flink/pipe/detail` just shows `pipeStatus: FAILURE` with no error detail).
Root cause: `BuildPipeline#execStep()` wraps every successful step result in `Optional.of(result)`:
```java
R result = process.call();
...
return java.util.Optional.of(result); // <-- throws NPE if result == null
```
`Optional.of(null)` always throws `NullPointerException` per the Java API contract (use `Optional.ofNullable` for that). Many pipeline steps legitimately have nothing meaningful to return — e.g. "create/clean the build workspace" just performs a side effect and returns `null` — and this is the *normal*, successful case, not an edge case. This is a classic Scala→Java migration bug (introduced by #4463, "[Migrate] Migrate streampark-flink-packer from Scala to Java"): the original Scala code almost certainly used `Option(x)`, which tolerates `null` and produces `None`, not the strict `Optional.of(x)`.
This is not specific to Flink SQL or remote/standalone deploy mode — the same `execStep(...).orElseThrow(() -> pipelineException())` pattern, with a step whose `Callable` returns `null`, appears at **~20 call sites across 6 files**: `BuildPipeline#runYarnSqlBuildSteps` (used by the Yarn pipelines), `FlinkRemoteBuildPipeline`, `FlinkK8sSessionBuildPipeline`, `AbstractK8sApplicationBuildPipeline`, `FlinkK8sApplicationBuildPipeline`, and `SparkK8sApplicationBuildPipeline`. Every deploy mode's build pipeline is affected wherever its first (or any null-returning) step runs.
Simply swapping `Optional.of` for `Optional.ofNullable` is **not sufficient** on its own: every call site relies on `.orElseThrow(() -> pipelineException())` to turn a *failed* step into a thrown exception, and `Optional<R>` cannot distinguish "empty because the step failed" from "empty because the step succeeded with a `null` result" — so a naive one-line fix just trades a crash for `.orElseThrow()` incorrectly firing on every successful null-returning step. The actual fix needs `execStep()` to signal failure via a thrown exception directly (matching what all 20 call sites already reconstruct via `orElseThrow`) rather than via `Optional` emptiness, and the corresponding call sites need to drop the now-redundant `.orElseThrow(...)`.
### Error Exception
```
06:31:23.581 [streampark-build-pipeline-executor-0] ERROR org.apache.streampark.flink.packer.pipeline.impl.FlinkRemoteBuildPipeline - [StreamPark] [streampark-packer] Building pipeline has failed. | appName=streampark_bounded_probe
java.lang.IllegalStateException: null
at org.apache.streampark.flink.packer.pipeline.BuildPipeline.pipelineException(BuildPipeline.java:259)
at org.apache.streampark.flink.packer.pipeline.impl.FlinkRemoteBuildPipeline.lambda$buildProcess$1(FlinkRemoteBuildPipeline.java:64)
at java.base/java.util.Optional.orElseThrow(Optional.java:403)
at org.apache.streampark.flink.packer.pipeline.impl.FlinkRemoteBuildPipeline.buildProcess(FlinkRemoteBuildPipeline.java:63)
...
Caused by: java.lang.NullPointerException: null
at java.base/java.util.Objects.requireNonNull(Objects.java:233)
at java.base/java.util.Optional.of(Optional.java:113)
at org.apache.streampark.flink.packer.pipeline.BuildPipeline.execStep(BuildPipeline.java:135)
at org.apache.streampark.flink.packer.pipeline.impl.FlinkRemoteBuildPipeline.buildProcess(FlinkRemoteBuildPipeline.java:59)
... 5 common frames omitted
```
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] BuildPipeline.execStep throws NullPointerException for any step that legitimately returns null [streampark]
Posted by "wolfboys (via GitHub)" <gi...@apache.org>.
wolfboys closed issue #4481: [Bug] BuildPipeline.execStep throws NullPointerException for any step that legitimately returns null
URL: https://github.com/apache/streampark/issues/4481
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org