[GH] (streampark/feat/gravitino-lineage): Workflow run "Backend" failed!

[GitBox <gi...@apache.org>]

The GitHub Actions job "Backend" on streampark.git/feat/gravitino-lineage has failed.
Run started by GitHub user 88fantasy (triggered by 88fantasy).

Head commit for run:
f69aad281a5d2fcb30e8b488474cc75919c99c09 / 88fantasy <88...@gmail.com>
[Console] Report Flink and Spark job lineage to Gravitino via OpenLineage

StreamPark currently emits no lineage at all, so jobs it submits are
invisible to a Gravitino lineage graph that other producers already write
into. This adds table-level lineage reporting for Flink SQL and Spark
applications, off unless configured.

Configuration (system settings, all new):
  lineage.gravitino.address              base URL; empty disables everything
  lineage.gravitino.token                bearer token, needed once Gravitino
                                         authentication is enabled
  lineage.gravitino.namespace            OpenLineage namespace to report under
  lineage.flink.native.listener.enable   also inject the official
                                         openlineage-flink listener config

Per-application opt-in via a new lineage_enable column on t_flink_app and
t_spark_app. Nothing is injected or emitted while the address is empty, so
an existing deployment is unaffected until it is configured — deliberate,
since injecting listener config for a jar the cluster's lib/ does not have
would break job startup.

Flink: the SQL text is planned in a throwaway TableEnvironment inside the
per-version shims classloader (via the existing FlinkShimsProxy, the same
mechanism SQL verification already uses), and the resulting CompiledPlan
JSON is walked from each sink backwards to the sources that reach it. The
console emits START on submission and COMPLETE/FAIL when the job reaches a
terminal state, so the lineage graph is only rebuilt for runs that actually
finished. Pairing inputs per sink rather than flattening them matters for
STATEMENT SET jobs: N independent INSERTs compile to N disconnected
subgraphs, and flattening would report N×M edges that do not exist.

Spark: OpenLineage's own listener is configured through appProperties,
leaving any key the user set explicitly untouched.

Dataset identity is resolved by connector (mysql-cdc, doris, catalog-backed
tables) into the exact (namespace, name) pair other producers into the same
Gravitino instance already use — a mismatch would split one physical table
into two nodes in the graph rather than fail visibly. An unrecognised
connector is skipped with a WARN instead of guessed at.

Everything on this path is fail-open: an unresolvable dataset, a plan that
will not compile, or an unreachable Gravitino logs and moves on. A lineage
gap must never fail a job submission.

Adds io.openlineage:openlineage-java (Apache-2.0) at 1.29.0. That client
needs httpclient5 5.4.x, so httpclient5 goes 5.1 -> 5.4.2 and httpcore5 is
pinned to 5.4.3 — without pinning, an older transitive httpcore5 wins
mediation and httpclient5 fails at runtime with NoSuchMethodError.

Schema changes ship as upgrade/mysql/3.0.0.sql and upgrade/pgsql/3.0.0.sql
alongside the install scripts. Both were verified by applying the previous
release's schema, running the upgrade, and diffing the result against a
fresh install: identical columns and t_setting rows on both databases. The
PostgreSQL script covers t_flink_app only, because pgsql-schema.sql defines
no Spark tables at all — a pre-existing gap this change does not widen.

Report URL: https://github.com/apache/streampark/actions/runs/31763788468

With regards,
GitHub Actions via GitBox