You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "chl-wxp (via GitHub)" <gi...@apache.org> on 2026/07/01 03:09:07 UTC

[PR] [Feature][Flink] Support MetadataProvider SPI in Flink starter [seatunnel]

chl-wxp opened a new pull request, #11237:
URL: https://github.com/apache/seatunnel/pull/11237

    ### Purpose
   
     This PR adds MetadataProvider SPI support for the Flink starter, so Flink jobs can resolve datasource connection options from the metadata center by using `metadata_datasource_id`.
   
     Before this change, Flink jobs with configs such as:
   
     ```hocon
     source {
       Jdbc {
         metadata_datasource_id = "test_catalog"
         query = "select * from source"
       }
     }
   
     could fail during connector option validation because required JDBC options like url and driver were not resolved before FactoryUtil created the source.
   
     ### Changes
   
     - Load metadata center configuration from seatunnel.yaml in the Flink starter.
     - Resolve datasource configs through MetadataProviderManager before applying legacy Metalake config handling.
     - Keep compatibility with existing Metalake placeholder replacement logic.
     - Update JDBC e2e coverage for MetadataProvider SPI on Flink.
     - Ensure the Flink e2e job process has SEATUNNEL_HOME so it can locate config/seatunnel.yaml.
     - Keep Metalake legacy env behavior using METALAKE_ENABLED, METALAKE_TYPE, and METALAKE_URL.
   
     ### Why
   
     The MetadataProvider SPI expects metadata settings from:
   
     seatunnel:
       engine:
         metadata:
           enabled: true
           kind: gravitino
           gravitino:
             uri: http://gravitino:8090
             metalake: test_metalake
   
     Flink starter needs this config before connector validation, otherwise datasource options are not merged into the job config.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Feature][Flink] Support MetadataProvider SPI in Flink starter [seatunnel]

Posted by "chl-wxp (via GitHub)" <gi...@apache.org>.
chl-wxp closed pull request #11237: [Feature][Flink] Support MetadataProvider SPI in Flink starter
URL: https://github.com/apache/seatunnel/pull/11237


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org