[jira] [Created] (FLINK-37586) YAML pipeline supports passing extra options to UDFs

["yux (Jira)" <ji...@apache.org>]

yux created FLINK-37586:
---------------------------

             Summary: YAML pipeline supports passing extra options to UDFs
                 Key: FLINK-37586
                 URL: https://issues.apache.org/jira/browse/FLINK-37586
             Project: Flink
          Issue Type: Improvement
          Components: Flink CDC
            Reporter: yux


Currently, the only way to customize UDF behavior is by passing literal values as arguments at runtime.

As an example scenario, assume we're creating a UDF that queries corresponding values by given keys in Redis. Such usage is simple and straightforward:

 
{code:java}
transform:
  - projection: id, redis_key, QUERY_REDIS(key) AS redis_value{code}
but sadly doesn't work because we can't configure Redis connection url elsewhere. An awkward workaround could be like this:

 

 
{code:java}
transform:
  - projection: id, redis_key, QUERY_REDIS('redis://user:pass@localhost:6379/something', key) AS redis_value{code}
which isn't an ideal solution.

 

---

It woule be nice if we can pass some static options to UDFs like this:
{code:java}
pipeline:
  user-defined-function:
    - name: QUERY_REDIS
      classpath: com.example.flink.cdc.udf.RedisQueryFunction
      options:
        hostname: redis://user:pass@localhost:6379/something
        cache: true
        ...{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)