[jira] [Created] (FLINK-27928) External Resource Framework: 'external-resources' config delimiter is not working as expected

["James Cho (Jira)" <ji...@apache.org>]

James Cho created FLINK-27928:
---------------------------------

             Summary: External Resource Framework: 'external-resources' config delimiter is not working as expected
                 Key: FLINK-27928
                 URL: https://issues.apache.org/jira/browse/FLINK-27928
             Project: Flink
          Issue Type: Bug
          Components: API / DataStream
    Affects Versions: 1.15.0
            Reporter: James Cho


For flink version 1.15.0, the configuration `external-resources` delimiter separation(";") functionality between external resource names is not working as expected. 

 

Reproducing the bug setting up a flink cluster running using docker with two external resources named `abc` and `efg`:

docker-compose.yml:

 
{code:java}
services:
  jobmanager:
    image: flink:1.15.0-scala_2.12
    ports:
      - "8081:8081"
      - "6123:6123"
    command: jobmanager
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager

  taskmanager:
    image: flink:1.15.0-scala_2.12
    depends_on:
      - jobmanager
    command: taskmanager
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager
        taskmanager.numberOfTaskSlots: 8
        external-resources: abc;efg
        external-resource.abc.amount: 1
        external-resource.efg.amount: 1{code}
 

Related task manager logs:

 
{code:java}
root@2873c5ae72c2:/opt/flink# cat log/flink* | grep ExternalResourceUtils
2022-06-07 05:42:34,232 WARN  org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - The amount of the abc;efg should be configured. Will ignore that resource.
2022-06-07 05:42:34,232 INFO  org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled external resources: []
2022-06-07 05:42:34,236 WARN  org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Could not find driver class name for abc;efg. Please make sure external-resource.abc;efg.driver-factory.class is configured.
2022-06-07 05:42:34,240 WARN  org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - The amount of the abc;efg should be configured. Will ignore that resource.
{code}
Notice flink recognizes `abc;efg` as a single external resource rather than as `abc` and `efg`

 

 

The same issue does not exist in flink version 1.14.4

docker-compose.yml:
{code:java}
services:
  jobmanager:
    image: flink:1.14.4-scala_2.12
    ports:
      - "8081:8081"
      - "6123:6123"
    command: jobmanager
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager

  taskmanager:
    image: flink:1.14.4-scala_2.12
    depends_on:
      - jobmanager
    command: taskmanager
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager
        taskmanager.numberOfTaskSlots: 8
        external-resources: abc;efg
        external-resource.abc.amount: 1
        external-resource.efg.amount: 1 {code}
related tm logs
{code:java}
docker logs ... | grep ExternalResourceUtils
2022-06-07 05:48:13,531 INFO  org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled external resources: [abc, efg]
2022-06-07 05:48:13,534 WARN  org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Could not find driver class name for abc. Please make sure external-resource.abc.driver-factory.class is configured.
2022-06-07 05:48:13,534 WARN  org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Could not find driver class name for efg. Please make sure external-resource.efg.driver-factory.class is configured.
 {code}
 

 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)