You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "James Cho (Jira)" <ji...@apache.org> on 2022/06/07 06:02:00 UTC
[jira] [Updated] (FLINK-27928) External Resource Framework: 'external-resources' config delimitation not working as expected
[ https://issues.apache.org/jira/browse/FLINK-27928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
James Cho updated FLINK-27928:
------------------------------
Summary: External Resource Framework: 'external-resources' config delimitation not working as expected (was: External Resource Framework: 'external-resources' config delimiter is not working as expected)
> External Resource Framework: 'external-resources' config delimitation not working as expected
> ---------------------------------------------------------------------------------------------
>
> Key: FLINK-27928
> URL: https://issues.apache.org/jira/browse/FLINK-27928
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
> Affects Versions: 1.15.0
> Reporter: James Cho
> Priority: Major
> Labels: external-resources
>
> For flink version 1.15.0, the configuration `external-resources` delimiter separation(";") functionality between external resource names is not working as expected.
>
> Reproducing the bug setting up a flink cluster running using docker with two external resources named `abc` and `efg`:
> docker-compose.yml:
>
> {code:java}
> services:
> jobmanager:
> image: flink:1.15.0-scala_2.12
> ports:
> - "8081:8081"
> - "6123:6123"
> command: jobmanager
> environment:
> - |
> FLINK_PROPERTIES=
> jobmanager.rpc.address: jobmanager
> taskmanager:
> image: flink:1.15.0-scala_2.12
> depends_on:
> - jobmanager
> command: taskmanager
> environment:
> - |
> FLINK_PROPERTIES=
> jobmanager.rpc.address: jobmanager
> taskmanager.numberOfTaskSlots: 8
> external-resources: abc;efg
> external-resource.abc.amount: 1
> external-resource.efg.amount: 1{code}
>
> Related task manager logs:
>
> {code:java}
> root@2873c5ae72c2:/opt/flink# cat log/flink* | grep ExternalResourceUtils
> 2022-06-07 05:42:34,232 WARN org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - The amount of the abc;efg should be configured. Will ignore that resource.
> 2022-06-07 05:42:34,232 INFO org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled external resources: []
> 2022-06-07 05:42:34,236 WARN org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Could not find driver class name for abc;efg. Please make sure external-resource.abc;efg.driver-factory.class is configured.
> 2022-06-07 05:42:34,240 WARN org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - The amount of the abc;efg should be configured. Will ignore that resource.
> {code}
> Notice flink recognizes `abc;efg` as a single external resource rather than as `abc` and `efg`
>
>
> The same issue does not exist in flink version 1.14.4
> docker-compose.yml:
> {code:java}
> services:
> jobmanager:
> image: flink:1.14.4-scala_2.12
> ports:
> - "8081:8081"
> - "6123:6123"
> command: jobmanager
> environment:
> - |
> FLINK_PROPERTIES=
> jobmanager.rpc.address: jobmanager
> taskmanager:
> image: flink:1.14.4-scala_2.12
> depends_on:
> - jobmanager
> command: taskmanager
> environment:
> - |
> FLINK_PROPERTIES=
> jobmanager.rpc.address: jobmanager
> taskmanager.numberOfTaskSlots: 8
> external-resources: abc;efg
> external-resource.abc.amount: 1
> external-resource.efg.amount: 1 {code}
> related tm logs
> {code:java}
> docker logs ... | grep ExternalResourceUtils
> 2022-06-07 05:48:13,531 INFO org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled external resources: [abc, efg]
> 2022-06-07 05:48:13,534 WARN org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Could not find driver class name for abc. Please make sure external-resource.abc.driver-factory.class is configured.
> 2022-06-07 05:48:13,534 WARN org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Could not find driver class name for efg. Please make sure external-resource.efg.driver-factory.class is configured.
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)