You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Flink Jira Bot (Jira)" <ji...@apache.org> on 2023/08/11 10:36:00 UTC
[jira] [Updated] (FLINK-27928) External Resource Framework: 'external-resources' name delimitation not working as expected
[ https://issues.apache.org/jira/browse/FLINK-27928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flink Jira Bot updated FLINK-27928:
-----------------------------------
Labels: auto-deprioritized-major external-resources stale-minor (was: auto-deprioritized-major external-resources)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized.
> External Resource Framework: 'external-resources' name delimitation not working as expected
> -------------------------------------------------------------------------------------------
>
> Key: FLINK-27928
> URL: https://issues.apache.org/jira/browse/FLINK-27928
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Configuration
> Affects Versions: 1.15.0
> Reporter: James Cho
> Priority: Minor
> Labels: auto-deprioritized-major, external-resources, stale-minor
>
> For flink version 1.15.0, the configuration `external-resources` delimiter separation(";") functionality between external resource names is not working as expected.
>
> Reproducing the bug setting up a flink cluster running using docker with two external resources named `abc` and `efg`:
> docker-compose.yml:
>
> {code:java}
> services:
> jobmanager:
> image: flink:1.15.0-scala_2.12
> ports:
> - "8081:8081"
> - "6123:6123"
> command: jobmanager
> environment:
> - |
> FLINK_PROPERTIES=
> jobmanager.rpc.address: jobmanager
> taskmanager:
> image: flink:1.15.0-scala_2.12
> depends_on:
> - jobmanager
> command: taskmanager
> environment:
> - |
> FLINK_PROPERTIES=
> jobmanager.rpc.address: jobmanager
> taskmanager.numberOfTaskSlots: 8
> external-resources: abc;efg
> external-resource.abc.amount: 1
> external-resource.efg.amount: 1{code}
>
> Related task manager logs:
>
> {code:java}
> root@2873c5ae72c2:/opt/flink# cat log/flink* | grep ExternalResourceUtils
> 2022-06-07 05:42:34,232 WARN org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - The amount of the abc;efg should be configured. Will ignore that resource.
> 2022-06-07 05:42:34,232 INFO org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled external resources: []
> 2022-06-07 05:42:34,236 WARN org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Could not find driver class name for abc;efg. Please make sure external-resource.abc;efg.driver-factory.class is configured.
> 2022-06-07 05:42:34,240 WARN org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - The amount of the abc;efg should be configured. Will ignore that resource.
> {code}
> Notice flink recognizes `abc;efg` as a single external resource rather than as `abc` and `efg`
>
>
> The same issue does not exist in flink version 1.14.4
> docker-compose.yml:
> {code:java}
> services:
> jobmanager:
> image: flink:1.14.4-scala_2.12
> ports:
> - "8081:8081"
> - "6123:6123"
> command: jobmanager
> environment:
> - |
> FLINK_PROPERTIES=
> jobmanager.rpc.address: jobmanager
> taskmanager:
> image: flink:1.14.4-scala_2.12
> depends_on:
> - jobmanager
> command: taskmanager
> environment:
> - |
> FLINK_PROPERTIES=
> jobmanager.rpc.address: jobmanager
> taskmanager.numberOfTaskSlots: 8
> external-resources: abc;efg
> external-resource.abc.amount: 1
> external-resource.efg.amount: 1 {code}
> related tm logs
> {code:java}
> docker logs ... | grep ExternalResourceUtils
> 2022-06-07 05:48:13,531 INFO org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled external resources: [abc, efg]
> 2022-06-07 05:48:13,534 WARN org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Could not find driver class name for abc. Please make sure external-resource.abc.driver-factory.class is configured.
> 2022-06-07 05:48:13,534 WARN org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Could not find driver class name for efg. Please make sure external-resource.efg.driver-factory.class is configured.
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)