You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by "Craig Condit (Jira)" <ji...@apache.org> on 2023/08/23 23:15:00 UTC
[jira] [Updated] (YUNIKORN-1931) Improve placeholder name generation
[ https://issues.apache.org/jira/browse/YUNIKORN-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Condit updated YUNIKORN-1931:
-----------------------------------
Description:
When creating placeholder pods, the name generation function is currently defined as:
{noformat}
tg-{tgID:20}-{appID:28}-{placeholderIndex}
{noformat}
This has several drawbacks:
* Duplicates can occur if multiple applications have a common 28-char prefix, or if task group IDs within an app have a common 20-character prefix.
* It is theoretically possible to ask for enough placeholders to overflow the maximum label length of 63 characters.
* The task group ID is probably less significant than appID when attempting to locate placeholder pods in listings.
If we introduce a random suffix instead of the placeholder index, we can ensure that we do not have name collisions or overflows:
{noformat}
tg-{appID:28}-{tgID:20}-{suffix:10}
{noformat}
This ensures a total length of 63 characters or less, places the appID foremost in the list, and if we use [a-z0-9] for each digit of the suffix, 36^10 possible combinations which will ensure a statistical impossibility of collision.
So instead of:
tg-driver-sparkapp-abcde-1
tg-executor-sparkapp-abcde-2
tg-executor-sparkapp-abcde-3
We would have:
tg-sparkapp-abcde-driver-38sh40fk58
tg-sparkapp-abcde-executor-2sg93hal23
tg-sparkapp-abcde-executor-a03xh5dl39
We should also document that placeholders always have unique names starting with "tg-" and may contain portions of the application ID and task group ID but that the specific formatting is subject to change.
was:
When creating placeholder pods, the name generation function is currently defined as:
tg-{tgID:20}-{appID:28}-{placeholderIndex}
This has several drawbacks:
* Duplicates can occur if multiple applications have a common 28-char prefix, or if task group IDs within an app have a common 20-character prefix.
* It is theoretically possible to ask for enough placeholders to overflow the maximum label length of 63 characters.
* The task group ID is probably less significant than appID when attempting to locate placeholder pods in listings.
If we introduce a random suffix instead of the placeholder index, we can ensure that we do not have name collisions or overflows:
tg-{appID:28}-{tgID:20}-{suffix:10}
This ensures a total length of 63 characters or less, places the appID foremost in the list, and if we use [a-z0-9] for each digit of the suffix, 36^10 possible combinations which will ensure a statistical impossibility of collision.
So instead of:
tg-driver-sparkapp-abcde-1
tg-executor-sparkapp-abcde-2
tg-executor-sparkapp-abcde-3
We would have:
tg-sparkapp-abcde-driver-38sh40fk58
tg-sparkapp-abcde-executor-2sg93hal23
tg-sparkapp-abcde-executor-a03xh5dl39
We should also document that placeholders always have unique names starting with "tg-" and may contain portions of the application ID and task group ID but that the specific formatting is subject to change.
> Improve placeholder name generation
> -----------------------------------
>
> Key: YUNIKORN-1931
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1931
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: shim - kubernetes
> Reporter: Craig Condit
> Assignee: Craig Condit
> Priority: Major
>
> When creating placeholder pods, the name generation function is currently defined as:
> {noformat}
> tg-{tgID:20}-{appID:28}-{placeholderIndex}
> {noformat}
> This has several drawbacks:
> * Duplicates can occur if multiple applications have a common 28-char prefix, or if task group IDs within an app have a common 20-character prefix.
> * It is theoretically possible to ask for enough placeholders to overflow the maximum label length of 63 characters.
> * The task group ID is probably less significant than appID when attempting to locate placeholder pods in listings.
> If we introduce a random suffix instead of the placeholder index, we can ensure that we do not have name collisions or overflows:
> {noformat}
> tg-{appID:28}-{tgID:20}-{suffix:10}
> {noformat}
> This ensures a total length of 63 characters or less, places the appID foremost in the list, and if we use [a-z0-9] for each digit of the suffix, 36^10 possible combinations which will ensure a statistical impossibility of collision.
> So instead of:
> tg-driver-sparkapp-abcde-1
> tg-executor-sparkapp-abcde-2
> tg-executor-sparkapp-abcde-3
> We would have:
> tg-sparkapp-abcde-driver-38sh40fk58
> tg-sparkapp-abcde-executor-2sg93hal23
> tg-sparkapp-abcde-executor-a03xh5dl39
> We should also document that placeholders always have unique names starting with "tg-" and may contain portions of the application ID and task group ID but that the specific formatting is subject to change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org