[jira] [Commented] (YUNIKORN-1931) Improve placeholder name generation

["Craig Condit (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YUNIKORN-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758297#comment-17758297 ] 

Craig Condit commented on YUNIKORN-1931:
----------------------------------------

One potential issue I've discovered is that since placeholder names are no longer predictable, it becomes more difficult during recovery to skip creating placeholders since the names will not be consistent after a scheduler restart.

[~wilfreds] any ideas on what to do here? One thought would be to simplify recovery and not recreate placeholders if any are already present, but that might not be the best path...

> Improve placeholder name generation
> -----------------------------------
>
>                 Key: YUNIKORN-1931
>                 URL: https://issues.apache.org/jira/browse/YUNIKORN-1931
>             Project: Apache YuniKorn
>          Issue Type: Improvement
>          Components: shim - kubernetes
>            Reporter: Craig Condit
>            Assignee: Craig Condit
>            Priority: Major
>
> When creating placeholder pods, the name generation function is currently defined as:
> {noformat}
> tg-{tgID:20}-{appID:28}-{placeholderIndex}
> {noformat}
> This has several drawbacks:
> * Duplicates can occur if multiple applications have a common 28-char prefix, or if task group IDs within an app have a common 20-character prefix.
> * It is theoretically possible to ask for enough placeholders to overflow the maximum label length of 63 characters.
> * The task group ID is probably less significant than appID when attempting to locate placeholder pods in listings.
> If we introduce a random suffix instead of the placeholder index, we can ensure that we do not have name collisions or overflows:
> {noformat}
> tg-{appID:28}-{tgID:20}-{suffix:10}
> {noformat}
> This ensures a total length of 63 characters or less, places the appID foremost in the list, and if we use [a-z0-9] for each digit of the suffix, 36^10 possible combinations which will ensure a statistical impossibility of collision.
> So instead of:
> tg-driver-sparkapp-abcde-1
> tg-executor-sparkapp-abcde-2
> tg-executor-sparkapp-abcde-3
> We would have:
> tg-sparkapp-abcde-driver-38sh40fk58
> tg-sparkapp-abcde-executor-2sg93hal23
> tg-sparkapp-abcde-executor-a03xh5dl39
> We should also document that placeholders always have unique names starting with "tg-" and may contain portions of the application ID and task group ID but that the specific formatting is subject to change.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org