[jira] [Commented] (YARN-4178) [storage implementation] app id as string in row keys can cause incorrect ordering

["Sangjin Lee (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934442#comment-14934442 ] 

Sangjin Lee commented on YARN-4178:
-----------------------------------

Thanks [~varun_saxena] for the proposed patch. I'm generally in agreement with the approach.

As for the "application" prefix, is there a discussion in the YARN community in general of making this prefix customizable? I'm just not familiar with the current state of that discussion, although I recall hearing about it in the past. That would help us determine whether this is significant enough for us to care about it.

One small nit on the patch: in {{TimelineReaderUtils.java}}, there is following code:

{code}
    long clusterTs = Long.MAX_VALUE - Bytes.toLong(appIdBytes, 0);
    int seqId = Integer.MAX_VALUE - Bytes.toInt(appIdBytes, Bytes.SIZEOF_LONG);
{code}

Although it is exactly equivalent, for readability it might be bit better to spell out the size and the offset explicitly, and also use {{invert()}}:

{code}
    long clusterTs = invert(Bytes.toLong(appIdBytes, 0, Bytes.SIZEOF_LONG));
    int seqId = invert(Bytes.toInt(appIdBytes, Bytes.SIZEOF_LONG, Bytes.SIZEOF_INT));
{code}

That would also make it a little more symmetric with the writer util. Also, use {{invert()}} for the writer util.

> [storage implementation] app id as string in row keys can cause incorrect ordering
> ----------------------------------------------------------------------------------
>
>                 Key: YARN-4178
>                 URL: https://issues.apache.org/jira/browse/YARN-4178
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-4178-YARN-2928.01.patch
>
>
> Currently the app id is used in various places as part of row keys. However, currently they are treated as strings. This will cause a problem with ordering when the id portion of the app id rolls over to the next digit.
> For example, "app_1234567890_10000" will be considered *earlier* than "app_1234567890_9999". We should correct this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)