[jira] [Updated] (YARN-6733) Add table for storing sub-application entities

["Vrushali C (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vrushali C updated YARN-6733:
-----------------------------
    Attachment: YARN-6733-YARN-5355.001.patch

Attaching v001.

- added a new package org.apache.hadoop.yarn.server.timelineservice.storage.subapplication and SubApplication* classes. 
- Updated test cases to include checks for this table's row key and row key converters (similar to tests for other tables).
- updated Timeline Schema creator to create this table.

The rowkey is 
 clusterId!subAppUserName!subAppName!entityType!entityPrefix!entityId

I have kept the schema like the entity table. Trying to remember if we wanted some other fields? Also, I am not sure if metrics will be part of this table, but I have added a metrics column family, since it's easier to do it now rather than change schema later.

The unit tests in timelineservice, timelineservice-hbase and timelineservice-hbase-tests pass. What I have not yet added is, checks for data to be stored and retrieved from this table. If anyone has any example records in their minds, I can add in some test around that data. 

Would appreciate some review on this.. 



> Add table for storing sub-application entities
> ----------------------------------------------
>
>                 Key: YARN-6733
>                 URL: https://issues.apache.org/jira/browse/YARN-6733
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Vrushali C
>            Assignee: Vrushali C
>         Attachments: YARN-6733-YARN-5355.001.patch
>
>
> After a discussion with Tez folks, we have been thinking over introducing a table to store  sub-application information.
> For example, if a Tez session runs for a certain period as User X and runs a few AMs. These AMs accept DAGs from other users. Tez will execute these dags with a doAs user. ATSv2 should store this information in a new table perhaps called as "sub_application" table. 
> This jira tracks the code changes needed for  table schema creation.
> I will file other jiras for writing to that table, updating the user name fields to include sub-application user etc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org