[jira] [Commented] (YARN-6259) Support pagination and optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16044062#comment-16044062 ] 

Hadoop QA commented on YARN-6259:
---------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  7s{color} | {color:red} YARN-6259 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-6259 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12855348/YARN-6259.001.patch |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/16165/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Support pagination and optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-6259
>                 URL: https://issues.apache.org/jira/browse/YARN-6259
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 2.8.1
>            Reporter: Tao Yang
>            Assignee: Tao Yang
>         Attachments: YARN-6259.001.patch
>
>
> Currently containerlogs REST API in NMWebServices will read and send the entire content of container logs. Most of container logs are large and it's useful to support pagination.
> * Add pagesize and pageindex parameters for containerlogs REST API
> {code}
> URL: http://<nm_address>/ws/v1/node/containerlogs/<container_id>/<file_name>
> QueryParams:
>   pagesize - max bytes of one page , default 1MB
>   pageindex - index of required page, default 0, can be nagative(set -1 will get the last page content)
> {code}
> * Add containerlogs-info REST API since sometimes we need to know the totalSize/pageSize/pageCount info of log 
> {code}
> URL: http://<nm_address>/ws/v1/node/containerlogs-info/<container_id>/<file_name>
> QueryParams:
>   pagesize - max bytes of one page , default 1MB
> Response example:
>   {"logInfo":{"totalSize":2497280,"pageSize":1048576,"pageCount":3}}
> {code}
> Moreover, the data transfer pipeline (disk --> read buffer --> NM buffer --> socket buffer) can be optimized to pipeline(disk --> read buffer --> socket buffer) with zero-copy approach.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org