You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "Dalton Matos Coelho Barreto (JIRA)" <ji...@apache.org> on 2018/01/31 14:50:01 UTC
[jira] [Updated] (MESOS-8517) Master http endpoint /tasks should
include "executor/directory" field
[ https://issues.apache.org/jira/browse/MESOS-8517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dalton Matos Coelho Barreto updated MESOS-8517:
-----------------------------------------------
Description:
Hello all,
I noticed that the Task information returned by a mesos agent is different from the task information returned by the mesos master.
Agent's {{/state}} endpoint, when returning each task, includes the {{directory}} field (inside the {{executors}} array), which is super useful if we want to browse this task's sandbox files. But at the same time, the agent HTTP API does not provide a way to get a task by its ID.
So we need to talk to the Mesos master to get a task by its ID and we can do it using the {{/tasks?task_id=<task_id}} endpoint. But here we do not have any information about this task's executor.
So given a TaskID, to be able to download/browse its sandbox files we need 3+ API calls:
* One to the master ({{/tasks}}) to get the task's JSON;
* Anoter to the master ({{/slaves?slave_id=<slave-id>}} to get the slave IP address;
* Another to this slave ({{//state}}) to get all its tasks. Then we iterate the lists of tasks (of each executor) searching for our TaskID; get the {{directory}} field.
* And then, the final API call, made on endpoint ({{/files/browse?path=}}) to get the contents of the required file
I would like to know if it's possible to the Master to include this {{directory}} field when returning info about a specific task.
Also, is there a easier way to do what I described here? GIven a TaskID, get the content of a file inside its sandbox (stdout/stderr would be the most common options)
Thanks a lot.
Here is one example of each JSON returned (only the important parts)
Master {{/tasks?task_id=<task_id>}} endpoint
{noformat}
{
"tasks" : [
{
"resources" : {
"disk" : 0,
"gpus" : 0,
"ports" : "[31720-31720]",
"cpus" : 0.3,
"mem" : 256
},
"executor_id" : "",
"labels" : [
],
"framework_id" : "c7fe8caf-3772-4570-bb36-149d600053af-0000",
"id" : "",
"slave_id" : "78a58d98-f011-41bf-b5d7-a92700bcda93-S20847",
"discovery" : { },
"role" : "*",
"state" : "TASK_RUNNING"
}
]
}
{noformat}
Slave's {{/state}} endpoint
{noformat}
{
"build_user" : "centos",
"frameworks" : [
{
"role" : "*",
"completed_executors" : ,
"executors" : [
{
"tasks" : [ ],
"directory" : "/tmp/mesos/slaves/78a58d98-f011-41bf-b5d7-a92700bcda93-S20847/frameworks/c7fe8caf-3772-4570-bb36-149d600053af-0000/executors/task_name.447d755d-0616-11e8-96d4-ee61be2ee5b4/runs/14f119de-708c-4978-b04e-60886a12be57",
"container" : "14f119de-708c-4978-b04e-60886a12be57",
"queued_tasks" : [],
"resources" : {
"cpus" : 0.4,
"mem" : 288,
"ports" : "[31720-31720]",
"disk" : 0,
"gpus" : 0
},
"completed_tasks" : [],
"labels" : [ ],
"role" : "*"
}
],
"id" : "c7fe8caf-3772-4570-bb36-149d600053af-0000",
"checkpoint" : true
}
]
}
{noformat}
Thanks,
was:
'Hello all,
I noticed that the Task information returned by a mesos agent is different from the task information returned by the mesos master.
Agent's {{/state}} endpoint, when returning each task, includes the {{directory}} field (inside the {{executors}} array), which is super useful if we want to browse this task's sandbox files. But at the same time, the agent HTTP API does not provide a way to get a task by its ID.
So we need to talk to the Mesos master to get a task by its ID and we can do it using the {{/tasks?task_id=<task_id}} endpoint. But here we do not have any information about this task's executor.
So given a TaskID, to be able to download/browse its sandbox files we need 3+ API calls:
* One to the master ({{/tasks}}) to get the task's JSON;
* Anoter to the master ({{/slaves?slave_id=<slave-id>}} to get the slave IP address;
* Another to this slave ({{//state}}) to get all its tasks. Then we iterate the lists of tasks (of each executor) searching for our TaskID; get the {{directory}} field.
* And then, the final API call, made on endpoint ({{/files/browse?path=}}) to get the contents of the required file
I would like to know if it's possible to the Master to include this {{directory}} field when returning info about a specific task.
Also, is there a easier way to do what I described here? GIven a TaskID, get the content of a file inside its sandbox (stdout/stderr would be the most common options)
Thanks a lot.
Here is one example of each JSON returned (only the important parts)
Master {{/tasks?task_id=<task_id>}} endpoint
{noformat}
{
"tasks" : [
{
"resources" : {
"disk" : 0,
"gpus" : 0,
"ports" : "[31720-31720]",
"cpus" : 0.3,
"mem" : 256
},
"executor_id" : "",
"labels" : [
],
"framework_id" : "c7fe8caf-3772-4570-bb36-149d600053af-0000",
"id" : "",
"slave_id" : "78a58d98-f011-41bf-b5d7-a92700bcda93-S20847",
"discovery" : { },
"role" : "*",
"state" : "TASK_RUNNING"
}
]
}
{noformat}
Slave's {{/state}} endpoint
{noformat}
{
"build_user" : "centos",
"frameworks" : [
{
"role" : "*",
"completed_executors" : ,
"executors" : [
{
"tasks" : [ ],
"directory" : "/tmp/mesos/slaves/78a58d98-f011-41bf-b5d7-a92700bcda93-S20847/frameworks/c7fe8caf-3772-4570-bb36-149d600053af-0000/executors/task_name.447d755d-0616-11e8-96d4-ee61be2ee5b4/runs/14f119de-708c-4978-b04e-60886a12be57",
"container" : "14f119de-708c-4978-b04e-60886a12be57",
"queued_tasks" : [],
"resources" : {
"cpus" : 0.4,
"mem" : 288,
"ports" : "[31720-31720]",
"disk" : 0,
"gpus" : 0
},
"completed_tasks" : [],
"labels" : [ ],
"role" : "*"
}
],
"id" : "c7fe8caf-3772-4570-bb36-149d600053af-0000",
"checkpoint" : true
}
]
}
{noformat}
Thanks,
> Master http endpoint /tasks should include "executor/directory" field
> ---------------------------------------------------------------------
>
> Key: MESOS-8517
> URL: https://issues.apache.org/jira/browse/MESOS-8517
> Project: Mesos
> Issue Type: Improvement
> Components: HTTP API
> Affects Versions: 1.4.1
> Reporter: Dalton Matos Coelho Barreto
> Priority: Minor
>
> Hello all,
> I noticed that the Task information returned by a mesos agent is different from the task information returned by the mesos master.
> Agent's {{/state}} endpoint, when returning each task, includes the {{directory}} field (inside the {{executors}} array), which is super useful if we want to browse this task's sandbox files. But at the same time, the agent HTTP API does not provide a way to get a task by its ID.
> So we need to talk to the Mesos master to get a task by its ID and we can do it using the {{/tasks?task_id=<task_id}} endpoint. But here we do not have any information about this task's executor.
> So given a TaskID, to be able to download/browse its sandbox files we need 3+ API calls:
> * One to the master ({{/tasks}}) to get the task's JSON;
> * Anoter to the master ({{/slaves?slave_id=<slave-id>}} to get the slave IP address;
> * Another to this slave ({{//state}}) to get all its tasks. Then we iterate the lists of tasks (of each executor) searching for our TaskID; get the {{directory}} field.
> * And then, the final API call, made on endpoint ({{/files/browse?path=}}) to get the contents of the required file
> I would like to know if it's possible to the Master to include this {{directory}} field when returning info about a specific task.
> Also, is there a easier way to do what I described here? GIven a TaskID, get the content of a file inside its sandbox (stdout/stderr would be the most common options)
> Thanks a lot.
> Here is one example of each JSON returned (only the important parts)
> Master {{/tasks?task_id=<task_id>}} endpoint
> {noformat}
> {
> "tasks" : [
> {
> "resources" : {
> "disk" : 0,
> "gpus" : 0,
> "ports" : "[31720-31720]",
> "cpus" : 0.3,
> "mem" : 256
> },
> "executor_id" : "",
> "labels" : [
> ],
> "framework_id" : "c7fe8caf-3772-4570-bb36-149d600053af-0000",
> "id" : "",
> "slave_id" : "78a58d98-f011-41bf-b5d7-a92700bcda93-S20847",
> "discovery" : { },
> "role" : "*",
> "state" : "TASK_RUNNING"
> }
> ]
> }
> {noformat}
> Slave's {{/state}} endpoint
> {noformat}
> {
> "build_user" : "centos",
> "frameworks" : [
> {
> "role" : "*",
> "completed_executors" : ,
> "executors" : [
> {
> "tasks" : [ ],
> "directory" : "/tmp/mesos/slaves/78a58d98-f011-41bf-b5d7-a92700bcda93-S20847/frameworks/c7fe8caf-3772-4570-bb36-149d600053af-0000/executors/task_name.447d755d-0616-11e8-96d4-ee61be2ee5b4/runs/14f119de-708c-4978-b04e-60886a12be57",
> "container" : "14f119de-708c-4978-b04e-60886a12be57",
> "queued_tasks" : [],
> "resources" : {
> "cpus" : 0.4,
> "mem" : 288,
> "ports" : "[31720-31720]",
> "disk" : 0,
> "gpus" : 0
> },
> "completed_tasks" : [],
> "labels" : [ ],
> "role" : "*"
> }
> ],
> "id" : "c7fe8caf-3772-4570-bb36-149d600053af-0000",
> "checkpoint" : true
> }
> ]
> }
> {noformat}
> Thanks,
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)