You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "Alexander Rojas (JIRA)" <ji...@apache.org> on 2017/11/30 15:45:00 UTC
[jira] [Updated] (MESOS-8283) Operator API `SUBSCRIBE` call doesn't
fully respect acls
[ https://issues.apache.org/jira/browse/MESOS-8283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Rojas updated MESOS-8283:
-----------------------------------
Description:
When an operator subscribes to the event stream using the {{SUBSCRIBE}} call, the initial response is properly authorized (with filter in the state). The same holds true for the events {{TASK_ADDED}} and {{TASK_UPDATED}} which, if a user has no right to see them he won't receive the updates, {{AGENT_ADDED}} seems to be respected too. However any user will get {{FRAMEWORK_ADDED}} and {{FRAMEWORK_UPDATED}} events.
I didn't {{AGENT_REMOVED}}
In order to test I used the following acls:
{code:javascript}
{
"permissive": true,
"view_roles" : [
{
"principals" : { "type" : "ANY" },
"roles" : { "values" : ["*"] }
},
{
"principals" : { "values" : ["hal-9000"] },
"roles" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"roles" : { "type" : "NONE" }
}
],
"view_framework": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
],
"view_tasks": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
],
"register_agent": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
]
}
{code}
the following credential files:
{noformat}
super super
hal-9000 hal-9000
glados glados
{noformat}
And launch a master and an agent as follows:
{noformat}
./mesos-master.sh \
--work_dir=/tmp/$USER/mesos/master \
--log_dir=/tmp/$USER/mesos/master/log \
--ip=${MASTER_IP} \
--agent_ping_timeout=5secs \
--max_agent_ping_timeouts=2 \
--agent_removal_rate_limit=1/1secs \
--http_framework_authenticators=basic \
--authenticate_http \
--authenticate_http_frameworks \
--acls=$HOME/testing/acls.json \
--credentials=$HOME/testing/credentials.txt
sudo ./mesos-agent.sh \
--work_dir=/tmp/$USER/mesos/agent \
--log_dir=/tmp/$USER/mesos/agent/log \
--containerizers=mesos,docker \
--master=${MASTER_IP}:5050 \
--authenticate_http_readwrite \
--http_authenticators=basic \
--acls=$HOME/Documents/workspace/testing/acls.json \
--http_credentials=$HOME/testing/credentials.txt
{noformat}
Once the master and agent are running, use the following script:
{code}
#! /usr/bin/env python3
import json
import recordio
import requests
import sys
from contextlib import closing
from typing import Dict
MACHINE_NAME='ip.to.machine'
MACHINE_IP='ip.to.machine'
def test_mesos_v1_master_subscribe(host: str, ip: str) -> None:
base_url = 'http://{}:5050'.format(host)
headers = {
'Content-Type': 'application/json',
'Accept': 'application/json',
'Connection': 'close',
}
decoder = recordio.Decoder(lambda s: json.loads(s.decode("UTF-8")))
with closing(requests.post(
base_url + '/api/v1',
json={'type': 'SUBSCRIBE'},
auth=('hal-9000', 'hal-9000'),
headers=headers,
stream=True,
)) as response:
assert response.status_code == 200, response.text
chunks = response.iter_content(chunk_size=None)
events = [] # type: ignore
for chunk in chunks:
for record in decoder.decode(chunk):
if record['type'] != 'HEARTBEAT':
print('got event "' + record['type'])
events.append(record['type'])
# It takes at least 10 mins for an agent to be marked as gone.
if len(events) >= 7:
print('got the following events "' + '", "'.join(events) + '"')
break
with closing(requests.post(
base_url + '/api/v1',
json={'type': 'SUBSCRIBE'},
auth=('glados', 'glados'),
headers=headers,
stream=True,
)) as response:
assert response.status_code == 200, response.text
chunks = response.iter_content(chunk_size=None)
for chunk in chunks:
for record in decoder.decode(chunk):
if record['type'] not in ('HEARTBEAT', 'SUBSCRIBED'):
print('It wasn\'t supposed to get "' + record['type'] + '"')
def main(args: Dict[str, str]) -> None:
test_mesos_v1_master_subscribe(MACHINE_NAME, MACHINE_IP)
if __name__ == '__main__':
main(sys.argv)
{code}
Finally while the machine is running you do the following:
1. Launch an extra agent:
{noformat}
sudo ./mesos-agent.sh \
--work_dir=/tmp/$USER/mesos/agent-2 \
--log_dir=/tmp/$USER/mesos/agent-2/log \
--containerizers=mesos,docker \
--port=5152 \
--master=${MASTER_IP}:5050 \
--authenticate_http_readwrite \
--http_authenticators=basic \
--acls=$HOME/testing/acls.json \
--http_credentials=$HOME/testing/credentials.txt
{noformat}
2. Teardown the agent.
3. Launch a framework and task:
{noformat}
./src/mesos-execute --master=${MASTER_IP}:5050 --command='while true; do echo "Hello World"; sleep 5; done;' --resources="cpus:1;mem:128;disk:32;ports:[31002-31003]" --name=hello-discovery --principal=hal-9000 --secret=hal-9000
{noformat}
4. Teardown the framework by killing the program {{mesos-execute}}.
5. Repeat (1) and (2).
The output of the script is the following:
{noformat}
got the following events "SUBSCRIBED", "FRAMEWORK_ADDED", "TASK_ADDED", "TASK_UPDATED", "TASK_UPDATED", "FRAMEWORK_REMOVED", "AGENT_ADDED"
It wasn't supposed to get "FRAMEWORK_ADDED"
It wasn't supposed to get "FRAMEWORK_REMOVED"
{noformat}
The last two lines should not appear.
was:
When an operator subscribes to the event stream using the {{SUBSCRIBE}} call, the initial response is properly authorized (with filter in the state). The same holds true for the events {{TASK_ADDED}} and {{TASK_UPDATED}} which, if a user has no right to see them he won't receive the updates, {{AGENT_ADDED}} seems to be respected too. However any user will get {{FRAMEWORK_ADDED}} and {{FRAMEWORK_UPDATED}} events.
I didn't {{AGENT_REMOVED}}
In order to test I used the following acls:
{code:javascript}
{
"permissive": true,
"view_roles" : [
{
"principals" : { "type" : "ANY" },
"roles" : { "values" : ["*"] }
},
{
"principals" : { "values" : ["hal-9000"] },
"roles" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"roles" : { "type" : "NONE" }
}
],
"view_framework": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
],
"view_tasks": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
],
"register_agent": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
]
}
{code}
the following credential files:
{noformat}
super super
hal-9000 hal-9000
glados glados
{noformat}
And launch a master and an agent as follows:
{noformat}
./mesos-master.sh \
--work_dir=/tmp/$USER/mesos/master \
--log_dir=/tmp/$USER/mesos/master/log \
--ip=${MASTER_IP} \
--agent_ping_timeout=5secs \
--max_agent_ping_timeouts=2 \
--agent_removal_rate_limit=1/1secs \
--http_framework_authenticators=basic \
--authenticate_http \
--authenticate_http_frameworks \
--acls=$HOME/testing/acls.json \
--credentials=$HOME/testing/credentials.txt
sudo ./mesos-agent.sh \
--work_dir=/tmp/$USER/mesos/agent \
--log_dir=/tmp/$USER/mesos/agent/log \
--containerizers=mesos,docker \
--master=${MASTER_IP}:5050 \
--authenticate_http_readwrite \
--http_authenticators=basic \
--acls=$HOME/Documents/workspace/testing/acls.json \
--http_credentials=$HOME/testing/credentials.txt
{noformat}
Once the master and agent are running, use the following script:
{code}
#! /usr/bin/env python3
import json
import recordio
import requests
import sys
from contextlib import closing
from typing import Dict
MACHINE_NAME='ip.to.machine'
MACHINE_IP='ip.to.machine'
def test_mesos_v1_master_subscribe(host: str, ip: str) -> None:
base_url = 'http://{}:5050'.format(host)
headers = {
'Content-Type': 'application/json',
'Accept': 'application/json',
'Connection': 'close',
}
decoder = recordio.Decoder(lambda s: json.loads(s.decode("UTF-8")))
with closing(requests.post(
base_url + '/api/v1',
json={'type': 'SUBSCRIBE'},
auth=('hal-9000', 'hal-9000'),
headers=headers,
stream=True,
)) as response:
assert response.status_code == 200, response.text
chunks = response.iter_content(chunk_size=None)
events = [] # type: ignore
for chunk in chunks:
for record in decoder.decode(chunk):
if record['type'] != 'HEARTBEAT':
print('got event "' + record['type'])
events.append(record['type'])
# It takes at least 10 mins for an agent to be marked as gone.
if len(events) >= 7:
print('got the following events "' + '", "'.join(events) + '"')
break
with closing(requests.post(
base_url + '/api/v1',
json={'type': 'SUBSCRIBE'},
auth=('glados', 'glados'),
headers=headers,
stream=True,
)) as response:
assert response.status_code == 200, response.text
chunks = response.iter_content(chunk_size=None)
for chunk in chunks:
for record in decoder.decode(chunk):
if record['type'] not in ('HEARTBEAT', 'SUBSCRIBED'):
print('It wasn\'t supposed to get "' + record['type'] + '"')
def main(args: Dict[str, str]) -> None:
test_mesos_v1_master_subscribe(MACHINE_NAME, MACHINE_IP)
if __name__ == '__main__':
main(sys.argv)
{code}
Finally while the machine is running you do the following:
1. Launch an extra agent:
{noformat}
sudo ./mesos-agent.sh \
--work_dir=/tmp/$USER/mesos/agent-2 \
--log_dir=/tmp/$USER/mesos/agent-2/log \
--containerizers=mesos,docker \
--port=5152 \
--master=${MASTER_IP}:5050 \
--authenticate_http_readwrite \
--http_authenticators=basic \
--acls=$HOME/testing/acls.json \
--http_credentials=$HOME/testing/credentials.txt
{noformat}
2. Teardown the agent.
3. Launch a framework and task:
{noformat}
./src/mesos-execute --master=${MASTER_IP}:5050 --command='while true; do echo "Hello World"; sleep 5; done;' --resources="cpus:1;mem:128;disk:32;ports:[31002-31003]" --name=hello-discovery --principal=hal-9000 --secret=hal-9000
{noformat}
4. Teardown the framework by killing the program {{mesos-execute}}.
5. Repeat (1) and (2).
The output of the script is the following:
{noformat}
got the following events "SUBSCRIBED", "FRAMEWORK_ADDED", "TASK_ADDED", "TASK_UPDATED", "TASK_UPDATED", "FRAMEWORK_REMOVED", "AGENT_ADDED", "AGENT_ADDED"
It wasn't supposed to get "FRAMEWORK_ADDED"
It wasn't supposed to get "FRAMEWORK_REMOVED"
{noformat}
> Operator API `SUBSCRIBE` call doesn't fully respect acls
> --------------------------------------------------------
>
> Key: MESOS-8283
> URL: https://issues.apache.org/jira/browse/MESOS-8283
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API, master
> Affects Versions: 1.4.0
> Reporter: Alexander Rojas
> Labels: mesosphere
>
> When an operator subscribes to the event stream using the {{SUBSCRIBE}} call, the initial response is properly authorized (with filter in the state). The same holds true for the events {{TASK_ADDED}} and {{TASK_UPDATED}} which, if a user has no right to see them he won't receive the updates, {{AGENT_ADDED}} seems to be respected too. However any user will get {{FRAMEWORK_ADDED}} and {{FRAMEWORK_UPDATED}} events.
> I didn't {{AGENT_REMOVED}}
> In order to test I used the following acls:
> {code:javascript}
> {
> "permissive": true,
> "view_roles" : [
> {
> "principals" : { "type" : "ANY" },
> "roles" : { "values" : ["*"] }
> },
> {
> "principals" : { "values" : ["hal-9000"] },
> "roles" : { "type" : "ANY" }
> },
> {
> "principals" : { "values" : ["glados"] },
> "roles" : { "type" : "NONE" }
> }
> ],
> "view_framework": [
> {
> "principals" : { "values" : ["hal-9000"] },
> "users" : { "type" : "ANY" }
> },
> {
> "principals" : { "values" : ["glados"] },
> "users" : { "type" : "NONE" }
> }
> ],
> "view_tasks": [
> {
> "principals" : { "values" : ["hal-9000"] },
> "users" : { "type" : "ANY" }
> },
> {
> "principals" : { "values" : ["glados"] },
> "users" : { "type" : "NONE" }
> }
> ],
> "register_agent": [
> {
> "principals" : { "values" : ["hal-9000"] },
> "users" : { "type" : "ANY" }
> },
> {
> "principals" : { "values" : ["glados"] },
> "users" : { "type" : "NONE" }
> }
> ]
> }
> {code}
> the following credential files:
> {noformat}
> super super
> hal-9000 hal-9000
> glados glados
> {noformat}
> And launch a master and an agent as follows:
> {noformat}
> ./mesos-master.sh \
> --work_dir=/tmp/$USER/mesos/master \
> --log_dir=/tmp/$USER/mesos/master/log \
> --ip=${MASTER_IP} \
> --agent_ping_timeout=5secs \
> --max_agent_ping_timeouts=2 \
> --agent_removal_rate_limit=1/1secs \
> --http_framework_authenticators=basic \
> --authenticate_http \
> --authenticate_http_frameworks \
> --acls=$HOME/testing/acls.json \
> --credentials=$HOME/testing/credentials.txt
> sudo ./mesos-agent.sh \
> --work_dir=/tmp/$USER/mesos/agent \
> --log_dir=/tmp/$USER/mesos/agent/log \
> --containerizers=mesos,docker \
> --master=${MASTER_IP}:5050 \
> --authenticate_http_readwrite \
> --http_authenticators=basic \
> --acls=$HOME/Documents/workspace/testing/acls.json \
> --http_credentials=$HOME/testing/credentials.txt
> {noformat}
> Once the master and agent are running, use the following script:
> {code}
> #! /usr/bin/env python3
> import json
> import recordio
> import requests
> import sys
> from contextlib import closing
> from typing import Dict
> MACHINE_NAME='ip.to.machine'
> MACHINE_IP='ip.to.machine'
> def test_mesos_v1_master_subscribe(host: str, ip: str) -> None:
> base_url = 'http://{}:5050'.format(host)
> headers = {
> 'Content-Type': 'application/json',
> 'Accept': 'application/json',
> 'Connection': 'close',
> }
> decoder = recordio.Decoder(lambda s: json.loads(s.decode("UTF-8")))
> with closing(requests.post(
> base_url + '/api/v1',
> json={'type': 'SUBSCRIBE'},
> auth=('hal-9000', 'hal-9000'),
> headers=headers,
> stream=True,
> )) as response:
> assert response.status_code == 200, response.text
> chunks = response.iter_content(chunk_size=None)
> events = [] # type: ignore
> for chunk in chunks:
> for record in decoder.decode(chunk):
> if record['type'] != 'HEARTBEAT':
> print('got event "' + record['type'])
> events.append(record['type'])
> # It takes at least 10 mins for an agent to be marked as gone.
> if len(events) >= 7:
> print('got the following events "' + '", "'.join(events) + '"')
> break
> with closing(requests.post(
> base_url + '/api/v1',
> json={'type': 'SUBSCRIBE'},
> auth=('glados', 'glados'),
> headers=headers,
> stream=True,
> )) as response:
> assert response.status_code == 200, response.text
> chunks = response.iter_content(chunk_size=None)
> for chunk in chunks:
> for record in decoder.decode(chunk):
> if record['type'] not in ('HEARTBEAT', 'SUBSCRIBED'):
> print('It wasn\'t supposed to get "' + record['type'] + '"')
> def main(args: Dict[str, str]) -> None:
> test_mesos_v1_master_subscribe(MACHINE_NAME, MACHINE_IP)
> if __name__ == '__main__':
> main(sys.argv)
> {code}
> Finally while the machine is running you do the following:
> 1. Launch an extra agent:
> {noformat}
> sudo ./mesos-agent.sh \
> --work_dir=/tmp/$USER/mesos/agent-2 \
> --log_dir=/tmp/$USER/mesos/agent-2/log \
> --containerizers=mesos,docker \
> --port=5152 \
> --master=${MASTER_IP}:5050 \
> --authenticate_http_readwrite \
> --http_authenticators=basic \
> --acls=$HOME/testing/acls.json \
> --http_credentials=$HOME/testing/credentials.txt
> {noformat}
> 2. Teardown the agent.
> 3. Launch a framework and task:
> {noformat}
> ./src/mesos-execute --master=${MASTER_IP}:5050 --command='while true; do echo "Hello World"; sleep 5; done;' --resources="cpus:1;mem:128;disk:32;ports:[31002-31003]" --name=hello-discovery --principal=hal-9000 --secret=hal-9000
> {noformat}
> 4. Teardown the framework by killing the program {{mesos-execute}}.
> 5. Repeat (1) and (2).
> The output of the script is the following:
> {noformat}
> got the following events "SUBSCRIBED", "FRAMEWORK_ADDED", "TASK_ADDED", "TASK_UPDATED", "TASK_UPDATED", "FRAMEWORK_REMOVED", "AGENT_ADDED"
> It wasn't supposed to get "FRAMEWORK_ADDED"
> It wasn't supposed to get "FRAMEWORK_REMOVED"
> {noformat}
> The last two lines should not appear.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)