You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Joseph Wu <jo...@mesosphere.io> on 2015/11/02 21:21:45 UTC
Review Request 39611: Remove binary `data` fields from state
endpoints.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/
-----------------------------------------------------------
Review request for mesos, Ben Mahler and Artem Harutyunyan.
Bugs: MESOS-3771
https://issues.apache.org/jira/browse/MESOS-3771
Repository: mesos
Description
-------
We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
Diffs
-----
src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
Diff: https://reviews.apache.org/r/39611/diff/
Testing
-------
`make check`
Apparently, none of our tests check these fields.
(Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
Thanks,
Joseph Wu
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Guangya Liu <gy...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/#review104823
-----------------------------------------------------------
Ship it!
I think that the upgrade documentshould also be updated if this field is removed.
- Guangya Liu
On 十一月 2, 2015, 8:21 p.m., Joseph Wu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39611/
> -----------------------------------------------------------
>
> (Updated 十一月 2, 2015, 8:21 p.m.)
>
>
> Review request for mesos, Ben Mahler and Artem Harutyunyan.
>
>
> Bugs: MESOS-3771
> https://issues.apache.org/jira/browse/MESOS-3771
>
>
> Repository: mesos
>
>
> Description
> -------
>
> We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
>
> The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
>
>
> Diffs
> -----
>
> src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
> src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
>
> Diff: https://reviews.apache.org/r/39611/diff/
>
>
> Testing
> -------
>
> `make check`
>
> Apparently, none of our tests check these fields.
>
> (Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
>
>
> Thanks,
>
> Joseph Wu
>
>
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Mesos ReviewBot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/#review104824
-----------------------------------------------------------
Patch looks great!
Reviews applied: [39611]
All tests passed.
- Mesos ReviewBot
On Nov. 2, 2015, 8:21 p.m., Joseph Wu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39611/
> -----------------------------------------------------------
>
> (Updated Nov. 2, 2015, 8:21 p.m.)
>
>
> Review request for mesos, Ben Mahler and Artem Harutyunyan.
>
>
> Bugs: MESOS-3771
> https://issues.apache.org/jira/browse/MESOS-3771
>
>
> Repository: mesos
>
>
> Description
> -------
>
> We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
>
> The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
>
>
> Diffs
> -----
>
> src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
> src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
>
> Diff: https://reviews.apache.org/r/39611/diff/
>
>
> Testing
> -------
>
> `make check`
>
> Apparently, none of our tests check these fields.
>
> (Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
>
>
> Thanks,
>
> Joseph Wu
>
>
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Mesos ReviewBot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/#review104855
-----------------------------------------------------------
Patch looks great!
Reviews applied: [39611]
All tests passed.
- Mesos ReviewBot
On Nov. 3, 2015, 1:06 a.m., Joseph Wu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39611/
> -----------------------------------------------------------
>
> (Updated Nov. 3, 2015, 1:06 a.m.)
>
>
> Review request for mesos, Ben Mahler and Artem Harutyunyan.
>
>
> Bugs: MESOS-3771
> https://issues.apache.org/jira/browse/MESOS-3771
>
>
> Repository: mesos
>
>
> Description
> -------
>
> We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
>
> The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
>
>
> Diffs
> -----
>
> docs/upgrades.md 3854e59f68d53892b2673a4542ea2aaf96936cbb
> src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
> src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
>
> Diff: https://reviews.apache.org/r/39611/diff/
>
>
> Testing
> -------
>
> `make check` and visually checked markdown for correctness.
>
> Apparently, none of our tests check these fields.
>
> (Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
>
>
> Thanks,
>
> Joseph Wu
>
>
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Joseph Wu <jo...@mesosphere.io>.
> On Nov. 4, 2015, 2:59 p.m., Ben Mahler wrote:
> > docs/upgrades.md, line 16
> > <https://reviews.apache.org/r/39611/diff/2/?file=1114331#file1114331line16>
> >
> > Can we be clear about exactly what is being removed here and give some background (link to the ticket)? Specifically, it would be helpful to say exactly which fields instead of referring to arbitrary binary data.
I added onto the message.
I'm not sure how understandable the JSON "search string" is. (Modelled after the syntax of `JSON::Object::find`.)
- Joseph
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/#review105169
-----------------------------------------------------------
On Nov. 4, 2015, 4:02 p.m., Joseph Wu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39611/
> -----------------------------------------------------------
>
> (Updated Nov. 4, 2015, 4:02 p.m.)
>
>
> Review request for mesos, Ben Mahler and Artem Harutyunyan.
>
>
> Bugs: MESOS-3771
> https://issues.apache.org/jira/browse/MESOS-3771
>
>
> Repository: mesos
>
>
> Description
> -------
>
> We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
>
> The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
>
>
> Diffs
> -----
>
> docs/upgrades.md 3854e59f68d53892b2673a4542ea2aaf96936cbb
> src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
> src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
>
> Diff: https://reviews.apache.org/r/39611/diff/
>
>
> Testing
> -------
>
> `make check` and visually checked markdown for correctness.
>
> Apparently, none of our tests check these fields.
>
> (Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
>
>
> Thanks,
>
> Joseph Wu
>
>
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/#review105169
-----------------------------------------------------------
Ship it!
Thanks Joseph, just a minor comment about the upgrade note.
docs/upgrades.md (line 16)
<https://reviews.apache.org/r/39611/#comment163589>
Can we be clear about exactly what is being removed here and give some background (link to the ticket)? Specifically, it would be helpful to say exactly which fields instead of referring to arbitrary binary data.
- Ben Mahler
On Nov. 3, 2015, 1:06 a.m., Joseph Wu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39611/
> -----------------------------------------------------------
>
> (Updated Nov. 3, 2015, 1:06 a.m.)
>
>
> Review request for mesos, Ben Mahler and Artem Harutyunyan.
>
>
> Bugs: MESOS-3771
> https://issues.apache.org/jira/browse/MESOS-3771
>
>
> Repository: mesos
>
>
> Description
> -------
>
> We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
>
> The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
>
>
> Diffs
> -----
>
> docs/upgrades.md 3854e59f68d53892b2673a4542ea2aaf96936cbb
> src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
> src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
>
> Diff: https://reviews.apache.org/r/39611/diff/
>
>
> Testing
> -------
>
> `make check` and visually checked markdown for correctness.
>
> Apparently, none of our tests check these fields.
>
> (Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
>
>
> Thanks,
>
> Joseph Wu
>
>
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Guangya Liu <gy...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/#review104846
-----------------------------------------------------------
docs/upgrades.md (line 16)
<https://reviews.apache.org/r/39611/#comment163113>
Shall we highlight `data` here?
- Guangya Liu
On 十一月 3, 2015, 1:06 a.m., Joseph Wu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39611/
> -----------------------------------------------------------
>
> (Updated 十一月 3, 2015, 1:06 a.m.)
>
>
> Review request for mesos, Ben Mahler and Artem Harutyunyan.
>
>
> Bugs: MESOS-3771
> https://issues.apache.org/jira/browse/MESOS-3771
>
>
> Repository: mesos
>
>
> Description
> -------
>
> We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
>
> The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
>
>
> Diffs
> -----
>
> docs/upgrades.md 3854e59f68d53892b2673a4542ea2aaf96936cbb
> src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
> src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
>
> Diff: https://reviews.apache.org/r/39611/diff/
>
>
> Testing
> -------
>
> `make check` and visually checked markdown for correctness.
>
> Apparently, none of our tests check these fields.
>
> (Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
>
>
> Thanks,
>
> Joseph Wu
>
>
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Guangya Liu <gy...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/#review105182
-----------------------------------------------------------
Ship it!
Ship It!
- Guangya Liu
On 十一月 5, 2015, 12:02 a.m., Joseph Wu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39611/
> -----------------------------------------------------------
>
> (Updated 十一月 5, 2015, 12:02 a.m.)
>
>
> Review request for mesos, Ben Mahler and Artem Harutyunyan.
>
>
> Bugs: MESOS-3771
> https://issues.apache.org/jira/browse/MESOS-3771
>
>
> Repository: mesos
>
>
> Description
> -------
>
> We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
>
> The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
>
>
> Diffs
> -----
>
> docs/upgrades.md 3854e59f68d53892b2673a4542ea2aaf96936cbb
> src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
> src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
>
> Diff: https://reviews.apache.org/r/39611/diff/
>
>
> Testing
> -------
>
> `make check` and visually checked markdown for correctness.
>
> Apparently, none of our tests check these fields.
>
> (Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
>
>
> Thanks,
>
> Joseph Wu
>
>
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/#review105188
-----------------------------------------------------------
Ship it!
Ship It!
- Ben Mahler
On Nov. 5, 2015, 12:36 a.m., Joseph Wu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39611/
> -----------------------------------------------------------
>
> (Updated Nov. 5, 2015, 12:36 a.m.)
>
>
> Review request for mesos, Ben Mahler and Artem Harutyunyan.
>
>
> Bugs: MESOS-3771
> https://issues.apache.org/jira/browse/MESOS-3771
>
>
> Repository: mesos
>
>
> Description
> -------
>
> We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
>
> The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
>
>
> Diffs
> -----
>
> docs/upgrades.md c7bf6a226fb6f5af8ef42bb9e9b843099fdd4e71
> src/common/http.cpp abb4c6bc2fe547bdc3a15682bfa680a52acc75a8
> src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
>
> Diff: https://reviews.apache.org/r/39611/diff/
>
>
> Testing
> -------
>
> `make check` and visually checked markdown for correctness.
>
> Apparently, none of our tests check these fields.
>
> (Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
>
>
> Thanks,
>
> Joseph Wu
>
>
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Joseph Wu <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/
-----------------------------------------------------------
(Updated Nov. 4, 2015, 4:36 p.m.)
Review request for mesos, Ben Mahler and Artem Harutyunyan.
Changes
-------
Tweak upgrades.md.
Bugs: MESOS-3771
https://issues.apache.org/jira/browse/MESOS-3771
Repository: mesos
Description
-------
We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
Diffs (updated)
-----
docs/upgrades.md c7bf6a226fb6f5af8ef42bb9e9b843099fdd4e71
src/common/http.cpp abb4c6bc2fe547bdc3a15682bfa680a52acc75a8
src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
Diff: https://reviews.apache.org/r/39611/diff/
Testing
-------
`make check` and visually checked markdown for correctness.
Apparently, none of our tests check these fields.
(Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
Thanks,
Joseph Wu
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Joseph Wu <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/
-----------------------------------------------------------
(Updated Nov. 4, 2015, 4:31 p.m.)
Review request for mesos, Ben Mahler and Artem Harutyunyan.
Changes
-------
Adjusted language in upgrades.md per offline discussion with BenM.
Bugs: MESOS-3771
https://issues.apache.org/jira/browse/MESOS-3771
Repository: mesos
Description
-------
We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
Diffs (updated)
-----
docs/upgrades.md c7bf6a226fb6f5af8ef42bb9e9b843099fdd4e71
src/common/http.cpp abb4c6bc2fe547bdc3a15682bfa680a52acc75a8
src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
Diff: https://reviews.apache.org/r/39611/diff/
Testing
-------
`make check` and visually checked markdown for correctness.
Apparently, none of our tests check these fields.
(Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
Thanks,
Joseph Wu
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Joseph Wu <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/
-----------------------------------------------------------
(Updated Nov. 4, 2015, 4:11 p.m.)
Review request for mesos, Ben Mahler and Artem Harutyunyan.
Changes
-------
Rebased on the other change to upgrades.md.
Bugs: MESOS-3771
https://issues.apache.org/jira/browse/MESOS-3771
Repository: mesos
Description
-------
We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
Diffs (updated)
-----
docs/upgrades.md c7bf6a226fb6f5af8ef42bb9e9b843099fdd4e71
src/common/http.cpp abb4c6bc2fe547bdc3a15682bfa680a52acc75a8
src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
Diff: https://reviews.apache.org/r/39611/diff/
Testing
-------
`make check` and visually checked markdown for correctness.
Apparently, none of our tests check these fields.
(Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
Thanks,
Joseph Wu
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Joseph Wu <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/
-----------------------------------------------------------
(Updated Nov. 4, 2015, 4:02 p.m.)
Review request for mesos, Ben Mahler and Artem Harutyunyan.
Changes
-------
Expand on the upgrades.md comment.
Bugs: MESOS-3771
https://issues.apache.org/jira/browse/MESOS-3771
Repository: mesos
Description
-------
We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
Diffs (updated)
-----
docs/upgrades.md 3854e59f68d53892b2673a4542ea2aaf96936cbb
src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
Diff: https://reviews.apache.org/r/39611/diff/
Testing
-------
`make check` and visually checked markdown for correctness.
Apparently, none of our tests check these fields.
(Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
Thanks,
Joseph Wu
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Joseph Wu <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/
-----------------------------------------------------------
(Updated Nov. 2, 2015, 5:06 p.m.)
Review request for mesos, Ben Mahler and Artem Harutyunyan.
Changes
-------
Add a note to `upgrades.md`.
Bugs: MESOS-3771
https://issues.apache.org/jira/browse/MESOS-3771
Repository: mesos
Description
-------
We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
Diffs (updated)
-----
docs/upgrades.md 3854e59f68d53892b2673a4542ea2aaf96936cbb
src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
Diff: https://reviews.apache.org/r/39611/diff/
Testing (updated)
-------
`make check` and visually checked markdown for correctness.
Apparently, none of our tests check these fields.
(Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
Thanks,
Joseph Wu
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Kapil Arya <ka...@mesosphere.io>.
> On Nov. 2, 2015, 3:52 p.m., Neil Conway wrote:
> > How about backward compatibility? Adding a note to docs/upgrades.md seems a good idea, at the very least. Are we pretty confident that no one else is looking at this data, and/or we're happy to break anyone that is?
Good point about backward compatibility. However, if the `data` field contains random bytes that might result into the entire json becoming "unrenderable", then it's broken anyways. Also note that we don't expose `TaskInfo::data` and `TaskStatus::data` fields over state.json either. We can use the dev/user mailing lists to get a feel for it.
- Kapil
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/#review104789
-----------------------------------------------------------
On Nov. 2, 2015, 3:21 p.m., Joseph Wu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39611/
> -----------------------------------------------------------
>
> (Updated Nov. 2, 2015, 3:21 p.m.)
>
>
> Review request for mesos, Ben Mahler and Artem Harutyunyan.
>
>
> Bugs: MESOS-3771
> https://issues.apache.org/jira/browse/MESOS-3771
>
>
> Repository: mesos
>
>
> Description
> -------
>
> We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
>
> The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
>
>
> Diffs
> -----
>
> src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
> src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
>
> Diff: https://reviews.apache.org/r/39611/diff/
>
>
> Testing
> -------
>
> `make check`
>
> Apparently, none of our tests check these fields.
>
> (Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
>
>
> Thanks,
>
> Joseph Wu
>
>
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Joseph Wu <jo...@mesosphere.io>.
> On Nov. 2, 2015, 12:52 p.m., Neil Conway wrote:
> > How about backward compatibility? Adding a note to docs/upgrades.md seems a good idea, at the very least. Are we pretty confident that no one else is looking at this data, and/or we're happy to break anyone that is?
>
> Kapil Arya wrote:
> Good point about backward compatibility. However, if the `data` field contains random bytes that might result into the entire json becoming "unrenderable", then it's broken anyways. Also note that we don't expose `TaskInfo::data` and `TaskStatus::data` fields over state.json either. We can use the dev/user mailing lists to get a feel for it.
Yeah, I emailed the dev list a while ago about this: http://www.mail-archive.com/dev@mesos.apache.org/msg33536.html
Note: The `TaskInfo::data` fields are part of the agent's state endpoint.
- Joseph
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/#review104789
-----------------------------------------------------------
On Nov. 2, 2015, 12:21 p.m., Joseph Wu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39611/
> -----------------------------------------------------------
>
> (Updated Nov. 2, 2015, 12:21 p.m.)
>
>
> Review request for mesos, Ben Mahler and Artem Harutyunyan.
>
>
> Bugs: MESOS-3771
> https://issues.apache.org/jira/browse/MESOS-3771
>
>
> Repository: mesos
>
>
> Description
> -------
>
> We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
>
> The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
>
>
> Diffs
> -----
>
> src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
> src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
>
> Diff: https://reviews.apache.org/r/39611/diff/
>
>
> Testing
> -------
>
> `make check`
>
> Apparently, none of our tests check these fields.
>
> (Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
>
>
> Thanks,
>
> Joseph Wu
>
>
Re: Review Request 39611: Remove binary `data` fields from state
endpoints.
Posted by Neil Conway <ne...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39611/#review104789
-----------------------------------------------------------
How about backward compatibility? Adding a note to docs/upgrades.md seems a good idea, at the very least. Are we pretty confident that no one else is looking at this data, and/or we're happy to break anyone that is?
- Neil Conway
On Nov. 2, 2015, 8:21 p.m., Joseph Wu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39611/
> -----------------------------------------------------------
>
> (Updated Nov. 2, 2015, 8:21 p.m.)
>
>
> Review request for mesos, Ben Mahler and Artem Harutyunyan.
>
>
> Bugs: MESOS-3771
> https://issues.apache.org/jira/browse/MESOS-3771
>
>
> Repository: mesos
>
>
> Description
> -------
>
> We do not encode or otherwise preprocess the raw binary in the ExecutorInfo or TaskInfo `data` fields before outputting them to JSON via the state endpoints. This means that the outputted JSON may be invalid (i.e. non-UTF8) if `data` was filled in with arbitrary bytes.
>
> The `data` fields are being removed because the state endpoints should not be writing binary data in the first place.
>
>
> Diffs
> -----
>
> src/common/http.cpp f56d8a178e1f61d91adfeaad32b47718adbc4379
> src/slave/http.cpp d6df97fb64077fb051dd78fe35bf04e8be0331ff
>
> Diff: https://reviews.apache.org/r/39611/diff/
>
>
> Testing
> -------
>
> `make check`
>
> Apparently, none of our tests check these fields.
>
> (Discovered via manual tests) Our JSON parser (PicoJson) is not restrictive enough to catch the invalid-JSON that results. It will happily parse the binary blobs (roundtripping successfully).
>
>
> Thanks,
>
> Joseph Wu
>
>